git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [BUG?] bulk checkin does not respect filters

From
Jeff King <peff@peff.net>
Date
Feb 24, 2012, 07:54 UTC
Message-ID
<20120224075425.GA18688@sigill.intra.peff.net>
In-Reply-To
<7vr4xk28z0.fsf@alter.siamese.dyndns.org>
On Thu, Feb 23, 2012 at 07:42:11PM -0800, Junio C Hamano wrote:
Show 14 quoted lines
> It is a bit curious that anything filtered even goes to the streaming
> codepath, given this piece of code in write_entry() in entry.c:
> 
> 	if (ce_mode_s_ifmt == S_IFREG) {
> 		struct stream_filter *filter = get_stream_filter(path, ce->sha1);
> 		if (filter &&
> 		    !streaming_write_entry(ce, path, filter,
> 					   state, to_tempfile,
> 					   &fstat_done, &st))
> 			goto finish;
> 	}
> 
> and get_stream_filter() in convert.c has an explicit exception for this
> case at the very beginning:

I think it is because we don't follow that code path at all. The stack trace for "git add" looks on a large file looks like:

#0 stream_to_pack (...) at bulk-checkin.c:101 #1 deflate_to_pack (...) at bulk-checkin.c:219 #2 index_bulk_checkin (...) at bulk-checkin.c:258 #3 index_stream (...) at sha1_file.c:2712 #4 index_fd (...) at sha1_file.c:2726 #5 index_path (...) at sha1_file.c:2742 #6 add_to_index (...) at read-cache.c:644 #7 add_file_to_index (...) at read-cache.c:673 #8 add_files (...) at builtin/add.c:363 #9 cmd_add (...) at builtin/add.c:474 #10 run_builtin (...) at git.c:324 #11 handle_internal_command (...) at git.c:484 #12 run_argv (...) at git.c:530 #13 main (...) at git.c:605

Isn't write_entry the _other_ side of things. I.e., checking out, not checking in? That side works fine (making the large file handling for git-add even more odd. We will fail to apply the "clean" filter when adding it, but we will apply the smudge filter when we write it out).

-Peff
Previous: Junio C HamanoNext: Junio C Hamano
Message 4 of 19 in “[BUG?] bulk checkin does not respect filters”
  1. Jeff KingFeb 24, 2012
  2. Junio C HamanoFeb 24, 2012
  3. Junio C HamanoFeb 24, 2012
  4. Jeff KingFeb 24, 2012
  5. Junio C HamanoFeb 24, 2012
  6. Jeff KingFeb 24, 2012
  7. Jeff KingFeb 24, 2012
  8. 1/2 teach convert_to_git a "dry run" modeJeff King, Feb 24, 2012
  9. 2/2 do not stream large files to pack when filters are in useJeff King, Feb 24, 2012
  10. Junio C HamanoFeb 24, 2012
  11. Jeff KingFeb 24, 2012
  12. Jeff KingFeb 24, 2012
  13. Junio C HamanoFeb 24, 2012
  14. Jeff KingFeb 24, 2012
  15. 1/3 teach convert_to_git a "dry run" modeJeff King, Feb 24, 2012
  16. 2/3 teach dry-run convert_to_git not to require a src bufferJeff King, Feb 24, 2012
  17. 3/3 do not stream large files to pack when filters are in useJeff King, Feb 24, 2012
  18. Junio C HamanoFeb 24, 2012
  19. Junio C HamanoFeb 24, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.