git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/5] combine-diff: handle binary files as binary

From
Junio C Hamano <gitster@pobox.com>
Date
May 31, 2011, 22:42 UTC
Message-ID
<7vlixmija0.fsf@alter.siamese.dyndns.org>
In-Reply-To
<20110530143627.GC31490@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> ... speaking of
> which, I need to polish that now that the streaming series seems to have
> settled...

I merely put it on the back burner, not necessarily declaring it as "settled".

I just created a pair of test repository with a single 800M binary junk file. One repository has it as a single loose object file, and the other repository has it in a packfile. The path is not explicitly marked with any attributes, but I have no CRLF funniness configured, so streaming would be used but there won't be any filtering involved.

Removing the file from the working tree and then checking it out of the index gave me a pleasant surprise. I originally did this only to help smaller machines that cannot comfortably fit inflated blob data in core, but it turns out that streaming write seems to perform better.

These are the typical /usr/bin/time output:
  $ /usr/bin/time git checkout a ;# w/o streaming
  1.39user 2.23system 0:03.62elapsed 99%CPU (0avgtext+0avgdata 6297056maxresident)k
  0inputs+1572872outputs (0major+399828minor)pagefaults 0swaps
  $ /usr/bin/time git checkout a ;# w/ streaming
  1.35user 1.22system 0:02.52elapsed 101%CPU (0avgtext+0avgdata 3151536maxresident)k
  0inputs+1572872outputs (0major+203226minor)pagefaults 0swaps

So now I can say I am reasonably happy with the series, but I haven't really exercised the code heavily, so there may still be bugs lurking ;-).

Previous: Junio C HamanoNext: Jeff King
Message 16 of 37 in “combined diff does not detect binary files and ignores -diff attribute”
  1. Jay SoffianMay 22, 2011
  2. Michael J GruberMay 23, 2011
  3. Jay SoffianMay 23, 2011
  4. Junio C HamanoMay 23, 2011
  5. Jeff KingMay 23, 2011
  6. Jeff KingMay 23, 2011
  7. 1/5 combine-diff: split header printing into its own functionJeff King, May 23, 2011
  8. 2/5 combine-diff: calculate mode_differs earlierJeff King, May 23, 2011
  9. 3/5 combine-diff: handle binary files as binaryJeff King, May 23, 2011
  10. Junio C HamanoMay 23, 2011
  11. Jeff KingMay 23, 2011
  12. Junio C HamanoMay 30, 2011
  13. Jeff KingMay 30, 2011
  14. Jeff KingMay 30, 2011
  15. Junio C HamanoMay 30, 2011
  16. Junio C HamanoMay 31, 2011
  17. 4/5 refactor get_textconv to not require diff_filespecJeff King, May 23, 2011
  18. 5/5 combine-diff: respect textconv attributesJeff King, May 23, 2011
  19. Junio C HamanoMay 23, 2011
  20. Jeff KingMay 23, 2011
  21. Junio C HamanoMay 24, 2011
  22. Jeff KingMay 24, 2011
  23. Jay SoffianMay 23, 2011
  24. Jay SoffianMay 23, 2011
  25. Jeff KingMay 23, 2011
  26. Jay SoffianMay 24, 2011
  27. Jeff KingMay 23, 2011
  28. Junio C HamanoMay 24, 2011
  29. Michael J GruberMay 24, 2011
  30. Junio C HamanoMay 24, 2011
  31. Michael J GruberMay 24, 2011
  32. Junio C HamanoMay 24, 2011
  33. Jay SoffianMay 24, 2011
  34. Jeff KingMay 24, 2011
  35. Michael J GruberMay 25, 2011
  36. Jeff KingMay 25, 2011
  37. Jay SoffianMay 24, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.