git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 1/2] pack-objects: break delta cycles before delta-search phase

From
Jeff King <peff@peff.net>
Date
Aug 11, 2016, 06:57 UTC
Message-ID
<20160811065751.p64bi3sngbeotwc3@sigill.intra.peff.net>
In-Reply-To
<20160811050252.g3iusy7bp3j6tzte@sigill.intra.peff.net>
On Thu, Aug 11, 2016 at 01:02:52AM -0400, Jeff King wrote:
Show 11 quoted lines
> > > + *   2. Updating our size; check_object() will have filled in the size of our
> > > + *      delta, but a non-delta object needs it true size.
> > 
> > Excellent point.
> 
> I was not clever enough to think of it; the pack-objects code is filled
> with nice assertions (Thanks, Nico!) that help out when you are stupid. :)
> 
> One thing to be careful of is that there are more things this
> drop_reused_delta() should be doing. But I looked through the rest of
> check_object() and could not find anything else.
Argh, I spoke too soon.

It is true that the size lookup is the only part of check_object() we skip if we are reusing the delta. But what I didn't notice is that when we choose to reuse a delta, we overwrite entry->type (the real type!) with the in_pack_type (OFS_DELTA, etc). We need to undo that so that later stages see the real type.

I'm not sure how my existing tests worked (I confirmed that they do indeed break the delta). It may be that only some code paths actually care about the real type. But when playing with the depth-limit (which uses the same drop_reused_delta() helper), I managed to make some pretty broken packs.

So please disregard the v4 patch I just sent; I haven't triggered it, but I imagine it has the same problem, and I just didn't manage to trigger it.

I'll clean that up and send out a new series.
-Peff
Previous: Jeff KingNext: Jeff King
Message 26 of 35 in “speed up pack-objects counting with many packs”
  1. 0/7 speed up pack-objects counting with many packsJeff King, Jul 29, 2016
  2. 1/7 t/perf: add tests for many-pack scenariosJeff King, Jul 29, 2016
  3. 2/7 sha1_file: drop free_pack_by_nameJeff King, Jul 29, 2016
  4. 3/7 add generic most-recently-used listJeff King, Jul 29, 2016
  5. 4/7 find_pack_entry: replace last_found_pack with MRU cacheJeff King, Jul 29, 2016
  6. 5/7 pack-objects: break out of want_object loop earlyJeff King, Jul 29, 2016
  7. 6/7 pack-objects: compute local/ignore_pack_keep earlyJeff King, Jul 29, 2016
  8. 7/7 pack-objects: use mru list when iterating over packsJeff King, Jul 29, 2016
  9. Jeff KingJul 29, 2016
  10. Junio C HamanoJul 29, 2016
  11. Jeff KingAug 8, 2016
  12. Junio C HamanoAug 8, 2016
  13. Jeff KingAug 8, 2016
  14. Junio C HamanoAug 8, 2016
  15. Jeff KingAug 9, 2016
  16. Jeff KingAug 9, 2016
  17. Junio C HamanoAug 9, 2016
  18. Junio C HamanoAug 9, 2016
  19. 0/2 pack-objects mruJeff King, Aug 10, 2016
  20. Junio C HamanoAug 10, 2016
  21. Jeff KingAug 11, 2016
  22. 1/2 pack-objects: break delta cycles before delta-search phaseJeff King, Aug 10, 2016
  23. Junio C HamanoAug 10, 2016
  24. Jeff KingAug 11, 2016
  25. 1/2 pack-objects: break delta cycles before delta-search phaseJeff King, Aug 11, 2016
  26. Jeff KingAug 11, 2016
  27. pack-objects mruJeff King, Aug 11, 2016
  28. 1/4 provide an initializer for "struct object_info"Jeff King, Aug 11, 2016
  29. 2/4 sha1_file: make packed_object_info publicJeff King, Aug 11, 2016
  30. 3/4 pack-objects: break delta cycles before delta-search phaseJeff King, Aug 11, 2016
  31. 4/4 pack-objects: use mru list when iterating over packsJeff King, Aug 11, 2016
  32. Jeff KingAug 11, 2016
  33. Junio C HamanoAug 11, 2016
  34. Jeff KingAug 11, 2016
  35. 2/2 pack-objects: use mru list when iterating over packsJeff King, Aug 10, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.