git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/7] speeding up cat-file by reordering object access

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 16, 2018, 18:52 UTC
Message-ID
<xmqqmutmdu02.fsf@gitster-ct.c.googlers.com>
In-Reply-To
<20180816173934.GC882@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> I think that makes sense. We already see duplicates from
> for_each_packed_object() when they're in multiple packs, and callers
> just need to be ready to deal with it (and depending on what you're
> doing, you may actually _want_ the duplicates).

You of course would also see dups between loose and packed until prune-packed is run.

I also was thinking about the same thing after Derrick's response, but unless you are very specialized caller, it does not allow you do very much to learn that object X exists as a loose object locally, also as a loose object in our alternate, and also in pack A, but not in other packs. You need a way to say "Read the contents of object X from that place, not from any other place", "Remove that copy of object X at that place, but not at any other place" etc. to make effective use of that kind of information.

The codepath that implements runtime access has "I found a copy here, but it is unusable, so let's go on to look for another usable copy" fallback. This is a tangent but it is something we should not lose in the midx-enabled world.

Previous: Jeff KingNext: Jeff King
Message 26 of 27 in “speeding up cat-file by reordering object access”
  1. 0/7 speeding up cat-file by reordering object accessJeff King, Aug 10, 2018
  2. 1/7 for_each_*_object: store flag definitions in a single locationJeff King, Aug 10, 2018
  3. Stefan BellerAug 10, 2018
  4. Jeff KingAug 10, 2018
  5. Jeff KingAug 10, 2018
  6. Stefan BellerAug 10, 2018
  7. Jeff KingAug 11, 2018
  8. 2/7 for_each_*_object: take flag arguments as enumJeff King, Aug 10, 2018
  9. 3/7 for_each_*_object: give more comprehensive docstringsJeff King, Aug 10, 2018
  10. 4/7 for_each_packed_object: support iterating in pack-orderJeff King, Aug 10, 2018
  11. Derrick StoleeAug 15, 2018
  12. Jeff KingAug 16, 2018
  13. 5/7 t1006: test cat-file --batch-all-objects with duplicatesJeff King, Aug 10, 2018
  14. 6/7 cat-file: rename batch_{loose,packed}_object callbacksJeff King, Aug 10, 2018
  15. 7/7 cat-file: support "unordered" output for --batch-all-objectsJeff King, Aug 10, 2018
  16. Jonathan TanAug 13, 2018
  17. 0/4 finishing touches on jk/for-each-object-iterationJeff King, Aug 14, 2018
  18. 1/4 cat-file: use oidset check-and-insertJeff King, Aug 14, 2018
  19. 2/4 cat-file: split batch "buf" into two variablesJeff King, Aug 14, 2018
  20. 3/4 cat-file: use a single strbuf for all outputJeff King, Aug 14, 2018
  21. René ScharfeAug 14, 2018
  22. Jeff KingAug 14, 2018
  23. 4/4 for_each_*_object: move declarations to object-store.hJeff King, Aug 14, 2018
  24. Derrick StoleeAug 15, 2018
  25. Jeff KingAug 16, 2018
  26. Junio C HamanoAug 16, 2018
  27. Jeff KingAug 16, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.