git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/2] optimizing pack access on "read only" fetch repos

From
Shawn Pearce <spearce@spearce.org>
Date
Jan 29, 2013, 08:06 UTC
Message-ID
<CAJo=hJtDn_6qsQ7uj2Pi=vnjNXR7WPN1VH1Mb8YdneD_bHUK8w@mail.gmail.com>
In-Reply-To
<7vlibfxhit.fsf@alter.siamese.dyndns.org>
On Sat, Jan 26, 2013 at 10:32 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 15 quoted lines
> Jeff King <peff@peff.net> writes:
>
>> This is a repost from here:
>>
>>   http://thread.gmane.org/gmane.comp.version-control.git/211176
>>
>> which got no response initially. Basically the issue is that read-only
>> repos (e.g., a CI server) whose workflow is something like:
>>
>>   git fetch $some_branch &&
>>   git checkout -f $some_branch &&
>>   make test
>>
>> will never run git-gc, and will accumulate a bunch of small packs and
>> loose objects, leading to poor performance.
...
> I also wonder if we would be helped by another "repack" mode that
> coalesces small packs into a single one with minimum overhead, and
> run that often from "gc --auto", so that we do not end up having to
> have 50 packfiles.
Yes. This does help....
Show 10 quoted lines
> When we have 2 or more small and young packs, we could:
>
>  - iterate over idx files for these packs to enumerate the objects
>    to be packed, replacing read_object_list_from_stdin() step;
>
>  - always choose to copy the data we have in these existing packs,
>    instead of doing a full prepare_pack(); and
>
>  - use the order the objects appear in the original packs, bypassing
>    compute_write_order().
Hmm, sounds familiar. Seems like its what we do in JGit for Android. :-)
Previous: Junio C HamanoNext: Jeff King
Message 9 of 21 in “optimizing pack access on "read only" fetch repos”
  1. 0/2 optimizing pack access on "read only" fetch reposJeff King, Jan 26, 2013
  2. 1/2 fetch: run gc --auto after fetchingJeff King, Jan 26, 2013
  3. Jonathan NiederJan 27, 2013
  4. 2/2 fetch-pack: avoid repeatedly re-scanning pack directoryJeff King, Jan 26, 2013
  5. Jonathan NiederJan 27, 2013
  6. Junio C HamanoJan 27, 2013
  7. Jonathan NiederJan 27, 2013
  8. Junio C HamanoJan 27, 2013
  9. Shawn PearceJan 29, 2013
  10. Jeff KingJan 29, 2013
  11. Martin FickJan 29, 2013
  12. Junio C HamanoJan 29, 2013
  13. Jeff KingJan 29, 2013
  14. Junio C HamanoJan 29, 2013
  15. Shawn PearceJan 31, 2013
  16. Jeff KingFeb 1, 2013
  17. Shawn PearceFeb 2, 2013
  18. Duy NguyenJan 29, 2013
  19. Jeff KingJul 12, 2017
  20. Ævar Arnfjörð BjarmasonJul 12, 2017
  21. Jeff KingJul 12, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.