git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: How to still kill git fetch with too many refs

From
MFMartin Fick <mfick@codeaurora.org>
Date
Jul 2, 2013, 16:58 UTC
Message-ID
<201307021058.58563.mfick@codeaurora.org>
In-Reply-To
<51D29C3E.5070600@alum.mit.edu>

On Tuesday, July 02, 2013 03:24:14 am Michael Haggerty wrote:

Show 7 quoted lines
> > git rev-list HEAD | for nn in $(seq 0 100) ; do for c
> > in $(seq 0 10000) ; do  read sha ; echo $sha
> > refs/c/$nn/$c$nn ; done ; done > .git/packed-refs
> 
> I believe this generates a packed-refs file that is not
> sorted lexicographically by refname, whereas all
> Git-generated packed-refs files are sorted.

Yes, you are indeed correct. I was attempting to be too clever with my sharding I guess. Thanks.

Show 7 quoted lines
> There are
> some optimizations in refs.c for adding references in
> order that might therefore be circumvented by your
> unsorted file.  Please try sorting the file by refname
> and see if that helps.  (You can do so by deleting one
> of the packed references; then git will sort the
> remainder while rewriting the file.)
A simple git pack-refs seems to clean it up.

The original test did complete in ~77mins last night. A rerun with a sorted file takes ~61mins,

-Martin
PS: This test was performed with git version 1.8.2.1 on 
linux 2.6.32-37-generic #81-Ubuntu SMP 
-- 
The Qualcomm Innovation Center, Inc. is a member of Code 
Aurora Forum, hosted by The Linux Foundation
Previous: Michael Haggerty
Message 15 of 15 in “How to still kill git fetch with too many refs”
  1. Martin FickJul 2, 2013
  2. Jeff KingJul 2, 2013
  3. Jeff KingJul 2, 2013
  4. Jeff KingJul 2, 2013
  5. Junio C HamanoJul 2, 2013
  6. Jeff KingJul 2, 2013
  7. 0/3 avoid quadratic behavior in fetch-packJeff King, Jul 2, 2013
  8. 1/3 fetch-pack: avoid quadratic list insertion in mark_completeJeff King, Jul 2, 2013
  9. 2/3 commit.c: make compare_commits_by_commit_date globalJeff King, Jul 2, 2013
  10. 3/3 fetch-pack: avoid quadratic behavior in rev_list_pushJeff King, Jul 2, 2013
  11. Eric SunshineJul 2, 2013
  12. Martin FickJul 2, 2013
  13. Brandon CaseyJul 2, 2013
  14. Michael HaggertyJul 2, 2013
  15. Martin FickJul 2, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.