git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Poor push performance with large number of refs

From
Duy Nguyen <pclouds@gmail.com>
Date
Dec 11, 2014, 01:41 UTC
Message-ID
<CACsJy8DxXdLsdgP7OKmv8bS1WVPSO_goBaNrttAqd0-ASBgbpQ@mail.gmail.com>
In-Reply-To
<20141210233443.GA130424@vauxhall.crustytoothpaste.net>

On Thu, Dec 11, 2014 at 6:34 AM, brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 14 quoted lines
> I looked at this more in depth today and I found that the bottleneck is
> --thin.  I tried git send-pack, which does not use --thin by default,
> which led me to further testing.  A particular push went from 24 seconds
> with --thin to 4 seconds without.
>
> I agree that the large number of refs is at least part of the problem,
> because reducing the number of refs has a slight but noticeable impact.
> It's also the factor I can least control.
>
> I have a patch which allows per-remote configuration of whether to use
> thin packs (which I will send shortly), but I'm wondering if we can do
> better, especially since --thin is the default.  It looks like --thin
> forces pack-objects to do its own lookup (essentially a rev-list)
> instead of using the values provided on stdin.

It could be a regression by fbd4a70 (list-objects: mark more commits as edges in mark_edges_uninteresting - 2013-08-16). That commit makes --thin a lot more agressive (reading lots of trees). You can try to revert that commit (or use a git version without that commit) and see if it improves performance. If so, we probably want to enable that code for shallow repos only.

-- 
Duy
Previous: brian m. carlsonNext: brian m. carlson
Message 6 of 14 in “Poor push performance with large number of refs”
  1. brian m. carlsonDec 10, 2014
  2. Shawn PearceDec 10, 2014
  3. brian m. carlsonDec 10, 2014
  4. push: add remote.pushThin to control thin pack generationbrian m. carlson, Dec 10, 2014
  5. brian m. carlsonDec 11, 2014
  6. Duy NguyenDec 11, 2014
  7. brian m. carlsonDec 11, 2014
  8. list-objects: mark fewer commits as edges for non-shallow clonesbrian m. carlson, Dec 11, 2014
  9. Duy NguyenDec 11, 2014
  10. brian m. carlsonDec 20, 2014
  11. Junio C HamanoDec 11, 2014
  12. Junio C HamanoDec 11, 2014
  13. brian m. carlsonDec 20, 2014
  14. Duy NguyenDec 10, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.