git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Huge push upload despite only having a tiny change

From
Derrick Stolee <stolee@gmail.com>
Date
Jun 2, 2020, 19:40 UTC
Message-ID
<29e6c05e-2d79-d2ac-a033-dab6342ebcaa@gmail.com>
In-Reply-To
<CABPp-BEswHLhymQ1_07g3qqu=7kFR3eQyAHR0qMgSvi6THy=zQ@mail.gmail.com>
On 6/2/2020 3:21 PM, Elijah Newren wrote:
> * The user was two commits behind the closely-related branch at the
> time of the first push, and 10 commits behind at the time of the
> second push.  Running format-patch on these 10 commits that were on
> the server at the time shows their size is at most about ~55 k.

This is most-likely the difference, since the pack-objects algorithm only looks at the _boundary_ between the server's commits and the commits-to-push. This also could have dramatically changed the delta-base matches.

Do you have exact object counts? It would help to know if somehow the object discovery algorithm is at fault or the delta-base algorithm is to blame.

For instance, `pack.useSparse` was enabled by default this release, and has some opportunity to push extra objects. See [1] for more details on both the "boundary" description (the "commit frontier") but also how that option changes the algorithm.

The only case I know of that could lead to sending extra objects (that was not the case before) is described in t5322-pack-objects-sparse.sh and 4f6d26b1 (list-objects: consume sparse tree walk, 2019-01-16). It involves doing a full _copy_ of a tree from one position to another, without "disturbing" the parent of the original tree.

(I mean: copy directory A/B to C/D and make sure nothing is different in directory A.)

However, if these two pushes were with the same config setting, I'm not sure what could have changed between the pushes to hit this very narrow case.

Thanks, -Stolee

[1] https://devblogs.microsoft.com/devops/exploring-new-frontiers-for-git-push-performance/
Previous: Elijah NewrenNext: Elijah Newren
Message 2 of 6 in “Huge push upload despite only having a tiny change”
  1. Elijah NewrenJun 2, 2020
  2. Derrick StoleeJun 2, 2020
  3. Elijah NewrenJun 3, 2020
  4. Jonathan NiederJun 3, 2020
  5. Elijah NewrenJun 3, 2020
  6. Junio C HamanoJun 3, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.