git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/1] send-pack: set core.warnAmbiguousRefs=false

From
Derrick Stolee <stolee@gmail.com>
Date
Nov 6, 2018, 20:00 UTC
Message-ID
<1e92f560-a5c0-2d58-5a4c-8530598d1025@gmail.com>
In-Reply-To
<20181106194442.GB8902@sigill.intra.peff.net>
On 11/6/2018 2:44 PM, Jeff King wrote:
Show 32 quoted lines
> On Tue, Nov 06, 2018 at 11:13:47AM -0800, Derrick Stolee via GitGitGadget wrote:
>
>> I've been looking into the performance of git push for very large repos. Our
>> users are reporting that 60-80% of git push time is spent during the
>> "Enumerating objects" phase of git pack-objects.
>>
>> A git push process runs several processes during its run, but one includes
>> git send-pack which calls git pack-objects and passes the known have/wants
>> into stdin using object ids. However, the default setting for
>> core.warnAmbiguousRefs requires git pack-objects to check for ref names
>> matching the ref_rev_parse_rules array in refs.c. This means that every
>> object is triggering at least six "file exists?" queries.
>>
>> When there are a lot of refs, this can add up significantly! My PerfView
>> trace for a simple push measured 3 seconds spent checking these paths.
> Some of this might be useful in the commit message. :)
>
>> The fix for this is simple: set core.warnAmbiguousRefs to false for this
>> specific call of git pack-objects coming from git send-pack. We don't want
>> to default it to false for all calls to git pack-objects, as it is valid to
>> pass ref names instead of object ids. This helps regain these seconds during
>> a push.
> I don't think you actually care about the ambiguity check between refs
> here; you just care about avoiding the ref check when we've seen (and
> are mostly expecting) a 40-hex sha1. We have a more specific flag for
> that: warn_on_object_refname_ambiguity.
>
> And I think it would be OK to enable that all the time for pack-objects,
> which is plumbing that does typically expect object names. See prior art
> in 25fba78d36 (cat-file: disable object/refname ambiguity check for
> batch mode, 2013-07-12) and 4c30d50402 (rev-list: disable object/refname
> ambiguity check with --stdin, 2014-03-12).

Thanks for these pointers. Helps to know there is precedent for shutting down the behavior without relying on "-c" flags.

Show 5 quoted lines
> Whenever I see a change like this to the pack-objects invocation for
> send-pack, it makes me wonder if upload-pack would want the same thing.
>
> It's a moot point if we just set the flag directly in inside
> pack-objects, though.
I'll send a v2 that does just that.

Thanks, -Stolee

Previous: Derrick StoleeNext: Derrick Stolee via GitGitGadget
Message 6 of 12 in “send-pack: set core.warnAmbiguousRefs=false”
  1. 0/1 send-pack: set core.warnAmbiguousRefs=falseDerrick Stolee via GitGitGadget, Nov 6, 2018
  2. 1/1 send-pack: set core.warnAmbiguousRefs=falseDerrick Stolee via GitGitGadget, Nov 6, 2018
  3. Jeff KingNov 6, 2018
  4. Jeff KingNov 6, 2018
  5. Derrick StoleeNov 6, 2018
  6. Derrick StoleeNov 6, 2018
  7. 0/1 send-pack: set core.warnAmbiguousRefs=falseDerrick Stolee via GitGitGadget, Nov 6, 2018
  8. 1/1 pack-objects: ignore ambiguous object warningsDerrick Stolee via GitGitGadget, Nov 6, 2018
  9. Jeff KingNov 6, 2018
  10. Junio C HamanoNov 7, 2018
  11. Junio C HamanoNov 7, 2018
  12. Jeff KingNov 7, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.