git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: A local shared clone is now much slower

From
Duy Nguyen <pclouds@gmail.com>
Date
Jul 8, 2013, 08:57 UTC
Message-ID
<CACsJy8Chmm0=wDV4NQ+4Gh7KZYpbd9qkb=pNzkPeG-a-xiwVmw@mail.gmail.com>
In-Reply-To
<20130708073041.GA25072@sigill.intra.peff.net>
On Mon, Jul 8, 2013 at 2:30 PM, Jeff King <peff@peff.net> wrote:
Show 16 quoted lines
> Subject: [PATCH] clone: drop connectivity check for local clones
>
> Commit 0433ad1 (clone: run check_everything_connected,
> 2013-03-25) added the same connectivity check to clone that
> we use for fetching. The intent was to provide enough safety
> checks that "git clone git://..." could be counted on to
> detect bit errors and other repo corruption, and not
> silently propagate them to the clone.
>
> For local clones, this turns out to be a bad idea, for two
> reasons:
>
>   1. Local clones use hard linking (or even shared object
>      stores), and so complete far more quickly. The time
>      spent on the connectivity check is therefore
>      proportionally much more painful.

There's also byte-to-byte copy when system does not support hardlinks (or the user does not want it) but I guess it's safe to trust the OS to copy correctly in most cases.

Show 9 quoted lines
>   2. Local clones do not actually meet our safety guarantee
>      anyway. The connectivity check makes sure we have all
>      of the objects we claim to, but it does not check for
>      bit errors. We will notice bit errors in commits and
>      trees, but we do not load blob objects at all. Whereas
>      over the pack transport, we actually recompute the sha1
>      of each object in the incoming packfile; bit errors
>      change the sha1 of the object, which is then caught by
>      the connectivity check.

We used to, before d21c463 (fetch/receive: remove over-pessimistic connectivity check - 2012-03-15). But back then we did not even do connectivity check in clone.

Show 8 quoted lines
> This patch drops the connectivity check in the local case.
> Note that we have to revert the changes from 0433ad1 to
> t5710, as we no longer notice the corruption during clone.
>
> We could go a step further and provide a "verify even local
> clones" option, but it is probably not worthwhile. You can
> already spell that as "cd foo.git && git fsck && git clone ."
> or as "git clone --no-local foo.git".

Faster clones make everybody happy :-) -- Duy

Previous: Jeff KingNext: Junio C Hamano
Message 5 of 10 in “A local shared clone is now much slower”
  1. Stephen RothwellJul 8, 2013
  2. Duy NguyenJul 8, 2013
  3. Stephen RothwellJul 8, 2013
  4. Jeff KingJul 8, 2013
  5. Duy NguyenJul 8, 2013
  6. Junio C HamanoJul 8, 2013
  7. Jeff KingJul 9, 2013
  8. Priming git clone with a local repo?Andreas Krey, Jul 11, 2013
  9. Junio C HamanoJul 11, 2013
  10. Junio C HamanoJul 8, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.