git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git-fetching from a big repository is slow

From
Nicolas Pitre <nico@cam.org>
Date
Dec 14, 2006, 18:14 UTC
Message-ID
<Pine.LNX.4.64.0612141311310.18171@xanadu.home>
In-Reply-To
<4581573E.40803@op5.se>
On Thu, 14 Dec 2006, Andreas Ericsson wrote:
Show 19 quoted lines
> Andy Parkins wrote:
> > Hello,
> > 
> > I've got a big repository.  I've got two computers.  One has the repository
> > up-to-date (164M after repack); one is behind (30M ish).
> > 
> > I used git-fetch to try and update; and the sync took HOURS.  I zipped the
> > .git directory and transferred that and it took about 15 minutes to
> > transfer.
> > 
> > Am I doing something wrong?  The git-fetch was done with a git+ssh:// URL.
> > The zip transfer with scp (so ssh shouldn't be a factor).
> > 
> 
> This seems to happen if your repository consists of many large binary files,
> especially many large binary files of several versions that do not deltify
> well against each other. Perhaps it's worth adding gzip compression detecion
> to git? I imagine more people than me are tracking gzipped/bzip2'ed content
> that pretty much never deltifies well against anything else.

If your remote repository is fully packed in a single pack that should not have any impact on the transfer latency since no attempt to redeltify objects against each other is attempted by default when those objects are in the same pack.

Previous: Junio C Hamano
Message 14 of 14 in “git-fetching from a big repository is slow”
  1. Andy ParkinsDec 14, 2006
  2. Andreas EricssonDec 14, 2006
  3. Johannes SchindelinDec 14, 2006
  4. Andreas EricssonDec 14, 2006
  5. Geert BoschDec 14, 2006
  6. Andreas EricssonDec 14, 2006
  7. Andy ParkinsDec 14, 2006
  8. Han-Wen NienhuysDec 14, 2006
  9. Andy ParkinsDec 14, 2006
  10. Johannes SchindelinDec 14, 2006
  11. Junio C HamanoDec 14, 2006
  12. Johannes SchindelinDec 14, 2006
  13. Junio C HamanoDec 15, 2006
  14. Nicolas PitreDec 14, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.