git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Optimizing cloning of a high object count repository

From
Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Date
Dec 13, 2008, 15:46 UTC
Message-ID
<fcaeb9bf0812130746l38a12f37wde26f31d5fa0d2a2@mail.gmail.com>
In-Reply-To
<200812131624.57618.Resul-Cetin@gmx.net>
On 12/13/08, Resul Cetin <Resul-Cetin@gmx.net> wrote:
Show 16 quoted lines
> Hi,
>  there are currently different ideas to move gentoo's cvs repository to an
>  other scm. Current tests showed that svn will not make anything better (it
>  gets in most perfomance and size based benchmarks even worse). Another idea is
>  to move to git. It looks really promising in size based benchmarks but cloning
>  seems nearly impossible. The current test repository is available at
>  git://git.overlays.gentoo.org/exp/gentoo-x86.git and is around 900MB in size
>  and has 4696137 objects. It really takes ages to do the counting of the
>  objects on the server and compressing takes much longer.
>  The size of the linux repository seems to be smaller but in the same range
>  object count and repository size but clones are much much faster. Is there any
>  way to optimize the server operations like counting and compressing of objects
>  to get the same speed as we get from git.kernel.org (which does it in nearly
>  no time and the only limiting factor seems to be my bandwith)?
>  The only other information I have is that Robin H. Johnson made a single
>  ~910MiB pack for the whole repository.

Make yearly packed repository snapshots and publish them via http. People can wget the latest snapshot, then pull updates later.

-- 
Duy
Previous: Resul CetinNext: Resul Cetin
Message 2 of 7 in “Optimizing cloning of a high object count repository”
  1. Resul CetinDec 13, 2008
  2. Nguyen Thai Ngoc DuyDec 13, 2008
  3. Resul CetinDec 13, 2008
  4. Jean-Luc HerrenDec 13, 2008
  5. Resul CetinDec 13, 2008
  6. Nicolas PitreDec 13, 2008
  7. Nicolas PitreDec 13, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.