threads / discuss / 7703

Initial push of a fully packed repo - why repack?

Subject: Initial push of a fully packed repo - why repack?

## tl;dr

4 messages between Apr 17, 2007 and Apr 17, 2007.

replies: 3people: 2as markdown or json

Martin Langhoff· Apr 17, 2007, 01:12 UTC · lore

Pushing to a new empty repo, git spends a lot of time repacking, even if we have just run git-repack. The process looks like

 - use parsecvs to import a large CVS repo
 - git-repack -a -d (packs to ~550MB, taking ~1hr)
 - git-push --all git+ssh://host/newlycreatedrepo.git

Does it make sense to detect and optimise for this case?

cheers,

martin

Nicolas Pitre· Apr 17, 2007, 02:21 UTC · re: Martin Langhoff · lore

Re: Initial push of a fully packed repo - why repack?

On Tue, 17 Apr 2007, Martin Langhoff wrote:

Show 8 quoted lines

> Pushing to a new empty repo, git spends a lot of time repacking, even
> if we have just run git-repack. The process looks like
> 
> - use parsecvs to import a large CVS repo
> - git-repack -a -d (packs to ~550MB, taking ~1hr)
> - git-push --all git+ssh://host/newlycreatedrepo.git
> 
> Does it make sense to detect and optimise for this case?

Maybe... Although the second repack during the push should be much much faster than the first one.

Nicolas

Martin Langhoff· Apr 17, 2007, 02:28 UTC · re: Nicolas Pitre · lore

Re: Initial push of a fully packed repo - why repack?

On 4/17/07, Nicolas Pitre <nico@cam.org> wrote:

> On Tue, 17 Apr 2007, Martin Langhoff wrote:
> > Does it make sense to detect and optimise for this case?
> Maybe...  Although the second repack during the push should be much much
> faster than the first one.

It is - but it still burns through perhaps 1 minute of CPU and IO rewriting the exact same pack as you can see:

Generating pack...
Done counting 290123 objects.
Deltifying 290123 objects.
 100% (290123/290123) done
Writing 290123 objects.
 100% (290123/290123) done
Total 290123 (delta 234293), reused 290123 (delta 234293)

cheers,

martin

Nicolas Pitre· Apr 17, 2007, 02:49 UTC · re: Martin Langhoff · lore

Re: Initial push of a fully packed repo - why repack?

On Tue, 17 Apr 2007, Martin Langhoff wrote:

Show 8 quoted lines

> On 4/17/07, Nicolas Pitre <nico@cam.org> wrote:
> > On Tue, 17 Apr 2007, Martin Langhoff wrote:
> > > Does it make sense to detect and optimise for this case?
> > Maybe...  Although the second repack during the push should be much much
> > faster than the first one.
> 
> It is - but it still burns through perhaps 1 minute of CPU and IO
> rewriting the exact same pack as you can see:

Sure. On the IO you can't save. You'll have to copy the packanyway and with all objects being "reused" the pack-objects code is basically not doing much more than a straight cp would do.

What is costly is figuring out if the single pack you have actually contains all the objects you wish to push, and _only_ the objects you wish to push. That is the real cost. By the time all those objects are listed and accounted then repacking is basically copying the data over with almost no CPU usage. In other words, the cost to determine if it is OK to simply send the pack you already have and actually sending it would be sensibly the same.

Nicolas