threads / discuss / 5870

is git-unpack-objects now redundant for 'git-push' and friends?

Subject: is git-unpack-objects now redundant for 'git-push' and friends?

## tl;dr

3 messages between Oct 10, 2006 and Oct 10, 2006.

replies: 2people: 3as markdown or json

Sam Vilain· Oct 10, 2006, 13:23 UTC · lore

When pushing or pulling to/from a repository, why unpack the objects? Why not just fsck and then throw the pack into $GIT_DIR/objects/pack?

If you're pushing the entire repository, for instance, currently you might create 10,000's of files, which will just be thrown away later when you `git-repack -d'.

I suspect that this was never changed, because there never used to be more than one packfile allowed, correct?

If the server *does* send us duplicates of objects we already have for some reason, well that's what `git-repack -a -d' is for.

I'm just wondering if there are any good reasons to do this any more.
Sam.
Shawn Pearce· Oct 10, 2006, 16:14 UTC · re: Sam Vilain · lore

Re: is git-unpack-objects now redundant for 'git-push' and friends?

Sam Vilain <sam@vilain.net> wrote:
Show 14 quoted lines
> When pushing or pulling to/from a repository, why unpack the objects?
> Why not just fsck and then throw the pack into $GIT_DIR/objects/pack?
> 
> If you're pushing the entire repository, for instance, currently you
> might create 10,000's of files, which will just be thrown away later
> when you `git-repack -d'.
> 
> I suspect that this was never changed, because there never used to be
> more than one packfile allowed, correct?
> 
> If the server *does* send us duplicates of objects we already have for
> some reason, well that's what `git-repack -a -d' is for.
> 
> I'm just wondering if there are any good reasons to do this any more.

There's still a few good reasons to unpack things, even though I hate sitting through 5000 objects unpacking on a Windows system, or a very slow NFS system (one NFS based system I work with unpacks small <1KiB objects at a rate of about 1 every 15 seconds).

 - We don't completely trust the remote peer.  If the remote sends
   us an object we already have we want to use the bytes we have
   and not the bytes they are sending.  That way even if the remote
   were able to generate a hash collision and produce a replacement
   byte sequence with the same hash, our known good byte sequence
   is not affected.
 - Packs sent over the network can be thin packs.  A think pack
   may not contain the object used as the delta base for a delta
   within that pack.  This is done to save network bandwidth when
   both sides already have the delta base for an object that is
   being transferred.
 - Local packs cannot access delta base objects not in the same
   pack.  This limitation simplifies the pack access code in a
   lot of the system.  It also means that you can in a worst case
   scenario obtain everything in that pack back when all you have
   is the pack file itself.

I'm sure there's more reasons than that, but those are the major ones.

-- 
Shawn.
Junio C Hamano· Oct 10, 2006, 17:44 UTC · re: Sam Vilain · lore

Re: is git-unpack-objects now redundant for 'git-push' and friends?

Sam Vilain <sam@vilain.net> writes:
Show 5 quoted lines
> When pushing or pulling to/from a repository, why unpack the objects?
> Why not just fsck and then throw the pack into $GIT_DIR/objects/pack?
>...
> I suspect that this was never changed, because there never used to be
> more than one packfile allowed, correct?

We've supported more than one packs at the same time from the day we supported packs.

Primary reason that push never keeps the pack is that explosion into loose objects hurts you only at the first round and later pushes are supposed to be much smaller, so changing it to keep the pack for the initial one has a very small value making it a very low priority change. Not keeping the pack on push makes it possible to always use --thin for pushing if we wanted to.

On the fetch side you could use --keep option when you know the fetch will be huge.

← back to recent threads