Remote corruption issue, linked to thin pack code?

Hi *,

This one sounds fairly similar to

  http://thread.gmane.org/gmane.comp.version-control.git/123244/

which was never resolved.

Jonas reported repo corruption on IRC along the lines of

  $ make gitpush
  cd dist && git push ssh://<user>@eloxoph.com/repos/mainrepo/bla.git master
   
    -  W e l c o m e   t o   E L O X O P H  -
               friendly landlord

  <user>@eloxoph.com's password: 
  Counting objects: 201, done.
  Delta compression using up to 2 threads.
  Compressing objects: 100% (132/132), done.
  Writing objects: 100% (133/133), 1.01 MiB, done.
  Total 133 (delta 118), reused 0 (delta 0)
  fatal: pack has 114 unresolved deltas
  error: unpack failed: index-pack abnormal exit
  To ssh://<user>@eloxoph.com/repos/mainrepo/bla.git
   ! [remote rejected] master -> master (n/a (unpacker error))
  error: failed to push some refs to 'ssh://<user>@eloxoph.com/repos/mainrepo/bla.git'
  make: *** [gitpush] Error 1

The respective repos show no errors with git-fsck. Jonas kindly provides a download link for both:

  http://eloxoph.com/localrepo.zip
  http://eloxoph.com/remoterepo.zip

Versions are 1.7.2.1 locally and 1.5.4.3 remotely.

Beware, there are hooks in both. Disabling them, however, only changes the error slightly:

  [...]
  Total 133 (delta 118), reused 0 (delta 0)
  fatal: pack has 114 unresolved deltas
  error: unpack failed: index-pack abnormal exit
  To ssh://mainrepo@eloxoph.com/repos/mainrepo/bla.git
   ! [remote rejected] master -> master (n/a (unpacker error))
  error: failed to push some refs to 'ssh://mainrepo@eloxoph.com/repos/mainrepo/bla.git'
  make: *** [gitpush] Error 1

What's even stranger is that fetching from the repo is also not possible:

  fatal: git-upload-pack: cannot find object e28ae6b61c384732c506544626c5083557dd2d75:
  fatal: The remote end hung up unexpectedly

despite the object being there.

What's also strange is that while there is a temporary pack inside objects/, I get

  $ git index-pack --stdin < objects/tmp_pack_oEUkIc
  fatal: pack has 114 unresolved deltas
  $ git index-pack --fix-thin -v --stdin < objects/tmp_pack_oEUkIc
  Receiving objects: 100% (133/133), 1.01 MiB, done.
  Resolving deltas: 100% (118/118), completed with 63 local objects.
  pack    061120577b0a1fec7ba636d6e3162f95f83543aa

So it seems the remote side got a thin pack and can't cope. But a4503a1 (Make --no-thin the default in git-push to save server resources, 2007-09-09), merged way back in 1.5.3.2, claims to enable --no-thin all the time. So how did a thin pack get there?

Any ideas?

On Wed, 25 Aug 2010, Thomas Rast wrote:

I don't think it actually is the same issue.

So... out of 118 deltas, 114 were relying on the remote to already have base objects for them (or some of them are deltas against unresolved deltas). In any case, the remote is missing some objects it advertised it should have.

What about 'git fsck --full'? Given git v1.5.4 on the remote, you should try --full with 'git fsck' as this wasn't the default back then.

$ wget http://eloxoph.com/remoterepo.zip --2010-08-25 21:24:52-- http://eloxoph.com/remoterepo.zip Resolving eloxoph.com... 85.214.104.74 Connecting to eloxoph.com|85.214.104.74|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2010-08-25 21:24:52 ERROR 404: Not Found.

How do you know it is there while git (on the remote) is telling you it is not?

What about 'git cat-file -t e28ae6b61c384732c506' ?

Hmmm... in this case it must be because git v1.5.4 is buggy.

/me looks

Well... apparently no.  In receive-pack.c:unpack() there is this code:

                keeper[0] = "index-pack";
                keeper[1] = "--stdin";
                keeper[2] = "--fix-thin";
                keeper[3] = hdr_arg;
                keeper[4] = keep_arg;
                keeper[5] = NULL;

so --fix-thin is always unconditionally passed to index-pack. This is also more or less the same code in v1.7.2.

Well, this was added back by accident a long time ago, and since then I also changed my mind about this, especially since 'git gc --auto' is executed on the remote after a push these days, so I never pushed a "fix" for it.

Regardless, the receiving end of a push shouldn't care either ways and always complete a thin pack.

So there are 2 mysteries here:

1) Why can't the remote repo be fetched? The error from git-upload-pack 
   is straight forward and has nothing to do with any thin packs.

2) Why adding --fix-thin to a manual invocation of index-pack does work 
   while the invocation from receive_pack which should have it doesn't?

Also, what is the OS version and filesystem used on the remote machine?

Maybe after I can reproduce this issue locally.

Nicolas

Nicolas Pitre wrote:

Well, I instructed him to try --full, as you can see in the conversation:

  http://colabti.org/irclogger/irclogger_log/git?date=2010-08-25#l2196

Nothing interesting.

Hrm.  Apparently a mis-edit on my part, it should be

  http://eloxoph.com/remoteserverrepo.zip

It's fine in my unpacked copy. Jonas, what were the results on the server?

As a side note, I couldn't reproduce locally with the above repos but then I didn't have the time to downgrade to 1.5.4.

Jonas also has to answer the rest of the questions for obvious reasons. (I'm away from the net for a few days anyway...)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

I'll try to provide the missing information as good as I can:

$ GIT_DIR=./ git fsck --full dangling commit d298351127861349846fe626a320c66101821d72 dangling commit a71ea8dd4c837c39ef26765574d515253ea2fd5a

$ GIT_DIR=./ git cat-file -t e28ae6b61c384732c506 commit

OS is Ubuntu 8.04 LTS. It's a virtual server by a hosting company, therefore I don't know much about the filesystem setup, but maybe that helps:

$ df -T:
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/vzfs reiserfs    10485760   4056808   6428952  39% /
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iF4EAREIAAYFAkx2Ph8ACgkQVizsUChSmww9TAEAptTyg+0yqidArbg2wMLmgac+ PrK9T3g4cU20EBOEuXkBALlqLapTQLlyO8UoFi+I3Mq64sDBBdjaEqH8bi6LcTor =ahRa -----END PGP SIGNATURE-----