{"thread":{"id":"5399","subject":"Re: Problem with pack","startedAt":"2006-08-26T18:53:52Z","lastAt":"2006-08-26T19:24:01Z","messageCount":2,"participants":["Sergio Callegari","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"25951","messageId":"44F098C0.8000202@arces.unibo.it","threadId":"5399","inReplyTo":null,"subject":"Re: Problem with pack","fromName":"Sergio Callegari","fromEmail":"scallegari@arces.unibo.it","sentAt":"2006-08-26T18:53:52Z","receivedAt":"2006-08-26T18:53:52Z","isPatch":false,"sender":{"key":"scallegari@arces.unibo.it","avatar":null},"body":"> Earlier you said that the mothership has 1.4.2, and the note has\n> 1.4.0.  The sequence of events as I understand are:\n>\n> \t- repack -a -d on the mothership with 1.4.2; no problems\n>           observed.\n>\n>         - transfer the results to note; this was done behind git so\n>           no problems observed.\n>\n>         - tried to repack on note with 1.4.0; got \"failed to\n>           read delta-pack base object\" error.\n>   \nYes... only before that I had a few more iterations PC<->notebook always\nsyncing with unison.\n> Can you make the pack/idx available to the public for\n> postmortem?\n>   \nYes... I can make them available... the pack/idx actually do not contain\nanything extremely confidential (just a bunch of LaTeX files).\nOnly, being that there is conference data and stuff by people who\nprofessionally organize conferences, I'd prefer to make it available\ndirectly to some specific people that I can trust not re-distributing it\nrather than putting it in the general public.\n> Also I wonder if the pack can be read by 1.4.2.\n>   \nNo it cannot.\n> Earlier you said \"unpack-objects <$that-pack.pack\" fails with\n> \"error code -3 in inflate...\"  What exact error do you get?\n> I am guessing that it is get_data() which says:\n>\n> \t\"inflate returned %d\\n\"\n>   \nThis is right...\n>   \nJust a question...\nMight the problem have come out of a scenario like the following...\n\n1) I use unison to sync my documents (rather than using the git tools...\nsilly me!)\n2) I get things wrong in controlling unison (without realizing that I\ndo) and the result is that I lose some blobs.\n3) I repack an unclean tree (missing some objects)\n\nCan this be the case?\n\nThanks\n\nSergio\n"},{"id":"25952","messageId":"Pine.LNX.4.64.0608261203220.11811@g5.osdl.org","threadId":"5399","inReplyTo":"44F098C0.8000202@arces.unibo.it","subject":"Re: Problem with pack","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-08-26T19:24:01Z","receivedAt":"2006-08-26T19:24:01Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 26 Aug 2006, Sergio Callegari wrote:\n>\n> Might the problem have come out of a scenario like the following...\n> \n> 1) I use unison to sync my documents (rather than using the git tools...\n> silly me!)\n> 2) I get things wrong in controlling unison (without realizing that I\n> do) and the result is that I lose some blobs.\n> 3) I repack an unclean tree (missing some objects)\n> \n> Can this be the case?\n\nI do think that your synchronization using unison is _somehow_ part of the \nreason why bad things happened, but I really can't see why it would cause \nproblems, and perhaps more importantly, git should have noticed them \nearlier (and, in particular, failed the repack). So a git bug and/or \nmisfeature is involved somehow.\n\nOne thing that may have happened is that the use of unison somehow \ncorrupted an older pack (or you had a disk corruption), and that it was \nmissed because the corruption was in a delta of the old pack that was \nsilently re-used for the new one.\n\nThat would explain how the SHA1 of the pack-file matches - the repack \nwould have re-computed the SHA1 properly, but since the source delta \nitself was corrupt, the resulting new pack is corrupt.\n\nIf you had used git itself to synchronize the two repositories, that \ncorruption of one repo would have been noticed when it transfers the data \nover to the other side, which is one reason why the native git syncing \ntools are so superior to doing a filesystem-level synchronization.\n\nWith a filesystem-level sync (unison or anything else - rsync, cp -r, \netc), a problem introduced in one repository will be copied to another one \nwithout any sanity checking.\n\n(Which is not to say that the native protocol might not miss something \ntoo, but it should be _much_ harder to trigger: for anything but the \ninitial close, the native protocol will unpack all objects and recompute \ntheir SHA1 hashes from first principles on the receiving side, rather than \ntrust the sender implicitly, so it's fundamentally safer. But maybe we \ncould be even _more_ anal somewhere).\n\nSo as a suggestion if you want to be careful:\n\n - only use \"git fetch/pull\" to synchronize two git repos, because that's \n   inherently safer than any non-native synchronization.\n\n - if you repack reasonably often, do \"git fsck-objects\" (which is very \n   cheap when there aren't a lot of unpacked objects) to verify the \n   archive before \"git repack -a -d\" to repack it.\n\n - the \"fsck-objects\" thing won't catch a corrupt pack (unless you ask for \n   it with \"--full\", which is expensive for bigger projects), but at least \n   with \"git fetch/pull\", such corruption should not be able to replicate \n   to another repository.\n\nbut in the meantime, when you find a place to put the corrupt pack/index \nfile, please include me and Junio at a minimum into the group of people \nwho you tell where to find it (and/or passwords to access it). I'll \nhappily keep your data private (I've done it before for others).\n\n\t\tLinus\n"}]}