{"thread":{"id":"16025","subject":"Verifying the whole repository","startedAt":"2008-10-23T13:59:59Z","lastAt":"2008-10-23T14:28:04Z","messageCount":4,"participants":["Alex Bennee","David Symonds","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"93783","messageId":"b2cdc9f30810230659n15f44f64l571a0df3dbe104d9@mail.gmail.com","threadId":"16025","inReplyTo":null,"subject":"Verifying the whole repository","fromName":"Alex Bennee","fromEmail":"kernel-hacker@bennee.com","sentAt":"2008-10-23T13:59:59Z","receivedAt":"2008-10-23T13:59:59Z","isPatch":false,"sender":{"key":"kernel-hacker@bennee.com","avatar":null},"body":"Hi,\n\nWhile I was debugging a crash in parsecvs while converting our CVS\nrepository I discovered it was because one of the CVS files had become\ncorrupted (truncated). This is a problem I've had before with RCS\nbased files which are prone to silent corruption that you won't notice\nuntil you try and checkout an old revision of the file.\n\nAs git is fundamentally hash based it's a lot easier to determine the\nhealth of the repository but I wonder if it's possible for silent\ncorruption to creep in which won't be noticed until you try and\ncheckout a historical commit of the tree. I notice there is a\ngit-verify-pack command that checks the pack files are OK. Do any of\nthe other commands implicitly ensure all objects in the repo are\ncorrect and valid? git-gc?\n\nAre there any other parts of the .git metadata that are crucial or is\nit enough to say if all objects and packs match their hashes you have\nall the information you may need to recover an arbitrary revision of\nthe repo?\n\n-- \nAlex, homepage: http://www.bennee.com/~alex/\n"},{"id":"93784","messageId":"ee77f5c20810230705l20339a1dj87b855bf3321f796@mail.gmail.com","threadId":"16025","inReplyTo":"b2cdc9f30810230659n15f44f64l571a0df3dbe104d9@mail.gmail.com","subject":"Re: Verifying the whole repository","fromName":"David Symonds","fromEmail":"dsymonds@gmail.com","sentAt":"2008-10-23T14:05:16Z","receivedAt":"2008-10-23T14:05:16Z","isPatch":false,"sender":{"key":"dsymonds@gmail.com","avatar":"https://gravatar.com/avatar/b22f5051cbfc11836e36cf7a690e6cde4e225d835e13295ff98d15c7a9ee3c0f?d=mp&s=160"},"body":"On Thu, Oct 23, 2008 at 6:59 AM, Alex Bennee wrote:\n\n> As git is fundamentally hash based it's a lot easier to determine the\n> health of the repository but I wonder if it's possible for silent\n> corruption to creep in which won't be noticed until you try and\n> checkout a historical commit of the tree. I notice there is a\n> git-verify-pack command that checks the pack files are OK. Do any of\n> the other commands implicitly ensure all objects in the repo are\n> correct and valid? git-gc?\n\nTry: git fsck --full --strict\n\n\nDave.\n"},{"id":"93785","messageId":"b2cdc9f30810230714x3301a15by9341de79d418f761@mail.gmail.com","threadId":"16025","inReplyTo":"ee77f5c20810230705l20339a1dj87b855bf3321f796@mail.gmail.com","subject":"Re: Verifying the whole repository","fromName":"Alex Bennee","fromEmail":"kernel-hacker@bennee.com","sentAt":"2008-10-23T14:14:12Z","receivedAt":"2008-10-23T14:14:12Z","isPatch":false,"sender":{"key":"kernel-hacker@bennee.com","avatar":null},"body":"On Thu, Oct 23, 2008 at 3:05 PM, David Symonds wrote:\n> On Thu, Oct 23, 2008 at 6:59 AM, Alex Bennee wrote:\n>> Do any of\n>> the other commands implicitly ensure all objects in the repo are\n>> correct and valid? git-gc?\n>\n> Try: git fsck --full --strict\n\nAhh, I forgot that git was written by a filesystem guy ;-)\n\nThanks.\n\n-- \nAlex, homepage: http://www.bennee.com/~alex/\n"},{"id":"93787","messageId":"20081023142804.GA14786@spearce.org","threadId":"16025","inReplyTo":"b2cdc9f30810230659n15f44f64l571a0df3dbe104d9@mail.gmail.com","subject":"Re: Verifying the whole repository","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-10-23T14:28:04Z","receivedAt":"2008-10-23T14:28:04Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Alex Bennee wrote:\n> As git is fundamentally hash based it's a lot easier to determine the\n> health of the repository but I wonder if it's possible for silent\n> corruption to creep in which won't be noticed until you try and\n> checkout a historical commit of the tree. I notice there is a\n> git-verify-pack command that checks the pack files are OK. Do any of\n> the other commands implicitly ensure all objects in the repo are\n> correct and valid? git-gc?\n\nAs David pointed out, git fsck can be used to verify all of the\nhashes, but git-gc also does a quick sanity check using a CRC code\nwhen it copies data from one pack to another pack.\n\nUnlike CVS Git has a write-once, read-many mentality, so with\nthe exception of git gc (err, actually the git repack it calls)\ngit never modifies an existing file. That really helps to reduce\nthe risk of corruption.\n\nIf you never do a gc or fsck operation (but still use say commit\nor push into the repository) then yes, silent corruption can still\nsneak up on you in the form of disk block corruption.\n\n> Are there any other parts of the .git metadata that are crucial or is\n> it enough to say if all objects and packs match their hashes you have\n> all the information you may need to recover an arbitrary revision of\n> the repo?\n\nDon't forget about the loose objects under .git/objects/?? but\notherwise yes, you just need the object data. The refs under\n.git/refs are also useful, but the tips can be recovered if the\nrefs space is lost by \"git fsck --unreachable\".\n\n-- \nShawn.\n"}]}