{"thread":{"id":"21276","subject":"git fsck not identifying corrupted packs","startedAt":"2009-10-19T07:56:59Z","lastAt":"2009-10-20T20:49:03Z","messageCount":21,"participants":["Sergio Callegari","Johannes Sixt","Johannes Schindelin","Gabor Gombas","Junio C Hamano","Wesley J. Landaker","Matthieu Moy","Alex Riesen","Robin Rosenberg","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"125351","messageId":"loom.20091019T094924-194@post.gmane.org","threadId":"21276","inReplyTo":null,"subject":"git fsck not identifying corrupted packs","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2009-10-19T07:56:59Z","receivedAt":"2009-10-19T07:56:59Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Hi,\n\nI have a pack that contains a corrupted object.\nIt is an old corrupted repo that I have conserved.\n\nAs expected, git gc cries out loud about it.\nIt indicates an inflate error (data stream error with incorrect data check),\nand then the impossibility to read an object from a certain offset in the pack.\n\nHowever, git fsck does not complain at all about the repo.\nI guess that for speed reasons, git fsck does not try to inflate the objects.\n\nIs there a means to have fsck to a truly full check on the sanity of a repo?\n\nThis both on git 1.6.5.1 and 1.6.4.2.\n\nThanks\n\nSergio Callegari\n"},{"id":"125363","messageId":"4ADC2D45.3020803@viscovery.net","threadId":"21276","inReplyTo":"loom.20091019T094924-194@post.gmane.org","subject":"Re: git fsck not identifying corrupted packs","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2009-10-19T09:11:33Z","receivedAt":"2009-10-19T09:11:33Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Sergio Callegari schrieb:\n> Is there a means to have fsck to a truly full check on the sanity of a repo?\n\ngit fsck --full\n\nRTFM, please.\n\n-- Hannes\n"},{"id":"125367","messageId":"alpine.DEB.1.00.0910191202020.4985@pacific.mpi-cbg.de","threadId":"21276","inReplyTo":"4ADC2D45.3020803@viscovery.net","subject":"Re: git fsck not identifying corrupted packs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-10-19T10:04:34Z","receivedAt":"2009-10-19T10:04:34Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 19 Oct 2009, Johannes Sixt wrote:\n\n> Sergio Callegari schrieb:\n> > Is there a means to have fsck to a truly full check on the sanity of a \n> > repo?\n> \n> git fsck --full\n> \n> RTFM, please.\n\nNow, now.\n\nIf you were to test a new filesystem, say, wonderfulfs, and wanted to \ncheck its integrity, would you not just run \"fsck-wonderfulfs\" if that \nexists, rather than reading the fantamagastic manual?  Would you not \nexpect that it Does The Right Thing?  Would you not expect that it \nfollows the Law Of Minimal Surprise?\n\nSo FWIW I can see where Sergio is coming from.\n\nCiao,\nDscho\n"},{"id":"125370","messageId":"4ADC45C7.6090907@gmail.com","threadId":"21276","inReplyTo":"4ADC2D45.3020803@viscovery.net","subject":"Re: git fsck not identifying corrupted packs","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2009-10-19T10:56:07Z","receivedAt":"2009-10-19T10:56:07Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Johannes Sixt wrote:\n> Sergio Callegari schrieb:\n>   \n>> Is there a means to have fsck to a truly full check on the sanity of a repo?\n>>     \n>\n> git fsck --full\n>\n> RTFM, please.\n>\n>   \nRight... sorry for the noise, I mismatched --strict for --full in a script.\n\nBTW, the short help for fsck at --full only says \"consider objects in \nalternate repositories\".\n\nMy apologize.\n\nSergio\n"},{"id":"125405","messageId":"20091019183655.GC3630@boogie.lpds.sztaki.hu","threadId":"21276","inReplyTo":"4ADC2D45.3020803@viscovery.net","subject":"Re: git fsck not identifying corrupted packs","fromName":"Gabor Gombas","fromEmail":"gombasg@sztaki.hu","sentAt":"2009-10-19T18:36:55Z","receivedAt":"2009-10-19T18:36:55Z","isPatch":false,"sender":{"key":"gombasg@sztaki.hu","avatar":null},"body":"On Mon, Oct 19, 2009 at 11:11:33AM +0200, Johannes Sixt wrote:\n\n> > Is there a means to have fsck to a truly full check on the sanity of a repo?\n> \n> git fsck --full\n> \n> RTFM, please.\n\nThat still does not catch everything. About a week ago I wanted to check\nout a branch in a local repo and I got an error that it was corrupt. But\n\"git fsck --full\" only complained about some dangling objects, it did\nnot notice the corruption. I used git version 1.6.4.3 (Debian unstable\nat that time). It would be nice to have a\n\"git fsck --i-really-want-to-check-everything\".\n\nGabor\n\n-- \n     ---------------------------------------------------------\n     MTA SZTAKI Computer and Automation Research Institute\n                Hungarian Academy of Sciences\n     ---------------------------------------------------------\n"},{"id":"125408","messageId":"7v7hur1a0h.fsf@alter.siamese.dyndns.org","threadId":"21276","inReplyTo":"alpine.DEB.1.00.0910191202020.4985@pacific.mpi-cbg.de","subject":"Re: git fsck not identifying corrupted packs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-10-19T19:03:42Z","receivedAt":"2009-10-19T19:03:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> On Mon, 19 Oct 2009, Johannes Sixt wrote:\n>\n>> Sergio Callegari schrieb:\n>> > Is there a means to have fsck to a truly full check on the sanity of a \n>> > repo?\n>> \n>> git fsck --full\n>> \n>> RTFM, please.\n>\n> Now, now.\n>\n> If you were to test a new filesystem, say, wonderfulfs, and wanted to \n> check its integrity, would you not just run \"fsck-wonderfulfs\" if that \n> exists, rather than reading the fantamagastic manual?  Would you not \n> expect that it Does The Right Thing?  Would you not expect that it \n> follows the Law Of Minimal Surprise?\n>\n> So FWIW I can see where Sergio is coming from.\n\nLinus and other git developers from the early days trained their fingers\nto type the command, every once in a while even without thinking, to check\nthe consistency of the repository back when the lower core part of the git\nwas still being developed.  Developers who wanted to make sure that git\ncorrectly dealt with packfiles could deliberately trigger their creation\nand checked them after they were created carefully, but loose objects are\nthe ones that are written by various commands from random codepaths.  It\nmade some technical sense to have a mode that checked only loose objects\nfrom the debugging point of view for that reason.\n\n    Side note.  I think the help description of --full option is wrong (or\n    at least stale).  We always look at alternate object store these days\n    since e15ef66 (fsck: check loose objects from alternate object stores\n    by default, 2009-01-30).  It probably should read \"check packed\n    objects fully\" or something.\n\nThe above paragraph is merely a historical background, and in this case\nthe \"history\" refers to early-to-mid 2005.  Even for git developers there\nno longer is any reason to type \"git fsck\" in fear of some newly created\nobjects might be corrupt due to recent change to git these days.\n\nThe reason we did not make \"--full\" the default is probably we trust our\nfilesystems a bit too much.  At least, we trusted filesystems more than we\ntrusted the lower core part of git that was under development ;-)\n\nOnce a packfile is created and we always use it read-only, there didn't\nseem to be much point in suspecting that the underlying filesystems or\ndisks may corrupt them in such a way that is not caught by the SHA-1\nchecksum over the entire packfile and per object checksum.  That trust in\nthe filesystems might have been a good tradeoff between fsck performance\nand reliability on platforms git was initially developed on and for, but\nit might not be true anymore as we run on more platforms these days.\n\nIt probably makes sense to ship 1.7.0 with a version of \"fsck\" in which\n\"--full\" is the default; it would still accept \"--full\" but it would be a\nno-op.  This would be a backward incompatible change, but the difference\nis primarily about performance (\"it takes a lot longer than before!\"), and\nnot correctness, so we probably can live with it.  As I already said,\nthere is not much reason to run \"fsck\" every five minutes anymore to begin\nwith (unless your filesystem is so unreliable that it might eat one file\nevery five minutes, that is).\n\nIt probably is also a good idea to add a \"--loose\" option that does what\n\"fsck\" currently does without \"--full\".  It is a good name because (1) to\npeople who do not know the internal of git, it means \"check only loosely\",\nwhich would discourage them from running \"fack\" with that option to begin\nwith, and (2) to others, it exactly tells what the option makes the\ncommand check.\n"},{"id":"125409","messageId":"200910191307.56989.wjl@icecavern.net","threadId":"21276","inReplyTo":"4ADC45C7.6090907@gmail.com","subject":"Re: git fsck not identifying corrupted packs","fromName":"Wesley J. Landaker","fromEmail":"wjl@icecavern.net","sentAt":"2009-10-19T19:07:56Z","receivedAt":"2009-10-19T19:07:56Z","isPatch":false,"sender":{"key":"wjl@icecavern.net","avatar":"https://avatars.githubusercontent.com/u/67229?v=4"},"body":"On Monday 19 October 2009 04:56:07 Sergio Callegari wrote:\n> Johannes Sixt wrote:\n> > Sergio Callegari schrieb:\n> >> Is there a means to have fsck to a truly full check on the sanity of a\n> >> repo?\n> >\n> > git fsck --full\n> >\n> > RTFM, please.\n>\n> Right... sorry for the noise, I mismatched --strict for --full in a\n> script.\n>\n> BTW, the short help for fsck at --full only says \"consider objects in\n> alternate repositories\".\n\nUntil I read this thread, I didn't realize you needed --full to check\nobjects in packs.\n\nSince just every git repository I ever use has 99%+ of it's objects in\npacks, this means every time I've run \"git fsck\" it's essentially been\na no-op and I didn't know it. I imagine this is a common confusion.\n\nAlso, having --full mean both \"check alternate object pools\", and \"check\nobjects in packs\" seems to be rolling up two orthogonal issues.\n\nBut anyway, here is a patch that at least fixes the short option help to\nmatch the manual and the current behavior:\n\n--- 8< ---\nFrom 8fc3cd68d496bf00faad4f0a7b6ae4fee9437e68 Mon Sep 17 00:00:00 2001\nFrom: Wesley J. Landaker <wjl@icecavern.net>\nDate: Mon, 19 Oct 2009 12:48:07 -0600\nSubject: [PATCH] Update git fsck --full short description to mention packs\n\nThe '--full' option to git fsck does two things:\n\n  1) Check objects in packs\n  2) Check alternate objects\n\nThis is documented in the git fsck manual; this patch reflects that in\nthe short git fsck option help message as well.\n\nSigned-off-by: Wesley J. Landaker <wjl@icecavern.net>\n---\n builtin-fsck.c |    2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-fsck.c b/builtin-fsck.c\nindex c58b0e3..63212ea 100644\n--- a/builtin-fsck.c\n+++ b/builtin-fsck.c\n@@ -576,7 +576,7 @@ static struct option fsck_opts[] = {\n \tOPT_BOOLEAN(0, \"root\", &show_root, \"report root nodes\"),\n \tOPT_BOOLEAN(0, \"cache\", &keep_cache_objects, \"make index objects head nodes\"),\n \tOPT_BOOLEAN(0, \"reflogs\", &include_reflogs, \"make reflogs head nodes (default)\"),\n-\tOPT_BOOLEAN(0, \"full\", &check_full, \"also consider alternate objects\"),\n+\tOPT_BOOLEAN(0, \"full\", &check_full, \"also consider packs and alternate objects\"),\n \tOPT_BOOLEAN(0, \"strict\", &check_strict, \"enable more strict checking\"),\n \tOPT_BOOLEAN(0, \"lost-found\", &write_lost_and_found,\n \t\t\t\t\"write dangling objects in .git/lost-found\"),\n-- \n1.6.5\n"},{"id":"125411","messageId":"200910191327.49092.wjl@icecavern.net","threadId":"21276","inReplyTo":"7v7hur1a0h.fsf@alter.siamese.dyndns.org","subject":"Re: git fsck not identifying corrupted packs","fromName":"Wesley J. Landaker","fromEmail":"wjl@icecavern.net","sentAt":"2009-10-19T19:27:48Z","receivedAt":"2009-10-19T19:27:48Z","isPatch":false,"sender":{"key":"wjl@icecavern.net","avatar":"https://avatars.githubusercontent.com/u/67229?v=4"},"body":"(Not CCing everyone, since this is mostly curiosa in the \"using git as it \nwas never intended\" section):\n\nOn Monday 19 October 2009 13:03:42 Junio C Hamano wrote:\n> Once a packfile is created and we always use it read-only, there didn't\n> seem to be much point in suspecting that the underlying filesystems or\n> disks may corrupt them in such a way that is not caught by the SHA-1\n> checksum over the entire packfile and per object checksum.  That trust in\n> the filesystems might have been a good tradeoff between fsck performance\n> and reliability on platforms git was initially developed on and for, but\n> it might not be true anymore as we run on more platforms these days.\n\nFilesystems are mostly reliable, but only until your crazy users do strange \nand terrible things. I have a real, non-toy environment where I use this \nstack as a [horrible] workaround for some issues beyond my control:\n\ngit -> ext4 -> lvm -> dmcrypt -> loop -> sshfs -> cygwin sshd -> SMB share\n\nAmazingly, this works pretty reliably with many gigabytes of data in a git \nrepository, even with the occasional crash because of flakiness with the \n\"sshfs -> cygwin sshd\" piece of the puzzle. But a good \"git fsck\" sure \ndoesn't hurt in this environment! =)\n"},{"id":"125441","messageId":"vpq3a5etwfm.fsf@bauges.imag.fr","threadId":"21276","inReplyTo":"200910191307.56989.wjl@icecavern.net","subject":"Re: git fsck not identifying corrupted packs","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2009-10-20T06:24:13Z","receivedAt":"2009-10-20T06:24:13Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"\"Wesley J. Landaker\" <wjl@icecavern.net> writes:\n\n> Until I read this thread, I didn't realize you needed --full to check\n> objects in packs.\n\nSame here.\n\n> -\tOPT_BOOLEAN(0, \"full\", &check_full, \"also consider alternate objects\"),\n> +\tOPT_BOOLEAN(0, \"full\", &check_full, \"also consider packs and alternate objects\"),\n\nIMHO, something like this should be applied to \"maint\", this is kind\nof a serious bug indeed. Just check the \"alternate\" thing, according\nto Junio:\n\nJunio C Hamano <gitster@pobox.com> writes:\n\n>     Side note.  I think the help description of --full option is wrong (or\n>     at least stale).  We always look at alternate object store these days\n>     since e15ef66 (fsck: check loose objects from alternate object stores\n>     by default, 2009-01-30).  It probably should read \"check packed\n>     objects fully\" or something.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"125442","messageId":"vpqy6n6shri.fsf@bauges.imag.fr","threadId":"21276","inReplyTo":"7v7hur1a0h.fsf@alter.siamese.dyndns.org","subject":"Re: git fsck not identifying corrupted packs","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2009-10-20T06:26:25Z","receivedAt":"2009-10-20T06:26:25Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Linus and other git developers from the early days [...]\n\nThanks for the historical background.\n\n> It probably makes sense to ship 1.7.0 with a version of \"fsck\" in which\n> \"--full\" is the default; it would still accept \"--full\" but it would be a\n> no-op.\n\n+1\n\n> It probably is also a good idea to add a \"--loose\" option that does what\n> \"fsck\" currently does without \"--full\".  It is a good name\n\n+1 too.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"125447","messageId":"7vfx9esgvt.fsf@alter.siamese.dyndns.org","threadId":"21276","inReplyTo":"vpqy6n6shri.fsf@bauges.imag.fr","subject":"Re: git fsck not identifying corrupted packs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-10-20T06:45:26Z","receivedAt":"2009-10-20T06:45:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n>> Linus and other git developers from the early days [...]\n>\n> Thanks for the historical background.\n>\n>> It probably makes sense to ship 1.7.0 with a version of \"fsck\" in which\n>> \"--full\" is the default; it would still accept \"--full\" but it would be a\n>> no-op.\n>\n> +1\n>\n>> It probably is also a good idea to add a \"--loose\" option that does what\n>> \"fsck\" currently does without \"--full\".  It is a good name\n>\n> +1 too.\n\nActually, I changed my mind.  I do not think this so big that we need to\nwait for a major version bump.  Why not shoot for 1.6.6?\n"},{"id":"125460","messageId":"81b0412b0910200225g47220cc9wa2e82290a853c85d@mail.gmail.com","threadId":"21276","inReplyTo":"7vfx9esgvt.fsf@alter.siamese.dyndns.org","subject":"Re: git fsck not identifying corrupted packs","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2009-10-20T09:25:00Z","receivedAt":"2009-10-20T09:25:00Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Tue, Oct 20, 2009 at 08:45, Junio C Hamano <gitster@pobox.com> wrote:\n> Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n>>> It probably is also a good idea to add a \"--loose\" option that does what\n>>> \"fsck\" currently does without \"--full\".  It is a good name\n>>\n>> +1 too.\n>\n> Actually, I changed my mind.  I do not think this so big that we need to\n> wait for a major version bump.  Why not shoot for 1.6.6?\n\n--no-full works\n"},{"id":"125464","messageId":"alpine.DEB.1.00.0910201221250.4985@pacific.mpi-cbg.de","threadId":"21276","inReplyTo":"81b0412b0910200225g47220cc9wa2e82290a853c85d@mail.gmail.com","subject":"Re: git fsck not identifying corrupted packs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-10-20T10:22:40Z","receivedAt":"2009-10-20T10:22:40Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 20 Oct 2009, Alex Riesen wrote:\n\n> On Tue, Oct 20, 2009 at 08:45, Junio C Hamano <gitster@pobox.com> wrote:\n> > Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n> >>> It probably is also a good idea to add a \"--loose\" option that does what\n> >>> \"fsck\" currently does without \"--full\".  It is a good name\n> >>\n> >> +1 too.\n> >\n> > Actually, I changed my mind.  I do not think this so big that we need to\n> > wait for a major version bump.  Why not shoot for 1.6.6?\n> \n> --no-full works\n\nIt works.  Technically.  For human users, though, --loose-objects-only \n(with a shortcut \"--loose\") would be better.\n\nDisclaimer: this email was written by a bot and is valid without signature"},{"id":"125469","messageId":"vpq1vkygtx6.fsf@bauges.imag.fr","threadId":"21276","inReplyTo":"alpine.DEB.1.00.0910201221250.4985@pacific.mpi-cbg.de","subject":"Re: git fsck not identifying corrupted packs","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2009-10-20T11:56:53Z","receivedAt":"2009-10-20T11:56:53Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> Hi,\n>\n> On Tue, 20 Oct 2009, Alex Riesen wrote:\n>\n>> On Tue, Oct 20, 2009 at 08:45, Junio C Hamano <gitster@pobox.com> wrote:\n>> > Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n>> >>> It probably is also a good idea to add a \"--loose\" option that does what\n>> >>> \"fsck\" currently does without \"--full\".  It is a good name\n>> \n>> --no-full works\n>\n> It works.  Technically.  For human users, though, --loose-objects-only \n> (with a shortcut \"--loose\") would be better.\n\nOTOH, the advantage of \"--no-full\" is that it's compatible with\nexisting Git versions. If I learn Git 1.6.6 with --no-full, and use it\nin a script, then my stript works also with older Gits.\n\nBut anyway, I think very few people are actually interested in \"git\n--no-full\" (or call it whatever you like), so I don't think this is\nvery important.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"125485","messageId":"200910201741.50764.robin.rosenberg@dewire.com","threadId":"21276","inReplyTo":"200910191327.49092.wjl@icecavern.net","subject":"Re: git fsck not identifying corrupted packs","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2009-10-20T15:41:50Z","receivedAt":"2009-10-20T15:41:50Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"måndag 19 oktober 2009 21:27:48 skrev  Wesley J. Landaker:\n> (Not CCing everyone, since this is mostly curiosa in the \"using git as it\n> was never intended\" section):\n>\n> On Monday 19 October 2009 13:03:42 Junio C Hamano wrote:\n> > Once a packfile is created and we always use it read-only, there didn't\n> > seem to be much point in suspecting that the underlying filesystems or\n> > disks may corrupt them in such a way that is not caught by the SHA-1\n> > checksum over the entire packfile and per object checksum.  That trust in\n> > the filesystems might have been a good tradeoff between fsck performance\n> > and reliability on platforms git was initially developed on and for, but\n> > it might not be true anymore as we run on more platforms these days.\n>\n> Filesystems are mostly reliable, but only until your crazy users do strange\n> and terrible things. I have a real, non-toy environment where I use this\n> stack as a [horrible] workaround for some issues beyond my control:\n>\n> git -> ext4 -> lvm -> dmcrypt -> loop -> sshfs -> cygwin sshd -> SMB share\n\nThe obvious follow up question here is: Why?\n\n-- robin\n"},{"id":"125487","messageId":"200910201020.18676.wjl@icecavern.net","threadId":"21276","inReplyTo":"200910201741.50764.robin.rosenberg@dewire.com","subject":"Re: git fsck not identifying corrupted packs","fromName":"Wesley J. Landaker","fromEmail":"wjl@icecavern.net","sentAt":"2009-10-20T16:20:18Z","receivedAt":"2009-10-20T16:20:18Z","isPatch":false,"sender":{"key":"wjl@icecavern.net","avatar":"https://avatars.githubusercontent.com/u/67229?v=4"},"body":"On Tuesday 20 October 2009 09:41:50 Robin Rosenberg wrote:\n> måndag 19 oktober 2009 21:27:48 skrev  Wesley J. Landaker:\n> > (Not CCing everyone, since this is mostly curiosa in the \"using git as\n> > it was never intended\" section):\n[...]\n> > Filesystems are mostly reliable, but only until your crazy users do\n> > strange and terrible things. I have a real, non-toy environment where I\n> > use this stack as a [horrible] workaround for some issues beyond my\n> > control:\n> >\n> > git -> ext4 -> lvm -> dmcrypt -> loop -> sshfs -> cygwin sshd -> SMB\n> > share\n\nMy main point was to illustrate that having \"git fsck\" do a REALLY GOOD \nCHECK is still desirable, as we still haven't reached the days of file-\nsystem utopia where nothing ever gets corrupted (even with a smaller, \nsimpler stack).\n\nThe actual application where I use this stack is because of odd requirements \nand circumstances like data must be physically stored on a particular \nWindows server on the network that uses a weird authentication method that \nsamba doesn't support, and it has to go over the network encrypted anyway, \nthere are lots of holes in the data, so I want ext4 for the extent support, \nfile-size limitations on the target, etc.\n\nIt's a really an exotic love-hate mix between an off-by-one-please-no-never-\nagain kind of situation coupled with a bit of \"because I can\".\n\n> The obvious follow up question here is: Why?\n\nIf you are both nerdy and morbidly curious enough to care, send me a \"but, \nno ... really, WHY?!\" with the git list CC dropped and we can talk about \ndetails and/or other crazy stuff. (I don't want to get wildly off-topic on \nthis list.)\n"},{"id":"125524","messageId":"alpine.LFD.2.00.0910201437240.21460@xanadu.home","threadId":"21276","inReplyTo":"7vfx9esgvt.fsf@alter.siamese.dyndns.org","subject":"Re: git fsck not identifying corrupted packs","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2009-10-20T18:39:15Z","receivedAt":"2009-10-20T18:39:15Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Mon, 19 Oct 2009, Junio C Hamano wrote:\n\n> Actually, I changed my mind.  I do not think this so big that we need to\n> wait for a major version bump.  Why not shoot for 1.6.6?\n\nAgreed.  With a prominent note in the release notes to point people at \nit when they don't read release notes and complain that fsck suddenly \nbecame very slow after they upgraded.\n\n\nNicolas\n"},{"id":"125505","messageId":"7v3a5doqcg.fsf_-_@alter.siamese.dyndns.org","threadId":"21276","inReplyTo":"vpq1vkygtx6.fsf@bauges.imag.fr","subject":"[RFC/PATCH] fsck: default to \"git fsck --full\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-10-20T18:46:55Z","receivedAt":"2009-10-20T18:46:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus and other git developers from the early days trained their fingers\nto type the command, every once in a while even without thinking, to check\nthe consistency of the repository back when the lower core part of the git\nwas still being developed.  Developers who wanted to make sure that git\ncorrectly dealt with packfiles could deliberately trigger their creation\nand checked them after they were created carefully, but loose objects are\nthe ones that are written by various commands from random codepaths.  It\nmade some technical sense to have a mode that checked only loose objects\nfrom the debugging point of view for that reason.\n\nEven for git developers, there no longer is any reason to type \"git fsck\"\nevery five minutes these days, worried that some newly created objects\nmight be corrupt due to recent change to git.\n\nThe reason we did not make \"--full\" the default is probably we trust our\nfilesystems a bit too much.  At least, we trusted filesystems more than we\ntrusted the lower core part of git that was under development.\n\nOnce a packfile is created and we always use it read-only, there didn't\nseem to be much point in suspecting that the underlying filesystems or\ndisks may corrupt them in such a way that is not caught by the SHA-1\nchecksum over the entire packfile and per object checksum.  That trust in\nthe filesystems might have been a good tradeoff between fsck performance\nand reliability on platforms git was initially developed on and for, but\nit may not be true anymore as we run on many more platforms these days.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\nMatthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n> ...\n>> On Tue, 20 Oct 2009, Alex Riesen wrote:\n>> ...\n>>> --no-full works\n>>\n>> It works.  Technically.  For human users, though, --loose-objects-only \n>> (with a shortcut \"--loose\") would be better.\n>\n> OTOH, the advantage of \"--no-full\" is that it's compatible with\n> existing Git versions. If I learn Git 1.6.6 with --no-full, and use it\n> in a script, then my stript works also with older Gits.\n>\n> But anyway, I think very few people are actually interested in \"git\n> --no-full\" (or call it whatever you like), so I don't think this is\n> very important.\n\nFor human users, I think --full vs --no-full is quite a nice suggestion,\ngiven that we already have advertised --full and people know the option.\n\nAlso people know that splicing \"no-\" after the double dash is often the\nway to negate a boolean-looking option.\n\nThe actual patch to do this is tiny, but that is just a bonus ;-)\n\n Documentation/RelNotes-1.6.6.txt |   10 ++++++++++\n Documentation/git-fsck.txt       |    5 +++--\n builtin-fsck.c                   |    2 +-\n 3 files changed, 14 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/RelNotes-1.6.6.txt b/Documentation/RelNotes-1.6.6.txt\nindex 5f1fecb..1896e05 100644\n--- a/Documentation/RelNotes-1.6.6.txt\n+++ b/Documentation/RelNotes-1.6.6.txt\n@@ -1,6 +1,13 @@\n GIT v1.6.6 Release Notes\n ========================\n \n+In this release, \"git fsck\" defaults to \"git fsck --full\" and checks\n+packfiles.  If you prefer a quicker check only on loose objects (the\n+old default), you can say \"git fsck --no-full\".  This has been\n+supported by 1.5.4 and newer versions of git, so it is safe to write\n+it in your script if you use slightly older git on some of your\n+machines.\n+\n In git 1.7.0, which is planned to be the release after 1.6.6, \"git\n push\" into a branch that is currently checked out will be refused by\n default.\n@@ -38,6 +45,9 @@ Updates since v1.6.5\n \n (usability, bells and whistles)\n \n+ * \"git fsck\" by default checks the packfiles (i.e. \"--full\" is the\n+   default); you can turn it off with \"git fsck --no-full\".\n+\n  * \"git log --decorate\" shows the location of HEAD as well.\n \n (developers)\ndiff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\nindex 287c4fc..6fe9484 100644\n--- a/Documentation/git-fsck.txt\n+++ b/Documentation/git-fsck.txt\n@@ -10,7 +10,7 @@ SYNOPSIS\n --------\n [verse]\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n-\t [--full] [--strict] [--verbose] [--lost-found] [<object>*]\n+\t [--[no-]full] [--strict] [--verbose] [--lost-found] [<object>*]\n \n DESCRIPTION\n -----------\n@@ -52,7 +52,8 @@ index file, all SHA1 references in .git/refs/*, and all reflogs (unless\n \tor $GIT_DIR/objects/info/alternates,\n \tand in packed git archives found in $GIT_DIR/objects/pack\n \tand corresponding pack subdirectories in alternate\n-\tobject pools.\n+\tobject pools.  This is now default; you can turn it off\n+\twith --no-full.\n \n --strict::\n \tEnable more strict checking, namely to catch a file mode\ndiff --git a/builtin-fsck.c b/builtin-fsck.c\nindex c58b0e3..2d88e45 100644\n--- a/builtin-fsck.c\n+++ b/builtin-fsck.c\n@@ -19,7 +19,7 @@ static int show_root;\n static int show_tags;\n static int show_unreachable;\n static int include_reflogs = 1;\n-static int check_full;\n+static int check_full = 1;\n static int check_strict;\n static int keep_cache_objects;\n static unsigned char head_sha1[20];\n"},{"id":"125507","messageId":"alpine.LFD.2.00.0910201457310.21460@xanadu.home","threadId":"21276","inReplyTo":"7v3a5doqcg.fsf_-_@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] fsck: default to \"git fsck --full\"","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2009-10-20T19:00:31Z","receivedAt":"2009-10-20T19:00:31Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 20 Oct 2009, Junio C Hamano wrote:\n\n> diff --git a/Documentation/RelNotes-1.6.6.txt b/Documentation/RelNotes-1.6.6.txt\n> index 5f1fecb..1896e05 100644\n> --- a/Documentation/RelNotes-1.6.6.txt\n> +++ b/Documentation/RelNotes-1.6.6.txt\n> @@ -1,6 +1,13 @@\n>  GIT v1.6.6 Release Notes\n>  ========================\n>  \n> +In this release, \"git fsck\" defaults to \"git fsck --full\" and checks\n> +packfiles.  If you prefer a quicker check only on loose objects (the\n             ^^\n\nMight be worth mentioning explicitly that, because of that change, plain \nfsck is now going to take much longer to complete.\n\n\nNicolas\n"},{"id":"125508","messageId":"7vy6n5namx.fsf@alter.siamese.dyndns.org","threadId":"21276","inReplyTo":"alpine.LFD.2.00.0910201457310.21460@xanadu.home","subject":"Re: [RFC/PATCH] fsck: default to \"git fsck --full\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-10-20T19:11:34Z","receivedAt":"2009-10-20T19:11:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nicolas Pitre <nico@fluxnic.net> writes:\n\n> On Tue, 20 Oct 2009, Junio C Hamano wrote:\n>\n>> diff --git a/Documentation/RelNotes-1.6.6.txt b/Documentation/RelNotes-1.6.6.txt\n>> index 5f1fecb..1896e05 100644\n>> --- a/Documentation/RelNotes-1.6.6.txt\n>> +++ b/Documentation/RelNotes-1.6.6.txt\n>> @@ -1,6 +1,13 @@\n>>  GIT v1.6.6 Release Notes\n>>  ========================\n>>  \n>> +In this release, \"git fsck\" defaults to \"git fsck --full\" and checks\n>> +packfiles.  If you prefer a quicker check only on loose objects (the\n>              ^^\n>\n> Might be worth mentioning explicitly that, because of that change, plain \n> fsck is now going to take much longer to complete.\n\nSounds fair; thanks.\n\ndiff --git a/Documentation/RelNotes-1.6.6.txt b/Documentation/RelNotes-1.6.6.txt\nindex 0adf998..fa0e11a 100644\n--- a/Documentation/RelNotes-1.6.6.txt\n+++ b/Documentation/RelNotes-1.6.6.txt\n@@ -2,7 +2,8 @@ GIT v1.6.6 Release Notes\n ========================\n \n In this release, \"git fsck\" defaults to \"git fsck --full\" and checks\n-packfiles.  If you prefer a quicker check only on loose objects (the\n+packfiles, and because of this it will take much longer to complete\n+than before.  If you prefer a quicker check only on loose objects (the\n old default), you can say \"git fsck --no-full\".  This has been\n supported by 1.5.4 and newer versions of git, so it is safe to write\n it in your script even if you use slightly older git on some of your\n"},{"id":"125527","messageId":"81b0412b0910201349o185e9e92l8cb737aa7b75513d@mail.gmail.com","threadId":"21276","inReplyTo":"alpine.LFD.2.00.0910201437240.21460@xanadu.home","subject":"Re: git fsck not identifying corrupted packs","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2009-10-20T20:49:03Z","receivedAt":"2009-10-20T20:49:03Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Tue, Oct 20, 2009 at 20:39, Nicolas Pitre <nico@fluxnic.net> wrote:\n> On Mon, 19 Oct 2009, Junio C Hamano wrote:\n>\n>> Actually, I changed my mind.  I do not think this so big that we need to\n>> wait for a major version bump.  Why not shoot for 1.6.6?\n>\n> Agreed.  With a prominent note in the release notes to point people at\n> it when they don't read release notes and complain that fsck suddenly\n> became very slow after they upgraded.\n\nI have a feeling that it either wont be noticed at all (i.e., I have run fsck\nwith something other than --full only one time in my whole bash history,\nand never shown it otherwise to anyone else), or people will immediately\nlike it (\"Oh, finallly! Now that feels like it is doing something!\" :)\n"}]}