# does a successful 'git gc' imply 'git fsck' 7 messages from 2012-12-02 to 2012-12-03. Participants: Sitaram Chamarty, Shawn Pearce, Junio C Hamano, Matthieu Moy. Thread: https://gitlist.dev/t/32260 ## Sitaram Chamarty, 2012-12-02 02:31 Subject: does a successful 'git gc' imply 'git fsck' Message-ID: URL: https://gitlist.dev/e/CAMK1S_iBq1ReGkjuy2UBENSQXOWyKj2ZzSCcg7Z89FVtVL2wMw%40mail.gmail.com ``` Hi, Background: I have a situation where I have to fix up a few hundred repos in terms of 'git gc' (the auto gc seems to have failed in many cases; they have far more than 6700 loose objects). I also found some corrupted objects in some cases that prevent the gc from completing. I am running "git gc" followed by "git fsck". The majority of the repos I have worked through so far appear to be fine, but in the larger repos (upwards of 2-3 GB) the git fsck is taking almost 5 times longer than the 'gc'. If I could assume that a successful 'git gc' means an fsck is not needed, I'd save a lot of time. Hence my question. -- Sitaram ``` ## Shawn Pearce, 2012-12-02 04:28 Subject: Re: does a successful 'git gc' imply 'git fsck' Message-ID: URL: https://gitlist.dev/e/CAJo%3DhJvHeA4g1fJZXm9VxpdvMFoHeEJW_r5YtXTDACEp09Xm_Q%40mail.gmail.com In-Reply-To: ``` On Sat, Dec 1, 2012 at 6:31 PM, Sitaram Chamarty wrote: > Background: I have a situation where I have to fix up a few hundred > repos in terms of 'git gc' (the auto gc seems to have failed in many > cases; they have far more than 6700 loose objects). I also found some > corrupted objects in some cases that prevent the gc from completing. > > I am running "git gc" followed by "git fsck". The majority of the > repos I have worked through so far appear to be fine, but in the > larger repos (upwards of 2-3 GB) the git fsck is taking almost 5 times > longer than the 'gc'. > > If I could assume that a successful 'git gc' means an fsck is not > needed, I'd save a lot of time. Hence my question. Not really. For example fsck verifies that every blob when decompressed and fully inflated matches its SHA-1. gc only checks connectivity of the commit and tree graph by making sure every object was accounted for. But when creating the output pack it only verifies a CRC-32 was correct when copying the bits from the source to the destination, it does not verify that the data decompresses and matches the SHA-1 it should match. So it depends on what level of check you need to feel safe. ``` ## Sitaram Chamarty, 2012-12-02 08:46 Subject: Re: does a successful 'git gc' imply 'git fsck' Message-ID: URL: https://gitlist.dev/e/CAMK1S_gUB6tpNQTLGTPPAvQ87%2BmX2HGiN1xpA7KoH33MX9tFBw%40mail.gmail.com In-Reply-To: ``` On Sun, Dec 2, 2012 at 9:58 AM, Shawn Pearce wrote: > On Sat, Dec 1, 2012 at 6:31 PM, Sitaram Chamarty wrote: >> Background: I have a situation where I have to fix up a few hundred >> repos in terms of 'git gc' (the auto gc seems to have failed in many >> cases; they have far more than 6700 loose objects). I also found some >> corrupted objects in some cases that prevent the gc from completing. >> >> I am running "git gc" followed by "git fsck". The majority of the >> repos I have worked through so far appear to be fine, but in the >> larger repos (upwards of 2-3 GB) the git fsck is taking almost 5 times >> longer than the 'gc'. >> >> If I could assume that a successful 'git gc' means an fsck is not >> needed, I'd save a lot of time. Hence my question. > > Not really. For example fsck verifies that every blob when > decompressed and fully inflated matches its SHA-1. gc only checks OK that makes sense. After I posted I happened to check using strace and kinda guessed this from what I saw, but it's nice to have confirmation. > connectivity of the commit and tree graph by making sure every object > was accounted for. But when creating the output pack it only verifies > a CRC-32 was correct when copying the bits from the source to the > destination, it does not verify that the data decompresses and matches > the SHA-1 it should match. > > So it depends on what level of check you need to feel safe. Yup; thanks. All the repos my internal client manages are mirrored in multiple places, and they set (or were at least told to set, heh!) receive.fsckObjects so the lesser check is fine in most cases. -- Sitaram ``` ## Junio C Hamano, 2012-12-02 09:31 Subject: Re: does a successful 'git gc' imply 'git fsck' Message-ID: <7vlidgls09.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vlidgls09.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` Sitaram Chamarty writes: > If I could assume that a successful 'git gc' means an fsck is not > needed, I'd save a lot of time. Hence my question. When it does "repack -a", it at least scans the whole history so you would be sure that all the commits and trees are readable for the purpose of enumerating the objects referred by them (and a bit flip in them will likely be noticed by zlib inflation). But a "gc" does not necessarily run "repack -a" when it does not see too many pack files, so it can end up scanning only the surface of the history to collect the recently created loose objects into a pack, and stop its traversal without going into existing packfiles. ``` ## Sitaram Chamarty, 2012-12-03 13:14 Subject: Re: does a successful 'git gc' imply 'git fsck' Message-ID: URL: https://gitlist.dev/e/CAMK1S_gNgQORT8XRSMn0OoKRGqosoEB3_Jv84s0fQ9cpsHqMDQ%40mail.gmail.com In-Reply-To: <7vlidgls09.fsf@alter.siamese.dyndns.org> ``` On Sun, Dec 2, 2012 at 3:01 PM, Junio C Hamano wrote: > Sitaram Chamarty writes: > >> If I could assume that a successful 'git gc' means an fsck is not >> needed, I'd save a lot of time. Hence my question. > > When it does "repack -a", it at least scans the whole history so you > would be sure that all the commits and trees are readable for the > purpose of enumerating the objects referred by them (and a bit flip > in them will likely be noticed by zlib inflation). > > But a "gc" does not necessarily run "repack -a" when it does not see > too many pack files, so it can end up scanning only the surface of > the history to collect the recently created loose objects into a > pack, and stop its traversal without going into existing packfiles. Thanks; I'd missed this nuance as well... -- Sitaram ``` ## Matthieu Moy, 2012-12-03 13:46 Subject: Re: does a successful 'git gc' imply 'git fsck' Message-ID: URL: https://gitlist.dev/e/vpq4nk3uu39.fsf%40grenoble-inp.fr In-Reply-To: <7vlidgls09.fsf@alter.siamese.dyndns.org> ``` Junio C Hamano writes: > But a "gc" does not necessarily run "repack -a" when it does not see > too many pack files, so it can end up scanning only the surface of > the history to collect the recently created loose objects into a > pack, and stop its traversal without going into existing packfiles. Isn't that the behavior of "git gc --auto", not plain "git gc" ? -- Matthieu Moy http://www-verimag.imag.fr/~moy/ ``` ## Junio C Hamano, 2012-12-03 14:06 Subject: Re: does a successful 'git gc' imply 'git fsck' Message-ID: <7vpq2rtele.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vpq2rtele.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` Matthieu Moy writes: > Junio C Hamano writes: > >> But a "gc" does not necessarily run "repack -a" when it does not see >> too many pack files, so it can end up scanning only the surface of >> the history to collect the recently created loose objects into a >> pack, and stop its traversal without going into existing packfiles. > > Isn't that the behavior of "git gc --auto", not plain "git gc" ? True; I missed that Sitaram was running "gc" manually. ```