{"thread":{"id":"65947","subject":"Incremental 'git fetch' downloaded everything again","startedAt":"2026-07-07T22:32:26Z","lastAt":"2026-07-07T22:51:09Z","messageCount":2,"participants":["SZEDER Gábor","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"547409","messageId":"ak1+dsNQIV8EeSIc@szeder.dev","threadId":"65947","inReplyTo":null,"subject":"Incremental 'git fetch' downloaded everything again","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2026-07-07T22:32:22Z","receivedAt":"2026-07-07T22:32:26Z","isPatch":false,"body":"I usually try to fetch from https://github.com/git/git daily, and\ntoday morning something unusual happened:\n\n  $ git fetch\n  remote: Enumerating objects: 406099, done.\n  remote: Counting objects: 100% (1229/1229), done.\n  remote: Compressing objects: 100% (1044/1044), done.\n  remote: Total 406093 (delta 207), reused 1189 (delta 185), pack-reused 404864 (from 2)\n  Receiving objects: 100% (406093/406093), 292.22 MiB | 4.62 MiB/s, done.\n  Resolving deltas: 100% (308943/308943), done.\n  From https://github.com/git/git\n   + ffe2b816f5...106a830b98 jch        -> origin/jch  (forced update)\n     e9019fcafe..f85a7e6620  master     -> origin/master\n   + c42f45431d...00534a21ce next       -> origin/next  (forced update)\n   + f6884212b2...73452939f9 seen       -> origin/seen  (forced update)\n\nNote that it downloaded over 400k objects in an almost 300MB packfile.\n\nLooking at the objects that I already had and the objects in the newly\ndownloaded packfile:\n\n  $ git rev-list --objects origin/master@{1} origin/next@{1} origin/jch@{1} origin/seen@{1} | cut -d' ' -f1 | sort >existing-objects\n  $ git verify-pack -v .git/objects/pack/pack-080fedc9c19f711dd1b22103b382ede8925b90a6.idx | sed -n -E -e 's/^([0-9a-f]{40}) .*/\\1/p' | sort >received-objects\n  $ wc -l existing-objects received-objects \n    406954 existing-objects\n    406093 received-objects\n    813047 total\n  $ git diff --no-index --stat existing-objects received-objects\n   existing-objects => received-objects | 3615 +++++++++++++---------------------\n   1 file changed, 1377 insertions(+), 2238 deletions(-)\n\nThe vast majority of objects were already available locally.\n\nWhat's going on?!\n\nThis might be a recurring issue: I remember a similar large download\nfrom 2 or 3 weeks ago, but back then I didn't have time to investigate\nor to report.\nI tried to reproduce this issue by attempting to recreate the state of\nmy git repository from yesterday in a new repo, but no luck, 'git\nfetch' only downloads what's necessary.\n\nI use a Git version based on next, with a bunch of my own patches on\ntop, but none of them has anything to do with object transfer and I've\nbeen using most of them for years.  I don't have any config set under\n'fetch.*' or 'transfer.*'.\n\n"},{"id":"547411","messageId":"xmqqh5maza5g.fsf@gitster.g","threadId":"65947","inReplyTo":"ak1+dsNQIV8EeSIc@szeder.dev","subject":"Re: Incremental 'git fetch' downloaded everything again","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-07T22:51:07Z","receivedAt":"2026-07-07T22:51:09Z","isPatch":false,"body":"SZEDER Gábor <szeder.dev@gmail.com> writes:\n\n> I usually try to fetch from https://github.com/git/git daily, and\n> today morning something unusual happened:\n>\n>   $ git fetch\n>   remote: Enumerating objects: 406099, done.\n>   remote: Counting objects: 100% (1229/1229), done.\n>   remote: Compressing objects: 100% (1044/1044), done.\n>   remote: Total 406093 (delta 207), reused 1189 (delta 185), pack-reused 404864 (from 2)\n>   Receiving objects: 100% (406093/406093), 292.22 MiB | 4.62 MiB/s, done.\n>   Resolving deltas: 100% (308943/308943), done.\n>   From https://github.com/git/git\n>    + ffe2b816f5...106a830b98 jch        -> origin/jch  (forced update)\n>      e9019fcafe..f85a7e6620  master     -> origin/master\n>    + c42f45431d...00534a21ce next       -> origin/next  (forced update)\n>    + f6884212b2...73452939f9 seen       -> origin/seen  (forced update)\n\nThe only difference I can think of is that 'next' has been rewound\nrecently, but because 'seen' and 'jch' are constantly reound, it\nwould be very strange that it made such a big difference.\n\nSo, sorry, no idea.  If I were bug-hunting this, I would first try\nto eliminate whatever GitHub runs on their server end from the\npicture.\n\n> Note that it downloaded over 400k objects in an almost 300MB packfile.\n>\n> Looking at the objects that I already had and the objects in the newly\n> downloaded packfile:\n>\n>   $ git rev-list --objects origin/master@{1} origin/next@{1} origin/jch@{1} origin/seen@{1} | cut -d' ' -f1 | sort >existing-objects\n>   $ git verify-pack -v .git/objects/pack/pack-080fedc9c19f711dd1b22103b382ede8925b90a6.idx | sed -n -E -e 's/^([0-9a-f]{40}) .*/\\1/p' | sort >received-objects\n>   $ wc -l existing-objects received-objects \n>     406954 existing-objects\n>     406093 received-objects\n>     813047 total\n>   $ git diff --no-index --stat existing-objects received-objects\n>    existing-objects => received-objects | 3615 +++++++++++++---------------------\n>    1 file changed, 1377 insertions(+), 2238 deletions(-)\n>\n> The vast majority of objects were already available locally.\n>\n> What's going on?!\n>\n> This might be a recurring issue: I remember a similar large download\n> from 2 or 3 weeks ago, but back then I didn't have time to investigate\n> or to report.\n> I tried to reproduce this issue by attempting to recreate the state of\n> my git repository from yesterday in a new repo, but no luck, 'git\n> fetch' only downloads what's necessary.\n>\n> I use a Git version based on next, with a bunch of my own patches on\n> top, but none of them has anything to do with object transfer and I've\n> been using most of them for years.  I don't have any config set under\n> 'fetch.*' or 'transfer.*'.\n"}]}