{"thread":{"id":"58634","subject":"git 2.38.0: Change in `git archive` output","startedAt":"2022-10-16T22:07:20Z","lastAt":"2022-10-17T19:14:16Z","messageCount":6,"participants":["kpcyrd","brian m. carlson","Jeff King","Konstantin Ryabitsev"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"465049","messageId":"20b14207-a6f2-033f-3419-271662bffba9@archlinux.org","threadId":"58634","inReplyTo":null,"subject":"git 2.38.0: Change in `git archive` output","fromName":"kpcyrd","fromEmail":"kpcyrd@archlinux.org","sentAt":"2022-10-16T21:57:40Z","receivedAt":"2022-10-16T22:07:20Z","isPatch":false,"sender":{"key":"kpcyrd@archlinux.org","avatar":null},"body":"hello,\n\nmultiple people in Arch Linux noticed the output of our `git archive` \ncommand doesn't match the tarball served by github anymore.\n\nFirst I suspected an update in our gzip package until I found this line \nin the git 2.38.0 release notes:\n\n > * Teach \"git archive\" to (optionally and then by default) avoid\n >   spawning an external \"gzip\" process when creating \".tar.gz\" (and\n >   \".tgz\") archives.\n\nI've then found this commit that could be considered a breaking change \nin `git archive`:\n\nhttps://github.com/git/git/commit/4f4be00d302bc52d0d9d5a3d4738bb525066c710\n\nI don't know if there's some kind of gzip standard that could be used to \nalign the git internal gzip implementation with gnu gzip.\n\nI'm not saying this is necessarily a bug or regression but it makes it \nharder to reproduce github tar balls from a git repository. Just sharing \nwhat I've debugged. :)\n\ncheers,\nkpcyrd\n"},{"id":"465051","messageId":"Y0yR5dPJyh8aetZA@tapette.crustytoothpaste.net","threadId":"58634","inReplyTo":"20b14207-a6f2-033f-3419-271662bffba9@archlinux.org","subject":"Re: git 2.38.0: Change in `git archive` output","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2022-10-16T23:21:09Z","receivedAt":"2022-10-16T23:21:16Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2022-10-16 at 21:57:40, kpcyrd wrote:\n> hello,\n\nHey,\n\n> multiple people in Arch Linux noticed the output of our `git archive`\n> command doesn't match the tarball served by github anymore.\n> \n> First I suspected an update in our gzip package until I found this line in\n> the git 2.38.0 release notes:\n> \n> > * Teach \"git archive\" to (optionally and then by default) avoid\n> >   spawning an external \"gzip\" process when creating \".tar.gz\" (and\n> >   \".tgz\") archives.\n> \n> I've then found this commit that could be considered a breaking change in\n> `git archive`:\n> \n> https://github.com/git/git/commit/4f4be00d302bc52d0d9d5a3d4738bb525066c710\n> \n> I don't know if there's some kind of gzip standard that could be used to\n> align the git internal gzip implementation with gnu gzip.\n> \n> I'm not saying this is necessarily a bug or regression but it makes it\n> harder to reproduce github tar balls from a git repository. Just sharing\n> what I've debugged. :)\n\nThis isn't a bug, because Git doesn't guarantee that archives produced\nby different versions will be bit-for-bit identical.  It does guarantee\nthat an archive of the same commit or tree using the same version of Git\nand associated tools running with the same configuration and environment\nwill be consistent (that is, a given version of Git produces\ndeterministic archives).\n\nI will also point out that GitHub also doesn't guarantee bit-for-bit\nidentical archives.  It does currently use git archive under the hood,\nbut that could change at any moment without notice.\n\nZip files also contain two sets of timestamps: local and UTC, and\ntherefore there's an additional element in which archives can differ\ndepending on the time zone.  In addition, using the export-subst\nfunctionality can result in short object IDs of different lengths\ndepending on the number of objects in the repository, so archive can\ndiffer for that reason as well.\n\nSo it's not the case that you can expect identical archives from Git and\nGitHub.  If you need to compute a hash over an archive, you need to\nstore the archive somewhere (on GitHub, that would be as a release\nasset).\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"465052","messageId":"Y0ybi66K40+uH+im@coredump.intra.peff.net","threadId":"58634","inReplyTo":"20b14207-a6f2-033f-3419-271662bffba9@archlinux.org","subject":"Re: git 2.38.0: Change in `git archive` output","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2022-10-17T00:02:19Z","receivedAt":"2022-10-17T00:02:25Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Oct 16, 2022 at 11:57:40PM +0200, kpcyrd wrote:\n\n> multiple people in Arch Linux noticed the output of our `git archive`\n> command doesn't match the tarball served by github anymore.\n> \n> First I suspected an update in our gzip package until I found this line in\n> the git 2.38.0 release notes:\n> \n> > * Teach \"git archive\" to (optionally and then by default) avoid\n> >   spawning an external \"gzip\" process when creating \".tar.gz\" (and\n> >   \".tgz\") archives.\n> \n> I've then found this commit that could be considered a breaking change in\n> `git archive`:\n> \n> https://github.com/git/git/commit/4f4be00d302bc52d0d9d5a3d4738bb525066c710\n> \n> I don't know if there's some kind of gzip standard that could be used to\n> align the git internal gzip implementation with gnu gzip.\n\nInteresting. For a small input, they seem to produce the same file for\nme:\n\n  git init repo\n  cd repo\n  seq 1000 >file\n  git add file\n  git commit -m foo\n \n  git -c tar.tar.gz.command='git archive gzip' \\\n    archive --format=tar.gz HEAD >internal.tar.gz\n  git -c tar.tar.gz.command='gzip -cn' \\\n    archive --format=tar.gz HEAD >external.tar.gz\n  cmp internal.tar.gz external.tar.gz && echo ok\n\nbut if I instead do \"seq 10000\", then the files differ. I didn't dig\ninto the actual binary to see the source of the change. It might be\nsomething we can tweak (e.g., if it's how a header is represented, or if\nwe can change the zlib parameters to find the same compressions).\n\n> I'm not saying this is necessarily a bug or regression but it makes it\n> harder to reproduce github tar balls from a git repository. Just sharing\n> what I've debugged. :)\n\nI don't think we make promises about stable output from \"git archive\".\nWe've fixed bugs in the tar-generating side before that lead to changes.\nBut if we can easily make them the same, that might be worth doing.\n\nIn the meantime, you can use the config option I showed above to get the\nold, external behavior. At some point GitHub will probably update their\nversion, though, at which point you'd want the internal (they may also\ntry to retain the old one, though; lots of distro/packaging projects get\nbroken when GitHub's archives aren't byte-for-byte identical).\n\n-Peff\n"},{"id":"465054","messageId":"Y0ynDbG8CxwAt4Fj@tapette.crustytoothpaste.net","threadId":"58634","inReplyTo":"Y0ybi66K40+uH+im@coredump.intra.peff.net","subject":"Re: git 2.38.0: Change in `git archive` output","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2022-10-17T00:51:25Z","receivedAt":"2022-10-17T00:51:32Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2022-10-17 at 00:02:19, Jeff King wrote:\n> Interesting. For a small input, they seem to produce the same file for\n> me:\n> \n>   git init repo\n>   cd repo\n>   seq 1000 >file\n>   git add file\n>   git commit -m foo\n> \n>   git -c tar.tar.gz.command='git archive gzip' \\\n>     archive --format=tar.gz HEAD >internal.tar.gz\n>   git -c tar.tar.gz.command='gzip -cn' \\\n>     archive --format=tar.gz HEAD >external.tar.gz\n>   cmp internal.tar.gz external.tar.gz && echo ok\n> \n> but if I instead do \"seq 10000\", then the files differ. I didn't dig\n> into the actual binary to see the source of the change. It might be\n> something we can tweak (e.g., if it's how a header is represented, or if\n> we can change the zlib parameters to find the same compressions).\n\nI will say that trying to make two compression implementations produce\nidentical output is likely futile because it's almost always the case\nthat there are multiple identical ways to encode the same data.  Most\nimplementations are going to prefer improving size over consistency, so\nthere's little incentive to copy the same algorithm across\nimplementations. I believe even GNU gzip has changed its output in the\npast as better optimizations were implemented.\n\nI mean, don't let me stop you from trying to tweak things to see if you\ncan make it work, but in general I think it's likely that some\ndivergence is going to occur between implementations no matter what.\n\n> I don't think we make promises about stable output from \"git archive\".\n> We've fixed bugs in the tar-generating side before that lead to changes.\n> But if we can easily make them the same, that might be worth doing.\n\nSince this is on the reproducible builds list, I would be interested in\nworking with tar implementations to specify a profile of the pax format\nthat _is_ standardized, stable, and consistent and that Git and other\nimplementations could use to produce bit-for-bit identical tar archives\nacross versions, since this is a thing lots of people seem to want.  (If\nthat's of interest, please contact me off list.)\n\nHowever, I don't think that trying to do that with compression formats\nis likely to lead to a productive work product, so users who cared about\nreproducibility would need to compare the uncompressed output.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"465115","messageId":"Y02K8bI1s74tEij4@coredump.intra.peff.net","threadId":"58634","inReplyTo":"Y0ynDbG8CxwAt4Fj@tapette.crustytoothpaste.net","subject":"Re: git 2.38.0: Change in `git archive` output","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2022-10-17T17:03:45Z","receivedAt":"2022-10-17T17:03:58Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 17, 2022 at 12:51:25AM +0000, brian m. carlson wrote:\n\n> > but if I instead do \"seq 10000\", then the files differ. I didn't dig\n> > into the actual binary to see the source of the change. It might be\n> > something we can tweak (e.g., if it's how a header is represented, or if\n> > we can change the zlib parameters to find the same compressions).\n> \n> I will say that trying to make two compression implementations produce\n> identical output is likely futile because it's almost always the case\n> that there are multiple identical ways to encode the same data.  Most\n> implementations are going to prefer improving size over consistency, so\n> there's little incentive to copy the same algorithm across\n> implementations. I believe even GNU gzip has changed its output in the\n> past as better optimizations were implemented.\n> \n> I mean, don't let me stop you from trying to tweak things to see if you\n> can make it work, but in general I think it's likely that some\n> divergence is going to occur between implementations no matter what.\n\nYeah, I definitely don't think it's something we ought to be promising,\nor do put a lot of work into. But if there's low-hanging fruit to reduce\nimmediate pain in practice, it seems worth considering.\n\n-Peff\n"},{"id":"465139","messageId":"20221017191406.uimmmmr6c6eiy56t@meerkat.local","threadId":"58634","inReplyTo":"20b14207-a6f2-033f-3419-271662bffba9@archlinux.org","subject":"Re: git 2.38.0: Change in `git archive` output","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2022-10-17T19:14:06Z","receivedAt":"2022-10-17T19:14:16Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Sun, Oct 16, 2022 at 11:57:40PM +0200, kpcyrd wrote:\n> I don't know if there's some kind of gzip standard that could be used to\n> align the git internal gzip implementation with gnu gzip.\n> \n> I'm not saying this is necessarily a bug or regression but it makes it\n> harder to reproduce github tar balls from a git repository. Just sharing\n> what I've debugged. :)\n\nI've previously complained about the output of .tar format changing, but I\nthink it's too much to expect the *compressed* output to remain the same. In\nenvironments where CPU time is more expensive than bandwidth, it's entirely\nnormal to expect the administrator to arbitrarily adjust compression levels\nwithout any warning, or, on the contrary, pre-generate and cache\nzopfli-compressed archives.\n\n-K\n"}]}