{"thread":{"id":"59258","subject":"Feature request: Add --mtime option to git archive","startedAt":"2023-02-16T19:41:47Z","lastAt":"2023-02-22T23:24:12Z","messageCount":16,"participants":["Raul E Rangel","Jeff King","Junio C Hamano","Raul Rangel","René Scharfe","demerphq","brian m. carlson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"472220","messageId":"Y+6G9n6cWRT9EKyl@google.com","threadId":"59258","inReplyTo":null,"subject":"Feature request: Add --mtime option to git archive","fromName":"Raul E Rangel","fromEmail":"rrangel@chromium.org","sentAt":"2023-02-16T19:41:42Z","receivedAt":"2023-02-16T19:41:47Z","isPatch":false,"sender":{"key":"rrangel@chromium.org","avatar":null},"body":"When generating a tarball with `git archive `, `git archive` will\nuse the current time as the mtime. This results in a non-hermetic\ntarball. Could we should add a --mtime option that allows passing in\nthe time? \n\nThanks!\n"},{"id":"472228","messageId":"Y+6akicTFG9n0eZy@coredump.intra.peff.net","threadId":"59258","inReplyTo":"Y+6G9n6cWRT9EKyl@google.com","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-02-16T21:05:22Z","receivedAt":"2023-02-16T21:05:28Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 16, 2023 at 12:41:42PM -0700, Raul E Rangel wrote:\n\n> When generating a tarball with `git archive `, `git archive` will\n> use the current time as the mtime. This results in a non-hermetic\n> tarball. Could we should add a --mtime option that allows passing in\n> the time? \n\nThat seems like a very reasonable feature to have. Just to sketch out\nthe implementation, in case anybody wants to work on it:\n\n - options are read first in cmd_archive(), but it passes unknown ones\n on to write_archive(), which is where we parse the interesting ones.\n\n - that relies on parse_archive_args() in archive.c. So we'd want a new\n option in opts[] there, and to write the value into a new field in\n the archiver_args struct.\n\n - the time is filled in via parse_treeish_arg(), where we use\n time(NULL) if there's no commit. And there we'd just use the value\n from archiver_args.\n\n If there is a commit and they've specified --mtime, what should\n happen? Quietly ignore it? Override the commit's timestamp? Complain\n and bail?\n\n - I didn't look at how this might interact with \"git archive\n --remote\". I think it might just all work, as we just ship the\n options to the other side, which parses them itself using the same\n code.\n\n-Peff\n"},{"id":"472229","messageId":"xmqq5yc1p7yn.fsf@gitster.g","threadId":"59258","inReplyTo":"Y+6akicTFG9n0eZy@coredump.intra.peff.net","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-02-16T22:21:36Z","receivedAt":"2023-02-16T22:21:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Feb 16, 2023 at 12:41:42PM -0700, Raul E Rangel wrote:\n>\n>> When generating a tarball with `git archive `, `git archive` will\n>> use the current time as the mtime. This results in a non-hermetic\n>> tarball. Could we should add a --mtime option that allows passing in\n>> the time? \n>\n> That seems like a very reasonable feature to have. Just to sketch out\n> the implementation, in case anybody wants to work on it: ...\n\nThere has been a discussion on not just \"fix mtime\" but coming up\nwith a lot more stable tar archive format specification.\n\nThe first link is about documenting the status quo. The second one\nis about the proposed format that aims to be stable.\n\nhttps://lore.kernel.org/git/20230203080629.31492-1-ray@ameretat.dev/\nhttps://lore.kernel.org/git/20230205221728.4179674-2-sandals@crustytoothpaste.net/\n\n"},{"id":"472243","messageId":"Y+7PcqpYhF5ZuApG@coredump.intra.peff.net","threadId":"59258","inReplyTo":"xmqq5yc1p7yn.fsf@gitster.g","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-02-17T00:50:58Z","receivedAt":"2023-02-17T00:51:24Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 16, 2023 at 02:21:36PM -0800, Junio C Hamano wrote:\n\n> >> When generating a tarball with `git archive `, `git archive` will\n> >> use the current time as the mtime. This results in a non-hermetic\n> >> tarball. Could we should add a --mtime option that allows passing in\n> >> the time? \n> >\n> > That seems like a very reasonable feature to have. Just to sketch out\n> > the implementation, in case anybody wants to work on it: ...\n> \n> There has been a discussion on not just \"fix mtime\" but coming up\n> with a lot more stable tar archive format specification.\n\nYes, I think in brian's proposal the mtime would always be 0. I don't\nmind that either (and really, I doubt anybody would really want to set\n--mtime to anything but a fixed, known value anyway). This is a much\nsmaller change that could be done in the meantime with less effort, but\nI guess we'd be stuck with --mtime forever, then.\n\nA similar option in is to simply start using \"0\" in the meantime, like:\n\ndiff --git a/archive.c b/archive.c\nindex 81ff76fce9..48d89785c3 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -470,7 +470,7 @@ static void parse_treeish_arg(const char **argv,\n \t\tarchive_time = commit->date;\n \t} else {\n \t\tcommit_oid = NULL;\n-\t\tarchive_time = time(NULL);\n+\t\tarchive_time = 0;\n \t}\n \n \ttree = parse_tree_indirect(&oid);\n\nNobody will complain about changing the byte-for-byte format, since by definition it\nwas already changing once per second (cue somebody complaining that they\nhave been using LD_PRELOAD tricks to simulate --mtime).\n\nI do wonder if people would complain (both with the patch above and with\nbrian's proposal) that the resulting tarballs extract everything with a\ndate in 1970. That's not functionally a problem, but it looks kind of\nweird in \"ls -l\".\n\n-Peff\n"},{"id":"472245","messageId":"xmqqpma9m4i1.fsf@gitster.g","threadId":"59258","inReplyTo":"Y+7PcqpYhF5ZuApG@coredump.intra.peff.net","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-02-17T02:04:38Z","receivedAt":"2023-02-17T02:04:43Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> A similar option in is to simply start using \"0\" in the meantime, like:\n>\n> diff --git a/archive.c b/archive.c\n> index 81ff76fce9..48d89785c3 100644\n> --- a/archive.c\n> +++ b/archive.c\n> @@ -470,7 +470,7 @@ static void parse_treeish_arg(const char **argv,\n> \t\tarchive_time = commit->date;\n> \t} else {\n> \t\tcommit_oid = NULL;\n> -\t\tarchive_time = time(NULL);\n> +\t\tarchive_time = 0;\n> \t}\n> \n> \ttree = parse_tree_indirect(&oid);\n>\n> Nobody will complain about changing the byte-for-byte format, since by definition it\n> was already changing once per second (cue somebody complaining that they\n> have been using LD_PRELOAD tricks to simulate --mtime).\n>\n> I do wonder if people would complain (both with the patch above and with\n> brian's proposal) that the resulting tarballs extract everything with a\n> date in 1970. That's not functionally a problem, but it looks kind of\n> weird in \"ls -l\".\n\nAnd owned by root:root ;-)\n\nI am sure people would complain. What matters is if these\ncomplaints have merit, and in this case, I doubt it. I especially\nlike your \"it has been already changing once per second\" reasoning\nfor this change.\n"},{"id":"472256","messageId":"CAHQZ30Dq1_vdJj_hakqXKFUbuqn9ysNsw-zzN83RmUVbibA3Gw@mail.gmail.com","threadId":"59258","inReplyTo":"xmqqpma9m4i1.fsf@gitster.g","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"Raul Rangel","fromEmail":"rrangel@chromium.org","sentAt":"2023-02-17T15:43:03Z","receivedAt":"2023-02-17T15:43:20Z","isPatch":false,"sender":{"key":"rrangel@chromium.org","avatar":null},"body":"On Thu, Feb 16, 2023 at 7:04 PM Junio C Hamano wrote:\n>\n> Jeff King writes:\n>\n> > A similar option in is to simply start using \"0\" in the meantime, like:\n> >\n> > diff --git a/archive.c b/archive.c\n> > index 81ff76fce9..48d89785c3 100644\n> > --- a/archive.c\n> > +++ b/archive.c\n> > @@ -470,7 +470,7 @@ static void parse_treeish_arg(const char **argv,\n> > archive_time = commit->date;\n> > } else {\n> > commit_oid = NULL;\n> > - archive_time = time(NULL);\n> > + archive_time = 0;\n> > }\n> >\n> > tree = parse_tree_indirect(&oid);\n> >\n> > Nobody will complain about changing the byte-for-byte format, since by definition it\n> > was already changing once per second (cue somebody complaining that they\n> > have been using LD_PRELOAD tricks to simulate --mtime).\n> >\n> > I do wonder if people would complain (both with the patch above and with\n> > brian's proposal) that the resulting tarballs extract everything with a\n> > date in 1970. That's not functionally a problem, but it looks kind of\n> > weird in \"ls -l\".\n>\n> And owned by root:root ;-)\n\nI fully support both of those easy changes. Only reason I proposed\n--mtime was that https://reproducible-builds.org/docs/archives/\nrecommends setting SOURCE_DATE_EPOCH, but honestly for my purposes I\nwould always use 0 and root:root.\n\n>\n> I am sure people would complain. What matters is if these\n> complaints have merit, and in this case, I doubt it. I especially\n> like your \"it has been already changing once per second\" reasoning\n> for this change.\n"},{"id":"472264","messageId":"Y+/iu7j+NsS/3sAX@coredump.intra.peff.net","threadId":"59258","inReplyTo":"xmqqpma9m4i1.fsf@gitster.g","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-02-17T20:25:31Z","receivedAt":"2023-02-17T20:26:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 16, 2023 at 06:04:38PM -0800, Junio C Hamano wrote:\n\n> > I do wonder if people would complain (both with the patch above and with\n> > brian's proposal) that the resulting tarballs extract everything with a\n> > date in 1970. That's not functionally a problem, but it looks kind of\n> > weird in \"ls -l\".\n> \n> And owned by root:root ;-)\n\nYeah, though I think it's pretty standard these days to use\n--no-same-owner or equivalent (and it's the default for non-root users).\nAnd ditto for modes. I wondered if there was an equivalent for\ntimestamps. It looks like \"-m\" works (it just sets everything to the\ncurrent timestamp at time of extraction), but I don't know how portable\nthat is (and POSIX is no help with tar, as usual).\n\n-Peff\n"},{"id":"472265","messageId":"83635ee4-02a2-dea7-38fa-c7b65c30c897@web.de","threadId":"59258","inReplyTo":"CAHQZ30Dq1_vdJj_hakqXKFUbuqn9ysNsw-zzN83RmUVbibA3Gw@mail.gmail.com","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2023-02-17T20:31:45Z","receivedAt":"2023-02-17T20:32:13Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 17.02.23 um 16:43 schrieb Raul Rangel:\n> On Thu, Feb 16, 2023 at 7:04 PM Junio C Hamano wrote:\n>>\n>> Jeff King writes:\n>>\n>>> A similar option in is to simply start using \"0\" in the meantime, like:\n>>>\n>>> diff --git a/archive.c b/archive.c\n>>> index 81ff76fce9..48d89785c3 100644\n>>> --- a/archive.c\n>>> +++ b/archive.c\n>>> @@ -470,7 +470,7 @@ static void parse_treeish_arg(const char **argv,\n>>> archive_time = commit->date;\n>>> } else {\n>>> commit_oid = NULL;\n>>> - archive_time = time(NULL);\n>>> + archive_time = 0;\n>>> }\n>>>\n>>> tree = parse_tree_indirect(&oid);\n>>>\n>>> Nobody will complain about changing the byte-for-byte format, since by definition it\n>>> was already changing once per second (cue somebody complaining that they\n>>> have been using LD_PRELOAD tricks to simulate --mtime).\n\nAnd how are you now going to tell the current time on a remote git\nserver? ;)\n\n>>> I do wonder if people would complain (both with the patch above and with\n>>> brian's proposal) that the resulting tarballs extract everything with a\n>>> date in 1970. That's not functionally a problem, but it looks kind of\n>>> weird in \"ls -l\".\n\nYes, 21st century artifacts being sent back in time looks a bit strange.\nGNU tar and bsdtar provide option -m to ignore mtimes when extracting.\n\n>> And owned by root:root ;-)\n>\n> I fully support both of those easy changes. Only reason I proposed\n> --mtime was that https://reproducible-builds.org/docs/archives/\n> recommends setting SOURCE_DATE_EPOCH, but honestly for my purposes I\n> would always use 0 and root:root.\n\ngit archive already records root as user and group.\n\n>> I am sure people would complain. What matters is if these\n>> complaints have merit, and in this case, I doubt it. I especially\n>> like your \"it has been already changing once per second\" reasoning\n>> for this change.\n\nI find it quite convincing as well.\n\nRené\n\n"},{"id":"472291","messageId":"CANgJU+VaF7-SJgGPqYGEV5VcJd_nTt2SMOQ5u9mNZ_wsArKT6g@mail.gmail.com","threadId":"59258","inReplyTo":"xmqqpma9m4i1.fsf@gitster.g","subject":"Re: Feature request: Add --mtime option to git archive","fromName":"demerphq","fromEmail":"demerphq@gmail.com","sentAt":"2023-02-18T03:04:12Z","receivedAt":"2023-02-18T03:04:27Z","isPatch":false,"sender":{"key":"demerphq@gmail.com","avatar":null},"body":"On Fri, 17 Feb 2023 at 03:39, Junio C Hamano wrote:\n> > I do wonder if people would complain (both with the patch above and with\n> > brian's proposal) that the resulting tarballs extract everything with a\n> > date in 1970. That's not functionally a problem, but it looks kind of\n> > weird in \"ls -l\".\n>\n> And owned by root:root ;-)\n>\n> I am sure people would complain. What matters is if these\n> complaints have merit, and in this case, I doubt it. I especially\n> like your \"it has been already changing once per second\" reasoning\n> for this change.\n\nI don't really get the argument as far as it is presented as a reason\n/not/ to support a --mtime option as was requested in this thread.\n\nYou yourself say \"people will complain\", and you seem to be planning\nto just ignore those complaints as lacking merit. I honestly don't\nunderstand that, wouldn't it be less noise, less stress, and better\n\"customer service\" to let the user choose what constant is in the\narchive? Then no complaints, and you don't have to explain/argue\nto/with everybody why their complaint is without merit over and over.\n\nI could understand this position if letting the user set the --mtime\nimplied some harm that must be mitigated, but it seems like an odd\nchoice if there is none. Especially given it would also future proof\ngit against people coming up with a good reason not to use the 0 epoch\nin the future that you haven't thought of right now. It is not like\nepoch's and unix time have a totally uncontroversial and stable\nhistory. 0 has meant multiple dates over time, and the current\ndefinition of 1 Jan 1970, 00:00:00 UTC is problematic as UTC didn't\nexist until 1972! Given it clearly wouldn't be hard to allow users to\nselect the epoch in these archives then why not do so?\n\nI have seen devs have issues with stuff like this in the past.\nUnpacking an archive on one machine showing a different date than one\nanother, or other weird artifacts. Mac used to use a different 0 epoch\nthan windows and linux as I recall, etc etc. I dont remember the gory\ndetails, but i have definitely seen people gnash their teeth over\nthese kind of decisions before. Why not side-step that if you can and\nlet people choose their own defaults?\n\nJust my $0.02.\n\ncheers,\nYves\n\n-- \nperl -Mre=debug -e \"/just|another|perl|hacker/\"\n"},{"id":"472298","messageId":"91a73f5d-ca3e-6cb0-4ba3-38d703074ee6@web.de","threadId":"59258","inReplyTo":"Y+6G9n6cWRT9EKyl@google.com","subject":"[PATCH] archive: add --mtime","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2023-02-18T08:36:23Z","receivedAt":"2023-02-18T08:36:44Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Allow users to specify the modification time of archive entries. The\nnew option --mtime uses approxidate() to parse a time specification and\noverrides the default of using the current time for trees and the commit\ntime for tags and commits. It can be used to create a reproducible\narchive for a tree, or to use a specific mtime without creating a commit\nwith GIT_COMMITTER_DATE set.\n\nThis implementation doesn't support the negated form of the new option,\ni.e. --no-mtime is not accepted. It is not possible to have no mtime at\nall. We could use the Unix epoch or revert to the default behavior, but\nsince negation is not necessary for the intended use it's left undecided\nfor now.\n\nRequested-by: Raul E Rangel \nSuggested-by: demerphq \nSigned-off-by: René Scharfe \n---\n Documentation/git-archive.txt | 5 +++++\n archive.c | 7 +++++++\n archive.h | 1 +\n t/t5000-tar-tree.sh | 19 +++++++++++++++++++\n 4 files changed, 32 insertions(+)\n\ndiff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt\nindex 60c040988b..6bab201d37 100644\n--- a/Documentation/git-archive.txt\n+++ b/Documentation/git-archive.txt\n@@ -86,6 +86,11 @@ cases, write an untracked file and use `--add-file` instead.\n \tLook for attributes in .gitattributes files in the working tree\n \tas well (see <>).\n\n+--mtime=