{"thread":{"id":"29529","subject":"[RFC/PATCH 0/2] Commits with ancient timestamps","startedAt":"2012-02-02T21:41:41Z","lastAt":"2012-02-03T18:01:24Z","messageCount":6,"participants":["Junio C Hamano","Thomas Rast","Han-Wen Nienhuys"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"183669","messageId":"1328218903-5681-1-git-send-email-gitster@pobox.com","threadId":"29529","inReplyTo":null,"subject":"[RFC/PATCH 0/2] Commits with ancient timestamps","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-02T21:41:41Z","receivedAt":"2012-02-02T21:41:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Result of conversion of ancient history from other SCMs, and output from\nother third-party tools, can record timestamps that predates inception of\nGit. They can cause \"git am\", \"git rebase\" and \"git commit --amend\" to\nmisbehave, because the raw git timestamp e.g.\n\n\tauthor 1328214896 -0800\n\nare read from the commit object and passed to parse_date() machinery,\nwhich rejects raw git timestamps without sufficient number of digits to\navoid misinterpreting human-written timestamp in other formats, and\ntimestamps before 1975 do not have enough number of digits in them.\n\nHere is a two-patch series that may improve the situation.\n\nNote that this is not meant for direct application, unless somebody uses\nthis extensively to make sure there is no regression.\n\n\nJunio C Hamano (2):\n parse_date(): allow ancient timestamps in in-header format\n parse_date(): '@' prefix forces git-timestamp interpretation\n\n date.c | 30 ++++++++++++++++++++++++++++++\n git-sh-setup.sh | 2 +-\n 2 files changed, 31 insertions(+), 1 deletions(-)\n\n-- \n1.7.9.172.ge26ae\n"},{"id":"183670","messageId":"1328218903-5681-2-git-send-email-gitster@pobox.com","threadId":"29529","inReplyTo":"1328218903-5681-1-git-send-email-gitster@pobox.com","subject":"[RFC/PATCH 1/2] parse_date(): allow ancient git-timestamp","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-02T21:41:42Z","receivedAt":"2012-02-02T21:41:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The date-time parser parses out a human-readble datestring piece by\npiece, so that it could even parse a string in a rather strange\nnotation like 'noon november 11, 2005', but restricts itself from\nparsing strings in \" \" format only\nfor reasonably new timestamps (like 1974 or newer) with 10 or more\ndigits. This is to prevent a string like \"20100917\" from getting\ninterpreted as seconds since epoch (we want to treat it as September\n17, 2010 instead) while doing so.\n\nThe same codepath is used to read back the timestamp that we have\nalready recorded in the headers of commit and tag objects; because\nof this, such a commit with timestamp \"0 +0000\" cannot be rebased or\namended very easily.\n\nTeach parse_date() codepath to special case a string of the form\n\" +<4-digits>\" to work this issue around, but require that\nthere is no other cruft around the string when parsing a timestamp\nof this format for safety.\n\nNote that this has a slight backward incompatibility implications.\n\nIf somebody writes \"git commit --date='20100917 +0900'\" and wants it\nto mean a timestamp in September 2010 in Japan, this change will\nbreak such a use case.\n\nSigned-off-by: Junio C Hamano \n---\n date.c | 29 +++++++++++++++++++++++++++++\n 1 files changed, 29 insertions(+), 0 deletions(-)\n\ndiff --git a/date.c b/date.c\nindex 896fbb4..c212946 100644\n--- a/date.c\n+++ b/date.c\n@@ -585,6 +585,33 @@ static int date_string(unsigned long date, int offset, char *buf, int len)\n \treturn snprintf(buf, len, \"%lu %c%02d%02d\", date, sign, offset / 60, offset % 60);\n }\n \n+/*\n+ * Parse a string like \"0 +0000\" as ancient timestamp near epoch, but\n+ * only when it appears not as part of any other string.\n+ */\n+static int match_object_header_date(const char *date, unsigned long *timestamp, int *offset)\n+{\n+\tchar *end;\n+\tunsigned long stamp;\n+\tint ofs;\n+\n+\tif (*date < '0' || '9' <= *date)\n+\t\treturn -1;\n+\tstamp = strtoul(date, &end, 10);\n+\tif (*end != ' ' || stamp == ULONG_MAX || (end[1] != '+' && end[1] != '-'))\n+\t\treturn -1;\n+\tdate = end + 2;\n+\tofs = strtol(date, &end, 10);\n+\tif ((*end != '\\0' && (*end != '\\n')) || end != date + 4)\n+\t\treturn -1;\n+\tofs = (ofs / 100) * 60 + (ofs % 100);\n+\tif (date[-1] == '-')\n+\t\tofs = -ofs;\n+\t*timestamp = stamp;\n+\t*offset = ofs;\n+\treturn 0;\n+}\n+\n /* Gr. strptime is crap for this; it doesn't have a way to require RFC2822\n (i.e. English) day/month names, and it doesn't work correctly with %z. */\n int parse_date_basic(const char *date, unsigned long *timestamp, int *offset)\n@@ -610,6 +637,8 @@ int parse_date_basic(const char *date, unsigned long *timestamp, int *offset)\n \t*offset = -1;\n \ttm_gmt = 0;\n \n+\tif (!match_object_header_date(date, timestamp, offset))\n+\t\treturn 0; /* success */\n \tfor (;;) {\n \t\tint match = 0;\n \t\tunsigned char c = *date;\n-- \n1.7.9.172.ge26ae\n"},{"id":"183671","messageId":"1328218903-5681-3-git-send-email-gitster@pobox.com","threadId":"29529","inReplyTo":"1328218903-5681-1-git-send-email-gitster@pobox.com","subject":"[RFC/PATCH 2/2] parse_date(): '@' prefix forces git-timestamp","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-02T21:41:43Z","receivedAt":"2012-02-02T21:41:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The only place that the issue this series addresses was observed\nwhere we read \"cat-file commit\" output and put it in GIT_AUTHOR_DATE\nin order to replay a commit with an ancient timestamp.\n\nWith the previous patch alone, \"git commit --date='20100917 +0900'\"\ncan be misinterpreted to mean an ancient timestamp, not September in\nyear 2010. Guard this codepath by requring an extra '@' in front of\nthe raw git timestamp on the parsing side. This of course needs to\nbe compensated by updating get_author_ident_from_commit and the code\nfor \"git commit --amend\" to prepend '@' to the string read from the\nexisting commit in the GIT_AUTHOR_DATE environment variable.\n\nSigned-off-by: Junio C Hamano \n---\n builtin/commit.c | 6 ++++++\n date.c | 3 ++-\n git-sh-setup.sh | 2 +-\n t/t3400-rebase.sh | 23 +++++++++++++++++++++++\n 4 files changed, 32 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/commit.c b/builtin/commit.c\nindex cbc9613..bcb0db2 100644\n--- a/builtin/commit.c\n+++ b/builtin/commit.c\n@@ -534,6 +534,7 @@ static void determine_author_info(struct strbuf *author_ident)\n \n \tif (author_message) {\n \t\tconst char *a, *lb, *rb, *eol;\n+\t\tsize_t len;\n \n \t\ta = strstr(author_message_buffer, \"\\nauthor \");\n \t\tif (!a)\n@@ -554,6 +555,11 @@ static void determine_author_info(struct strbuf *author_ident)\n \t\t\t\t\t (a + strlen(\"\\nauthor \"))));\n \t\temail = xmemdupz(lb + strlen(\"<\"), rb - (lb + strlen(\"<\")));\n \t\tdate = xmemdupz(rb + strlen(\"> \"), eol - (rb + strlen(\"> \")));\n+\t\tlen = eol - (rb + strlen(\"> \"));\n+\t\tdate = xmalloc(len + 2);\n+\t\t*date = '@';\n+\t\tmemcpy(date + 1, rb + strlen(\"> \"), len);\n+\t\tdate[len + 1] = '\\0';\n \t}\n \n \tif (force_author) {\ndiff --git a/date.c b/date.c\nindex c212946..ca60767 100644\n--- a/date.c\n+++ b/date.c\n@@ -637,7 +637,8 @@ int parse_date_basic(const char *date, unsigned long *timestamp, int *offset)\n \t*offset = -1;\n \ttm_gmt = 0;\n \n-\tif (!match_object_header_date(date, timestamp, offset))\n+\tif (*date == '@' &&\n+\t !match_object_header_date(date + 1, timestamp, offset))\n \t\treturn 0; /* success */\n \tfor (;;) {\n \t\tint match = 0;\ndiff --git a/git-sh-setup.sh b/git-sh-setup.sh\nindex 8e427da..015fe6e 100644\n--- a/git-sh-setup.sh\n+++ b/git-sh-setup.sh\n@@ -200,7 +200,7 @@ get_author_ident_from_commit () {\n \t\ts/.*/GIT_AUTHOR_EMAIL='\\''&'\\''/p\n \n \t\tg\n-\t\ts/^author [^<]* <[^>]*> \\(.*\\)$/\\1/\n+\t\ts/^author [^<]* <[^>]*> \\(.*\\)$/@\\1/\n \t\ts/.*/GIT_AUTHOR_DATE='\\''&'\\''/p\n \n \t\tq\ndiff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh\nindex 6eaecec..e26e14d 100755\n--- a/t/t3400-rebase.sh\n+++ b/t/t3400-rebase.sh\n@@ -218,4 +218,27 @@ test_expect_success 'rebase -m can copy notes' '\n \ttest \"a note\" = \"$(git notes show HEAD)\"\n '\n \n+test_expect_success 'rebase commit with an ancient timestamp' '\n+\tgit reset --hard &&\n+\n+\t>old.one && git add old.one && test_tick &&\n+\tgit commit --date=\"@12345 +0400\" -m \"Old one\" &&\n+\t>old.two && git add old.two && test_tick &&\n+\tgit commit --date=\"@23456 +0500\" -m \"Old two\" &&\n+\t>old.three && git add old.three && test_tick &&\n+\tgit commit --date=\"@34567 +0600\" -m \"Old three\" &&\n+\n+\tgit cat-file commit HEAD^^ >actual &&\n+\tgrep \"author .* 12345 +0400$\" actual &&\n+\tgit cat-file commit HEAD^ >actual &&\n+\tgrep \"author .* 23456 +0500$\" actual &&\n+\tgit cat-file commit HEAD >actual &&\n+\tgrep \"author .* 34567 +0600$\" actual &&\n+\n+\tgit rebase --onto HEAD^^ HEAD^ &&\n+\n+\tgit cat-file commit HEAD >actual &&\n+\tgrep \"author .* 34567 +0600$\" actual\n+'\n+\n test_done\n-- \n1.7.9.172.ge26ae\n"},{"id":"183706","messageId":"87mx90yz5y.fsf@thomas.inf.ethz.ch","threadId":"29529","inReplyTo":"1328218903-5681-1-git-send-email-gitster@pobox.com","subject":"Re: [RFC/PATCH 0/2] Commits with ancient timestamps","fromName":"Thomas Rast","fromEmail":"trast@inf.ethz.ch","sentAt":"2012-02-03T10:44:09Z","receivedAt":"2012-02-03T10:44:09Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Junio C Hamano writes:\n\n> avoid misinterpreting human-written timestamp in other formats, and\n> timestamps before 1975 do not have enough number of digits in them.\n>\n> Here is a two-patch series that may improve the situation.\n\nDoing this just makes me wonder how important exactly the 1970-1975\nrange is. Is there a notable software history from that era that can be\nrecovered?\n\n(Your [1/2] does not seem to parse negative offsets from the unix epoch,\nso anything before 1970 is still out.)\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"183737","messageId":"CAFQ2z_Nb-3gai8aXRv+03N0YrAYOJ_sDPn+jruGCvYkTQAr-Pg@mail.gmail.com","threadId":"29529","inReplyTo":"1328218903-5681-1-git-send-email-gitster@pobox.com","subject":"Re: [RFC/PATCH 0/2] Commits with ancient timestamps","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@google.com","sentAt":"2012-02-03T14:53:32Z","receivedAt":"2012-02-03T14:53:32Z","isPatch":true,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"On Thu, Feb 2, 2012 at 7:41 PM, Junio C Hamano wrote:\n> Result of conversion of ancient history from other SCMs, and output from\n> other third-party tools, can record timestamps that predates inception of\n> Git. They can cause \"git am\", \"git rebase\" and \"git commit --amend\" to\n> misbehave, because the raw git timestamp e.g.\n>\n>        author 1328214896 -0800\n>\n> are read from the commit object and passed to parse_date() machinery,\n\nAs a bit of context: we have some internal tools at Google that create\nadministrative commits that should have no timestamp. I am using \"0\"\nand \"1\" as a deterministic timestamps in these cases (ie. the start of\nthe epoch). While this works well in general, there are some git\nsubcommands that barf on this, causing user-unhappiness.\n\nThis patch will hopefully resolve these breakages.\n\n-- \nHan-Wen Nienhuys\nGoogle Engineering Belo Horizonte\nhanwen@google.com\n"},{"id":"183746","messageId":"7v1uqbssnf.fsf@alter.siamese.dyndns.org","threadId":"29529","inReplyTo":"87mx90yz5y.fsf@thomas.inf.ethz.ch","subject":"Re: [RFC/PATCH 0/2] Commits with ancient timestamps","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-02-03T18:01:24Z","receivedAt":"2012-02-03T18:01:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast writes:\n\n> Doing this just makes me wonder how important exactly the 1970-1975\n> range is. Is there a notable software history from that era that can be\n> recovered?\n\nThat is not really a valid question. People who wrote private stuff in\nthat era deserve to be users of Git, too.\n\n> (Your [1/2] does not seem to parse negative offsets from the unix epoch,\n> so anything before 1970 is still out.)\n\nYes, pre-epoch timestamps are also useful for projects like US\nConstitution.\n\n http://thread.gmane.org/gmane.comp.version-control.git/152433/focus=152725\n\nFor that, we would need to use and pass around time_t (or intmax_t if we\nfollow the reason why originally Linus chose to avoid time_t) throughout\nthe codebase. If you actually write commit objects that record pre-epoch\ntimestamps, however, they will become unreadable by the current versions\nof Git (as they would not understand such a negative raw timestamp).\n\nIn any case, that is a goal for a much longer term.\n\nBut even after such a change happens, you still need a way for Git to\nreplay a raw timestamp stored in commit objects without regressing the\nparse_date() interface too much. These two patches show one way to do so\nwith minimum disruption.\n\nAs an added bonus, with the second patch, the way to spell a raw timestamp\nhappens to become compatible with how GNU date accepts one, i.e.\n\n $ date -d @1000000000\n\neven though we do not have to encourage the use of this notation by humans,\ntools and script writers may find it useful.\n"}]}