{"thread":{"id":"13884","subject":"Date parsing","startedAt":"2008-06-10T14:58:27Z","lastAt":"2008-06-21T22:40:42Z","messageCount":11,"participants":["Sverre Rabbelier","Johannes Sixt","Brandon Casey","Linus Torvalds","Stephan Beyer"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"79353","messageId":"bd6139dc0806100758xb41d08dh18e3051088b707e5@mail.gmail.com","threadId":"13884","inReplyTo":null,"subject":"Date parsing","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T14:58:27Z","receivedAt":"2008-06-10T14:58:27Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"Heya,\n\nI noticed that we parse\nGIT_COMMITTER_DATE=\"31-05-2008 11:30pm\" as \"31st of July at 23:30 +0200\"\nwhereas we parse\nGIT_COMMITTER_DATE=\"01-06-2008 11:30pm\" as \"6th of January at 23:30 +0100\"\n\nNot only is it strange that we accept both orders (since apparently we\ndon't resort to the latter order unless the first digit > 12), but\nit's also weird that the timezone changes based on which option was\nchosen! What does the list think about this? Is this \"normal\"\nbehavior? If not, what (if anything) should we do about it?\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"79355","messageId":"484E98BA.70808@viscovery.net","threadId":"13884","inReplyTo":"bd6139dc0806100758xb41d08dh18e3051088b707e5@mail.gmail.com","subject":"Re: Date parsing","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-06-10T15:07:38Z","receivedAt":"2008-06-10T15:07:38Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Sverre Rabbelier schrieb:\n> Heya,\n> \n> I noticed that we parse\n> GIT_COMMITTER_DATE=\"31-05-2008 11:30pm\" as \"31st of July at 23:30 +0200\"\n> whereas we parse\n> GIT_COMMITTER_DATE=\"01-06-2008 11:30pm\" as \"6th of January at 23:30 +0100\"\n> \n> Not only is it strange that we accept both orders (since apparently we\n> don't resort to the latter order unless the first digit > 12), but\n> it's also weird that the timezone changes based on which option was\n> chosen! What does the list think about this? Is this \"normal\"\n> behavior? If not, what (if anything) should we do about it?\n\nSince you did not give a timezone, your local timezone is taken,\npresumably MET, which has daylight saving time in July, but not in\nJanuary. This is OK.\n\n-- Hannes\n"},{"id":"79357","messageId":"Zx2uvEYSssjj9E0HrmUL8wYASyWX9L9w8LkR-gGRAmnD9isjoEeyKg@cipher.nrlssc.navy.mil","threadId":"13884","inReplyTo":"bd6139dc0806100758xb41d08dh18e3051088b707e5@mail.gmail.com","subject":"Re: Date parsing","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-06-10T15:10:33Z","receivedAt":"2008-06-10T15:10:33Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Sverre Rabbelier wrote:\n> Heya,\n> \n> I noticed that we parse\n> GIT_COMMITTER_DATE=\"31-05-2008 11:30pm\" as \"31st of July at 23:30 +0200\"\n> whereas we parse\n> GIT_COMMITTER_DATE=\"01-06-2008 11:30pm\" as \"6th of January at 23:30 +0100\"\n> \n> Not only is it strange that we accept both orders (since apparently we\n> don't resort to the latter order unless the first digit > 12),\n\nTake a look at match_multi_number in date.c\nEuropean ordering is preferred when the separator is '.'\n\n> but\n> it's also weird that the timezone changes based on which option was\n> chosen!\n\nThe timezone didn't change. One date is in daylight saving time, the other\nis not. The offset from GMT is different to reflect this.\n\n> What does the list think about this? Is this \"normal\"\n> behavior? If not, what (if anything) should we do about it?\n\nThere will be flaws in any system that tries to automatically guess the\nformat. If there is not already one, perhaps a statement should be added\nto the appropriate documentation describing how to ensure that european\nordering is used to parse the date.\n\n-brandon\n"},{"id":"79359","messageId":"bd6139dc0806100827m434748a6u8ce232e1ad967377@mail.gmail.com","threadId":"13884","inReplyTo":"484E98BA.70808@viscovery.net","subject":"Re: Date parsing","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T15:27:58Z","receivedAt":"2008-06-10T15:27:58Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Tue, Jun 10, 2008 at 5:07 PM, Johannes Sixt wrote:\n> Since you did not give a timezone, your local timezone is taken,\n> presumably MET, which has daylight saving time in July, but not in\n> January. This is OK.\n\nAh, then I agree the timezone change makes sense, although magical\nordering switching is still bothering me.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"79360","messageId":"bd6139dc0806100831y7a00a0f4sbc8dee9df7a8c16a@mail.gmail.com","threadId":"13884","inReplyTo":"Zx2uvEYSssjj9E0HrmUL8wYASyWX9L9w8LkR-gGRAmnD9isjoEeyKg@cipher.nrlssc.navy.mil","subject":"Re: Date parsing","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T15:31:07Z","receivedAt":"2008-06-10T15:31:07Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Tue, Jun 10, 2008 at 5:10 PM, Brandon Casey wrote:\n> Take a look at match_multi_number in date.c\n> European ordering is preferred when the separator is '.'\n\nOk, then I'll use . in the future, that's nice :).\n\n> The timezone didn't change. One date is in daylight saving time, the other\n> is not. The offset from GMT is different to reflect this.\n\nAs Hannes said, no bug there, my mistake.\n\n>> What does the list think about this? Is this \"normal\"\n>> behavior? If not, what (if anything) should we do about it?\n>\n> There will be flaws in any system that tries to automatically guess the\n> format. If there is not already one, perhaps a statement should be added\n> to the appropriate documentation describing how to ensure that european\n> ordering is used to parse the date.\n\nI think that it should bail out when it encounters \"20-01-2008\"\ninstead of automagically going for european notation. Even more\nhelpfull would be to inform the user that \"20.01.2008\" is the proper\nnotation.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"79364","messageId":"alpine.LFD.1.10.0806100942500.3101@woody.linux-foundation.org","threadId":"13884","inReplyTo":"bd6139dc0806100831y7a00a0f4sbc8dee9df7a8c16a@mail.gmail.com","subject":"Re: Date parsing","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-06-10T16:55:30Z","receivedAt":"2008-06-10T16:55:30Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 10 Jun 2008, Sverre Rabbelier wrote:\n> On Tue, Jun 10, 2008 at 5:10 PM, Brandon Casey wrote:\n> > Take a look at match_multi_number in date.c\n> > European ordering is preferred when the separator is '.'\n> \n> Ok, then I'll use . in the future, that's nice :).\n\nWell, there are safer ways to give the date.\n\nIf you do it in strict rfc822 format, you'll never have any confusion \nwhat-so-ever. The \"approxidate()\" thing tries to parse any random input, \nbut it *is* meant to be excessively liberal.\n\nIOW, you can literally say \"at tea-time two weeks ago\" and get a date, and \nit will even work. But you can also say \"my 3rd child was born in \nDecember\", and it will also give you a date. The date will not make \n_sense_, but it will give you one (it will decide that what you meant \nis \"Dec 3rd\").\n\nSo if you want to be precise and safe, you should be precise. I'd \npersonally suggest using yyyy-mm-dd, which is the ISO date format, \nalthough if that fails approxidate will still try the admittedly \ncrazy yyyy-dd-mm.\n\nAnd always set the timezone explicitly if you really care. Again, we try \nout best if you don't explicitly say which timezone to use, but if you \ndon't want any guessing - even _informed_ guessing - you really should \nstate things explicitly.\n\n> I think that it should bail out when it encounters \"20-01-2008\"\n> instead of automagically going for european notation. Even more\n> helpfull would be to inform the user that \"20.01.2008\" is the proper\n> notation.\n\nSee above. git approxidate() tries the exact reverse: it's extremely \nwilling to turn absolutely any line noise into a date.\n\nWhich is really nice when you do\n\n\tgit log @{last.week}..\n\nbut if you actually want to state an exact date it really means that the \nonus is on _you_ to be exact, and use a well-defined standard format.\n\n\t\t\tLinus\n"},{"id":"79373","messageId":"bd6139dc0806101051t5af7d14an57970f305b09173b@mail.gmail.com","threadId":"13884","inReplyTo":"alpine.LFD.1.10.0806100942500.3101@woody.linux-foundation.org","subject":"Re: Date parsing","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T17:51:19Z","receivedAt":"2008-06-10T17:51:19Z","isPatch":false,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Tue, Jun 10, 2008 at 6:55 PM, Linus Torvalds\n wrote:\n> On Tue, 10 Jun 2008, Sverre Rabbelier wrote:\n>> On Tue, Jun 10, 2008 at 5:10 PM, Brandon Casey wrote:\n>> > Take a look at match_multi_number in date.c\n>> > European ordering is preferred when the separator is '.'\n>>\n>> Ok, then I'll use . in the future, that's nice :).\n>\n> Well, there are safer ways to give the date.\n>\n> If you do it in strict rfc822 format, you'll never have any confusion\n> what-so-ever. The \"approxidate()\" thing tries to parse any random input,\n> but it *is* meant to be excessively liberal.\n>\n> IOW, you can literally say \"at tea-time two weeks ago\" and get a date, and\n> it will even work. But you can also say \"my 3rd child was born in\n> December\", and it will also give you a date. The date will not make\n> _sense_, but it will give you one (it will decide that what you meant\n> is \"Dec 3rd\").\n\nOk, so it'll eat anything it can make a date of, that makes sense I\nguess, considering what you refer to below.\n\n> So if you want to be precise and safe, you should be precise. I'd\n> personally suggest using yyyy-mm-dd, which is the ISO date format,\n> although if that fails approxidate will still try the admittedly\n> crazy yyyy-dd-mm.\n>\n> And always set the timezone explicitly if you really care. Again, we try\n> out best if you don't explicitly say which timezone to use, but if you\n> don't want any guessing - even _informed_ guessing - you really should\n> state things explicitly.\n\nI overlooked the fact that in Januari there is no DST, so that my\nlocal timezone would indeed be +1, so it handled it as desired.\n\n>> I think that it should bail out when it encounters \"20-01-2008\"\n>> instead of automagically going for european notation. Even more\n>> helpfull would be to inform the user that \"20.01.2008\" is the proper\n>> notation.\n>\n> See above. git approxidate() tries the exact reverse: it's extremely\n> willing to turn absolutely any line noise into a date.\n>\n> Which is really nice when you do\n>\n> git log @{last.week}..\n>\n> but if you actually want to state an exact date it really means that the\n> onus is on _you_ to be exact, and use a well-defined standard format.\n\nThank you, this explained why it works the way it does very well. I'll\ngo with ISO notation in the future :).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"80572","messageId":"20080621215240.GD15111@leksak.fem-net","threadId":"13884","inReplyTo":"alpine.LFD.1.10.0806100942500.3101@woody.linux-foundation.org","subject":"Re: Date parsing","fromName":"Stephan Beyer","fromEmail":"s-beyer@gmx.net","sentAt":"2008-06-21T21:52:40Z","receivedAt":"2008-06-21T21:52:40Z","isPatch":false,"sender":{"key":"s-beyer@gmx.net","avatar":"https://avatars.githubusercontent.com/u/143889?v=4"},"body":"Hi,\n\n> > On Tue, Jun 10, 2008 at 5:10 PM, Brandon Casey wrote:\n> > > Take a look at match_multi_number in date.c\n> > > European ordering is preferred when the separator is '.'\n> > \n> > Ok, then I'll use . in the future, that's nice :).\n> \n> Well, there are safer ways to give the date.\n> \n> If you do it in strict rfc822 format, you'll never have any confusion \n> what-so-ever. The \"approxidate()\" thing tries to parse any random input, \n> but it *is* meant to be excessively liberal.\n\nToday I've been playing around with approxidate(), too, and I think I\nfound some bug in date parsing. I let copy&paste speak...\n\nCorrect:\n\n$ ./test-date \"2008-07-01 23:59:59 +0200\"\n2008-07-01 23:59:59 +0200 -> 1214949599 +0200 -> Tue Jul 1 23:59:59 2008\n2008-07-01 23:59:59 +0200 -> Tue Jul 1 23:59:59 2008\n\nAnd even:\n\n$ ./test-date \"2008-07-01 24:00:00 +0200\"\n2008-07-01 24:00:00 +0200 -> 1214949600 +0200 -> Wed Jul 2 00:00:00 2008\n2008-07-01 24:00:00 +0200 -> Wed Jul 2 00:00:00 2008\n\nBut then there's a jump in time:\n\n$ ./test-date \"2008-07-02 00:00:00 +0200\"\n2008-07-02 00:00:00 +0200 -> 1202335200 +0200 -> Wed Feb 6 23:00:00 2008\n2008-07-02 00:00:00 +0200 -> Wed Feb 6 23:00:00 2008\n\n$ ./test-date \"2008-07-02 01:00:00 +0200\"\n2008-07-02 01:00:00 +0200 -> 1202338800 +0200 -> Thu Feb 7 00:00:00 2008\n2008-07-02 01:00:00 +0200 -> Thu Feb 7 00:00:00 2008\n\n\nIf we let test-date just print the timestamp...\n\ndiff --git a/test-date.c b/test-date.c\nindex 62e8f23..18d53c1 100644\n--- a/test-date.c\n+++ b/test-date.c\n@@ -11,10 +11,10 @@ int main(int argc, char **argv)\n \t\tmemcpy(result, \"bad\", 4);\n \t\tparse_date(argv[i], result, sizeof(result));\n \t\tt = strtoul(result, NULL, 0);\n-\t\tprintf(\"%s -> %s -> %s\", argv[i], result, ctime(&t));\n+\t\tprintf(\"%s -> %lu\\n\", argv[i], t);\n \n \t\tt = approxidate(argv[i]);\n-\t\tprintf(\"%s -> %s\\n\", argv[i], ctime(&t));\n+\t\tprintf(\"%s -> %lu\\n\", argv[i], t);\n \t}\n \treturn 0;\n }\n-- -- -- --\n\n... then we get:\n\n$ ./test-date \"2008-07-01 23:59:59 +0200\"\n2008-07-01 23:59:59 +0200 -> 1214949599\n2008-07-01 23:59:59 +0200 -> 1214949599\n\n$ ./test-date \"2008-07-02 00:00:00 +0200\"\n2008-07-02 00:00:00 +0200 -> 1202335200\n2008-07-02 00:00:00 +0200 -> 1202335200\n\nAlso, with another timezone, we get:\n\n$ ./test-date \"2008-07-01 23:59:59 +0000\"\n2008-07-01 23:59:59 +0000 -> 1214956799\n2008-07-01 23:59:59 +0000 -> 1214956799\n\n$ ./test-date \"2008-07-02 00:00:00 +0000\"\n2008-07-02 00:00:00 +0000 -> 1202342400\n2008-07-02 00:00:00 +0000 -> 1202342400\n\nIs something wrong in my format or is there a bug?\n\nProviding timestamps works, of course ;-)\n\nRegards,\n Stephan\n\n-- \nStephan Beyer , PGP 0x6EDDD207FCC5040F\n"},{"id":"80575","messageId":"alpine.LFD.1.10.0806211507370.2926@woody.linux-foundation.org","threadId":"13884","inReplyTo":"20080621215240.GD15111@leksak.fem-net","subject":"Re: Date parsing","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-06-21T22:28:56Z","receivedAt":"2008-06-21T22:28:56Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 21 Jun 2008, Stephan Beyer wrote:\n> \n> Today I've been playing around with approxidate(), too, and I think I\n> found some bug in date parsing. I let copy&paste speak...\n> \n> Correct:\n> \n> $ ./test-date \"2008-07-01 23:59:59 +0200\"\n> 2008-07-01 23:59:59 +0200 -> 1214949599 +0200 -> Tue Jul 1 23:59:59 2008\n> 2008-07-01 23:59:59 +0200 -> Tue Jul 1 23:59:59 2008\n> \n> And even:\n> \n> $ ./test-date \"2008-07-01 24:00:00 +0200\"\n> 2008-07-01 24:00:00 +0200 -> 1214949600 +0200 -> Wed Jul 2 00:00:00 2008\n> 2008-07-01 24:00:00 +0200 -> Wed Jul 2 00:00:00 2008\n> \n> But then there's a jump in time:\n> \n> $ ./test-date \"2008-07-02 00:00:00 +0200\"\n> 2008-07-02 00:00:00 +0200 -> 1202335200 +0200 -> Wed Feb 6 23:00:00 2008\n> 2008-07-02 00:00:00 +0200 -> Wed Feb 6 23:00:00 2008\n\nHeh.\n\nWhat you're seeing is that approxidate() does not like dates in the \nfuture. You're hitting this case:\n\n /* Be it commit time or author time, it does not make\n * sense to specify timestamp way into the future. Make\n * sure it is not later than ten days from now...\n */\n if (now + 10*24*3600 < specified)\n return 0;\n\nso approxidate() refuses to think that \"2008-07-02\" is a valid date in \nJuly, because it is more than ten days in the future. So it decides that \nif somebody tried to feed it a date like that, it must be the seventh of \nFebruary instead of July.\n\nSo the refusal to look at future dates is part of trying to disambiguate \nthe \"dd.mm\" form from the \"mm.dd\" form, where it will decide that if it's \nfar in the future (where \"far\" is 10 days), it cannot be right.\n\nRemember: the git date handling was _not_ meant to be a generic date \nlibrary. It is very much meant to be a *git* date library. This is why you \ncan say things like \"7 days ago\", and it will return something sane, but \nif you say \"7 days from now\", it will still think you're talking about \nseven days ago - it simply doesn't have the concept of \"future date\".\n\nThat said, I think that in this case the thing just doesn't make sense. \nFor the specific case of iso time format, we perhaps shouldn't even try to \nrefuse future dates. Does anybody use the insane yyyy-dd-mm format? \n\nTo avoid the future check, you could try something like the appended.\n\nOf course, the whole parser was really designed to parse email dates from \nthe beginning, and rfc2822 actually ends up being totally unambiguous and \nalso won't ever hit the \"refuse future\" case.\n\nSo I was wrong. Rather than using the European ISO format (yyyy-mm-dd), \nthe really safest format is the \"English month name spelled out\" format, \nie\n\n\t./test-date \"12 Jul 2010\"\n\nparses correctly, but\n\n\t./test-date 2010-07-12\n\ndoes not, because the latter is found suspect due to being in the future.\n\n\t\tLinus\n\n---\n date.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/date.c b/date.c\nindex 1a4eb87..870419a 100644\n--- a/date.c\n+++ b/date.c\n@@ -374,7 +374,7 @@ static int match_multi_number(unsigned long num, char c, const char *date, char\n \n \t\tif (num > 70) {\n \t\t\t/* yyyy-mm-dd? */\n-\t\t\tif (is_date(num, num2, num3, refuse_future, now, tm))\n+\t\t\tif (is_date(num, num2, num3, NULL, now, tm))\n \t\t\t\tbreak;\n \t\t\t/* yyyy-dd-mm? */\n \t\t\tif (is_date(num, num3, num2, refuse_future, now, tm))\n"},{"id":"80576","messageId":"alpine.LFD.1.10.0806211530560.2926@woody.linux-foundation.org","threadId":"13884","inReplyTo":"alpine.LFD.1.10.0806211507370.2926@woody.linux-foundation.org","subject":"Re: Date parsing","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-06-21T22:37:04Z","receivedAt":"2008-06-21T22:37:04Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 21 Jun 2008, Linus Torvalds wrote:\n> \n> Remember: the git date handling was _not_ meant to be a generic date \n> library. It is very much meant to be a *git* date library.\n\nBtw, I'm not proud of that. The whole of \"date.c\" was a huge hack to \novercome the fact that\n\n - standard library date parsing is pure and utter shit, even for standard \n date formats.\n\n - Never mind the standard date formats, very few seem to do the relative \n dateslike \"10 days ago\".\n\n - So I couldn't find anybody elses code to \"borrow\" that did a reasonable \n job and handled the nice relative dates too.\n\nThe GNU \"date\" command has pretty good support for doing things like \"5 \nminutes ago\", but I think I tried to look at the code, and quicky decided \nthat it was not an option (when I say \"I think I tried\", I have to admit \nthat I don't have strong memories from it, but I suspect it was all so \nutterly horrid that I repressed the episode).\n\nI don't mind using other peoples code (the whole xdiff library and the \nSHA1 stuff was certainly all from outside), but it needs to be in a format \nthat doesn't cause projectile vomiting.\n\nIf somebody has a good date parser, I think we could drop the git-specific \none in an instant.\n\nOr if somebody wants to just improve on it... Hint, hint.\n\n\t\tLinus\n"},{"id":"80578","messageId":"20080621224042.GE15111@leksak.fem-net","threadId":"13884","inReplyTo":"alpine.LFD.1.10.0806211507370.2926@woody.linux-foundation.org","subject":"Re: Date parsing","fromName":"Stephan Beyer","fromEmail":"s-beyer@gmx.net","sentAt":"2008-06-21T22:40:42Z","receivedAt":"2008-06-21T22:40:42Z","isPatch":false,"sender":{"key":"s-beyer@gmx.net","avatar":"https://avatars.githubusercontent.com/u/143889?v=4"},"body":"Hi,\n\nOn Sat, Jun 21, 2008 at 03:28:56PM -0700,\nLinus Torvalds :\n[...]\n> /* Be it commit time or author time, it does not make\n> * sense to specify timestamp way into the future. Make\n> * sure it is not later than ten days from now...\n> */\n> if (now + 10*24*3600 < specified)\n> return 0;\n[...]\n\nGreat.\nSo it really turns out that this bug is a nifty feature. (Except if we receive\npatches from the future with a numeric date format.)\n\nThanks for the explanation,\n Stephan\n\n-- \nStephan Beyer , PGP 0x6EDDD207FCC5040F\n"}]}