{"thread":{"id":"52669","subject":"Y2038 vs struct cache_time/time_t","startedAt":"2020-01-20T19:38:55Z","lastAt":"2020-01-21T11:48:20Z","messageCount":5,"participants":["Johannes Schindelin","Michal Suchánek","Randall S. Becker"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"390123","messageId":"nycvar.QRO.7.76.6.2001201555220.46@tvgsbejvaqbjf.bet","threadId":"52669","inReplyTo":null,"subject":"Y2038 vs struct cache_time/time_t","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-01-20T19:38:51Z","receivedAt":"2020-01-20T19:38:55Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Team,\n\ntoday, in quite an entertaining thread on Twitter\n(https://twitter.com/jxxf/status/1219009308438024200) I read about yet\nanother account how the Year 2038 problem already bites people. And costs\nreal amounts of money.\n\nAnd after I stopped shaking my head in disbelief, I had a quick look, and\nit seems that we're safe at least until February 7th, 2106. That's not\ngreat, but I plan on not being around at that date anymore, so there. That\ndate is when the unsigned 32-bit Unix epoch will roll over and play\ndead^W^Wwreak havoc (iff the human species manages to actually turn around\nand reverse the climate catastrophe it caused, and that's a big iff):\nhttps://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2106\n\nConcretely, it looks as if we store our own timestamps on disk (in the\nindex file) as uint32_t:\n\n\t/*\n\t * The \"cache_time\" is just the low 32 bits of the\n\t * time. It doesn't matter if it overflows - we only\n\t * check it for equality in the 32 bits we save.\n\t */\n\tstruct cache_time {\n\t\tuint32_t sec;\n\t\tuint32_t nsec;\n\t};\n\nThe comment seems to indicate that we are still safe even if 2106 comes\naround, but I am not _quite_ that sure, as I expect us to have \"greater\nthan\" checks, not only equality checks.\n\nBut wait, we're still not quite safe. If I remember correctly, 32-bit\nLinux still uses _signed_ 32-bit integers as `time_t`, so when we render\ndates, for example, and use system-provided functions, on 32-bit Linux we\nwill at least show the wrong dates starting 2038.\n\nThis got me thinking, and I put on my QA hat. Kids, try this at home:\n\n\t$ git log --until=1.january.1960\n\n\t$ git log --since=1.january.2200\n\nGit does not really do what you expected, eh?\n\nMaybe we want to do something about that, and while at it also fix the\noverflow problems, probably requiring a new index format?\n\nCiao,\nJohannes\n"},{"id":"390124","messageId":"20200120194515.GE4113@kitsune.suse.cz","threadId":"52669","inReplyTo":"nycvar.QRO.7.76.6.2001201555220.46@tvgsbejvaqbjf.bet","subject":"Re: Y2038 vs struct cache_time/time_t","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2020-01-20T19:45:15Z","receivedAt":"2020-01-20T19:45:19Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Mon, Jan 20, 2020 at 08:38:51PM +0100, Johannes Schindelin wrote:\n> Team,\n> \n> today, in quite an entertaining thread on Twitter\n> (https://twitter.com/jxxf/status/1219009308438024200) I read about yet\n> another account how the Year 2038 problem already bites people. And costs\n> real amounts of money.\n> \n> And after I stopped shaking my head in disbelief, I had a quick look, and\n> it seems that we're safe at least until February 7th, 2106. That's not\n> great, but I plan on not being around at that date anymore, so there. That\n> date is when the unsigned 32-bit Unix epoch will roll over and play\n> dead^W^Wwreak havoc (iff the human species manages to actually turn around\n> and reverse the climate catastrophe it caused, and that's a big iff):\n> https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2106\n> \n> Concretely, it looks as if we store our own timestamps on disk (in the\n> index file) as uint32_t:\n> \n> \t/*\n> \t * The \"cache_time\" is just the low 32 bits of the\n> \t * time. It doesn't matter if it overflows - we only\n> \t * check it for equality in the 32 bits we save.\n> \t */\n> \tstruct cache_time {\n> \t\tuint32_t sec;\n> \t\tuint32_t nsec;\n> \t};\n> \n> The comment seems to indicate that we are still safe even if 2106 comes\n> around, but I am not _quite_ that sure, as I expect us to have \"greater\n> than\" checks, not only equality checks.\n> \n> But wait, we're still not quite safe. If I remember correctly, 32-bit\n> Linux still uses _signed_ 32-bit integers as `time_t`, so when we render\n> dates, for example, and use system-provided functions, on 32-bit Linux we\n> will at least show the wrong dates starting 2038.\n> \n> This got me thinking, and I put on my QA hat. Kids, try this at home:\n> \n> \t$ git log --until=1.january.1960\n> \n> \t$ git log --since=1.january.2200\n> \n> Git does not really do what you expected, eh?\n> \n> Maybe we want to do something about that, and while at it also fix the\n> overflow problems, probably requiring a new index format?\n\nWhich means we can split off the timestamps to a separate file, too ;-)\n\nThanks\n\nMichal\n"},{"id":"390127","messageId":"017401d5cfcf$8791d900$96b58b00$@nexbridge.com","threadId":"52669","inReplyTo":"nycvar.QRO.7.76.6.2001201555220.46@tvgsbejvaqbjf.bet","subject":"RE: Y2038 vs struct cache_time/time_t","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2020-01-20T20:23:46Z","receivedAt":"2020-01-20T20:23:57Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On January 19, 2038 (no really January 20, 2020 2:39 PM), Johannes\nSchindelin wrote:\n> today, in quite an entertaining thread on Twitter\n> (https://twitter.com/jxxf/status/1219009308438024200) I read about yet\n> another account how the Year 2038 problem already bites people. And costs\n> real amounts of money.\n> \n> And after I stopped shaking my head in disbelief, I had a quick look, and\nit\n> seems that we're safe at least until February 7th, 2106. That's not great,\nbut I\n> plan on not being around at that date anymore, so there. That date is when\n> the unsigned 32-bit Unix epoch will roll over and play dead^W^Wwreak\n> havoc (iff the human species manages to actually turn around and reverse\n> the climate catastrophe it caused, and that's a big iff):\n> https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_21\n> 06\n> \n> Concretely, it looks as if we store our own timestamps on disk (in the\nindex\n> file) as uint32_t:\n> \n> \t/*\n> \t * The \"cache_time\" is just the low 32 bits of the\n> \t * time. It doesn't matter if it overflows - we only\n> \t * check it for equality in the 32 bits we save.\n> \t */\n> \tstruct cache_time {\n> \t\tuint32_t sec;\n> \t\tuint32_t nsec;\n> \t};\n> \n> The comment seems to indicate that we are still safe even if 2106 comes\n> around, but I am not _quite_ that sure, as I expect us to have \"greater\nthan\"\n> checks, not only equality checks.\n> \n> But wait, we're still not quite safe. If I remember correctly, 32-bit\nLinux still\n> uses _signed_ 32-bit integers as `time_t`, so when we render dates, for\n> example, and use system-provided functions, on 32-bit Linux we will at\nleast\n> show the wrong dates starting 2038.\n> \n> This got me thinking, and I put on my QA hat. Kids, try this at home:\n> \n> \t$ git log --until=1.january.1960\n> \n> \t$ git log --since=1.january.2200\n> \n> Git does not really do what you expected, eh?\n> \n> Maybe we want to do something about that, and while at it also fix the\n> overflow problems, probably requiring a new index format?\n\nThe preferred way of fixing this is traditionally - for those of us who have\nbeen through it (4-ish times), to convert to time64_t where available (big\nlegacy machines, like z/OS and NonStop), or in gcc, time_t is 64 bit on 64\nbit systems. It has been 64 bit on Windows since VS 2005. I have a\nrelatively some relatively old Linux distros on 64 bit processors that also\nhave time_t set as 64 bit in gcc. Those seem to be the standard approaches.\nTo cover it, I suggest we move to a gittime_t which is always 64 bit (or 128\nbit if you don't want to be resurrected after the sun turns into a red giant\nor later when we are left with evaporating black holes), no matter what the\nplatform, and build the selection of what gittime_t is (time_t or time64_t)\ninto our config and/or compat.h. That way, hopefully, people will rebuild\ntheir git before 2038 or before someone decides to stick a fake date into a\nGithub repo just to mess with us.\n\nCheers,\nRandall\n\n"},{"id":"390164","messageId":"nycvar.QRO.7.76.6.2001211245380.46@tvgsbejvaqbjf.bet","threadId":"52669","inReplyTo":"20200120194515.GE4113@kitsune.suse.cz","subject":"Re: Y2038 vs struct cache_time/time_t","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-01-21T11:46:07Z","receivedAt":"2020-01-21T11:46:27Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Michal,\n\nOn Mon, 20 Jan 2020, Michal Suchánek wrote:\n\n> On Mon, Jan 20, 2020 at 08:38:51PM +0100, Johannes Schindelin wrote:\n> > Team,\n> >\n> > today, in quite an entertaining thread on Twitter\n> > (https://twitter.com/jxxf/status/1219009308438024200) I read about yet\n> > another account how the Year 2038 problem already bites people. And costs\n> > real amounts of money.\n> >\n> > And after I stopped shaking my head in disbelief, I had a quick look, and\n> > it seems that we're safe at least until February 7th, 2106. That's not\n> > great, but I plan on not being around at that date anymore, so there. That\n> > date is when the unsigned 32-bit Unix epoch will roll over and play\n> > dead^W^Wwreak havoc (iff the human species manages to actually turn around\n> > and reverse the climate catastrophe it caused, and that's a big iff):\n> > https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2106\n> >\n> > Concretely, it looks as if we store our own timestamps on disk (in the\n> > index file) as uint32_t:\n> >\n> > \t/*\n> > \t * The \"cache_time\" is just the low 32 bits of the\n> > \t * time. It doesn't matter if it overflows - we only\n> > \t * check it for equality in the 32 bits we save.\n> > \t */\n> > \tstruct cache_time {\n> > \t\tuint32_t sec;\n> > \t\tuint32_t nsec;\n> > \t};\n> >\n> > The comment seems to indicate that we are still safe even if 2106 comes\n> > around, but I am not _quite_ that sure, as I expect us to have \"greater\n> > than\" checks, not only equality checks.\n> >\n> > But wait, we're still not quite safe. If I remember correctly, 32-bit\n> > Linux still uses _signed_ 32-bit integers as `time_t`, so when we render\n> > dates, for example, and use system-provided functions, on 32-bit Linux we\n> > will at least show the wrong dates starting 2038.\n> >\n> > This got me thinking, and I put on my QA hat. Kids, try this at home:\n> >\n> > \t$ git log --until=1.january.1960\n> >\n> > \t$ git log --since=1.january.2200\n> >\n> > Git does not really do what you expected, eh?\n> >\n> > Maybe we want to do something about that, and while at it also fix the\n> > overflow problems, probably requiring a new index format?\n>\n> Which means we can split off the timestamps to a separate file, too ;-)\n\nSure. We could also jump from a cliff at the same time. Just because you\ncan do something does not mean that it is a good idea to actually do it.\n\nCiao,\nJohannes\n"},{"id":"390165","messageId":"nycvar.QRO.7.76.6.2001211247180.46@tvgsbejvaqbjf.bet","threadId":"52669","inReplyTo":"017401d5cfcf$8791d900$96b58b00$@nexbridge.com","subject":"RE: Y2038 vs struct cache_time/time_t","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-01-21T11:47:49Z","receivedAt":"2020-01-21T11:48:20Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Randall,\n\nOn Mon, 20 Jan 2020, Randall S. Becker wrote:\n\n> On January 19, 2038 (no really January 20, 2020 2:39 PM), Johannes\n> Schindelin wrote:\n> > today, in quite an entertaining thread on Twitter\n> > (https://twitter.com/jxxf/status/1219009308438024200) I read about yet\n> > another account how the Year 2038 problem already bites people. And costs\n> > real amounts of money.\n> >\n> > And after I stopped shaking my head in disbelief, I had a quick look, and\n> it\n> > seems that we're safe at least until February 7th, 2106. That's not great,\n> but I\n> > plan on not being around at that date anymore, so there. That date is when\n> > the unsigned 32-bit Unix epoch will roll over and play dead^W^Wwreak\n> > havoc (iff the human species manages to actually turn around and reverse\n> > the climate catastrophe it caused, and that's a big iff):\n> > https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_21\n> > 06\n> >\n> > Concretely, it looks as if we store our own timestamps on disk (in the\n> index\n> > file) as uint32_t:\n> >\n> > \t/*\n> > \t * The \"cache_time\" is just the low 32 bits of the\n> > \t * time. It doesn't matter if it overflows - we only\n> > \t * check it for equality in the 32 bits we save.\n> > \t */\n> > \tstruct cache_time {\n> > \t\tuint32_t sec;\n> > \t\tuint32_t nsec;\n> > \t};\n> >\n> > The comment seems to indicate that we are still safe even if 2106 comes\n> > around, but I am not _quite_ that sure, as I expect us to have \"greater\n> than\"\n> > checks, not only equality checks.\n> >\n> > But wait, we're still not quite safe. If I remember correctly, 32-bit\n> Linux still\n> > uses _signed_ 32-bit integers as `time_t`, so when we render dates, for\n> > example, and use system-provided functions, on 32-bit Linux we will at\n> least\n> > show the wrong dates starting 2038.\n> >\n> > This got me thinking, and I put on my QA hat. Kids, try this at home:\n> >\n> > \t$ git log --until=1.january.1960\n> >\n> > \t$ git log --since=1.january.2200\n> >\n> > Git does not really do what you expected, eh?\n> >\n> > Maybe we want to do something about that, and while at it also fix the\n> > overflow problems, probably requiring a new index format?\n>\n> The preferred way of fixing this is traditionally - for those of us who have\n> been through it (4-ish times), to convert to time64_t where available (big\n> legacy machines, like z/OS and NonStop), or in gcc, time_t is 64 bit on 64\n> bit systems. It has been 64 bit on Windows since VS 2005. I have a\n> relatively some relatively old Linux distros on 64 bit processors that also\n> have time_t set as 64 bit in gcc. Those seem to be the standard approaches.\n> To cover it, I suggest we move to a gittime_t which is always 64 bit (or 128\n> bit if you don't want to be resurrected after the sun turns into a red giant\n> or later when we are left with evaporating black holes), no matter what the\n> platform, and build the selection of what gittime_t is (time_t or time64_t)\n> into our config and/or compat.h. That way, hopefully, people will rebuild\n> their git before 2038 or before someone decides to stick a fake date into a\n> Github repo just to mess with us.\n\nI like it. If I had time to tackle this, I would definitely go for\n`git_time64_t`.\n\nCiao,\nDscho\n"}]}