# [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll 7 messages from 2024-06-25 to 2024-06-26. Participants: Junio C Hamano, Eric Sunshine, Phillip Wood. Thread: https://gitlist.dev/t/61681 ## Junio C Hamano, 2024-06-25 23:12 Subject: [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll Message-ID: <20240625231248.4070257-1-gitster@pobox.com> URL: https://gitlist.dev/e/20240625231248.4070257-1-gitster%40pobox.com ``` So it has been some time since we discussed this topic. Let's clean up the messy "SQUASH???" patches I had to queue on top of the main patch to keep the CI working and make them into a preliminary patch. The tree at the end of the series is identical to what has been queued in 'seen' for the past few weeks. The only difference is that we first lay groundwork to skip certain time-parsing tests on 32-bit systems first, and then use Darcy's patch with minimum adjustments for 32-bit systems. Darcy Burke (1): date: detect underflow/overflow when parsing dates with timezone offset Junio C Hamano (1): t0006: simplify prerequisites date.c | 12 +++++++++++- t/t0006-date.sh | 51 +++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 56 insertions(+), 7 deletions(-) -- 2.45.2-796-g2ef7a3d713 ``` ## Junio C Hamano, 2024-06-25 23:12 Subject: [PATCH v4 1/2] t0006: simplify prerequisites Message-ID: <20240625231248.4070257-2-gitster@pobox.com> URL: https://gitlist.dev/e/20240625231248.4070257-2-gitster%40pobox.com In-Reply-To: <20240625231248.4070257-1-gitster@pobox.com> ``` The system must support 64-bit time and its time_t must be 64-bit wide to pass these tests. Combine these two prerequisites together to simplify the tests. In theory, they could be fulfilled independently and tests could require only one without the other, but in practice, but in practice these must come hand-in-hand. Update the "check_parse" test helper to pay attention to the REQUIRE_64BIT_TIME variable, which can be set to the HAVE_64BIT_TIME prerequisite so that a parse test can be skipped on 32-bit systems. This will be used in the next step to skip tests for timestamps near the end of year 2099, as 32-bit systems will not be able to express a timestamp beyond 2038 anyway. Signed-off-by: Junio C Hamano --- t/t0006-date.sh | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/t/t0006-date.sh b/t/t0006-date.sh index 3031256d14..24e8647f26 100755 --- a/t/t0006-date.sh +++ b/t/t0006-date.sh @@ -8,6 +8,11 @@ TEST_PASSES_SANITIZE_LEAK=true # arbitrary reference time: 2009-08-30 19:20:00 GIT_TEST_DATE_NOW=1251660000; export GIT_TEST_DATE_NOW +if test_have_prereq TIME_IS_64BIT,TIME_T_IS_64BIT +then + test_set_prereq HAVE_64BIT_TIME +fi + check_relative() { t=$(($GIT_TEST_DATE_NOW - $1)) echo "$t -> $2" >expect @@ -80,14 +85,15 @@ check_show raw "$TIME" '1466000000 -0200' # arbitrary time absurdly far in the future FUTURE="5758122296 -0400" -check_show iso "$FUTURE" "2152-06-19 18:24:56 -0400" TIME_IS_64BIT,TIME_T_IS_64BIT -check_show iso-local "$FUTURE" "2152-06-19 22:24:56 +0000" TIME_IS_64BIT,TIME_T_IS_64BIT +check_show iso "$FUTURE" "2152-06-19 18:24:56 -0400" HAVE_64BIT_TIME +check_show iso-local "$FUTURE" "2152-06-19 22:24:56 +0000" HAVE_64BIT_TIME -check_parse() { +REQUIRE_64BIT_TIME= +check_parse () { echo "$1 -> $2" >expect - test_expect_${4:-success} "parse date ($1${3:+ TZ=$3})" " - TZ=${3:-$TZ} test-tool date parse '$1' >actual && - test_cmp expect actual + test_expect_success $REQUIRE_64BIT_TIME "parse date ($1${3:+ TZ=$3}) -> $2" " + TZ=${3:-$TZ} test-tool date parse '$1' >actual && + test_cmp expect actual " } -- 2.45.2-796-g2ef7a3d713 ``` ## Junio C Hamano, 2024-06-25 23:12 Subject: [PATCH v4 2/2] date: detect underflow/overflow when parsing dates with timezone offset Message-ID: <20240625231248.4070257-3-gitster@pobox.com> URL: https://gitlist.dev/e/20240625231248.4070257-3-gitster%40pobox.com In-Reply-To: <20240625231248.4070257-1-gitster@pobox.com> ``` From: Darcy Burke Overriding the date of a commit to be close to "1970-01-01 00:00:00" with a large enough positive timezone for the equivelant GMT time to be before the epoch is considered valid by `parse_date_basic`. Similar behaviour occurs when using a date close to "2099-12-31 23:59:59" (the maximum date allowed by `tm_to_time_t`) with a large enough negative timezone offset. This leads to an integer underflow or underflow respectively in the commit timestamp, which is not caught by `git-commit`, but will cause other services to fail, such as `git-fsck`, which, for the first case, reports "badDateOverflow: invalid author/committer line - date causes integer overflow". Instead check the timezone offset and fail if the resulting time comes before the epoch "1970-01-01T00:00:00Z" or after the maximum date "2099-12-31T23:59:59Z". Using the REQUIRE_64BIT_TIME prerequisite, make sure that the tests near the end of Git time (aka end of year 2099) are not attempted on purely 32-bit systems, as they cannot express timestamp beyond 2038 anyway. Signed-off-by: Darcy Burke [jc: fixups for 32-bit platforms] Signed-off-by: Junio C Hamano --- date.c | 12 +++++++++++- t/t0006-date.sh | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/date.c b/date.c index 7365a4ad24..bee9fe8f10 100644 --- a/date.c +++ b/date.c @@ -868,6 +868,10 @@ static int match_object_header_date(const char *date, timestamp_t *timestamp, in return 0; } + +/* timestamp of 2099-12-31T23:59:59Z, including 32 leap days */ +static const timestamp_t timestamp_max = (((timestamp_t)2100 - 1970) * 365 + 32) * 24 * 60 * 60 - 1; + /* Gr. strptime is crap for this; it doesn't have a way to require RFC2822 (i.e. English) day/month names, and it doesn't work correctly with %z. */ int parse_date_basic(const char *date, timestamp_t *timestamp, int *offset) @@ -937,8 +941,14 @@ int parse_date_basic(const char *date, timestamp_t *timestamp, int *offset) } } - if (!tm_gmt) + if (!tm_gmt) { + if (*offset > 0 && *offset * 60 > *timestamp) + return -1; + if (*offset < 0 && -*offset * 60 > timestamp_max - *timestamp) + return -1; *timestamp -= *offset * 60; + } + return 0; /* success */ } diff --git a/t/t0006-date.sh b/t/t0006-date.sh index 24e8647f26..fd373e1b39 100755 --- a/t/t0006-date.sh +++ b/t/t0006-date.sh @@ -123,6 +123,39 @@ check_parse '2008-02-14 20:30:45 -05:00' '2008-02-14 20:30:45 -0500' check_parse '2008-02-14 20:30:45' '2008-02-14 20:30:45 -0500' EST5 check_parse 'Thu, 7 Apr 2005 15:14:13 -0700' '2005-04-07 15:14:13 -0700' +check_parse '1970-01-01 00:00:00' '1970-01-01 00:00:00 +0000' +check_parse '1970-01-01 00:00:00 +00' '1970-01-01 00:00:00 +0000' +check_parse '1970-01-01 00:00:00 Z' '1970-01-01 00:00:00 +0000' +check_parse '1970-01-01 00:00:00 -01' '1970-01-01 00:00:00 -0100' +check_parse '1970-01-01 00:00:00 +01' bad +check_parse '1970-01-01 00:00:00 +11' bad +check_parse '1970-01-01 00:59:59 +01' bad +check_parse '1970-01-01 01:00:00 +01' '1970-01-01 01:00:00 +0100' +check_parse '1970-01-01 01:00:00 +11' bad +check_parse '1970-01-02 00:00:00 +11' '1970-01-02 00:00:00 +1100' +check_parse '1969-12-31 23:59:59' bad +check_parse '1969-12-31 23:59:59 +00' bad +check_parse '1969-12-31 23:59:59 Z' bad +check_parse '1969-12-31 23:59:59 +11' bad +check_parse '1969-12-31 23:59:59 -11' bad + +REQUIRE_64BIT_TIME=HAVE_64BIT_TIME +check_parse '2099-12-31 23:59:59' '2099-12-31 23:59:59 +0000' +check_parse '2099-12-31 23:59:59 +00' '2099-12-31 23:59:59 +0000' +check_parse '2099-12-31 23:59:59 Z' '2099-12-31 23:59:59 +0000' +check_parse '2099-12-31 23:59:59 +01' '2099-12-31 23:59:59 +0100' +check_parse '2099-12-31 23:59:59 -01' bad +check_parse '2099-12-31 23:59:59 -11' bad +check_parse '2099-12-31 23:00:00 -01' bad +check_parse '2099-12-31 22:59:59 -01' '2099-12-31 22:59:59 -0100' +check_parse '2100-00-00 00:00:00' bad +check_parse '2099-12-30 00:00:00 -11' '2099-12-30 00:00:00 -1100' +check_parse '2100-00-00 00:00:00 +00' bad +check_parse '2100-00-00 00:00:00 Z' bad +check_parse '2100-00-00 00:00:00 -11' bad +check_parse '2100-00-00 00:00:00 +11' bad +REQUIRE_64BIT_TIME= + check_approxidate() { echo "$1 -> $2 +0000" >expect test_expect_${3:-success} "parse approxidate ($1)" " -- 2.45.2-796-g2ef7a3d713 ``` ## Eric Sunshine, 2024-06-25 23:30 Subject: Re: [PATCH v4 1/2] t0006: simplify prerequisites Message-ID: URL: https://gitlist.dev/e/CAPig%2BcRaQbWTLnvbx-Yec3mGEJDOQQr8RX%2BbT%3DJ1FzVDqjq2_g%40mail.gmail.com In-Reply-To: <20240625231248.4070257-2-gitster@pobox.com> ``` On Tue, Jun 25, 2024 at 7:13 PM Junio C Hamano wrote: > The system must support 64-bit time and its time_t must be 64-bit > wide to pass these tests. Combine these two prerequisites together > to simplify the tests. In theory, they could be fulfilled > independently and tests could require only one without the other, > but in practice, but in practice these must come hand-in-hand. s/but in practice, but in practice/but in practice/ > Update the "check_parse" test helper to pay attention to the > REQUIRE_64BIT_TIME variable, which can be set to the HAVE_64BIT_TIME > prerequisite so that a parse test can be skipped on 32-bit systems. > This will be used in the next step to skip tests for timestamps near > the end of year 2099, as 32-bit systems will not be able to express > a timestamp beyond 2038 anyway. > > Signed-off-by: Junio C Hamano ``` ## Junio C Hamano, 2024-06-26 00:04 Subject: Re: [PATCH v4 1/2] t0006: simplify prerequisites Message-ID: URL: https://gitlist.dev/e/xmqq5xtw603e.fsf%40gitster.g In-Reply-To: ``` Eric Sunshine writes: > On Tue, Jun 25, 2024 at 7:13 PM Junio C Hamano wrote: >> The system must support 64-bit time and its time_t must be 64-bit >> wide to pass these tests. Combine these two prerequisites together >> to simplify the tests. In theory, they could be fulfilled >> independently and tests could require only one without the other, >> but in practice, but in practice these must come hand-in-hand. > > s/but in practice, but in practice/but in practice/ Thanks, always, for your sharp eyes. ``` ## Phillip Wood, 2024-06-26 15:21 Subject: Re: [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll Message-ID: URL: https://gitlist.dev/e/dedaa590-b9bf-498f-b743-ba058cba8486%40gmail.com In-Reply-To: <20240625231248.4070257-1-gitster@pobox.com> ``` Hi Junio On 26/06/2024 00:12, Junio C Hamano wrote: > So it has been some time since we discussed this topic. Let's clean > up the messy "SQUASH???" patches I had to queue on top of the main > patch to keep the CI working and make them into a preliminary patch. > > The tree at the end of the series is identical to what has been > queued in 'seen' for the past few weeks. The only difference is > that we first lay groundwork to skip certain time-parsing tests on > 32-bit systems first, and then use Darcy's patch with minimum > adjustments for 32-bit systems. I've had a read through this verison and I don't have anything to add to Eric's comments. Best Wishes Phillip > Darcy Burke (1): > date: detect underflow/overflow when parsing dates with timezone offset > > Junio C Hamano (1): > t0006: simplify prerequisites > > date.c | 12 +++++++++++- > t/t0006-date.sh | 51 +++++++++++++++++++++++++++++++++++++++++++------ > 2 files changed, 56 insertions(+), 7 deletions(-) > ``` ## Junio C Hamano, 2024-06-26 18:32 Subject: Re: [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll Message-ID: URL: https://gitlist.dev/e/xmqqle2rzh9o.fsf%40gitster.g In-Reply-To: ``` Phillip Wood writes: > Hi Junio > > On 26/06/2024 00:12, Junio C Hamano wrote: >> So it has been some time since we discussed this topic. Let's clean >> up the messy "SQUASH???" patches I had to queue on top of the main >> patch to keep the CI working and make them into a preliminary patch. >> The tree at the end of the series is identical to what has been >> queued in 'seen' for the past few weeks. The only difference is >> that we first lay groundwork to skip certain time-parsing tests on >> 32-bit systems first, and then use Darcy's patch with minimum >> adjustments for 32-bit systems. > > I've had a read through this verison and I don't have anything to add > to Eric's comments. > > Best Wishes Thanks, let's mark it for 'next' then. ```