threads / patch / 61681

v4, 2 partsDarcy's "date underflow fix" topic, final reroll

Subject: [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll

## tl;dr

7 messages between Jun 25, 2024 and Jun 26, 2024. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Junio C Hamano· Jun 25, 2024, 23:12 UTC · lore

So it has been some time since we discussed this topic. Let's clean up the messy "SQUASH???" patches I had to queue on top of the main patch to keep the CI working and make them into a preliminary patch.

The tree at the end of the series is identical to what has been queued in 'seen' for the past few weeks. The only difference is that we first lay groundwork to skip certain time-parsing tests on 32-bit systems first, and then use Darcy's patch with minimum adjustments for 32-bit systems.

Darcy Burke (1):
  date: detect underflow/overflow when parsing dates with timezone offset
Junio C Hamano (1):
  t0006: simplify prerequisites
 date.c          | 12 +++++++++++-
 t/t0006-date.sh | 51 +++++++++++++++++++++++++++++++++++++++++++------
 2 files changed, 56 insertions(+), 7 deletions(-)
-- 
2.45.2-796-g2ef7a3d713
Junio C Hamano· Jun 25, 2024, 23:12 UTC · re: Junio C Hamano · lore

[PATCH v4 1/2] t0006: simplify prerequisites

The system must support 64-bit time and its time_t must be 64-bit wide to pass these tests. Combine these two prerequisites together to simplify the tests. In theory, they could be fulfilled independently and tests could require only one without the other, but in practice, but in practice these must come hand-in-hand.

Update the "check_parse" test helper to pay attention to the REQUIRE_64BIT_TIME variable, which can be set to the HAVE_64BIT_TIME prerequisite so that a parse test can be skipped on 32-bit systems. This will be used in the next step to skip tests for timestamps near the end of year 2099, as 32-bit systems will not be able to express a timestamp beyond 2038 anyway.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t0006-date.sh | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
Show changes to t/t0006-date.sh +12 −6
diff --git a/t/t0006-date.sh b/t/t0006-date.sh
index 3031256d14..24e8647f26 100755
--- a/t/t0006-date.sh
+++ b/t/t0006-date.sh
@@ -8,6 +8,11 @@ TEST_PASSES_SANITIZE_LEAK=true
 # arbitrary reference time: 2009-08-30 19:20:00
 GIT_TEST_DATE_NOW=1251660000; export GIT_TEST_DATE_NOW
 
+if test_have_prereq TIME_IS_64BIT,TIME_T_IS_64BIT
+then
+	test_set_prereq HAVE_64BIT_TIME
+fi
+
 check_relative() {
 	t=$(($GIT_TEST_DATE_NOW - $1))
 	echo "$t -> $2" >expect
@@ -80,14 +85,15 @@ check_show raw "$TIME" '1466000000 -0200'
 
 # arbitrary time absurdly far in the future
 FUTURE="5758122296 -0400"
-check_show iso       "$FUTURE" "2152-06-19 18:24:56 -0400" TIME_IS_64BIT,TIME_T_IS_64BIT
-check_show iso-local "$FUTURE" "2152-06-19 22:24:56 +0000" TIME_IS_64BIT,TIME_T_IS_64BIT
+check_show iso       "$FUTURE" "2152-06-19 18:24:56 -0400" HAVE_64BIT_TIME
+check_show iso-local "$FUTURE" "2152-06-19 22:24:56 +0000" HAVE_64BIT_TIME
 
-check_parse() {
+REQUIRE_64BIT_TIME=
+check_parse () {
 	echo "$1 -> $2" >expect
-	test_expect_${4:-success} "parse date ($1${3:+ TZ=$3})" "
-	TZ=${3:-$TZ} test-tool date parse '$1' >actual &&
-	test_cmp expect actual
+	test_expect_success $REQUIRE_64BIT_TIME "parse date ($1${3:+ TZ=$3}) -> $2" "
+		TZ=${3:-$TZ} test-tool date parse '$1' >actual &&
+		test_cmp expect actual
 	"
 }
 

-- 
2.45.2-796-g2ef7a3d713
Eric Sunshine· Jun 25, 2024, 23:30 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 1/2] t0006: simplify prerequisites

On Tue, Jun 25, 2024 at 7:13 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 5 quoted lines
> The system must support 64-bit time and its time_t must be 64-bit
> wide to pass these tests.  Combine these two prerequisites together
> to simplify the tests.  In theory, they could be fulfilled
> independently and tests could require only one without the other,
> but in practice, but in practice these must come hand-in-hand.
s/but in practice, but in practice/but in practice/
Show 8 quoted lines
> Update the "check_parse" test helper to pay attention to the
> REQUIRE_64BIT_TIME variable, which can be set to the HAVE_64BIT_TIME
> prerequisite so that a parse test can be skipped on 32-bit systems.
> This will be used in the next step to skip tests for timestamps near
> the end of year 2099, as 32-bit systems will not be able to express
> a timestamp beyond 2038 anyway.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano· Jun 26, 2024, 00:04 UTC · re: Eric Sunshine · lore

Re: [PATCH v4 1/2] t0006: simplify prerequisites

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 8 quoted lines
> On Tue, Jun 25, 2024 at 7:13 PM Junio C Hamano <gitster@pobox.com> wrote:
>> The system must support 64-bit time and its time_t must be 64-bit
>> wide to pass these tests.  Combine these two prerequisites together
>> to simplify the tests.  In theory, they could be fulfilled
>> independently and tests could require only one without the other,
>> but in practice, but in practice these must come hand-in-hand.
>
> s/but in practice, but in practice/but in practice/
Thanks, always, for your sharp eyes.
Junio C Hamano· Jun 25, 2024, 23:12 UTC · re: Junio C Hamano · lore

[PATCH v4 2/2] date: detect underflow/overflow when parsing dates with timezone offset

From: Darcy Burke <acednes@gmail.com>

Overriding the date of a commit to be close to "1970-01-01 00:00:00" with a large enough positive timezone for the equivelant GMT time to be before the epoch is considered valid by `parse_date_basic`. Similar behaviour occurs when using a date close to "2099-12-31 23:59:59" (the maximum date allowed by `tm_to_time_t`) with a large enough negative timezone offset.

This leads to an integer underflow or underflow respectively in the commit timestamp, which is not caught by `git-commit`, but will cause other services to fail, such as `git-fsck`, which, for the first case, reports "badDateOverflow: invalid author/committer line - date causes integer overflow".

Instead check the timezone offset and fail if the resulting time comes before the epoch "1970-01-01T00:00:00Z" or after the maximum date "2099-12-31T23:59:59Z".

Using the REQUIRE_64BIT_TIME prerequisite, make sure that the tests near the end of Git time (aka end of year 2099) are not attempted on purely 32-bit systems, as they cannot express timestamp beyond 2038 anyway.

Signed-off-by: Darcy Burke <acednes@gmail.com>
[jc: fixups for 32-bit platforms]
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 date.c          | 12 +++++++++++-
 t/t0006-date.sh | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)
Show changes to 2 files +44 −1

date.c, t/t0006-date.sh

diff --git a/date.c b/date.c
index 7365a4ad24..bee9fe8f10 100644
--- a/date.c
+++ b/date.c
@@ -868,6 +868,10 @@ static int match_object_header_date(const char *date, timestamp_t *timestamp, in
 	return 0;
 }
 
+
+/* timestamp of 2099-12-31T23:59:59Z, including 32 leap days */
+static const timestamp_t timestamp_max = (((timestamp_t)2100 - 1970) * 365 + 32) * 24 * 60 * 60 - 1;
+
 /* Gr. strptime is crap for this; it doesn't have a way to require RFC2822
    (i.e. English) day/month names, and it doesn't work correctly with %z. */
 int parse_date_basic(const char *date, timestamp_t *timestamp, int *offset)
@@ -937,8 +941,14 @@ int parse_date_basic(const char *date, timestamp_t *timestamp, int *offset)
 		}
 	}
 
-	if (!tm_gmt)
+	if (!tm_gmt) {
+		if (*offset > 0 && *offset * 60 > *timestamp)
+			return -1;
+		if (*offset < 0 && -*offset * 60 > timestamp_max - *timestamp)
+			return -1;
 		*timestamp -= *offset * 60;
+	}
+
 	return 0; /* success */
 }
 
diff --git a/t/t0006-date.sh b/t/t0006-date.sh
index 24e8647f26..fd373e1b39 100755
--- a/t/t0006-date.sh
+++ b/t/t0006-date.sh
@@ -123,6 +123,39 @@ check_parse '2008-02-14 20:30:45 -05:00' '2008-02-14 20:30:45 -0500'
 check_parse '2008-02-14 20:30:45' '2008-02-14 20:30:45 -0500' EST5
 check_parse 'Thu, 7 Apr 2005 15:14:13 -0700' '2005-04-07 15:14:13 -0700'
 
+check_parse '1970-01-01 00:00:00' '1970-01-01 00:00:00 +0000'
+check_parse '1970-01-01 00:00:00 +00' '1970-01-01 00:00:00 +0000'
+check_parse '1970-01-01 00:00:00 Z' '1970-01-01 00:00:00 +0000'
+check_parse '1970-01-01 00:00:00 -01' '1970-01-01 00:00:00 -0100'
+check_parse '1970-01-01 00:00:00 +01' bad
+check_parse '1970-01-01 00:00:00 +11' bad
+check_parse '1970-01-01 00:59:59 +01' bad
+check_parse '1970-01-01 01:00:00 +01' '1970-01-01 01:00:00 +0100'
+check_parse '1970-01-01 01:00:00 +11' bad
+check_parse '1970-01-02 00:00:00 +11' '1970-01-02 00:00:00 +1100'
+check_parse '1969-12-31 23:59:59' bad
+check_parse '1969-12-31 23:59:59 +00' bad
+check_parse '1969-12-31 23:59:59 Z' bad
+check_parse '1969-12-31 23:59:59 +11' bad
+check_parse '1969-12-31 23:59:59 -11' bad
+
+REQUIRE_64BIT_TIME=HAVE_64BIT_TIME
+check_parse '2099-12-31 23:59:59' '2099-12-31 23:59:59 +0000'
+check_parse '2099-12-31 23:59:59 +00' '2099-12-31 23:59:59 +0000'
+check_parse '2099-12-31 23:59:59 Z' '2099-12-31 23:59:59 +0000'
+check_parse '2099-12-31 23:59:59 +01' '2099-12-31 23:59:59 +0100'
+check_parse '2099-12-31 23:59:59 -01' bad
+check_parse '2099-12-31 23:59:59 -11' bad
+check_parse '2099-12-31 23:00:00 -01' bad
+check_parse '2099-12-31 22:59:59 -01' '2099-12-31 22:59:59 -0100'
+check_parse '2100-00-00 00:00:00' bad
+check_parse '2099-12-30 00:00:00 -11' '2099-12-30 00:00:00 -1100'
+check_parse '2100-00-00 00:00:00 +00' bad
+check_parse '2100-00-00 00:00:00 Z' bad
+check_parse '2100-00-00 00:00:00 -11' bad
+check_parse '2100-00-00 00:00:00 +11' bad
+REQUIRE_64BIT_TIME=
+
 check_approxidate() {
 	echo "$1 -> $2 +0000" >expect
 	test_expect_${3:-success} "parse approxidate ($1)" "

-- 
2.45.2-796-g2ef7a3d713
Phillip Wood· Jun 26, 2024, 15:21 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll

Hi Junio
On 26/06/2024 00:12, Junio C Hamano wrote:
Show 9 quoted lines
> So it has been some time since we discussed this topic.  Let's clean
> up the messy "SQUASH???" patches I had to queue on top of the main
> patch to keep the CI working and make them into a preliminary patch.
> 
> The tree at the end of the series is identical to what has been
> queued in 'seen' for the past few weeks.  The only difference is
> that we first lay groundwork to skip certain time-parsing tests on
> 32-bit systems first, and then use Darcy's patch with minimum
> adjustments for 32-bit systems.

I've had a read through this verison and I don't have anything to add to Eric's comments.

Best Wishes
Phillip
Show 10 quoted lines
> Darcy Burke (1):
>    date: detect underflow/overflow when parsing dates with timezone offset
> 
> Junio C Hamano (1):
>    t0006: simplify prerequisites
> 
>   date.c          | 12 +++++++++++-
>   t/t0006-date.sh | 51 +++++++++++++++++++++++++++++++++++++++++++------
>   2 files changed, 56 insertions(+), 7 deletions(-)
>
Junio C Hamano· Jun 26, 2024, 18:32 UTC · re: Phillip Wood · lore

Re: [PATCH v4 0/2] Darcy's "date underflow fix" topic, final reroll

Phillip Wood <phillip.wood123@gmail.com> writes:
Show 16 quoted lines
> Hi Junio
>
> On 26/06/2024 00:12, Junio C Hamano wrote:
>> So it has been some time since we discussed this topic.  Let's clean
>> up the messy "SQUASH???" patches I had to queue on top of the main
>> patch to keep the CI working and make them into a preliminary patch.
>> The tree at the end of the series is identical to what has been
>> queued in 'seen' for the past few weeks.  The only difference is
>> that we first lay groundwork to skip certain time-parsing tests on
>> 32-bit systems first, and then use Darcy's patch with minimum
>> adjustments for 32-bit systems.
>
> I've had a read through this verison and I don't have anything to add
> to Eric's comments.
>
> Best Wishes
Thanks, let's mark it for 'next' then.

← back to recent threads