# Reftable reflog timezone encoding differs from specification 5 messages from 2026-09-28 to 2026-09-29. Participants: Josh McKinney, Patrick Steinhardt, Junio C Hamano. Thread: https://gitlist.dev/t/66403 ## Josh McKinney, 2026-09-28 07:00 Subject: Reftable reflog timezone encoding differs from specification Message-ID: <85f7daa8-d60b-4348-ac2f-b1a68628af7b@app.fastmail.com> ``` Hi, Git 2.55.0 appears to store reftable reflog timezone offsets as signed HHMM integers, whereas the specification requires signed minutes. https://git-scm.com/docs/reftable#_log_record states: "tz_offset is the absolute number of minutes from GMT the committer was at the time of the update." The specification also gives GMT+0230 as an example encoded as 150. I reproduced the discrepancy on macOS arm64 by creating a SHA-1 reftable repository and committing with this date: 2026-09-27T12:00:00+05:30 An independent Python/zlib inspection of the resulting reftable found: Stored timezone bytes: 02 12 = 530 Expected signed minutes: 01 4a = 330 Both HEAD and refs/heads/main reflog entries contained 530. Git reads its own entries back correctly as +05:30, so its writer and reader appear internally consistent, but disagree with the specification. Here is a reproducer using only Git and Python's standard library: ( set -eu proof_dir=$(mktemp -d) cd "$proof_dir" echo "Repository retained at: $proof_dir" export GIT_CONFIG_NOSYSTEM=1 export GIT_CONFIG_GLOBAL=/dev/null export GIT_AUTHOR_DATE='2026-09-27T12:00:00+05:30' export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" git --version git init --initial-branch=main --object-format=sha1 \ --ref-format=reftable example git -C example \ -c user.name=Example \ -c user.email=example@example.com \ -c commit.gpgsign=false \ -c core.logAllRefUpdates=true \ commit --allow-empty -m "Timezone example" git -C example reflog show --format='%gD' \ --date=iso-strict refs/heads/main python3 - <<'PY' from pathlib import Path import zlib directory = Path("example/.git/reftable") for name in (directory / "tables.list").read_text().splitlines(): table = (directory / name).read_bytes() assert table[:5] == b"REFT\x01" footer = table[-68:] log_start = int.from_bytes(footer[48:56], "big") if not log_start: continue assert table[log_start:log_start + 1] == b"g" log = zlib.decompress(table[log_start + 4:]) # Fixture-specific: email, variable-length time, then timezone. email = b"example@example.com" search_from = 0 while (position := log.find(email, search_from)) != -1: position += len(email) while log[position] & 0x80: position += 1 position += 1 raw = log[position:position + 2] offset = int.from_bytes(raw, "big", signed=True) print(f"Timezone bytes: {raw.hex(' ')}; integer: {offset}") search_from = position + 2 PY ) The relevant output is: refs/heads/main@{2026-09-27T12:00:00+05:30} Timezone bytes: 02 12; integer: 530 Timezone bytes: 02 12; integer: 530 Is this a known discrepancy? Which representation should interoperable implementations use? If either the implementation or specification changes, how should existing tables be interpreted, given that values such as 330 are valid under both interpretations? Given that Git consistently writes and reads HHMM values, I suspect the practical resolution is to update the specification to match existing behavior. Are there other implementations or compatibility considerations that would prevent that? Thanks, Josh -- Josh McKinney joshka.net ``` ## Patrick Steinhardt, 2026-09-28 12:13 Subject: Re: Reftable reflog timezone encoding differs from specification Message-ID: In-Reply-To: <85f7daa8-d60b-4348-ac2f-b1a68628af7b@app.fastmail.com> ``` Hi, On Mon, Sep 28, 2026 at 12:00:43AM -0700, Josh McKinney wrote: > Hi, > > Git 2.55.0 appears to store reftable reflog timezone offsets as signed > HHMM integers, whereas the specification requires signed minutes. > > https://git-scm.com/docs/reftable#_log_record states: > > "tz_offset is the absolute number of minutes from GMT the > committer was at the time of the update." > > The specification also gives GMT+0230 as an example encoded as 150. Oh dear, that's indeed the case. I was able to reproduce the issue, as well. > I reproduced the discrepancy on macOS arm64 by creating a SHA-1 > reftable repository and committing with this date: > > 2026-09-27T12:00:00+05:30 > > An independent Python/zlib inspection of the resulting reftable found: > > Stored timezone bytes: 02 12 = 530 > Expected signed minutes: 01 4a = 330 Yup. It's plain wrong the way we store it. > Both HEAD and refs/heads/main reflog entries contained 530. Git reads > its own entries back correctly as +05:30, so its writer and reader > appear internally consistent, but disagree with the specification. > > Here is a reproducer using only Git and Python's standard library: And here's a Git reproducer: diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c index fc49fafc34..e3cfb6a3fa 100644 --- a/t/helper/test-reftable.c +++ b/t/helper/test-reftable.c @@ -103,7 +103,16 @@ static int dump_table(struct reftable_merged_table *mt) if (err < 0) return err; - algop = &hash_algos[hash_algo_by_id(reftable_merged_table_hash_id(mt))]; + switch (reftable_merged_table_hash_id(mt)) { + case REFTABLE_HASH_SHA1: + algop = &hash_algos[GIT_HASH_SHA1]; + break; + case REFTABLE_HASH_SHA256: + algop = &hash_algos[GIT_HASH_SHA256]; + break; + default: + die("unsupported hash algorithm: %d", reftable_merged_table_hash_id(mt)); + } while (1) { err = reftable_iterator_next_ref(&it, &ref); diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh index 35e98b43db..e1d714077c 100755 --- a/t/t0610-reftable-basics.sh +++ b/t/t0610-reftable-basics.sh @@ -1163,4 +1163,19 @@ test_expect_success 'writes do not persist peeled value for invalid tags' ' ) ' +test_expect_success 'writes do not persist peeled value for invalid tags' ' + test_when_finished rm -rf repo && + git init repo && + ( + cd repo && + + export GIT_AUTHOR_DATE='2026-09-27T12:00:00+05:30' && + export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" && + git commit --allow-empty --message "Timezone example" && + git refs optimize && + test-tool dump-reftable -t .git/reftable/*.ref >table && + test_grep "log{HEAD(2) C O Mitter 1790490600 0530" table + ) +' + test_done And yes, the test-helper for reftables is broken, so we also have to fix that. [snip] > Is this a known discrepancy? No, it's not, I wasn't aware of it at all. > Which representation should interoperable implementations use? We should use the one that we have in our specification, so in my opinion we should fix Git itself. This is also because JGit, which had a reftable implementation for far longer compared to us, implements the specification correctly: private PersonIdent readPersonIdent() { String name = readValueString(); String email = readValueString(); long epochSeconds = readVarint64(); ZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60); return new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz); } You can see that we indeed treat the integer as number of minutes there, as expected. > If either the implementation or specification changes, how should > existing tables be interpreted, given that values such as 330 are > valid under both interpretations? > > Given that Git consistently writes and reads HHMM values, I suspect > the practical resolution is to update the specification to match > existing behavior. Are there other implementations or compatibility > considerations that would prevent that? That's a very good question. Given that JGit interprets the value as expected my take is that we should fix this in Git and keep the spec as-is. The question is how much we really lose by "just" fixing the bug. Sure, timezones would be wrong in that case. For example, if we had an entry with original timezone of +0200 we'd now interpret that as +0320. It's of course wrong given the original intent, but is it the end of the world? I dunno. Overall, the amount of damage is kind of limited here as the discrepancy is limited: ┌──────┬──────────────┬────────────────┬───────────┐ │tz │HHMM encoding │correct minutes │divergence │ ├──────┼──────────────┼────────────────┼───────────┤ │+1400 │1400 │840 │560 │ ├──────┼──────────────┼────────────────┼───────────┤ │-1200 │-1200 │-720 │480 │ ├──────┼──────────────┼────────────────┼───────────┤ │+0530 │530 │330 │200 │ ├──────┼──────────────┼────────────────┼───────────┤ │+0000 │0 │0 │0 │ └──────┴──────────────┴────────────────┴───────────┘ We could of course retroactively declare that version 2 of the format uses the syntax that Git uses right now. After all, JGit only knows to read version 1 of it anyway, so that could kind of fix it. But for any repository that uses SHA1 we used to write version 1 anyway, so this does not really buy us anything, I'd claim. In summary: - We have an upper limit in divergence of <10h. - This only matters in the context of reflogs, we don't use these anywhere else. - The risk for data loss by a change is limited as our default grace period for garbage collecting reflog entries is 30 days. With these points I'm inclined to call it a bug and just fix it, without handling backwards compatibility. I'm very happy to hear alternative takes though. In any case, thanks for your report! Patrick ``` ## Junio C Hamano, 2026-09-28 14:42 Subject: Re: Reftable reflog timezone encoding differs from specification Message-ID: In-Reply-To: ``` Patrick Steinhardt writes: > We should use the one that we have in our specification, so in my > opinion we should fix Git itself. This is also because JGit, which had a > reftable implementation for far longer compared to us, implements the > specification correctly: > > > private PersonIdent readPersonIdent() { > String name = readValueString(); > String email = readValueString(); > long epochSeconds = readVarint64(); > ZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60); > return new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz); > } Thanks for checking. I (unfortunately) agree with the (unfortunate) conclusion. We do not ship reftable files over networks and reflogs at the conceptual level is not shared across repositories, so the issue, other than the trivial part of updating the implementation, is how to migrate the data in a local repository that uses reftable. One time offline conversion may be the simplest but I do not know if it is worth it, given ... > We could of course retroactively declare that version 2 of the format > uses the syntax that Git uses right now. After all, JGit only knows to > read version 1 of it anyway, so that could kind of fix it. But for any > repository that uses SHA1 we used to write version 1 anyway, so this > does not really buy us anything, I'd claim. > > In summary: > > - We have an upper limit in divergence of <10h. ... this. > > - This only matters in the context of reflogs, we don't use these > anywhere else. > > - The risk for data loss by a change is limited as our default grace > period for garbage collecting reflog entries is 30 days. > > With these points I'm inclined to call it a bug and just fix it, without > handling backwards compatibility. ``` ## Josh McKinney, 2026-09-28 16:02 Subject: Re: Reftable reflog timezone encoding differs from specification Message-ID: <2abba760-d331-4cad-bb8b-6e567b517beb@app.fastmail.com> In-Reply-To: ``` I think my main concern here is mostly around using multiple tools on the same repo and how they should interpret on disk formats (my clanker picked up the problem when comparing git's output with a library it's writing). Anyway, nothing urgent on the problem from me because I noticed it purely in a development context. Thanks for filling in the bits about the real world impact on this too. Josh -- Josh McKinney joshka.net On Mon, Sep 28, 2026, at 7:42 AM, Junio C Hamano wrote: > Patrick Steinhardt writes: > >> We should use the one that we have in our specification, so in my >> opinion we should fix Git itself. This is also because JGit, which had a >> reftable implementation for far longer compared to us, implements the >> specification correctly: >> >> >> private PersonIdent readPersonIdent() { >> String name = readValueString(); >> String email = readValueString(); >> long epochSeconds = readVarint64(); >> ZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60); >> return new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz); >> } > > Thanks for checking. I (unfortunately) agree with the (unfortunate) > conclusion. > > We do not ship reftable files over networks and reflogs at the > conceptual level is not shared across repositories, so the issue, > other than the trivial part of updating the implementation, is how > to migrate the data in a local repository that uses reftable. One > time offline conversion may be the simplest but I do not know if it > is worth it, given ... > >> We could of course retroactively declare that version 2 of the format >> uses the syntax that Git uses right now. After all, JGit only knows to >> read version 1 of it anyway, so that could kind of fix it. But for any >> repository that uses SHA1 we used to write version 1 anyway, so this >> does not really buy us anything, I'd claim. >> >> In summary: >> >> - We have an upper limit in divergence of <10h. > > ... this. > >> >> - This only matters in the context of reflogs, we don't use these >> anywhere else. >> >> - The risk for data loss by a change is limited as our default grace >> period for garbage collecting reflog entries is 30 days. >> >> With these points I'm inclined to call it a bug and just fix it, without >> handling backwards compatibility. ``` ## Patrick Steinhardt, 2026-09-29 07:10 Subject: Re: Reftable reflog timezone encoding differs from specification Message-ID: In-Reply-To: <2abba760-d331-4cad-bb8b-6e567b517beb@app.fastmail.com> ``` On Mon, Sep 28, 2026 at 09:02:36AM -0700, Josh McKinney wrote: > I think my main concern here is mostly around using multiple tools on > the same repo and how they should interpret on disk formats (my > clanker picked up the problem when comparing git's output with a > library it's writing). Yeah, and sticking to the spec we have is the best way to fix that, I'd think. > Anyway, nothing urgent on the problem from me because I noticed it > purely in a development context. Thanks for filling in the bits about > the real world impact on this too. Well, I think fixing it is somewhat urgent -- the longer we have the inconsistency the more problems it causes. I'll aim for having a fix for this ready later this week. Thanks for detecting this! Patrick ```