{"thread":{"id":"66403","subject":"Reftable reflog timezone encoding differs from specification","startedAt":"2026-09-28T07:01:05Z","lastAt":"2026-09-29T07:10:20Z","messageCount":5,"participants":["Josh McKinney","Patrick Steinhardt","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"553409","messageId":"85f7daa8-d60b-4348-ac2f-b1a68628af7b@app.fastmail.com","threadId":"66403","inReplyTo":null,"subject":"Reftable reflog timezone encoding differs from specification","fromName":"Josh McKinney","fromEmail":"git-bugs@lists.joshka.net","sentAt":"2026-09-28T07:00:43Z","receivedAt":"2026-09-28T07:01:05Z","isPatch":false,"body":"Hi,\n\nGit 2.55.0 appears to store reftable reflog timezone offsets as signed\nHHMM integers, whereas the specification requires signed minutes.\n\nhttps://git-scm.com/docs/reftable#_log_record states:\n\n    \"tz_offset is the absolute number of minutes from GMT the\n    committer was at the time of the update.\"\n\nThe specification also gives GMT+0230 as an example encoded as 150.\n\nI reproduced the discrepancy on macOS arm64 by creating a SHA-1\nreftable repository and committing with this date:\n\n    2026-09-27T12:00:00+05:30\n\nAn independent Python/zlib inspection of the resulting reftable found:\n\n    Stored timezone bytes: 02 12 = 530\n    Expected signed minutes: 01 4a = 330\n\nBoth HEAD and refs/heads/main reflog entries contained 530. Git reads\nits own entries back correctly as +05:30, so its writer and reader\nappear internally consistent, but disagree with the specification.\n\nHere is a reproducer using only Git and Python's standard library:\n\n(\n    set -eu\n    proof_dir=$(mktemp -d)\n    cd \"$proof_dir\"\n    echo \"Repository retained at: $proof_dir\"\n\n    export GIT_CONFIG_NOSYSTEM=1\n    export GIT_CONFIG_GLOBAL=/dev/null\n    export GIT_AUTHOR_DATE='2026-09-27T12:00:00+05:30'\n    export GIT_COMMITTER_DATE=\"$GIT_AUTHOR_DATE\"\n\n    git --version\n    git init --initial-branch=main --object-format=sha1 \\\n        --ref-format=reftable example\n\n    git -C example \\\n        -c user.name=Example \\\n        -c user.email=example@example.com \\\n        -c commit.gpgsign=false \\\n        -c core.logAllRefUpdates=true \\\n        commit --allow-empty -m \"Timezone example\"\n\n    git -C example reflog show --format='%gD' \\\n        --date=iso-strict refs/heads/main\n\n    python3 - <<'PY'\nfrom pathlib import Path\nimport zlib\n\ndirectory = Path(\"example/.git/reftable\")\nfor name in (directory / \"tables.list\").read_text().splitlines():\n    table = (directory / name).read_bytes()\n    assert table[:5] == b\"REFT\\x01\"\n    footer = table[-68:]\n    log_start = int.from_bytes(footer[48:56], \"big\")\n    if not log_start:\n        continue\n\n    assert table[log_start:log_start + 1] == b\"g\"\n    log = zlib.decompress(table[log_start + 4:])\n\n    # Fixture-specific: email, variable-length time, then timezone.\n    email = b\"example@example.com\"\n    search_from = 0\n    while (position := log.find(email, search_from)) != -1:\n        position += len(email)\n        while log[position] & 0x80:\n            position += 1\n        position += 1\n\n        raw = log[position:position + 2]\n        offset = int.from_bytes(raw, \"big\", signed=True)\n        print(f\"Timezone bytes: {raw.hex(' ')}; integer: {offset}\")\n        search_from = position + 2\nPY\n)\n\nThe relevant output is:\n\n    refs/heads/main@{2026-09-27T12:00:00+05:30}\n    Timezone bytes: 02 12; integer: 530\n    Timezone bytes: 02 12; integer: 530\n\nIs this a known discrepancy? Which representation should interoperable\nimplementations use? If either the implementation or specification\nchanges, how should existing tables be interpreted, given that values\nsuch as 330 are valid under both interpretations?\n\nGiven that Git consistently writes and reads HHMM values, I suspect the practical resolution is to update the specification to match existing behavior. Are there other implementations or compatibility considerations that would prevent that?\n\nThanks,\nJosh\n\n-- \nJosh McKinney\njoshka.net\n"},{"id":"553452","messageId":"arpZ5xCwFXc9ikrj@pks.im","threadId":"66403","inReplyTo":"85f7daa8-d60b-4348-ac2f-b1a68628af7b@app.fastmail.com","subject":"Re: Reftable reflog timezone encoding differs from specification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-28T12:13:27Z","receivedAt":"2026-09-28T12:13:37Z","isPatch":false,"body":"Hi,\n\nOn Mon, Sep 28, 2026 at 12:00:43AM -0700, Josh McKinney wrote:\n> Hi,\n> \n> Git 2.55.0 appears to store reftable reflog timezone offsets as signed\n> HHMM integers, whereas the specification requires signed minutes.\n> \n> https://git-scm.com/docs/reftable#_log_record states:\n> \n>     \"tz_offset is the absolute number of minutes from GMT the\n>     committer was at the time of the update.\"\n> \n> The specification also gives GMT+0230 as an example encoded as 150.\n\nOh dear, that's indeed the case. I was able to reproduce the issue, as\nwell.\n\n> I reproduced the discrepancy on macOS arm64 by creating a SHA-1\n> reftable repository and committing with this date:\n> \n>     2026-09-27T12:00:00+05:30\n> \n> An independent Python/zlib inspection of the resulting reftable found:\n> \n>     Stored timezone bytes: 02 12 = 530\n>     Expected signed minutes: 01 4a = 330\n\nYup. It's plain wrong the way we store it.\n\n> Both HEAD and refs/heads/main reflog entries contained 530. Git reads\n> its own entries back correctly as +05:30, so its writer and reader\n> appear internally consistent, but disagree with the specification.\n> \n> Here is a reproducer using only Git and Python's standard library:\n\nAnd here's a Git reproducer:\n\ndiff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c\nindex fc49fafc34..e3cfb6a3fa 100644\n--- a/t/helper/test-reftable.c\n+++ b/t/helper/test-reftable.c\n@@ -103,7 +103,16 @@ static int dump_table(struct reftable_merged_table *mt)\n \tif (err < 0)\n \t\treturn err;\n \n-\talgop = &hash_algos[hash_algo_by_id(reftable_merged_table_hash_id(mt))];\n+\tswitch (reftable_merged_table_hash_id(mt)) {\n+\tcase REFTABLE_HASH_SHA1:\n+\t\talgop = &hash_algos[GIT_HASH_SHA1];\n+\t\tbreak;\n+\tcase REFTABLE_HASH_SHA256:\n+\t\talgop = &hash_algos[GIT_HASH_SHA256];\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"unsupported hash algorithm: %d\", reftable_merged_table_hash_id(mt));\n+\t}\n \n \twhile (1) {\n \t\terr = reftable_iterator_next_ref(&it, &ref);\ndiff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh\nindex 35e98b43db..e1d714077c 100755\n--- a/t/t0610-reftable-basics.sh\n+++ b/t/t0610-reftable-basics.sh\n@@ -1163,4 +1163,19 @@ test_expect_success 'writes do not persist peeled value for invalid tags' '\n \t)\n '\n \n+test_expect_success 'writes do not persist peeled value for invalid tags' '\n+\ttest_when_finished rm -rf repo &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\n+\t\texport GIT_AUTHOR_DATE='2026-09-27T12:00:00+05:30' &&\n+\t\texport GIT_COMMITTER_DATE=\"$GIT_AUTHOR_DATE\" &&\n+\t\tgit commit --allow-empty --message \"Timezone example\" &&\n+\t\tgit refs optimize &&\n+\t\ttest-tool dump-reftable -t .git/reftable/*.ref >table &&\n+\t\ttest_grep \"log{HEAD(2) C O Mitter <committer@example.com> 1790490600 0530\" table\n+\t)\n+'\n+\n test_done\n\nAnd yes, the test-helper for reftables is broken, so we also have to fix\nthat.\n\n[snip]\n> Is this a known discrepancy?\n\nNo, it's not, I wasn't aware of it at all.\n\n> Which representation should interoperable implementations use?\n\nWe should use the one that we have in our specification, so in my\nopinion we should fix Git itself. This is also because JGit, which had a\nreftable implementation for far longer compared to us, implements the\nspecification correctly:\n\n\tprivate PersonIdent readPersonIdent() {\n\t\tString name = readValueString();\n\t\tString email = readValueString();\n\t\tlong epochSeconds = readVarint64();\n\t\tZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60);\n\t\treturn new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz);\n\t}\n\nYou can see that we indeed treat the integer as number of minutes there,\nas expected.\n\n> If either the implementation or specification changes, how should\n> existing tables be interpreted, given that values such as 330 are\n> valid under both interpretations?\n> \n> Given that Git consistently writes and reads HHMM values, I suspect\n> the practical resolution is to update the specification to match\n> existing behavior. Are there other implementations or compatibility\n> considerations that would prevent that?\n\nThat's a very good question. Given that JGit interprets the value as\nexpected my take is that we should fix this in Git and keep the spec\nas-is.\n\nThe question is how much we really lose by \"just\" fixing the bug. Sure,\ntimezones would be wrong in that case. For example, if we had an entry\nwith original timezone of +0200 we'd now interpret that as +0320. It's\nof course wrong given the original intent, but is it the end of the\nworld? I dunno. Overall, the amount of damage is kind of limited here as\nthe discrepancy is limited:\n\n  ┌──────┬──────────────┬────────────────┬───────────┐\n  │tz    │HHMM encoding │correct minutes │divergence │\n  ├──────┼──────────────┼────────────────┼───────────┤\n  │+1400 │1400          │840             │560        │\n  ├──────┼──────────────┼────────────────┼───────────┤\n  │-1200 │-1200         │-720            │480        │\n  ├──────┼──────────────┼────────────────┼───────────┤\n  │+0530 │530           │330             │200        │\n  ├──────┼──────────────┼────────────────┼───────────┤\n  │+0000 │0             │0               │0          │\n  └──────┴──────────────┴────────────────┴───────────┘\n\nWe could of course retroactively declare that version 2 of the format\nuses the syntax that Git uses right now. After all, JGit only knows to\nread version 1 of it anyway, so that could kind of fix it. But for any\nrepository that uses SHA1 we used to write version 1 anyway, so this\ndoes not really buy us anything, I'd claim.\n\nIn summary:\n\n  - We have an upper limit in divergence of <10h.\n\n  - This only matters in the context of reflogs, we don't use these\n    anywhere else.\n\n  - The risk for data loss by a change is limited as our default grace\n    period for garbage collecting reflog entries is 30 days.\n\nWith these points I'm inclined to call it a bug and just fix it, without\nhandling backwards compatibility.\n\nI'm very happy to hear alternative takes though.\n\nIn any case, thanks for your report!\n\nPatrick\n"},{"id":"553482","messageId":"xmqq33utphdy.fsf@gitster.g","threadId":"66403","inReplyTo":"arpZ5xCwFXc9ikrj@pks.im","subject":"Re: Reftable reflog timezone encoding differs from specification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-09-28T14:42:01Z","receivedAt":"2026-09-28T14:42:05Z","isPatch":false,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> We should use the one that we have in our specification, so in my\n> opinion we should fix Git itself. This is also because JGit, which had a\n> reftable implementation for far longer compared to us, implements the\n> specification correctly:\n>\n>\n> \tprivate PersonIdent readPersonIdent() {\n> \t\tString name = readValueString();\n> \t\tString email = readValueString();\n> \t\tlong epochSeconds = readVarint64();\n> \t\tZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60);\n> \t\treturn new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz);\n> \t}\n\nThanks for checking.  I (unfortunately) agree with the (unfortunate)\nconclusion.\n\nWe do not ship reftable files over networks and reflogs at the\nconceptual level is not shared across repositories, so the issue,\nother than the trivial part of updating the implementation, is how\nto migrate the data in a local repository that uses reftable.  One\ntime offline conversion may be the simplest but I do not know if it\nis worth it, given ...\n\n> We could of course retroactively declare that version 2 of the format\n> uses the syntax that Git uses right now. After all, JGit only knows to\n> read version 1 of it anyway, so that could kind of fix it. But for any\n> repository that uses SHA1 we used to write version 1 anyway, so this\n> does not really buy us anything, I'd claim.\n>\n> In summary:\n>\n>   - We have an upper limit in divergence of <10h.\n\n... this.\n\n>\n>   - This only matters in the context of reflogs, we don't use these\n>     anywhere else.\n>\n>   - The risk for data loss by a change is limited as our default grace\n>     period for garbage collecting reflog entries is 30 days.\n>\n> With these points I'm inclined to call it a bug and just fix it, without\n> handling backwards compatibility.\n\n"},{"id":"553503","messageId":"2abba760-d331-4cad-bb8b-6e567b517beb@app.fastmail.com","threadId":"66403","inReplyTo":"xmqq33utphdy.fsf@gitster.g","subject":"Re: Reftable reflog timezone encoding differs from specification","fromName":"Josh McKinney","fromEmail":"git-bugs@lists.joshka.net","sentAt":"2026-09-28T16:02:36Z","receivedAt":"2026-09-28T16:02:59Z","isPatch":false,"body":"I think my main concern here is mostly around using multiple tools on the same repo and how they should interpret on disk formats (my clanker picked up the problem when comparing git's output with a library it's writing).\n\nAnyway, nothing urgent on the problem from me because I noticed it purely in a development context.\nThanks for filling in the bits about the real world impact on this too.\n\nJosh\n\n-- \nJosh McKinney\njoshka.net\n\nOn Mon, Sep 28, 2026, at 7:42 AM, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n>\n>> We should use the one that we have in our specification, so in my\n>> opinion we should fix Git itself. This is also because JGit, which had a\n>> reftable implementation for far longer compared to us, implements the\n>> specification correctly:\n>>\n>>\n>> \tprivate PersonIdent readPersonIdent() {\n>> \t\tString name = readValueString();\n>> \t\tString email = readValueString();\n>> \t\tlong epochSeconds = readVarint64();\n>> \t\tZoneOffset tz = ZoneOffset.ofTotalSeconds(readInt16() * 60);\n>> \t\treturn new PersonIdent(name, email, Instant.ofEpochSecond(epochSeconds), tz);\n>> \t}\n>\n> Thanks for checking.  I (unfortunately) agree with the (unfortunate)\n> conclusion.\n>\n> We do not ship reftable files over networks and reflogs at the\n> conceptual level is not shared across repositories, so the issue,\n> other than the trivial part of updating the implementation, is how\n> to migrate the data in a local repository that uses reftable.  One\n> time offline conversion may be the simplest but I do not know if it\n> is worth it, given ...\n>\n>> We could of course retroactively declare that version 2 of the format\n>> uses the syntax that Git uses right now. After all, JGit only knows to\n>> read version 1 of it anyway, so that could kind of fix it. But for any\n>> repository that uses SHA1 we used to write version 1 anyway, so this\n>> does not really buy us anything, I'd claim.\n>>\n>> In summary:\n>>\n>>   - We have an upper limit in divergence of <10h.\n>\n> ... this.\n>\n>>\n>>   - This only matters in the context of reflogs, we don't use these\n>>     anywhere else.\n>>\n>>   - The risk for data loss by a change is limited as our default grace\n>>     period for garbage collecting reflog entries is 30 days.\n>>\n>> With these points I'm inclined to call it a bug and just fix it, without\n>> handling backwards compatibility.\n"},{"id":"553552","messageId":"artkVoGEP0iLOGgr@pks.im","threadId":"66403","inReplyTo":"2abba760-d331-4cad-bb8b-6e567b517beb@app.fastmail.com","subject":"Re: Reftable reflog timezone encoding differs from specification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-29T07:10:14Z","receivedAt":"2026-09-29T07:10:20Z","isPatch":false,"body":"On Mon, Sep 28, 2026 at 09:02:36AM -0700, Josh McKinney wrote:\n> I think my main concern here is mostly around using multiple tools on\n> the same repo and how they should interpret on disk formats (my\n> clanker picked up the problem when comparing git's output with a\n> library it's writing).\n\nYeah, and sticking to the spec we have is the best way to fix that, I'd\nthink.\n\n> Anyway, nothing urgent on the problem from me because I noticed it\n> purely in a development context. Thanks for filling in the bits about\n> the real world impact on this too.\n\nWell, I think fixing it is somewhat urgent -- the longer we have the\ninconsistency the more problems it causes. I'll aim for having a fix for\nthis ready later this week.\n\nThanks for detecting this!\n\nPatrick\n"}]}