git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 0/3] compatObjectFormat: fix a BUG(), wrong ID and unknown headers

From
Adrian Ratiu <adrian.ratiu@collabora.com>
Date
Nov 19, 2026, 14:18 UTC
Message-ID
<cover.1794932695.git.adrian.ratiu@collabora.com>
In-Reply-To
<20261112195740.2834341-1-adrian.ratiu@collabora.com>
Hello all,

This is v2 of the compatObjectFormat fixes found while testing sha1-sha256 interop between Git, Gitoxide and Jujutsu.

Because patch 3 drops a check that 08a45903cb (object-file-convert: convert commits that embed signed tags, 2023-10-01) added on purpose, I'd still welcome a look from Eric or Brian.

Changes in v2:
 * Check repo_oid_to_algop() for "< 0" and reword the note on which
   rev-parse forms are covered. The test now writes the object before
   enabling compat instead of copying it into the object directory by
   hand. (Patrick)
 * Match the "tree ", "parent " and "mergetag " headers by their key,
   like read_commit_extra_header_lines() does. (Patrick)
 * Minor casing, spacing, doc and commit msg fixes (Patrick)
v1: https://lore.kernel.org/git/20261112195740.2834341-1-adrian.ratiu@collabora.com/

Many thanks, Adrian

Adrian Ratiu (3):
  setup: reject compatObjectFormat == objectFormat
  rev-parse: die when --output-object-format cannot map an object
  object-file-convert: copy unknown commit headers unchanged
 builtin/rev-parse.c           |  8 +++++---
 object-file-convert.c         | 35 ++++++++++++++++++++++-------------
 setup.c                       |  7 +++++++
 t/t1016-compatObjectFormat.sh | 34 ++++++++++++++++++++++++++++++++++
 t/t1302-repo-version.sh       | 14 ++++++++++++++
 t/t1500-rev-parse.sh          | 14 ++++++++++++++
 6 files changed, 96 insertions(+), 16 deletions(-)
Range-diff against v1:
1:  53c814a1e1 ! 1:  f74a0fd67a setup: reject compatObjectFormat == objectFormat
    @@ setup.c: int verify_repository_format(const struct repository_format *format,
      
     +	if (format->compat_hash_algo == format->hash_algo) {
     +		strbuf_addf(err, _("'%s' is '%s', the same as the object format"),
    -+			    "extensions.compatobjectformat",
    ++			    "extensions.compatObjectFormat",
     +			    hash_algos[format->hash_algo].name);
     +		return -1;
     +	}
    @@ t/t1302-repo-version.sh: abort 0 noop-v1
     +	test_expect_success "compatObjectFormat equal to objectFormat $hash" "
     +		test_when_finished 'rm -rf same' &&
     +		git init --object-format=$hash same &&
    -+		git -C same config core.repositoryformatversion 1 &&
    -+		git -C same config extensions.compatObjectFormat $hash &&
    ++		git -C same config set core.repositoryformatversion 1 &&
    ++		git -C same config set extensions.compatObjectFormat $hash &&
     +		test_must_fail git -C same rev-parse --git-dir 2>err &&
    -+		test_grep \"'extensions.compatobjectformat' is '$hash', the same as the object format\" err &&
    -+		git config --file=same/.git/config --unset extensions.compatObjectFormat &&
    ++		test_grep \"'extensions.compatObjectFormat' is '$hash', the same as the object format\" err &&
    ++		git config unset --file=same/.git/config extensions.compatObjectFormat &&
     +		git -C same rev-parse --git-dir
     +	"
     +done
2:  066fc59207 ! 2:  5b91372748 rev-parse: die when --output-object-format cannot map an object
    @@ Commit message
         rev-parse: die when --output-object-format cannot map an object
     
         "git rev-parse --output-object-format=<compat-format>" should print
    -    an object compat id when extensions.compatObjectFormat is enabled.
    +    the compat ID of an object when extensions.compatObjectFormat is
    +    enabled.
     
    -    Git however doesn't record a compat id for every object, e.g. for
    +    Git however doesn't record a compat ID for every object, e.g. for
         objects received by a fetch that goes through index-pack.
    -    For such objects, rev-parse silently prints the wrong id: it prints
    -    the storage id where the compat id is expected and exits with 0.
    +    For such objects, rev-parse silently prints the wrong ID: it prints
    +    the storage ID where the compat ID is expected and exits with 0.
     
         This happens because rev-parse ignores the repo_oid_to_algop() return
         value. When no mapping is found, that function fails and leaves the
    -    id unchanged, so the storage id is what gets printed.
    +    ID unchanged, so the storage ID is what gets printed.
     
         Fix it to die with the same message as pack-objects. The other callers
         of repo_oid_to_algop() already check its result.
     
    -    This only covers a single revision argument, the one case where
    -    rev-parse converts its output. Other forms, such as "A..B" or "A^@",
    -    print storage ids whatever --output-object-format says. That is left
    -    for a separate change.
    -
    -    The test copies a loose object into the repository by hand, so it
    -    doesn't depend on whether fetch writes compat mappings.
    +    Note that this only covers the case where the user asks for a single
    +    revision, as it is the only case where rev-parse passes the desired
    +    output hash explicitly. All the other cases, like for example when the
    +    user passes a range, do not use the desired output hash at all. Fixing
    +    that would be a bigger change though, so it is left for a future patch
    +    series.
     
         Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
     
    @@ builtin/rev-parse.c: int cmd_rev_parse(int argc,
     -						  output_algo, &oid);
     +			if (output_algo &&
     +			    repo_oid_to_algop(the_repository, &oid,
    -+					      output_algo, &oid))
    ++					      output_algo, &oid) < 0)
     +				die(_("cannot map object %s to %s"),
     +				    oid_to_hex(&oid), output_algo->name);
      			if (verify)
    @@ t/t1500-rev-parse.sh: test_expect_success RUST 'rev-parse --show-object-format i
      '
      
     +test_expect_success RUST 'rev-parse --output-object-format with unmapped object' '
    -+	test_when_finished "rm -rf plain compat" &&
    -+	git init --object-format=sha256 plain &&
    -+	git init --object-format=sha256 compat &&
    -+	git -C compat config extensions.compatobjectformat sha1 &&
    -+	oid=$(echo unmapped | git -C plain hash-object -w --stdin) &&
    -+	# copy the object by hand, so that no compat mapping is written for it
    -+	path=$(test_oid_to_path $oid) &&
    -+	mkdir compat/.git/objects/${path%/*} &&
    -+	cp plain/.git/objects/$path compat/.git/objects/$path &&
    ++	test_when_finished "rm -rf unmapped" &&
    ++	git init --object-format=sha256 unmapped &&
    ++	# write the object before enabling compat, so it gets no compat mapping
    ++	oid=$(echo unmapped | git -C unmapped hash-object -w --stdin) &&
    ++	git -C unmapped config set extensions.compatObjectFormat sha1 &&
     +	echo $oid >expect &&
    -+	git -C compat rev-parse --output-object-format=sha256 $oid >actual &&
    ++	git -C unmapped rev-parse --output-object-format=sha256 $oid >actual &&
     +	test_cmp expect actual &&
    -+	test_must_fail git -C compat rev-parse --output-object-format=sha1 $oid >actual 2>err &&
    ++	test_must_fail git -C unmapped rev-parse --output-object-format=sha1 $oid >actual 2>err &&
     +	test_must_be_empty actual &&
     +	test_grep "cannot map object $oid to sha1" err
     +'
3:  a171e86b54 ! 3:  26080a7a38 object-file-convert: copy unknown commit headers unchanged
    @@ Commit message
         don't know:
     
         1. When a commit is created with commit_tree_extended(), as
    -       "git commit" does, convert_commit_extra_headers() copies unknown
    +       git-commit(1) does, convert_commit_extra_headers() copies unknown
            headers unchanged.
     
    -    2. When an existing commit object is converted (git hash-object -w)
    -       or when it's read by its compat id, convert_commit_object() fails
    +    2. When an existing commit object is converted (git-hash-object(1) -w)
    +       or when it's read by its compat ID, convert_commit_object() fails
            on them instead.
     
    -    So git hash-object refuses such a commit, while git commit --amend
    -    records a compat id for it that can't be read back:
    +    So git-hash-object(1) refuses such a commit, while "git commit --amend"
    +    records a compat ID for it that can't be read back:
            "Failed to convert object from sha256 to sha1".
     
         The rejection was added in 08a45903cb (object-file-convert: convert
         commits that embed signed tags, 2023-10-01). Git can't know what an
    -    unknown header holds. It might hold an object id that needs
    +    unknown header holds. It might hold an object ID that needs
         converting, so convert_commit_object() rejects all of them.
     
         Some tools rely on such headers though. Jujutsu, for example, writes
    @@ Commit message
         converted.
     
         Make convert_commit_object() copy unknown headers and their
    -    continuation lines unchanged, just like convert_commit_extra_headers()
    +    continuation lines unchanged, just like convert_commit_extra_headers(),
         so both give the same compat object.
     
         The author, committer, encoding and gpgsig headers were already copied
         the same, so their separate cases are no longer needed.
     
    -    A "tree", "parent" or "mergetag" line with nothing after the keyword
    -    used to be rejected as unknown and is now copied as well. Such a
    -    commit is malformed in both formats anyway.
    +    The "tree ", "parent " and "mergetag " cases only matched a line with
    +    something after the space. That didn't matter while all other lines
    +    were rejected, but they would now be copied. For a "mergetag " line
    +    whose tag starts on the next line, the object ID in the tag would be
    +    copied unconverted, even though Git reads that header as a mergetag and
    +    convert_commit_extra_headers() refuses to convert it.
    +
    +    Match these headers by their key alone instead, the way
    +    read_commit_extra_header_lines() does. Such a line with an empty or bad
    +    value is then rejected, as before. A "tree", "parent" or "mergetag"
    +    line without a space is not one of these headers to Git, so it is
    +    copied like any other unknown header, which is also what
    +    convert_commit_extra_headers() does.
     
         Alternatives considered:
     
    @@ Commit message
            Both paths would then agree, but commits from tools that add
            their own headers could not be used in a compat repo.
     
    -     - Copy only headers known to hold no object ids. Every tool would
    +     - Copy only headers known to hold no object IDs. Every tool would
            first need to get its headers added to that list.
     
    -    The test writes a commit with an unknown header with "git hash-object",
    -    amends it with "git commit", and checks that both commits map to the
    -    same objects in the SHA-1 and the SHA-256 repo.
    +     - Convert values that look like object IDs. Any other hex string
    +       could be taken for one, and Git shouldn't interpret headers it
    +       doesn't know.
    +
    +     - Also reject lines that don't look like "<key> <value>". This would
    +       make the two paths disagree again, because
    +       convert_commit_extra_headers() copies such lines, so
    +       "git commit --amend" would record compat IDs that can't be read
    +       back.
     
         Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
     
      ## object-file-convert.c ##
    +@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
    + 		if (!eol)
    + 			return error(_("bad %s in commit"), "line");
    + 
    +-		if (((bufptr + 5) < eol) && !memcmp(bufptr, "tree ", 5))
    ++		if (((bufptr + 5) <= eol) && !memcmp(bufptr, "tree ", 5))
    + 		{
    + 			if (((bufptr + tree_entry_len) != eol) ||
    + 			    parse_oid_hex_algop(bufptr + 5, &oid, &p, from) ||
    +@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
    + 					     "tree", oid_to_hex(&oid));
    + 			strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid));
    + 		}
    +-		else if (((bufptr + 7) < eol) && !memcmp(bufptr, "parent ", 7))
    ++		else if (((bufptr + 7) <= eol) && !memcmp(bufptr, "parent ", 7))
    + 		{
    + 			if (((bufptr + parent_entry_len) != eol) ||
    + 			    parse_oid_hex_algop(bufptr + 7, &oid, &p, from) ||
    +@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
    + 
    + 			strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid));
    + 		}
    +-		else if (((bufptr + 9) < eol) && !memcmp(bufptr, "mergetag ", 9))
    ++		else if (((bufptr + 9) <= eol) && !memcmp(bufptr, "mergetag ", 9))
    + 		{
    + 			struct strbuf tag = STRBUF_INIT, new_tag = STRBUF_INIT;
    + 
     @@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
      			strbuf_release(&tag);
      			strbuf_release(&new_tag);
    @@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
     -			/* Unknown line fail it might embed an oid */
     -			return -1;
     +			/*
    -+			 * Copy all other headers unchanged. author, committer, encoding and gpgsig
    -+			 * hold no object ids. For unknown headers we can't tell, so copy them too,
    -+			 * like convert_commit_extra_headers() does when the commit is written. Both
    -+			 * must give the same compat object.
    ++			 * Copy all other headers unchanged. author, committer,
    ++			 * encoding and gpgsig hold no object IDs. For unknown
    ++			 * headers we can't tell, so copy them too, like
    ++			 * convert_commit_extra_headers() does when the commit
    ++			 * is written. Both must give the same compat object.
    ++			 *
    ++			 * This means that a header holding object IDs is
    ++			 * copied without converting them. Any such header added
    ++			 * to Git needs its own case above, like parent or
    ++			 * mergetag, and the same conversion in
    ++			 * convert_commit_extra_headers().
     +			 *
    -+			 * This means that a header holding object ids is copied without converting
    -+			 * them. Any such header added to Git needs its own case above, like parent
    -+			 * or mergetag, and the same conversion in convert_commit_extra_headers().
    ++			 * The cases above match a header by its key alone, as
    ++			 * read_commit_extra_header_lines() does, so that e.g. a
    ++			 * "mergetag " line with nothing after the space is
    ++			 * still handled as a mergetag instead of being copied.
     +			 */
     +			strbuf_add(out, bufptr, (eol - bufptr) + 1);
      		}
    @@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
     
      ## t/t1016-compatObjectFormat.sh ##
     @@ t/t1016-compatObjectFormat.sh: do
    - 		eval signedtag3_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag3) &&
      		eval signedtag4_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag4)
      	'
    + 
     +	test_expect_success "create a $hash commit with an unknown header" '
     +		git cat-file commit $(commit2_oid $hash) >../${hash}_commit2 &&
     +		awk "{ print } /^committer / { print \"x-unknown one\"; print \" two\" }" \
    @@ t/t1016-compatObjectFormat.sh: do
     +		eval unknownheader_${hash}_oid=$oid &&
     +		git branch unknownheader $oid
     +	'
    ++
     +	test_expect_success "amend a $hash commit with an unknown header" '
     +		git checkout unknownheader &&
     +		git commit --amend -m "Amend a commit with an unknown header" &&
    @@ t/t1016-compatObjectFormat.sh: do
     +		oid=$(git rev-parse HEAD) &&
     +		eval unknownheader2_${hash}_oid=$oid
     +	'
    - 
    ++
    ++	test_expect_success "reject a $hash commit with an empty mergetag line" '
    ++		git cat-file tag committag >../${hash}_committag &&
    ++		{
    ++			sed -n "/^committer /q;p" ../${hash}_commit2 &&
    ++			grep "^committer " ../${hash}_commit2 &&
    ++			echo "mergetag " &&
    ++			sed "s/^/ /" ../${hash}_committag &&
    ++			echo &&
    ++			echo message
    ++		} >../${hash}_emptymergetag &&
    ++		test_must_fail git hash-object -t commit -w ../${hash}_emptymergetag 2>err &&
    ++		test_grep "Failed to convert object" err
    ++	'
    ++
      	test_expect_success 'rev-parse maps oid of object borrowed from alternate' '
      		for repo in alt borrow
    + 		do
     @@ t/t1016-compatObjectFormat.sh: compare_oids 'blob' more "$more_sha1_oid" "$more_sha256_oid"
      compare_oids 'blob' another "$another_sha1_oid" "$another_sha256_oid"
      compare_oids 'tree' tree2 "$tree2_sha1_oid" "$tree2_sha256_oid"
-- 
2.56.1
Previous: Adrian RatiuNext: Adrian Ratiu
Message 11 of 14 in “compatObjectFormat: fix a BUG(), wrong id and unknown headers”
  1. 0/3 compatObjectFormat: fix a BUG(), wrong id and unknown headersAdrian Ratiu, Nov 12, 2026
  2. 1/3 setup: reject compatObjectFormat == objectFormatAdrian Ratiu, Nov 12, 2026
  3. Patrick SteinhardtNov 16, 2026
  4. Adrian RatiuNov 17, 2026
  5. 2/3 rev-parse: die when --output-object-format cannot map an objectAdrian Ratiu, Nov 12, 2026
  6. Patrick SteinhardtNov 16, 2026
  7. Adrian RatiuNov 17, 2026
  8. 3/3 object-file-convert: copy unknown commit headers unchangedAdrian Ratiu, Nov 12, 2026
  9. Patrick SteinhardtNov 16, 2026
  10. Adrian RatiuNov 17, 2026
  11. 0/3 compatObjectFormat: fix a BUG(), wrong ID and unknown headersAdrian Ratiu, Nov 19, 2026
  12. 1/3 setup: reject compatObjectFormat == objectFormatAdrian Ratiu, Nov 19, 2026
  13. 2/3 rev-parse: die when --output-object-format cannot map an objectAdrian Ratiu, Nov 19, 2026
  14. 3/3 object-file-convert: copy unknown commit headers unchangedAdrian Ratiu, Nov 19, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.