git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/3] commit-graph: error out on invalid commit oids in 'write --stdin-commits'

From
Taylor Blau <me@ttaylorr.com>
Date
Apr 3, 2020, 18:49 UTC
Message-ID
<20200403184933.GA57202@syl.local>
In-Reply-To
<20200403183057.GA659224@coredump.intra.peff.net>
On Fri, Apr 03, 2020 at 02:30:57PM -0400, Jeff King wrote:
Show 65 quoted lines
> On Mon, Aug 05, 2019 at 10:02:40AM +0200, SZEDER Gábor wrote:
>
> > While 'git commit-graph write --stdin-commits' expects commit object
> > ids as input, it accepts and silently skips over any invalid commit
> > object ids, and still exits with success:
> >
> >   # nonsense
> >   $ echo not-a-commit-oid | git commit-graph write --stdin-commits
> >   $ echo $?
> >   0
> >   # sometimes I forgot that refs are not good...
> >   $ echo HEAD | git commit-graph write --stdin-commits
> >   $ echo $?
> >   0
> >   # valid tree OID, but not a commit OID
> >   $ git rev-parse HEAD^{tree} | git commit-graph write --stdin-commits
> >   $ echo $?
> >   0
> >   $ ls -l .git/objects/info/commit-graph
> >   ls: cannot access '.git/objects/info/commit-graph': No such file or directory
> >
> > Check that all input records are indeed valid commit object ids and
> > return with error otherwise, the same way '--stdin-packs' handles
> > invalid input; see e103f7276f (commit-graph: return with errors during
> > write, 2019-06-12).
>
> Can you explain more why the old behavior is a problem? For reasons (see
> below), we want to do something like:
>
>   git for-each-ref --format='%(objectname)' |
>   git commit-graph write --stdin-commits
>
> In v2.23 and earlier, that worked exactly like --reachable, but now it
> will blow up if there are any refs that point to a non-commit (e.g., a
> tag of a blob).
>
> It can be worked around by asking for %(objecttype) and %(*objecttype)
> and grepping the result, but that's awkward and much less efficient
> (especially if you have a lot of annotated tags, as we may have to open
> and parse each one).
>
> Now obviously you could just use --reachable for the code above. But
> here are two plausible cases where you might not want to do that:
>
>  - you're limiting the graph to only a subset of refs (e.g., you want to
>    graph refs/heads/ and refs/tags, but not refs/some-other-weird-area/).
>
>  - you're generating an incremental graph update. You know somehow that
>    a few refs were updated, and you want to feed those tips to generate
>    the incremental, but not the rest of the refs (not because it would
>    be wrong to do so, but in the name of keeping it O(size of change)
>    and not O(number of refs in the repo).
>
> The latter is the actual case that bit us. I suppose one could do
> something like:
>
>   git rev-list --no-walk <maybe-commits |
>   git commit-graph write --stdin-commits
>
> to use rev-list as a filter, but that feels kind of baroque.
>
> Normally I'm in favor of more error checking instead of less, but in
> this case it feels like it's putting scripted use at a disadvantage
> versus the internal code (e.g., the auto-write for git-fetch uses the
> "--reachable" semantics for its internal invocation).

For what it's worth, (and in case it wasn't obvious) this came about because we feed '--stdin-commits' at GitHub, and observed exactly this error case. I wasn't sure what approach would be more palatable, so I prepared both in my fork at https://github.com/ttaylorr/git:

  - Branch 'tb/commit-graph-dont-check-oids' drops this checking
    entirely.
  - Branch 'tb/commit-graph-check-oids-option' adds a
    '--[no-]check-oids', in case that this is generally desirable
    behavior, by offering an opt-out of this OID checking.

Please let me know if you find either of these to be good options, and I'll happily send one of them to the list. Thanks.

Show 5 quoted lines
> -Peff
>
> PS As an aside, I think the internal git-fetch write could benefit from
>    this same trick: feed the set of newly-stored ref tips to the
>    commit-graph machinery, rather than using for_each_ref().

Thanks, Taylor

Previous: Jeff KingNext: SZEDER Gábor
Message 8 of 19 in “commit-graph: error out on invalid commit oids in 'write --stdin-commits'”
  1. 0/3 commit-graph: error out on invalid commit oids in 'write --stdin-commits'SZEDER Gábor, Aug 5, 2019
  2. 1/3 t5318-commit-graph: use 'test_expect_code'SZEDER Gábor, Aug 5, 2019
  3. 2/3 commit-graph: turn a group of write-related macro flags into an enumSZEDER Gábor, Aug 5, 2019
  4. 3/3 commit-graph: error out on invalid commit oids in 'write --stdin-commits'SZEDER Gábor, Aug 5, 2019
  5. Derrick StoleeAug 5, 2019
  6. SZEDER GáborAug 5, 2019
  7. Jeff KingApr 3, 2020
  8. Taylor BlauApr 3, 2020
  9. SZEDER GáborApr 3, 2020
  10. Jeff KingApr 3, 2020
  11. SZEDER GáborApr 3, 2020
  12. Jeff KingApr 3, 2020
  13. Taylor BlauApr 13, 2020
  14. Jeff KingApr 13, 2020
  15. Taylor BlauApr 14, 2020
  16. Taylor BlauApr 3, 2020
  17. Junio C HamanoApr 3, 2020
  18. Taylor BlauApr 3, 2020
  19. SZEDER GáborAug 5, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.