git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 8/8] commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flag

From
Taylor Blau <me@ttaylorr.com>
Date
May 5, 2020, 16:16 UTC
Message-ID
<20200505161649.GG69300@syl.local>
In-Reply-To
<81fb0f5e-e7e1-7541-d6c7-36828190c95b@gmail.com>
On Tue, May 05, 2020 at 08:10:40AM -0400, Derrick Stolee wrote:
Show 27 quoted lines
> On 5/4/2020 9:14 PM, Taylor Blau wrote:
> > Since 7c5c9b9c57 (commit-graph: error out on invalid commit oids in
> > 'write --stdin-commits', 2019-08-05), the commit-graph builtin dies on
> > receiving non-commit OIDs as input to '--stdin-commits'.
> >
> > This behavior can be cumbersome to work around in, say, the case of
> > piping 'git for-each-ref' to 'git commit-graph write --stdin-commits' if
> > the caller does not want to cull out non-commits themselves. In this
> > situation, it would be ideal if 'git commit-graph write' wrote the graph
> > containing the inputs that did pertain to commits, and silently ignored
> > the remainder of the input.
> >
> > Some options have been proposed to the effect of '--[no-]check-oids'
> > which would allow callers to have the commit-graph builtin do just that.
> > After some discussion, it is difficult to imagine a caller who wouldn't
> > want to pass '--no-check-oids', suggesting that we should get rid of the
> > behavior of complaining about non-commit inputs altogether.
> >
> > If callers do wish to retain this behavior, they can easily work around
> > this change by doing the following:
> >
> >     git for-each-ref --format='%(objectname) %(objecttype) %(*objecttype)' |
> >     awk '/commit/ { print $1 }' |
> >     git commit-graph write --stdin-commits
>
> I appreciate that you included the workaround here for posterity. That
> allows anyone complaining to bisect to these instructions.

Well, now that I linked provided a link to [2] in the cover letter, you'll see that it was Peff's suggestion ;).

But, I wrote something similar to this in one of GitHub's post-processing jobs before we had this series, so I think I can take some of the credit!

Show 80 quoted lines
> > To make it so that valid OIDs that refer to non-existent objects are
> > indeed an error after loosening the error handling, perform an extra
> > lookup to make sure that object indeed exists before sending it to the
> > commit-graph internals.
> >
> > Signed-off-by: Taylor Blau <me@ttaylorr.com>
> > ---
> >  Documentation/git-commit-graph.txt |  6 ++++--
> >  builtin/commit-graph.c             | 10 +++++-----
> >  commit-graph.c                     |  2 --
> >  commit-graph.h                     | 10 ++++------
> >  t/t5318-commit-graph.sh            | 15 +++++++++++----
> >  5 files changed, 24 insertions(+), 19 deletions(-)
> >
> > diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt
> > index 53a650225a..fcac7d12e1 100644
> > --- a/Documentation/git-commit-graph.txt
> > +++ b/Documentation/git-commit-graph.txt
> > @@ -47,8 +47,10 @@ with `--stdin-commits` or `--reachable`.)
> >  +
> >  With the `--stdin-commits` option, generate the new commit graph by
> >  walking commits starting at the commits specified in stdin as a list
> > -of OIDs in hex, one OID per line. (Cannot be combined with
> > -`--stdin-packs` or `--reachable`.)
> > +of OIDs in hex, one OID per line. OIDs that resolve to non-commits
> > +(either directly, or by peeling tags) are silently ignored. OIDs that
> > +are malformed, or do not exist generate an error. (Cannot be combined
> > +with `--stdin-packs` or `--reachable`.)
> >  +
> >  With the `--reachable` option, generate the new commit graph by walking
> >  commits starting at all refs. (Cannot be combined with `--stdin-commits`
> > diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
> > index 9eec68572f..3637d079fb 100644
> > --- a/builtin/commit-graph.c
> > +++ b/builtin/commit-graph.c
> > @@ -153,13 +153,14 @@ static int read_one_commit(struct oidset *commits, struct progress *progress,
> >
> >  	display_progress(progress, oidset_size(commits) + 1);
> >
> > +	if (oid_object_info(the_repository, &oid, NULL) < 0) {
> > +		error(_("object %s does not exist"), hash);
> > +		return 1;
> > +	}
> > +
>
> If we get a non-existent object, then this will cause us to fail
> the command, right?
>
> > diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> > index 89020d3d44..74f93f0a17 100755
> > --- a/t/t5318-commit-graph.sh
> > +++ b/t/t5318-commit-graph.sh
> > @@ -84,11 +84,18 @@ graph_read_expect() {
> >
> >  test_expect_success 'exit with correct error on bad input to --stdin-commits' '
> >  	cd "$TRASH_DIRECTORY/full" &&
> > -	echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr &&
> > +	# invalid, non-hex OID
> > +	echo HEAD >in &&
> > +	test_expect_code 1 git commit-graph write --stdin-commits <in 2>stderr &&
> >  	test_i18ngrep "unexpected non-hex object ID: HEAD" stderr &&
> > -	# valid tree OID, but not a commit OID
> > -	git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr &&
> > -	test_i18ngrep "invalid commit object id" stderr
> > +	# non-existent OID
> > +	echo $ZERO_OID >in &&
> > +	test_expect_code 1 git commit-graph write --stdin-commits <in 2>stderr &&
> > +	test_i18ngrep "does not exist" stderr &&
>
> And here you verify that it fails at that point. Excellent!
>
> > +	# valid commit and tree OID
> > +	git rev-parse HEAD HEAD^{tree} >in &&
> > +	git commit-graph write --stdin-commits <in &&
> > +	graph_read_expect 3
> >  '
>
> This is an excellent series! Thanks.
>
> Reviewed-by: Derrick Stolee <dstolee@microsoft.com>

Thanks for your review. I applied your suggested changes locally (which came down to tweaking the progress meter's messages in the 2nd and 5th patches), but I'll sit on them for a day or so to make sure that nobody else has any thoughts before sending a v2.

Thanks, Taylor

Previous: Derrick StoleeNext: Jeff King
Message 23 of 59 in “commit-graph: drop CHECK_OIDS, peel in callers”
  1. 0/8 commit-graph: drop CHECK_OIDS, peel in callersTaylor Blau, May 5, 2020
  2. 1/8 commit-graph.c: extract 'refs_cb_data'Taylor Blau, May 5, 2020
  3. 2/8 commit-graph.c: show progress of finding reachable commitsTaylor Blau, May 5, 2020
  4. Derrick StoleeMay 5, 2020
  5. Taylor BlauMay 5, 2020
  6. 3/8 commit-graph.c: peel refs in 'add_ref_to_set'Taylor Blau, May 5, 2020
  7. Jeff KingMay 7, 2020
  8. Taylor BlauMay 13, 2020
  9. Jeff KingMay 13, 2020
  10. 4/8 builtin/commit-graph.c: extract 'read_one_commit()'Taylor Blau, May 5, 2020
  11. Jeff KingMay 7, 2020
  12. Taylor BlauMay 13, 2020
  13. 5/8 builtin/commit-graph.c: dereference tags in builtinTaylor Blau, May 5, 2020
  14. Derrick StoleeMay 5, 2020
  15. Taylor BlauMay 5, 2020
  16. Jeff KingMay 7, 2020
  17. 6/8 commit-graph.c: simplify 'fill_oids_from_commits'Taylor Blau, May 5, 2020
  18. Derrick StoleeMay 5, 2020
  19. Jeff KingMay 7, 2020
  20. 7/8 t5318: reorder test below 'graph_read_expect'Taylor Blau, May 5, 2020
  21. 8/8 commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flagTaylor Blau, May 5, 2020
  22. Derrick StoleeMay 5, 2020
  23. Taylor BlauMay 5, 2020
  24. Jeff KingMay 7, 2020
  25. Taylor BlauMay 13, 2020
  26. Derrick StoleeMay 5, 2020
  27. Taylor BlauMay 5, 2020
  28. 0/8 commit-graph: drop CHECK_OIDS, peel in callersTaylor Blau, May 6, 2020
  29. 1/8 commit-graph.c: extract 'refs_cb_data'Taylor Blau, May 6, 2020
  30. 2/8 commit-graph.c: show progress of finding reachable commitsTaylor Blau, May 6, 2020
  31. 3/8 commit-graph.c: peel refs in 'add_ref_to_set'Taylor Blau, May 6, 2020
  32. 4/8 builtin/commit-graph.c: extract 'read_one_commit()'Taylor Blau, May 6, 2020
  33. 5/8 builtin/commit-graph.c: dereference tags in builtinTaylor Blau, May 6, 2020
  34. 6/8 commit-graph.c: simplify 'fill_oids_from_commits'Taylor Blau, May 6, 2020
  35. 7/8 t5318: reorder test below 'graph_read_expect'Taylor Blau, May 6, 2020
  36. 8/8 commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flagTaylor Blau, May 6, 2020
  37. Jeff KingMay 7, 2020
  38. 0/8 commit-graph: drop CHECK_OIDS, peel in callersTaylor Blau, May 13, 2020
  39. 1/8 commit-graph.c: extract 'refs_cb_data'Taylor Blau, May 13, 2020
  40. 2/8 commit-graph.c: show progress of finding reachable commitsTaylor Blau, May 13, 2020
  41. 3/8 commit-graph.c: peel refs in 'add_ref_to_set'Taylor Blau, May 13, 2020
  42. 4/8 builtin/commit-graph.c: extract 'read_one_commit()'Taylor Blau, May 13, 2020
  43. Jeff KingMay 14, 2020
  44. Taylor BlauMay 14, 2020
  45. Junio C HamanoMay 14, 2020
  46. Taylor BlauMay 18, 2020
  47. 5/8 builtin/commit-graph.c: dereference tags in builtinTaylor Blau, May 13, 2020
  48. Jeff KingMay 14, 2020
  49. Taylor BlauMay 14, 2020
  50. commit-graph: fix "Collecting commits from input" progress lineSZEDER Gábor, Jul 10, 2020
  51. Taylor BlauJul 10, 2020
  52. SZEDER GáborJul 15, 2020
  53. Derrick StoleeJul 15, 2020
  54. Junio C HamanoJul 15, 2020
  55. 6/8 commit-graph.c: simplify 'fill_oids_from_commits'Taylor Blau, May 13, 2020
  56. 7/8 t5318: reorder test below 'graph_read_expect'Taylor Blau, May 13, 2020
  57. 8/8 commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flagTaylor Blau, May 13, 2020
  58. Jeff KingMay 14, 2020
  59. Jeff KingMay 14, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.