git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 7/8] commit-graph write: don't die if the existing graph is corrupt

From
Junio C Hamano <gitster@pobox.com>
Date
Feb 22, 2019, 23:13 UTC
Message-ID
<xmqqzhqno1r3.fsf@gitster-ct.c.googlers.com>
In-Reply-To
<20190221223753.20070-8-avarab@gmail.com>
Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
> When the commit-graph is written we end up calling
> parse_commit(). This will in turn invoke code that'll consult the
> existing commit-graph about the commit, if the graph is corrupted we
> die.
Irony ;-).
Show 5 quoted lines
> Change the "commit-graph write" codepath to use a new
> parse_commit_no_graph() helper instead of parse_commit() to avoid
> this. The latter will call repo_parse_commit_internal() with
> use_commit_graph=1 as seen in 177722b344 ("commit: integrate commit
> graph with commit parsing", 2018-04-10).

That may slow down writing the graph but would be a sensible way to prevent an error in the existing commit-graph from spreading.

> This might need to be re-visited if we learn to write the commit-graph
> incrementally.

Probably yes. If we were doing "repack -a -d (without -f)" style of "incremental", then we do need to revisit this, which is a moral equivalent of saying "do not reuse delta" to "repack", and it would make it impossible to be incremental.

Hopefully a true "incremental" shouldn't even have to touch existing data, perhaps similar to "repack (without -a)", in which case the equation may be different. If we'd be computing reachability for only new things, there is nothing gained by allowing parse_commit() to peek into existing commit-graph file (by definition, these new things are not in there---that's the reason why we are incrementally extending the commit-graph by computing the reachability for them). I dunno.

Show 25 quoted lines
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index 1ee00fa333..6db658ed66 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -377,7 +377,13 @@ corrupt_graph_verify() {
>  	test_must_fail git commit-graph verify 2>test_err &&
>  	grep -v "^+" test_err >err &&
>  	test_i18ngrep "$grepstr" err &&
> -	git status --short
> +	if test -z "$NO_WRITE_TEST_BACKUP"
> +	then
> +		cp $objdir/info/commit-graph commit-graph-pre-write-test
> +	fi &&
> +	git status --short &&
> +	GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD=true git commit-graph write &&
> +	git commit-graph verify
>  }
>  
>  # usage: corrupt_graph_and_verify <position> <data> <string> [<zero_pos>]
> @@ -408,7 +414,7 @@ test_expect_success 'detect permission problem' '
>  	# "chmod 000 file" does not yield EACCES on e.g. "cat file"
>  	if ! test -r $objdir/info/commit-graph
>  	then
> -		corrupt_graph_verify "Could not open"
> +		NO_WRITE_TEST_BACKUP=1 corrupt_graph_verify "Could not open"

This would not work as you think it would; corrupt_graph_verify is a shell function, so you cannot VAR=VAL prefix to export an environment variable only for the duration of the command.

Show 10 quoted lines
>  	fi
>  '
>  
> @@ -528,6 +534,7 @@ test_expect_success 'git fsck (checks commit-graph)' '
>  	git fsck &&
>  	corrupt_graph_and_verify $GRAPH_BYTE_FOOTER "\00" \
>  		"incorrect checksum" &&
> +	cp commit-graph-pre-write-test $objdir/info/commit-graph &&
>  	test_must_fail git fsck
>  '
Previous: Ævar Arnfjörð BjarmasonNext: Eric Sunshine
Message 10 of 36 in “commit-graph: segfault & other fixes for broken graphs”
  1. 0/8 commit-graph: segfault & other fixes for broken graphsÆvar Arnfjörð Bjarmason, Feb 21, 2019
  2. 1/8 commit-graph tests: split up corrupt_graph_and_verify()Ævar Arnfjörð Bjarmason, Feb 21, 2019
  3. 2/8 commit-graph tests: test a graph that's too smallÆvar Arnfjörð Bjarmason, Feb 21, 2019
  4. 3/8 commit-graph: fix segfault on e.g. "git status"Ævar Arnfjörð Bjarmason, Feb 21, 2019
  5. 4/8 commit-graph: don't early exit(1) on e.g. "git status"Ævar Arnfjörð Bjarmason, Feb 21, 2019
  6. 6/8 commit-graph verify: detect inability to read the graphÆvar Arnfjörð Bjarmason, Feb 21, 2019
  7. SZEDER GáborFeb 21, 2019
  8. 5/8 commit-graph: don't pass filename to load_commit_graph_one_fd_st()Ævar Arnfjörð Bjarmason, Feb 21, 2019
  9. 7/8 commit-graph write: don't die if the existing graph is corruptÆvar Arnfjörð Bjarmason, Feb 21, 2019
  10. Junio C HamanoFeb 22, 2019
  11. Eric SunshineFeb 23, 2019
  12. 8/8 commit-graph: improve & i18n error messagesÆvar Arnfjörð Bjarmason, Feb 21, 2019
  13. 0/8 commit-graph: segfault & other fixes for broken graphsÆvar Arnfjörð Bjarmason, Mar 14, 2019
  14. Eric SunshineMar 15, 2019
  15. Ævar Arnfjörð BjarmasonMar 15, 2019
  16. 0/8 commit-graph: segfault & other fixes for broken graphsÆvar Arnfjörð Bjarmason, Mar 25, 2019
  17. 1/8 commit-graph tests: split up corrupt_graph_and_verify()Ævar Arnfjörð Bjarmason, Mar 25, 2019
  18. 2/8 commit-graph tests: test a graph that's too smallÆvar Arnfjörð Bjarmason, Mar 25, 2019
  19. 3/8 commit-graph: fix segfault on e.g. "git status"Ævar Arnfjörð Bjarmason, Mar 25, 2019
  20. 4/8 commit-graph: don't early exit(1) on e.g. "git status"Ævar Arnfjörð Bjarmason, Mar 25, 2019
  21. Ævar Arnfjörð BjarmasonApr 27, 2019
  22. Derrick StoleeApr 29, 2019
  23. Ævar Arnfjörð BjarmasonApr 29, 2019
  24. Jeff KingMay 1, 2019
  25. 5/8 commit-graph: don't pass filename to load_commit_graph_one_fd_st()Ævar Arnfjörð Bjarmason, Mar 25, 2019
  26. 8/8 commit-graph: improve & i18n error messagesÆvar Arnfjörð Bjarmason, Mar 25, 2019
  27. 7/8 commit-graph write: don't die if the existing graph is corruptÆvar Arnfjörð Bjarmason, Mar 25, 2019
  28. 6/8 commit-graph verify: detect inability to read the graphÆvar Arnfjörð Bjarmason, Mar 25, 2019
  29. 1/8 commit-graph tests: split up corrupt_graph_and_verify()Ævar Arnfjörð Bjarmason, Mar 14, 2019
  30. 2/8 commit-graph tests: test a graph that's too smallÆvar Arnfjörð Bjarmason, Mar 14, 2019
  31. 3/8 commit-graph: fix segfault on e.g. "git status"Ævar Arnfjörð Bjarmason, Mar 14, 2019
  32. 5/8 commit-graph: don't pass filename to load_commit_graph_one_fd_st()Ævar Arnfjörð Bjarmason, Mar 14, 2019
  33. 4/8 commit-graph: don't early exit(1) on e.g. "git status"Ævar Arnfjörð Bjarmason, Mar 14, 2019
  34. 6/8 commit-graph verify: detect inability to read the graphÆvar Arnfjörð Bjarmason, Mar 14, 2019
  35. 7/8 commit-graph write: don't die if the existing graph is corruptÆvar Arnfjörð Bjarmason, Mar 14, 2019
  36. 8/8 commit-graph: improve & i18n error messagesÆvar Arnfjörð Bjarmason, Mar 14, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.