git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 4/4] commit-graph: fix generation number v2 overflow values

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Feb 28, 2022, 15:40 UTC
Message-ID
<220228.868rtv3sfx.gmgdl@evledraar.gmail.com>
In-Reply-To
<193217c71e0aaf3f56a02d9abec6753bd19aba71.1646056423.git.gitgitgadget@gmail.com>
On Mon, Feb 28 2022, Derrick Stolee via GitGitGadget wrote:
Show 19 quoted lines
> From: Derrick Stolee <derrickstolee@github.com>
>
> The Generation Data Chunk was implemented and tested in e8b63005c
> (commit-graph: implement generation data chunk, 2021-01-16), but the
> test was carefully constructed to work on systems with 32-bit dates.
> Since the corrected commit date offsets still required more than 31
> bits, this triggered writing the generation_data_overflow chunk.
>
> However, upon closer look, the
> write_graph_chunk_generation_data_overflow() method writes the offsets
> to the chunk (as dictated by the format) but fill_commit_graph_info()
> treats the value in the chunk as if it is the full corrected commit date
> (not an offset). For some reason, this does not cause an issue when
> using the FUTURE_DATE specified in t5318-commit-graph.sh, but it does
> show up as a failure in 'git commit-graph verify' if we increase that
> FUTURE_DATE to be above four billion.
>
> Fix this error and update the test to require 64-bit dates so we can
> safely use this large value in our test.

Hrm, so perhaps re my comment on 2/4 @2147483646 was never used? I'm not sure I understand this.

Show 20 quoted lines
> diff --git a/commit-graph.c b/commit-graph.c
> index 8e52bb09552..b86a6a634fe 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -806,7 +806,7 @@ static void fill_commit_graph_info(struct commit *item, struct commit_graph *g,
>  				die(_("commit-graph requires overflow generation data but has none"));
>  
>  			offset_pos = offset ^ CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW;
> -			graph_data->generation = get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
> +			graph_data->generation = item->date + get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
>  		} else
>  			graph_data->generation = item->date + offset;
>  	} else
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index 1afee1c2705..f4ffaad661d 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -815,6 +815,15 @@ test_expect_success 'corrupt commit-graph write (missing tree)' '
>  	)
>  '
This goes back to my comment on 3/4 but:
Show 9 quoted lines
> +# The remaining tests check timestamps that flow over
> +# 32-bits. The graph_git_behavior checks can't take a
> +# prereq, so just stop here if we are on a 32-bit machine.
> +
> +if ! test_have_prereq TIME_IS_64BIT || ! test_have_prereq TIME_T_IS_64BIT
> +then
> +	test_done
> +fi
> +
This...(continued)...
Show 13 quoted lines
>  # We test the overflow-related code with the following repo history:
>  #
>  #               4:F - 5:N - 6:U
> @@ -832,10 +841,10 @@ test_expect_success 'corrupt commit-graph write (missing tree)' '
>  # The largest offset observed is 2 ^ 31, just large enough to overflow.
>  #
>  
> -test_expect_success 'set up and verify repo with generation data overflow chunk' '
> +test_expect_success TIME_IS_64BIT,TIME_T_IS_64BIT 'set up and verify repo with generation data overflow chunk' '
>  	objdir=".git/objects" &&
>  	UNIX_EPOCH_ZERO="@0 +0000" &&
> -	FUTURE_DATE="@2147483646 +0000" &&
> +	FUTURE_DATE="@4000000000 +0000" &&

Hrm, again this may be over my head, but @4147483646 instead of @2147483646 in the other test, but @4000000000 instead of the same here?

Show 12 quoted lines
>  	test_oid_cache <<-EOF &&
>  	oid_version sha1:1
>  	oid_version sha256:2
> @@ -867,4 +876,8 @@ test_expect_success 'set up and verify repo with generation data overflow chunk'
>  
>  graph_git_behavior 'generation data overflow chunk repo' repo left right
>  
> +# Do not add tests at the end of this file, unless they require 64-bit
> +# timestamps, since this portion of the script is only executed when
> +# time data types have 64 bits.
> +
>  test_done

...and this would really be much nicer if we split this test up into its own file, which would be obviously named to note the issue. tXXXX-commit-graph-64bit-timestamp.sh or something.

As shown in my recent 0a2bfccb9c8 (t0051: use "skip_all" under !MINGW in
single-test file, 2022-02-04) you'll also get much nicer output from
"prove" in that case.
 
Previous: Derrick Stolee via GitGitGadgetNext: Ævar Arnfjörð Bjarmason
Message 60 of 70 in “Commit-graph: Generation Number v2 Fixes, v3 implementation”
  1. 0/7 Commit-graph: Generation Number v2 Fixes, v3 implementationDerrick Stolee via GitGitGadget, Feb 24, 2022
  2. 2/7 commit-graph: fix ordering bug in generation numbersDerrick Stolee via GitGitGadget, Feb 24, 2022
  3. Junio C HamanoFeb 24, 2022
  4. Derrick StoleeFeb 25, 2022
  5. Junio C HamanoFeb 25, 2022
  6. 1/7 test-read-graph: include extra post-parse infoDerrick Stolee via GitGitGadget, Feb 24, 2022
  7. 3/7 commit-graph: start parsing generation v2 (again)Derrick Stolee via GitGitGadget, Feb 24, 2022
  8. Patrick SteinhardtFeb 28, 2022
  9. Derrick StoleeFeb 28, 2022
  10. Patrick SteinhardtFeb 28, 2022
  11. Derrick StoleeFeb 28, 2022
  12. Patrick SteinhardtMar 1, 2022
  13. Patrick SteinhardtMar 1, 2022
  14. Derrick StoleeMar 1, 2022
  15. Patrick SteinhardtMar 1, 2022
  16. Derrick StoleeMar 1, 2022
  17. Patrick SteinhardtMar 2, 2022
  18. Derrick StoleeMar 2, 2022
  19. Junio C HamanoMar 2, 2022
  20. Derrick StoleeMar 2, 2022
  21. Junio C HamanoMar 2, 2022
  22. Patrick SteinhardtMar 3, 2022
  23. Derrick StoleeMar 3, 2022
  24. Derrick StoleeMar 4, 2022
  25. Patrick SteinhardtMar 7, 2022
  26. Derrick StoleeMar 7, 2022
  27. Junio C HamanoMar 7, 2022
  28. Patrick SteinhardtMar 10, 2022
  29. Derrick StoleeMar 10, 2022
  30. 4/7 commit-graph: fix generation number v2 overflow valuesDerrick Stolee via GitGitGadget, Feb 24, 2022
  31. Junio C HamanoFeb 24, 2022
  32. Derrick StoleeFeb 25, 2022
  33. Junio C HamanoFeb 25, 2022
  34. 5/7 commit-graph: document file format v2Derrick Stolee via GitGitGadget, Feb 24, 2022
  35. Junio C HamanoFeb 24, 2022
  36. Ævar Arnfjörð BjarmasonFeb 25, 2022
  37. Derrick StoleeFeb 28, 2022
  38. Ævar Arnfjörð BjarmasonFeb 28, 2022
  39. Derrick StoleeFeb 28, 2022
  40. Ævar Arnfjörð BjarmasonFeb 28, 2022
  41. Derrick StoleeMar 1, 2022
  42. Ævar Arnfjörð BjarmasonMar 1, 2022
  43. Derrick StoleeMar 1, 2022
  44. 6/7 commit-graph: parse file format v2Derrick Stolee via GitGitGadget, Feb 24, 2022
  45. Junio C HamanoFeb 24, 2022
  46. Derrick StoleeFeb 25, 2022
  47. 7/7 commit-graph: write file format v2Derrick Stolee via GitGitGadget, Feb 24, 2022
  48. Junio C HamanoFeb 24, 2022
  49. Junio C HamanoFeb 24, 2022
  50. Derrick StoleeFeb 25, 2022
  51. 0/4 Commit-graph: Generation Number v2 FixesDerrick Stolee via GitGitGadget, Feb 28, 2022
  52. 1/4 test-read-graph: include extra post-parse infoDerrick Stolee via GitGitGadget, Feb 28, 2022
  53. Ævar Arnfjörð BjarmasonFeb 28, 2022
  54. 2/4 commit-graph: fix ordering bug in generation numbersDerrick Stolee via GitGitGadget, Feb 28, 2022
  55. Ævar Arnfjörð BjarmasonFeb 28, 2022
  56. 3/4 commit-graph: start parsing generation v2 (again)Derrick Stolee via GitGitGadget, Feb 28, 2022
  57. Ævar Arnfjörð BjarmasonFeb 28, 2022
  58. Derrick StoleeFeb 28, 2022
  59. 4/4 commit-graph: fix generation number v2 overflow valuesDerrick Stolee via GitGitGadget, Feb 28, 2022
  60. Ævar Arnfjörð BjarmasonFeb 28, 2022
  61. Ævar Arnfjörð BjarmasonMar 1, 2022
  62. 0/5 Commit-graph: Generation Number v2 FixesDerrick Stolee via GitGitGadget, Mar 1, 2022
  63. 3/5 commit-graph: fix ordering bug in generation numbersDerrick Stolee via GitGitGadget, Mar 1, 2022
  64. Junio C HamanoMar 1, 2022
  65. Junio C HamanoMar 1, 2022
  66. Derrick StoleeMar 2, 2022
  67. 5/5 commit-graph: fix generation number v2 overflow valuesDerrick Stolee via GitGitGadget, Mar 1, 2022
  68. 4/5 commit-graph: start parsing generation v2 (again)Derrick Stolee via GitGitGadget, Mar 1, 2022
  69. 1/5 test-read-graph: include extra post-parse infoDerrick Stolee via GitGitGadget, Mar 1, 2022
  70. 2/5 t5318: extract helpers to lib-commit-graph.shDerrick Stolee via GitGitGadget, Mar 1, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.