git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 0/3] Changed path filter hash fix and version bump

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jun 7, 2023, 21:37 UTC
Message-ID
<20230607213712.2589505-1-jonathantanmy@google.com>
In-Reply-To
<20230607163056.1602219-1-jonathantanmy@google.com>
Jonathan Tan <jonathantanmy@google.com> writes:
Show 6 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> > Seems to break t4216 when merged to 'seen' to replace the previous
> > round.  Could you take a look?
> 
> OK - I see that it fails CI when I upload the merge to GitHub (although
> it passes locally). I'll take a look.

Hmm...so when the test writes a file with a high bit filename, the filename written is different. Here's the output from my machine (Linux):

        ++ git -C highbit1/ commit -m c1                                                                                                                          
        [main (root-commit) 808e481] c1                                                                                                                           
         Author: A U Thor <author@example.com>                                                                                                                    
         1 file changed, 1 insertion(+)                                                                                                                           
         create mode 100644 "\302\242"                                                                                                                            
        ++ case "$tag" in                                                                                                                                         
        ++ git -C highbit1/ tag c1                                                                                                                                
        ++ touch $'\302\242\302\242'                                                                                                                              
        ++ ls                                                                                                                                                     
         A        expect   file5_renamed   limits        log_wo_bloom   trace2-auto.txt  ''$'\302\242\302\242'                                                    
         actual   file4    highbit1        log_w_bloom   trace.perf     trace2.txt                                                                                
        ++ git -C highbit1 commit-graph write --reachable --changed-paths                                                                                         
        bloom calculating -62,-94 version 1                                                                                                                       
        ok 141 - set up repo with high bit path, version 1 changed-path       
and here's the output from the CI on GitHub on linux-gcc-default:
        [main (root-commit) 3f37a43] c1
         Author: A U Thor <author@example.com>
         1 file changed, 1 insertion(+)
         create mode 100644 "\\xc2\\xa2"
        + git -C highbit1/ tag c1
        + touch \xc2\xa2\xc2\xa2
        + ls
        A
        \xc2\xa2\xc2\xa2
        actual
        expect
        file4
        file5_renamed
        highbit1
        limits
        log_w_bloom
        log_wo_bloom
        trace.perf
        trace2-auto.txt
        trace2.txt
        + git -C highbit1 commit-graph write --reachable --changed-paths
        bloom calculating 92,120 version 1
        ok 141 - set up repo with high bit path, version 1 changed-path
These were run with some extra code, here for completeness:
        diff --git a/bloom.c b/bloom.c
        index dea883d8d6..ccc3e0ce80 100644
        --- a/bloom.c
        +++ b/bloom.c
        @@ -192,6 +192,8 @@ void fill_bloom_key(const char *data,
                        ? murmur3_seeded_v2 : murmur3_seeded_v1)(seed0, data, len);
                const uint32_t hash1 = (settings->hash_version == 2
                        ? murmur3_seeded_v2 : murmur3_seeded_v1)(seed1, data, len);
        +       if (len > 0)
        +               fprintf(stderr, "bloom calculating %d,%d version %d\n", data[0], data[1], settings->hash_version);
 
                key->hashes = (uint32_t *)xcalloc(settings->num_hashes, sizeof(uint32_t));
                for (i = 0; i < settings->num_hashes; i++)
        diff --git a/t/t4216-log-bloom.sh b/t/t4216-log-bloom.sh
        index 764c6dee0f..1121e7d4cd 100755
        --- a/t/t4216-log-bloom.sh
        +++ b/t/t4216-log-bloom.sh
        @@ -423,6 +423,8 @@ CENT=$(printf "\xc2\xa2")
         test_expect_success 'set up repo with high bit path, version 1 changed-path' '
                git init highbit1 &&
                test_commit -C highbit1 c1 "$CENT" &&
        +       touch $CENT$CENT &&
        +       ls &&
                git -C highbit1 commit-graph write --reachable --changed-paths
         '

Notice the different filename after "create mode", and the different "ls" output. I'll investigate some more, but if anyone offhand knows what's going on (and/or knows how to write a high-bit filename portably, even across Linux), please let me know.

An alternative is to drop the tests from these patches - so, leave them in during the review period and reviewers would see that the tests pass the CI jobs for Windows and Mac OS X pass, and then before we merge it, delete the tests from the patches. This also solves needing to prevent unsigned char platforms from running the version 1 tests - there's no prereq for that yet and we would have to investigate making one.

Previous: Jonathan TanNext: Jonathan Tan
Message 18 of 116 in “Changed path filter hash fix and version bump”
  1. 0/2 Changed path filter hash fix and version bumpJonathan Tan, May 22, 2023
  2. 1/2 t4216: test wrong bloom filter version rejectionJonathan Tan, May 22, 2023
  3. 2/2 commit-graph: fix murmur3, bump filter ver. to 2Jonathan Tan, May 22, 2023
  4. Derrick StoleeMay 23, 2023
  5. Jonathan TanMay 23, 2023
  6. Junio C HamanoMay 23, 2023
  7. Jonathan TanMay 24, 2023
  8. Derrick StoleeMay 26, 2023
  9. Jonathan TanMay 30, 2023
  10. Junio C HamanoMay 23, 2023
  11. 0/3 Changed path filter hash fix and version bumpJonathan Tan, May 31, 2023
  12. 1/3 t4216: test changed path filters with high bit pathsJonathan Tan, May 31, 2023
  13. 2/3 repo-settings: introduce commitgraph.changedPathsVersionJonathan Tan, May 31, 2023
  14. 3/3 commit-graph: new filter ver. that fixes murmur3Jonathan Tan, May 31, 2023
  15. Junio C HamanoJun 3, 2023
  16. Junio C HamanoJun 3, 2023
  17. Jonathan TanJun 7, 2023
  18. Jonathan TanJun 7, 2023
  19. 0/4 Changed path filter hash fix and version bumpJonathan Tan, Jun 8, 2023
  20. 1/4 gitformat-commit-graph: describe version 2 of BDATJonathan Tan, Jun 8, 2023
  21. Ramsay JonesJun 8, 2023
  22. Junio C HamanoJun 12, 2023
  23. 2/4 t4216: test changed path filters with high bit pathsJonathan Tan, Jun 8, 2023
  24. 3/4 repo-settings: introduce commitgraph.changedPathsVersionJonathan Tan, Jun 8, 2023
  25. 4/4 commit-graph: new filter ver. that fixes murmur3Jonathan Tan, Jun 8, 2023
  26. Ramsay JonesJun 8, 2023
  27. Jonathan TanJun 9, 2023
  28. Junio C HamanoJun 12, 2023
  29. Jonathan TanJun 13, 2023
  30. CodingGuidelines: use octal escapes, not hexJonathan Tan, Jun 13, 2023
  31. Eric SunshineJun 13, 2023
  32. Jonathan TanJun 13, 2023
  33. Eric SunshineJun 13, 2023
  34. Junio C HamanoJun 13, 2023
  35. Junio C HamanoJun 13, 2023
  36. 0/4 Changed path filter hash fix and version bumpJonathan Tan, Jun 13, 2023
  37. 2/4 t4216: test changed path filters with high bit pathsJonathan Tan, Jun 13, 2023
  38. 1/4 gitformat-commit-graph: describe version 2 of BDATJonathan Tan, Jun 13, 2023
  39. Junio C HamanoJun 13, 2023
  40. Derrick StoleeJun 20, 2023
  41. Taylor BlauJun 21, 2023
  42. Jonathan TanJun 22, 2023
  43. Derrick StoleeJun 23, 2023
  44. 3/4 repo-settings: introduce commitgraph.changedPathsVersionJonathan Tan, Jun 13, 2023
  45. Derrick StoleeJun 20, 2023
  46. Taylor BlauJun 21, 2023
  47. 4/4 commit-graph: new filter ver. that fixes murmur3Jonathan Tan, Jun 13, 2023
  48. Derrick StoleeJun 20, 2023
  49. Junio C HamanoJun 20, 2023
  50. Junio C HamanoJun 13, 2023
  51. Derrick StoleeJun 20, 2023
  52. Jonathan TanJun 20, 2023
  53. Taylor BlauJun 21, 2023
  54. Derrick StoleeJun 21, 2023
  55. Jonathan TanJun 22, 2023
  56. Derrick StoleeJun 23, 2023
  57. 0/4 Changed path filter hash fix and version bumpJonathan Tan, Jul 13, 2023
  58. 3/4 repo-settings: introduce commitgraph.changedPathsVersionJonathan Tan, Jul 13, 2023
  59. Taylor BlauJul 19, 2023
  60. Jonathan TanJul 20, 2023
  61. Taylor BlauJul 20, 2023
  62. 2/4 t4216: test changed path filters with high bit pathsJonathan Tan, Jul 13, 2023
  63. Junio C HamanoJul 13, 2023
  64. Taylor BlauJul 19, 2023
  65. 0/4 commit-graph: avoid looking at Bloom filter data directlyTaylor Blau, Jul 19, 2023
  66. 1/4 t/helper/test-read-graph.c: extract `dump_graph_info()`Taylor Blau, Jul 19, 2023
  67. 2/4 bloom.h: make `load_bloom_filter_from_graph()` publicTaylor Blau, Jul 19, 2023
  68. 3/4 t/helper/test-read-graph: implement `bloom-filters` modeTaylor Blau, Jul 19, 2023
  69. 4/4 fixup! t4216: test changed path filters with high bit pathsTaylor Blau, Jul 19, 2023
  70. Junio C HamanoJul 19, 2023
  71. Jonathan TanJul 20, 2023
  72. 1/4 gitformat-commit-graph: describe version 2 of BDATJonathan Tan, Jul 13, 2023
  73. Taylor BlauJul 19, 2023
  74. Jonathan TanJul 20, 2023
  75. Taylor BlauJul 21, 2023
  76. 4/4 commit-graph: new filter ver. that fixes murmur3Jonathan Tan, Jul 13, 2023
  77. Taylor BlauJul 19, 2023
  78. Jonathan TanJul 20, 2023
  79. Taylor BlauJul 26, 2023
  80. Junio C HamanoJul 13, 2023
  81. Junio C HamanoJul 13, 2023
  82. Jonathan TanJul 14, 2023
  83. 0/7 Changed path filter hash fix and version bumpJonathan Tan, Jul 20, 2023
  84. 1/7 gitformat-commit-graph: describe version 2 of BDATJonathan Tan, Jul 20, 2023
  85. 2/7 t/helper/test-read-graph.c: extract `dump_graph_info()`Jonathan Tan, Jul 20, 2023
  86. Taylor BlauJul 26, 2023
  87. 3/7 bloom.h: make `load_bloom_filter_from_graph()` publicJonathan Tan, Jul 20, 2023
  88. 5/7 t4216: test changed path filters with high bit pathsJonathan Tan, Jul 20, 2023
  89. Taylor BlauJul 26, 2023
  90. 4/7 t/helper/test-read-graph: implement `bloom-filters` modeJonathan Tan, Jul 20, 2023
  91. 6/7 repo-settings: introduce commitgraph.changedPathsVersionJonathan Tan, Jul 20, 2023
  92. 7/7 commit-graph: new filter ver. that fixes murmur3Jonathan Tan, Jul 20, 2023
  93. Junio C HamanoJul 25, 2023
  94. Junio C HamanoJul 26, 2023
  95. Taylor BlauJul 27, 2023
  96. Junio C HamanoJul 27, 2023
  97. Jonathan TanJul 27, 2023
  98. Taylor BlauJul 27, 2023
  99. Jonathan TanJul 27, 2023
  100. Taylor BlauAug 1, 2023
  101. Jonathan TanAug 1, 2023
  102. Taylor BlauAug 3, 2023
  103. Derrick StoleeAug 3, 2023
  104. Taylor BlauAug 3, 2023
  105. Junio C HamanoJul 27, 2023
  106. 0/7 Changed path filter hash fix and version bumpJonathan Tan, Aug 1, 2023
  107. 1/7 gitformat-commit-graph: describe version 2 of BDATJonathan Tan, Aug 1, 2023
  108. 2/7 t/helper/test-read-graph.c: extract `dump_graph_info()`Jonathan Tan, Aug 1, 2023
  109. 3/7 bloom.h: make `load_bloom_filter_from_graph()` publicJonathan Tan, Aug 1, 2023
  110. 4/7 t/helper/test-read-graph: implement `bloom-filters` modeJonathan Tan, Aug 1, 2023
  111. 5/7 t4216: test changed path filters with high bit pathsJonathan Tan, Aug 1, 2023
  112. 6/7 repo-settings: introduce commitgraph.changedPathsVersionJonathan Tan, Aug 1, 2023
  113. 7/7 commit-graph: new filter ver. that fixes murmur3Jonathan Tan, Aug 1, 2023
  114. Junio C HamanoAug 1, 2023
  115. Taylor BlauAug 1, 2023
  116. Junio C HamanoAug 1, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.