{"thread":{"id":"57611","subject":"[PATCH v1 0/3] trace2 output for bitmap decision path","startedAt":"2022-03-24T11:44:10Z","lastAt":"2022-07-22T06:17:30Z","messageCount":128,"participants":["Teng Long","Ævar Arnfjörð Bjarmason","Junio C Hamano","Taylor Blau","Jeff Hostetler","Eric Sunshine","tenglong.tl"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"452104","messageId":"cover.1648119652.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":null,"subject":"[PATCH v1 0/3] trace2 output for bitmap decision path","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-24T11:43:58Z","receivedAt":"2022-03-24T11:44:10Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"A Git repo can be chosen to use the normal bitmap (before MIDX) and MIDX bitmap.\n\nI recently tried to understand this part of the MIDX implementation because I found\na bug which has been discovered and repaired in community [1].\n\nI am grateful to Taylor Blau for his help and for introducing me to the testing\nmethod according to the `git rev-list --test-bitmap `.\n\nIn the process of understanding and troubleshooting by using this command, I found\nwhen the execution is failed it will output a single line of\n\"fatal: failed to load bitmap indexes\", sometimes will be more informations like\nif the bitmap file is broken, the outputs maybe contain\n\"error: Corrupted bitmap index file (wrong header)\".), but most of time are single\nline output I mentioned above. So, this brings a little obstacle for debugging and\ntroubleshooting I think, because \"failed to load bitmap indexes\" can represent\nto much informations (many causes like: midx config turn off, bitmap inexistence, etc.)\n\nTherefore, as a git repo can be chosen to use the normal bitmap (before MIDX) or\nMIDX bitmap, or they can both exist and let git make the decision. I think why not add\nsome extra informations based on TRACE2 to help for showing the bitmap decision path\nclearer and more plentiful, so when the failure occurs the user can use it to debug\naround bitmap in a quicker way.\n\nThanks.\n\nLinks:\n\t1. https://public-inbox.org/git/cover.1638991570.git.me@ttaylorr.com/)\n\nTeng Long (3):\n pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"\n pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"\n bitmap: add trace outputs during open \"bitmap\" file\n\n midx.c | 2 ++\n pack-bitmap.c | 17 +++++++++++++----\n 2 files changed, 15 insertions(+), 4 deletions(-)\n\nRange-diff against v0:\n-: ---------- > 1: 3048b4dd29 pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"\n-: ---------- > 2: 70500b6343 pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"\n-: ---------- > 3: 9912450fc1 bitmap: add trace outputs during open \"bitmap\" file\n-- \n2.35.1.579.g70500b6343.dirty\n\n"},{"id":"452105","messageId":"3048b4dd2982932fa11ba8393895fa33a00a5b58.1648119652.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1648119652.git.dyroneteng@gmail.com","subject":"[PATCH v1 1/3] pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-24T11:43:59Z","receivedAt":"2022-03-24T11:44:14Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Let's use \"ret\" value for \"return\" statement in \"open_midx_bitmap()\"\njust as the same way as int \"open_pack_bitmap()\".\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 9 ++++++---\n 1 file changed, 6 insertions(+), 3 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..931219adf0 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -494,15 +494,18 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n-\t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\tif (!open_midx_bitmap_1(bitmap_git, midx)) {\n+\t\t\tret = 0;\n+\t\t\tbreak;\n+\t\t}\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.1.579.g70500b6343.dirty\n\n"},{"id":"452106","messageId":"70500b63434f6e454631b3a8fde07c62b5adfef0.1648119652.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1648119652.git.dyroneteng@gmail.com","subject":"[PATCH v1 2/3] pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-24T11:44:00Z","receivedAt":"2022-03-24T11:44:16Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"There will be only one operant \".bitmap\" file in repo, so let's\nadd \"break\" statement in \"open_pack_bitmap()\" when looping all\nthe packs in repo.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 4 +++-\n 1 file changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 931219adf0..b1357137bf 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -484,8 +484,10 @@ static int open_pack_bitmap(struct repository *r,\n \tassert(!bitmap_git->map);\n \n \tfor (p = get_all_packs(r); p; p = p->next) {\n-\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0)\n+\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n \t\t\tret = 0;\n+\t\t\tbreak;\n+\t\t}\n \t}\n \n \treturn ret;\n-- \n2.35.1.579.g70500b6343.dirty\n\n"},{"id":"452107","messageId":"9912450fc11f869ad1b6409d7adcfe20f06c17ff.1648119652.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1648119652.git.dyroneteng@gmail.com","subject":"[PATCH v1 3/3] bitmap: add trace outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-24T11:44:01Z","receivedAt":"2022-03-24T11:44:23Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"It's supported for a Git repo to use bitmap in both normal bitmap way\nor a multi-pack-index bitmap.\n\nSometimes the debug path is not obvious, for example, when executing:\n\n\t $git rev-list --test-bitmap HEAD\n\t fatal: failed to load bitmap indexes\n\nIf we see the output like this, we are not sure about what's happened,\nbecause the cause should be :\n\n\t 1. neither normal nor midx bitmap exists\n\t 2. only midx bitmap exists but core.multipackIndex=\"false\"\n 3. core.multipackIndex=\"true\" but midx bitmap file is currupt\n\t 4. core.multipackIndex=\"true\" and no midx bitmap exists but\n\t normal bitmap file is currupt\n\t ....\n\nThese are some of the scenarios I briefly tested, but maybe there are\nothers (some scenarios is produced manually like \"currupt bitmap file\",\nbut it's not represent it's an existed bug.).\n\nTherefore, we added some TRACE2 code so that when we read the bitmap\nwe can be more clear about the decision path, such as whether it is\nworking on midx bitmap or normal bitmap, or is it simply because the\nrelated configuration is disabled. This may help with logging, user\ntroubleshooting, and development debugging.\n\nHere are some of the outputs of\n\n \"$GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\"\n\nbased on this commit:\n\n 1. core.multipackindex=\"true\" and midx bitmap exists\n\n\t17:12:50.496112 common-main.c:49 | d0 | main | version | | | | | 2.35.1.579.g70500b6343.dirty\n\t17:12:50.496141 common-main.c:50 | d0 | main | start | | 0.000306 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n\t17:12:50.496208 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n\t17:12:50.496333 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n\t17:12:50.496536 midx.c:395 | d0 | main | data | r0 | 0.000706 | 0.000706 | midx | core.multipackIndex:true\n\t17:12:50.496563 midx.c:185 | d0 | main | data | r0 | 0.000733 | 0.000733 | midx | load/num_packs:1\n\t17:12:50.496571 midx.c:186 | d0 | main | data | r0 | 0.000741 | 0.000741 | midx | load/num_objects:15\n\t17:12:50.496677 pack-revindex.c:315 | d0 | main | data | r0 | 0.000847 | 0.000847 | load_midx_re | source:midx\n\t17:12:50.496685 pack-bitmap.c:512 | d0 | main | data | r0 | 0.000856 | 0.000856 | midx | open bitmap (midx):ok\n\tBitmap v1 test (4 entries loaded)\n\tFound bitmap for 9a0a2928a280d91f5c8ded46e952add4c4296979. 64 bits / f15de25e checksum\n\t17:12:50.497016 progress.c:268 | d0 | main | region_enter | r0 | 0.001186 | | progress | label:Verifying bitmap entries\n\tVerifying bitmap entries: 100% (15/15), done.\n\t17:12:50.497179 progress.c:339 | d0 | main | data | r0 | 0.001350 | 0.000164 | progress | ..total_objects:15\n\t17:12:50.497188 progress.c:346 | d0 | main | region_leave | r0 | 0.001358 | 0.000172 | progress | label:Verifying bitmap entries\n\tOK!\n\t17:12:50.497204 git.c:718 | d0 | main | exit | | 0.001374 | | | code:0\n\t17:12:50.497213 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001384 | | | code:0\n\n 2. core.multipackindex=\"false\" and only midx bitmap exists\n\n\t17:18:38.423328 common-main.c:49 | d0 | main | version | | | | | 2.35.1.579.g70500b6343.dirty\n\t17:18:38.423357 common-main.c:50 | d0 | main | start | | 0.000263 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n\t17:18:38.423443 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n\t17:18:38.423587 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n\t17:18:38.423809 midx.c:395 | d0 | main | data | r0 | 0.000720 | 0.000720 | midx | core.multipackIndex:false\n\t17:18:38.423933 pack-bitmap.c:512 | d0 | main | data | r0 | 0.000845 | 0.000845 | midx | open bitmap (midx):failed\n\t17:18:38.423947 pack-bitmap.c:493 | d0 | main | data | r0 | 0.000858 | 0.000858 | bitmap | open bitmap (non-midx):failed\n\t17:18:38.423956 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n\tfatal: failed to load bitmap indexes\n\t17:18:38.423967 usage.c:74 | d0 | main | exit | | 0.000879 | | | code:128\n\t17:18:38.423976 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.000888 | | | code:128\n\n 3. core.multipackindex=\"false\" and only normal bitmap exists\n\n\t17:21:25.519233 common-main.c:49 | d0 | main | version | | | | | 2.35.1.579.g70500b6343.dirty\n\t17:21:25.519261 common-main.c:50 | d0 | main | start | | 0.000284 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n\t17:21:25.519335 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n\t17:21:25.519468 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n\t17:21:25.519673 midx.c:395 | d0 | main | data | r0 | 0.000701 | 0.000701 | midx | core.multipackIndex:false\n\t17:21:25.520051 pack-bitmap.c:512 | d0 | main | data | r0 | 0.001080 | 0.001080 | midx | open bitmap (midx):failed\n\t17:21:25.520070 pack-bitmap.c:493 | d0 | main | data | r0 | 0.001099 | 0.001099 | bitmap | open bitmap (non-midx):ok\n\tBitmap v1 test (4 entries loaded)\n\tFound bitmap for 9a0a2928a280d91f5c8ded46e952add4c4296979. 64 bits / f15de25e checksum\n\t17:21:25.520395 progress.c:268 | d0 | main | region_enter | r0 | 0.001423 | | progress | label:Verifying bitmap entries\n\tVerifying bitmap entries: 100% (15/15), done.\n\t17:21:25.520526 progress.c:339 | d0 | main | data | r0 | 0.001555 | 0.000132 | progress | ..total_objects:15\n\t17:21:25.520533 progress.c:346 | d0 | main | region_leave | r0 | 0.001562 | 0.000139 | progress | label:Verifying bitmap entries\n\tOK!\n\t17:21:25.520549 git.c:718 | d0 | main | exit | | 0.001578 | | | code:0\n\t17:21:25.520557 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001586 | | | code:0\n\n 4. core.multipackindex=\"false\" and both midx and normal bitmaps do not exists\n\n\t17:22:27.680275 common-main.c:49 | d0 | main | version | | | | | 2.35.1.579.g70500b6343.dirty\n\t17:22:27.680298 common-main.c:50 | d0 | main | start | | 0.000284 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n\t17:22:27.680375 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n\t17:22:27.680511 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n\t17:22:27.680718 midx.c:395 | d0 | main | data | r0 | 0.000709 | 0.000709 | midx | core.multipackIndex:false\n\t17:22:27.681085 pack-bitmap.c:512 | d0 | main | data | r0 | 0.001075 | 0.001075 | midx | open bitmap (midx):failed\n\t17:22:27.681102 pack-bitmap.c:493 | d0 | main | data | r0 | 0.001093 | 0.001093 | bitmap | open bitmap (non-midx):failed\n\t17:22:27.681110 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n\tfatal: failed to load bitmap indexes\n\t17:22:27.681120 usage.c:74 | d0 | main | exit | | 0.001112 | | | code:128\n\t17:22:27.681129 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001120 | | | code:128\n\nSigned-off-by: Teng Long \n---\n midx.c | 2 ++\n pack-bitmap.c | 4 ++++\n 2 files changed, 6 insertions(+)\n\ndiff --git a/midx.c b/midx.c\nindex 865170bad0..fda9644028 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -392,6 +392,8 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i\n \tstruct multi_pack_index *m_search;\n \n \tprepare_repo_settings(r);\n+\ttrace2_data_string(\"midx\", r, \"core.multipackIndex\",\n+\t\t\t\t\t r->settings.core_multi_pack_index ? \"true\" : \"false\");\n \tif (!r->settings.core_multi_pack_index)\n \t\treturn 0;\n \ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex b1357137bf..14cf8abebd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -490,6 +490,8 @@ static int open_pack_bitmap(struct repository *r,\n \t\t}\n \t}\n \n+\ttrace2_data_string(\"bitmap\", the_repository, \"open bitmap (non-midx)\",\n+\t\t\t\t\t ret ? \"failed\" : \"ok\");\n \treturn ret;\n }\n \n@@ -507,6 +509,8 @@ static int open_midx_bitmap(struct repository *r,\n \t\t\tbreak;\n \t\t}\n \t}\n+\ttrace2_data_string(\"midx\", the_repository, \"open bitmap (midx)\",\n+\t\t\t\t\t ret ? \"failed\" : \"ok\");\n \treturn ret;\n }\n \n-- \n2.35.1.579.g70500b6343.dirty\n\n"},{"id":"452116","messageId":"220324.867d8jo45p.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"cover.1648119652.git.dyroneteng@gmail.com","subject":"Re: [PATCH v1 0/3] trace2 output for bitmap decision path","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-24T13:22:17Z","receivedAt":"2022-03-24T13:45:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Mar 24 2022, Teng Long wrote:\n\n> A Git repo can be chosen to use the normal bitmap (before MIDX) and MIDX bitmap.\n>\n> I recently tried to understand this part of the MIDX implementation because I found\n> a bug which has been discovered and repaired in community [1].\n>\n> I am grateful to Taylor Blau for his help and for introducing me to the testing\n> method according to the `git rev-list --test-bitmap `.\n>\n> In the process of understanding and troubleshooting by using this command, I found\n> when the execution is failed it will output a single line of\n> \"fatal: failed to load bitmap indexes\", sometimes will be more informations like\n> if the bitmap file is broken, the outputs maybe contain\n> \"error: Corrupted bitmap index file (wrong header)\".), but most of time are single\n> line output I mentioned above. So, this brings a little obstacle for debugging and\n> troubleshooting I think, because \"failed to load bitmap indexes\" can represent\n> to much informations (many causes like: midx config turn off, bitmap inexistence, etc.)\n>\n> Therefore, as a git repo can be chosen to use the normal bitmap (before MIDX) or\n> MIDX bitmap, or they can both exist and let git make the decision. I think why not add\n> some extra informations based on TRACE2 to help for showing the bitmap decision path\n> clearer and more plentiful, so when the failure occurs the user can use it to debug\n> around bitmap in a quicker way.\n>\n> Thanks.\n>\n> Links:\n> \t1. https://public-inbox.org/git/cover.1638991570.git.me@ttaylorr.com/)\n>\n> Teng Long (3):\n> pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"\n> pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"\n> bitmap: add trace outputs during open \"bitmap\" file\n>\n> midx.c | 2 ++\n> pack-bitmap.c | 17 +++++++++++++----\n> 2 files changed, 15 insertions(+), 4 deletions(-)\n>\n> Range-diff against v0:\n> -: ---------- > 1: 3048b4dd29 pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"\n> -: ---------- > 2: 70500b6343 pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"\n> -: ---------- > 3: 9912450fc1 bitmap: add trace outputs during open \"bitmap\" file\n\nWas there an on-list v0 (RFC?) or is this a range-diff against nothing?\nBest not to include it until a v2 then.\n\nComments:\n\nSometimes it's better to split up patches, but I think these 3x should\nreally be squashed together. We make incremental progress to nowhere in\n1/3 and 2/3, and it all comes together in 3/3. The 1-2/3 are trivial\nenough that we can squash them in.\n\nWe then end up with this, with my comments added:\n\t\n\t midx.c | 2 ++\n\t pack-bitmap.c | 17 +++++++++++++----\n\t 2 files changed, 15 insertions(+), 4 deletions(-)\n\t\n\tdiff --git a/midx.c b/midx.c\n\tindex 865170bad05..fda96440287 100644\n\t--- a/midx.c\n\t+++ b/midx.c\n\t@@ -392,6 +392,8 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i\n\t \tstruct multi_pack_index *m_search;\n\t \n\t \tprepare_repo_settings(r);\n\t+\ttrace2_data_string(\"midx\", r, \"core.multipackIndex\",\n\t+\t\t\t\t\t r->settings.core_multi_pack_index ? \"true\" : \"false\");\n\nWeird indentation here.\n\nAlso, if we think it's a good idea to log these shouldn't it be in\nrepo_cfg_bool() in repo-settings.c, why is core.multipackIndex out of\nall in r->settings special?\n\n\t \tif (!r->settings.core_multi_pack_index)\n\t \t\treturn 0;\n\t \n\tdiff --git a/pack-bitmap.c b/pack-bitmap.c\n\tindex 97909d48da3..cac8d4a978f 100644\n\t--- a/pack-bitmap.c\n\t+++ b/pack-bitmap.c\n\t@@ -484,25 +484,34 @@ static int open_pack_bitmap(struct repository *r,\n\t \tassert(!bitmap_git->map);\n\t \n\t \tfor (p = get_all_packs(r); p; p = p->next) {\n\t-\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0)\n\t+\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n\nAside: If we end up changing this line anyway, it's OK to just change it\nto \"if (!open...\".\n\n\n\t \t\t\tret = 0;\n\t+\t\t\tbreak;\n\t+\t\t}\n\t \t}\n\t \n\t+\ttrace2_data_string(\"bitmap\", the_repository, \"open bitmap (non-midx)\",\n\t+\t\t\t\t\t ret ? \"failed\" : \"ok\");\n\t \treturn ret;\n\t }\n\t \n\t static int open_midx_bitmap(struct repository *r,\n\t \t\t\t struct bitmap_index *bitmap_git)\n\t {\n\t+\tint ret = -1;\n\t \tstruct multi_pack_index *midx;\n\t \n\t \tassert(!bitmap_git->map);\n\t \n\t \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n\t-\t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n\t-\t\t\treturn 0;\n\t+\t\tif (!open_midx_bitmap_1(bitmap_git, midx)) {\n\t+\t\t\tret = 0;\n\t+\t\t\tbreak;\n\t+\t\t}\n\t \t}\n\t-\treturn -1;\n\t+\ttrace2_data_string(\"midx\", the_repository, \"open bitmap (midx)\",\n\t+\t\t\t\t\t ret ? \"failed\" : \"ok\");\n\t+\treturn ret;\n\t }\n\t \n\t static int open_bitmap(struct repository *r,\n\nIt seems odd not to use trace2 regions for this, and to not add add this\ndata logging open_bitmap(). I came up with this on top of this when\ntesting this:\n\t\n\tdiff --git a/pack-bitmap.c b/pack-bitmap.c\n\tindex cac8d4a978f..ba71a7ea5cd 100644\n\t--- a/pack-bitmap.c\n\t+++ b/pack-bitmap.c\n\t@@ -318,11 +318,14 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n\t \n\t \tfree(idx_name);\n\t \n\t-\tif (fd < 0)\n\t+\tif (fd < 0) {\n\t+\t\t/* TODO: Log trace2_data_string() here, do we care? */\n\t \t\treturn -1;\n\t+\t}\n\t \n\t \tif (fstat(fd, &st)) {\n\t \t\tclose(fd);\n\t+\t\t/* TODO: Log trace2_data_string() here, do we care? */\n\t \t\treturn -1;\n\t \t}\n\t \n\t@@ -330,6 +333,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n\t \t\tstruct strbuf buf = STRBUF_INIT;\n\t \t\tget_midx_filename(&buf, midx->object_dir);\n\t \t\t/* ignore extra bitmap file; we can only handle one */\n\t+\t\t/* NOTE: You'll already get a warning (well, \"error\") event due to this, and it'll be in your region */\n\t \t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n\t \t\tclose(fd);\n\t \t\tstrbuf_release(&buf);\n\t@@ -344,9 +348,11 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n\t \tclose(fd);\n\t \n\t \tif (load_bitmap_header(bitmap_git) < 0)\n\t+\t\t/* TODO: Add trace2_data_string() or warning/error here? */\n\t \t\tgoto cleanup;\n\t \n\t \tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n\t+\t\t/* TODO: Add trace2_data_string() or warning/error here? */\n\t \t\tgoto cleanup;\n\t \n\t \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n\t@@ -479,49 +485,44 @@ static int open_pack_bitmap(struct repository *r,\n\t \t\t\t struct bitmap_index *bitmap_git)\n\t {\n\t \tstruct packed_git *p;\n\t-\tint ret = -1;\n\t-\n\t-\tassert(!bitmap_git->map);\n\t \n\t \tfor (p = get_all_packs(r); p; p = p->next) {\n\t \t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n\t-\t\t\tret = 0;\n\t-\t\t\tbreak;\n\t+\t\t\treturn 0;\n\t \t\t}\n\t \t}\n\t-\n\t-\ttrace2_data_string(\"bitmap\", the_repository, \"open bitmap (non-midx)\",\n\t-\t\t\t\t\t ret ? \"failed\" : \"ok\");\n\t-\treturn ret;\n\t+\treturn -1;\n\t }\n\t \n\t static int open_midx_bitmap(struct repository *r,\n\t \t\t\t struct bitmap_index *bitmap_git)\n\t {\n\t-\tint ret = -1;\n\t \tstruct multi_pack_index *midx;\n\t \n\t-\tassert(!bitmap_git->map);\n\t-\n\t \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n\t \t\tif (!open_midx_bitmap_1(bitmap_git, midx)) {\n\t-\t\t\tret = 0;\n\t-\t\t\tbreak;\n\t+\t\t\treturn 0;\n\t \t\t}\n\t \t}\n\t-\ttrace2_data_string(\"midx\", the_repository, \"open bitmap (midx)\",\n\t-\t\t\t\t\t ret ? \"failed\" : \"ok\");\n\t-\treturn ret;\n\t+\treturn -1;\n\t }\n\t \n\t static int open_bitmap(struct repository *r,\n\t \t\t struct bitmap_index *bitmap_git)\n\t {\n\t+\tint ret;\n\t+\n\t \tassert(!bitmap_git->map);\n\t \n\t-\tif (!open_midx_bitmap(r, bitmap_git))\n\t-\t\treturn 0;\n\t-\treturn open_pack_bitmap(r, bitmap_git);\n\t+\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n\t+\tif (!open_midx_bitmap(r, bitmap_git)) {\n\t+\t\tret = 0;\n\t+\t\tgoto done;\n\t+\t}\n\t+\tret = open_pack_bitmap(r, bitmap_git);\n\t+done:\n\t+\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n\t+\treturn ret;\n\t }\n\t \n\t struct bitmap_index *prepare_bitmap_git(struct repository *r)\n\nI.e. surely you just want to create a region, and if you care enough to\nlog failure shouldn't we log that in open_midx_bitmap_1() if we care,\nand perhaps error() there instead of silently returning -1?\n"},{"id":"452211","messageId":"xmqq7d8ji48t.fsf@gitster.g","threadId":"57611","inReplyTo":"70500b63434f6e454631b3a8fde07c62b5adfef0.1648119652.git.dyroneteng@gmail.com","subject":"Re: [PATCH v1 2/3] pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-24T18:40:02Z","receivedAt":"2022-03-24T18:40:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> There will be only one operant \".bitmap\" file in repo, so let's\n> add \"break\" statement in \"open_pack_bitmap()\" when looping all\n> the packs in repo.\n\nGoogling \"operant\" only gives a psychology term X-<.\n\n : behavior (such as bar pressing by a rat to obtain food) that\n operates on the environment to produce rewarding and reinforcing\n effects.\n\nI do not quite get this. We expect that there may be more than one\n.pack in the object store, and that is why we iterate over the packs\nget_all_packs() gives us. Then for each of these packs, we try to\nopen pack bitmap.\n\nAh, are you referring to the current limitation that we can have\nonly one .bitmap file, even when there are many packs in the object\nstore?\n\nI wonder if that limitation is something we need to hardcode here.\n\nIOW, when we ever extend the feature to allow more than one, we may\nhave to extend the \"struct bitmap_index\" so that we can have bitmap\ninfo from more than one on-disk files, and the implementation of\nopen_pack_bitmap_1() (which is \"read bitmap for this one pack\" as\nopposed to \"read bitmpa for this repository\" which is this function)\nwould certainly have to change, but would we need to revert this\nchange to the function? If we do not omit this patch, i.e. even if\nwe know there can be only one pack that may have a corresponding\nbitmap but if the code does not know it and has to try git_open()\nonly to see it fail, how much wasted effort are we talking about?\n\n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 4 +++-\n> 1 file changed, 3 insertions(+), 1 deletion(-)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 931219adf0..b1357137bf 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -484,8 +484,10 @@ static int open_pack_bitmap(struct repository *r,\n> \tassert(!bitmap_git->map);\n> \n> \tfor (p = get_all_packs(r); p; p = p->next) {\n> -\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0)\n> +\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n> \t\t\tret = 0;\n> +\t\t\tbreak;\n> +\t\t}\n> \t}\n> \n> \treturn ret;\n"},{"id":"452212","messageId":"xmqqy20zgpkq.fsf@gitster.g","threadId":"57611","inReplyTo":"9912450fc11f869ad1b6409d7adcfe20f06c17ff.1648119652.git.dyroneteng@gmail.com","subject":"Re: [PATCH v1 3/3] bitmap: add trace outputs during open \"bitmap\" file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-24T18:42:13Z","receivedAt":"2022-03-24T18:42:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> It's supported for a Git repo to use bitmap in both normal bitmap way\n> or a multi-pack-index bitmap.\n\nThis step out of 3 patches does match what was sold on the cover\nletter, and it does not seem to even need the other two.\n\n> Signed-off-by: Teng Long \n> ---\n> midx.c | 2 ++\n> pack-bitmap.c | 4 ++++\n> 2 files changed, 6 insertions(+)\n>\n> diff --git a/midx.c b/midx.c\n> index 865170bad0..fda9644028 100644\n> --- a/midx.c\n> +++ b/midx.c\n> @@ -392,6 +392,8 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i\n> \tstruct multi_pack_index *m_search;\n> \n> \tprepare_repo_settings(r);\n> +\ttrace2_data_string(\"midx\", r, \"core.multipackIndex\",\n> +\t\t\t\t\t r->settings.core_multi_pack_index ? \"true\" : \"false\");\n\nThat's an overly deep contination line.\n\n> \tif (!r->settings.core_multi_pack_index)\n> \t\treturn 0;\n> \n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index b1357137bf..14cf8abebd 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -490,6 +490,8 @@ static int open_pack_bitmap(struct repository *r,\n> \t\t}\n> \t}\n> \n> +\ttrace2_data_string(\"bitmap\", the_repository, \"open bitmap (non-midx)\",\n> +\t\t\t\t\t ret ? \"failed\" : \"ok\");\n\nLikewise.\n\n> \treturn ret;\n> }\n> \n> @@ -507,6 +509,8 @@ static int open_midx_bitmap(struct repository *r,\n> \t\t\tbreak;\n> \t\t}\n> \t}\n> +\ttrace2_data_string(\"midx\", the_repository, \"open bitmap (midx)\",\n> +\t\t\t\t\t ret ? \"failed\" : \"ok\");\n\nLikewise.\n\n> \treturn ret;\n> }\n"},{"id":"452216","messageId":"YjzAbSJfuu06smnD@nand.local","threadId":"57611","inReplyTo":"70500b63434f6e454631b3a8fde07c62b5adfef0.1648119652.git.dyroneteng@gmail.com","subject":"Re: [PATCH v1 2/3] pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-03-24T19:03:09Z","receivedAt":"2022-03-24T19:03:16Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Mar 24, 2022 at 07:44:00PM +0800, Teng Long wrote:\n> There will be only one operant \".bitmap\" file in repo, so let's\n> add \"break\" statement in \"open_pack_bitmap()\" when looping all\n> the packs in repo.\n>\n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 4 +++-\n> 1 file changed, 3 insertions(+), 1 deletion(-)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 931219adf0..b1357137bf 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -484,8 +484,10 @@ static int open_pack_bitmap(struct repository *r,\n> \tassert(!bitmap_git->map);\n>\n> \tfor (p = get_all_packs(r); p; p = p->next) {\n> -\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0)\n> +\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n> \t\t\tret = 0;\n> +\t\t\tbreak;\n\nThe lack of a break here is intentional, I think, since having more than\none bitmap of the same kind in a repository is an error.\n\n(This is behavior we inherited from the pre-MIDX bitmap days, when\nhaving more than one pack bitmap caused Git to signal an error, since it\ncould only use the results from a single bitmap).\n\nYou can see in pack-bitmap.c::open_pack_bitmap_1() that we have a\ncondition saying:\n\n if (bitmap_git->pack || bitmap_git->midx) {\n /* ignore extra bitmap file; we can only handle one */\n warning(\"...\")\n close(fd;)\n return -1;\n }\n\nWe do want to call that open_pack_bitmap_1() function on every pack we\nknow about to make sure that one and only one of them corresponds to a\n.bitmap.\n\nThanks,\nTaylor\n"},{"id":"452218","messageId":"YjzBRjrYQWIpdsk3@nand.local","threadId":"57611","inReplyTo":"xmqq7d8ji48t.fsf@gitster.g","subject":"Re: [PATCH v1 2/3] pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-03-24T19:06:46Z","receivedAt":"2022-03-24T19:06:50Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Mar 24, 2022 at 11:40:02AM -0700, Junio C Hamano wrote:\n> Teng Long writes:\n>\n> > There will be only one operant \".bitmap\" file in repo, so let's\n> > add \"break\" statement in \"open_pack_bitmap()\" when looping all\n> > the packs in repo.\n>\n> Googling \"operant\" only gives a psychology term X-<.\n>\n> : behavior (such as bar pressing by a rat to obtain food) that\n> operates on the environment to produce rewarding and reinforcing\n> effects.\n\nI suspect that the author meant \"operating\", which I would probably have\nwritten as \"Git will only use information from a single .bitmap file in\nthe repository\" or similar.\n\nBut...\n\n> I do not quite get this. We expect that there may be more than one\n> .pack in the object store, and that is why we iterate over the packs\n> get_all_packs() gives us. Then for each of these packs, we try to\n> open pack bitmap.\n>\n> Ah, are you referring to the current limitation that we can have\n> only one .bitmap file, even when there are many packs in the object\n> store?\n>\n> I wonder if that limitation is something we need to hardcode here.\n\n...you're exactly right: if we see more than one pack bitmap in the\nrepository, we can and should warn there, since we'll only ever *use*\ninformation from a single bitmap at a time.\n\nThat's the behavior before this patch which is guaranteed by t5310.74.\nThis patch causes that test to break, and I would suggest that is\ntelling us to drop this part of the series.\n\nThanks,\nTaylor\n"},{"id":"452219","messageId":"YjzCTLLDCby+kJrZ@nand.local","threadId":"57611","inReplyTo":"3048b4dd2982932fa11ba8393895fa33a00a5b58.1648119652.git.dyroneteng@gmail.com","subject":"Re: [PATCH v1 1/3] pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-03-24T19:11:08Z","receivedAt":"2022-03-24T19:11:12Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Mar 24, 2022 at 07:43:59PM +0800, Teng Long wrote:\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 9c666cdb8b..931219adf0 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -494,15 +494,18 @@ static int open_pack_bitmap(struct repository *r,\n> static int open_midx_bitmap(struct repository *r,\n> \t\t\t struct bitmap_index *bitmap_git)\n> {\n> +\tint ret = -1;\n> \tstruct multi_pack_index *midx;\n>\n> \tassert(!bitmap_git->map);\n>\n> \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n> -\t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n> -\t\t\treturn 0;\n> +\t\tif (!open_midx_bitmap_1(bitmap_git, midx)) {\n> +\t\t\tret = 0;\n> +\t\t\tbreak;\n> +\t\t}\n> \t}\n> -\treturn -1;\n> +\treturn ret;\n\nThis all looks like good clean-up to me, and it indeed preserves the\nbehavior before and after this patch is applied.\n\nBut thinking about some of my comments on patch 2/3 here, I think that\nwe don't want to break out of that loop until we have visited both the\nMIDX in our repository, as well as any alternates (along with _their_\nalternates, recursively).\n\nThat _is_ a behavior change with respect to the existing implementation\non master, but I think that what's on master is wrong to stop after\nlooking at the first MIDX bitmap. At least, it's wrong in the same sense\nof: \"we will only load _one_ of these MIDX bitmaps, so if there is more\nthan one to choose from, the caller is mistaken\".\n\nI think instead we'd want to do something like this on top:\n\n--- 8< ---\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 410020c4d3..0c6640b0f6 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -500,10 +500,8 @@ static int open_midx_bitmap(struct repository *r,\n \tassert(!bitmap_git->map);\n\n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n-\t\tif (!open_midx_bitmap_1(bitmap_git, midx)) {\n+\t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n \t\t\tret = 0;\n-\t\t\tbreak;\n-\t\t}\n \t}\n \treturn ret;\n }\n\n--- >8 ---\n\nThanks,\nTaylor\n"},{"id":"452459","messageId":"20220328075907.75529-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"YjzCTLLDCby+kJrZ@nand.local","subject":"Re: [PATCH v1 1/3] pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-28T07:59:07Z","receivedAt":"2022-03-28T07:59:13Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Thu, 24 Mar 2022 15:11:08 -0400, Taylor Blau wrote:\n\n\n> But thinking about some of my comments on patch 2/3 here, I think that\n> we don't want to break out of that loop until we have visited both the\n> MIDX in our repository, as well as any alternates (along with _their_\n> alternates, recursively). \n>\n> That _is_ a behavior change with respect to the existing implementation\n> on master, but I think that what's on master is wrong to stop after\n> looking at the first MIDX bitmap. At least, it's wrong in the same sense\n> of: \"we will only load _one_ of these MIDX bitmaps, so if there is more\n> than one to choose from, the caller is mistaken\".\n\nI'm a little wondering that what's the practial meaning for _ do not _\nstop after looking at the first MIDX bitmap? \n\nAlthough all MIDX bitmap are scanned, only one of them will eventually work\n, and there seems to be no guarantee that the last MIDX that works is the\nmost appropriate one? (with the same comfusion applies to non-MIDX\nbitmap's behavour...)\n\nThanks.\n"},{"id":"452544","messageId":"20220329024949.62091-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"YjzAbSJfuu06smnD@nand.local","subject":"Re: [PATCH v1 2/3] pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-29T02:49:49Z","receivedAt":"2022-03-29T02:49:57Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Thu, 24 Mar 2022 15:03:09 -0400, Taylor Blau wrote:\n\n> The lack of a break here is intentional, I think, since having more than\n> one bitmap of the same kind in a repository is an error.\n\nYeah. I quite sure that it's intentional now.\n\nThanks for your explanations.\n\n> (This is behavior we inherited from the pre-MIDX bitmap days, when\n> having more than one pack bitmap caused Git to signal an error, since it\n> could only use the results from a single bitmap).\n> \n> You can see in pack-bitmap.c::open_pack_bitmap_1() that we have a\n> condition saying:\n> \n> if (bitmap_git->pack || bitmap_git->midx) {\n> /* ignore extra bitmap file; we can only handle one */\n> warning(\"...\")\n> close(fd;)\n> return -1;\n> }\n> \n> We do want to call that open_pack_bitmap_1() function on every pack we\n> know about to make sure that one and only one of them corresponds to a\n> .bitmap.\n\nYeah, the condition and \"warning\" make it clear to me which is if already\nexists a bitmap of the pack or MIDX is ready, we will give warnings and just\nlet it fail (return -1 means a return of NULL in \"prepare_bitmap_git()\", and\nwill then die() in usage cases I found).\n\nIn addition of above, I had a question that why we need \"bitmap_git->midx\"\nin the condition? Because here in \"open_pack_bitmap_1()\" we intent to open the \nnon-midx-bitmap and it's after we deal with \"open_midx_bitmap()\" in \"open_bitmap()\":\n\nstatic int open_bitmap(struct repository *r,\n struct bitmap_index *bitmap_git)\n{\t\t\t \n\tassert(!bitmap_git->map);\n\n\tif (!open_midx_bitmap(r, bitmap_git))\n\t return 0;\n\t return open_pack_bitmap(r, bitmap_git);\n}\n\nSo if I understood correct, maybe we can made condition of \"bitmap_git->midx\" a little\nearlier so that we can avoid to open every packfile, maybe it's like:\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..38f53b8f1c 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -483,11 +483,12 @@ static int open_pack_bitmap(struct repository *r,\n \n assert(!bitmap_git->map);\n \n- for (p = get_all_packs(r); p; p = p->next) {\n- if (open_pack_bitmap_1(bitmap_git, p) == 0)\n- ret = 0;\n+ if (!bitmap_git->midx) {\n+ for (p = get_all_packs(r); p; p = p->next) {\n+ if (open_pack_bitmap_1(bitmap_git, p) == 0)\n+ ret = 0;\n+ }\n }\n-\n return ret;\n }\n\n\nThanks.\n"},{"id":"452546","messageId":"20220329073851.80702-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220324.867d8jo45p.gmgdl@evledraar.gmail.com","subject":"Teng Long","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-29T07:38:51Z","receivedAt":"2022-03-29T07:39:05Z","isPatch":false,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\n> Was there an on-list v0 (RFC?) or is this a range-diff against nothing?\n> Best not to include it until a v2 then.\n\nMy careless, there's no RFC, actually this patch should begin with v0 and\nwithout range-diff.\n\n\n> Sometimes it's better to split up patches, but I think these 3x should\n> really be squashed together. We make incremental progress to nowhere in\n> 1/3 and 2/3, and it all comes together in 3/3. The 1-2/3 are trivial\n> enough that we can squash them in.\n\nSure.\n\n>\t+\ttrace2_data_string(\"midx\", r, \"core.multipackIndex\",\n>\t+\t\t\t\t\t\t r->settings.core_multi_pack_index ? \"true\" : \"false\");\n>\n> Weird indentation here.\n\nWill fix.\n\n> I.e. surely you just want to create a region, and if you care enough to\n> log failure shouldn't we log that in open_midx_bitmap_1() if we care,\n\nActually, I just want to use \"trace2_data_string()\" at first because which informations I\nwant to append is not so many, or does \"create a region\" a preferred principle for using\nTRACE2 APIs?\n\n> and perhaps error() there instead of silently returning -1?\n\nI think so, after I checked function \"error_builtin()\" and there is a \"trace2_cmd_error_va()\"\nusage in it which is already support to print some debug infos using TRACE2 envs.\n\nThanks.\n"},{"id":"452547","messageId":"220329.86pmm52kzf.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"20220329073851.80702-1-dyroneteng@gmail.com","subject":"Re: Teng Long","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-29T08:54:25Z","receivedAt":"2022-03-29T08:58:54Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Mar 29 2022, Teng Long wrote:\n\n>> Was there an on-list v0 (RFC?) or is this a range-diff against nothing?\n>> Best not to include it until a v2 then.\n>\n> My careless, there's no RFC, actually this patch should begin with v0 and\n> without range-diff.\n\nNo worries.\n\n>> Sometimes it's better to split up patches, but I think these 3x should\n>> really be squashed together. We make incremental progress to nowhere in\n>> 1/3 and 2/3, and it all comes together in 3/3. The 1-2/3 are trivial\n>> enough that we can squash them in.\n>\n> Sure.\n>\n>>\t+\ttrace2_data_string(\"midx\", r, \"core.multipackIndex\",\n>>\t+\t\t\t\t\t\t r->settings.core_multi_pack_index ? \"true\" : \"false\");\n>>\n>> Weird indentation here.\n>\n> Will fix.\n\n*Nod*\n\n>> I.e. surely you just want to create a region, and if you care enough to\n>> log failure shouldn't we log that in open_midx_bitmap_1() if we care,\n>\n> Actually, I just want to use \"trace2_data_string()\" at first because which informations I\n> want to append is not so many, or does \"create a region\" a preferred principle for using\n> TRACE2 APIs?\n\nI think a begin/end region gives you everything you want here and more, i.e.:\n\n * We'll get start/end times on this (potentially expensive?) operation\n automatically.\n\n * You just log a \"failed\", but as the RFC-on-top shows we can just log\n the specific reason instead, either via error(), warning() or perhaps\n even trace2_data_string(). because it's within a region it'll be\n clear from the data what failed.\n\nB.t.w. I noticed after sending that that this ad-hoc patch fails the\ntests, it passed the one bitmap test I hacked it up against.\n\nIt was just there to make the point/demo that it looked easier to do\nthis one function call above where you were adding this.\n\n>> and perhaps error() there instead of silently returning -1?\n>\n> I think so, after I checked function \"error_builtin()\" and there is a \"trace2_cmd_error_va()\"\n> usage in it which is already support to print some debug infos using TRACE2 envs.\n\nExactly.\n"},{"id":"452636","messageId":"YkPCxRDelmwpdey3@nand.local","threadId":"57611","inReplyTo":"20220328075907.75529-1-dyroneteng@gmail.com","subject":"Re: [PATCH v1 1/3] pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-03-30T02:39:01Z","receivedAt":"2022-03-30T02:39:05Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Mar 28, 2022 at 03:59:07PM +0800, Teng Long wrote:\n> Although all MIDX bitmap are scanned, only one of them will eventually work\n> , and there seems to be no guarantee that the last MIDX that works is the\n> most appropriate one? (with the same comfusion applies to non-MIDX\n> bitmap's behavour...)\n\nI think your \"which is the most appropriate one?\" exactly highlights the\nproblem. Since Git can only handle one .bitmap file at a time, we should\ntell the user when we found more than one candidate.\n\nIn this case, they may be expecting or depending on a particular choice\nfor performance reasons, and so may be rather surprised or upset if Git\nchose a different available .bitmap.\n\nThe one exception to this is when we have both a single- and multi-pack\nbitmap available, in which case it almost always makes more sense to\npick the multi-pack bitmap. So we don't complain in this case (i.e., if\nopen_midx_bitmap() succeeds, we immediately return success from\nopen_bitmap() and proceed to have prepare_bitmap_git() call\nload_bitmap() on the previous result without even bothering to check for\npack bitmaps).\n\nThanks,\nTaylor\n"},{"id":"452638","messageId":"YkPGq0mDL4NG6D1o@nand.local","threadId":"57611","inReplyTo":"20220329024949.62091-1-dyroneteng@gmail.com","subject":"Re: [PATCH v1 2/3] pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-03-30T02:55:39Z","receivedAt":"2022-03-30T02:55:44Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Mar 29, 2022 at 10:49:49AM +0800, Teng Long wrote:\n> Yeah, the condition and \"warning\" make it clear to me which is if\n> already exists a bitmap of the pack or MIDX is ready, we will give\n> warnings and just let it fail (return -1 means a return of NULL in\n> \"prepare_bitmap_git()\", and will then die() in usage cases I found).\n>\n> In addition of above, I had a question that why we need\n> \"bitmap_git->midx\" in the condition? Because here in\n> \"open_pack_bitmap_1()\" we intent to open the non-midx-bitmap and it's\n> after we deal with \"open_midx_bitmap()\" in \"open_bitmap()\":\n\nYou're right; open_pack_bitmap_1() doesn't need to care about whether or\nnot bitmap_git->midx is or isn't non-NULL, since:\n\n - if we did open a MIDX bitmap (which we will always attempt first\n before trying to open single-pack bitmaps), then we won't even\n bother to call open_pack_bitmap() at all.\n\n - if we _do_ end up within open_pack_bitmap_1(), then we _know_ that\n no MIDX bitmap could be found/opened, so there is no need to check\n in that case, either.\n\nSo I think we realistically could do something like:\n\n--- 8< ---\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 97909d48da..6e7c89826d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -387,3 +387,3 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n\n-\tif (bitmap_git->pack || bitmap_git->midx) {\n+\tif (bitmap_git->pack) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n\n--- >8 ---\n\n...but having the conditional there from the pre-image doesn't hurt,\neither, and it makes the error clearer in case of an accidental\nregression where we start looking for single-pack bitmaps after\nsuccessfully opening a multi-pack one.\n\n> static int open_bitmap(struct repository *r,\n> struct bitmap_index *bitmap_git)\n> {\n> \tassert(!bitmap_git->map);\n>\n> \tif (!open_midx_bitmap(r, bitmap_git))\n> \t return 0;\n> \t return open_pack_bitmap(r, bitmap_git);\n> }\n>\n> So if I understood correct, maybe we can made condition of \"bitmap_git->midx\" a little\n> earlier so that we can avoid to open every packfile, maybe it's like:\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 9c666cdb8b..38f53b8f1c 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -483,11 +483,12 @@ static int open_pack_bitmap(struct repository *r,\n>\n> assert(!bitmap_git->map);\n>\n> - for (p = get_all_packs(r); p; p = p->next) {\n> - if (open_pack_bitmap_1(bitmap_git, p) == 0)\n> - ret = 0;\n> + if (!bitmap_git->midx) {\n> + for (p = get_all_packs(r); p; p = p->next) {\n> + if (open_pack_bitmap_1(bitmap_git, p) == 0)\n> + ret = 0;\n> + }\n> }\n> -\n> return ret;\n> }\n\nThis shouldn't be necessary, since we don't bother calling\nopen_pack_bitmap() at all if open_midx_bitmap() returns success. In\nother words, based on the way that open_bitmap() (which is the caller\nfor both of these functions) is written, we know that once we're in\nopen_pack_bitmap(), that `bitmap_git->midx` is definitely NULL, which\nmakes this change a noop.\n\nThanks,\nTaylor\n"},{"id":"452659","messageId":"20220330073230.61358-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"YkPGq0mDL4NG6D1o@nand.local","subject":"Teng Long","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-03-30T07:32:30Z","receivedAt":"2022-03-30T07:33:07Z","isPatch":false,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Tue, 29 Mar 2022 22:55:39 -0400, Taylor Blau wrote:\n\n> You're right; open_pack_bitmap_1() doesn't need to care about whether or\n> not bitmap_git->midx is or isn't non-NULL, since:\n> \n> - if we did open a MIDX bitmap (which we will always attempt first\n> before trying to open single-pack bitmaps), then we won't even\n> bother to call open_pack_bitmap() at all.\n> \n> - if we _do_ end up within open_pack_bitmap_1(), then we _know_ that\n> no MIDX bitmap could be found/opened, so there is no need to check\n> in that case, either.\n> \n> So I think we realistically could do something like:\n> \n> --- 8< ---\n> \n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 97909d48da..6e7c89826d 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -387,3 +387,3 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \n> - if (bitmap_git->pack || bitmap_git->midx) {\n> + if (bitmap_git->pack) {\n> /* ignore extra bitmap file; we can only handle one */\n> \n> --- >8 ---\n> ...but having the conditional there from the pre-image doesn't hurt,\n> either, and it makes the error clearer in case of an accidental\n> regression where we start looking for single-pack bitmaps after\n> successfully opening a multi-pack one.\n\nI agree with the \"accidental regression\", it's a protection without\nany disadvantages so far. So, if we don't remove the \"bitmap_git->midx\"\ncondition for some robust reason, then I think maybe we can let the\nwarning more detailed if the \"accident\" happens, like:\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 91a6be358d..e64a85bc59 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -333,7 +333,15 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n struct strbuf buf = STRBUF_INIT;\n get_midx_filename(&buf, midx->object_dir);\n /* ignore extra bitmap file; we can only handle one */\n- warning(\"ignoring extra bitmap file: %s\", buf.buf);\n+ warning(\"%signoring extra bitmap file: %s\",\n+ bitmap_git->pack ?\n+ xstrfmt(\"A non-MIDX bitmap has been opened: %s, \",\n+ bitmap_git->pack->pack_name) :\n+ bitmap_git->midx ?\n+ xstrfmt(\"A MIDX bitmap has been opened : %s, \",\n+ midx_bitmap_filename(bitmap_git->midx)) :\n+ \"\",\n+ buf.buf);\n close(fd);\n strbuf_release(&buf);\n return -1;\n@@ -398,9 +406,17 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n return -1;\n }\n \n- if (bitmap_git->pack) {\n+ if (bitmap_git->pack || bitmap_git->midx) {\n /* ignore extra bitmap file; we can only handle one */\n- warning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+ warning(\"%signoring extra bitmap file: %s\",\n+ bitmap_git->pack ?\n+ xstrfmt(\"A non-MIDX bitmap has been opened: %s, \",\n+ bitmap_git->pack->pack_name) :\n+ bitmap_git->midx ?\n+ xstrfmt(\"A MIDX bitmap has been opened : %s, \",\n+ midx_bitmap_filename(bitmap_git->midx)) :\n+ \"\",\n+ packfile->pack_name);\n close(fd);\n return -1;\n }\ndiff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh\nindex f775fc1ce6..a962f6861f 100755\n--- a/t/t5310-pack-bitmaps.sh\n+++ b/t/t5310-pack-bitmaps.sh\n@@ -421,6 +421,7 @@ test_expect_success 'complains about multiple pack bitmaps' '\n test_line_count = 2 bitmaps &&\n \n git rev-list --use-bitmap-index HEAD 2>err &&\n+ grep \"A non-MIDX bitmap has been opened\" err &&\n grep \"ignoring extra bitmap file\" err\n )\n\n\nDoes that make sense to you? \n\nAnd also I look codes in \"open_midx_bitmap_1()\", it's really nice for me to read and\nunderstand, but the \"char *idx_name = ...\" declaration made me a litte confusing at\nthe beginning, I think maybe it's better to name it as \"bitmap_name\" for it, I know\nbitmap is also a kind of index structure for using, but when I saw \"idx\", it brings\nme to the \".idx\" or \"multi-pack-index\" like files naturally in git context, of course\nthis is not a problem and maybe it's just me.\n"},{"id":"452662","messageId":"220330.868rsrpohm.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"20220330073230.61358-1-dyroneteng@gmail.com","subject":"Re: Teng Long","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-30T13:17:44Z","receivedAt":"2022-03-30T13:19:08Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Mar 30 2022, Teng Long wrote:\n\n> On Tue, 29 Mar 2022 22:55:39 -0400, Taylor Blau wrote:\n>\n>> You're right; open_pack_bitmap_1() doesn't need to care about whether or\n>> not bitmap_git->midx is or isn't non-NULL, since:\n>> \n>> - if we did open a MIDX bitmap (which we will always attempt first\n>> before trying to open single-pack bitmaps), then we won't even\n>> bother to call open_pack_bitmap() at all.\n>> \n>> - if we _do_ end up within open_pack_bitmap_1(), then we _know_ that\n>> no MIDX bitmap could be found/opened, so there is no need to check\n>> in that case, either.\n>> \n>> So I think we realistically could do something like:\n>> \n>> --- 8< ---\n>> \n>> diff --git a/pack-bitmap.c b/pack-bitmap.c\n>> index 97909d48da..6e7c89826d 100644\n>> --- a/pack-bitmap.c\n>> +++ b/pack-bitmap.c\n>> @@ -387,3 +387,3 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n>> \n>> - if (bitmap_git->pack || bitmap_git->midx) {\n>> + if (bitmap_git->pack) {\n>> /* ignore extra bitmap file; we can only handle one */\n>> \n>> --- >8 ---\n>> ...but having the conditional there from the pre-image doesn't hurt,\n>> either, and it makes the error clearer in case of an accidental\n>> regression where we start looking for single-pack bitmaps after\n>> successfully opening a multi-pack one.\n>\n> I agree with the \"accidental regression\", it's a protection without\n> any disadvantages so far. So, if we don't remove the \"bitmap_git->midx\"\n> condition for some robust reason, then I think maybe we can let the\n> warning more detailed if the \"accident\" happens, like:\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 91a6be358d..e64a85bc59 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -333,7 +333,15 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> struct strbuf buf = STRBUF_INIT;\n> get_midx_filename(&buf, midx->object_dir);\n> /* ignore extra bitmap file; we can only handle one */\n> - warning(\"ignoring extra bitmap file: %s\", buf.buf);\n> + warning(\"%signoring extra bitmap file: %s\",\n> + bitmap_git->pack ?\n> + xstrfmt(\"A non-MIDX bitmap has been opened: %s, \",\n> + bitmap_git->pack->pack_name) :\n> + bitmap_git->midx ?\n> + xstrfmt(\"A MIDX bitmap has been opened : %s, \",\n> + midx_bitmap_filename(bitmap_git->midx)) :\n> + \"\",\n> + buf.buf);\n> close(fd);\n\nAside from any bitmap-specific issues, let's not compose messages like\nthat via concat, it makes them harder to translate.\n\nIn this case the original message doesn't have _(), but looking at the\ncontext that seems to be a simple omission.\n\nIt's more verbose, but it's better to split this out into 3x warning()\ncalls.\n\nIt also has the advantage of not leaking memory from the xstrfmt().\n"},{"id":"454071","messageId":"cover.1650547400.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1648119652.git.dyroneteng@gmail.com","subject":"[PATCH v2 0/5] trace2 output for bitmap decision path","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-04-21T13:26:35Z","receivedAt":"2022-04-21T13:27:03Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Sorry for the late patch. Main diffs from v1:\n\n1. Add a new commit to rename \"idx_name\" and this may clear up\nsome misconceptions about the naming\n\n2. Add the warnings more detailed when opening bitmap and also\nlet the texts support translation.\n\n3. using error() instead of silently returning -1 when opening\nbitmap.\n\n4. Continue looping when first MIDX bitmap is found.\n\nThanks.\n\nTeng Long (5):\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: make warnings more detailed when opening bitmap\n bitmap: add trace2 outputs during open \"bitmap\" file\n pack-bitmap.c: using error() instead of silently returning -1\n\n pack-bitmap.c | 73 ++++++++++++++++++++++++++++-------------\n repo-settings.c | 1 +\n t/t5310-pack-bitmaps.sh | 3 +-\n 3 files changed, 54 insertions(+), 23 deletions(-)\n\nRange-diff against v1:\n1: 3048b4dd29 ! 1: 1bfd2fb6ab pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"\n @@ Metadata\n Author: Teng Long \n \n ## Commit message ##\n - pack-bitmap.c: use \"ret\" in \"open_midx_bitmap()\"\n + pack-bitmap.c: continue looping when first MIDX bitmap is found\n \n - Let's use \"ret\" value for \"return\" statement in \"open_midx_bitmap()\"\n - just as the same way as int \"open_pack_bitmap()\".\n + In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\n + the first one has been found, then will break out by a \"return\"\n + directly.\n \n + But actually, it's better to don't stop the loop until we have visited\n + both the MIDX in our repository, as well as any alternates (along with\n + _their_ alternates, recursively).\n +\n + The discussion link of community:\n +\n + https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n +\n + Helped-by: Taylor Blau \n Signed-off-by: Teng Long \n \n ## pack-bitmap.c ##\n @@ pack-bitmap.c: static int open_pack_bitmap(struct repository *r,\n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n --\t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n + \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n -\t\t\treturn 0;\n -+\t\tif (!open_midx_bitmap_1(bitmap_git, midx)) {\n +\t\t\tret = 0;\n -+\t\t\tbreak;\n -+\t\t}\n \t}\n -\treturn -1;\n +\treturn ret;\n2: 70500b6343 < -: ---------- pack-bitmap.c: add \"break\" statement in \"open_pack_bitmap()\"\n3: 9912450fc1 < -: ---------- bitmap: add trace outputs during open \"bitmap\" file\n-: ---------- > 2: 1fff3b3ca7 pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n-: ---------- > 3: 5a8f5afccf pack-bitmap.c: make warnings more detailed when opening bitmap\n-: ---------- > 4: 2016ef2e34 bitmap: add trace2 outputs during open \"bitmap\" file\n-: ---------- > 5: 1a169d7b5e pack-bitmap.c: using error() instead of silently returning -1\n-- \n2.35.1.583.g30faa5f068\n\n"},{"id":"454072","messageId":"1bfd2fb6ab01af689fc6495e2f32d3f64c19f4b7.1650547400.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1650547400.git.dyroneteng@gmail.com","subject":"[PATCH v2 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-04-21T13:26:36Z","receivedAt":"2022-04-21T13:27:06Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to don't stop the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..112c2b12c6 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -494,15 +494,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.1.583.g30faa5f068\n\n"},{"id":"454073","messageId":"1fff3b3ca76d0b5e026270fdf746c55b582405d5.1650547400.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1650547400.git.dyroneteng@gmail.com","subject":"[PATCH v2 2/5] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-04-21T13:26:37Z","receivedAt":"2022-04-21T13:27:10Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index\n, let's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 112c2b12c6..f13a6bfe3a 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -313,10 +313,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -368,14 +368,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.1.583.g30faa5f068\n\n"},{"id":"454074","messageId":"5a8f5afccf6e2b451c76306e04ca9ef300c92fdd.1650547400.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1650547400.git.dyroneteng@gmail.com","subject":"[PATCH v2 3/5] pack-bitmap.c: make warnings more detailed when opening bitmap","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-04-21T13:26:38Z","receivedAt":"2022-04-21T13:27:14Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"When calling the \"open_midx_bitmap_1()\" or \"open_pack_bitmap_1()\", there\nwill be a warning if a normal bitmap or MIDX bitmap already has been\nopened, then let's make the warning information more detailed. For\nexample, it makes the error clearer in case of an accidental\nregression where we start looking for single-pack bitmaps after\nsuccessfully opening a multi-pack one.\n\nAt the same time, we made the previous and new warning texts support\ntranslation.\n\nDiscussion in community:\n\n 1. https://public-inbox.org/git/YkPGq0mDL4NG6D1o@nand.local/#t\n 2. https://public-inbox.org/git/220330.868rsrpohm.gmgdl@evledraar.gmail.com/\n\nHelped-by: Ævar Arnfjörð Bjarmason \nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 6 ++++--\n t/t5310-pack-bitmaps.sh | 1 +\n 2 files changed, 5 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex f13a6bfe3a..1b268f655e 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -330,7 +330,8 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(_(\"a normal or midx bitmap already has been opened\"));\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +388,8 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(_(\"a normal or midx bitmap already has been opened \"));\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\ndiff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh\nindex f775fc1ce6..eb63b71852 100755\n--- a/t/t5310-pack-bitmaps.sh\n+++ b/t/t5310-pack-bitmaps.sh\n@@ -421,6 +421,7 @@ test_expect_success 'complains about multiple pack bitmaps' '\n \t\ttest_line_count = 2 bitmaps &&\n \n \t\tgit rev-list --use-bitmap-index HEAD 2>err &&\n+\t\tgrep \"a bitmap has been opened\" err &&\n \t\tgrep \"ignoring extra bitmap file\" err\n \t)\n '\n-- \n2.35.1.583.g30faa5f068\n\n"},{"id":"454075","messageId":"1a169d7b5ea4b6aec030a48f718dc2c4e922a9f4.1650547400.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1650547400.git.dyroneteng@gmail.com","subject":"[PATCH v2 5/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-04-21T13:26:40Z","receivedAt":"2022-04-21T13:27:16Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex a1d06c4252..e0dcd06db3 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -328,7 +328,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\ttrace2_data_string(\"midx\", the_repository, \"stat bitmap file\",\n \t\t\t\t \"failed\");\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(\"cannot stat bitmap file\");\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -374,7 +374,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tbitmap_git->map_pos = 0;\n \tbitmap_git->map = NULL;\n \tbitmap_git->midx = NULL;\n-\treturn -1;\n+\treturn error(\"cannot open midx bitmap file\");\n }\n \n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n@@ -399,7 +399,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\ttrace2_data_string(\"bitmap\", the_repository, \"stat bitmap file\",\n \t\t\t\t \"failed\");\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(\"cannot stat bitmap file\");\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -413,7 +413,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \tif (!is_pack_valid(packfile)) {\n \t\ttrace2_data_string(\"bitmap\", the_repository, \"packfile\", \"invalid\");\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(\"packfile is invalid\");\n \t}\n \n \tbitmap_git->pack = packfile;\n@@ -430,7 +430,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\tbitmap_git->map_size = 0;\n \t\tbitmap_git->map_pos = 0;\n \t\tbitmap_git->pack = NULL;\n-\t\treturn -1;\n+\t\treturn error(\"bitmap header is invalid\");\n \t}\n \n \treturn 0;\n-- \n2.35.1.583.g30faa5f068\n\n"},{"id":"454076","messageId":"2016ef2e342c2ec6517afa8ec3e57035021fb965.1650547400.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1650547400.git.dyroneteng@gmail.com","subject":"[PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-04-21T13:26:39Z","receivedAt":"2022-04-21T13:27:18Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"It's supported for a Git repo to use bitmap in both normal bitmap way\nor a multi-pack-index bitmap.\n\nSometimes the debug path is not obvious, for example, when executing:\n\n $git rev-list --test-bitmap HEAD\n fatal: failed to load bitmap indexes\n\nIf we see the output like this, we are not sure about what's happened,\nbecause the cause should be :\n\n 1. neither normal nor midx bitmap exists\n 2. only midx bitmap exists but core.multipackIndex=\"false\"\n 3. core.multipackIndex=\"true\" but midx bitmap file is currupt\n 4. core.multipackIndex=\"true\" and no midx bitmap exists but\n normal bitmap file is currupt\n ....\n\nThese are some of the scenarios I briefly tested, but maybe there are\nothers (some scenarios is produced manually like \"currupt bitmap file\",\nbut it's not represent it's an existed bug.).\n\nTherefore, we added some TRACE2 code so that when we read the bitmap\nwe can be more clear about the decision path, such as whether it is\nworking on midx bitmap or normal bitmap, or is it simply because the\nrelated configuration is disabled. This may help with logging, user\ntroubleshooting, and development debugging.\n\nHere are some output examples when executing\n\"$GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\" under different\nsituations:\n\n1. _HAVE_ MIDX bitmap and NORMAL bitmap file, but the\n\"core.multipackIndex\" configures as \"false\":\n\n 19:38:43.007840 common-main.c:49 | d0 | main | version | | | | | 2.35.1.582.g8e9092487a\n 19:38:43.007874 common-main.c:50 | d0 | main | start | | 0.000305 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n 19:38:43.007950 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n 19:38:43.008091 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n 19:38:43.008284 repo-settings.c:11 | d0 | main | data | r0 | 0.000720 | 0.000720 | config | feature.manyfiles:false\n 19:38:43.008297 repo-settings.c:11 | d0 | main | data | r0 | 0.000734 | 0.000734 | config | feature.experimental:false\n 19:38:43.008305 repo-settings.c:11 | d0 | main | data | r0 | 0.000742 | 0.000742 | config | core.commitgraph:true\n 19:38:43.008313 repo-settings.c:11 | d0 | main | data | r0 | 0.000749 | 0.000749 | config | commitgraph.readchangedpaths:true\n 19:38:43.008320 repo-settings.c:11 | d0 | main | data | r0 | 0.000756 | 0.000756 | config | gc.writecommitgraph:true\n 19:38:43.008327 repo-settings.c:11 | d0 | main | data | r0 | 0.000764 | 0.000764 | config | fetch.writecommitgraph:false\n 19:38:43.008334 repo-settings.c:11 | d0 | main | data | r0 | 0.000770 | 0.000770 | config | pack.usesparse:true\n 19:38:43.008341 repo-settings.c:11 | d0 | main | data | r0 | 0.000777 | 0.000777 | config | core.multipackindex:false\n 19:38:43.008348 repo-settings.c:11 | d0 | main | data | r0 | 0.000784 | 0.000784 | config | index.sparse:false\n 19:38:43.008724 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n 19:38:43.008738 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001173 | 0.000013 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n 19:38:43.008754 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001191 | 0.000031 | pack-bitmap | label:open_bitmap\n Bitmap v1 test (8 entries loaded)\n Found bitmap for 0a7df7ae92f8d8ed879c240f8ae9cdd33d18085e. 64 bits / 1801edc6 checksum\n 19:38:43.009099 progress.c:268 | d0 | main | region_enter | r0 | 0.001535 | | progress | label:Verifying bitmap entries\n Verifying bitmap entries: 100% (27/27), done.\n 19:38:43.009294 progress.c:339 | d0 | main | data | r0 | 0.001730 | 0.000195 | progress | ..total_objects:27\n 19:38:43.009302 progress.c:346 | d0 | main | region_leave | r0 | 0.001739 | 0.000204 | progress | label:Verifying bitmap entries\n OK!\n 19:38:43.009321 git.c:718 | d0 | main | exit | | 0.001757 | | | code:0\n 19:38:43.009329 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001766 | | | code:0\n\n2. _HAVE_ MIDX bitmap and NORMAL bitmap file, but the\n\"core.multipackIndex\" configures as \"true\":\n\n (omit duplicate outputs)\n ...\n 20:02:31.288797 repo-settings.c:11 | d0 | main | data | r0 | 0.000811 | 0.000811 | config | core.multipackindex:true\n 20:02:31.288806 repo-settings.c:11 | d0 | main | data | r0 | 0.000819 | 0.000819 | config | index.sparse:false\n 20:02:31.288836 midx.c:185 | d0 | main | data | r0 | 0.000849 | 0.000849 | midx | load/num_packs:1\n 20:02:31.288843 midx.c:186 | d0 | main | data | r0 | 0.000857 | 0.000857 | midx | load/num_objects:27\n 20:02:31.289217 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001229 | | pack-bitmap | label:open_bitmap\n 20:02:31.289230 pack-bitmap.c:318 | d0 | main | data | r0 | 0.001242 | 0.000013 | midx | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/multi-pack-index-b6b04fbe053bd500d9ca13354466d3249dc275ac.bitmap\n 20:02:31.289252 pack-revindex.c:315 | d0 | main | data | r0 | 0.001265 | 0.000036 | load_midx_re | ..source:midx\n 20:02:31.289261 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001274 | 0.000045 | pack-bitmap | label:open_bitmap\n Bitmap v1 test (8 entries loaded)\n Found bitmap for 0a7df7ae92f8d8ed879c240f8ae9cdd33d18085e. 64 bits / 1801edc6 checksum\n 20:02:31.289594 progress.c:268 | d0 | main | region_enter | r0 | 0.001607 | | progress | label:Verifying bitmap entries\n Verifying bitmap entries: 100% (27/27), done.\n 20:02:31.289810 progress.c:339 | d0 | main | data | r0 | 0.001823 | 0.000216 | progress | ..total_objects:27\n 20:02:31.289824 progress.c:346 | d0 | main | region_leave | r0 | 0.001837 | 0.000230 | progress | label:Verifying bitmap entries\n OK!\n 20:02:31.289843 git.c:718 | d0 | main | exit | | 0.001856 | | | code:0\n 20:02:31.289860 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001873 | | | code:0\n\n3. _HAVE_ MIDX bitmap and a corrupt NORMAL bitmap file, but the\n\"core.multipackIndex\" configures as \"false\" :\n\n (omit duplicate outputs)\n ...\n 20:14:06.539305 repo-settings.c:11 | d0 | main | data | r0 | 0.000794 | 0.000794 | config | core.multipackindex:false\n 20:14:06.539310 repo-settings.c:11 | d0 | main | data | r0 | 0.000799 | 0.000799 | config | index.sparse:false\n 20:14:06.539658 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001146 | | pack-bitmap | label:open_bitmap\n 20:14:06.539671 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001160 | 0.000014 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n 20:14:06.539686 usage.c:79 | d0 | main | error | | | | | ..Corrupted bitmap index file (wrong header)\n error: Corrupted bitmap index file (wrong header)\n 20:14:06.539696 pack-bitmap.c:426 | d0 | main | data | r0 | 0.001185 | 0.000039 | bitmap | ..load bitmap header:failed\n 20:14:06.539709 usage.c:79 | d0 | main | error | | | | | ..bitmap header is invalid\n error: bitmap header is invalid\n 20:14:06.539719 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001208 | 0.000062 | pack-bitmap | label:open_bitmap\n 20:14:06.539726 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n fatal: failed to load bitmap indexes\n 20:14:06.539735 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n 20:14:06.539744 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001233 | | | code:128\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 40 +++++++++++++++++++++++++++++++++-------\n repo-settings.c | 1 +\n t/t5310-pack-bitmaps.sh | 4 ++--\n 3 files changed, 36 insertions(+), 9 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 1b268f655e..a1d06c4252 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -312,9 +312,12 @@ char *pack_bitmap_filename(struct packed_git *p)\n static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n+\tint fd;\n \tstruct stat st;\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(bitmap_name);\n+\ttrace2_data_string(\"midx\", the_repository, \"try to open bitmap\",\n+\t\t\t bitmap_name);\n+\tfd = git_open(bitmap_name);\n \n \tfree(bitmap_name);\n \n@@ -322,6 +325,8 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\treturn -1;\n \n \tif (fstat(fd, &st)) {\n+\t\ttrace2_data_string(\"midx\", the_repository, \"stat bitmap file\",\n+\t\t\t\t \"failed\");\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -344,12 +349,19 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t\tMAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tif (load_bitmap_header(bitmap_git) < 0)\n+\tif (load_bitmap_header(bitmap_git) < 0) {\n+\t\ttrace2_data_string(\"midx\", the_repository, \"load bitmap header\",\n+\t\t\t\t \"failed\");\n \t\tgoto cleanup;\n+\t}\n \n-\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n+\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n+\t\ttrace2_data_string(\"midx\", the_repository, \"verify checksum\",\n+\t\t\t\t \"mismatch\");\n \t\tgoto cleanup;\n \n+\t}\n+\n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n \t\tgoto cleanup;\n@@ -375,6 +387,8 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\treturn -1;\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n+\ttrace2_data_string(\"bitmap\", the_repository, \"try to open bitmap\",\n+\t\t\t bitmap_name);\n \tfd = git_open(bitmap_name);\n \tfree(bitmap_name);\n \n@@ -382,6 +396,8 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\treturn -1;\n \n \tif (fstat(fd, &st)) {\n+\t\ttrace2_data_string(\"bitmap\", the_repository, \"stat bitmap file\",\n+\t\t\t\t \"failed\");\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -395,6 +411,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t}\n \n \tif (!is_pack_valid(packfile)) {\n+\t\ttrace2_data_string(\"bitmap\", the_repository, \"packfile\", \"invalid\");\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -406,6 +423,8 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \tclose(fd);\n \n \tif (load_bitmap_header(bitmap_git) < 0) {\n+\t\ttrace2_data_string(\"bitmap\", the_repository,\n+\t\t\t\t \"load bitmap header\", \"failed\");\n \t\tmunmap(bitmap_git->map, bitmap_git->map_size);\n \t\tbitmap_git->map = NULL;\n \t\tbitmap_git->map_size = 0;\n@@ -511,11 +530,18 @@ static int open_midx_bitmap(struct repository *r,\n static int open_bitmap(struct repository *r,\n \t\t struct bitmap_index *bitmap_git)\n {\n-\tassert(!bitmap_git->map);\n+\tint ret = -1;\n \n-\tif (!open_midx_bitmap(r, bitmap_git))\n-\t\treturn 0;\n-\treturn open_pack_bitmap(r, bitmap_git);\n+\tassert(!bitmap_git->map);\n+\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n+\tif (!open_midx_bitmap(r, bitmap_git)) {\n+\t\tret = 0;\n+\t\tgoto done;\n+\t}\n+\tret = open_pack_bitmap(r, bitmap_git);\n+done:\n+\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n+\treturn ret;\n }\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r)\ndiff --git a/repo-settings.c b/repo-settings.c\nindex b4fbd16cdc..5bc7a97a6d 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -8,6 +8,7 @@ static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n {\n \tif (repo_config_get_bool(r, key, dest))\n \t\t*dest = def;\n+\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n }\n \n void prepare_repo_settings(struct repository *r)\ndiff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh\nindex eb63b71852..664cb88b0b 100755\n--- a/t/t5310-pack-bitmaps.sh\n+++ b/t/t5310-pack-bitmaps.sh\n@@ -421,8 +421,8 @@ test_expect_success 'complains about multiple pack bitmaps' '\n \t\ttest_line_count = 2 bitmaps &&\n \n \t\tgit rev-list --use-bitmap-index HEAD 2>err &&\n-\t\tgrep \"a bitmap has been opened\" err &&\n-\t\tgrep \"ignoring extra bitmap file\" err\n+\t\tgrep \"warning: a normal or midx bitmap already has been opened\" err &&\n+\t\tgrep \"warning: ignoring extra bitmap file\" err\n \t)\n '\n \n-- \n2.35.1.583.g30faa5f068\n\n"},{"id":"454086","messageId":"220421.86o80u77xj.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"1a169d7b5ea4b6aec030a48f718dc2c4e922a9f4.1650547400.git.dyroneteng@gmail.com","subject":"Re: [PATCH v2 5/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-21T15:41:36Z","receivedAt":"2022-04-21T15:50:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Apr 21 2022, Teng Long wrote:\n\n> In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\n> return error() instead of \"-1\" when some unexpected error occurs like\n> \"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\n> mismatch\", etc.\n>\n> There are places where we do not replace, such as when the bitmap\n> does not exist (no bitmap in repository is allowed) or when another\n> bitmap has already been opened (in which case it should be a warning\n> rather than an error).\n>\n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 10 +++++-----\n> 1 file changed, 5 insertions(+), 5 deletions(-)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index a1d06c4252..e0dcd06db3 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -328,7 +328,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\ttrace2_data_string(\"midx\", the_repository, \"stat bitmap file\",\n> \t\t\t\t \"failed\");\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error(\"cannot stat bitmap file\");\n> \t}\n\nFirst, I wondered if we were missing _(), but looking at other string in\nthe file they're not using that already, looks like these all should,\nbut we can fix this all up some other time in some i18n commit. It's\nfine to keep this for now.\n\nBut more importantly: I think this should be your 4/5. I.e. just make\nthese an error() and you won't need to add e.g. this\ntrace2_data_string() for a failed stat.\n\nYou will be inside your trace2 region, so any failure to stat etc. will\nbe obvious from the structure of the data and the \"error\" event, no\nreason to have an additional trace2_data_string().\n\nAside from that & as a general matter: Unless you have some use-case for\ntrace2 data in this detail I'd think that it would be better just to\nskip logging it (except if we get it for free, such as with error()).\n\nI.e. is this particular callsite really different from other places we\nfail to stat() or open() something?\n\nIt's all a moot point with the region + error, but just something to\nkeep in mind.\n"},{"id":"454087","messageId":"220421.86k0bi77mb.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"2016ef2e342c2ec6517afa8ec3e57035021fb965.1650547400.git.dyroneteng@gmail.com","subject":"Re: [PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-21T15:51:30Z","receivedAt":"2022-04-21T15:56:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Apr 21 2022, Teng Long wrote:\n\nThanks for following up..\n\n> 19:38:43.007840 common-main.c:49 | d0 | main | version | | | | | 2.35.1.582.g8e9092487a\n> 19:38:43.007874 common-main.c:50 | d0 | main | start | | 0.000305 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n\nIt's really helpful to have these full examples in the commit\nmessage. Thanks.\n\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -312,9 +312,12 @@ char *pack_bitmap_filename(struct packed_git *p)\n> static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\t\t struct multi_pack_index *midx)\n> {\n> +\tint fd;\n> \tstruct stat st;\n> \tchar *bitmap_name = midx_bitmap_filename(midx);\n> -\tint fd = git_open(bitmap_name);\n> +\ttrace2_data_string(\"midx\", the_repository, \"try to open bitmap\",\n> +\t\t\t bitmap_name);\n> +\tfd = git_open(bitmap_name);\n> \n> \tfree(bitmap_name);\n\nHrm, so re my comment on 5/5 are you trying to use the \"try to open\" as\na timer to see what our start time is?\n\nI think probably not, in which case this whole variable flip-around is\nsomething we won't need.\n\nBut if we do actually need it perhaps a sub-region for the timing?\n\n\n> @@ -511,11 +530,18 @@ static int open_midx_bitmap(struct repository *r,\n> static int open_bitmap(struct repository *r,\n> \t\t struct bitmap_index *bitmap_git)\n> {\n> -\tassert(!bitmap_git->map);\n> +\tint ret = -1;\n> \n> -\tif (!open_midx_bitmap(r, bitmap_git))\n> -\t\treturn 0;\n> -\treturn open_pack_bitmap(r, bitmap_git);\n> +\tassert(!bitmap_git->map);\n> +\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n> +\tif (!open_midx_bitmap(r, bitmap_git)) {\n> +\t\tret = 0;\n\nNit: I think these \"goto\" patterns are best if your \"int ret = N\" picks\nan \"N\" which is the default that you'll \"goto\", i.e. if you pick \"ret =\n0\" you'll just need \"goto done\" here....\n\n> +\t\tgoto done;\n> +\t}\n> +\tret = open_pack_bitmap(r, bitmap_git);\n\n...which we may then override here.\n\nJust saves you the assignment and the {}, but it tends to add up in\nlonger functions.\n\n> +done:\n> +\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n> +\treturn ret;\n> }\n\nLooks good, aside from the 5/5 comments that much of the data string\nlogging looks like it becomes redundant in the end due to error() giving\nus the same thing.\n\n> struct bitmap_index *prepare_bitmap_git(struct repository *r)\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b4fbd16cdc..5bc7a97a6d 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -8,6 +8,7 @@ static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n> {\n> \tif (repo_config_get_bool(r, key, dest))\n> \t\t*dest = def;\n> +\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n> }\n> \n> void prepare_repo_settings(struct repository *r)\n> diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh\n> index eb63b71852..664cb88b0b 100755\n> --- a/t/t5310-pack-bitmaps.sh\n> +++ b/t/t5310-pack-bitmaps.sh\n> @@ -421,8 +421,8 @@ test_expect_success 'complains about multiple pack bitmaps' '\n> \t\ttest_line_count = 2 bitmaps &&\n> \n> \t\tgit rev-list --use-bitmap-index HEAD 2>err &&\n> -\t\tgrep \"a bitmap has been opened\" err &&\n> -\t\tgrep \"ignoring extra bitmap file\" err\n> +\t\tgrep \"warning: a normal or midx bitmap already has been opened\" err &&\n> +\t\tgrep \"warning: ignoring extra bitmap file\" err\n> \t)\n> '\n\n\nI haven't tested but part of this test change looks like it'll break\nunder bisect, i.e. you changed 1/2 of these strings in 3/5. Did you try\nwith \"git rebase -i -x 'make test T=t*bitmap*\" or similar?\n"},{"id":"454089","messageId":"8005642d-979f-8e48-7a93-07b8a888bdc7@jeffhostetler.com","threadId":"57611","inReplyTo":"2016ef2e342c2ec6517afa8ec3e57035021fb965.1650547400.git.dyroneteng@gmail.com","subject":"Re: [PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-04-21T16:32:48Z","receivedAt":"2022-04-21T16:33:04Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 4/21/22 9:26 AM, Teng Long wrote:\n> It's supported for a Git repo to use bitmap in both normal bitmap way\n> or a multi-pack-index bitmap.\n> \n> Sometimes the debug path is not obvious, for example, when executing:\n> \n> $git rev-list --test-bitmap HEAD\n> fatal: failed to load bitmap indexes\n> \n> If we see the output like this, we are not sure about what's happened,\n> because the cause should be :\n> \n> 1. neither normal nor midx bitmap exists\n> 2. only midx bitmap exists but core.multipackIndex=\"false\"\n> 3. core.multipackIndex=\"true\" but midx bitmap file is currupt\n> 4. core.multipackIndex=\"true\" and no midx bitmap exists but\n> normal bitmap file is currupt\n> ....\n> \n> These are some of the scenarios I briefly tested, but maybe there are\n> others (some scenarios is produced manually like \"currupt bitmap file\",\n> but it's not represent it's an existed bug.).\n> \n> Therefore, we added some TRACE2 code so that when we read the bitmap\n> we can be more clear about the decision path, such as whether it is\n> working on midx bitmap or normal bitmap, or is it simply because the\n> related configuration is disabled. This may help with logging, user\n> troubleshooting, and development debugging.\n> \n> Here are some output examples when executing\n> \"$GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\" under different\n> situations:\n> \n> 1. _HAVE_ MIDX bitmap and NORMAL bitmap file, but the\n> \"core.multipackIndex\" configures as \"false\":\n> \n> 19:38:43.007840 common-main.c:49 | d0 | main | version | | | | | 2.35.1.582.g8e9092487a\n> 19:38:43.007874 common-main.c:50 | d0 | main | start | | 0.000305 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n> 19:38:43.007950 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n> 19:38:43.008091 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n> 19:38:43.008284 repo-settings.c:11 | d0 | main | data | r0 | 0.000720 | 0.000720 | config | feature.manyfiles:false\n> 19:38:43.008297 repo-settings.c:11 | d0 | main | data | r0 | 0.000734 | 0.000734 | config | feature.experimental:false\n> 19:38:43.008305 repo-settings.c:11 | d0 | main | data | r0 | 0.000742 | 0.000742 | config | core.commitgraph:true\n> 19:38:43.008313 repo-settings.c:11 | d0 | main | data | r0 | 0.000749 | 0.000749 | config | commitgraph.readchangedpaths:true\n> 19:38:43.008320 repo-settings.c:11 | d0 | main | data | r0 | 0.000756 | 0.000756 | config | gc.writecommitgraph:true\n> 19:38:43.008327 repo-settings.c:11 | d0 | main | data | r0 | 0.000764 | 0.000764 | config | fetch.writecommitgraph:false\n> 19:38:43.008334 repo-settings.c:11 | d0 | main | data | r0 | 0.000770 | 0.000770 | config | pack.usesparse:true\n> 19:38:43.008341 repo-settings.c:11 | d0 | main | data | r0 | 0.000777 | 0.000777 | config | core.multipackindex:false\n> 19:38:43.008348 repo-settings.c:11 | d0 | main | data | r0 | 0.000784 | 0.000784 | config | index.sparse:false\n> 19:38:43.008724 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n> 19:38:43.008738 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001173 | 0.000013 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n> 19:38:43.008754 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001191 | 0.000031 | pack-bitmap | label:open_bitmap\n> Bitmap v1 test (8 entries loaded)\n> Found bitmap for 0a7df7ae92f8d8ed879c240f8ae9cdd33d18085e. 64 bits / 1801edc6 checksum\n> 19:38:43.009099 progress.c:268 | d0 | main | region_enter | r0 | 0.001535 | | progress | label:Verifying bitmap entries\n> Verifying bitmap entries: 100% (27/27), done.\n> 19:38:43.009294 progress.c:339 | d0 | main | data | r0 | 0.001730 | 0.000195 | progress | ..total_objects:27\n> 19:38:43.009302 progress.c:346 | d0 | main | region_leave | r0 | 0.001739 | 0.000204 | progress | label:Verifying bitmap entries\n> OK!\n> 19:38:43.009321 git.c:718 | d0 | main | exit | | 0.001757 | | | code:0\n> 19:38:43.009329 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001766 | | | code:0\n> \n> 2. _HAVE_ MIDX bitmap and NORMAL bitmap file, but the\n> \"core.multipackIndex\" configures as \"true\":\n> \n> (omit duplicate outputs)\n> ...\n> 20:02:31.288797 repo-settings.c:11 | d0 | main | data | r0 | 0.000811 | 0.000811 | config | core.multipackindex:true\n> 20:02:31.288806 repo-settings.c:11 | d0 | main | data | r0 | 0.000819 | 0.000819 | config | index.sparse:false\n> 20:02:31.288836 midx.c:185 | d0 | main | data | r0 | 0.000849 | 0.000849 | midx | load/num_packs:1\n> 20:02:31.288843 midx.c:186 | d0 | main | data | r0 | 0.000857 | 0.000857 | midx | load/num_objects:27\n> 20:02:31.289217 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001229 | | pack-bitmap | label:open_bitmap\n> 20:02:31.289230 pack-bitmap.c:318 | d0 | main | data | r0 | 0.001242 | 0.000013 | midx | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/multi-pack-index-b6b04fbe053bd500d9ca13354466d3249dc275ac.bitmap\n> 20:02:31.289252 pack-revindex.c:315 | d0 | main | data | r0 | 0.001265 | 0.000036 | load_midx_re | ..source:midx\n> 20:02:31.289261 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001274 | 0.000045 | pack-bitmap | label:open_bitmap\n> Bitmap v1 test (8 entries loaded)\n> Found bitmap for 0a7df7ae92f8d8ed879c240f8ae9cdd33d18085e. 64 bits / 1801edc6 checksum\n> 20:02:31.289594 progress.c:268 | d0 | main | region_enter | r0 | 0.001607 | | progress | label:Verifying bitmap entries\n> Verifying bitmap entries: 100% (27/27), done.\n> 20:02:31.289810 progress.c:339 | d0 | main | data | r0 | 0.001823 | 0.000216 | progress | ..total_objects:27\n> 20:02:31.289824 progress.c:346 | d0 | main | region_leave | r0 | 0.001837 | 0.000230 | progress | label:Verifying bitmap entries\n> OK!\n> 20:02:31.289843 git.c:718 | d0 | main | exit | | 0.001856 | | | code:0\n> 20:02:31.289860 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001873 | | | code:0\n> \n> 3. _HAVE_ MIDX bitmap and a corrupt NORMAL bitmap file, but the\n> \"core.multipackIndex\" configures as \"false\" :\n> \n> (omit duplicate outputs)\n> ...\n> 20:14:06.539305 repo-settings.c:11 | d0 | main | data | r0 | 0.000794 | 0.000794 | config | core.multipackindex:false\n> 20:14:06.539310 repo-settings.c:11 | d0 | main | data | r0 | 0.000799 | 0.000799 | config | index.sparse:false\n> 20:14:06.539658 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001146 | | pack-bitmap | label:open_bitmap\n> 20:14:06.539671 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001160 | 0.000014 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n> 20:14:06.539686 usage.c:79 | d0 | main | error | | | | | ..Corrupted bitmap index file (wrong header)\n> error: Corrupted bitmap index file (wrong header)\n> 20:14:06.539696 pack-bitmap.c:426 | d0 | main | data | r0 | 0.001185 | 0.000039 | bitmap | ..load bitmap header:failed\n> 20:14:06.539709 usage.c:79 | d0 | main | error | | | | | ..bitmap header is invalid\n> error: bitmap header is invalid\n> 20:14:06.539719 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001208 | 0.000062 | pack-bitmap | label:open_bitmap\n> 20:14:06.539726 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n> fatal: failed to load bitmap indexes\n> 20:14:06.539735 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n> 20:14:06.539744 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001233 | | | code:128\n> \n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 40 +++++++++++++++++++++++++++++++++-------\n> repo-settings.c | 1 +\n> t/t5310-pack-bitmaps.sh | 4 ++--\n> 3 files changed, 36 insertions(+), 9 deletions(-)\n> \n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 1b268f655e..a1d06c4252 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -312,9 +312,12 @@ char *pack_bitmap_filename(struct packed_git *p)\n> static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\t\t struct multi_pack_index *midx)\n> {\n> +\tint fd;\n> \tstruct stat st;\n> \tchar *bitmap_name = midx_bitmap_filename(midx);\n> -\tint fd = git_open(bitmap_name);\n> +\ttrace2_data_string(\"midx\", the_repository, \"try to open bitmap\",\n> +\t\t\t bitmap_name);\n> +\tfd = git_open(bitmap_name);\n\nThis might just be a style thing, but rather than logging the pathname\nin a separate data_string message, you can use the _printf version of\nregion_enter and region_leave to also print the name of the\npath -- like I did in read-cache.c for the \"do_read_index\"\ncalls.\n\n... | region_enter | ... | index | label:do_read_index .git/index\n...\n... | region_leave | ... | index | label:do_read_index .git/index\n\nAs AEvar suggests in another message in this thread, I'm not sure if\nyou need the region timing here for reading the bitmap, but all of\nthe error and any other data messages will be bounded between the\nregion_enter and region_leave events and that might (or might not)\nbe helpful.\n\n\nAlso, I agree with AEvar's statements about using error() and getting\nthe trace2 error messages for free and not needing some of the\ntrace2_data_string() messages that you have later in this file.\n\nI wonder if it would be worth adding the pathname of the invalid\nfile to those new error messages. Granted you'll have it in the\ntrace2 log, but then you'll also get it on stderr if that would\nbe helpful.\n\n\nJeff\n\n\n\n\n"},{"id":"454091","messageId":"YmGTphnsVJn7CX5V@nand.local","threadId":"57611","inReplyTo":"5a8f5afccf6e2b451c76306e04ca9ef300c92fdd.1650547400.git.dyroneteng@gmail.com","subject":"Re: [PATCH v2 3/5] pack-bitmap.c: make warnings more detailed when opening bitmap","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-04-21T17:25:58Z","receivedAt":"2022-04-21T17:26:03Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Apr 21, 2022 at 09:26:38PM +0800, Teng Long wrote:\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index f13a6bfe3a..1b268f655e 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -330,7 +330,8 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\tstruct strbuf buf = STRBUF_INIT;\n> \t\tget_midx_filename(&buf, midx->object_dir);\n> \t\t/* ignore extra bitmap file; we can only handle one */\n> -\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n> +\t\twarning(_(\"a normal or midx bitmap already has been opened\"));\n> +\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n\nI'm glad that the existing warning has been marked for translation.\nThere's no reason that it couldn't have been before, so I'm glad to see\nit added now.\n\nBut I'm not sure that the new warning tells the user anything that the\nold one didn't. The old warning says \"ignoring extra bitmap file: ...\",\nleading us to believe that another one has already been opened. The new\nwarning makes the latter part explicit, but I don't think it adds any\nnew information that wasn't already readily available.\n\n> diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh\n> index f775fc1ce6..eb63b71852 100755\n> --- a/t/t5310-pack-bitmaps.sh\n> +++ b/t/t5310-pack-bitmaps.sh\n> @@ -421,6 +421,7 @@ test_expect_success 'complains about multiple pack bitmaps' '\n> \t\ttest_line_count = 2 bitmaps &&\n>\n> \t\tgit rev-list --use-bitmap-index HEAD 2>err &&\n> +\t\tgrep \"a bitmap has been opened\" err &&\n\nThis is out of sync with the warning you added, causing t5310.74 to\nfail.\n\nThanks,\nTaylor\n"},{"id":"454905","messageId":"20220506090833.17033-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"YmGTphnsVJn7CX5V@nand.local","subject":"Re: [PATCH v2 3/5] pack-bitmap.c: make warnings more detailed when opening bitmap","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-05-06T09:08:33Z","receivedAt":"2022-05-06T09:09:00Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Thu, 21 Apr 2022 13:25:58 -0400, Taylor Blau wrote:\n\n> I'm glad that the existing warning has been marked for translation.\n> There's no reason that it couldn't have been before, so I'm glad to see\n> it added now.\n\nThanks and it's actually suggested by Ævar Arnfjörð Bjarmason.\n\n> But I'm not sure that the new warning tells the user anything that the\n> old one didn't. The old warning says \"ignoring extra bitmap file: ...\",\n> leading us to believe that another one has already been opened. The new\n> warning makes the latter part explicit, but I don't think it adds any\n> new information that wasn't already readily available.\n> ... \n> This is out of sync with the warning you added, causing t5310.74 to\n> fail.\n\nYeah, I agree with you about this. \nWill roll back on here and fix the broken test in next patch.\n\n\nThanks\n"},{"id":"454911","messageId":"20220506112722.33760-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220421.86k0bi77mb.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-05-06T11:27:22Z","receivedAt":"2022-05-06T11:27:39Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Thu, 21 Apr 2022 17:51:30 +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> It's really helpful to have these full examples in the commit\n> message. Thanks.\n\nFrom the previous contribution process, I think it is necessary\nto provide sufficient information in patch, which can help to save\nreviewer's time and find problems faster.\n\n> Hrm, so re my comment on 5/5 are you trying to use the \"try to open\" as\n> a timer to see what our start time is?\n> \n> I think probably not, in which case this whole variable flip-around is\n> something we won't need.\n> \n> But if we do actually need it perhaps a sub-region for the timing?\n\nYes, I looked back on it and found it's redundant and unnecessary now.\nWill delete the two related references in the code in next patch.\n\n\n> Nit: I think these \"goto\" patterns are best if your \"int ret = N\" picks\n> an \"N\" which is the default that you'll \"goto\", i.e. if you pick \"ret =\n> 0\" you'll just need \"goto done\" here....\n> \n> > +\t goto done;\n> > +\t }\n> > +\t ret = open_pack_bitmap(r, bitmap_git);\n> \n> ...which we may then override here.\n> \n> Just saves you the assignment and the {}, but it tends to add up in\n> longer functions\n\nMake sense.\n\n> > +done:\n> > +\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n> > +\treturn ret;\n> > }\n> \n> Looks good, aside from the 5/5 comments that much of the data string\n> logging looks like it becomes redundant in the end due to error() giving\n> us the same thing.\n\nOops, I will read the comments and reply them in 5/5. \n\n> I haven't tested but part of this test change looks like it'll break\n> under bisect, i.e. you changed 1/2 of these strings in 3/5. Did you try\n> with \"git rebase -i -x 'make test T=t*bitmap*\" or similar?\n\nYes, they should be in the same commit in 3/5, but now I the new warning\ntext seems like is much verbose which suggested by Taylor Blau, and I will\nroll back the new warning message and fix the broken test under bisect.\n\nThanks. \n"},{"id":"454912","messageId":"20220506115356.100838-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220421.86k0bi77mb.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-05-06T11:53:56Z","receivedAt":"2022-05-06T11:54:06Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"> Hrm, so re my comment on 5/5 are you trying to use the \"try to open\" as\n> a timer to see what our start time is?\n> \n> I think probably not, in which case this whole variable flip-around is\n> something we won't need.\n> \n> But if we do actually need it perhaps a sub-region for the timing?\n\nBy the way, I think it will be better replace \"try to open\" with the\nbitmap name so we can also know about what kind of bitmap we are opening\nby filename.\n\nBefore: \n\n 19:38:43.008724 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n 19:38:43.008738 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001173 | 0.000013 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n 19:38:43.008754 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001191 | 0.000031 | pack-bitmap | label:open_bitmap\n\nAfter:\n \n 19:38:43.008724 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n 19:38:43.008738 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001173 | 0.000013 | bitmap | ../home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n 19:38:43.008754 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001191 | 0.000031 | pack-bitmap | label:open_bitmap\n\nThanks.\n"},{"id":"454915","messageId":"20220506124333.58956-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"8005642d-979f-8e48-7a93-07b8a888bdc7@jeffhostetler.com","subject":"Re: [PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-05-06T12:43:33Z","receivedAt":"2022-05-06T12:43:44Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"> This might just be a style thing, but rather than logging the pathname\n> in a separate data_string message, you can use the _printf version of\n> region_enter and region_leave to also print the name of the\n> path -- like I did in read-cache.c for the \"do_read_index\"\n> calls.\n> \n> ... | region_enter | ... | index | label:do_read_index .git/index\n> ...\n> ... | region_leave | ... | index | label:do_read_index .git/index\n\nAppreciate for the input about the _printf version, we can choose to\nlet the region_enter and region_leave to print the pathname by moving\nthe related \"midx_bitmap_filename()\" and \"pack_bitmap_filename\" at\nfront, but it's not enough because both midx and normal bitmap support\nmultiple opening, so it's likely we keep on the current way using\n\"trace2_data_string()\" in \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\"\nis a simpler solution.\n\nI'm not sure If I totally get the meaning about your suggestion,\nso correct me if I understand you wrong.\n\n> As AEvar suggests in another message in this thread, I'm not sure if\n> you need the region timing here for reading the bitmap, but all of\n> the error and any other data messages will be bounded between the\n> region_enter and region_leave events and that might (or might not)\n> be helpful.\n\nI think it's needed in my opinion, the bounded between the region is\nhelpful, especially when we want to know the detailed debug info like\nwe do in \"open_midx_bitmap_1()\". \n\n\n> Also, I agree with AEvar's statements about using error() and getting\n> the trace2 error messages for free and not needing some of the\n> trace2_data_string() messages that you have later in this file.\n> \n> I wonder if it would be worth adding the pathname of the invalid\n> file to those new error messages. Granted you'll have it in the\n> trace2 log, but then you'll also get it on stderr if that would\n> be helpful.\n\nI think I will remove the redundant \"trace2_data_string()\" code when\nit will return by \"error()\".\n\n\nThanks.\n"},{"id":"454916","messageId":"20220506125530.70989-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220421.86o80u77xj.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2 5/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-05-06T12:55:30Z","receivedAt":"2022-05-06T12:56:15Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Thu, 21 Apr 2022 17:41:36 +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> First, I wondered if we were missing _(), but looking at other string in\n> the file they're not using that already, looks like these all should,\n> but we can fix this all up some other time in some i18n commit. It's\n> fine to keep this for now.\n\nYes, I agree with you.\n\nI also think I have a willingness to make another patchset to solve\nthe _() missing problems recently.\n\n> But more importantly: I think this should be your 4/5. I.e. just make\n> these an error() and you won't need to add e.g. this\n> trace2_data_string() for a failed stat.\n\nMake sense. Will adjust the order in next path.\n \n> You will be inside your trace2 region, so any failure to stat etc. will\n> be obvious from the structure of the data and the \"error\" event, no\n> reason to have an additional trace2_data_string().\n\nYeah, I forgot about the \"error()\" already load the trace2 functions in.\nWill remove the redundant trace2_data_string() where it's obviously will\nreturn error().\n\n> Aside from that & as a general matter: Unless you have some use-case for\n> trace2 data in this detail I'd think that it would be better just to\n> skip logging it (except if we get it for free, such as with error()).\n> \n> I.e. is this particular callsite really different from other places we\n> fail to stat() or open() something?\n> \n> It's all a moot point with the region + error, but just something to\n> keep in mind.\n\nMake sense.\n\nAnd I think it's a good case in \"open_midx_bitmap_1()\" to add related\ntrace2_data_string() because there only a general error info in \"cleanup:\".\n\n\nThanks.\n"},{"id":"455118","messageId":"7047d8f1-5959-a553-5a53-ace7b5b925c1@jeffhostetler.com","threadId":"57611","inReplyTo":"20220506124333.58956-1-dyroneteng@gmail.com","subject":"Re: [PATCH v2 4/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-05-10T20:47:37Z","receivedAt":"2022-05-10T20:47:44Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 5/6/22 8:43 AM, Teng Long wrote:\n>> This might just be a style thing, but rather than logging the pathname\n>> in a separate data_string message, you can use the _printf version of\n>> region_enter and region_leave to also print the name of the\n>> path -- like I did in read-cache.c for the \"do_read_index\"\n>> calls.\n>>\n>> ... | region_enter | ... | index | label:do_read_index .git/index\n>> ...\n>> ... | region_leave | ... | index | label:do_read_index .git/index\n> \n> Appreciate for the input about the _printf version, we can choose to\n> let the region_enter and region_leave to print the pathname by moving\n> the related \"midx_bitmap_filename()\" and \"pack_bitmap_filename\" at\n> front, but it's not enough because both midx and normal bitmap support\n> multiple opening, so it's likely we keep on the current way using\n> \"trace2_data_string()\" in \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\"\n> is a simpler solution.\n> \n> I'm not sure If I totally get the meaning about your suggestion,\n> so correct me if I understand you wrong.\n\nThat's fine. I was assuming that you were only operating\non a single file at a time and opening, using, closing it\nin a nicely bracketed fashion. The region_enter and _leave\nis good for capturing that. But if you might be handling\nboth files concurrently, then your messages would be better\nsince we don't want mix or improperly nest the regions.\n\n(I must admit that I haven't studied the bitmap code, so\nI defer to your judgement here.)\n\n> \n>> As AEvar suggests in another message in this thread, I'm not sure if\n>> you need the region timing here for reading the bitmap, but all of\n>> the error and any other data messages will be bounded between the\n>> region_enter and region_leave events and that might (or might not)\n>> be helpful.\n> \n> I think it's needed in my opinion, the bounded between the region is\n> helpful, especially when we want to know the detailed debug info like\n> we do in \"open_midx_bitmap_1()\".\n> \n> \n>> Also, I agree with AEvar's statements about using error() and getting\n>> the trace2 error messages for free and not needing some of the\n>> trace2_data_string() messages that you have later in this file.\n>>\n>> I wonder if it would be worth adding the pathname of the invalid\n>> file to those new error messages. Granted you'll have it in the\n>> trace2 log, but then you'll also get it on stderr if that would\n>> be helpful.\n> \n> I think I will remove the redundant \"trace2_data_string()\" code when\n> it will return by \"error()\".\n> \n> \n> Thanks.\n> \n"},{"id":"455176","messageId":"YnwrHHPregT5wCH1@nand.local","threadId":"57611","inReplyTo":"1bfd2fb6ab01af689fc6495e2f32d3f64c19f4b7.1650547400.git.dyroneteng@gmail.com","subject":"Re: [PATCH v2 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-05-11T21:31:08Z","receivedAt":"2022-05-11T21:31:16Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Apr 21, 2022 at 09:26:36PM +0800, Teng Long wrote:\n> In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\n> the first one has been found, then will break out by a \"return\"\n> directly.\n>\n> But actually, it's better to don't stop the loop until we have visited\n\ns/don't stop/continue\n\n> both the MIDX in our repository, as well as any alternates (along with\n> _their_ alternates, recursively).\n>\n> The discussion link of community:\n>\n> https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nIn the future, it is often worth summarizing the discussion, optionally\nlinking off to the list archive. In this case, I wouldn't mind a little\nmore detail beyond \"it's better to [continue] the loop until ...\".\n\nIn particular, you say it's better without saying why that is the case.\nThe link to explains, but this commit\nmessage as-is leaves out some details that are important IMHO.\n\n> ---\n> pack-bitmap.c | 5 +++--\n> 1 file changed, 3 insertions(+), 2 deletions(-)\n\nThe actual changes look good to me.\n\nThanks,\nTaylor\n"},{"id":"455177","messageId":"YnwrThYqiHYbyZgM@nand.local","threadId":"57611","inReplyTo":"1fff3b3ca76d0b5e026270fdf746c55b582405d5.1650547400.git.dyroneteng@gmail.com","subject":"Re: [PATCH v2 2/5] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-05-11T21:31:58Z","receivedAt":"2022-05-11T21:32:05Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Apr 21, 2022 at 09:26:37PM +0800, Teng Long wrote:\n> although bitmap is essentially can be understood as a kind of index\n> , let's define this name a little more accurate here.\n\nThe placement of this comma is a little odd, but otherwise the patch\nlooks good to me.\n\nThanks,\nTaylor\n"},{"id":"457080","messageId":"cover.1655018322.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 0/5] trace2 output for bitmap decision path","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-12T07:44:15Z","receivedAt":"2022-06-12T07:44:39Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"*** BLURB HERE ***\n\nTeng Long (5):\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: make warnings support i18N when opening bitmap\n pack-bitmap.c: using error() instead of silently returning -1\n bitmap: add trace2 outputs during open \"bitmap\" file\n\n pack-bitmap.c | 58 +++++++++++++++++++++++++++++++------------------\n repo-settings.c | 1 +\n 2 files changed, 38 insertions(+), 21 deletions(-)\n\nRange-diff against v2:\n1: 1bfd2fb6ab ! 1: 589e3f4075 pack-bitmap.c: continue looping when first MIDX bitmap is found\n @@ Commit message\n the first one has been found, then will break out by a \"return\"\n directly.\n \n - But actually, it's better to don't stop the loop until we have visited\n + But actually, it's better to continue the loop until we have visited\n both the MIDX in our repository, as well as any alternates (along with\n _their_ alternates, recursively).\n \n + The reason for this is, there may exist more than one MIDX file in\n + a repo. The \"multi_pack_index\" struct is actually designed as a singly\n + linked list, and if a MIDX file has been already opened successfully,\n + then the other MIDX files will be skipped and left with a warning\n + \"ignoring extra bitmap file.\" to the output.\n +\n The discussion link of community:\n \n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n2: 1fff3b3ca7 ! 2: b6b30047fc pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n @@ Commit message\n \n There may bring some confusion in this \"idx_name\" naming, which\n might lead us to think of \".idx \"or\" multi-pack-index\" files,\n - although bitmap is essentially can be understood as a kind of index\n - , let's define this name a little more accurate here.\n + although bitmap is essentially can be understood as a kind of index,\n + let's define this name a little more accurate here.\n \n Signed-off-by: Teng Long \n \n3: 5a8f5afccf ! 3: d8dfe53dd4 pack-bitmap.c: make warnings more detailed when opening bitmap\n @@ Metadata\n Author: Teng Long \n \n ## Commit message ##\n - pack-bitmap.c: make warnings more detailed when opening bitmap\n + pack-bitmap.c: make warnings support i18N when opening bitmap\n \n When calling the \"open_midx_bitmap_1()\" or \"open_pack_bitmap_1()\", there\n will be a warning if a normal bitmap or MIDX bitmap already has been\n - opened, then let's make the warning information more detailed. For\n - example, it makes the error clearer in case of an accidental\n - regression where we start looking for single-pack bitmaps after\n - successfully opening a multi-pack one.\n -\n - At the same time, we made the previous and new warning texts support\n - translation.\n + opened, then let's make the warning text supporting for translation.\n \n Discussion in community:\n \n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n -\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n -+\t\twarning(_(\"a normal or midx bitmap already has been opened\"));\n +\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st\n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n -\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n -+\t\twarning(_(\"a normal or midx bitmap already has been opened \"));\n +\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n -\n - ## t/t5310-pack-bitmaps.sh ##\n -@@ t/t5310-pack-bitmaps.sh: test_expect_success 'complains about multiple pack bitmaps' '\n - \t\ttest_line_count = 2 bitmaps &&\n - \n - \t\tgit rev-list --use-bitmap-index HEAD 2>err &&\n -+\t\tgrep \"a bitmap has been opened\" err &&\n - \t\tgrep \"ignoring extra bitmap file\" err\n - \t)\n - '\n5: 1a169d7b5e ! 4: 72da3b5844 pack-bitmap.c: using error() instead of silently returning -1\n @@ Commit message\n \n ## pack-bitmap.c ##\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n - \t\ttrace2_data_string(\"midx\", the_repository, \"stat bitmap file\",\n - \t\t\t\t \"failed\");\n + \n + \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n -\t\treturn -1;\n -+\t\treturn error(\"cannot stat bitmap file\");\n ++\t\treturn error(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n - \t\ttrace2_data_string(\"bitmap\", the_repository, \"stat bitmap file\",\n - \t\t\t\t \"failed\");\n + \n + \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n -\t\treturn -1;\n -+\t\treturn error(\"cannot stat bitmap file\");\n ++\t\treturn error(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n + \n \tif (!is_pack_valid(packfile)) {\n - \t\ttrace2_data_string(\"bitmap\", the_repository, \"packfile\", \"invalid\");\n \t\tclose(fd);\n -\t\treturn -1;\n -+\t\treturn error(\"packfile is invalid\");\n ++\t\treturn error(_(\"packfile is invalid\"));\n \t}\n \n \tbitmap_git->pack = packfile;\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st\n \t\tbitmap_git->map_pos = 0;\n \t\tbitmap_git->pack = NULL;\n -\t\treturn -1;\n -+\t\treturn error(\"bitmap header is invalid\");\n ++\t\treturn error(_(\"bitmap header is invalid\"));\n \t}\n \n \treturn 0;\n4: 2016ef2e34 ! 5: e118758d1d bitmap: add trace2 outputs during open \"bitmap\" file\n @@ Metadata\n ## Commit message ##\n bitmap: add trace2 outputs during open \"bitmap\" file\n \n - It's supported for a Git repo to use bitmap in both normal bitmap way\n - or a multi-pack-index bitmap.\n -\n - Sometimes the debug path is not obvious, for example, when executing:\n + It's supported for a repo to use bitmap in both \"NORMAL\" bitmap way\n + or a MIDX (multi-pack-index) bitmap. Either of two bitmap kinds can\n + exist in the repository, or both can be stored but let the config\n + controls which kind of bitmap is used (like \"core.multipackIndex\",\n + etc.). Because of this, sometimes the bitmap debug path is not\n + obvious enough, for example, when executing:\n \n $git rev-list --test-bitmap HEAD\n fatal: failed to load bitmap indexes\n \n - If we see the output like this, we are not sure about what's happened,\n - because the cause should be :\n + If we see the output like this, It's not sure for us to know\n + what's happened concretely, because the cause should be :\n \n - 1. neither normal nor midx bitmap exists\n - 2. only midx bitmap exists but core.multipackIndex=\"false\"\n - 3. core.multipackIndex=\"true\" but midx bitmap file is currupt\n - 4. core.multipackIndex=\"true\" and no midx bitmap exists but\n - normal bitmap file is currupt\n + 1. Neither normal nor MIDX bitmap exists.\n + 2. Only MIDX bitmap exists but core.multipackIndex=\"false\".\n + 3. Config core.multipackIndex set to \"true\" but MIDX bitmap is\n + corrupted.\n + 4. Config core.multipackIndex set to \"true\" and no MIDX bitmap\n + exists but normal bitmap file is corrupted.\n ....\n \n These are some of the scenarios I briefly tested, but maybe there are\n - others (some scenarios is produced manually like \"currupt bitmap file\",\n + others (some scenarios is produced manually like \"corrupted bitmap file\",\n but it's not represent it's an existed bug.).\n \n Therefore, we added some TRACE2 code so that when we read the bitmap\n we can be more clear about the decision path, such as whether it is\n - working on midx bitmap or normal bitmap, or is it simply because the\n - related configuration is disabled. This may help with logging, user\n - troubleshooting, and development debugging.\n -\n - Here are some output examples when executing\n - \"$GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\" under different\n - situations:\n -\n - 1. _HAVE_ MIDX bitmap and NORMAL bitmap file, but the\n - \"core.multipackIndex\" configures as \"false\":\n -\n - 19:38:43.007840 common-main.c:49 | d0 | main | version | | | | | 2.35.1.582.g8e9092487a\n - 19:38:43.007874 common-main.c:50 | d0 | main | start | | 0.000305 | | | /opt/git/master/bin/git rev-list --test-bitmap HEAD\n - 19:38:43.007950 compat/linux/procinfo.c:170 | d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n - 19:38:43.008091 git.c:460 | d0 | main | cmd_name | | | | | rev-list (rev-list)\n - 19:38:43.008284 repo-settings.c:11 | d0 | main | data | r0 | 0.000720 | 0.000720 | config | feature.manyfiles:false\n - 19:38:43.008297 repo-settings.c:11 | d0 | main | data | r0 | 0.000734 | 0.000734 | config | feature.experimental:false\n - 19:38:43.008305 repo-settings.c:11 | d0 | main | data | r0 | 0.000742 | 0.000742 | config | core.commitgraph:true\n - 19:38:43.008313 repo-settings.c:11 | d0 | main | data | r0 | 0.000749 | 0.000749 | config | commitgraph.readchangedpaths:true\n - 19:38:43.008320 repo-settings.c:11 | d0 | main | data | r0 | 0.000756 | 0.000756 | config | gc.writecommitgraph:true\n - 19:38:43.008327 repo-settings.c:11 | d0 | main | data | r0 | 0.000764 | 0.000764 | config | fetch.writecommitgraph:false\n - 19:38:43.008334 repo-settings.c:11 | d0 | main | data | r0 | 0.000770 | 0.000770 | config | pack.usesparse:true\n - 19:38:43.008341 repo-settings.c:11 | d0 | main | data | r0 | 0.000777 | 0.000777 | config | core.multipackindex:false\n - 19:38:43.008348 repo-settings.c:11 | d0 | main | data | r0 | 0.000784 | 0.000784 | config | index.sparse:false\n - 19:38:43.008724 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n - 19:38:43.008738 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001173 | 0.000013 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n - 19:38:43.008754 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001191 | 0.000031 | pack-bitmap | label:open_bitmap\n - Bitmap v1 test (8 entries loaded)\n - Found bitmap for 0a7df7ae92f8d8ed879c240f8ae9cdd33d18085e. 64 bits / 1801edc6 checksum\n - 19:38:43.009099 progress.c:268 | d0 | main | region_enter | r0 | 0.001535 | | progress | label:Verifying bitmap entries\n - Verifying bitmap entries: 100% (27/27), done.\n - 19:38:43.009294 progress.c:339 | d0 | main | data | r0 | 0.001730 | 0.000195 | progress | ..total_objects:27\n - 19:38:43.009302 progress.c:346 | d0 | main | region_leave | r0 | 0.001739 | 0.000204 | progress | label:Verifying bitmap entries\n - OK!\n - 19:38:43.009321 git.c:718 | d0 | main | exit | | 0.001757 | | | code:0\n - 19:38:43.009329 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001766 | | | code:0\n -\n - 2. _HAVE_ MIDX bitmap and NORMAL bitmap file, but the\n - \"core.multipackIndex\" configures as \"true\":\n -\n - (omit duplicate outputs)\n - ...\n - 20:02:31.288797 repo-settings.c:11 | d0 | main | data | r0 | 0.000811 | 0.000811 | config | core.multipackindex:true\n - 20:02:31.288806 repo-settings.c:11 | d0 | main | data | r0 | 0.000819 | 0.000819 | config | index.sparse:false\n - 20:02:31.288836 midx.c:185 | d0 | main | data | r0 | 0.000849 | 0.000849 | midx | load/num_packs:1\n - 20:02:31.288843 midx.c:186 | d0 | main | data | r0 | 0.000857 | 0.000857 | midx | load/num_objects:27\n - 20:02:31.289217 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001229 | | pack-bitmap | label:open_bitmap\n - 20:02:31.289230 pack-bitmap.c:318 | d0 | main | data | r0 | 0.001242 | 0.000013 | midx | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/multi-pack-index-b6b04fbe053bd500d9ca13354466d3249dc275ac.bitmap\n - 20:02:31.289252 pack-revindex.c:315 | d0 | main | data | r0 | 0.001265 | 0.000036 | load_midx_re | ..source:midx\n - 20:02:31.289261 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001274 | 0.000045 | pack-bitmap | label:open_bitmap\n - Bitmap v1 test (8 entries loaded)\n - Found bitmap for 0a7df7ae92f8d8ed879c240f8ae9cdd33d18085e. 64 bits / 1801edc6 checksum\n - 20:02:31.289594 progress.c:268 | d0 | main | region_enter | r0 | 0.001607 | | progress | label:Verifying bitmap entries\n - Verifying bitmap entries: 100% (27/27), done.\n - 20:02:31.289810 progress.c:339 | d0 | main | data | r0 | 0.001823 | 0.000216 | progress | ..total_objects:27\n - 20:02:31.289824 progress.c:346 | d0 | main | region_leave | r0 | 0.001837 | 0.000230 | progress | label:Verifying bitmap entries\n - OK!\n - 20:02:31.289843 git.c:718 | d0 | main | exit | | 0.001856 | | | code:0\n - 20:02:31.289860 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001873 | | | code:0\n -\n - 3. _HAVE_ MIDX bitmap and a corrupt NORMAL bitmap file, but the\n - \"core.multipackIndex\" configures as \"false\" :\n -\n - (omit duplicate outputs)\n - ...\n - 20:14:06.539305 repo-settings.c:11 | d0 | main | data | r0 | 0.000794 | 0.000794 | config | core.multipackindex:false\n - 20:14:06.539310 repo-settings.c:11 | d0 | main | data | r0 | 0.000799 | 0.000799 | config | index.sparse:false\n - 20:14:06.539658 pack-bitmap.c:536 | d0 | main | region_enter | r0 | 0.001146 | | pack-bitmap | label:open_bitmap\n - 20:14:06.539671 pack-bitmap.c:390 | d0 | main | data | r0 | 0.001160 | 0.000014 | bitmap | ..try to open bitmap:/home/tenglong.tl/test/dyrone/.git/objects/pack/pack-2d1b3f749fb859b874710e33263d0847ef009e03.bitmap\n - 20:14:06.539686 usage.c:79 | d0 | main | error | | | | | ..Corrupted bitmap index file (wrong header)\n - error: Corrupted bitmap index file (wrong header)\n - 20:14:06.539696 pack-bitmap.c:426 | d0 | main | data | r0 | 0.001185 | 0.000039 | bitmap | ..load bitmap header:failed\n - 20:14:06.539709 usage.c:79 | d0 | main | error | | | | | ..bitmap header is invalid\n - error: bitmap header is invalid\n - 20:14:06.539719 pack-bitmap.c:543 | d0 | main | region_leave | r0 | 0.001208 | 0.000062 | pack-bitmap | label:open_bitmap\n - 20:14:06.539726 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n - fatal: failed to load bitmap indexes\n - 20:14:06.539735 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n - 20:14:06.539744 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001233 | | | code:128\n + working on MIDX or NORMAL bitmap at present, or the related config is\n + enabled or not. This may help with logging, user troubleshooting, and\n + development debugging.\n +\n + Here are some brief output examples on different scenarios when\n + executing:\n +\n + $GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n +\n + Scenario 1: core.multipackIndex [false], midx bitmap exists [Y],\n + normal bitmap exists [N]\n +\n + 19:21:56.580349 repo-settings.c:11 | d0 | main | data | r1 | 0.000827 | 0.000827 | config | core.multipackindex:false\n + 19:21:56.580356 repo-settings.c:11 | d0 | main | data | r1 | 0.000834 | 0.000834 | config | index.sparse:false\n + 19:21:56.580706 pack-bitmap.c:525 | d0 | main | region_enter | r1 | 0.001183 | | pack-bitmap | label:open_bitmap\n + 19:21:56.580719 pack-bitmap.c:386 | d0 | main | data | r1 | 0.001196 | 0.000013 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n + 19:21:56.580729 pack-bitmap.c:530 | d0 | main | region_leave | r1 | 0.001207 | 0.000024 | pack-bitmap | label:open_bitmap\n + 19:21:56.580737 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n + fatal: failed to load bitmap indexes\n + 19:21:56.580746 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n + 19:21:56.580754 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001232 | | | code:128\n +\n + Scenario 2: core.multipackIndex [false], midx bitmap exists [Y],\n + normal bitmap exists [Y]\n +\n + 19:23:44.692384 repo-settings.c:11 | d0 | main | data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false\n + 19:23:44.692755 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap\n + 19:23:44.692768 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001149 | 0.000014 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n + 19:23:44.692790 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap\n + Bitmap v1 test (1 entries loaded)\n + Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n + 19:23:44.693119 progress.c:268 | d0 | main | region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries\n + Verifying bitmap entries: 100% (3/3), done.\n + 19:23:44.693208 progress.c:339 | d0 | main | data | r0 | 0.001589 | 0.000089 | progress | ..total_objects:3\n + 19:23:44.693216 progress.c:346 | d0 | main | region_leave | r0 | 0.001597 | 0.000097 | progress | label:Verifying bitmap entries\n + OK!\n + 19:23:44.693234 git.c:718 | d0 | main | exit | | 0.001615 | | | code:0\n + 19:23:44.693244 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001625 | | | code:0\n +\n + Scenario 3: core.multipackIndex [true], midx bitmap exists [Y],\n + normal bitmap exists [Y]\n +\n + 19:26:03.625055 repo-settings.c:11 | d0 | main | data | r0 | 0.000760 | 0.000760 | config | core.multipackindex:true\n + 19:26:03.625090 midx.c:185 | d0 | main | data | r0 | 0.000795 | 0.000795 | midx | load/num_packs:1\n + 19:26:03.625097 midx.c:186 | d0 | main | data | r0 | 0.000803 | 0.000803 | midx | load/num_objects:3\n + 19:26:03.625455 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n + 19:26:03.625470 pack-bitmap.c:318 | d0 | main | data | r0 | 0.001175 | 0.000015 | midx | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/multi-pack-index-fe8e96790bd34926423bdf3efd762dbbea9f3213.bitmap\n + 19:26:03.625489 pack-revindex.c:315 | d0 | main | data | r0 | 0.001194 | 0.000034 | load_midx_re | ..source:midx\n + 19:26:03.625496 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001202 | 0.000042 | pack-bitmap | label:open_bitmap\n + Bitmap v1 test (1 entries loaded)\n + Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n + 19:26:03.625818 progress.c:268 | d0 | main | region_enter | r0 | 0.001523 | | progress | label:Verifying bitmap entries\n + Verifying bitmap entries: 100% (3/3), done.\n + 19:26:03.625916 progress.c:339 | d0 | main | data | r0 | 0.001622 | 0.000099 | progress | ..total_objects:3\n + 19:26:03.625925 progress.c:346 | d0 | main | region_leave | r0 | 0.001630 | 0.000107 | progress | label:Verifying bitmap entries\n + OK!\n + 19:26:03.625943 git.c:718 | d0 | main | exit | | 0.001648 | | | code:0\n + 19:26:03.625952 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001658 | | | code:0\n +\n + Situation 4: core.multipackIndex [false], midx bitmap exists [N],\n + normal bitmap exists [Y].\n +\n + 19:27:15.383037 repo-settings.c:11 | d0 | main | data | r0 | 0.000746 | 0.000746 | config | core.multipackindex:true\n + 19:27:15.383397 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001105 | | pack-bitmap | label:open_bitmap\n + 19:27:15.383408 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001116 | 0.000011 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n + 19:27:15.383419 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001128 | 0.000023 | pack-bitmap | label:open_bitmap\n + Bitmap v1 test (1 entries loaded)\n + Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n + 19:27:15.383730 progress.c:268 | d0 | main | region_enter | r0 | 0.001439 | | progress | label:Verifying bitmap entries\n + Verifying bitmap entries: 100% (3/3), done.\n + 19:27:15.383822 progress.c:339 | d0 | main | data | r0 | 0.001531 | 0.000092 | progress | ..total_objects:3\n + 19:27:15.383830 progress.c:346 | d0 | main | region_leave | r0 | 0.001539 | 0.000100 | progress | label:Verifying bitmap entries\n + OK!\n + 19:27:15.383848 git.c:718 | d0 | main | exit | | 0.001557 | | | code:0\n + 19:27:15.383867 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001576 | | | code:0\n +\n + Scenario 5: core.multipackIndex [true], midx bitmap exists [Y] but corrupted,\n + normal bitmap exists [Y]\n +\n + 19:29:25.888233 repo-settings.c:11 | d0 | main | data | r0 | 0.000794 | 0.000794 | config | core.multipackindex:true\n + 19:29:25.888591 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001152 | | pack-bitmap | label:open_bitmap\n + 19:29:25.888603 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001163 | 0.000011 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n + 19:29:25.888622 usage.c:79 | d0 | main | error | | | | | ..Corrupted bitmap index file (wrong header)\n + error: Corrupted bitmap index file (wrong header)\n + 19:29:25.888638 usage.c:79 | d0 | main | error | | | | | ..bitmap header is invalid\n + error: bitmap header is invalid\n + 19:29:25.888650 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001211 | 0.000059 | pack-bitmap | label:open_bitmap\n + 19:29:25.888659 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n + fatal: failed to load bitmap indexes\n + 19:29:25.888670 usage.c:74 | d0 | main | exit | | 0.001231 | | | code:128\n + 19:29:25.888680 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001241 | | | code:128\n \n Signed-off-by: Teng Long \n \n @@ pack-bitmap.c: char *pack_bitmap_filename(struct packed_git *p)\n \tstruct stat st;\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n -\tint fd = git_open(bitmap_name);\n -+\ttrace2_data_string(\"midx\", the_repository, \"try to open bitmap\",\n -+\t\t\t bitmap_name);\n ++\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n +\tfd = git_open(bitmap_name);\n \n \tfree(bitmap_name);\n \n -@@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n - \t\treturn -1;\n - \n - \tif (fstat(fd, &st)) {\n -+\t\ttrace2_data_string(\"midx\", the_repository, \"stat bitmap file\",\n -+\t\t\t\t \"failed\");\n - \t\tclose(fd);\n - \t\treturn -1;\n - \t}\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t\tMAP_PRIVATE, fd, 0);\n \tclose(fd);\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st\n \t\treturn -1;\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n -+\ttrace2_data_string(\"bitmap\", the_repository, \"try to open bitmap\",\n -+\t\t\t bitmap_name);\n ++\ttrace2_data_string(\"bitmap\", the_repository, \"path\", bitmap_name);\n \tfd = git_open(bitmap_name);\n \tfree(bitmap_name);\n \n -@@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n - \t\treturn -1;\n - \n - \tif (fstat(fd, &st)) {\n -+\t\ttrace2_data_string(\"bitmap\", the_repository, \"stat bitmap file\",\n -+\t\t\t\t \"failed\");\n - \t\tclose(fd);\n - \t\treturn -1;\n - \t}\n -@@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n - \t}\n - \n - \tif (!is_pack_valid(packfile)) {\n -+\t\ttrace2_data_string(\"bitmap\", the_repository, \"packfile\", \"invalid\");\n - \t\tclose(fd);\n - \t\treturn -1;\n - \t}\n -@@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n - \tclose(fd);\n - \n - \tif (load_bitmap_header(bitmap_git) < 0) {\n -+\t\ttrace2_data_string(\"bitmap\", the_repository,\n -+\t\t\t\t \"load bitmap header\", \"failed\");\n - \t\tmunmap(bitmap_git->map, bitmap_git->map_size);\n - \t\tbitmap_git->map = NULL;\n - \t\tbitmap_git->map_size = 0;\n @@ pack-bitmap.c: static int open_midx_bitmap(struct repository *r,\n static int open_bitmap(struct repository *r,\n \t\t struct bitmap_index *bitmap_git)\n {\n -\tassert(!bitmap_git->map);\n -+\tint ret = -1;\n ++\tint ret = 0;\n \n --\tif (!open_midx_bitmap(r, bitmap_git))\n --\t\treturn 0;\n --\treturn open_pack_bitmap(r, bitmap_git);\n +\tassert(!bitmap_git->map);\n +\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n -+\tif (!open_midx_bitmap(r, bitmap_git)) {\n -+\t\tret = 0;\n + \tif (!open_midx_bitmap(r, bitmap_git))\n +-\t\treturn 0;\n +-\treturn open_pack_bitmap(r, bitmap_git);\n +\t\tgoto done;\n -+\t}\n +\tret = open_pack_bitmap(r, bitmap_git);\n +done:\n +\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n @@ repo-settings.c: static void repo_cfg_bool(struct repository *r, const char *key\n }\n \n void prepare_repo_settings(struct repository *r)\n -\n - ## t/t5310-pack-bitmaps.sh ##\n -@@ t/t5310-pack-bitmaps.sh: test_expect_success 'complains about multiple pack bitmaps' '\n - \t\ttest_line_count = 2 bitmaps &&\n - \n - \t\tgit rev-list --use-bitmap-index HEAD 2>err &&\n --\t\tgrep \"a bitmap has been opened\" err &&\n --\t\tgrep \"ignoring extra bitmap file\" err\n -+\t\tgrep \"warning: a normal or midx bitmap already has been opened\" err &&\n -+\t\tgrep \"warning: ignoring extra bitmap file\" err\n - \t)\n - '\n - \n-- \n2.35.1.582.g320e881567\n\n"},{"id":"457081","messageId":"589e3f4075513b13dcd057242fa207b95371b114.1655018322.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-12T07:44:16Z","receivedAt":"2022-06-12T07:44:50Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to continue the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe reason for this is, there may exist more than one MIDX file in\na repo. The \"multi_pack_index\" struct is actually designed as a singly\nlinked list, and if a MIDX file has been already opened successfully,\nthen the other MIDX files will be skipped and left with a warning\n\"ignoring extra bitmap file.\" to the output.\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..112c2b12c6 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -494,15 +494,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.1.582.g320e881567\n\n"},{"id":"457082","messageId":"d8dfe53dd47b05aa55cfa2ce9eed428212fd17b8.1655018322.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 3/5] pack-bitmap.c: make warnings support i18N when opening bitmap","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-12T07:44:18Z","receivedAt":"2022-06-12T07:44:50Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"When calling the \"open_midx_bitmap_1()\" or \"open_pack_bitmap_1()\", there\nwill be a warning if a normal bitmap or MIDX bitmap already has been\nopened, then let's make the warning text supporting for translation.\n\nDiscussion in community:\n\n 1. https://public-inbox.org/git/YkPGq0mDL4NG6D1o@nand.local/#t\n 2. https://public-inbox.org/git/220330.868rsrpohm.gmgdl@evledraar.gmail.com/\n\nHelped-by: Ævar Arnfjörð Bjarmason \nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex f13a6bfe3a..af0f41833e 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -330,7 +330,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +387,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n-- \n2.35.1.582.g320e881567\n\n"},{"id":"457083","messageId":"b6b30047fc92933d254ee2034b946a10ac37bb31.1655018322.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 2/5] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-12T07:44:17Z","receivedAt":"2022-06-12T07:44:51Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index,\nlet's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 112c2b12c6..f13a6bfe3a 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -313,10 +313,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -368,14 +368,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.1.582.g320e881567\n\n"},{"id":"457084","messageId":"72da3b584490467c2492578a8125cbcfe05aad9a.1655018322.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 4/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-12T07:44:19Z","receivedAt":"2022-06-12T07:44:54Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex af0f41833e..5654eb7b8d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -323,7 +323,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -361,7 +361,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tbitmap_git->map_pos = 0;\n \tbitmap_git->map = NULL;\n \tbitmap_git->midx = NULL;\n-\treturn -1;\n+\treturn error(\"cannot open midx bitmap file\");\n }\n \n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n@@ -382,7 +382,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -394,7 +394,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (!is_pack_valid(packfile)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(_(\"packfile is invalid\"));\n \t}\n \n \tbitmap_git->pack = packfile;\n@@ -409,7 +409,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\tbitmap_git->map_size = 0;\n \t\tbitmap_git->map_pos = 0;\n \t\tbitmap_git->pack = NULL;\n-\t\treturn -1;\n+\t\treturn error(_(\"bitmap header is invalid\"));\n \t}\n \n \treturn 0;\n-- \n2.35.1.582.g320e881567\n\n"},{"id":"457085","messageId":"e118758d1dada378d65d58579cc1372fa547d720.1655018322.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-12T07:44:20Z","receivedAt":"2022-06-12T07:44:54Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"It's supported for a repo to use bitmap in both \"NORMAL\" bitmap way\nor a MIDX (multi-pack-index) bitmap. Either of two bitmap kinds can\nexist in the repository, or both can be stored but let the config\ncontrols which kind of bitmap is used (like \"core.multipackIndex\",\netc.). Because of this, sometimes the bitmap debug path is not\nobvious enough, for example, when executing:\n\n $git rev-list --test-bitmap HEAD\n fatal: failed to load bitmap indexes\n\nIf we see the output like this, It's not sure for us to know\nwhat's happened concretely, because the cause should be :\n\n 1. Neither normal nor MIDX bitmap exists.\n 2. Only MIDX bitmap exists but core.multipackIndex=\"false\".\n 3. Config core.multipackIndex set to \"true\" but MIDX bitmap is\n corrupted.\n 4. Config core.multipackIndex set to \"true\" and no MIDX bitmap\n exists but normal bitmap file is corrupted.\n ....\n\nThese are some of the scenarios I briefly tested, but maybe there are\nothers (some scenarios is produced manually like \"corrupted bitmap file\",\nbut it's not represent it's an existed bug.).\n\nTherefore, we added some TRACE2 code so that when we read the bitmap\nwe can be more clear about the decision path, such as whether it is\nworking on MIDX or NORMAL bitmap at present, or the related config is\nenabled or not. This may help with logging, user troubleshooting, and\ndevelopment debugging.\n\nHere are some brief output examples on different scenarios when\nexecuting:\n\n $GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n\nScenario 1: core.multipackIndex [false], midx bitmap exists [Y],\nnormal bitmap exists [N]\n\n 19:21:56.580349 repo-settings.c:11 | d0 | main | data | r1 | 0.000827 | 0.000827 | config | core.multipackindex:false\n 19:21:56.580356 repo-settings.c:11 | d0 | main | data | r1 | 0.000834 | 0.000834 | config | index.sparse:false\n 19:21:56.580706 pack-bitmap.c:525 | d0 | main | region_enter | r1 | 0.001183 | | pack-bitmap | label:open_bitmap\n 19:21:56.580719 pack-bitmap.c:386 | d0 | main | data | r1 | 0.001196 | 0.000013 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n 19:21:56.580729 pack-bitmap.c:530 | d0 | main | region_leave | r1 | 0.001207 | 0.000024 | pack-bitmap | label:open_bitmap\n 19:21:56.580737 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n fatal: failed to load bitmap indexes\n 19:21:56.580746 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n 19:21:56.580754 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001232 | | | code:128\n\nScenario 2: core.multipackIndex [false], midx bitmap exists [Y],\nnormal bitmap exists [Y]\n\n\t19:23:44.692384 repo-settings.c:11 | d0 | main | data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false\n\t19:23:44.692755 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap\n\t19:23:44.692768 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001149 | 0.000014 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n\t19:23:44.692790 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap\n\tBitmap v1 test (1 entries loaded)\n\tFound bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n\t19:23:44.693119 progress.c:268 | d0 | main | region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries\n\tVerifying bitmap entries: 100% (3/3), done.\n\t19:23:44.693208 progress.c:339 | d0 | main | data | r0 | 0.001589 | 0.000089 | progress | ..total_objects:3\n\t19:23:44.693216 progress.c:346 | d0 | main | region_leave | r0 | 0.001597 | 0.000097 | progress | label:Verifying bitmap entries\n\tOK!\n\t19:23:44.693234 git.c:718 | d0 | main | exit | | 0.001615 | | | code:0\n\t19:23:44.693244 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001625 | | | code:0\n\nScenario 3: core.multipackIndex [true], midx bitmap exists [Y],\nnormal bitmap exists [Y]\n\n\t19:26:03.625055 repo-settings.c:11 | d0 | main | data | r0 | 0.000760 | 0.000760 | config | core.multipackindex:true\n\t19:26:03.625090 midx.c:185 | d0 | main | data | r0 | 0.000795 | 0.000795 | midx | load/num_packs:1\n\t19:26:03.625097 midx.c:186 | d0 | main | data | r0 | 0.000803 | 0.000803 | midx | load/num_objects:3\n\t19:26:03.625455 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n\t19:26:03.625470 pack-bitmap.c:318 | d0 | main | data | r0 | 0.001175 | 0.000015 | midx | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/multi-pack-index-fe8e96790bd34926423bdf3efd762dbbea9f3213.bitmap\n\t19:26:03.625489 pack-revindex.c:315 | d0 | main | data | r0 | 0.001194 | 0.000034 | load_midx_re | ..source:midx\n\t19:26:03.625496 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001202 | 0.000042 | pack-bitmap | label:open_bitmap\n\tBitmap v1 test (1 entries loaded)\n\tFound bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n\t19:26:03.625818 progress.c:268 | d0 | main | region_enter | r0 | 0.001523 | | progress | label:Verifying bitmap entries\n\tVerifying bitmap entries: 100% (3/3), done.\n\t19:26:03.625916 progress.c:339 | d0 | main | data | r0 | 0.001622 | 0.000099 | progress | ..total_objects:3\n\t19:26:03.625925 progress.c:346 | d0 | main | region_leave | r0 | 0.001630 | 0.000107 | progress | label:Verifying bitmap entries\n\tOK!\n\t19:26:03.625943 git.c:718 | d0 | main | exit | | 0.001648 | | | code:0\n\t19:26:03.625952 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001658 | | | code:0\n\nSituation 4: core.multipackIndex [false], midx bitmap exists [N],\nnormal bitmap exists [Y].\n\n\t19:27:15.383037 repo-settings.c:11 | d0 | main | data | r0 | 0.000746 | 0.000746 | config | core.multipackindex:true\n\t19:27:15.383397 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001105 | | pack-bitmap | label:open_bitmap\n\t19:27:15.383408 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001116 | 0.000011 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n\t19:27:15.383419 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001128 | 0.000023 | pack-bitmap | label:open_bitmap\n\tBitmap v1 test (1 entries loaded)\n\tFound bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n\t19:27:15.383730 progress.c:268 | d0 | main | region_enter | r0 | 0.001439 | | progress | label:Verifying bitmap entries\n\tVerifying bitmap entries: 100% (3/3), done.\n\t19:27:15.383822 progress.c:339 | d0 | main | data | r0 | 0.001531 | 0.000092 | progress | ..total_objects:3\n\t19:27:15.383830 progress.c:346 | d0 | main | region_leave | r0 | 0.001539 | 0.000100 | progress | label:Verifying bitmap entries\n\tOK!\n\t19:27:15.383848 git.c:718 | d0 | main | exit | | 0.001557 | | | code:0\n\t19:27:15.383867 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001576 | | | code:0\n\nScenario 5: core.multipackIndex [true], midx bitmap exists [Y] but corrupted,\nnormal bitmap exists [Y]\n\n\t19:29:25.888233 repo-settings.c:11 | d0 | main | data | r0 | 0.000794 | 0.000794 | config | core.multipackindex:true\n\t19:29:25.888591 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001152 | | pack-bitmap | label:open_bitmap\n\t19:29:25.888603 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001163 | 0.000011 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n\t19:29:25.888622 usage.c:79 | d0 | main | error | | | | | ..Corrupted bitmap index file (wrong header)\n\terror: Corrupted bitmap index file (wrong header)\n\t19:29:25.888638 usage.c:79 | d0 | main | error | | | | | ..bitmap header is invalid\n\terror: bitmap header is invalid\n\t19:29:25.888650 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001211 | 0.000059 | pack-bitmap | label:open_bitmap\n\t19:29:25.888659 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n\tfatal: failed to load bitmap indexes\n\t19:29:25.888670 usage.c:74 | d0 | main | exit | | 0.001231 | | | code:128\n\t19:29:25.888680 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001241 | | | code:128\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 27 +++++++++++++++++++++------\n repo-settings.c | 1 +\n 2 files changed, 22 insertions(+), 6 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 5654eb7b8d..ced5993560 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -312,9 +312,11 @@ char *pack_bitmap_filename(struct packed_git *p)\n static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n+\tint fd;\n \tstruct stat st;\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(bitmap_name);\n+\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n+\tfd = git_open(bitmap_name);\n \n \tfree(bitmap_name);\n \n@@ -343,12 +345,19 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t\tMAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tif (load_bitmap_header(bitmap_git) < 0)\n+\tif (load_bitmap_header(bitmap_git) < 0) {\n+\t\ttrace2_data_string(\"midx\", the_repository, \"load bitmap header\",\n+\t\t\t\t \"failed\");\n \t\tgoto cleanup;\n+\t}\n \n-\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n+\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n+\t\ttrace2_data_string(\"midx\", the_repository, \"verify checksum\",\n+\t\t\t\t \"mismatch\");\n \t\tgoto cleanup;\n \n+\t}\n+\n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n \t\tgoto cleanup;\n@@ -374,6 +383,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\treturn -1;\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n+\ttrace2_data_string(\"bitmap\", the_repository, \"path\", bitmap_name);\n \tfd = git_open(bitmap_name);\n \tfree(bitmap_name);\n \n@@ -509,11 +519,16 @@ static int open_midx_bitmap(struct repository *r,\n static int open_bitmap(struct repository *r,\n \t\t struct bitmap_index *bitmap_git)\n {\n-\tassert(!bitmap_git->map);\n+\tint ret = 0;\n \n+\tassert(!bitmap_git->map);\n+\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n \tif (!open_midx_bitmap(r, bitmap_git))\n-\t\treturn 0;\n-\treturn open_pack_bitmap(r, bitmap_git);\n+\t\tgoto done;\n+\tret = open_pack_bitmap(r, bitmap_git);\n+done:\n+\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n+\treturn ret;\n }\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r)\ndiff --git a/repo-settings.c b/repo-settings.c\nindex b4fbd16cdc..5bc7a97a6d 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -8,6 +8,7 @@ static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n {\n \tif (repo_config_get_bool(r, key, dest))\n \t\t*dest = def;\n+\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n }\n \n void prepare_repo_settings(struct repository *r)\n-- \n2.35.1.582.g320e881567\n\n"},{"id":"457122","messageId":"xmqq5yl45lgm.fsf@gitster.g","threadId":"57611","inReplyTo":"e118758d1dada378d65d58579cc1372fa547d720.1655018322.git.dyroneteng@gmail.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-13T20:59:05Z","receivedAt":"2022-06-13T21:17:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 5654eb7b8d..ced5993560 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -312,9 +312,11 @@ char *pack_bitmap_filename(struct packed_git *p)\n> static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\t\t struct multi_pack_index *midx)\n> {\n> +\tint fd;\n> \tstruct stat st;\n> \tchar *bitmap_name = midx_bitmap_filename(midx);\n> -\tint fd = git_open(bitmap_name);\n> +\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n> +\tfd = git_open(bitmap_name);\n> \n> \tfree(bitmap_name);\n\nThe patch adds new statements at a wrong place. The block of\ndeclarations and the first statement in the block were separated by\na blank line, but they no longer are.\n\nThese things tend to show up in merges quite clearly. They do not\ncause more unnecessary conflicts but can make resolution more error\nprone.\n\nThanks.\n"},{"id":"457161","messageId":"YqfhR/PV6HhGyhT5@nand.local","threadId":"57611","inReplyTo":"72da3b584490467c2492578a8125cbcfe05aad9a.1655018322.git.dyroneteng@gmail.com","subject":"Re: [PATCH v3 4/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-14T01:15:51Z","receivedAt":"2022-06-14T01:16:02Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sun, Jun 12, 2022 at 03:44:19PM +0800, Teng Long wrote:\n> @@ -323,7 +323,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n>\n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error(_(\"cannot stat bitmap file\"));\n\nSince we are handling an error from fstat here, the errno variable\ncontains useful information that we should include in the error via\nerror_errno().\n\n> @@ -361,7 +361,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \tbitmap_git->map_pos = 0;\n> \tbitmap_git->map = NULL;\n> \tbitmap_git->midx = NULL;\n> -\treturn -1;\n> +\treturn error(\"cannot open midx bitmap file\");\n\nThe other error strings are marked for translation, but this one is not.\nWas that intentional, or just a typo / oversight?\n\n> static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n> @@ -382,7 +382,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n>\n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error(_(\"cannot stat bitmap file\"));\n\nSame note here about using error_errno() instead of just error().\n\nThanks,\nTaylor\n"},{"id":"457162","messageId":"Yqfm+rsP5DWyj95L@nand.local","threadId":"57611","inReplyTo":"e118758d1dada378d65d58579cc1372fa547d720.1655018322.git.dyroneteng@gmail.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-06-14T01:40:10Z","receivedAt":"2022-06-14T01:40:15Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sun, Jun 12, 2022 at 03:44:20PM +0800, Teng Long wrote:\n> It's supported for a repo to use bitmap in both \"NORMAL\" bitmap way\n\nI'm nitpicking, but I usually say \"single-pack bitmap\" or \"multi-pack\n(MIDX) bitmap\" when distinguishing between the two.\n\n> or a MIDX (multi-pack-index) bitmap. Either of two bitmap kinds can\n> exist in the repository, or both can be stored but let the config\n> controls which kind of bitmap is used (like \"core.multipackIndex\",\n> etc.). Because of this, sometimes the bitmap debug path is not\n> obvious enough, for example, when executing:\n>\n> $git rev-list --test-bitmap HEAD\n> fatal: failed to load bitmap indexes\n\nOdd spacing, and there should be a single space character separating \"$\"\nfrom \"git\" (like \"$ git\").\n\nWhile I'm thinking about it: is the error message here up-to-date with\nthe changes made by the previous patch?\n\n> If we see the output like this, It's not sure for us to know\n> what's happened concretely, because the cause should be :\n>\n> 1. Neither normal nor MIDX bitmap exists.\n> 2. Only MIDX bitmap exists but core.multipackIndex=\"false\".\n> 3. Config core.multipackIndex set to \"true\" but MIDX bitmap is\n> corrupted.\n> 4. Config core.multipackIndex set to \"true\" and no MIDX bitmap\n> exists but normal bitmap file is corrupted.\n> ....\n>\n> These are some of the scenarios I briefly tested, but maybe there are\n> others (some scenarios is produced manually like \"corrupted bitmap file\",\n> but it's not represent it's an existed bug.).\n\nThis could probably be trimmed down for brevity, but I don't feel\nstrongly about it. If you wanted to make your commit message a tad\nshorter, perhaps something like:\n\n When a user sees output like this, it's unclear which kind(s) of\n .bitmap exist, and which were read. For example, it's possible a MIDX\n bitmap exists, but was not read (e.g., because\n core.multiPackIndex=false), among many other scenarios.\n\nwould suffice.\n\n> Therefore, we added some TRACE2 code so that when we read the bitmap\n\ns/TRACE2/trace2\n\n> we can be more clear about the decision path, such as whether it is\n> working on MIDX or NORMAL bitmap at present, or the related config is\n\ns/NORMAL/pack\n\n> enabled or not. This may help with logging, user troubleshooting, and\n> development debugging.\n>\n> Here are some brief output examples on different scenarios when\n> executing:\n>\n> $GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n\ns/$GIT/$ GIT\n\n> Scenario 1: core.multipackIndex [false], midx bitmap exists [Y],\n> normal bitmap exists [N]\n\nThe output here is quite wide, and I wonder if this whole section could\nbe shortened. For example, scenario 2 is arguably more interesting than\nscenario 1 (I think readers would reasonably infer what happens in\nscenario 1 by reading what happens in scenario 2).\n\n> 19:21:56.580349 repo-settings.c:11 | d0 | main | data | r1 | 0.000827 | 0.000827 | config | core.multipackindex:false\n> 19:21:56.580356 repo-settings.c:11 | d0 | main | data | r1 | 0.000834 | 0.000834 | config | index.sparse:false\n> 19:21:56.580706 pack-bitmap.c:525 | d0 | main | region_enter | r1 | 0.001183 | | pack-bitmap | label:open_bitmap\n> 19:21:56.580719 pack-bitmap.c:386 | d0 | main | data | r1 | 0.001196 | 0.000013 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n> 19:21:56.580729 pack-bitmap.c:530 | d0 | main | region_leave | r1 | 0.001207 | 0.000024 | pack-bitmap | label:open_bitmap\n> 19:21:56.580737 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n> fatal: failed to load bitmap indexes\n> 19:21:56.580746 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n> 19:21:56.580754 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001232 | | | code:128\n>\n> Scenario 2: core.multipackIndex [false], midx bitmap exists [Y],\n> normal bitmap exists [Y]\n>\n> \t19:23:44.692384 repo-settings.c:11 | d0 | main | data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false\n> \t19:23:44.692755 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap\n> \t19:23:44.692768 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001149 | 0.000014 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n> \t19:23:44.692790 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap\n> \tBitmap v1 test (1 entries loaded)\n> \tFound bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n> \t19:23:44.693119 progress.c:268 | d0 | main | region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries\n> \tVerifying bitmap entries: 100% (3/3), done.\n> \t19:23:44.693208 progress.c:339 | d0 | main | data | r0 | 0.001589 | 0.000089 | progress | ..total_objects:3\n> \t19:23:44.693216 progress.c:346 | d0 | main | region_leave | r0 | 0.001597 | 0.000097 | progress | label:Verifying bitmap entries\n> \tOK!\n> \t19:23:44.693234 git.c:718 | d0 | main | exit | | 0.001615 | | | code:0\n> \t19:23:44.693244 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001625 | | | code:0\n\nAnd scenario 2 could be cleaned up by just showing a few of the columns\nfrom the trace2 output. Perhaps along the lines of:\n\n> \t| data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false\n> \t| region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap\n> \t| data | r0 | 0.001149 | 0.000014 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n> \t| region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap\n> \t| region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries\n\nReading the below scenarios, I think just showing this example is more\nthan sufficient for illustrating your point.\n\n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 27 +++++++++++++++++++++------\n> repo-settings.c | 1 +\n> 2 files changed, 22 insertions(+), 6 deletions(-)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 5654eb7b8d..ced5993560 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -312,9 +312,11 @@ char *pack_bitmap_filename(struct packed_git *p)\n> static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\t\t struct multi_pack_index *midx)\n> {\n> +\tint fd;\n> \tstruct stat st;\n> \tchar *bitmap_name = midx_bitmap_filename(midx);\n> -\tint fd = git_open(bitmap_name);\n> +\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n> +\tfd = git_open(bitmap_name);\n>\n> \tfree(bitmap_name);\n>\n> @@ -343,12 +345,19 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\t\t\tMAP_PRIVATE, fd, 0);\n> \tclose(fd);\n>\n> -\tif (load_bitmap_header(bitmap_git) < 0)\n> +\tif (load_bitmap_header(bitmap_git) < 0) {\n> +\t\ttrace2_data_string(\"midx\", the_repository, \"load bitmap header\",\n> +\t\t\t\t \"failed\");\n\nI wonder, why don't we show these errors to the user? Should we\nintroduce a \"ret\" variable and set it to (for e.g.)\n\n ret = error(_(\"failed to load bitmap header\"));\n\nor something?\n\n> struct bitmap_index *prepare_bitmap_git(struct repository *r)\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b4fbd16cdc..5bc7a97a6d 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -8,6 +8,7 @@ static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n> {\n> \tif (repo_config_get_bool(r, key, dest))\n> \t\t*dest = def;\n> +\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n\nI'm not sure we want to dump the whole repository configuration into\ntrace2 output. Is there a more convenient place to log any important\nvalue(s) _after_ they have been read?\n\nThanks,\nTaylor\n"},{"id":"457556","messageId":"20220620131730.23409-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"YqfhR/PV6HhGyhT5@nand.local","subject":"Re: [PATCH v3 4/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-20T13:17:30Z","receivedAt":"2022-06-20T13:50:18Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Mon, 13 Jun 2022 21:15:51 -0400, Taylor Blau wrote:\n\n> Since we are handling an error from fstat here, the errno variable\n> contains useful information that we should include in the error via\n> error_errno().\n\nAppreciate for reminding this.\nWill fix.\n\n> The other error strings are marked for translation, but this one is not.\n> Was that intentional, or just a typo / oversight?\n\nIt's an oversight.\nWill fix.\n\n> Same note here about using error_errno() instead of just error().\n\nAs same as the first one, will fix.\n\nThanks.\n"},{"id":"457557","messageId":"20220620133212.35937-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"xmqq5yl45lgm.fsf@gitster.g","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-20T13:32:12Z","receivedAt":"2022-06-20T14:18:21Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Mon, 13 Jun 2022 13:59:05 -0700, Junio C Hamano wrote:\n\n\n> The patch adds new statements at a wrong place. The block of\n> declarations and the first statement in the block were separated by\n> a blank line, but they no longer are.\n> \n> These things tend to show up in merges quite clearly. They do not\n> cause more unnecessary conflicts but can make resolution more error\n> prone.\n\nYou are absolutely right.\nWill fix.\n\nThanks.\n"},{"id":"457592","messageId":"20220621065853.126718-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"Yqfm+rsP5DWyj95L@nand.local","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T06:58:53Z","receivedAt":"2022-06-21T06:59:04Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Mon, 13 Jun 2022 21:40:10 -0400, Taylor Blau wrote:\n\n> I'm nitpicking, but I usually say \"single-pack bitmap\" or \"multi-pack\n> (MIDX) bitmap\" when distinguishing between the two.\n\nHa, it's OK, I'm hungry. Your description is a better way I think.\n\nWill optimize the relevant places.\n\n> Odd spacing, and there should be a single space character separating \"$\"\n> from \"git\" (like \"$ git\").\n\nYeah, will Fix.\n\n> While I'm thinking about it: is the error message here up-to-date with\n> the changes made by the previous patch?\n\nIt's still the same after the remake and test (actually it's just a case\nnot all the cases have the same error message. now by the previous path\nif \"core.multipackindex=false\" and single-pack doesn't exist, the error\nmessage will look like this).\n\n> This could probably be trimmed down for brevity, but I don't feel\n> strongly about it. If you wanted to make your commit message a tad\n> shorter, perhaps something like:\n> \n> When a user sees output like this, it's unclear which kind(s) of\n> .bitmap exist, and which were read. For example, it's possible a MIDX\n> bitmap exists, but was not read (e.g., because\n> core.multiPackIndex=false), among many other scenarios.\n> \n> would suffice.\n\nI think yours is better, I will use it instead the listed way.\n\n> s/TRACE2/trace2\n\nWill fix.\n\n> s/NORMAL/pack\n\nWill fix as s/NORMAL/single-pack.\n\n> s/$GIT/$ GIT\n\nWill fix.\n\n> The output here is quite wide, and I wonder if this whole section could\n> be shortened. For example, scenario 2 is arguably more interesting than\n> scenario 1 (I think readers would reasonably infer what happens in\n> scenario 1 by reading what happens in scenario 2).\n> ...\n> And scenario 2 could be cleaned up by just showing a few of the columns\n> from the trace2 output. Perhaps along the lines of:\n> ...\n> Reading the below scenarios, I think just showing this example is more\n> than sufficient for illustrating your point.\n\nThat makes sense. I'll keep two scenarios and omit some of the output to\nkeep it shorter in next patch.\n\n\n> I wonder, why don't we show these errors to the user? Should we\n> introduce a \"ret\" variable and set it to (for e.g.)\n> \n> ret = error(_(\"failed to load bitmap header\"));\n> \n> or something?\n\nJust this \"load header error\", I think it's unnecessary to show more\nerrors to uses because in \"load_bitmap_header()\", the function already\ndoes, but in my patch, it's unnecessary to add \"trace2_data_string()\"\ntoo, so here I will remove it (directly \"goto cleanup\" maybe) in\nnext patch.\n\n> I'm not sure we want to dump the whole repository configuration into\n> trace2 output. Is there a more convenient place to log any important\n> value(s) _after_ they have been read?\n\nI think maybe I can add a argument like \"int trace\" in next patch, so\nthe caller can decide whether to print to trace or not.\n\nThanks for your help, Taylor.\n"},{"id":"457609","messageId":"cover.1655817253.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655018322.git.dyroneteng@gmail.com","subject":"[PATCH v3 0/5] trace2 output for bitmap decision path","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T13:25:00Z","receivedAt":"2022-06-21T13:31:17Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Main diff from v4 to v3:\n\n* Optimize the commit message of [5/5].\n* Fix the missing blank line in [5/5].\n* Only print the config we want to instead of the whole configs .\n* use \"error_errno()\" instead in [4/5].\n\nThanks for Junio C Hamano and Taylor Blau for their help in v3.\n\n\nTeng Long (5):\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: make warnings support i18N when opening bitmap\n pack-bitmap.c: using error() instead of silently returning -1\n bitmap: add trace2 outputs during open \"bitmap\" file\n\n pack-bitmap.c | 52 ++++++++++++++++++++++++++++++-------------------\n repo-settings.c | 22 +++++++++++----------\n 2 files changed, 44 insertions(+), 30 deletions(-)\n\nRange-diff against v2:\n-: ---------- > 1: 589e3f4075 pack-bitmap.c: continue looping when first MIDX bitmap is found\n-: ---------- > 2: b6b30047fc pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n-: ---------- > 3: d8dfe53dd4 pack-bitmap.c: make warnings support i18N when opening bitmap\n1: 72da3b5844 ! 4: 917551f2b5 pack-bitmap.c: using error() instead of silently returning -1\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n -\t\treturn -1;\n -+\t\treturn error(_(\"cannot stat bitmap file\"));\n ++\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tbitmap_git->map = NULL;\n \tbitmap_git->midx = NULL;\n -\treturn -1;\n -+\treturn error(\"cannot open midx bitmap file\");\n ++\treturn error(_(\"cannot open midx bitmap file\"));\n }\n \n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st\n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n -\t\treturn -1;\n -+\t\treturn error(_(\"cannot stat bitmap file\"));\n ++\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n2: e118758d1d ! 5: 8735ae9979 bitmap: add trace2 outputs during open \"bitmap\" file\n @@ Metadata\n ## Commit message ##\n bitmap: add trace2 outputs during open \"bitmap\" file\n \n - It's supported for a repo to use bitmap in both \"NORMAL\" bitmap way\n - or a MIDX (multi-pack-index) bitmap. Either of two bitmap kinds can\n + It's supported for a repo to use bitmap in both single-pack bitmap\n + way or a multi-pack(MIDX) bitmap. Either of two bitmap kinds can\n exist in the repository, or both can be stored but let the config\n controls which kind of bitmap is used (like \"core.multipackIndex\",\n etc.). Because of this, sometimes the bitmap debug path is not\n obvious enough, for example, when executing:\n \n - $git rev-list --test-bitmap HEAD\n + $ git rev-list --test-bitmap HEAD\n fatal: failed to load bitmap indexes\n \n - If we see the output like this, It's not sure for us to know\n - what's happened concretely, because the cause should be :\n + When the output look like this, it's unclear which kind(s) of\n + .bitmap exist, and which were read. For example, it's possible a MIDX\n + bitmap exists, but was not read (e.g., because\n + core.multiPackIndex=false), among many other scenarios.\n \n - 1. Neither normal nor MIDX bitmap exists.\n - 2. Only MIDX bitmap exists but core.multipackIndex=\"false\".\n - 3. Config core.multipackIndex set to \"true\" but MIDX bitmap is\n - corrupted.\n - 4. Config core.multipackIndex set to \"true\" and no MIDX bitmap\n - exists but normal bitmap file is corrupted.\n - ....\n -\n - These are some of the scenarios I briefly tested, but maybe there are\n - others (some scenarios is produced manually like \"corrupted bitmap file\",\n - but it's not represent it's an existed bug.).\n -\n - Therefore, we added some TRACE2 code so that when we read the bitmap\n + Therefore, we added some trace2 code so that when we read the bitmap\n we can be more clear about the decision path, such as whether it is\n - working on MIDX or NORMAL bitmap at present, or the related config is\n - enabled or not. This may help with logging, user troubleshooting, and\n + working on MIDX or single-pack bitmap at present, or the related config\n + is enabled or not. This may help with logging, user troubleshooting, and\n development debugging.\n \n - Here are some brief output examples on different scenarios when\n + Here are some brief output (omitted some unrelated or repetitive rows\n + and columns, using \"...\" instead of) examples on two scenarios when\n executing:\n \n - $GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n -\n - Scenario 1: core.multipackIndex [false], midx bitmap exists [Y],\n - normal bitmap exists [N]\n -\n - 19:21:56.580349 repo-settings.c:11 | d0 | main | data | r1 | 0.000827 | 0.000827 | config | core.multipackindex:false\n - 19:21:56.580356 repo-settings.c:11 | d0 | main | data | r1 | 0.000834 | 0.000834 | config | index.sparse:false\n - 19:21:56.580706 pack-bitmap.c:525 | d0 | main | region_enter | r1 | 0.001183 | | pack-bitmap | label:open_bitmap\n - 19:21:56.580719 pack-bitmap.c:386 | d0 | main | data | r1 | 0.001196 | 0.000013 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n - 19:21:56.580729 pack-bitmap.c:530 | d0 | main | region_leave | r1 | 0.001207 | 0.000024 | pack-bitmap | label:open_bitmap\n - 19:21:56.580737 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n - fatal: failed to load bitmap indexes\n - 19:21:56.580746 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128\n - 19:21:56.580754 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001232 | | | code:128\n -\n - Scenario 2: core.multipackIndex [false], midx bitmap exists [Y],\n - normal bitmap exists [Y]\n -\n - 19:23:44.692384 repo-settings.c:11 | d0 | main | data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false\n - 19:23:44.692755 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap\n - 19:23:44.692768 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001149 | 0.000014 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n - 19:23:44.692790 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap\n - Bitmap v1 test (1 entries loaded)\n - Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n - 19:23:44.693119 progress.c:268 | d0 | main | region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries\n - Verifying bitmap entries: 100% (3/3), done.\n - 19:23:44.693208 progress.c:339 | d0 | main | data | r0 | 0.001589 | 0.000089 | progress | ..total_objects:3\n - 19:23:44.693216 progress.c:346 | d0 | main | region_leave | r0 | 0.001597 | 0.000097 | progress | label:Verifying bitmap entries\n - OK!\n - 19:23:44.693234 git.c:718 | d0 | main | exit | | 0.001615 | | | code:0\n - 19:23:44.693244 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001625 | | | code:0\n -\n - Scenario 3: core.multipackIndex [true], midx bitmap exists [Y],\n - normal bitmap exists [Y]\n -\n - 19:26:03.625055 repo-settings.c:11 | d0 | main | data | r0 | 0.000760 | 0.000760 | config | core.multipackindex:true\n - 19:26:03.625090 midx.c:185 | d0 | main | data | r0 | 0.000795 | 0.000795 | midx | load/num_packs:1\n - 19:26:03.625097 midx.c:186 | d0 | main | data | r0 | 0.000803 | 0.000803 | midx | load/num_objects:3\n - 19:26:03.625455 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001160 | | pack-bitmap | label:open_bitmap\n - 19:26:03.625470 pack-bitmap.c:318 | d0 | main | data | r0 | 0.001175 | 0.000015 | midx | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/multi-pack-index-fe8e96790bd34926423bdf3efd762dbbea9f3213.bitmap\n - 19:26:03.625489 pack-revindex.c:315 | d0 | main | data | r0 | 0.001194 | 0.000034 | load_midx_re | ..source:midx\n - 19:26:03.625496 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001202 | 0.000042 | pack-bitmap | label:open_bitmap\n - Bitmap v1 test (1 entries loaded)\n - Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n - 19:26:03.625818 progress.c:268 | d0 | main | region_enter | r0 | 0.001523 | | progress | label:Verifying bitmap entries\n - Verifying bitmap entries: 100% (3/3), done.\n - 19:26:03.625916 progress.c:339 | d0 | main | data | r0 | 0.001622 | 0.000099 | progress | ..total_objects:3\n - 19:26:03.625925 progress.c:346 | d0 | main | region_leave | r0 | 0.001630 | 0.000107 | progress | label:Verifying bitmap entries\n - OK!\n - 19:26:03.625943 git.c:718 | d0 | main | exit | | 0.001648 | | | code:0\n - 19:26:03.625952 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001658 | | | code:0\n -\n - Situation 4: core.multipackIndex [false], midx bitmap exists [N],\n - normal bitmap exists [Y].\n -\n - 19:27:15.383037 repo-settings.c:11 | d0 | main | data | r0 | 0.000746 | 0.000746 | config | core.multipackindex:true\n - 19:27:15.383397 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001105 | | pack-bitmap | label:open_bitmap\n - 19:27:15.383408 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001116 | 0.000011 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n - 19:27:15.383419 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001128 | 0.000023 | pack-bitmap | label:open_bitmap\n - Bitmap v1 test (1 entries loaded)\n - Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n - 19:27:15.383730 progress.c:268 | d0 | main | region_enter | r0 | 0.001439 | | progress | label:Verifying bitmap entries\n - Verifying bitmap entries: 100% (3/3), done.\n - 19:27:15.383822 progress.c:339 | d0 | main | data | r0 | 0.001531 | 0.000092 | progress | ..total_objects:3\n - 19:27:15.383830 progress.c:346 | d0 | main | region_leave | r0 | 0.001539 | 0.000100 | progress | label:Verifying bitmap entries\n - OK!\n - 19:27:15.383848 git.c:718 | d0 | main | exit | | 0.001557 | | | code:0\n - 19:27:15.383867 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001576 | | | code:0\n -\n - Scenario 5: core.multipackIndex [true], midx bitmap exists [Y] but corrupted,\n - normal bitmap exists [Y]\n -\n - 19:29:25.888233 repo-settings.c:11 | d0 | main | data | r0 | 0.000794 | 0.000794 | config | core.multipackindex:true\n - 19:29:25.888591 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001152 | | pack-bitmap | label:open_bitmap\n - 19:29:25.888603 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001163 | 0.000011 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n - 19:29:25.888622 usage.c:79 | d0 | main | error | | | | | ..Corrupted bitmap index file (wrong header)\n - error: Corrupted bitmap index file (wrong header)\n - 19:29:25.888638 usage.c:79 | d0 | main | error | | | | | ..bitmap header is invalid\n - error: bitmap header is invalid\n - 19:29:25.888650 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001211 | 0.000059 | pack-bitmap | label:open_bitmap\n - 19:29:25.888659 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes\n - fatal: failed to load bitmap indexes\n - 19:29:25.888670 usage.c:74 | d0 | main | exit | | 0.001231 | | | code:128\n - 19:29:25.888680 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001241 | | | code:128\n + $ GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n +\n + Scenario 1:\n + core.multipackIndex [false], MIDX bitmap exists [Y], single-pack bitmap exists [N]\n +\n + ...\n + ... | main | data | r1 | ... | config | core.multipackindex:false\n + ... | d0 | main | region_enter | r1 | ... | pack-bitmap | label:open_bitmap\n + ... | d0 | main | data | r1 | ... | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n + ... | main | region_leave | r1 | ... | pack-bitmap | label:open_bitmap\n + ... | main | error | | ... | | failed to load bitmap indexes\n + fatal: failed to load bitmap indexes\n + ... | d0 | main | exit | | ... | | code:128\n + ...\n +\n + Scenario 2:\n + core.multipackIndex [false], MIDX bitmap exists [Y], single-pack bitmap exists [Y]\n +\n + ... | d0 | main | region_enter | r0 | ... | pack-bitmap | label:open_bitmap\n + ... | d0 | main | data | r0 | ... | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n + ... | main | region_leave | r0 | ... | pack-bitmap | label:open_bitmap\n + Bitmap v1 test (1 entries loaded)\n + Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n + ... | main | region_enter | r0 | ... | progress | label:Verifying bitmap entries\n + Verifying bitmap entries: 100% (3/3), done.\n + ... | main | data | r0 | ... | progress | ..total_objects:3\n + ... | main | region_leave | r0 | ... | progress | label:Verifying bitmap entries\n + OK!\n + ... | d0 | main | exit | | ... | | code:0\n + ...\n \n Signed-off-by: Teng Long \n \n @@ pack-bitmap.c: char *pack_bitmap_filename(struct packed_git *p)\n {\n +\tint fd;\n \tstruct stat st;\n ++\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n -\tint fd = git_open(bitmap_name);\n +\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n @@ pack-bitmap.c: char *pack_bitmap_filename(struct packed_git *p)\n \tfree(bitmap_name);\n \n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n - \t\t\t\tMAP_PRIVATE, fd, 0);\n - \tclose(fd);\n - \n --\tif (load_bitmap_header(bitmap_git) < 0)\n -+\tif (load_bitmap_header(bitmap_git) < 0) {\n -+\t\ttrace2_data_string(\"midx\", the_repository, \"load bitmap header\",\n -+\t\t\t\t \"failed\");\n + \tif (load_bitmap_header(bitmap_git) < 0)\n \t\tgoto cleanup;\n -+\t}\n \n -\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n +\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n -+\t\ttrace2_data_string(\"midx\", the_repository, \"verify checksum\",\n -+\t\t\t\t \"mismatch\");\n ++\t\terror(_(\"midx and bitmap checksum don't match\"));\n \t\tgoto cleanup;\n - \n +\t}\n -+\n + \n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n - \t\tgoto cleanup;\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\treturn -1;\n \n @@ pack-bitmap.c: static int open_midx_bitmap(struct repository *r,\n struct bitmap_index *prepare_bitmap_git(struct repository *r)\n \n ## repo-settings.c ##\n -@@ repo-settings.c: static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n +@@\n + #include \"midx.h\"\n + \n + static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n +-\t\t\t int def)\n ++\t\t\t int def, int trace)\n {\n \tif (repo_config_get_bool(r, key, dest))\n \t\t*dest = def;\n -+\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n ++\tif (trace)\n ++\t\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n }\n \n void prepare_repo_settings(struct repository *r)\n +@@ repo-settings.c: void prepare_repo_settings(struct repository *r)\n + \tr->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_CONSECUTIVE;\n + \n + \t/* Booleans config or default, cascades to other settings */\n +-\trepo_cfg_bool(r, \"feature.manyfiles\", &manyfiles, 0);\n +-\trepo_cfg_bool(r, \"feature.experimental\", &experimental, 0);\n ++\trepo_cfg_bool(r, \"feature.manyfiles\", &manyfiles, 0, 0);\n ++\trepo_cfg_bool(r, \"feature.experimental\", &experimental, 0, 0);\n + \n + \t/* Defaults modified by feature.* */\n + \tif (experimental) {\n +@@ repo-settings.c: void prepare_repo_settings(struct repository *r)\n + \t}\n + \n + \t/* Boolean config or default, does not cascade (simple) */\n +-\trepo_cfg_bool(r, \"core.commitgraph\", &r->settings.core_commit_graph, 1);\n +-\trepo_cfg_bool(r, \"commitgraph.readchangedpaths\", &r->settings.commit_graph_read_changed_paths, 1);\n +-\trepo_cfg_bool(r, \"gc.writecommitgraph\", &r->settings.gc_write_commit_graph, 1);\n +-\trepo_cfg_bool(r, \"fetch.writecommitgraph\", &r->settings.fetch_write_commit_graph, 0);\n +-\trepo_cfg_bool(r, \"pack.usesparse\", &r->settings.pack_use_sparse, 1);\n +-\trepo_cfg_bool(r, \"core.multipackindex\", &r->settings.core_multi_pack_index, 1);\n +-\trepo_cfg_bool(r, \"index.sparse\", &r->settings.sparse_index, 0);\n ++\trepo_cfg_bool(r, \"core.commitgraph\", &r->settings.core_commit_graph, 1, 0);\n ++\trepo_cfg_bool(r, \"commitgraph.readchangedpaths\", &r->settings.commit_graph_read_changed_paths, 1, 0);\n ++\trepo_cfg_bool(r, \"gc.writecommitgraph\", &r->settings.gc_write_commit_graph, 1, 0);\n ++\trepo_cfg_bool(r, \"fetch.writecommitgraph\", &r->settings.fetch_write_commit_graph, 0, 0);\n ++\trepo_cfg_bool(r, \"pack.usesparse\", &r->settings.pack_use_sparse, 1, 0);\n ++\trepo_cfg_bool(r, \"core.multipackindex\", &r->settings.core_multi_pack_index, 1, 1);\n ++\trepo_cfg_bool(r, \"index.sparse\", &r->settings.sparse_index, 0, 0);\n + \n + \t/*\n + \t * The GIT_TEST_MULTI_PACK_INDEX variable is special in that\n-- \n2.35.1.582.g270d558070.dirty\n\n"},{"id":"457610","messageId":"589e3f4075513b13dcd057242fa207b95371b114.1655817253.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655817253.git.dyroneteng@gmail.com","subject":"[PATCH v3 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T13:25:01Z","receivedAt":"2022-06-21T13:33:48Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to continue the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe reason for this is, there may exist more than one MIDX file in\na repo. The \"multi_pack_index\" struct is actually designed as a singly\nlinked list, and if a MIDX file has been already opened successfully,\nthen the other MIDX files will be skipped and left with a warning\n\"ignoring extra bitmap file.\" to the output.\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..112c2b12c6 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -494,15 +494,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.1.582.g270d558070.dirty\n\n"},{"id":"457611","messageId":"d8dfe53dd47b05aa55cfa2ce9eed428212fd17b8.1655817253.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655817253.git.dyroneteng@gmail.com","subject":"[PATCH v3 3/5] pack-bitmap.c: make warnings support i18N when opening bitmap","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T13:25:03Z","receivedAt":"2022-06-21T13:34:12Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"When calling the \"open_midx_bitmap_1()\" or \"open_pack_bitmap_1()\", there\nwill be a warning if a normal bitmap or MIDX bitmap already has been\nopened, then let's make the warning text supporting for translation.\n\nDiscussion in community:\n\n 1. https://public-inbox.org/git/YkPGq0mDL4NG6D1o@nand.local/#t\n 2. https://public-inbox.org/git/220330.868rsrpohm.gmgdl@evledraar.gmail.com/\n\nHelped-by: Ævar Arnfjörð Bjarmason \nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex f13a6bfe3a..af0f41833e 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -330,7 +330,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +387,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n-- \n2.35.1.582.g270d558070.dirty\n\n"},{"id":"457612","messageId":"b6b30047fc92933d254ee2034b946a10ac37bb31.1655817253.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655817253.git.dyroneteng@gmail.com","subject":"[PATCH v3 2/5] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T13:25:02Z","receivedAt":"2022-06-21T13:34:14Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index,\nlet's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 112c2b12c6..f13a6bfe3a 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -313,10 +313,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -368,14 +368,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.1.582.g270d558070.dirty\n\n"},{"id":"457613","messageId":"8735ae99798a43650acb9a1df7ba2072c7758476.1655817253.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655817253.git.dyroneteng@gmail.com","subject":"[PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T13:25:05Z","receivedAt":"2022-06-21T13:34:16Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"It's supported for a repo to use bitmap in both single-pack bitmap\nway or a multi-pack(MIDX) bitmap. Either of two bitmap kinds can\nexist in the repository, or both can be stored but let the config\ncontrols which kind of bitmap is used (like \"core.multipackIndex\",\netc.). Because of this, sometimes the bitmap debug path is not\nobvious enough, for example, when executing:\n\n $ git rev-list --test-bitmap HEAD\n fatal: failed to load bitmap indexes\n\nWhen the output look like this, it's unclear which kind(s) of\n.bitmap exist, and which were read. For example, it's possible a MIDX\nbitmap exists, but was not read (e.g., because\ncore.multiPackIndex=false), among many other scenarios.\n\nTherefore, we added some trace2 code so that when we read the bitmap\nwe can be more clear about the decision path, such as whether it is\nworking on MIDX or single-pack bitmap at present, or the related config\nis enabled or not. This may help with logging, user troubleshooting, and\ndevelopment debugging.\n\nHere are some brief output (omitted some unrelated or repetitive rows\nand columns, using \"...\" instead of) examples on two scenarios when\nexecuting:\n\n $ GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n\nScenario 1:\n core.multipackIndex [false], MIDX bitmap exists [Y], single-pack bitmap exists [N]\n\n ...\n ... | main | data | r1 | ... | config | core.multipackindex:false\n ... | d0 | main | region_enter | r1 | ... | pack-bitmap | label:open_bitmap\n ... | d0 | main | data | r1 | ... | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n ... | main | region_leave | r1 | ... | pack-bitmap | label:open_bitmap\n ... | main | error | | ... | | failed to load bitmap indexes\n fatal: failed to load bitmap indexes\n ... | d0 | main | exit | | ... | | code:128\n ...\n\nScenario 2:\n core.multipackIndex [false], MIDX bitmap exists [Y], single-pack bitmap exists [Y]\n\n ... | d0 | main | region_enter | r0 | ... | pack-bitmap | label:open_bitmap\n ... | d0 | main | data | r0 | ... | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n ... | main | region_leave | r0 | ... | pack-bitmap | label:open_bitmap\n Bitmap v1 test (1 entries loaded)\n Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n ... | main | region_enter | r0 | ... | progress | label:Verifying bitmap entries\n Verifying bitmap entries: 100% (3/3), done.\n ... | main | data | r0 | ... | progress | ..total_objects:3\n ... | main | region_leave | r0 | ... | progress | label:Verifying bitmap entries\n OK!\n ... | d0 | main | exit | | ... | | code:0\n ...\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 21 ++++++++++++++++-----\n repo-settings.c | 22 ++++++++++++----------\n 2 files changed, 28 insertions(+), 15 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex a54d5a0c9f..c39d722592 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -312,9 +312,12 @@ char *pack_bitmap_filename(struct packed_git *p)\n static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n+\tint fd;\n \tstruct stat st;\n+\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(bitmap_name);\n+\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n+\tfd = git_open(bitmap_name);\n \n \tfree(bitmap_name);\n \n@@ -346,8 +349,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (load_bitmap_header(bitmap_git) < 0)\n \t\tgoto cleanup;\n \n-\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n+\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n+\t\terror(_(\"midx and bitmap checksum don't match\"));\n \t\tgoto cleanup;\n+\t}\n \n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n@@ -374,6 +379,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\treturn -1;\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n+\ttrace2_data_string(\"bitmap\", the_repository, \"path\", bitmap_name);\n \tfd = git_open(bitmap_name);\n \tfree(bitmap_name);\n \n@@ -509,11 +515,16 @@ static int open_midx_bitmap(struct repository *r,\n static int open_bitmap(struct repository *r,\n \t\t struct bitmap_index *bitmap_git)\n {\n-\tassert(!bitmap_git->map);\n+\tint ret = 0;\n \n+\tassert(!bitmap_git->map);\n+\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n \tif (!open_midx_bitmap(r, bitmap_git))\n-\t\treturn 0;\n-\treturn open_pack_bitmap(r, bitmap_git);\n+\t\tgoto done;\n+\tret = open_pack_bitmap(r, bitmap_git);\n+done:\n+\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n+\treturn ret;\n }\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r)\ndiff --git a/repo-settings.c b/repo-settings.c\nindex b4fbd16cdc..115d96ece3 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -4,10 +4,12 @@\n #include \"midx.h\"\n \n static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n-\t\t\t int def)\n+\t\t\t int def, int trace)\n {\n \tif (repo_config_get_bool(r, key, dest))\n \t\t*dest = def;\n+\tif (trace)\n+\t\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n }\n \n void prepare_repo_settings(struct repository *r)\n@@ -29,8 +31,8 @@ void prepare_repo_settings(struct repository *r)\n \tr->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_CONSECUTIVE;\n \n \t/* Booleans config or default, cascades to other settings */\n-\trepo_cfg_bool(r, \"feature.manyfiles\", &manyfiles, 0);\n-\trepo_cfg_bool(r, \"feature.experimental\", &experimental, 0);\n+\trepo_cfg_bool(r, \"feature.manyfiles\", &manyfiles, 0, 0);\n+\trepo_cfg_bool(r, \"feature.experimental\", &experimental, 0, 0);\n \n \t/* Defaults modified by feature.* */\n \tif (experimental) {\n@@ -42,13 +44,13 @@ void prepare_repo_settings(struct repository *r)\n \t}\n \n \t/* Boolean config or default, does not cascade (simple) */\n-\trepo_cfg_bool(r, \"core.commitgraph\", &r->settings.core_commit_graph, 1);\n-\trepo_cfg_bool(r, \"commitgraph.readchangedpaths\", &r->settings.commit_graph_read_changed_paths, 1);\n-\trepo_cfg_bool(r, \"gc.writecommitgraph\", &r->settings.gc_write_commit_graph, 1);\n-\trepo_cfg_bool(r, \"fetch.writecommitgraph\", &r->settings.fetch_write_commit_graph, 0);\n-\trepo_cfg_bool(r, \"pack.usesparse\", &r->settings.pack_use_sparse, 1);\n-\trepo_cfg_bool(r, \"core.multipackindex\", &r->settings.core_multi_pack_index, 1);\n-\trepo_cfg_bool(r, \"index.sparse\", &r->settings.sparse_index, 0);\n+\trepo_cfg_bool(r, \"core.commitgraph\", &r->settings.core_commit_graph, 1, 0);\n+\trepo_cfg_bool(r, \"commitgraph.readchangedpaths\", &r->settings.commit_graph_read_changed_paths, 1, 0);\n+\trepo_cfg_bool(r, \"gc.writecommitgraph\", &r->settings.gc_write_commit_graph, 1, 0);\n+\trepo_cfg_bool(r, \"fetch.writecommitgraph\", &r->settings.fetch_write_commit_graph, 0, 0);\n+\trepo_cfg_bool(r, \"pack.usesparse\", &r->settings.pack_use_sparse, 1, 0);\n+\trepo_cfg_bool(r, \"core.multipackindex\", &r->settings.core_multi_pack_index, 1, 1);\n+\trepo_cfg_bool(r, \"index.sparse\", &r->settings.sparse_index, 0, 0);\n \n \t/*\n \t * The GIT_TEST_MULTI_PACK_INDEX variable is special in that\n-- \n2.35.1.582.g270d558070.dirty\n\n"},{"id":"457614","messageId":"917551f2b53196c4754d8881a383e989a02d571e.1655817253.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655817253.git.dyroneteng@gmail.com","subject":"[PATCH v3 4/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-21T13:25:04Z","receivedAt":"2022-06-21T13:34:22Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex af0f41833e..a54d5a0c9f 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -323,7 +323,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -361,7 +361,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tbitmap_git->map_pos = 0;\n \tbitmap_git->map = NULL;\n \tbitmap_git->midx = NULL;\n-\treturn -1;\n+\treturn error(_(\"cannot open midx bitmap file\"));\n }\n \n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n@@ -382,7 +382,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -394,7 +394,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (!is_pack_valid(packfile)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(_(\"packfile is invalid\"));\n \t}\n \n \tbitmap_git->pack = packfile;\n@@ -409,7 +409,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\tbitmap_git->map_size = 0;\n \t\tbitmap_git->map_pos = 0;\n \t\tbitmap_git->pack = NULL;\n-\t\treturn -1;\n+\t\treturn error(_(\"bitmap header is invalid\"));\n \t}\n \n \treturn 0;\n-- \n2.35.1.582.g270d558070.dirty\n\n"},{"id":"457692","messageId":"78966fb1-d5be-2fb5-0f68-0fce0b93d249@jeffhostetler.com","threadId":"57611","inReplyTo":"e118758d1dada378d65d58579cc1372fa547d720.1655018322.git.dyroneteng@gmail.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-06-22T12:51:18Z","receivedAt":"2022-06-22T12:51:22Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/12/22 3:44 AM, Teng Long wrote:\n[...]\n> \n> struct bitmap_index *prepare_bitmap_git(struct repository *r)\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b4fbd16cdc..5bc7a97a6d 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -8,6 +8,7 @@ static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n> {\n> \tif (repo_config_get_bool(r, key, dest))\n> \t\t*dest = def;\n> +\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n> }\n\nWe should not be doing. This would dump every repo-related\nboolean value on every command.\n\nI already have a GIT_TRACE2_CONFIG_PARAMS and trace2.configparams\nthat will dump \"interesting\" config values to the trace2 log.\nJust set one of them to a list of regex's. Look at the comment above\ntrace2_cmd_list_config() in trace2.h for details.\n\nWe also have GIT_TRACE2_ENV_VARS and trace2.envvars that will dump\nthe values of \"interesting\" env vars.\n\nYou can use these in your testing to log the config and env var\nvalues that you are interested in.\n\nJeff\n\n"},{"id":"457693","messageId":"4176c5af-43fd-b7a1-346c-4d4aa9dbd4ae@jeffhostetler.com","threadId":"57611","inReplyTo":"8735ae99798a43650acb9a1df7ba2072c7758476.1655817253.git.dyroneteng@gmail.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-06-22T13:04:32Z","receivedAt":"2022-06-22T13:04:43Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/21/22 9:25 AM, Teng Long wrote:\n> It's supported for a repo to use bitmap in both single-pack bitmap\n> way or a multi-pack(MIDX) bitmap. Either of two bitmap kinds can\n> exist in the repository, or both can be stored but let the config\n> controls which kind of bitmap is used (like \"core.multipackIndex\",\n> etc.). Because of this, sometimes the bitmap debug path is not\n> obvious enough, for example, when executing:\n> \n> $ git rev-list --test-bitmap HEAD\n> fatal: failed to load bitmap indexes\n> \n> When the output look like this, it's unclear which kind(s) of\n> .bitmap exist, and which were read. For example, it's possible a MIDX\n> bitmap exists, but was not read (e.g., because\n> core.multiPackIndex=false), among many other scenarios.\n> \n> Therefore, we added some trace2 code so that when we read the bitmap\n> we can be more clear about the decision path, such as whether it is\n> working on MIDX or single-pack bitmap at present, or the related config\n> is enabled or not. This may help with logging, user troubleshooting, and\n> development debugging.\n> \n> Here are some brief output (omitted some unrelated or repetitive rows\n> and columns, using \"...\" instead of) examples on two scenarios when\n> executing:\n> \n> $ GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD\n> \n> Scenario 1:\n> core.multipackIndex [false], MIDX bitmap exists [Y], single-pack bitmap exists [N]\n> \n> ...\n> ... | main | data | r1 | ... | config | core.multipackindex:false\n> ... | d0 | main | region_enter | r1 | ... | pack-bitmap | label:open_bitmap\n> ... | d0 | main | data | r1 | ... | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n> ... | main | region_leave | r1 | ... | pack-bitmap | label:open_bitmap\n> ... | main | error | | ... | | failed to load bitmap indexes\n> fatal: failed to load bitmap indexes\n> ... | d0 | main | exit | | ... | | code:128\n> ...\n> \n> Scenario 2:\n> core.multipackIndex [false], MIDX bitmap exists [Y], single-pack bitmap exists [Y]\n> \n> ... | d0 | main | region_enter | r0 | ... | pack-bitmap | label:open_bitmap\n> ... | d0 | main | data | r0 | ... | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap\n> ... | main | region_leave | r0 | ... | pack-bitmap | label:open_bitmap\n> Bitmap v1 test (1 entries loaded)\n> Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum\n> ... | main | region_enter | r0 | ... | progress | label:Verifying bitmap entries\n> Verifying bitmap entries: 100% (3/3), done.\n> ... | main | data | r0 | ... | progress | ..total_objects:3\n> ... | main | region_leave | r0 | ... | progress | label:Verifying bitmap entries\n> OK!\n> ... | d0 | main | exit | | ... | | code:0\n> ...\n> \n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 21 ++++++++++++++++-----\n> repo-settings.c | 22 ++++++++++++----------\n> 2 files changed, 28 insertions(+), 15 deletions(-)\n> \n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index a54d5a0c9f..c39d722592 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -312,9 +312,12 @@ char *pack_bitmap_filename(struct packed_git *p)\n> static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \t\t\t struct multi_pack_index *midx)\n> {\n> +\tint fd;\n> \tstruct stat st;\n> +\n> \tchar *bitmap_name = midx_bitmap_filename(midx);\n> -\tint fd = git_open(bitmap_name);\n> +\ttrace2_data_string(\"midx\", the_repository, \"path\", bitmap_name);\n> +\tfd = git_open(bitmap_name);\n> \n> \tfree(bitmap_name);\n> \n> @@ -346,8 +349,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \tif (load_bitmap_header(bitmap_git) < 0)\n> \t\tgoto cleanup;\n> \n> -\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n> +\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n> +\t\terror(_(\"midx and bitmap checksum don't match\"));\n> \t\tgoto cleanup;\n> +\t}\n> \n> \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n> \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n> @@ -374,6 +379,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \t\treturn -1;\n> \n> \tbitmap_name = pack_bitmap_filename(packfile);\n> +\ttrace2_data_string(\"bitmap\", the_repository, \"path\", bitmap_name);\n> \tfd = git_open(bitmap_name);\n> \tfree(bitmap_name);\n> \n> @@ -509,11 +515,16 @@ static int open_midx_bitmap(struct repository *r,\n> static int open_bitmap(struct repository *r,\n> \t\t struct bitmap_index *bitmap_git)\n> {\n> -\tassert(!bitmap_git->map);\n> +\tint ret = 0;\n> \n> +\tassert(!bitmap_git->map);\n> +\ttrace2_region_enter(\"pack-bitmap\", \"open_bitmap\", r);\n> \tif (!open_midx_bitmap(r, bitmap_git))\n> -\t\treturn 0;\n> -\treturn open_pack_bitmap(r, bitmap_git);\n> +\t\tgoto done;\n> +\tret = open_pack_bitmap(r, bitmap_git);\n> +done:\n> +\ttrace2_region_leave(\"pack-bitmap\", \"open_bitmap\", r);\n> +\treturn ret;\n> }\n> \n> struct bitmap_index *prepare_bitmap_git(struct repository *r)\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b4fbd16cdc..115d96ece3 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -4,10 +4,12 @@\n> #include \"midx.h\"\n> \n> static void repo_cfg_bool(struct repository *r, const char *key, int *dest,\n> -\t\t\t int def)\n> +\t\t\t int def, int trace)\n> {\n> \tif (repo_config_get_bool(r, key, dest))\n> \t\t*dest = def;\n> +\tif (trace)\n> +\t\ttrace2_data_string(\"config\", r, key, *dest ? \"true\" : \"false\");\n> }\n> \n\n(I just sent a response to your V2 before I saw your V3, so I'll\nmy response here so that it doesn't get lost.)\n\n\nWe should not be doing this. This would dump every repo-related\nboolean value on every command. I see that in V3 that you have\na \"trace\" flag to control this. But again, this seems wrong here.\n\nI already have a GIT_TRACE2_CONFIG_PARAMS and trace2.configparams\nthat will dump \"interesting\" config values to the trace2 log.\nJust set one of them to a list of regex's. Look at the comment above\ntrace2_cmd_list_config() in trace2.h for details.\n\nWe also have GIT_TRACE2_ENV_VARS and trace2.envvars that will dump\nthe values of \"interesting\" env vars.\n\nYou can use these in your testing to log the config and env var\nvalues that you are interested in.\n\n\n> void prepare_repo_settings(struct repository *r)\n> @@ -29,8 +31,8 @@ void prepare_repo_settings(struct repository *r)\n> \tr->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_CONSECUTIVE;\n> \n> \t/* Booleans config or default, cascades to other settings */\n> -\trepo_cfg_bool(r, \"feature.manyfiles\", &manyfiles, 0);\n> -\trepo_cfg_bool(r, \"feature.experimental\", &experimental, 0);\n> +\trepo_cfg_bool(r, \"feature.manyfiles\", &manyfiles, 0, 0);\n> +\trepo_cfg_bool(r, \"feature.experimental\", &experimental, 0, 0);\n> \n> \t/* Defaults modified by feature.* */\n> \tif (experimental) {\n> @@ -42,13 +44,13 @@ void prepare_repo_settings(struct repository *r)\n> \t}\n> \n> \t/* Boolean config or default, does not cascade (simple) */\n> -\trepo_cfg_bool(r, \"core.commitgraph\", &r->settings.core_commit_graph, 1);\n> -\trepo_cfg_bool(r, \"commitgraph.readchangedpaths\", &r->settings.commit_graph_read_changed_paths, 1);\n> -\trepo_cfg_bool(r, \"gc.writecommitgraph\", &r->settings.gc_write_commit_graph, 1);\n> -\trepo_cfg_bool(r, \"fetch.writecommitgraph\", &r->settings.fetch_write_commit_graph, 0);\n> -\trepo_cfg_bool(r, \"pack.usesparse\", &r->settings.pack_use_sparse, 1);\n> -\trepo_cfg_bool(r, \"core.multipackindex\", &r->settings.core_multi_pack_index, 1);\n> -\trepo_cfg_bool(r, \"index.sparse\", &r->settings.sparse_index, 0);\n> +\trepo_cfg_bool(r, \"core.commitgraph\", &r->settings.core_commit_graph, 1, 0);\n> +\trepo_cfg_bool(r, \"commitgraph.readchangedpaths\", &r->settings.commit_graph_read_changed_paths, 1, 0);\n> +\trepo_cfg_bool(r, \"gc.writecommitgraph\", &r->settings.gc_write_commit_graph, 1, 0);\n> +\trepo_cfg_bool(r, \"fetch.writecommitgraph\", &r->settings.fetch_write_commit_graph, 0, 0);\n> +\trepo_cfg_bool(r, \"pack.usesparse\", &r->settings.pack_use_sparse, 1, 0);\n> +\trepo_cfg_bool(r, \"core.multipackindex\", &r->settings.core_multi_pack_index, 1, 1);\n> +\trepo_cfg_bool(r, \"index.sparse\", &r->settings.sparse_index, 0, 0);\n\nSo if I'm reading this right, this will only print the value of\n\"core.multipackindex\".\n\nYou can get this without the changes here using\n\nGIT_TRACE2_CONFIG_PARAMS=\"core.multipackindex,...any_other_values_of_interest...\"\nbefore running the command (or use the config setting) before running\nyour commandss.\n\nAlternatively, just emit a \"trace2_data_intmax()\" on the one value your\nwant to print here.\n\nJeff\n\n\n> \n> \t/*\n> \t * The GIT_TEST_MULTI_PACK_INDEX variable is special in that\n> \n"},{"id":"457711","messageId":"xmqqfsjw7mwe.fsf@gitster.g","threadId":"57611","inReplyTo":"4176c5af-43fd-b7a1-346c-4d4aa9dbd4ae@jeffhostetler.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-22T15:12:17Z","receivedAt":"2022-06-22T15:12:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff Hostetler writes:\n\n> (I just sent a response to your V2 before I saw your V3, so I'll\n> my response here so that it doesn't get lost.)\n>\n>\n> We should not be doing this. This would dump every repo-related\n> boolean value on every command. I see that in V3 that you have\n> a \"trace\" flag to control this. But again, this seems wrong here.\n\nI noticed that \"flag\" while merging this with another topic to\n\"seen\" and found it strange, too. It \"allows\" the caller to choose\nwhich one gets logged per variable, but in a very hard-coded way; we\nprobably would have been much better off to have a table of what\ngets logged and have repo_cfg_*() calls consult it. Even without\nconfigurability, at least it would move the hard-coded choice of\nwhat gets logged from code to data.\n\nBut ...\n\n> I already have a GIT_TRACE2_CONFIG_PARAMS and trace2.configparams\n> that will dump \"interesting\" config values to the trace2 log.\n> Just set one of them to a list of regex's. Look at the comment above\n> trace2_cmd_list_config() in trace2.h for details.\n\n... that does sound like the right way to go.\n\n> ...\n> So if I'm reading this right, this will only print the value of\n> \"core.multipackindex\".\n>\n> You can get this without the changes here using\n>\n> GIT_TRACE2_CONFIG_PARAMS=\"core.multipackindex,...any_other_values_of_interest...\"\n> before running the command (or use the config setting) before running\n> your commandss.\n\nThanks.\n"},{"id":"457765","messageId":"20220623093836.113288-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"78966fb1-d5be-2fb5-0f68-0fce0b93d249@jeffhostetler.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-23T09:38:36Z","receivedAt":"2022-06-23T09:38:47Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Wed, 22 Jun 2022 08:51:18 -0400, Jeff Hostetler wrote:\n\n> We should not be doing. This would dump every repo-related\n> boolean value on every command.\n\nYes. in v4 patch I use a new arg in repo_cfg_bool to decide\nwhether to trace or not, but after the above you mentioned, I\nthink it's still not a correct solution in my v4.\n\n> I already have a GIT_TRACE2_CONFIG_PARAMS and trace2.configparams\n> that will dump \"interesting\" config values to the trace2 log.\n> Just set one of them to a list of regex's. Look at the comment above\n> trace2_cmd_list_config() in trace2.h for details.\n\nThat's fresh for me, thanks for telling me that.\n\nI remove the changes of \"repo_config_bool\" from my v4 patch\nthen I try to know about \"GIT_TRACE2_CONFIG_PARAMS\" and\n\"trace2.configparams\". Thereby, when I:\n\nExecute \"GIT_TRACE2_PERF=1 GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex git rev-list --test-bitmap HEAD\"\n\n15:21:38.812782 git.c:461 | d0 | main | def_param | | | | | core.multipackindex:false\n15:21:38.812797 git.c:461 | d0 | main | def_param | | | | | core.multipackindex:false\n\nI checked my configs, I found if there exists multiple level configs.\nit'll print multiple times. Like If I config all the global, system\nand local on \"core.multipackIndex=false\" , the output will be:\n\n15:41:50.614108 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n15:41:50.614123 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n15:41:50.614136 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n\nAnd if I modified the local scope of core.multipackIndex to \"true\",\nthe output will be:\n\n15:45:39.200172 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n15:45:39.200186 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n15:45:39.200200 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:true\n\nI'm not sure it's an intentional design or here should be only\nprint the final value that takes effect or should print all the\nvalues if config multiple scopes on the same config.\n\nHence, I made a temporary patch below to try to add some\nidentifying information to know why we output these lines, like:\n\n\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex c5c8cfbbaa..37a3163be1 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n const char *event_name = \"def_param\";\n struct json_writer jw = JSON_WRITER_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n \ncommit a089800b9dbc93a5dff9a46da7e66111e6e4343e (HEAD -> master, dyrone/master, dyrone/HEAD)\nAuthor: Teng Long \nDate: Thu Jun 23 17:24:15 2022 +0800\n\n tr2: append scope info when same configs exist in multiple scopes\n \n Signed-off-by: Teng Long \n\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex c5c8cfbbaa..37a3163be1 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n const char *event_name = \"def_param\";\n struct json_writer jw = JSON_WRITER_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n \n struct json_writer jw = JSON_WRITER_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n \n jw_object_begin(&jw, 0);\n event_fmt_prepare(event_name, file, line, NULL, &jw);\n+ jw_object_string(&jw, \"scope\", scope_name);\n jw_object_string(&jw, \"param\", param);\n jw_object_string(&jw, \"value\", value);\n jw_end(&jw);\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex c42fbade7f..e1b036a60c 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -298,8 +298,10 @@ static void fn_param_fl(const char *file, int line, const char *param,\n const char *value)\n {\n struct strbuf buf_payload = STRBUF_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n \n- strbuf_addf(&buf_payload, \"def_param %s=%s\", param, value);\n+ strbuf_addf(&buf_payload, \"def_param scope:%s %s=%s\", scope_name, param, value);\n normal_io_write_fl(file, line, &buf_payload);\n strbuf_release(&buf_payload);\n }\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a1eff8bea3..c21bf8e651 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -441,10 +441,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n const char *event_name = \"def_param\";\n struct strbuf buf_payload = STRBUF_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n \n strbuf_addf(&buf_payload, \"%s:%s\", param, value);\n \n- perf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n\n\n----\nThe partial tr2 log looks like:\n\nPERF:\n17:00:35.933094 git.c:462 | d0 | main | def_param | | | | system | core.multipackindex:false\n17:00:35.933110 git.c:462 | d0 | main | def_param | | | | global | core.multipackindex:false\n17:00:35.933128 git.c:462 | d0 | main | def_param | | | | local | core.multipackindex:true\n\nNORMAL:\n17:14:32.905359 git.c:462 def_param scope:system core.multipackindex=false\n17:14:32.905370 git.c:462 def_param scope:global core.multipackindex=false\n17:14:32.905383 git.c:462 def_param scope:local core.multipackindex=true\n\nEVENT:\n{\"event\":\"def_param\",\"sid\":\"20220623T092115.703660Z-H82fddc29-P0001812a\",\"thread\":\"main\",\"time\":\"2022-06-23T09:21:15.703920Z\",\"file\":\"git.c\",\"line\":462,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n{\"event\":\"def_param\",\"sid\":\"20220623T092115.703660Z-H82fddc29-P0001812a\",\"thread\":\"main\",\"time\":\"2022-06-23T09:21:15.703936Z\",\"file\":\"git.c\",\"line\":462,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n{\"event\":\"def_param\",\"sid\":\"20220623T092115.703660Z-H82fddc29-P0001812a\",\"thread\":\"main\",\"time\":\"2022-06-23T09:21:15.703952Z\",\"file\":\"git.c\",\"line\":462,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"true\"}\n\n\n> We also have GIT_TRACE2_ENV_VARS and trace2.envvars that will dump\n> the values of \"interesting\" env vars.\n\n> You can use these in your testing to log the config and env var\n> values that you are interested in.\n\nWow, cool as GIT_TRACE2_CONFIG_PARAMS and trace2.configparams\nyou mentioned before. \n\nThanks.\n"},{"id":"457791","messageId":"a4dda985-a319-2c24-2957-5116831142b8@jeffhostetler.com","threadId":"57611","inReplyTo":"20220623093836.113288-1-dyroneteng@gmail.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\" file","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-06-23T15:14:27Z","receivedAt":"2022-06-23T15:14:35Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/23/22 5:38 AM, Teng Long wrote:\n> On Wed, 22 Jun 2022 08:51:18 -0400, Jeff Hostetler wrote:\n> \n[...]\n> I remove the changes of \"repo_config_bool\" from my v4 patch\n> then I try to know about \"GIT_TRACE2_CONFIG_PARAMS\" and\n> \"trace2.configparams\". Thereby, when I:\n> \n> Execute \"GIT_TRACE2_PERF=1 GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex git rev-list --test-bitmap HEAD\"\n> \n> 15:21:38.812782 git.c:461 | d0 | main | def_param | | | | | core.multipackindex:false\n> 15:21:38.812797 git.c:461 | d0 | main | def_param | | | | | core.multipackindex:false\n> \n> I checked my configs, I found if there exists multiple level configs.\n> it'll print multiple times. Like If I config all the global, system\n> and local on \"core.multipackIndex=false\" , the output will be:\n> \n> 15:41:50.614108 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n> 15:41:50.614123 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n> 15:41:50.614136 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n> \n> And if I modified the local scope of core.multipackIndex to \"true\",\n> the output will be:\n> \n> 15:45:39.200172 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n> 15:45:39.200186 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:false\n> 15:45:39.200200 git.c:462 | d0 | main | def_param | | | | | core.multipackindex:true\n> \n> I'm not sure it's an intentional design or here should be only\n> print the final value that takes effect or should print all the\n> values if config multiple scopes on the same config.\n> \n> Hence, I made a temporary patch below to try to add some\n> identifying information to know why we output these lines, like:\n\nI had intended it to only print the final \"effective\" value\nthat the command would actually use. I don't remember if I\never experimented with config values that are set at multiple\nlevels, but testing it now shows that you're right it does\nprint a \"def_param\" message for each level.\n\nI would suggest we fix it to only print the final value\nsince we really just want to know whether the command will\nor will not use a feature. The inheritance is somewhat\nirrelevant.\n\nBut the change to do that may be quite a mess, so printing\nthe scope name as you have here is fine. And may help with\nsupport when debugging users having confusing problems, so\nI'm fine with it.\n\nThanks,\nJeff\n\n"},{"id":"457835","messageId":"20220624082750.90840-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"a4dda985-a319-2c24-2957-5116831142b8@jeffhostetler.com","subject":"Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open \"bitmap\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-24T08:27:50Z","receivedAt":"2022-06-24T08:28:18Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Nn Thu, 23 Jun 2022 11:14:27 -0400, Jeff Hostetler wrote:\n\n> I had intended it to only print the final \"effective\" value\n> that the command would actually use. I don't remember if I\n> ever experimented with config values that are set at multiple\n> levels, but testing it now shows that you're right it does\n> print a \"def_param\" message for each level.\n> \n> I would suggest we fix it to only print the final value\n> since we really just want to know whether the command will\n> or will not use a feature. The inheritance is somewhat\n> irrelevant.\n> \n> But the change to do that may be quite a mess, so printing\n> the scope name as you have here is fine. And may help with\n> support when debugging users having confusing problems, so\n> I'm fine with it.\n\nYes, as I replied both way is OK for me, but sorry for only\nreplied the append-scope way. I will post a new patch v5\nand include the related changes to fix this. \n\nThanks.\n"},{"id":"457992","messageId":"cover.1656403084.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1655817253.git.dyroneteng@gmail.com","subject":"[PATCH v5 0/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-28T08:17:45Z","receivedAt":"2022-06-28T08:19:31Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Changes since v4:\n\nBy Jeff Hostetler's suggestion, we can use GIT_TRACE2_CONFIG_PARAMS and\ntrace2.configparams to dump \"interesting\" config values to the trace2 log.\nSo I drop the commit [5/5] 'bitmap: add trace2 outputs during open \"bitmap\"\nfile' in v4.\n\nThen I found if a config key exist multiple config values in different scope\nthe trace2 log will print the value repeatly but not only print the final\nactive value, so a new commit \"avoid to print \"interesting\" config repeatedly\"\nis appended in v5.\n\nFinally, compared with [3/5] in v4, commit \"pack-bitmap.c: retrieve missing\ni18n translations\" fix every place which was missing translation in pack-bitmap.c.\n\nThanks.\n\nTeng Long (5):\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: using error() instead of silently returning -1\n pack-bitmap.c: retrieve missing i18n translations\n tr2: avoid to print \"interesting\" config repeatedly\n\n pack-bitmap.c | 105 ++++++++++++++++++++++++-----------------------\n trace2/tr2_cfg.c | 9 +++-\n 2 files changed, 61 insertions(+), 53 deletions(-)\n\nRange-diff against v4:\n1: d8dfe53dd4 < -: ---------- pack-bitmap.c: make warnings support i18N when opening bitmap\n-: ---------- > 1: 589e3f4075 pack-bitmap.c: continue looping when first MIDX bitmap is found\n-: ---------- > 2: b6b30047fc pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n2: 917551f2b5 = 3: 82d4493a6e pack-bitmap.c: using error() instead of silently returning -1\n3: 8735ae9979 < -: ---------- bitmap: add trace2 outputs during open \"bitmap\" file\n-: ---------- > 4: 065b7c9ccb pack-bitmap.c: retrieve missing i18n translations\n-: ---------- > 5: f3b87a33da tr2: avoid to print \"interesting\" config repeatedly\n-- \n2.35.1.582.gf3b87a33da\n\n"},{"id":"457993","messageId":"589e3f4075513b13dcd057242fa207b95371b114.1656403084.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1656403084.git.dyroneteng@gmail.com","subject":"[PATCH v5 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-28T08:17:46Z","receivedAt":"2022-06-28T08:19:37Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to continue the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe reason for this is, there may exist more than one MIDX file in\na repo. The \"multi_pack_index\" struct is actually designed as a singly\nlinked list, and if a MIDX file has been already opened successfully,\nthen the other MIDX files will be skipped and left with a warning\n\"ignoring extra bitmap file.\" to the output.\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..112c2b12c6 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -494,15 +494,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.1.582.gf3b87a33da\n\n"},{"id":"457994","messageId":"b6b30047fc92933d254ee2034b946a10ac37bb31.1656403084.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1656403084.git.dyroneteng@gmail.com","subject":"[PATCH v5 2/5] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-28T08:17:47Z","receivedAt":"2022-06-28T08:19:49Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index,\nlet's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 112c2b12c6..f13a6bfe3a 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -313,10 +313,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -368,14 +368,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.1.582.gf3b87a33da\n\n"},{"id":"457995","messageId":"82d4493a6ee7d18063e0feb72d0bc1cc450f2682.1656403084.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1656403084.git.dyroneteng@gmail.com","subject":"[PATCH v5 3/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-28T08:17:48Z","receivedAt":"2022-06-28T08:19:50Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex f13a6bfe3a..9f60dbf282 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -323,7 +323,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -361,7 +361,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tbitmap_git->map_pos = 0;\n \tbitmap_git->map = NULL;\n \tbitmap_git->midx = NULL;\n-\treturn -1;\n+\treturn error(_(\"cannot open midx bitmap file\"));\n }\n \n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n@@ -382,7 +382,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -394,7 +394,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (!is_pack_valid(packfile)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error(_(\"packfile is invalid\"));\n \t}\n \n \tbitmap_git->pack = packfile;\n@@ -409,7 +409,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \t\tbitmap_git->map_size = 0;\n \t\tbitmap_git->map_pos = 0;\n \t\tbitmap_git->pack = NULL;\n-\t\treturn -1;\n+\t\treturn error(_(\"bitmap header is invalid\"));\n \t}\n \n \treturn 0;\n-- \n2.35.1.582.gf3b87a33da\n\n"},{"id":"457996","messageId":"065b7c9ccb5a412526a934f9b67c8be64a40fc7f.1656403084.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1656403084.git.dyroneteng@gmail.com","subject":"[PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-28T08:17:49Z","receivedAt":"2022-06-28T08:19:52Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In pack-bitmap.c, some printed texts are translated,some are not.\nLet's support the translations of the bitmap related output.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 76 +++++++++++++++++++++++++--------------------------\n 1 file changed, 38 insertions(+), 38 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9f60dbf282..922b9cbc54 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -138,7 +138,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"Failed to load bitmap index (corrupted?)\");\n+\t\terror(_(\"Failed to load bitmap index (corrupted?)\"));\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -160,14 +160,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"Corrupted bitmap index (too small)\");\n+\t\treturn error(_(\"Corrupted bitmap index (too small)\"));\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"Corrupted bitmap index file (wrong header)\");\n+\t\treturn error(_(\"Corrupted bitmap index file (wrong header)\"));\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"Unsupported version for bitmap index file (%d)\", index->version);\n+\t\treturn error(_(\"Unsupported version for bitmap index file (%d)\"), index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -176,12 +176,12 @@ static int load_bitmap_header(struct bitmap_index *index)\n \t\tunsigned char *index_end = index->map + index->map_size - the_hash_algo->rawsz;\n \n \t\tif ((flags & BITMAP_OPT_FULL_DAG) == 0)\n-\t\t\treturn error(\"Unsupported options for bitmap index file \"\n-\t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n+\t\t\treturn error(_(\"Unsupported options for bitmap index file \"\n+\t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\"));\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n \t\t\tif (cache_size > index_end - index->map - header_size)\n-\t\t\t\treturn error(\"corrupted bitmap index file (too short to fit hash cache)\");\n+\t\t\t\treturn error(_(\"corrupted bitmap index file (too short to fit hash cache)\"));\n \t\t\tindex->hashes = (void *)(index_end - cache_size);\n \t\t\tindex_end -= cache_size;\n \t\t}\n@@ -215,7 +215,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"Duplicate entry in bitmap index: %s\", oid_to_hex(oid));\n+\t\terror(_(\"Duplicate entry in bitmap index: %s\"), oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -259,14 +259,14 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\tstruct object_id oid;\n \n \t\tif (index->map_size - index->map_pos < 6)\n-\t\t\treturn error(\"corrupt ewah bitmap: truncated header for entry %d\", i);\n+\t\t\treturn error(_(\"corrupt ewah bitmap: truncated header for entry %d)\"), i);\n \n \t\tcommit_idx_pos = read_be32(index->map, &index->map_pos);\n \t\txor_offset = read_u8(index->map, &index->map_pos);\n \t\tflags = read_u8(index->map, &index->map_pos);\n \n \t\tif (nth_bitmap_object_oid(index, &oid, commit_idx_pos) < 0)\n-\t\t\treturn error(\"corrupt ewah bitmap: commit index %u out of range\",\n+\t\t\treturn error(_(\"corrupt ewah bitmap: commit index %u out of range\"),\n \t\t\t\t (unsigned)commit_idx_pos);\n \n \t\tbitmap = read_bitmap_1(index);\n@@ -274,13 +274,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"Corrupted bitmap pack index\");\n+\t\t\treturn error(_(\"Corrupted bitmap pack index\"));\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"Invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(_(\"Invalid XOR offset in bitmap pack index\"));\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -305,7 +305,7 @@ char *pack_bitmap_filename(struct packed_git *p)\n \tsize_t len;\n \n \tif (!strip_suffix(p->pack_name, \".pack\", &len))\n-\t\tBUG(\"pack_name does not end in .pack\");\n+\t\tBUG(_(\"pack_name does not end in .pack\"));\n \treturn xstrfmt(\"%.*s.bitmap\", (int)len, p->pack_name);\n }\n \n@@ -330,7 +330,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +387,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -819,7 +819,7 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,\n \t\trevs->include_check_data = &incdata;\n \n \t\tif (prepare_revision_walk(revs))\n-\t\t\tdie(\"revision walk setup failed\");\n+\t\t\tdie(_(\"revision walk setup failed\"));\n \n \t\tshow_data.bitmap_git = bitmap_git;\n \t\tshow_data.base = base;\n@@ -1134,7 +1134,7 @@ static void filter_bitmap_tree_depth(struct bitmap_index *bitmap_git,\n \t\t\t\t unsigned long limit)\n {\n \tif (limit)\n-\t\tBUG(\"filter_bitmap_tree_depth given non-zero limit\");\n+\t\tBUG(_(\"filter_bitmap_tree_depth given non-zero limit\"));\n \n \tfilter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter,\n \t\t\t\t OBJ_TREE);\n@@ -1148,7 +1148,7 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n \t\t\t\t enum object_type object_type)\n {\n \tif (object_type < OBJ_COMMIT || object_type > OBJ_TAG)\n-\t\tBUG(\"filter_bitmap_object_type given invalid object\");\n+\t\tBUG(_(\"filter_bitmap_object_type given invalid object\"));\n \n \tif (object_type != OBJ_TAG)\n \t\tfilter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TAG);\n@@ -1304,14 +1304,14 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,\n \t\trevs->ignore_missing_links = 0;\n \n \t\tif (haves_bitmap == NULL)\n-\t\t\tBUG(\"failed to perform bitmap walk\");\n+\t\t\tBUG(_(\"failed to perform bitmap walk\"));\n \t}\n \n \twants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap,\n \t\t\t\t filter);\n \n \tif (!wants_bitmap)\n-\t\tBUG(\"failed to perform bitmap walk\");\n+\t\tBUG(_(\"failed to perform bitmap walk\"));\n \n \tif (haves_bitmap)\n \t\tbitmap_and_not(wants_bitmap, haves_bitmap);\n@@ -1432,7 +1432,7 @@ uint32_t midx_preferred_pack(struct bitmap_index *bitmap_git)\n {\n \tstruct multi_pack_index *m = bitmap_git->midx;\n \tif (!m)\n-\t\tBUG(\"midx_preferred_pack: requires non-empty MIDX\");\n+\t\tBUG(_(\"midx_preferred_pack: requires non-empty MIDX\"));\n \treturn nth_midxed_pack_int_id(m, pack_pos_to_midx(bitmap_git->midx, 0));\n }\n \n@@ -1629,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object %s not found in type bitmaps\",\n+\t\tdie(_(\"object %s not found in type bitmaps\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object %s does not have a unique type\",\n+\t\tdie(_(\"object %s does not have a unique type\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object %s: real type %s, expected: %s\",\n+\t\tdie(_(\"object %s: real type %s, expected: %s\"),\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1651,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&object->oid));\n+\t\tdie(_(\"Object not in bitmap: %s\\n\"), oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1666,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&commit->object.oid));\n+\t\tdie(_(\"Object not in bitmap: %s\\n\"), oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1683,26 +1683,26 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstruct ewah_bitmap *bm;\n \n \tif (!(bitmap_git = prepare_bitmap_git(revs->repo)))\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tif (revs->pending.nr != 1)\n-\t\tdie(\"you must specify exactly one commit to test\");\n+\t\tdie(_(\"you must specify exactly one commit to test\"));\n \n-\tfprintf(stderr, \"Bitmap v%d test (%d entries loaded)\\n\",\n+\tfprintf(stderr, _(\"Bitmap v%d test (%d entries loaded)\\n\"),\n \t\tbitmap_git->version, bitmap_git->entry_count);\n \n \troot = revs->pending.objects[0].item;\n \tbm = bitmap_for_commit(bitmap_git, (struct commit *)root);\n \n \tif (bm) {\n-\t\tfprintf(stderr, \"Found bitmap for %s. %d bits / %08x checksum\\n\",\n+\t\tfprintf(stderr, _(\"Found bitmap for %s. %d bits / %08x checksum\\n\"),\n \t\t\toid_to_hex(&root->oid), (int)bm->bit_size, ewah_checksum(bm));\n \n \t\tresult = ewah_to_bitmap(bm);\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"Commit %s doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(_(\"Commit %s doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1711,7 +1711,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tresult_popcnt = bitmap_popcount(result);\n \n \tif (prepare_revision_walk(revs))\n-\t\tdie(\"revision walk setup failed\");\n+\t\tdie(_(\"revision walk setup failed\"));\n \n \ttdata.bitmap_git = bitmap_git;\n \ttdata.base = bitmap_new();\n@@ -1719,7 +1719,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \ttdata.trees = ewah_to_bitmap(bitmap_git->trees);\n \ttdata.blobs = ewah_to_bitmap(bitmap_git->blobs);\n \ttdata.tags = ewah_to_bitmap(bitmap_git->tags);\n-\ttdata.prg = start_progress(\"Verifying bitmap entries\", result_popcnt);\n+\ttdata.prg = start_progress(_(\"Verifying bitmap entries\"), result_popcnt);\n \ttdata.seen = 0;\n \n \ttraverse_commit_list(revs, &test_show_commit, &test_show_object, &tdata);\n@@ -1727,9 +1727,9 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstop_progress(&tdata.prg);\n \n \tif (bitmap_equals(result, tdata.base))\n-\t\tfprintf(stderr, \"OK!\\n\");\n+\t\tfprintf(stderr, _(\"OK!\\n\"));\n \telse\n-\t\tdie(\"mismatch in bitmap results\");\n+\t\tdie(_(\"mismatch in bitmap results\"));\n \n \tbitmap_free(result);\n \tbitmap_free(tdata.base);\n@@ -1747,7 +1747,7 @@ int test_bitmap_commits(struct repository *r)\n \tMAYBE_UNUSED void *value;\n \n \tif (!bitmap_git)\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n \t\tprintf(\"%s\\n\", oid_to_hex(&oid));\n@@ -1825,8 +1825,8 @@ uint32_t *create_bitmap_mapping(struct bitmap_index *bitmap_git,\n \tif (!bitmap_is_midx(bitmap_git))\n \t\tload_reverse_index(bitmap_git);\n \telse if (load_midx_revindex(bitmap_git->midx) < 0)\n-\t\tBUG(\"rebuild_existing_bitmaps: missing required rev-cache \"\n-\t\t \"extension\");\n+\t\tBUG(_(\"rebuild_existing_bitmaps: missing required rev-cache \"\n+\t\t \"extension\"));\n \n \tnum_objects = bitmap_num_objects(bitmap_git);\n \tCALLOC_ARRAY(reposition, num_objects);\n-- \n2.35.1.582.gf3b87a33da\n\n"},{"id":"457997","messageId":"f3b87a33da2a9bd2cd5148fcfe1d55a6281d8b99.1656403084.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1656403084.git.dyroneteng@gmail.com","subject":"[PATCH v5 5/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-06-28T08:17:50Z","receivedAt":"2022-06-28T08:19:55Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"We can use GIT_TRACE2_CONFIG_PARAMS and trace2.configparams to\ndump the config which we are inteseted in to the tr2 log. If\nan \"interesting\" config exists in multiple scopes, it will be\ndumped multiple times. So, let's fix it to only print the\nfinal value instead.\n\nSigned-off-by: Teng Long \nHelped-by: Jeff Hostetler \n---\n trace2/tr2_cfg.c | 9 ++++++++-\n 1 file changed, 8 insertions(+), 1 deletion(-)\n\ndiff --git a/trace2/tr2_cfg.c b/trace2/tr2_cfg.c\nindex ec9ac1a6ef..632bb6feec 100644\n--- a/trace2/tr2_cfg.c\n+++ b/trace2/tr2_cfg.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"config.h\"\n+#include \"strmap.h\"\n #include \"trace2/tr2_cfg.h\"\n #include \"trace2/tr2_sysenv.h\"\n \n@@ -10,6 +11,7 @@ static int tr2_cfg_loaded;\n static struct strbuf **tr2_cfg_env_vars;\n static int tr2_cfg_env_vars_count;\n static int tr2_cfg_env_vars_loaded;\n+static struct strset tr_cfg_set = STRSET_INIT;\n \n /*\n * Parse a string containing a comma-delimited list of config keys\n@@ -101,12 +103,17 @@ static int tr2_cfg_cb(const char *key, const char *value, void *d)\n {\n \tstruct strbuf **s;\n \tstruct tr2_cfg_data *data = (struct tr2_cfg_data *)d;\n+\tconst char *prior_value;\n \n \tfor (s = tr2_cfg_patterns; *s; s++) {\n \t\tstruct strbuf *buf = *s;\n \t\tint wm = wildmatch(buf->buf, key, WM_CASEFOLD);\n \t\tif (wm == WM_MATCH) {\n-\t\t\ttrace2_def_param_fl(data->file, data->line, key, value);\n+\t\t\tif (strset_contains(&tr_cfg_set, key)\n+\t\t\t || git_config_get_value(key, &prior_value))\n+\t\t\t\tcontinue;\n+\t\t\ttrace2_def_param_fl(data->file, data->line, key, prior_value);\n+\t\t\tstrset_add(&tr_cfg_set, key);\n \t\t\treturn 0;\n \t\t}\n \t}\n-- \n2.35.1.582.gf3b87a33da\n\n"},{"id":"458001","messageId":"220628.86sfnpf96x.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"065b7c9ccb5a412526a934f9b67c8be64a40fc7f.1656403084.git.dyroneteng@gmail.com","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-06-28T08:58:56Z","receivedAt":"2022-06-28T09:07:08Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jun 28 2022, Teng Long wrote:\n\n> In pack-bitmap.c, some printed texts are translated,some are not.\n> Let's support the translations of the bitmap related output.\n\nUsually we don't go for cleanup-while-at-it, but in this case we're\nmarking messages that don't conform to our CodingGudielines for\ntranslation, mostly because they're error messages that start with an\nupper-case letter.\n\nSo I think we should fix those issues first, to avoid double-work for\ntranslators (well, a bit less, since they're the translation memory, but\nit's quite a bit of churn...).\n\n> -\t\terror(\"Failed to load bitmap index (corrupted?)\");\n> +\t\terror(_(\"Failed to load bitmap index (corrupted?)\"));\n\ne.g. here.\n\n> -\t\treturn error(\"Corrupted bitmap index (too small)\");\n> +\t\treturn error(_(\"Corrupted bitmap index (too small)\"));\n\n..and here, etc. etc.\n\n> -\t\treturn error(\"Unsupported version for bitmap index file (%d)\", index->version);\n> +\t\treturn error(_(\"Unsupported version for bitmap index file (%d)\"), index->version);\n\nLet's say \"unsupported version '%d' for ...\" instead?\n\n> -\t\t\treturn error(\"Unsupported options for bitmap index file \"\n> -\t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n> +\t\t\treturn error(_(\"Unsupported options for bitmap index file \"\n> +\t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\"));\n\nI'm not sure, but shouldn't this be a BUG()?\n\n> -\t\terror(\"Duplicate entry in bitmap index: %s\", oid_to_hex(oid));\n> +\t\terror(_(\"Duplicate entry in bitmap index: %s\"), oid_to_hex(oid));\n\nDitto upper-case, but add '%s' while at it.\n\n> \tif (!strip_suffix(p->pack_name, \".pack\", &len))\n> -\t\tBUG(\"pack_name does not end in .pack\");\n> +\t\tBUG(_(\"pack_name does not end in .pack\"));\n\nDo not translate BUG() messages.\n\n> -\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n> +\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n\nQuote the name.\n\n> -\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n> +\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n\nditto.\n\n> \t\tif (prepare_revision_walk(revs))\n> -\t\t\tdie(\"revision walk setup failed\");\n> +\t\t\tdie(_(\"revision walk setup failed\"));\n\nLooks good, but aside from this we should really have a tree-wide\nxprepare_revision_walk() or something, we have this copy/pasted all over\nthe place...\n\n\n> -\t\tBUG(\"filter_bitmap_tree_depth given non-zero limit\");\n> +\t\tBUG(_(\"filter_bitmap_tree_depth given non-zero limit\"));\n\nDitto BUG.\n> \n> \tfilter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter,\n> \t\t\t\t OBJ_TREE);\n> @@ -1148,7 +1148,7 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n> \t\t\t\t enum object_type object_type)\n> {\n> \tif (object_type < OBJ_COMMIT || object_type > OBJ_TAG)\n> -\t\tBUG(\"filter_bitmap_object_type given invalid object\");\n> +\t\tBUG(_(\"filter_bitmap_object_type given invalid object\"));\n> \n> \tif (object_type != OBJ_TAG)\n> \t\tfilter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TAG);\n> @@ -1304,14 +1304,14 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,\n> \t\trevs->ignore_missing_links = 0;\n> \n> \t\tif (haves_bitmap == NULL)\n> -\t\t\tBUG(\"failed to perform bitmap walk\");\n> +\t\t\tBUG(_(\"failed to perform bitmap walk\"));\n> \t}\n\netc. etc.\n\n> \n> \twants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap,\n> \t\t\t\t filter);\n> \n> \tif (!wants_bitmap)\n> -\t\tBUG(\"failed to perform bitmap walk\");\n> +\t\tBUG(_(\"failed to perform bitmap walk\"));\n> \n> \tif (haves_bitmap)\n> \t\tbitmap_and_not(wants_bitmap, haves_bitmap);\n> @@ -1432,7 +1432,7 @@ uint32_t midx_preferred_pack(struct bitmap_index *bitmap_git)\n> {\n> \tstruct multi_pack_index *m = bitmap_git->midx;\n> \tif (!m)\n> -\t\tBUG(\"midx_preferred_pack: requires non-empty MIDX\");\n> +\t\tBUG(_(\"midx_preferred_pack: requires non-empty MIDX\"));\n\netc. etc.\n\n> \treturn nth_midxed_pack_int_id(m, pack_pos_to_midx(bitmap_git->midx, 0));\n> }\n> \n> @@ -1629,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n> \t}\n> \n> \tif (bitmap_type == OBJ_NONE)\n> -\t\tdie(\"object %s not found in type bitmaps\",\n> +\t\tdie(_(\"object %s not found in type bitmaps\"),\n> \t\t oid_to_hex(&obj->oid));\n> \n> \tif (bitmaps_nr > 1)\n> -\t\tdie(\"object %s does not have a unique type\",\n> +\t\tdie(_(\"object %s does not have a unique type\"),\n> \t\t oid_to_hex(&obj->oid));\n> \n> \tif (bitmap_type != obj->type)\n> -\t\tdie(\"object %s: real type %s, expected: %s\",\n> +\t\tdie(_(\"object %s: real type %s, expected: %s\"),\n> \t\t oid_to_hex(&obj->oid),\n> \t\t type_name(obj->type),\n> \t\t type_name(bitmap_type));\n\nquote %s for these.\n\n> @@ -1651,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n> \n> \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n> \tif (bitmap_pos < 0)\n> -\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&object->oid));\n> +\t\tdie(_(\"Object not in bitmap: %s\\n\"), oid_to_hex(&object->oid));\n\nLose the \\n here, in addition to lower-case & quote %s.\n\n> \ttest_bitmap_type(tdata, object, bitmap_pos);\n> \n> \tbitmap_set(tdata->base, bitmap_pos);\n> @@ -1666,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n> \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n> \t\t\t\t &commit->object.oid);\n> \tif (bitmap_pos < 0)\n> -\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&commit->object.oid));\n> +\t\tdie(_(\"Object not in bitmap: %s\\n\"), oid_to_hex(&commit->object.oid));\n\nDitto.\n\n> \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n> \n> \tbitmap_set(tdata->base, bitmap_pos);\n> @@ -1683,26 +1683,26 @@ void test_bitmap_walk(struct rev_info *revs)\n> \tstruct ewah_bitmap *bm;\n> \n> \tif (!(bitmap_git = prepare_bitmap_git(revs->repo)))\n> -\t\tdie(\"failed to load bitmap indexes\");\n> +\t\tdie(_(\"failed to load bitmap indexes\"));\n> \n> \tif (revs->pending.nr != 1)\n> -\t\tdie(\"you must specify exactly one commit to test\");\n> +\t\tdie(_(\"you must specify exactly one commit to test\"));\n> \n> -\tfprintf(stderr, \"Bitmap v%d test (%d entries loaded)\\n\",\n> +\tfprintf(stderr, _(\"Bitmap v%d test (%d entries loaded)\\n\"),\n> \t\tbitmap_git->version, bitmap_git->entry_count);\n> \n> \troot = revs->pending.objects[0].item;\n> \tbm = bitmap_for_commit(bitmap_git, (struct commit *)root);\n> \n> \tif (bm) {\n> -\t\tfprintf(stderr, \"Found bitmap for %s. %d bits / %08x checksum\\n\",\n> +\t\tfprintf(stderr, _(\"Found bitmap for %s. %d bits / %08x checksum\\n\"),\n> \t\t\toid_to_hex(&root->oid), (int)bm->bit_size, ewah_checksum(bm));\n> \n> \t\tresult = ewah_to_bitmap(bm);\n> \t}\n> \n> \tif (result == NULL)\n> -\t\tdie(\"Commit %s doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n> +\t\tdie(_(\"Commit %s doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n> \n> \trevs->tag_objects = 1;\n> \trevs->tree_objects = 1;\n> @@ -1711,7 +1711,7 @@ void test_bitmap_walk(struct rev_info *revs)\n> \tresult_popcnt = bitmap_popcount(result);\n> \n> \tif (prepare_revision_walk(revs))\n> -\t\tdie(\"revision walk setup failed\");\n> +\t\tdie(_(\"revision walk setup failed\"));\n> \n> \ttdata.bitmap_git = bitmap_git;\n> \ttdata.base = bitmap_new();\n> @@ -1719,7 +1719,7 @@ void test_bitmap_walk(struct rev_info *revs)\n> \ttdata.trees = ewah_to_bitmap(bitmap_git->trees);\n> \ttdata.blobs = ewah_to_bitmap(bitmap_git->blobs);\n> \ttdata.tags = ewah_to_bitmap(bitmap_git->tags);\n> -\ttdata.prg = start_progress(\"Verifying bitmap entries\", result_popcnt);\n> +\ttdata.prg = start_progress(_(\"Verifying bitmap entries\"), result_popcnt);\n\nGood catch!\n> \ttdata.seen = 0;\n> \n> \ttraverse_commit_list(revs, &test_show_commit, &test_show_object, &tdata);\n> @@ -1727,9 +1727,9 @@ void test_bitmap_walk(struct rev_info *revs)\n> \tstop_progress(&tdata.prg);\n> \n> \tif (bitmap_equals(result, tdata.base))\n> -\t\tfprintf(stderr, \"OK!\\n\");\n> +\t\tfprintf(stderr, _(\"OK!\\n\"));\n\nDitto don't include \\n.\n> \telse\n> -\t\tdie(\"mismatch in bitmap results\");\n> +\t\tdie(_(\"mismatch in bitmap results\"));\n> \n> \tbitmap_free(result);\n> \tbitmap_free(tdata.base);\n> @@ -1747,7 +1747,7 @@ int test_bitmap_commits(struct repository *r)\n> \tMAYBE_UNUSED void *value;\n> \n> \tif (!bitmap_git)\n> -\t\tdie(\"failed to load bitmap indexes\");\n> +\t\tdie(_(\"failed to load bitmap indexes\"));\n> \n> \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n> \t\tprintf(\"%s\\n\", oid_to_hex(&oid));\n> @@ -1825,8 +1825,8 @@ uint32_t *create_bitmap_mapping(struct bitmap_index *bitmap_git,\n> \tif (!bitmap_is_midx(bitmap_git))\n> \t\tload_reverse_index(bitmap_git);\n> \telse if (load_midx_revindex(bitmap_git->midx) < 0)\n> -\t\tBUG(\"rebuild_existing_bitmaps: missing required rev-cache \"\n> -\t\t \"extension\");\n> +\t\tBUG(_(\"rebuild_existing_bitmaps: missing required rev-cache \"\n> +\t\t \"extension\"));\n> \n\nDitto don't translate BUG().\n"},{"id":"458004","messageId":"220628.86o7ydf8gi.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"f3b87a33da2a9bd2cd5148fcfe1d55a6281d8b99.1656403084.git.dyroneteng@gmail.com","subject":"Re: [PATCH v5 5/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-06-28T09:13:21Z","receivedAt":"2022-06-28T09:23:10Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jun 28 2022, Teng Long wrote:\n\n> We can use GIT_TRACE2_CONFIG_PARAMS and trace2.configparams to\n> dump the config which we are inteseted in to the tr2 log. If\n> an \"interesting\" config exists in multiple scopes, it will be\n> dumped multiple times. So, let's fix it to only print the\n> final value instead.\n>\n> Signed-off-by: Teng Long \n> Helped-by: Jeff Hostetler \n> ---\n> trace2/tr2_cfg.c | 9 ++++++++-\n> 1 file changed, 8 insertions(+), 1 deletion(-)\n>\n> diff --git a/trace2/tr2_cfg.c b/trace2/tr2_cfg.c\n> index ec9ac1a6ef..632bb6feec 100644\n> --- a/trace2/tr2_cfg.c\n> +++ b/trace2/tr2_cfg.c\n> @@ -1,5 +1,6 @@\n> #include \"cache.h\"\n> #include \"config.h\"\n> +#include \"strmap.h\"\n> #include \"trace2/tr2_cfg.h\"\n> #include \"trace2/tr2_sysenv.h\"\n> \n> @@ -10,6 +11,7 @@ static int tr2_cfg_loaded;\n> static struct strbuf **tr2_cfg_env_vars;\n> static int tr2_cfg_env_vars_count;\n> static int tr2_cfg_env_vars_loaded;\n> +static struct strset tr_cfg_set = STRSET_INIT;\n> \n> /*\n> * Parse a string containing a comma-delimited list of config keys\n> @@ -101,12 +103,17 @@ static int tr2_cfg_cb(const char *key, const char *value, void *d)\n> {\n> \tstruct strbuf **s;\n> \tstruct tr2_cfg_data *data = (struct tr2_cfg_data *)d;\n> +\tconst char *prior_value;\n> \n> \tfor (s = tr2_cfg_patterns; *s; s++) {\n> \t\tstruct strbuf *buf = *s;\n> \t\tint wm = wildmatch(buf->buf, key, WM_CASEFOLD);\n> \t\tif (wm == WM_MATCH) {\n> -\t\t\ttrace2_def_param_fl(data->file, data->line, key, value);\n> +\t\t\tif (strset_contains(&tr_cfg_set, key)\n> +\t\t\t || git_config_get_value(key, &prior_value))\n> +\t\t\t\tcontinue;\n> +\t\t\ttrace2_def_param_fl(data->file, data->line, key, prior_value);\n> +\t\t\tstrset_add(&tr_cfg_set, key);\n> \t\t\treturn 0;\n> \t\t}\n> \t}\n\n\nIs tr2_cfg_load_patterns() run at early startup, but this perhaps at\nruntime? I wonder if this is OK under threading, i.e. concurrent access\nto your strset.\n\nThe assumption you're making here doesn't hold in general, some config\nis \"last vaule wins\", but for some other config all configured values\nare important.\n\nDo we care in this case? I really don't know, but perhaps we can declare\n\"dedup\" using the same facility we're using to wildcard-match keys, and\neither make that optional or the default, e.g.:\n\n\tGIT_TRACE2_CONFIG_PARAMS='*:dedup,core.*:'\n\nI.e. to make it a list of [:].\n\nMaybe not worth the effort...\n"},{"id":"458053","messageId":"CAPig+cR9XL3dRuRrH9kCrqUqu5HwAJstgkp9eTJVBZ0TBZ+Wjw@mail.gmail.com","threadId":"57611","inReplyTo":"220628.86sfnpf96x.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-06-28T17:28:31Z","receivedAt":"2022-06-28T17:28:46Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jun 28, 2022 at 5:13 AM Ævar Arnfjörð Bjarmason\n wrote:\n> On Tue, Jun 28 2022, Teng Long wrote:\n> > if (bitmap_equals(result, tdata.base))\n> > - fprintf(stderr, \"OK!\\n\");\n> > + fprintf(stderr, _(\"OK!\\n\"));\n\nAs a minor additional bit of help, you can use fprintf_ln() from\nstrbuf.h which will automatically output the trailing \"\\n\":\n\n frpintf_ln(stderr, _(\"OK!\"));\n\n(Aside: Use of fprintf() here is a bit odd since there are no\nformatting directives in the argument, thus fputs() would have been a\nbetter choice, but that's a cleanup for another day and a different\npatch series, probably.)\n"},{"id":"458056","messageId":"xmqqmtdwk6li.fsf@gitster.g","threadId":"57611","inReplyTo":"82d4493a6ee7d18063e0feb72d0bc1cc450f2682.1656403084.git.dyroneteng@gmail.com","subject":"Re: [PATCH v5 3/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-28T18:04:09Z","receivedAt":"2022-06-28T18:04:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n\n\"stat\" -> \"fstat\" perhaps?\n\nNot a new problem, but before this hunk, we have\n\n\tfd = git_open(...);\n\tif (fd < 0)\n\t\treturn -1;\n\nwhere it probably is legitimate to ignore ENOENT silently. In\npractice, it may be OK to treat a file that exists but cannot be\nopened for whatever reason, but I do not think it would hurt to\nreport such a rare anomaly with a warning, e.g.\n\n\tif (fd < 0) {\n\t\tif (errno != ENOENT)\n\t\t\twarning(\"'%s' cannot open '%s'\",\n\t\t\t\tstrerror(errno), bitmap_name);\n\t\tfree(bitmap_name);\n\t\treturn -1;\n\t}\n\nor something like that perhaps.\n\n> @@ -361,7 +361,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \tbitmap_git->map_pos = 0;\n> \tbitmap_git->map = NULL;\n> \tbitmap_git->midx = NULL;\n> -\treturn -1;\n> +\treturn error(_(\"cannot open midx bitmap file\"));\n\nThis is not exactly \"cannot open\". We opened the file, and found\nthere was something we do not like in it. If we failed to find midx\nlacks revindex chunk, we already would have given a warning, and the\nerror is not just misleading but is redundant. load_bitmap_header()\nalso gives an error() on its own.\n\nI think this patch aims for a good end result, but needs more work.\nAt least, checksum mismatch that goes to cleanup also needs to issue\nits own error() and then we can remove this \"cannot open\" at the end.\n\n> @@ -382,7 +382,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error_errno(_(\"cannot stat bitmap file\"));\n\n\"stat\" -> \"fstat\"\n\n> @@ -394,7 +394,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \n> \tif (!is_pack_valid(packfile)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error(_(\"packfile is invalid\"));\n\nSame \"sometimes redundant\" comment applies here, but not due to this\npart of the code but due to the helpers called from is_pack_valid().\n\nNamely, packfile.c::open_packed_git_1() is mostly chatty, but is\nsilent upon \"unable to open\" and \"unable to fstat\" (which I think is\nsafe to make chatty as well---we do not want to special case ENOENT\nin open_packed_git(), so \"cannot open because it is not there\" is an\nerror).\n\nAnd then, this \"invalid\" will become redundant and fuzzier message\nthan what is_pack_valid() would have already given, so you can leave\nit to silently return -1 here.\n\n> @@ -409,7 +409,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \t\tbitmap_git->map_size = 0;\n> \t\tbitmap_git->map_pos = 0;\n> \t\tbitmap_git->pack = NULL;\n> -\t\treturn -1;\n> +\t\treturn error(_(\"bitmap header is invalid\"));\n\nHaving shown how to analize these kind of things twice in the above,\nI would not bother checking myself, but you should look at the\nexisting error messages from load_bitmap_header() and see if this\nextra message should really be here. I suspect not.\n\n> \t}\n> \n> \treturn 0;\n"},{"id":"458057","messageId":"xmqqfsjok6g1.fsf@gitster.g","threadId":"57611","inReplyTo":"065b7c9ccb5a412526a934f9b67c8be64a40fc7f.1656403084.git.dyroneteng@gmail.com","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-28T18:07:26Z","receivedAt":"2022-06-28T18:07:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> Subject: Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations\n\nThe verb \"retrieve\" is puzzling.\n\n> In pack-bitmap.c, some printed texts are translated,some are not.\n\n\",\" -> \", \".\n\n> Let's support the translations of the bitmap related output.\n\n> \tif (bitmap_size < 0) {\n> -\t\terror(\"Failed to load bitmap index (corrupted?)\");\n> +\t\terror(_(\"Failed to load bitmap index (corrupted?)\"));\n\nIf we were to do this, to avoid burdening translators with double\nwork, we probably would want to fix the \"C\" locale version of the\nstring, either as a preliminary clean-up before this step, or as\npart of this step. From Documentation/CodingGuidelines:\n\n Error Messages\n\n - Do not end error messages with a full stop.\n\n - Do not capitalize the first word, only because it is the first word\n in the message (\"unable to open %s\", not \"Unable to open %s\"). But\n \"SHA-3 not supported\" is fine, because the reason the first word is\n capitalized is not because it is at the beginning of the sentence,\n but because the word would be spelled in capital letters even when\n it appeared in the middle of the sentence.\n\n - Say what the error is first (\"cannot open %s\", not \"%s: cannot open\")\n\n"},{"id":"458058","messageId":"xmqqbkuck66t.fsf@gitster.g","threadId":"57611","inReplyTo":"220628.86o7ydf8gi.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 5/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-28T18:12:58Z","receivedAt":"2022-06-28T18:13:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Do we care in this case? I really don't know, but perhaps we can declare\n> \"dedup\" using the same facility we're using to wildcard-match keys, and\n> either make that optional or the default, e.g.:\n>\n> \tGIT_TRACE2_CONFIG_PARAMS='*:dedup,core.*:'\n>\n> I.e. to make it a list of [:].\n>\n> Maybe not worth the effort...\n\nI happen to think that the trace output is primarily for machine\nconsumption (i.e. if you want to make it readable by humans, it is\nOK to require them to pre-process).\n\nHow does this \"duplicate output\" come about? If it is because\nend-user configuration files have multiple entries that are either\nlist-valued or relying on last-one-wins semantics, I suspect that it\nis better not to prematurely filter these at the trace generation\nside (which by definition is an opration that loses information).\n\nSo, to me, it smells that the whole \"dedup at the source\" thing is\nnot just not worth the effort but is misguided.\n"},{"id":"458388","messageId":"838e1273-b704-bd11-e9ed-e25390989027@jeffhostetler.com","threadId":"57611","inReplyTo":"xmqqbkuck66t.fsf@gitster.g","subject":"Re: [PATCH v5 5/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-07-01T14:31:08Z","receivedAt":"2022-07-01T14:36:16Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/28/22 2:12 PM, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n> \n>> Do we care in this case? I really don't know, but perhaps we can declare\n>> \"dedup\" using the same facility we're using to wildcard-match keys, and\n>> either make that optional or the default, e.g.:\n>>\n>> \tGIT_TRACE2_CONFIG_PARAMS='*:dedup,core.*:'\n>>\n>> I.e. to make it a list of [:].\n>>\n>> Maybe not worth the effort...\n> \n> I happen to think that the trace output is primarily for machine\n> consumption (i.e. if you want to make it readable by humans, it is\n> OK to require them to pre-process).\n> \n> How does this \"duplicate output\" come about? If it is because\n> end-user configuration files have multiple entries that are either\n> list-valued or relying on last-one-wins semantics, I suspect that it\n> is better not to prematurely filter these at the trace generation\n> side (which by definition is an opration that loses information).\n> \n> So, to me, it smells that the whole \"dedup at the source\" thing is\n> not just not worth the effort but is misguided.\n> \n\nI agree. Let's not try to dedup these. IIRC, the tr2_cfg_cb()\nmachinery runs during a \"read early\" or \"read very early\" scan of\nthe config values and the program is still starting up (in some\nsense). For example, the command line may not have been fully\nprocessed yet (again, IIRC). So injecting a call here to force an \nexplicit lookup could cause problems.\n\nAnd you'll be arbitrarily taking the first value, which is probably\nthe system level setting rather than the last value, which is probably\nthe local setting. So the output would be misleading.\n\nAnd I don't think it is worth the effort. Let's just log the\ncontext (system, global, local) as you described in the previous\nversion and be happy that we get multiple-but-now-qualified values.\n\nThis was intended as a machine-readable feature to allow telemetry\npost-processing to group or filter command events. For example, if\nwe want to do some \"feature-x\" is ON or OFF and compute averages\nfor the ONs and averages for the OFFs and compare them. Such post-\nprocessing has (I'm assuming) in the past always looked at the last\nvalue logged. And I assume that that would not change here with the\nadditional qualification.\n\nAlso, your new qualification would help us answer support questions\nusing telemetry where a user thought they had a feature on (say\nglobally) but was actually off at another level (say locally).\n\nJeff\n"},{"id":"458470","messageId":"20220705090454.10686-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"xmqqmtdwk6li.fsf@gitster.g","subject":"Re: [PATCH v5 3/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-05T09:04:54Z","receivedAt":"2022-07-05T09:05:07Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\nOn 28 Jun 2022 11:04:09 -0700, Junio C Hamano wrote:\n\n> \"stat\" -> \"fstat\" perhaps?\n\nYes.\n\n> Not a new problem, but before this hunk, we have\n>\n> \tfd = git_open(...);\n> \tif (fd < 0)\n> \t\treturn -1;\n>\n> where it probably is legitimate to ignore ENOENT silently. In\n> practice, it may be OK to treat a file that exists but cannot be\n> opened for whatever reason, but I do not think it would hurt to\n> report such a rare anomaly with a warning, e.g.\n>\n> \tif (fd < 0) {\n> \t\tif (errno != ENOENT)\n> \t\t\twarning(\"'%s' cannot open '%s'\",\n> \t\t\t\tstrerror(errno), bitmap_name);\n> \t\tfree(bitmap_name);\n> \t\treturn -1;\n> \t}\n>\n> or something like that perhaps.\n\nYes.\n\nIt's more accurate here with your suggestion. At the same time I\nfound there actually exists many similar place like \"ignore ENOENT\nsilently\" in repo. And I think it's not worth to impove them right now\nin this patch, if you want to do that it could in another pathset and\nplease tell me.\n\n> > @@ -361,7 +361,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> > \tbitmap_git->map_pos = 0;\n> > \tbitmap_git->map = NULL;\n> > \tbitmap_git->midx = NULL;\n> > -\treturn -1;\n> > +\treturn error(_(\"cannot open midx bitmap file\"));\n>\n> This is not exactly \"cannot open\". We opened the file, and found\n> there was something we do not like in it. If we failed to find midx\n> lacks revindex chunk, we already would have given a warning, and the\n> error is not just misleading but is redundant. load_bitmap_header()\n> also gives an error() on its own.\n>\n> I think this patch aims for a good end result, but needs more work.\n> At least, checksum mismatch that goes to cleanup also needs to issue\n> its own error() and then we can remove this \"cannot open\" at the end.\n\nYes, It's detailed here and I missed it, I will fix this in next\npatch: return error() when checksum doesn't match and let \"cleanup\"\nreturn \"-1\" but not an inaccurate error().\n\n> \"stat\" -> \"fstat\"\n\nYes.\n\n> > @@ -394,7 +394,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> >\n> > \tif (!is_pack_valid(packfile)) {\n> > \t\tclose(fd);\n> > -\t\treturn -1;\n> > +\t\treturn error(_(\"packfile is invalid\"));\n>\n> Same \"sometimes redundant\" comment applies here, but not due to this\n> part of the code but due to the helpers called from is_pack_valid().\n\nLet me try to know about it, the \"helpers\" means the place where invoke\nis_pack_valid() like here. In fact, in is_pack_valid() they already\nreturn the errors in various scenarios, so it would be no need to\nreturn another error.\n\n> Namely, packfile.c::open_packed_git_1() is mostly chatty, but is\n> silent upon \"unable to open\" and \"unable to fstat\" (which I think is\n> safe to make chatty as well---we do not want to special case ENOENT\n> in open_packed_git(), so \"cannot open because it is not there\" is an\n> error).\n\n\"chatty\" means the code that regularly gives information about its\ninternal operations. Did I understand it right because I'm actually\nfirstly know this description.\n\n\"cannot open because it is not there\" is an error, but I think it will\nalso could be a BUG, actually I'm not very sure for clarify the\ndifference bwtween the use of the two, but I will look into it to dig\nsomething out.\n\n> And then, this \"invalid\" will become redundant and fuzzier message\n> than what is_pack_valid() would have already given, so you can leave\n> it to silently return -1 here.\n\nClear now.\n\nThanks.\n"},{"id":"458491","messageId":"xmqq8rp7ju4n.fsf@gitster.g","threadId":"57611","inReplyTo":"20220705090454.10686-1-dyroneteng@gmail.com","subject":"Re: [PATCH v5 3/5] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-05T18:23:52Z","receivedAt":"2022-07-05T18:24:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> It's more accurate here with your suggestion. At the same time I\n> found there actually exists many similar place like \"ignore ENOENT\n> silently\" in repo. And I think it's not worth to impove them right now\n> in this patch, if you want to do that it could in another pathset and\n> please tell me.\n\nIt is sufficien to just mark it as #leftoverbits to find and fix\nplaces where the code means to ignore only a missing optional file\nbut ignores all other errors it gets from open/fopen instead.\n\n>> > @@ -394,7 +394,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n>> >\n>> > \tif (!is_pack_valid(packfile)) {\n>> > \t\tclose(fd);\n>> > -\t\treturn -1;\n>> > +\t\treturn error(_(\"packfile is invalid\"));\n>>\n>> Same \"sometimes redundant\" comment applies here, but not due to this\n>> part of the code but due to the helpers called from is_pack_valid().\n>\n> Let me try to know about it, the \"helpers\" means the place where invoke\n> is_pack_valid() like here. In fact, in is_pack_valid() they already\n> return the errors in various scenarios, so it would be no need to\n> return another error.\n\nYup. is_pack_valid() calls open_packed_git() and the helper\nfunctions that are called from that call chain are full of calls to\nerror() that tell specifically what exactly went wrong. But ...\n\n>> Namely, packfile.c::open_packed_git_1() is mostly chatty, but is\n>> silent upon \"unable to open\" and \"unable to fstat\" (which I think is\n>> safe to make chatty as well---we do not want to special case ENOENT\n>> in open_packed_git(), so \"cannot open because it is not there\" is an\n>> error).\n\n... the error coverage is not complete. There are some (rare) code\npaths that silently \"return -1\", not \"return error(_(\"...\"))\". They\nshould be updated to say something; otherwise we will silently fail\nin these \"rare\" codepaths.\n\n> \"cannot open because it is not there\" is an error, but I think it will\n> also could be a BUG, actually I'm not very sure for clarify the\n> difference bwtween the use of the two, but I will look into it to dig\n> something out.\n\nThe code may have many reasons to believe that a file should exist\nthere and try to open it, but it may find the file missing, but I\nwould suspect that it is never a BUG. You may have run stat() on\nthe path earlier and you know it existed, but by the time you try to\nopen it, some other process may have removed it. You may have found\nthe .idx file and expect that the corresponding .pack file to exist,\nbut the .pack file may be missing. You may have just written a file\nand you expect to be able to open it, but some other process may\nhave removed it already, or you may have run out of file descriptors\nand cannot open it. These are all runtime errors that deserve to be\nreported via error() or die(), but never BUG().\n\nThanks.\n"},{"id":"458523","messageId":"20220706140603.57564-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220628.86sfnpf96x.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-06T14:06:03Z","receivedAt":"2022-07-06T14:06:14Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"> Usually we don't go for cleanup-while-at-it, but in this case we're\n> marking messages that don't conform to our CodingGudielines for\n> translation, mostly because they're error messages that start with an\n> upper-case letter.\n>\n> So I think we should fix those issues first, to avoid double-work for\n> translators (well, a bit less, since they're the translation memory, but\n> it's quite a bit of churn...).\n\nYes. I think it's need to make a cleanup commit first.\n\n> Lose the \\n here, in addition to lower-case & quote %s.\n> ...\n> Ditto don't include \\n.\n> ...\n> Ditto don't translate BUG().\n> ..\n\netc.Will fix.\n\nThanks.\n"},{"id":"458524","messageId":"20220706141923.58074-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"CAPig+cR9XL3dRuRrH9kCrqUqu5HwAJstgkp9eTJVBZ0TBZ+Wjw@mail.gmail.com","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-06T14:19:23Z","receivedAt":"2022-07-06T14:19:33Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"On Tue, 28 Jun 2022 13:28:31 -0400, Eric Sunshine wrote:\n\n> > On Tue, Jun 28 2022, Teng Long wrote:\n> > > if (bitmap_equals(result, tdata.base))\n> > > - fprintf(stderr, \"OK!\\n\");\n> > > + fprintf(stderr, _(\"OK!\\n\"));\n>\n> As a minor additional bit of help, you can use fprintf_ln() from\n> strbuf.h which will automatically output the trailing \"\\n\":\n\nYes, agree.\n\n> (Aside: Use of fprintf() here is a bit odd since there are no\n> formatting directives in the argument, thus fputs() would have been a\n> better choice, but that's a cleanup for another day and a different\n> patch series, probably.)\n\n*nod*, by the way, I think I will remove the translation on \"OK!\", currently I\nthink it doesnt mean much.\n\nThanks.\n"},{"id":"458561","messageId":"20220707115912.75774-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"xmqqfsjok6g1.fsf@gitster.g","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-07T11:59:12Z","receivedAt":"2022-07-07T12:00:31Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\nOn Date: Tue, 28 Jun 2022 11:07:26 -0700, Junio C Hamano wrote:\n\n\n> The verb \"retrieve\" is puzzling.\n\nI use \"retrieve\" because I think they should be there but actually missing.\nBut If it's not appropriate here I will change another word like \"add\".\n\n> \",\" -> \", \".\n\nYes.\n\n> If we were to do this, to avoid burdening translators with double\n> work, we probably would want to fix the \"C\" locale version of the\n> string, either as a preliminary clean-up before this step, or as\n> part of this step. From Documentation/CodingGuidelines:\n\nYes.\n\nDoes git have any NOT \"C\" Locale string?\n\nAnother doublt is I found something like in:\n\n File: ./contrib/completion/git-completion.bash\n 923 LANG=C LC_ALL=C git merge -s help 2>&1\n\nI think LC_ALL=C will override LANG=C in these cases, so I think\n`LC_ALL=C git merge -s help 2>&1` is OK here.\n\nFrom: https://www.gnu.org/software/gettext/manual/html_node/Locale-Environment-Variables.html#Locale-Environment-Variables.\n\n> Error Messages\n>\n> - Do not end error messages with a full stop.\n>\n> - Do not capitalize the first word, only because it is the first word\n> in the message (\"unable to open %s\", not \"Unable to open %s\"). But\n> \"SHA-3 not supported\" is fine, because the reason the first word is\n> capitalized is not because it is at the beginning of the sentence,\n> but because the word would be spelled in capital letters even when\n> it appeared in the middle of the sentence.\n>\n> - Say what the error is first (\"cannot open %s\", not \"%s: cannot open\")\n\nHelpful, fill in blind spots.\n\nActually, in tr2 some text are printed as capitalized the first word. Is this\nentry appropriate for this situation? Whether we should unify the both?\n"},{"id":"458576","messageId":"xmqq8rp43m7v.fsf@gitster.g","threadId":"57611","inReplyTo":"20220707115912.75774-1-dyroneteng@gmail.com","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-07T16:45:56Z","receivedAt":"2022-07-07T16:46:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> On Date: Tue, 28 Jun 2022 11:07:26 -0700, Junio C Hamano wrote:\n>\n>\n>> The verb \"retrieve\" is puzzling.\n>\n> I use \"retrieve\" because I think they should be there but actually missing.\n> But If it's not appropriate here I will change another word like \"add\".\n\nTo retrieve is to get/bring something back and regaining possession\nof, which implies that the thing existed somewhere already but at a\nwrong/different place, and the only thing you are doing is to move\nit to the right place, but in this case, the translations did not\nexist. The patch is marking more strings for translation. And the\nact of marking them for translation will cause i18n/l10n folks to\ntranslate these strings, which will (finally) allow _(\"...\") to\nretrieve the translated strings at runtime.\n\nSo \"retrieve\" is indeed involved somewhere in the process, but using\nthe verb skips a few steps.\n\n Subject: [PATCH 4/5] pack-bitmap.c: mark more strings for translations\n\nperhaps?\n\n>> If we were to do this, to avoid burdening translators with double\n>> work, we probably would want to fix the \"C\" locale version of the\n>> string, either as a preliminary clean-up before this step, or as\n>> part of this step. From Documentation/CodingGuidelines:\n>\n> Yes.\n>\n> Does git have any NOT \"C\" Locale string?\n\nSorry, but I am not sure what you are asking. What I meant is that\na hunk like this from the patch in discussion:\n\n \tif (bitmap_size < 0) {\n-\t\terror(\"Failed to load bitmap index (corrupted?)\");\n+\t\terror(_(\"Failed to load bitmap index (corrupted?)\"));\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n\nmakes translators to first translate the above string, but we will\nfix the \"C\" locale version (that is, the string inside _() that is\nused as the key to the .po database when retrieving the translated\nversion) to follow our error message formatting convention to read\nsomething like\n\n\terror(_(\"failed to load bitmap index (corrupted?)\"));\n\nor even\n\n\terror(_(\"failed to load bitmap index (corrupted?): '%s'\"),\n\t filename);\n\nAnd the translators have to redo the work. If a preliminary patch\nfixed these up before bothering translators with more strings to\ntranslate, they do not need to translate the current, known to be\nfaulty, version of messages.\n\n> Another doublt is I found something like in:\n>\n> File: ./contrib/completion/git-completion.bash\n> 923 LANG=C LC_ALL=C git merge -s help 2>&1\n>\n> I think LC_ALL=C will override LANG=C in these cases, so I think\n> `LC_ALL=C git merge -s help 2>&1` is OK here.\n\nIn practice, yes, but the code is following the convention to reduce\ncommon confusion caused by leaving some lower precedence but common\nenvironment variables (i.e. LANG) as their original values.\n\nDoes the line in the completion script have anything to do with\n[PATCH 4/5], or is this merely your curiosity? Avoid mixing in\nunrelated things into the topic, which will only make the review\ncycle unnecessarily longer, but raise a separate discussion if you\nhave to.\n\nThanks.\n"},{"id":"458746","messageId":"20220711035141.12793-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220628.86o7ydf8gi.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 5/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T03:51:41Z","receivedAt":"2022-07-11T03:51:52Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\n> Is tr2_cfg_load_patterns() run at early startup, but this perhaps at\n> runtime? I wonder if this is OK under threading, i.e. concurrent access\n> to your strset.\n\nYes. I think it may have at runtime.\nNot for that reason, for now I think it's better to dump each config and\nscope name is a better way in terms of other replied suggestions.\n\n> The assumption you're making here doesn't hold in general, some config\n> is \"last vaule wins\", but for some other config all configured values\n> are important.\n\nYes. Also based on the above replies.\n\n> Do we care in this case? I really don't know, but perhaps we can declare\n> \"dedup\" using the same facility we're using to wildcard-match keys, and\n> either make that optional or the default, e.g.:\n>\n> \tGIT_TRACE2_CONFIG_PARAMS='*:dedup,core.*:'\n>\n> I.e. to make it a list of [:].\n>\n> Maybe not worth the effort...\n\nI think if we could print config and scope names is enough maybe. When users\nspecify such as GIT_TRACE2_CONFIG_PARAMS, no one will cares about the extra\nfew lines config output in my opinion, but it may be useful for some\ntroubleshooting, so in my opinion this may not be implemented in this patch\nfor the time being.\n\nThanks.\n"},{"id":"458748","messageId":"20220711041148.15760-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"838e1273-b704-bd11-e9ed-e25390989027@jeffhostetler.com","subject":"Re: [PATCH v5 5/5] tr2: avoid to print \"interesting\" config repeatedly","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T04:11:48Z","receivedAt":"2022-07-11T04:12:00Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\n> I agree. Let's not try to dedup these. IIRC, the tr2_cfg_cb()\n> machinery runs during a \"read early\" or \"read very early\" scan of\n> the config values and the program is still starting up (in some\n> sense). For example, the command line may not have been fully\n> processed yet (again, IIRC). So injecting a call here to force an\n> explicit lookup could cause problems.\n\n\nThe \"read early\" or \"read very early\" is a little abtract for me I think, I will\ntake a further look.\n\n\n> And I don't think it is worth the effort. Let's just log the\n> context (system, global, local) as you described in the previous\n> version and be happy that we get multiple-but-now-qualified values.\n\nYes.\n\n> Also, your new qualification would help us answer support questions\n> using telemetry where a user thought they had a feature on (say\n> globally) but was actually off at another level (say locally).\n\nThat's certainly my original thought.\n\n\nThanks.\n"},{"id":"458770","messageId":"20220711110403.47391-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"xmqq8rp43m7v.fsf@gitster.g","subject":"Re: [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T11:04:03Z","receivedAt":"2022-07-11T11:27:51Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"> To retrieve is to get/bring something back and regaining possession\n> of, which implies that the thing existed somewhere already but at a\n> wrong/different place, and the only thing you are doing is to move\n> it to the right place, but in this case, the translations did not\n> exist. The patch is marking more strings for translation. And the\n> act of marking them for translation will cause i18n/l10n folks to\n> translate these strings, which will (finally) allow _(\"...\") to\n> retrieve the translated strings at runtime.\n>\n> So \"retrieve\" is indeed involved somewhere in the process, but using\n> the verb skips a few steps.\n>\n> Subject: [PATCH 4/5] pack-bitmap.c: mark more strings for translations\n>\n> perhaps?\n\nYes. The explanation is clear.\n\n\n> Sorry, but I am not sure what you are asking. What I meant is that\n> a hunk like this from the patch in discussion:\n> \n> \tif (bitmap_size < 0) {\n> -\t\terror(\"Failed to load bitmap index (corrupted?)\");\n> +\t\terror(_(\"Failed to load bitmap index (corrupted?)\"));\n> \t\tewah_pool_free(b);\n> \t\treturn NULL;\n> \t}\n> \n> makes translators to first translate the above string, but we will\n> fix the \"C\" locale version (that is, the string inside _() that is\n> used as the key to the .po database when retrieving the translated\n> version) to follow our error message formatting convention to read\n> something like\n> \n> \terror(_(\"failed to load bitmap index (corrupted?)\"));\n> \n> or even\n> \n> \terror(_(\"failed to load bitmap index (corrupted?): '%s'\"),\n> \t filename);\n> \n> And the translators have to redo the work. If a preliminary patch\n> fixed these up before bothering translators with more strings to\n> translate, they do not need to translate the current, known to be\n> faulty, version of messages.\n\nYes. I understand a bit of that, maybe. So, if the string is not C locale,\ntranslator will redo because it cannot be a translate key. Another scence is,\nif the string is not following the guideline like capitalized first letter,\ntranslator will redo too, we should avoid that. \n\n> In practice, yes, but the code is following the convention to reduce\n> common confusion caused by leaving some lower precedence but common\n> environment variables (i.e. LANG) as their original values.\n\nOK.\n\n> Does the line in the completion script have anything to do with\n> [PATCH 4/5], or is this merely your curiosity? Avoid mixing in\n> unrelated things into the topic, which will only make the review\n> cycle unnecessarily longer, but raise a separate discussion if you\n> have to.\n\nCuriosity. \nSorry for that, I will raise a separate one next time.\n\nThanks.\n"},{"id":"458777","messageId":"cover.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1656403084.git.dyroneteng@gmail.com","subject":"[PATCH v6 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:43:56Z","receivedAt":"2022-07-11T12:44:47Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Since v5:\n\n1. [1/7] New commit, pre-clean the issues of the strings.\n2. [2/7] Optimize commit subject (the word \"retrieve\").\n3. [4/7] New commit, do not ignore ENOENT silently when fail to open file. \n4. [5/7] Replace \"stat\" to \"fstat\" in output string and let \"cleanup\" to\n return -1 instead of an unaccurate error tip as \"cannot open\".\n5. [7/7] dump corresponding scope-name when \"interesting\" config is print to\n trace2 log.\n\nThanks.\n\nTeng Long (7):\n clean: fixed issues related to text output format\n pack-bitmap.c: mark more strings for translations\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: don't ignore ENOENT silently\n pack-bitmap.c: using error() instead of silently returning -1\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n tr2: dump names if config exist in multiple scopes\n\n pack-bitmap.c | 105 ++++++++++++++++++++++------------------\n trace2/tr2_tgt_event.c | 3 ++\n trace2/tr2_tgt_normal.c | 5 +-\n trace2/tr2_tgt_perf.c | 9 +++-\n 4 files changed, 72 insertions(+), 50 deletions(-)\n\nRange-diff against v5:\n-: ---------- > 1: 94a64ba895 clean: fixed issues related to text output format\n4: 065b7c9ccb ! 2: cadecd6b84 pack-bitmap.c: retrieve missing i18n translations\n @@ Metadata\n Author: Teng Long \n \n ## Commit message ##\n - pack-bitmap.c: retrieve missing i18n translations\n + pack-bitmap.c: mark more strings for translations\n \n - In pack-bitmap.c, some printed texts are translated,some are not.\n + In pack-bitmap.c, some printed texts are translated, some are not.\n Let's support the translations of the bitmap related output.\n \n Signed-off-by: Teng Long \n @@ pack-bitmap.c: static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *ind\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n --\t\terror(\"Failed to load bitmap index (corrupted?)\");\n -+\t\terror(_(\"Failed to load bitmap index (corrupted?)\"));\n +-\t\terror(\"failed to load bitmap index (corrupted?)\");\n ++\t\terror(_(\"failed to load bitmap index (corrupted?)\"));\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n @@ pack-bitmap.c: static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n --\t\treturn error(\"Corrupted bitmap index (too small)\");\n -+\t\treturn error(_(\"Corrupted bitmap index (too small)\"));\n +-\t\treturn error(\"corrupted bitmap index (too small)\");\n ++\t\treturn error(_(\"corrupted bitmap index (too small)\"));\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n --\t\treturn error(\"Corrupted bitmap index file (wrong header)\");\n -+\t\treturn error(_(\"Corrupted bitmap index file (wrong header)\"));\n +-\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n ++\t\treturn error(_(\"corrupted bitmap index file (wrong header)\"));\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n --\t\treturn error(\"Unsupported version for bitmap index file (%d)\", index->version);\n -+\t\treturn error(_(\"Unsupported version for bitmap index file (%d)\"), index->version);\n +-\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n ++\t\treturn error(_(\"unsupported version '%d' for bitmap index file\"), index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n @@ pack-bitmap.c: static int load_bitmap_header(struct bitmap_index *index)\n - \t\tunsigned char *index_end = index->map + index->map_size - the_hash_algo->rawsz;\n - \n - \t\tif ((flags & BITMAP_OPT_FULL_DAG) == 0)\n --\t\t\treturn error(\"Unsupported options for bitmap index file \"\n --\t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n -+\t\t\treturn error(_(\"Unsupported options for bitmap index file \"\n -+\t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\"));\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n \t\t\tif (cache_size > index_end - index->map - header_size)\n @@ pack-bitmap.c: static struct stored_bitmap *store_bitmap(struct bitmap_index *in\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n --\t\terror(\"Duplicate entry in bitmap index: %s\", oid_to_hex(oid));\n -+\t\terror(_(\"Duplicate entry in bitmap index: %s\"), oid_to_hex(oid));\n +-\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n ++\t\terror(_(\"duplicate entry in bitmap index: '%s'\"), oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n @@ pack-bitmap.c: static int load_bitmap_entries_v1(struct bitmap_index *index)\n \n \t\tif (index->map_size - index->map_pos < 6)\n -\t\t\treturn error(\"corrupt ewah bitmap: truncated header for entry %d\", i);\n -+\t\t\treturn error(_(\"corrupt ewah bitmap: truncated header for entry %d)\"), i);\n ++\t\t\treturn error(_(\"corrupt ewah bitmap: truncated header for entry %d\"), i);\n \n \t\tcommit_idx_pos = read_be32(index->map, &index->map_pos);\n \t\txor_offset = read_u8(index->map, &index->map_pos);\n @@ pack-bitmap.c: static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n --\t\t\treturn error(\"Corrupted bitmap pack index\");\n -+\t\t\treturn error(_(\"Corrupted bitmap pack index\"));\n +-\t\t\treturn error(\"corrupted bitmap pack index\");\n ++\t\t\treturn error(_(\"corrupted bitmap pack index\"));\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n --\t\t\t\treturn error(\"Invalid XOR offset in bitmap pack index\");\n -+\t\t\t\treturn error(_(\"Invalid XOR offset in bitmap pack index\"));\n +-\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n ++\t\t\t\treturn error(_(\"invalid XOR offset in bitmap pack index\"));\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n -@@ pack-bitmap.c: char *pack_bitmap_filename(struct packed_git *p)\n - \tsize_t len;\n - \n - \tif (!strip_suffix(p->pack_name, \".pack\", &len))\n --\t\tBUG(\"pack_name does not end in .pack\");\n -+\t\tBUG(_(\"pack_name does not end in .pack\"));\n - \treturn xstrfmt(\"%.*s.bitmap\", (int)len, p->pack_name);\n - }\n - \n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n --\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n -+\t\twarning(_(\"ignoring extra bitmap file: %s\"), buf.buf);\n +-\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n ++\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n --\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n -+\t\twarning(_(\"ignoring extra bitmap file: %s\"), packfile->pack_name);\n +-\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n ++\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n @@ pack-bitmap.c: static struct bitmap *find_objects(struct bitmap_index *bitmap_gi\n \n \t\tshow_data.bitmap_git = bitmap_git;\n \t\tshow_data.base = base;\n -@@ pack-bitmap.c: static void filter_bitmap_tree_depth(struct bitmap_index *bitmap_git,\n - \t\t\t\t unsigned long limit)\n - {\n - \tif (limit)\n --\t\tBUG(\"filter_bitmap_tree_depth given non-zero limit\");\n -+\t\tBUG(_(\"filter_bitmap_tree_depth given non-zero limit\"));\n - \n - \tfilter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter,\n - \t\t\t\t OBJ_TREE);\n -@@ pack-bitmap.c: static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n - \t\t\t\t enum object_type object_type)\n - {\n - \tif (object_type < OBJ_COMMIT || object_type > OBJ_TAG)\n --\t\tBUG(\"filter_bitmap_object_type given invalid object\");\n -+\t\tBUG(_(\"filter_bitmap_object_type given invalid object\"));\n - \n - \tif (object_type != OBJ_TAG)\n - \t\tfilter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TAG);\n -@@ pack-bitmap.c: struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,\n - \t\trevs->ignore_missing_links = 0;\n - \n - \t\tif (haves_bitmap == NULL)\n --\t\t\tBUG(\"failed to perform bitmap walk\");\n -+\t\t\tBUG(_(\"failed to perform bitmap walk\"));\n - \t}\n - \n - \twants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap,\n - \t\t\t\t filter);\n - \n - \tif (!wants_bitmap)\n --\t\tBUG(\"failed to perform bitmap walk\");\n -+\t\tBUG(_(\"failed to perform bitmap walk\"));\n - \n - \tif (haves_bitmap)\n - \t\tbitmap_and_not(wants_bitmap, haves_bitmap);\n -@@ pack-bitmap.c: uint32_t midx_preferred_pack(struct bitmap_index *bitmap_git)\n - {\n - \tstruct multi_pack_index *m = bitmap_git->midx;\n - \tif (!m)\n --\t\tBUG(\"midx_preferred_pack: requires non-empty MIDX\");\n -+\t\tBUG(_(\"midx_preferred_pack: requires non-empty MIDX\"));\n - \treturn nth_midxed_pack_int_id(m, pack_pos_to_midx(bitmap_git->midx, 0));\n - }\n - \n @@ pack-bitmap.c: static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n --\t\tdie(\"object %s not found in type bitmaps\",\n -+\t\tdie(_(\"object %s not found in type bitmaps\"),\n +-\t\tdie(\"object '%s' not found in type bitmaps\",\n ++\t\tdie(_(\"object '%s' not found in type bitmaps\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n --\t\tdie(\"object %s does not have a unique type\",\n -+\t\tdie(_(\"object %s does not have a unique type\"),\n +-\t\tdie(\"object '%s' does not have a unique type\",\n ++\t\tdie(_(\"object '%s' does not have a unique type\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n --\t\tdie(\"object %s: real type %s, expected: %s\",\n -+\t\tdie(_(\"object %s: real type %s, expected: %s\"),\n +-\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n ++\t\tdie(_(\"object '%s': real type '%s', expected: '%s'\"),\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n @@ pack-bitmap.c: static void test_show_object(struct object *object, const char *n\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n --\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&object->oid));\n -+\t\tdie(_(\"Object not in bitmap: %s\\n\"), oid_to_hex(&object->oid));\n +-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n ++\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n @@ pack-bitmap.c: static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n --\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&commit->object.oid));\n -+\t\tdie(_(\"Object not in bitmap: %s\\n\"), oid_to_hex(&commit->object.oid));\n +-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n ++\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n @@ pack-bitmap.c: void test_bitmap_walk(struct rev_info *revs)\n -\t\tdie(\"you must specify exactly one commit to test\");\n +\t\tdie(_(\"you must specify exactly one commit to test\"));\n \n --\tfprintf(stderr, \"Bitmap v%d test (%d entries loaded)\\n\",\n -+\tfprintf(stderr, _(\"Bitmap v%d test (%d entries loaded)\\n\"),\n + \tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n - \n - \troot = revs->pending.objects[0].item;\n - \tbm = bitmap_for_commit(bitmap_git, (struct commit *)root);\n - \n - \tif (bm) {\n --\t\tfprintf(stderr, \"Found bitmap for %s. %d bits / %08x checksum\\n\",\n -+\t\tfprintf(stderr, _(\"Found bitmap for %s. %d bits / %08x checksum\\n\"),\n - \t\t\toid_to_hex(&root->oid), (int)bm->bit_size, ewah_checksum(bm));\n - \n - \t\tresult = ewah_to_bitmap(bm);\n +@@ pack-bitmap.c: void test_bitmap_walk(struct rev_info *revs)\n \t}\n \n \tif (result == NULL)\n --\t\tdie(\"Commit %s doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n -+\t\tdie(_(\"Commit %s doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n +-\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n ++\t\tdie(_(\"commit '%s' doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n @@ pack-bitmap.c: void test_bitmap_walk(struct rev_info *revs)\n \ttdata.bitmap_git = bitmap_git;\n \ttdata.base = bitmap_new();\n @@ pack-bitmap.c: void test_bitmap_walk(struct rev_info *revs)\n - \ttdata.trees = ewah_to_bitmap(bitmap_git->trees);\n - \ttdata.blobs = ewah_to_bitmap(bitmap_git->blobs);\n - \ttdata.tags = ewah_to_bitmap(bitmap_git->tags);\n --\ttdata.prg = start_progress(\"Verifying bitmap entries\", result_popcnt);\n -+\ttdata.prg = start_progress(_(\"Verifying bitmap entries\"), result_popcnt);\n - \ttdata.seen = 0;\n - \n - \ttraverse_commit_list(revs, &test_show_commit, &test_show_object, &tdata);\n -@@ pack-bitmap.c: void test_bitmap_walk(struct rev_info *revs)\n - \tstop_progress(&tdata.prg);\n - \n \tif (bitmap_equals(result, tdata.base))\n --\t\tfprintf(stderr, \"OK!\\n\");\n -+\t\tfprintf(stderr, _(\"OK!\\n\"));\n + \t\tfprintf_ln(stderr, \"OK!\");\n \telse\n -\t\tdie(\"mismatch in bitmap results\");\n +\t\tdie(_(\"mismatch in bitmap results\"));\n @@ pack-bitmap.c: int test_bitmap_commits(struct repository *r)\n +\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n - \t\tprintf(\"%s\\n\", oid_to_hex(&oid));\n -@@ pack-bitmap.c: uint32_t *create_bitmap_mapping(struct bitmap_index *bitmap_git,\n - \tif (!bitmap_is_midx(bitmap_git))\n - \t\tload_reverse_index(bitmap_git);\n - \telse if (load_midx_revindex(bitmap_git->midx) < 0)\n --\t\tBUG(\"rebuild_existing_bitmaps: missing required rev-cache \"\n --\t\t \"extension\");\n -+\t\tBUG(_(\"rebuild_existing_bitmaps: missing required rev-cache \"\n -+\t\t \"extension\"));\n - \n - \tnum_objects = bitmap_num_objects(bitmap_git);\n - \tCALLOC_ARRAY(reposition, num_objects);\n + \t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n2: b6b30047fc = 3: d8a2235cb0 pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n-: ---------- > 4: 009cc49a18 pack-bitmap.c: don't ignore ENOENT silently\n-: ---------- > 5: 52783555e2 pack-bitmap.c: using error() instead of silently returning -1\n1: 589e3f4075 = 6: 95832190b8 pack-bitmap.c: continue looping when first MIDX bitmap is found\n3: 82d4493a6e < -: ---------- pack-bitmap.c: using error() instead of silently returning -1\n5: f3b87a33da < -: ---------- tr2: avoid to print \"interesting\" config repeatedly\n-: ---------- > 7: c45ead51ff tr2: dump names if config exist in multiple scopes\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458778","messageId":"94a64ba895a8eb9eb7d502e94cf3d9fdfb9d2887.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 1/7] clean: fixed issues related to text output format","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:43:57Z","receivedAt":"2022-07-11T12:44:51Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"There are some text output issues in 'pack-bitmap.c', they exist in\ndie(), error() etc. This includes issues with capitalization the\nfirst letter, newlines, error() instead of BUG(), and substitution\nthat don't have quotes around them.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 47 ++++++++++++++++++++++++-----------------------\n 1 file changed, 24 insertions(+), 23 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..0fdba5e8c5 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"strbuf.h\"\n #include \"tag.h\"\n #include \"diff.h\"\n #include \"revision.h\"\n@@ -138,7 +139,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"Failed to load bitmap index (corrupted?)\");\n+\t\terror(\"failed to load bitmap index (corrupted?)\");\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -160,14 +161,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"Corrupted bitmap index (too small)\");\n+\t\treturn error(\"corrupted bitmap index (too small)\");\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"Corrupted bitmap index file (wrong header)\");\n+\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"Unsupported version for bitmap index file (%d)\", index->version);\n+\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -176,7 +177,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \t\tunsigned char *index_end = index->map + index->map_size - the_hash_algo->rawsz;\n \n \t\tif ((flags & BITMAP_OPT_FULL_DAG) == 0)\n-\t\t\treturn error(\"Unsupported options for bitmap index file \"\n+\t\t\tBUG(\"unsupported options for bitmap index file \"\n \t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n@@ -215,7 +216,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"Duplicate entry in bitmap index: %s\", oid_to_hex(oid));\n+\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -274,13 +275,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"Corrupted bitmap pack index\");\n+\t\t\treturn error(\"corrupted bitmap pack index\");\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"Invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -330,7 +331,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +388,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -1628,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object %s not found in type bitmaps\",\n+\t\tdie(\"object '%s' not found in type bitmaps\",\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object %s does not have a unique type\",\n+\t\tdie(\"object '%s' does not have a unique type\",\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object %s: real type %s, expected: %s\",\n+\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1650,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&object->oid));\n+\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1665,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&commit->object.oid));\n+\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1687,21 +1688,21 @@ void test_bitmap_walk(struct rev_info *revs)\n \tif (revs->pending.nr != 1)\n \t\tdie(\"you must specify exactly one commit to test\");\n \n-\tfprintf(stderr, \"Bitmap v%d test (%d entries loaded)\\n\",\n+\tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n \n \troot = revs->pending.objects[0].item;\n \tbm = bitmap_for_commit(bitmap_git, (struct commit *)root);\n \n \tif (bm) {\n-\t\tfprintf(stderr, \"Found bitmap for %s. %d bits / %08x checksum\\n\",\n+\t\tfprintf_ln(stderr, \"Found bitmap for '%s'. %d bits / %08x checksum\",\n \t\t\toid_to_hex(&root->oid), (int)bm->bit_size, ewah_checksum(bm));\n \n \t\tresult = ewah_to_bitmap(bm);\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"Commit %s doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1726,7 +1727,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstop_progress(&tdata.prg);\n \n \tif (bitmap_equals(result, tdata.base))\n-\t\tfprintf(stderr, \"OK!\\n\");\n+\t\tfprintf_ln(stderr, \"OK!\");\n \telse\n \t\tdie(\"mismatch in bitmap results\");\n \n@@ -1749,7 +1750,7 @@ int test_bitmap_commits(struct repository *r)\n \t\tdie(\"failed to load bitmap indexes\");\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n-\t\tprintf(\"%s\\n\", oid_to_hex(&oid));\n+\t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n \t});\n \n \tfree_bitmap_index(bitmap_git);\n@@ -1774,7 +1775,7 @@ int test_bitmap_hashes(struct repository *r)\n \n \t\tnth_bitmap_object_oid(bitmap_git, &oid, index_pos);\n \n-\t\tprintf(\"%s %\"PRIu32\"\\n\",\n+\t\tprintf_ln(\"%s %\"PRIu32\"\",\n \t\t oid_to_hex(&oid), get_be32(bitmap_git->hashes + index_pos));\n \t}\n \n@@ -1936,7 +1937,7 @@ static off_t get_disk_usage_for_type(struct bitmap_index *bitmap_git,\n \t\t\t\t\tstruct object_id oid;\n \t\t\t\t\tnth_midxed_object_oid(&oid, bitmap_git->midx, midx_pos);\n \n-\t\t\t\t\tdie(_(\"could not find %s in pack %s at offset %\"PRIuMAX),\n+\t\t\t\t\tdie(_(\"could not find '%s' in pack '%s' at offset %\"PRIuMAX),\n \t\t\t\t\t oid_to_hex(&oid),\n \t\t\t\t\t pack->pack_name,\n \t\t\t\t\t (uintmax_t)offset);\n@@ -1972,7 +1973,7 @@ static off_t get_disk_usage_for_extended(struct bitmap_index *bitmap_git)\n \t\t\tcontinue;\n \n \t\tif (oid_object_info_extended(the_repository, &obj->oid, &oi, 0) < 0)\n-\t\t\tdie(_(\"unable to get disk usage of %s\"),\n+\t\t\tdie(_(\"unable to get disk usage of '%s'\"),\n \t\t\t oid_to_hex(&obj->oid));\n \n \t\ttotal += object_size;\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458779","messageId":"cadecd6b8461e5e993d7b07d3bbeb8c44cf2b2d2.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 2/7] pack-bitmap.c: mark more strings for translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:43:58Z","receivedAt":"2022-07-11T12:44:57Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In pack-bitmap.c, some printed texts are translated, some are not.\nLet's support the translations of the bitmap related output.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 48 ++++++++++++++++++++++++------------------------\n 1 file changed, 24 insertions(+), 24 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 0fdba5e8c5..c970ab46dd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -139,7 +139,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"failed to load bitmap index (corrupted?)\");\n+\t\terror(_(\"failed to load bitmap index (corrupted?)\"));\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -161,14 +161,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"corrupted bitmap index (too small)\");\n+\t\treturn error(_(\"corrupted bitmap index (too small)\"));\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n+\t\treturn error(_(\"corrupted bitmap index file (wrong header)\"));\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n+\t\treturn error(_(\"unsupported version '%d' for bitmap index file\"), index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -182,7 +182,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n \t\t\tif (cache_size > index_end - index->map - header_size)\n-\t\t\t\treturn error(\"corrupted bitmap index file (too short to fit hash cache)\");\n+\t\t\t\treturn error(_(\"corrupted bitmap index file (too short to fit hash cache)\"));\n \t\t\tindex->hashes = (void *)(index_end - cache_size);\n \t\t\tindex_end -= cache_size;\n \t\t}\n@@ -216,7 +216,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n+\t\terror(_(\"duplicate entry in bitmap index: '%s'\"), oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -260,14 +260,14 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\tstruct object_id oid;\n \n \t\tif (index->map_size - index->map_pos < 6)\n-\t\t\treturn error(\"corrupt ewah bitmap: truncated header for entry %d\", i);\n+\t\t\treturn error(_(\"corrupt ewah bitmap: truncated header for entry %d\"), i);\n \n \t\tcommit_idx_pos = read_be32(index->map, &index->map_pos);\n \t\txor_offset = read_u8(index->map, &index->map_pos);\n \t\tflags = read_u8(index->map, &index->map_pos);\n \n \t\tif (nth_bitmap_object_oid(index, &oid, commit_idx_pos) < 0)\n-\t\t\treturn error(\"corrupt ewah bitmap: commit index %u out of range\",\n+\t\t\treturn error(_(\"corrupt ewah bitmap: commit index %u out of range\"),\n \t\t\t\t (unsigned)commit_idx_pos);\n \n \t\tbitmap = read_bitmap_1(index);\n@@ -275,13 +275,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"corrupted bitmap pack index\");\n+\t\t\treturn error(_(\"corrupted bitmap pack index\"));\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(_(\"invalid XOR offset in bitmap pack index\"));\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -331,7 +331,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n+\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -388,7 +388,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n+\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -819,7 +819,7 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,\n \t\trevs->include_check_data = &incdata;\n \n \t\tif (prepare_revision_walk(revs))\n-\t\t\tdie(\"revision walk setup failed\");\n+\t\t\tdie(_(\"revision walk setup failed\"));\n \n \t\tshow_data.bitmap_git = bitmap_git;\n \t\tshow_data.base = base;\n@@ -1629,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object '%s' not found in type bitmaps\",\n+\t\tdie(_(\"object '%s' not found in type bitmaps\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object '%s' does not have a unique type\",\n+\t\tdie(_(\"object '%s' does not have a unique type\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n+\t\tdie(_(\"object '%s': real type '%s', expected: '%s'\"),\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1651,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n+\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1666,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n+\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1683,10 +1683,10 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstruct ewah_bitmap *bm;\n \n \tif (!(bitmap_git = prepare_bitmap_git(revs->repo)))\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tif (revs->pending.nr != 1)\n-\t\tdie(\"you must specify exactly one commit to test\");\n+\t\tdie(_(\"you must specify exactly one commit to test\"));\n \n \tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n@@ -1702,7 +1702,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(_(\"commit '%s' doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1711,7 +1711,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tresult_popcnt = bitmap_popcount(result);\n \n \tif (prepare_revision_walk(revs))\n-\t\tdie(\"revision walk setup failed\");\n+\t\tdie(_(\"revision walk setup failed\"));\n \n \ttdata.bitmap_git = bitmap_git;\n \ttdata.base = bitmap_new();\n@@ -1729,7 +1729,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tif (bitmap_equals(result, tdata.base))\n \t\tfprintf_ln(stderr, \"OK!\");\n \telse\n-\t\tdie(\"mismatch in bitmap results\");\n+\t\tdie(_(\"mismatch in bitmap results\"));\n \n \tbitmap_free(result);\n \tbitmap_free(tdata.base);\n@@ -1747,7 +1747,7 @@ int test_bitmap_commits(struct repository *r)\n \tMAYBE_UNUSED void *value;\n \n \tif (!bitmap_git)\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n \t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458780","messageId":"d8a2235cb01a01f7f815db4055cf1aa3c0818cc9.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 3/7] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:43:59Z","receivedAt":"2022-07-11T12:45:11Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index,\nlet's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex c970ab46dd..7d8cc063fc 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -314,10 +314,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -369,14 +369,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458781","messageId":"009cc49a18f2846c24256102e07437894ac16908.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 4/7] pack-bitmap.c: don't ignore ENOENT silently","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:44:00Z","receivedAt":"2022-07-11T12:45:13Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"When finished call git_open(), instead of ignoring ENOENT silently\nand return error_errno(_(\"cannot stat...\")), it's better to check\nthe ENOENT before then output the warning.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 7d8cc063fc..319eb721d8 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -317,10 +317,13 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n \tint fd = git_open(bitmap_name);\n \n-\tfree(bitmap_name);\n-\n-\tif (fd < 0)\n+\tif (fd < 0) {\n+\t\tif (errno != ENOENT)\n+\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n+\t\tfree(bitmap_name);\n \t\treturn -1;\n+\t}\n+\tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n@@ -376,10 +379,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n \tfd = git_open(bitmap_name);\n-\tfree(bitmap_name);\n \n-\tif (fd < 0)\n+\tif (fd < 0) {\n+\t\tif (errno != ENOENT)\n+\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n+\t\tfree(bitmap_name);\n \t\treturn -1;\n+\t}\n+\tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458782","messageId":"52783555e206060465743b5587580a6bd4a1f008.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 5/7] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:44:01Z","receivedAt":"2022-07-11T12:45:22Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 319eb721d8..fbe3f58aff 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -327,7 +327,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n@@ -350,8 +350,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (load_bitmap_header(bitmap_git) < 0)\n \t\tgoto cleanup;\n \n-\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n+\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n+\t\terror(_(\"checksum doesn't match in MIDX and bitmap\"));\n \t\tgoto cleanup;\n+\t}\n \n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n@@ -390,7 +392,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-\t\treturn -1;\n+\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n \t}\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458783","messageId":"95832190b8bba72622ffe1474e99990a29706db8.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 6/7] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:44:02Z","receivedAt":"2022-07-11T12:45:29Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to continue the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe reason for this is, there may exist more than one MIDX file in\na repo. The \"multi_pack_index\" struct is actually designed as a singly\nlinked list, and if a MIDX file has been already opened successfully,\nthen the other MIDX files will be skipped and left with a warning\n\"ignoring extra bitmap file.\" to the output.\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex fbe3f58aff..7e69093d5a 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -504,15 +504,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458784","messageId":"c45ead51ffc5a9176493d627da8332d35a31d87c.1657540174.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v6 7/7] tr2: dump names if config exist in multiple scopes","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-11T12:44:03Z","receivedAt":"2022-07-11T12:45:34Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\ntrace2 will prints \"interesting\" config values to log. Sometimes,\nwhen a config set in multiple scope files, the following output\nlooks like (the irrelevant fields are omitted here as \"...\"):\n\n...| def_param | ... | core.multipackindex:false\n...| def_param | ... | core.multipackindex:false\n...| def_param | ... | core.multipackindex:false\n\nAs the log shows, even each config in different scope is dumped, but\nwe don't know which scope it comes from. Therefore, it's better to\nadd the scope names as well to make them be more recognizable. For\nexample, when execute:\n\n $ GIT_TRACE2_PERF=1 \\\n > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n > git rev-list --test-bitmap HEAD\"\n\nThe following is the ouput (the irrelevant fields are omitted here\nas \"...\"):\n\nFormat normal:\n... git.c:461 ... def_param scope:system core.multipackindex=false\n... git.c:461 ... def_param scope:global core.multipackindex=false\n... git.c:461 ... def_param scope:local core.multipackindex=false\n\nFormat perf:\n\n... | def_param | ... | scope:system | core.multipackindex:false\n... | def_param | ... | scope:global | core.multipackindex:false\n... | def_param | ... | scope:local | core.multipackindex:false\n\nFormat event:\n\n{\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n{\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n{\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n\nSigned-off-by: Teng Long \n---\n trace2/tr2_tgt_event.c | 3 +++\n trace2/tr2_tgt_normal.c | 5 ++++-\n trace2/tr2_tgt_perf.c | 9 +++++++--\n 3 files changed, 14 insertions(+), 3 deletions(-)\n\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex c5c8cfbbaa..37a3163be1 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n \tconst char *event_name = \"def_param\";\n \tstruct json_writer jw = JSON_WRITER_INIT;\n+\tenum config_scope scope = current_config_scope();\n+\tconst char *scope_name = config_scope_name(scope);\n \n \tjw_object_begin(&jw, 0);\n \tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_string(&jw, \"scope\", scope_name);\n \tjw_object_string(&jw, \"param\", param);\n \tjw_object_string(&jw, \"value\", value);\n \tjw_end(&jw);\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex c42fbade7f..69f8033077 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -298,8 +298,11 @@ static void fn_param_fl(const char *file, int line, const char *param,\n \t\t\tconst char *value)\n {\n \tstruct strbuf buf_payload = STRBUF_INIT;\n+\tenum config_scope scope = current_config_scope();\n+\tconst char *scope_name = config_scope_name(scope);\n \n-\tstrbuf_addf(&buf_payload, \"def_param %s=%s\", param, value);\n+\tstrbuf_addf(&buf_payload, \"def_param scope:%s %s=%s\", scope_name, param,\n+\t\t value);\n \tnormal_io_write_fl(file, line, &buf_payload);\n \tstrbuf_release(&buf_payload);\n }\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a1eff8bea3..8cb792488c 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -441,12 +441,17 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n \tconst char *event_name = \"def_param\";\n \tstruct strbuf buf_payload = STRBUF_INIT;\n+\tstruct strbuf scope_payload = STRBUF_INIT;\n+\tenum config_scope scope = current_config_scope();\n+\tconst char *scope_name = config_scope_name(scope);\n \n \tstrbuf_addf(&buf_payload, \"%s:%s\", param, value);\n+\tstrbuf_addf(&scope_payload, \"%s:%s\", \"scope\", scope_name);\n \n-\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n-\t\t\t &buf_payload);\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL,\n+\t\t\t scope_payload.buf, &buf_payload);\n \tstrbuf_release(&buf_payload);\n+\tstrbuf_release(&scope_payload);\n }\n \n static void fn_repo_fl(const char *file, int line,\n-- \n2.35.0.rc0.676.g60105b7097.dirty\n\n"},{"id":"458791","messageId":"220711.86k08j7lyn.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"009cc49a18f2846c24256102e07437894ac16908.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 4/7] pack-bitmap.c: don't ignore ENOENT silently","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-11T14:38:01Z","receivedAt":"2022-07-11T14:39:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 11 2022, Teng Long wrote:\n\n> When finished call git_open(), instead of ignoring ENOENT silently\n> and return error_errno(_(\"cannot stat...\")), it's better to check\n> the ENOENT before then output the warning.\n\nMakes sense, but...\n\n> -\tif (fd < 0)\n> +\tif (fd < 0) {\n> +\t\tif (errno != ENOENT)\n> +\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n\nThis should just be warning_errno(\"cannot open '%s'\", bitmap_name),\nunless I'm missing something...\n\n\n> +\t\tfree(bitmap_name);\n> \t\treturn -1;\n> +\t}\n> +\tfree(bitmap_name);\n> \n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> @@ -376,10 +379,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \n> \tbitmap_name = pack_bitmap_filename(packfile);\n> \tfd = git_open(bitmap_name);\n> -\tfree(bitmap_name);\n> \n> -\tif (fd < 0)\n> +\tif (fd < 0) {\n> +\t\tif (errno != ENOENT)\n> +\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n\n...ditto.\n\n\n> +\t\tfree(bitmap_name);\n> \t\treturn -1;\n> +\t}\n> +\tfree(bitmap_name);\n> \n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n\n"},{"id":"458793","messageId":"220711.86fsj77lur.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"c45ead51ffc5a9176493d627da8332d35a31d87c.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 7/7] tr2: dump names if config exist in multiple scopes","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-11T14:40:21Z","receivedAt":"2022-07-11T14:41:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 11 2022, Teng Long wrote:\n\n> When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\n> trace2 will prints \"interesting\" config values to log. Sometimes,\n> when a config set in multiple scope files, the following output\n> looks like (the irrelevant fields are omitted here as \"...\"):\n>\n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n>\n> As the log shows, even each config in different scope is dumped, but\n> we don't know which scope it comes from. Therefore, it's better to\n> add the scope names as well to make them be more recognizable. For\n> example, when execute:\n>\n> $ GIT_TRACE2_PERF=1 \\\n> > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n> > git rev-list --test-bitmap HEAD\"\n>\n> The following is the ouput (the irrelevant fields are omitted here\n> as \"...\"):\n>\n> Format normal:\n> ... git.c:461 ... def_param scope:system core.multipackindex=false\n> ... git.c:461 ... def_param scope:global core.multipackindex=false\n> ... git.c:461 ... def_param scope:local core.multipackindex=false\n>\n> Format perf:\n>\n> ... | def_param | ... | scope:system | core.multipackindex:false\n> ... | def_param | ... | scope:global | core.multipackindex:false\n> ... | def_param | ... | scope:local | core.multipackindex:false\n>\n> Format event:\n>\n> {\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n\nThis seems sensible on its face, but...\n\n> Signed-off-by: Teng Long \n> ---\n> trace2/tr2_tgt_event.c | 3 +++\n> trace2/tr2_tgt_normal.c | 5 ++++-\n> trace2/tr2_tgt_perf.c | 9 +++++++--\n> 3 files changed, 14 insertions(+), 3 deletions(-)\n\n... we really should update Documentation/technical/api-trace2.txt here too.\n\nIt does say \"...\" currently, so we're not lying there, but since we now\nadd \"scope\" unconditionally...\n"},{"id":"458796","messageId":"220711.86bktv7l5v.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"52783555e206060465743b5587580a6bd4a1f008.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 5/7] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-11T14:53:48Z","receivedAt":"2022-07-11T14:56:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 11 2022, Teng Long wrote:\n\nGood to report errno now, however...\n\n> Signed-off-by: Teng Long \n> ---\n> pack-bitmap.c | 8 +++++---\n> 1 file changed, 5 insertions(+), 3 deletions(-)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 319eb721d8..fbe3f58aff 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -327,7 +327,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n\nThis is a logic error, as fstat() failed, but you're reproting errno()\nafter our call to close(), at which point it's anyone's guess what\n\"errno\" is. It's only meaningful immediately after a system call.\n\nSo either:\n\n error_errno(....);\n close(fd);\n return -1;\n\nOr even better:\n\n error_errno(...)\n if (close(fd))\n warning_errno(\"cannot close() bitmap file\");\n return -1;\n\nAlthough that last one may be too pedantic.\n\n> \t}\n> \n> \tif (bitmap_git->pack || bitmap_git->midx) {\n> @@ -350,8 +350,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n> \tif (load_bitmap_header(bitmap_git) < 0)\n> \t\tgoto cleanup;\n> \n> -\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n> +\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n> +\t\terror(_(\"checksum doesn't match in MIDX and bitmap\"));\n> \t\tgoto cleanup;\n> +\t}\n> \n> \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n> \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n> @@ -390,7 +392,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n> \n> \tif (fstat(fd, &st)) {\n> \t\tclose(fd);\n> -\t\treturn -1;\n> +\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n\nSame issue here.\n"},{"id":"458797","messageId":"220711.867d4j7kuj.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-11T14:59:23Z","receivedAt":"2022-07-11T15:03:24Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 11 2022, Teng Long wrote:\n\n> Since v5:\n>\n> 1. [1/7] New commit, pre-clean the issues of the strings.\n> 2. [2/7] Optimize commit subject (the word \"retrieve\").\n> 3. [4/7] New commit, do not ignore ENOENT silently when fail to open file. \n> 4. [5/7] Replace \"stat\" to \"fstat\" in output string and let \"cleanup\" to\n> return -1 instead of an unaccurate error tip as \"cannot open\".\n> 5. [7/7] dump corresponding scope-name when \"interesting\" config is print to\n> trace2 log.\n\nI left some minor comments, another thing I'd just like to note, but it\nlargely pre-dates this series, so we can leave it for now is that I\nfound our mismatching reporting of the *.bitmap and/or *.pack kind of odd.\n\nI.e. this bit (a diff on top):\n\t\n\tdiff --git a/pack-bitmap.c b/pack-bitmap.c\n\tindex 7e69093d5a8..8a8be5f6cad 100644\n\t--- a/pack-bitmap.c\n\t+++ b/pack-bitmap.c\n\t@@ -381,23 +381,25 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n\t \n\t \tbitmap_name = pack_bitmap_filename(packfile);\n\t \tfd = git_open(bitmap_name);\n\t+\tfree(bitmap_name);\n\t \n\t \tif (fd < 0) {\n\t \t\tif (errno != ENOENT)\n\t-\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n\t-\t\tfree(bitmap_name);\n\t+\t\t\twarning_errno(\"cannot open bitmap for pack '%s'\",\n\t+\t\t\t\t packfile->pack_name);\n\t \t\treturn -1;\n\t \t}\n\t-\tfree(bitmap_name);\n\t \n\t \tif (fstat(fd, &st)) {\n\t \t\tclose(fd);\n\t-\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n\t+\t\treturn error_errno(_(\"cannot fstat() bitmap file for pack '%s'\"),\n\t+\t\t\t\t packfile->pack_name);\n\t \t}\n\t \n\t \tif (bitmap_git->pack || bitmap_git->midx) {\n\t \t\t/* ignore extra bitmap file; we can only handle one */\n\t \t\twarning(_(\"ignoring extra bitmap file: '%s'\"), packfile->pack_name);\n\t+\t\t/* ^^ because here we refer to the pack file, not the bitmap file */\n\t \t\tclose(fd);\n\t \t\treturn -1;\n\t \t}\n\nAs tha shows we sometimes talk about the path for the pack when\nsomething is wrong with the bitmap, but other times the *.bitmap\nassociated with the pack.\n\nIt would be a tad simpler to always talk about the *.pack, as we can\nfree() the bitamp filename right away, but arguably we shoudl do it the\nother way around. After all we have issues with the bitmap file, let's\nmention it (or mention a filename at all).\n\nAlso a pre-existing issue: Why are we getting as far as opening a\n*.bitmap file, if we then find that we have \"bitmap_git->pack ||\nbitmap_git->midx\" already, and then ignore it? Maybe I've missed\nsomething...\n\nAll of the same oddness applies to open_midx_bitmap_1, except with \"path\nto the midx\" in place of \"*.pack\" above.\n\n"},{"id":"458810","messageId":"9934c512-7e82-b024-5292-71a0ebcf2351@jeffhostetler.com","threadId":"57611","inReplyTo":"c45ead51ffc5a9176493d627da8332d35a31d87c.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 7/7] tr2: dump names if config exist in multiple scopes","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-07-11T19:19:53Z","receivedAt":"2022-07-11T19:19:59Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 7/11/22 8:44 AM, Teng Long wrote:\n> When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\n> trace2 will prints \"interesting\" config values to log. Sometimes,\n> when a config set in multiple scope files, the following output\n> looks like (the irrelevant fields are omitted here as \"...\"):\n> \n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n> \n> As the log shows, even each config in different scope is dumped, but\n> we don't know which scope it comes from. Therefore, it's better to\n> add the scope names as well to make them be more recognizable. For\n> example, when execute:\n> \n> $ GIT_TRACE2_PERF=1 \\\n> > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n> > git rev-list --test-bitmap HEAD\"\n> \n> The following is the ouput (the irrelevant fields are omitted here\n> as \"...\"):\n> \n> Format normal:\n> ... git.c:461 ... def_param scope:system core.multipackindex=false\n> ... git.c:461 ... def_param scope:global core.multipackindex=false\n> ... git.c:461 ... def_param scope:local core.multipackindex=false\n> \n> Format perf:\n> \n> ... | def_param | ... | scope:system | core.multipackindex:false\n> ... | def_param | ... | scope:global | core.multipackindex:false\n> ... | def_param | ... | scope:local | core.multipackindex:false\n> \n> Format event:\n> \n> {\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> \n> Signed-off-by: Teng Long \n> ---\n> trace2/tr2_tgt_event.c | 3 +++\n> trace2/tr2_tgt_normal.c | 5 ++++-\n> trace2/tr2_tgt_perf.c | 9 +++++++--\n> 3 files changed, 14 insertions(+), 3 deletions(-)\n> \n> diff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\n> index c5c8cfbbaa..37a3163be1 100644\n> --- a/trace2/tr2_tgt_event.c\n> +++ b/trace2/tr2_tgt_event.c\n> @@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n> {\n> \tconst char *event_name = \"def_param\";\n> \tstruct json_writer jw = JSON_WRITER_INIT;\n> +\tenum config_scope scope = current_config_scope();\n> +\tconst char *scope_name = config_scope_name(scope);\n> \n> \tjw_object_begin(&jw, 0);\n> \tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n> +\tjw_object_string(&jw, \"scope\", scope_name);\n> \tjw_object_string(&jw, \"param\", param);\n> \tjw_object_string(&jw, \"value\", value);\n> \tjw_end(&jw);\n> diff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\n> index c42fbade7f..69f8033077 100644\n> --- a/trace2/tr2_tgt_normal.c\n> +++ b/trace2/tr2_tgt_normal.c\n> @@ -298,8 +298,11 @@ static void fn_param_fl(const char *file, int line, const char *param,\n> \t\t\tconst char *value)\n> {\n> \tstruct strbuf buf_payload = STRBUF_INIT;\n> +\tenum config_scope scope = current_config_scope();\n> +\tconst char *scope_name = config_scope_name(scope);\n> \n> -\tstrbuf_addf(&buf_payload, \"def_param %s=%s\", param, value);\n> +\tstrbuf_addf(&buf_payload, \"def_param scope:%s %s=%s\", scope_name, param,\n> +\t\t value);\n> \tnormal_io_write_fl(file, line, &buf_payload);\n> \tstrbuf_release(&buf_payload);\n> }\n> diff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\n> index a1eff8bea3..8cb792488c 100644\n> --- a/trace2/tr2_tgt_perf.c\n> +++ b/trace2/tr2_tgt_perf.c\n> @@ -441,12 +441,17 @@ static void fn_param_fl(const char *file, int line, const char *param,\n> {\n> \tconst char *event_name = \"def_param\";\n> \tstruct strbuf buf_payload = STRBUF_INIT;\n> +\tstruct strbuf scope_payload = STRBUF_INIT;\n> +\tenum config_scope scope = current_config_scope();\n> +\tconst char *scope_name = config_scope_name(scope);\n> \n> \tstrbuf_addf(&buf_payload, \"%s:%s\", param, value);\n> +\tstrbuf_addf(&scope_payload, \"%s:%s\", \"scope\", scope_name);\n> \n> -\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n> -\t\t\t &buf_payload);\n> +\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL,\n> +\t\t\t scope_payload.buf, &buf_payload);\n> \tstrbuf_release(&buf_payload);\n> +\tstrbuf_release(&scope_payload);\n> }\n> \n> static void fn_repo_fl(const char *file, int line,\n> \n\nNicely done.\n\nThanks for your attention on this.\nJeff\n\n"},{"id":"458822","messageId":"xmqqwncjmk76.fsf@gitster.g","threadId":"57611","inReplyTo":"94a64ba895a8eb9eb7d502e94cf3d9fdfb9d2887.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 1/7] clean: fixed issues related to text output format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-11T21:08:13Z","receivedAt":"2022-07-11T21:08:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> Subject: Re: [PATCH v6 1/7] clean: fixed issues related to text output format\n\nThis does not seem to be about \"git clean\". Mistitled?\n\npack-bitmap.c: fix formatting of error messages\n\nperhaps?\n"},{"id":"458824","messageId":"xmqqpmibmjjs.fsf@gitster.g","threadId":"57611","inReplyTo":"009cc49a18f2846c24256102e07437894ac16908.1657540174.git.dyroneteng@gmail.com","subject":"Re: [PATCH v6 4/7] pack-bitmap.c: don't ignore ENOENT silently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-11T21:22:15Z","receivedAt":"2022-07-11T21:22:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> When finished call git_open(), instead of ignoring ENOENT silently\n> and return error_errno(_(\"cannot stat...\")), it's better to check\n> the ENOENT before then output the warning.\n\nBoth the title and the body describes a complete opposite of what\nthe patch does, doesn't it?\n\n\tpack-bitmap.c: do not ignore error when opening a bitmap file\n\n\tCalls to git_open() to open the pack bitmap file and\n\tmulti-pack bitmap file do not report any error when they\n\tfail. These files are optional and it is not an error if\n\topen failed due to ENOENT, but we shouldn't be ignoring\n\tother kinds of errors.\n\nor something?\n\n> +\tif (fd < 0) {\n> +\t\tif (errno != ENOENT)\n> +\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n> +\t\tfree(bitmap_name);\n\nShowing the errno is good, but I do not think it should be enclosed\nin single quotes.\n\nOne thing you should consider when writing a \"this is an optional\nfile and not finding it is perfectly fine\" logic like this one is\nthat you may or may not want to ignore ENOTDIR. ENOTDIR is what you\nget when you say open(\"a/b\") and \"a\" exists as something other than\na directory. If you have a path with a slash in bitmap_name, and if\nin a sane and healthy repository the leading path should always\nexist (i.e. the file \"a/b\" may be missing, but the directory \"a/\"\nshould be there), then getting ENOTDIR is a noteworthy event. There\nmay be cases where ENOTDIR can justifiably and should be ignored.\n\n"},{"id":"458949","messageId":"20220713114450.53250-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"xmqqwncjmk76.fsf@gitster.g","subject":"Re: [PATCH v6 1/7] clean: fixed issues related to text output format","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-13T11:44:50Z","receivedAt":"2022-07-13T11:45:02Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\nOn Mon, 11 Jul 2022 14:08:13 -070 Junio C Hamano wrote:\n\n> This does not seem to be about \"git clean\". Mistitled?\n\nYes, will fix.\n\n> pack-bitmap.c: fix formatting of error messages\n>\n> perhaps?\n\nWill accept.\n\nThanks.\n"},{"id":"458982","messageId":"20220713141421.57768-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220711.86k08j7lyn.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 4/7] pack-bitmap.c: don't ignore ENOENT silently","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-13T14:14:21Z","receivedAt":"2022-07-13T14:14:34Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\nOn Mon, 11 Jul 2022 16:38:01 +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> Makes sense, but...\n> ...\n> This should just be warning_errno(\"cannot open '%s'\", bitmap_name),\n> unless I'm missing something...\n> ...\n> ...ditto.\n>\n\nYou are right, I didn't notice warning_errno() before, will improve.\n\nThanks.\n"},{"id":"459046","messageId":"20220714152552.3932-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"xmqqpmibmjjs.fsf@gitster.g","subject":"Re: [PATCH v6 4/7] pack-bitmap.c: don't ignore ENOENT silently","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-14T15:25:52Z","receivedAt":"2022-07-14T15:26:05Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Junio C Hamano writes:\n\n> Both the title and the body describes a complete opposite of what\n> the patch does, doesn't it?\n>\n> \tpack-bitmap.c: do not ignore error when opening a bitmap file\n>\n> \tCalls to git_open() to open the pack bitmap file and\n> \tmulti-pack bitmap file do not report any error when they\n> \tfail. These files are optional and it is not an error if\n> \topen failed due to ENOENT, but we shouldn't be ignoring\n> \tother kinds of errors.\n>\n> or something?\n\nYes, will apply, thanks for this detailed assistent.\n\n> > +\tif (fd < 0) {\n> > +\t\tif (errno != ENOENT)\n> > +\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n> > +\t\tfree(bitmap_name);\n>\n> Showing the errno is good, but I do not think it should be enclosed\n> in single quotes.\n\nYes, it should not be enclosed around and by Ævar Arnfjörð Bjarmason's advice\nthe \"warning\" should be replaced by \"warning_errno\", I agree with that,\nso, I think at the same time it solves this problem.\n\n> One thing you should consider when writing a \"this is an optional\n> file and not finding it is perfectly fine\" logic like this one is\n> that you may or may not want to ignore ENOTDIR.\n\n$ errno ENOTDIR\nENOTDIR 20 Not a directory\n\n>ENOTDIR is what you\n> get when you say open(\"a/b\") and \"a\" exists as something other than\n> a directory. If you have a path with a slash in bitmap_name, and if\n> in a sane and healthy repository the leading path should always\n> exist (i.e. the file \"a/b\" may be missing, but the directory \"a/\"\n> should be there), then getting ENOTDIR is a noteworthy event.\n\nMake sense, and I tried to move \"pack\" away and touch a new file named \"pack\",\nthen executed \"git fsck\", it will show the result about ENOTDIR:\n\n$ git fsck\nChecking object directories: 100% (256/256), done.\nerror: unable to open object pack directory: /Users/tenglong.tl/Downloads/dyrone/.git/objects/pack: Not a directory\n\nSo, this is how it works in git I think, caller should also print the \"errno\"\nwhen deal error and warning, append errno string as the last part and the\nfirst part is to describe what's wrong happened in git.\n\n> There may be cases where ENOTDIR can justifiably and should be ignored.\n\nYes. If I understand it correctly, I think it's the current scene.\n\nFor example, if this happens in case like \"./pack\" is a non-directory, then\nwe print warning which contains ENOTDIR description under current change.\n\nThanks.\n"},{"id":"459110","messageId":"20220715023422.6257-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220711.86bktv7l5v.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 5/7] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-15T02:34:22Z","receivedAt":"2022-07-15T02:34:34Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n\n> This is a logic error, as fstat() failed, but you're reproting errno()\n> after our call to close(), at which point it's anyone's guess what\n> \"errno\" is. It's only meaningful immediately after a system call.\n>\n> So either:\n>\n> error_errno(....);\n> close(fd);\n> return -1;\n>\n> Or even better:\n>\n> error_errno(...)\n> if (close(fd))\n> warning_errno(\"cannot close() bitmap file\");\n> return -1;\n>\n> Although that last one may be too pedantic.\n> ...\n> Same issue here.\n\n\nYes, this will be fixed in next patch.\n\nThanks.\n"},{"id":"459242","messageId":"20220718083600.11803-1-dyroneteng@gmail.com","threadId":"57611","inReplyTo":"220711.867d4j7kuj.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T08:36:00Z","receivedAt":"2022-07-18T08:36:16Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> I left some minor comments, another thing I'd just like to note, but it\n> largely pre-dates this series, so we can leave it for now is that I\n> found our mismatching reporting of the *.bitmap and/or *.pack kind of odd.\n\nOk.\n\n> I.e. this bit (a diff on top):\n>\n> \tdiff --git a/pack-bitmap.c b/pack-bitmap.c\n> \tindex 7e69093d5a8..8a8be5f6cad 100644\n> \t--- a/pack-bitmap.c\n> \t+++ b/pack-bitmap.c\n> \t@@ -381,23 +381,25 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n>\n> \t \tbitmap_name = pack_bitmap_filename(packfile);\n> \t \tfd = git_open(bitmap_name);\n> \t+\tfree(bitmap_name);\n>\n> \t \tif (fd < 0) {\n> \t \t\tif (errno != ENOENT)\n> \t-\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n> \t-\t\tfree(bitmap_name);\n> \t+\t\t\twarning_errno(\"cannot open bitmap for pack '%s'\",\n> \t+\t\t\t\t packfile->pack_name);\n> \t \t\treturn -1;\n> \t \t}\n> \t-\tfree(bitmap_name);\n>\n> \t \tif (fstat(fd, &st)) {\n> \t \t\tclose(fd);\n> \t-\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n> \t+\t\treturn error_errno(_(\"cannot fstat() bitmap file for pack '%s'\"),\n> \t+\t\t\t\t packfile->pack_name);\n> \t \t}\n>\n> \t \tif (bitmap_git->pack || bitmap_git->midx) {\n> \t \t\t/* ignore extra bitmap file; we can only handle one */\n> \t \t\twarning(_(\"ignoring extra bitmap file: '%s'\"), packfile->pack_name);\n> \t+\t\t/* ^^ because here we refer to the pack file, not the bitmap file */\n> \t \t\tclose(fd);\n> \t \t\treturn -1;\n> \t \t}\n>\n> As tha shows we sometimes talk about the path for the pack when\n> something is wrong with the bitmap, but other times the *.bitmap\n> associated with the pack.\n\nI think the reason is, the single-pack-bitmap filename based on the pack\nfilename as \"pack-$hash.bitmap\" format and the multi-pack-bitmap filename based\non the multi-pack-index as \"multi-pack-index-$hash.bitmap\" format. So, if want\nto open single-pack-bitmap, the corresponding packfile is needed. if want to\nopen multi-pack-bitmap, the multi-pack-index file is needed, then initialize\nfields in \"bitmap_index\" like mmap, etc.\n\nThat'\"s probably where these two methods and inputs come from and make the\n\"path\" different:\n\n static int open_pack_bitmap_1(struct bitmap_index *bitmap_git,\n struct packed_git *packfile)\n static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n struct multi_pack_index *midx)\n\nBy the way, I think maybe the \"multi-pack-index\" filename maybe could change\nto \"multi-pack-index-$hash.idx\", it could map the current multi-pack-bitmap\nfilename \"multi-pack-index-$sha.bitmap\", and \".idx\" could be consistent with\nsingle-pack-index file. Maybe it's better I think, but it also should consider\nthe backend compatibility.\n\n> Also a pre-existing issue: Why are we getting as far as opening a\n> *.bitmap file, if we then find that we have \"bitmap_git->pack ||\n> bitmap_git->midx\" already, and then ignore it? Maybe I've missed\n> something...\n>\n> All of the same oddness applies to open_midx_bitmap_1, except with \"path\n> to the midx\" in place of \"*.pack\" above.\n\nI'm not quite sure but in the code I think it's purposed to get the \"fd\" first\nby \"open..()\", then get the bitmap file size by \"fstat()\", finally \"mmap\" the\nbitmap file for some struct data initialization work.\n\nThanks.\n"},{"id":"459288","messageId":"cover.1658159745.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1657540174.git.dyroneteng@gmail.com","subject":"[PATCH v7 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:45:59Z","receivedAt":"2022-07-18T16:46:31Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Changes since v6:\n\n1. [1/7] Fixed the mistitled commit mesage.\n\n [PATCH v6 1/7] \"clean: fixed issues related to text output format\"\n\n to:\n\n [PATCH v7 1/7] \"pack-bitmap.c: fix formatting of error messages\"\n\n2. [4/7] replace \"warning()\" to \"warning_errno()\" and rewrite commit message.\n\n3. [5/7] fix the logic error, move \"error_errno()\" before close(fd) to\n avoid errno lost.\n\n4. [7/7] update Documentation/technical/api-trace2.txt here too.\n\nThanks.\n\nTeng Long (7):\n pack-bitmap.c: fix formatting of error messages\n pack-bitmap.c: mark more strings for translations\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: do not ignore error when opening a bitmap file\n pack-bitmap.c: using error() instead of silently returning -1\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n tr2: dump names if config exist in multiple scopes\n\n Documentation/technical/api-trace2.txt | 45 +++++++++++\n pack-bitmap.c | 103 ++++++++++++++-----------\n trace2/tr2_tgt_event.c | 3 +\n trace2/tr2_tgt_normal.c | 5 +-\n trace2/tr2_tgt_perf.c | 9 ++-\n 5 files changed, 117 insertions(+), 48 deletions(-)\n\nRange-diff against v6:\n1: 94a64ba895 ! 1: 7426b277ba clean: fixed issues related to text output format\n @@ Metadata\n Author: Teng Long \n \n ## Commit message ##\n - clean: fixed issues related to text output format\n + pack-bitmap.c: fix formatting of error messages\n \n There are some text output issues in 'pack-bitmap.c', they exist in\n die(), error() etc. This includes issues with capitalization the\n2: cadecd6b84 = 2: 4e493426a0 pack-bitmap.c: mark more strings for translations\n3: d8a2235cb0 = 3: 263f45ba96 pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n4: 009cc49a18 ! 4: d11ea092d5 pack-bitmap.c: don't ignore ENOENT silently\n @@ Metadata\n Author: Teng Long \n \n ## Commit message ##\n - pack-bitmap.c: don't ignore ENOENT silently\n + pack-bitmap.c: do not ignore error when opening a bitmap file\n \n - When finished call git_open(), instead of ignoring ENOENT silently\n - and return error_errno(_(\"cannot stat...\")), it's better to check\n - the ENOENT before then output the warning.\n + Calls to git_open() to open the pack bitmap file and\n + multi-pack bitmap file do not report any error when they\n + fail. These files are optional and it is not an error if\n + open failed due to ENOENT, but we shouldn't be ignoring\n + other kinds of errors.\n \n Signed-off-by: Teng Long \n \n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n -\tif (fd < 0)\n +\tif (fd < 0) {\n +\t\tif (errno != ENOENT)\n -+\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n ++\t\t\twarning_errno(\"cannot open '%s'\", bitmap_name);\n +\t\tfree(bitmap_name);\n \t\treturn -1;\n +\t}\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, st\n -\tif (fd < 0)\n +\tif (fd < 0) {\n +\t\tif (errno != ENOENT)\n -+\t\t\twarning(\"'%s' cannot open '%s'\", strerror(errno), bitmap_name);\n ++\t\t\twarning_errno(\"cannot open '%s'\", bitmap_name);\n +\t\tfree(bitmap_name);\n \t\treturn -1;\n +\t}\n5: 52783555e2 ! 5: f60efe78d6 pack-bitmap.c: using error() instead of silently returning -1\n @@ Commit message\n \n ## pack-bitmap.c ##\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n + \tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n ++\t\terror_errno(_(\"cannot fstat bitmap file\"));\n \t\tclose(fd);\n --\t\treturn -1;\n -+\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n + \t\treturn -1;\n \t}\n - \n - \tif (bitmap_git->pack || bitmap_git->midx) {\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (load_bitmap_header(bitmap_git) < 0)\n \t\tgoto cleanup;\n @@ pack-bitmap.c: static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n @@ pack-bitmap.c: static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n + \tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n ++\t\terror_errno(_(\"cannot fstat bitmap file\"));\n \t\tclose(fd);\n --\t\treturn -1;\n -+\t\treturn error_errno(_(\"cannot fstat bitmap file\"));\n + \t\treturn -1;\n \t}\n - \n - \tif (bitmap_git->pack || bitmap_git->midx) {\n6: 95832190b8 = 6: 83090308ad pack-bitmap.c: continue looping when first MIDX bitmap is found\n7: c45ead51ff ! 7: a01ae8478d tr2: dump names if config exist in multiple scopes\n @@ Commit message\n \n Signed-off-by: Teng Long \n \n + ## Documentation/technical/api-trace2.txt ##\n +@@ Documentation/technical/api-trace2.txt: at offset 508.\n + This example also shows that thread names are assigned in a racy manner\n + as each thread starts and allocates TLS storage.\n + \n ++Print Configs::\n ++\n ++\t Dump \"interesting\" config values to trace2 log.\n +++\n ++The environment variable `GIT_TRACE2_CONFIG_PARAMS` and configuration\n ++`trace2.configparams` can be used to output config values which you care\n ++about. For example, assume that we want to config different `color.ui`\n ++values in multiple scopes, such as:\n +++\n ++----------------\n ++$ git config --system color.ui never\n ++$ git config --global color.ui always\n ++$ git config --local color.ui auto\n ++$ git config --list --show-scope | grep 'color.ui'\n ++system color.ui=never\n ++global color.ui=always\n ++local color.ui=auto\n ++----------------\n +++\n ++Then, mark the config `color.ui` as \"interesting\" config with\n ++`GIT_TRACE2_CONFIG_PARAMS`:\n +++\n ++----------------\n ++$ export GIT_TRACE2_PERF_BRIEF=1\n ++$ export GIT_TRACE2_PERF=~/log.perf\n ++$ export GIT_TRACE2_CONFIG_PARAMS=color.ui\n ++$ git version\n ++...\n ++$ cat ~/log.perf\n ++d0 | main | version | | | | | ...\n ++d0 | main | start | | 0.000284 | | | /opt/git/master/bin/git version\n ++d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n ++d0 | main | cmd_name | | | | | version (version)\n ++d0 | main | exit | | 0.000419 | | | code:0\n ++d0 | main | atexit | | 0.000426 | | | code:0\n ++d0 | main | version | | | | | ...\n ++d0 | main | start | | 0.000275 | | | /opt/git/master/bin/git version\n ++d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n ++d0 | main | cmd_name | | | | | version (version)\n ++d0 | main | def_param | | | | | color.ui:never\n ++d0 | main | def_param | | | | | color.ui:always\n ++d0 | main | def_param | | | | | color.ui:auto\n ++d0 | main | exit | | 0.000543 | | | code:0\n ++d0 | main | atexit | | 0.000549 | | | code:0\n ++----------------\n + == Future Work\n + \n + === Relationship to the Existing Trace Api (api-trace.txt)\n +\n ## trace2/tr2_tgt_event.c ##\n @@ trace2/tr2_tgt_event.c: static void fn_param_fl(const char *file, int line, const char *param,\n {\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459289","messageId":"7426b277baed2df132fcb6bc75f0650769a5c7a1.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 1/7] pack-bitmap.c: fix formatting of error messages","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:00Z","receivedAt":"2022-07-18T16:46:43Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"There are some text output issues in 'pack-bitmap.c', they exist in\ndie(), error() etc. This includes issues with capitalization the\nfirst letter, newlines, error() instead of BUG(), and substitution\nthat don't have quotes around them.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 47 ++++++++++++++++++++++++-----------------------\n 1 file changed, 24 insertions(+), 23 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..0fdba5e8c5 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"strbuf.h\"\n #include \"tag.h\"\n #include \"diff.h\"\n #include \"revision.h\"\n@@ -138,7 +139,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"Failed to load bitmap index (corrupted?)\");\n+\t\terror(\"failed to load bitmap index (corrupted?)\");\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -160,14 +161,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"Corrupted bitmap index (too small)\");\n+\t\treturn error(\"corrupted bitmap index (too small)\");\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"Corrupted bitmap index file (wrong header)\");\n+\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"Unsupported version for bitmap index file (%d)\", index->version);\n+\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -176,7 +177,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \t\tunsigned char *index_end = index->map + index->map_size - the_hash_algo->rawsz;\n \n \t\tif ((flags & BITMAP_OPT_FULL_DAG) == 0)\n-\t\t\treturn error(\"Unsupported options for bitmap index file \"\n+\t\t\tBUG(\"unsupported options for bitmap index file \"\n \t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n@@ -215,7 +216,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"Duplicate entry in bitmap index: %s\", oid_to_hex(oid));\n+\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -274,13 +275,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"Corrupted bitmap pack index\");\n+\t\t\treturn error(\"corrupted bitmap pack index\");\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"Invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -330,7 +331,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +388,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -1628,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object %s not found in type bitmaps\",\n+\t\tdie(\"object '%s' not found in type bitmaps\",\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object %s does not have a unique type\",\n+\t\tdie(\"object '%s' does not have a unique type\",\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object %s: real type %s, expected: %s\",\n+\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1650,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&object->oid));\n+\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1665,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&commit->object.oid));\n+\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1687,21 +1688,21 @@ void test_bitmap_walk(struct rev_info *revs)\n \tif (revs->pending.nr != 1)\n \t\tdie(\"you must specify exactly one commit to test\");\n \n-\tfprintf(stderr, \"Bitmap v%d test (%d entries loaded)\\n\",\n+\tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n \n \troot = revs->pending.objects[0].item;\n \tbm = bitmap_for_commit(bitmap_git, (struct commit *)root);\n \n \tif (bm) {\n-\t\tfprintf(stderr, \"Found bitmap for %s. %d bits / %08x checksum\\n\",\n+\t\tfprintf_ln(stderr, \"Found bitmap for '%s'. %d bits / %08x checksum\",\n \t\t\toid_to_hex(&root->oid), (int)bm->bit_size, ewah_checksum(bm));\n \n \t\tresult = ewah_to_bitmap(bm);\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"Commit %s doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1726,7 +1727,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstop_progress(&tdata.prg);\n \n \tif (bitmap_equals(result, tdata.base))\n-\t\tfprintf(stderr, \"OK!\\n\");\n+\t\tfprintf_ln(stderr, \"OK!\");\n \telse\n \t\tdie(\"mismatch in bitmap results\");\n \n@@ -1749,7 +1750,7 @@ int test_bitmap_commits(struct repository *r)\n \t\tdie(\"failed to load bitmap indexes\");\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n-\t\tprintf(\"%s\\n\", oid_to_hex(&oid));\n+\t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n \t});\n \n \tfree_bitmap_index(bitmap_git);\n@@ -1774,7 +1775,7 @@ int test_bitmap_hashes(struct repository *r)\n \n \t\tnth_bitmap_object_oid(bitmap_git, &oid, index_pos);\n \n-\t\tprintf(\"%s %\"PRIu32\"\\n\",\n+\t\tprintf_ln(\"%s %\"PRIu32\"\",\n \t\t oid_to_hex(&oid), get_be32(bitmap_git->hashes + index_pos));\n \t}\n \n@@ -1936,7 +1937,7 @@ static off_t get_disk_usage_for_type(struct bitmap_index *bitmap_git,\n \t\t\t\t\tstruct object_id oid;\n \t\t\t\t\tnth_midxed_object_oid(&oid, bitmap_git->midx, midx_pos);\n \n-\t\t\t\t\tdie(_(\"could not find %s in pack %s at offset %\"PRIuMAX),\n+\t\t\t\t\tdie(_(\"could not find '%s' in pack '%s' at offset %\"PRIuMAX),\n \t\t\t\t\t oid_to_hex(&oid),\n \t\t\t\t\t pack->pack_name,\n \t\t\t\t\t (uintmax_t)offset);\n@@ -1972,7 +1973,7 @@ static off_t get_disk_usage_for_extended(struct bitmap_index *bitmap_git)\n \t\t\tcontinue;\n \n \t\tif (oid_object_info_extended(the_repository, &obj->oid, &oi, 0) < 0)\n-\t\t\tdie(_(\"unable to get disk usage of %s\"),\n+\t\t\tdie(_(\"unable to get disk usage of '%s'\"),\n \t\t\t oid_to_hex(&obj->oid));\n \n \t\ttotal += object_size;\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459290","messageId":"4e493426a0049854c7eec64029928e10ad2b414c.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 2/7] pack-bitmap.c: mark more strings for translations","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:01Z","receivedAt":"2022-07-18T16:46:59Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In pack-bitmap.c, some printed texts are translated, some are not.\nLet's support the translations of the bitmap related output.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 48 ++++++++++++++++++++++++------------------------\n 1 file changed, 24 insertions(+), 24 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 0fdba5e8c5..c970ab46dd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -139,7 +139,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"failed to load bitmap index (corrupted?)\");\n+\t\terror(_(\"failed to load bitmap index (corrupted?)\"));\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -161,14 +161,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"corrupted bitmap index (too small)\");\n+\t\treturn error(_(\"corrupted bitmap index (too small)\"));\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n+\t\treturn error(_(\"corrupted bitmap index file (wrong header)\"));\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n+\t\treturn error(_(\"unsupported version '%d' for bitmap index file\"), index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -182,7 +182,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n \t\t\tif (cache_size > index_end - index->map - header_size)\n-\t\t\t\treturn error(\"corrupted bitmap index file (too short to fit hash cache)\");\n+\t\t\t\treturn error(_(\"corrupted bitmap index file (too short to fit hash cache)\"));\n \t\t\tindex->hashes = (void *)(index_end - cache_size);\n \t\t\tindex_end -= cache_size;\n \t\t}\n@@ -216,7 +216,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n+\t\terror(_(\"duplicate entry in bitmap index: '%s'\"), oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -260,14 +260,14 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\tstruct object_id oid;\n \n \t\tif (index->map_size - index->map_pos < 6)\n-\t\t\treturn error(\"corrupt ewah bitmap: truncated header for entry %d\", i);\n+\t\t\treturn error(_(\"corrupt ewah bitmap: truncated header for entry %d\"), i);\n \n \t\tcommit_idx_pos = read_be32(index->map, &index->map_pos);\n \t\txor_offset = read_u8(index->map, &index->map_pos);\n \t\tflags = read_u8(index->map, &index->map_pos);\n \n \t\tif (nth_bitmap_object_oid(index, &oid, commit_idx_pos) < 0)\n-\t\t\treturn error(\"corrupt ewah bitmap: commit index %u out of range\",\n+\t\t\treturn error(_(\"corrupt ewah bitmap: commit index %u out of range\"),\n \t\t\t\t (unsigned)commit_idx_pos);\n \n \t\tbitmap = read_bitmap_1(index);\n@@ -275,13 +275,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"corrupted bitmap pack index\");\n+\t\t\treturn error(_(\"corrupted bitmap pack index\"));\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(_(\"invalid XOR offset in bitmap pack index\"));\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -331,7 +331,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n+\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -388,7 +388,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n+\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -819,7 +819,7 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,\n \t\trevs->include_check_data = &incdata;\n \n \t\tif (prepare_revision_walk(revs))\n-\t\t\tdie(\"revision walk setup failed\");\n+\t\t\tdie(_(\"revision walk setup failed\"));\n \n \t\tshow_data.bitmap_git = bitmap_git;\n \t\tshow_data.base = base;\n@@ -1629,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object '%s' not found in type bitmaps\",\n+\t\tdie(_(\"object '%s' not found in type bitmaps\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object '%s' does not have a unique type\",\n+\t\tdie(_(\"object '%s' does not have a unique type\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n+\t\tdie(_(\"object '%s': real type '%s', expected: '%s'\"),\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1651,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n+\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1666,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n+\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1683,10 +1683,10 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstruct ewah_bitmap *bm;\n \n \tif (!(bitmap_git = prepare_bitmap_git(revs->repo)))\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tif (revs->pending.nr != 1)\n-\t\tdie(\"you must specify exactly one commit to test\");\n+\t\tdie(_(\"you must specify exactly one commit to test\"));\n \n \tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n@@ -1702,7 +1702,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(_(\"commit '%s' doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1711,7 +1711,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tresult_popcnt = bitmap_popcount(result);\n \n \tif (prepare_revision_walk(revs))\n-\t\tdie(\"revision walk setup failed\");\n+\t\tdie(_(\"revision walk setup failed\"));\n \n \ttdata.bitmap_git = bitmap_git;\n \ttdata.base = bitmap_new();\n@@ -1729,7 +1729,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tif (bitmap_equals(result, tdata.base))\n \t\tfprintf_ln(stderr, \"OK!\");\n \telse\n-\t\tdie(\"mismatch in bitmap results\");\n+\t\tdie(_(\"mismatch in bitmap results\"));\n \n \tbitmap_free(result);\n \tbitmap_free(tdata.base);\n@@ -1747,7 +1747,7 @@ int test_bitmap_commits(struct repository *r)\n \tMAYBE_UNUSED void *value;\n \n \tif (!bitmap_git)\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n \t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459291","messageId":"263f45ba96c322665f102e22e1899c93541d87c4.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 3/7] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:02Z","receivedAt":"2022-07-18T16:47:14Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index,\nlet's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex c970ab46dd..7d8cc063fc 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -314,10 +314,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -369,14 +369,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459292","messageId":"d11ea092d511f3be4601f65ec54d492474fe821c.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 4/7] pack-bitmap.c: do not ignore error when opening a bitmap file","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:03Z","receivedAt":"2022-07-18T16:47:28Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Calls to git_open() to open the pack bitmap file and\nmulti-pack bitmap file do not report any error when they\nfail. These files are optional and it is not an error if\nopen failed due to ENOENT, but we shouldn't be ignoring\nother kinds of errors.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 7d8cc063fc..f8f9937c9e 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -317,10 +317,13 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n \tint fd = git_open(bitmap_name);\n \n-\tfree(bitmap_name);\n-\n-\tif (fd < 0)\n+\tif (fd < 0) {\n+\t\tif (errno != ENOENT)\n+\t\t\twarning_errno(\"cannot open '%s'\", bitmap_name);\n+\t\tfree(bitmap_name);\n \t\treturn -1;\n+\t}\n+\tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n@@ -376,10 +379,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n \tfd = git_open(bitmap_name);\n-\tfree(bitmap_name);\n \n-\tif (fd < 0)\n+\tif (fd < 0) {\n+\t\tif (errno != ENOENT)\n+\t\t\twarning_errno(\"cannot open '%s'\", bitmap_name);\n+\t\tfree(bitmap_name);\n \t\treturn -1;\n+\t}\n+\tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459293","messageId":"f60efe78d6a426f4a834680adf62a55937089aa9.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 5/7] pack-bitmap.c: using error() instead of silently returning -1","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:04Z","receivedAt":"2022-07-18T16:47:50Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex f8f9937c9e..318edd8e0d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -326,6 +326,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n+\t\terror_errno(_(\"cannot fstat bitmap file\"));\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -350,8 +351,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (load_bitmap_header(bitmap_git) < 0)\n \t\tgoto cleanup;\n \n-\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n+\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n+\t\terror(_(\"checksum doesn't match in MIDX and bitmap\"));\n \t\tgoto cleanup;\n+\t}\n \n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n@@ -389,6 +392,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n+\t\terror_errno(_(\"cannot fstat bitmap file\"));\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459294","messageId":"83090308ad63543c9becfb4cec9c1cedcffb1fdc.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 6/7] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:05Z","receivedAt":"2022-07-18T16:48:25Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"In \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to continue the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe reason for this is, there may exist more than one MIDX file in\na repo. The \"multi_pack_index\" struct is actually designed as a singly\nlinked list, and if a MIDX file has been already opened successfully,\nthen the other MIDX files will be skipped and left with a warning\n\"ignoring extra bitmap file.\" to the output.\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 318edd8e0d..d013ea0277 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -506,15 +506,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459295","messageId":"a01ae8478d3a8545241c5b064b6d369a330ee59f.1658159746.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v7 7/7] tr2: dump names if config exist in multiple scopes","fromName":"Teng Long","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-18T16:46:06Z","receivedAt":"2022-07-18T16:48:46Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\ntrace2 will prints \"interesting\" config values to log. Sometimes,\nwhen a config set in multiple scope files, the following output\nlooks like (the irrelevant fields are omitted here as \"...\"):\n\n...| def_param | ... | core.multipackindex:false\n...| def_param | ... | core.multipackindex:false\n...| def_param | ... | core.multipackindex:false\n\nAs the log shows, even each config in different scope is dumped, but\nwe don't know which scope it comes from. Therefore, it's better to\nadd the scope names as well to make them be more recognizable. For\nexample, when execute:\n\n $ GIT_TRACE2_PERF=1 \\\n > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n > git rev-list --test-bitmap HEAD\"\n\nThe following is the ouput (the irrelevant fields are omitted here\nas \"...\"):\n\nFormat normal:\n... git.c:461 ... def_param scope:system core.multipackindex=false\n... git.c:461 ... def_param scope:global core.multipackindex=false\n... git.c:461 ... def_param scope:local core.multipackindex=false\n\nFormat perf:\n\n... | def_param | ... | scope:system | core.multipackindex:false\n... | def_param | ... | scope:global | core.multipackindex:false\n... | def_param | ... | scope:local | core.multipackindex:false\n\nFormat event:\n\n{\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n{\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n{\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n\nSigned-off-by: Teng Long \n---\n Documentation/technical/api-trace2.txt | 45 ++++++++++++++++++++++++++\n trace2/tr2_tgt_event.c | 3 ++\n trace2/tr2_tgt_normal.c | 5 ++-\n trace2/tr2_tgt_perf.c | 9 ++++--\n 4 files changed, 59 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex bb13ca3db8..48205a5ac5 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -1207,6 +1207,51 @@ at offset 508.\n This example also shows that thread names are assigned in a racy manner\n as each thread starts and allocates TLS storage.\n \n+Print Configs::\n+\n+\t Dump \"interesting\" config values to trace2 log.\n++\n+The environment variable `GIT_TRACE2_CONFIG_PARAMS` and configuration\n+`trace2.configparams` can be used to output config values which you care\n+about. For example, assume that we want to config different `color.ui`\n+values in multiple scopes, such as:\n++\n+----------------\n+$ git config --system color.ui never\n+$ git config --global color.ui always\n+$ git config --local color.ui auto\n+$ git config --list --show-scope | grep 'color.ui'\n+system color.ui=never\n+global color.ui=always\n+local color.ui=auto\n+----------------\n++\n+Then, mark the config `color.ui` as \"interesting\" config with\n+`GIT_TRACE2_CONFIG_PARAMS`:\n++\n+----------------\n+$ export GIT_TRACE2_PERF_BRIEF=1\n+$ export GIT_TRACE2_PERF=~/log.perf\n+$ export GIT_TRACE2_CONFIG_PARAMS=color.ui\n+$ git version\n+...\n+$ cat ~/log.perf\n+d0 | main | version | | | | | ...\n+d0 | main | start | | 0.000284 | | | /opt/git/master/bin/git version\n+d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n+d0 | main | cmd_name | | | | | version (version)\n+d0 | main | exit | | 0.000419 | | | code:0\n+d0 | main | atexit | | 0.000426 | | | code:0\n+d0 | main | version | | | | | ...\n+d0 | main | start | | 0.000275 | | | /opt/git/master/bin/git version\n+d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n+d0 | main | cmd_name | | | | | version (version)\n+d0 | main | def_param | | | | | color.ui:never\n+d0 | main | def_param | | | | | color.ui:always\n+d0 | main | def_param | | | | | color.ui:auto\n+d0 | main | exit | | 0.000543 | | | code:0\n+d0 | main | atexit | | 0.000549 | | | code:0\n+----------------\n == Future Work\n \n === Relationship to the Existing Trace Api (api-trace.txt)\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex c5c8cfbbaa..37a3163be1 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n \tconst char *event_name = \"def_param\";\n \tstruct json_writer jw = JSON_WRITER_INIT;\n+\tenum config_scope scope = current_config_scope();\n+\tconst char *scope_name = config_scope_name(scope);\n \n \tjw_object_begin(&jw, 0);\n \tevent_fmt_prepare(event_name, file, line, NULL, &jw);\n+\tjw_object_string(&jw, \"scope\", scope_name);\n \tjw_object_string(&jw, \"param\", param);\n \tjw_object_string(&jw, \"value\", value);\n \tjw_end(&jw);\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex c42fbade7f..69f8033077 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -298,8 +298,11 @@ static void fn_param_fl(const char *file, int line, const char *param,\n \t\t\tconst char *value)\n {\n \tstruct strbuf buf_payload = STRBUF_INIT;\n+\tenum config_scope scope = current_config_scope();\n+\tconst char *scope_name = config_scope_name(scope);\n \n-\tstrbuf_addf(&buf_payload, \"def_param %s=%s\", param, value);\n+\tstrbuf_addf(&buf_payload, \"def_param scope:%s %s=%s\", scope_name, param,\n+\t\t value);\n \tnormal_io_write_fl(file, line, &buf_payload);\n \tstrbuf_release(&buf_payload);\n }\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a1eff8bea3..8cb792488c 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -441,12 +441,17 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n \tconst char *event_name = \"def_param\";\n \tstruct strbuf buf_payload = STRBUF_INIT;\n+\tstruct strbuf scope_payload = STRBUF_INIT;\n+\tenum config_scope scope = current_config_scope();\n+\tconst char *scope_name = config_scope_name(scope);\n \n \tstrbuf_addf(&buf_payload, \"%s:%s\", param, value);\n+\tstrbuf_addf(&scope_payload, \"%s:%s\", \"scope\", scope_name);\n \n-\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n-\t\t\t &buf_payload);\n+\tperf_io_write_fl(file, line, event_name, NULL, NULL, NULL,\n+\t\t\t scope_payload.buf, &buf_payload);\n \tstrbuf_release(&buf_payload);\n+\tstrbuf_release(&scope_payload);\n }\n \n static void fn_repo_fl(const char *file, int line,\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459310","messageId":"xmqqa696fdva.fsf@gitster.g","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"Re: [PATCH v7 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-18T18:57:13Z","receivedAt":"2022-07-18T18:57:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> Changes since v6:\n>\n> 1. [1/7] Fixed the mistitled commit mesage.\n>\n> [PATCH v6 1/7] \"clean: fixed issues related to text output format\"\n>\n> to:\n>\n> [PATCH v7 1/7] \"pack-bitmap.c: fix formatting of error messages\"\n>\n> 2. [4/7] replace \"warning()\" to \"warning_errno()\" and rewrite commit message.\n>\n> 3. [5/7] fix the logic error, move \"error_errno()\" before close(fd) to\n> avoid errno lost.\n>\n> 4. [7/7] update Documentation/technical/api-trace2.txt here too.\n>\n> Thanks.\n>\n> Teng Long (7):\n> pack-bitmap.c: fix formatting of error messages\n> pack-bitmap.c: mark more strings for translations\n> pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n> pack-bitmap.c: do not ignore error when opening a bitmap file\n> pack-bitmap.c: using error() instead of silently returning -1\n> pack-bitmap.c: continue looping when first MIDX bitmap is found\n> tr2: dump names if config exist in multiple scopes\n\nThis has been \"cooking\" on the list for quite some time and I found\nthat all parts that I had comments on earlier iterations are now in\ngood shape.\n\nFellow reviewers, how does this round look? The only gripe I have\nis that the last one seems totally disconnected from the rest, but\nThat's minor.\n\nThanks.\n\n"},{"id":"459314","messageId":"220718.86edyiw82r.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"xmqqa696fdva.fsf@gitster.g","subject":"Re: [PATCH v7 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-18T19:07:30Z","receivedAt":"2022-07-18T19:10:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 18 2022, Junio C Hamano wrote:\n\n> Teng Long writes:\n>\n>> Changes since v6:\n>>\n>> 1. [1/7] Fixed the mistitled commit mesage.\n>>\n>> [PATCH v6 1/7] \"clean: fixed issues related to text output format\"\n>>\n>> to:\n>>\n>> [PATCH v7 1/7] \"pack-bitmap.c: fix formatting of error messages\"\n>>\n>> 2. [4/7] replace \"warning()\" to \"warning_errno()\" and rewrite commit message.\n>>\n>> 3. [5/7] fix the logic error, move \"error_errno()\" before close(fd) to\n>> avoid errno lost.\n>>\n>> 4. [7/7] update Documentation/technical/api-trace2.txt here too.\n>>\n>> Thanks.\n>>\n>> Teng Long (7):\n>> pack-bitmap.c: fix formatting of error messages\n>> pack-bitmap.c: mark more strings for translations\n>> pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n>> pack-bitmap.c: do not ignore error when opening a bitmap file\n>> pack-bitmap.c: using error() instead of silently returning -1\n>> pack-bitmap.c: continue looping when first MIDX bitmap is found\n>> tr2: dump names if config exist in multiple scopes\n>\n> This has been \"cooking\" on the list for quite some time and I found\n> that all parts that I had comments on earlier iterations are now in\n> good shape.\n>\n> Fellow reviewers, how does this round look? The only gripe I have\n> is that the last one seems totally disconnected from the rest, but\n> That's minor.\n\nYeah likewise, it even applies directly on master. But I can live with\nit :)\n\nOne minor nit is that something like this (which needs to be fleshened\nup) should be fixed up into 7/7 (and maybe we want to keep the \"...\"?):\n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex 49bb1ca1924..ce544982a37 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -716,7 +716,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n ------------\n {\n \t\"event\":\"def_param\",\n-\t...\n+\t\"scope\": ...\n \t\"param\":\"core.abbrev\",\n \t\"value\":\"7\"\n }\n\nAnd that the addition to api-trace2.txt seems to partially be something\nthat should just link to Documentation/config/trace2.txt, i.e. it's\ngenerally documenting an existing facility.\n\nI think it would be great in any case to have that 7/7 split into what\nwe do now & docs for that, and then the minor addition of \"scope\".\n\nThe rest all looks good to me, and seems to fully address the feedback\nin previous rounds.\n"},{"id":"459325","messageId":"d66f6bc7-4cd0-f585-83e9-9f7e2ee94d68@jeffhostetler.com","threadId":"57611","inReplyTo":"a01ae8478d3a8545241c5b064b6d369a330ee59f.1658159746.git.dyroneteng@gmail.com","subject":"Re: [PATCH v7 7/7] tr2: dump names if config exist in multiple scopes","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-07-18T20:13:23Z","receivedAt":"2022-07-18T20:13:41Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 7/18/22 12:46 PM, Teng Long wrote:\n> When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\n> trace2 will prints \"interesting\" config values to log. Sometimes,\n> when a config set in multiple scope files, the following output\n> looks like (the irrelevant fields are omitted here as \"...\"):\n> \n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n> \n> As the log shows, even each config in different scope is dumped, but\n> we don't know which scope it comes from. Therefore, it's better to\n> add the scope names as well to make them be more recognizable. For\n> example, when execute:\n> \n> $ GIT_TRACE2_PERF=1 \\\n> > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n> > git rev-list --test-bitmap HEAD\"\n> \n> The following is the ouput (the irrelevant fields are omitted here\n> as \"...\"):\n> \n> Format normal:\n> ... git.c:461 ... def_param scope:system core.multipackindex=false\n> ... git.c:461 ... def_param scope:global core.multipackindex=false\n> ... git.c:461 ... def_param scope:local core.multipackindex=false\n> \n> Format perf:\n> \n> ... | def_param | ... | scope:system | core.multipackindex:false\n> ... | def_param | ... | scope:global | core.multipackindex:false\n> ... | def_param | ... | scope:local | core.multipackindex:false\n> \n> Format event:\n> \n> {\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> \n> Signed-off-by: Teng Long \n> ---\n> Documentation/technical/api-trace2.txt | 45 ++++++++++++++++++++++++++\n> trace2/tr2_tgt_event.c | 3 ++\n> trace2/tr2_tgt_normal.c | 5 ++-\n> trace2/tr2_tgt_perf.c | 9 ++++--\n> 4 files changed, 59 insertions(+), 3 deletions(-)\n> \n> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n> index bb13ca3db8..48205a5ac5 100644\n> --- a/Documentation/technical/api-trace2.txt\n> +++ b/Documentation/technical/api-trace2.txt\n> @@ -1207,6 +1207,51 @@ at offset 508.\n> This example also shows that thread names are assigned in a racy manner\n> as each thread starts and allocates TLS storage.\n> \n> +Print Configs::\n> +\n> +\t Dump \"interesting\" config values to trace2 log.\n> ++\n> +The environment variable `GIT_TRACE2_CONFIG_PARAMS` and configuration\n> +`trace2.configparams` can be used to output config values which you care\n> +about. For example, assume that we want to config different `color.ui`\n> +values in multiple scopes, such as:\n> ++\n> +----------------\n> +$ git config --system color.ui never\n> +$ git config --global color.ui always\n> +$ git config --local color.ui auto\n> +$ git config --list --show-scope | grep 'color.ui'\n> +system color.ui=never\n> +global color.ui=always\n> +local color.ui=auto\n> +----------------\n> ++\n> +Then, mark the config `color.ui` as \"interesting\" config with\n> +`GIT_TRACE2_CONFIG_PARAMS`:\n> ++\n> +----------------\n> +$ export GIT_TRACE2_PERF_BRIEF=1\n> +$ export GIT_TRACE2_PERF=~/log.perf\n> +$ export GIT_TRACE2_CONFIG_PARAMS=color.ui\n> +$ git version\n> +...\n> +$ cat ~/log.perf\n> +d0 | main | version | | | | | ...\n> +d0 | main | start | | 0.000284 | | | /opt/git/master/bin/git version\n> +d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n> +d0 | main | cmd_name | | | | | version (version)\n> +d0 | main | exit | | 0.000419 | | | code:0\n> +d0 | main | atexit | | 0.000426 | | | code:0\n> +d0 | main | version | | | | | ...\n> +d0 | main | start | | 0.000275 | | | /opt/git/master/bin/git version\n> +d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n> +d0 | main | cmd_name | | | | | version (version)\n> +d0 | main | def_param | | | | | color.ui:never\n> +d0 | main | def_param | | | | | color.ui:always\n> +d0 | main | def_param | | | | | color.ui:auto\n\nshouldn't this example in the docs have the scope value in the\nnext to last column as you noted in the cover letter ?\n\n\n> +d0 | main | exit | | 0.000543 | | | code:0\n> +d0 | main | atexit | | 0.000549 | | | code:0\n> +----------------\n> == Future Work\n...\n\nOtherwise, this all LGTM.\nThanks\nJeff\n"},{"id":"459347","messageId":"20220719074053.37282-1-tenglong.tl@tenglongtldeMacBook-Pro.local","threadId":"57611","inReplyTo":"d66f6bc7-4cd0-f585-83e9-9f7e2ee94d68@jeffhostetler.com","subject":"Re: [PATCH v7 7/7] tr2: dump names if config exist in multiple scopes","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-19T07:40:53Z","receivedAt":"2022-07-19T07:41:48Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Jeff Hostetler writes:\n\n> shouldn't this example in the docs have the scope value in the\n> next to last column as you noted in the cover letter ?\n\noops, sorry for that. I will send a quick fix today.\n\nThanks.\n"},{"id":"459351","messageId":"20220719112612.46679-1-tenglong.tl@tenglongtldeMacBook-Pro.local","threadId":"57611","inReplyTo":"220718.86edyiw82r.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v7 0/7] trace2: dump scope when print \"interesting\" config","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-19T11:26:12Z","receivedAt":"2022-07-19T11:26:26Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"> Yeah likewise, it even applies directly on master. But I can live with\n> it :)\n\nSorry, may I ask what's the recommanded way to deal this, alway use\ntag or master on update?\n\n> One minor nit is that something like this (which needs to be fleshened\n> up) should be fixed up into 7/7 (and maybe we want to keep the \"...\"?):\n>\n> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n> index 49bb1ca1924..ce544982a37 100644\n> --- a/Documentation/technical/api-trace2.txt\n> +++ b/Documentation/technical/api-trace2.txt\n> @@ -716,7 +716,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n> ------------\n> {\n> \t\"event\":\"def_param\",\n> -\t...\n> +\t\"scope\": ...\n> \t\"param\":\"core.abbrev\",\n> \t\"value\":\"7\"\n> }\n\nI do a little check, the json format in this scenario is like :\n\n{\n \"event\": \"def_param\",\n \"sid\": \"20220719T075535.279369Z-H1b0a19dc-P000093db\",\n \"thread\": \"main\",\n \"time\": \"2022-07-19T07:55:35.280720Z\",\n \"file\": \"git.c\",\n \"line\": 461,\n \"scope\": \"global\",\n \"param\": \"color.ui\",\n \"value\": \"always\"\n}\n\nSo, I think this is ok maybe:\n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex bb13ca3db8..d66da52686 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -717,6 +717,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n {\n \"event\":\"def_param\",\n ...\n+ \"scope\":...\n \"param\":\"core.abbrev\",\n \"value\":\"7\"\n }\n\n> And that the addition to api-trace2.txt seems to partially be something\n> that should just link to Documentation/config/trace2.txt, i.e. it's\n> generally documenting an existing facility.\n\nDo you mean the modification about `trace2.configParams` and you suggest\nto make a link to Documentation/config/trace2.txt(actually as git-config[1])\nin /Documentation/technical/api-trace2.txt?\n\n> I think it would be great in any case to have that 7/7 split into what\n> we do now & docs for that, and then the minor addition of \"scope\".\n\nLet me try to understand this, it's better to split [7/7] into two commits.\n\nFirst commit is to add docs in /Documentation/technical/api-trace2.txt to let\nreader to find the print-config ability, by the way link the doc about the\nGIT_TRACE2_CONFIG_PARAM and trace2.configparami in git-config[1].\n\nLike:\n\ncommit 2db47572d4462e3788a92fd355b97df13b9bcc39\nAuthor: Teng Long \nDate: Tue Jul 19 17:30:35 2022 +0800\n\n api-trace2.txt: add docs to print config\n\n It's supported to print \"interesting\" config value to tr2 log by\n setting the \"GIT_TRACE2_CONFIG_PARAMS\" environment variable and\n the \"trace2.configparam\" config, let's add the related docs.\n\n Signed-off-by: Teng Long \n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex bb13ca3db8..4e411f3306 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -1207,6 +1207,45 @@ at offset 508.\n This example also shows that thread names are assigned in a racy manner\n as each thread starts and allocates TLS storage.\n\n+Print Configs::\n+\n+ Dump \"interesting\" config values to trace2 log.\n++\n+The environment variable `GIT_TRACE2_CONFIG_PARAMS` and configuration\n+`trace2.configparams` can be used to output config values which you care\n+about(see linkgit:git-config[1). For example, assume that we want to\n+config different `color.ui` values in multiple scopes, such as:\n++\n+----------------\n+$ git config --system color.ui never\n+$ git config --global color.ui always\n+$ git config --local color.ui auto\n+$ git config --list --show-scope | grep 'color.ui'\n+system color.ui=never\n+global color.ui=always\n+local color.ui=auto\n+----------------\n++\n+Then, mark the config `color.ui` as \"interesting\" config with\n+`GIT_TRACE2_CONFIG_PARAMS`:\n++\n+----------------\n+$ export GIT_TRACE2_PERF_BRIEF=1\n+$ export GIT_TRACE2_PERF=~/log.perf\n+$ export GIT_TRACE2_CONFIG_PARAMS=color.ui\n+$ git version\n+...\n+$ cat ~/log.perf\n+d0 | main | version | | | | | ...\n+d0 | main | start | | 0.000260 | | | /opt/git/master/bin/git version\n+d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n+d0 | main | cmd_name | | | | | version (version)\n+d0 | main | def_param | | | | | color.ui:never\n+d0 | main | def_param | | | | | color.ui:always\n+d0 | main | def_param | | | | | color.ui:auto\n+d0 | main | exit | | 0.000470 | | | code:0\n+d0 | main | atexit | | 0.000477 | | | code:0\n+----------------\n == Future Work\n\nSecond is actually what we do in [7/7], modify to support print scope names\nand update the docs which based on what we just add in the first commit\n\nLike:\n\n\ncommit 615ab4864fce6b8042778aa78799ac2656785710 (HEAD -> tl/bitmap-append-trace2-outputs)\nAuthor: Teng Long \nDate: Thu Jul 7 21:48:32 2022 +0800\n\n tr2: dump names if config exist in multiple scopes\n\n When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\n trace2 will prints \"interesting\" config values to log. Sometimes,\n when a config set in multiple scope files, the following output\n looks like (the irrelevant fields are omitted here as \"...\"):\n\n ...| def_param | ... | core.multipackindex:false\n ...| def_param | ... | core.multipackindex:false\n ...| def_param | ... | core.multipackindex:false\n\n As the log shows, even each config in different scope is dumped, but\n we don't know which scope it comes from. Therefore, it's better to\n add the scope names as well to make them be more recognizable. For\n example, when execute:\n\n $ GIT_TRACE2_PERF=1 \\\n > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n > git rev-list --test-bitmap HEAD\"\n\n The following is the ouput (the irrelevant fields are omitted here\n as \"...\"):\n\n Format normal:\n ... git.c:461 ... def_param scope:system core.multipackindex=false\n ... git.c:461 ... def_param scope:global core.multipackindex=false\n ... git.c:461 ... def_param scope:local core.multipackindex=false\n\n Format perf:\n\n ... | def_param | ... | scope:system | core.multipackindex:false\n ... | def_param | ... | scope:global | core.multipackindex:false\n ... | def_param | ... | scope:local | core.multipackindex:false\n\n Format event:\n\n {\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n {\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n {\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n\n Signed-off-by: Teng Long \n\ndiff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\nindex 4e411f3306..50f1e0c259 100644\n--- a/Documentation/technical/api-trace2.txt\n+++ b/Documentation/technical/api-trace2.txt\n@@ -717,6 +717,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n {\n \"event\":\"def_param\",\n ...\n+ \"scope\":...\n \"param\":\"core.abbrev\",\n \"value\":\"7\"\n }\n@@ -1240,9 +1241,9 @@ d0 | main | version | | | |\n d0 | main | start | | 0.000260 | | | /opt/git/master/bin/git version\n d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n d0 | main | cmd_name | | | | | version (version)\n-d0 | main | def_param | | | | | color.ui:never\n-d0 | main | def_param | | | | | color.ui:always\n-d0 | main | def_param | | | | | color.ui:auto\n+d0 | main | def_param | | | | scope:system | color.ui:never\n+d0 | main | def_param | | | | scope:global | color.ui:always\n+d0 | main | def_param | | | | scope:local | color.ui:auto\n d0 | main | exit | | 0.000470 | | | code:0\n d0 | main | atexit | | 0.000477 | | | code:0\n ----------------\ndiff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\nindex c5c8cfbbaa..37a3163be1 100644\n--- a/trace2/tr2_tgt_event.c\n+++ b/trace2/tr2_tgt_event.c\n@@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n const char *event_name = \"def_param\";\n struct json_writer jw = JSON_WRITER_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n\n jw_object_begin(&jw, 0);\n event_fmt_prepare(event_name, file, line, NULL, &jw);\n+ jw_object_string(&jw, \"scope\", scope_name);\n jw_object_string(&jw, \"param\", param);\n jw_object_string(&jw, \"value\", value);\n jw_end(&jw);\ndiff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\nindex c42fbade7f..69f8033077 100644\n--- a/trace2/tr2_tgt_normal.c\n+++ b/trace2/tr2_tgt_normal.c\n@@ -298,8 +298,11 @@ static void fn_param_fl(const char *file, int line, const char *param,\n const char *value)\n {\n struct strbuf buf_payload = STRBUF_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n\n- strbuf_addf(&buf_payload, \"def_param %s=%s\", param, value);\n+ strbuf_addf(&buf_payload, \"def_param scope:%s %s=%s\", scope_name, param,\n+ value);\n normal_io_write_fl(file, line, &buf_payload);\n strbuf_release(&buf_payload);\n }\ndiff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\nindex a1eff8bea3..8cb792488c 100644\n--- a/trace2/tr2_tgt_perf.c\n+++ b/trace2/tr2_tgt_perf.c\n@@ -441,12 +441,17 @@ static void fn_param_fl(const char *file, int line, const char *param,\n {\n const char *event_name = \"def_param\";\n struct strbuf buf_payload = STRBUF_INIT;\n+ struct strbuf scope_payload = STRBUF_INIT;\n+ enum config_scope scope = current_config_scope();\n+ const char *scope_name = config_scope_name(scope);\n\n strbuf_addf(&buf_payload, \"%s:%s\", param, value);\n+ strbuf_addf(&scope_payload, \"%s:%s\", \"scope\", scope_name);\n\n- perf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n- &buf_payload);\n+ perf_io_write_fl(file, line, event_name, NULL, NULL, NULL,\n+ scope_payload.buf, &buf_payload);\n strbuf_release(&buf_payload);\n+ strbuf_release(&scope_payload);\n }\n\n static void fn_repo_fl(const char *file, int line,\n\n\nAm I understand accurately?\nThanks.\n"},{"id":"459354","messageId":"220719.86tu7duxl3.gmgdl@evledraar.gmail.com","threadId":"57611","inReplyTo":"20220719112612.46679-1-tenglong.tl@tenglongtldeMacBook-Pro.local","subject":"Re: [PATCH v7 0/7] trace2: dump scope when print \"interesting\" config","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-19T11:42:37Z","receivedAt":"2022-07-19T11:54:42Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 19 2022, tenglong.tl wrote:\n\n>> Yeah likewise, it even applies directly on master. But I can live with\n>> it :)\n>\n> Sorry, may I ask what's the recommanded way to deal this, alway use\n> tag or master on update?\n\nWe generally submit patches on a single \"topic\", what a \"topic\" is is\noften fuzzy, and sometimes a topic that's mostly trying to do X will fix\nor change some unrelated Y \"while at it\".\n\nBut patch or patches at the end of a series don't depend on anything\nthat comes before them, and could be \"cherry-pick\"'d directly on top of\n\"master\" that's generally a sign that you should be submitting two sets\nof patches, not one.\n\nPer\nhttps://lore.kernel.org/git/2016ef2e342c2ec6517afa8ec3e57035021fb965.1650547400.git.dyroneteng@gmail.com/\nthe \"let's log config\" is just something you happened to run into on\nthis topic, but it might have just as well been some other command.\n\nSo I think it's better to split it up into its own topic.\n\n>> One minor nit is that something like this (which needs to be fleshened\n>> up) should be fixed up into 7/7 (and maybe we want to keep the \"...\"?):\n>>\n>> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n>> index 49bb1ca1924..ce544982a37 100644\n>> --- a/Documentation/technical/api-trace2.txt\n>> +++ b/Documentation/technical/api-trace2.txt\n>> @@ -716,7 +716,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n>> ------------\n>> {\n>> \t\"event\":\"def_param\",\n>> -\t...\n>> +\t\"scope\": ...\n>> \t\"param\":\"core.abbrev\",\n>> \t\"value\":\"7\"\n>> }\n>\n> I do a little check, the json format in this scenario is like :\n>\n> {\n> \"event\": \"def_param\",\n> \"sid\": \"20220719T075535.279369Z-H1b0a19dc-P000093db\",\n> \"thread\": \"main\",\n> \"time\": \"2022-07-19T07:55:35.280720Z\",\n> \"file\": \"git.c\",\n> \"line\": 461,\n> \"scope\": \"global\",\n> \"param\": \"color.ui\",\n> \"value\": \"always\"\n> }\n>\n> So, I think this is ok maybe:\n>\n> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n> index bb13ca3db8..d66da52686 100644\n> --- a/Documentation/technical/api-trace2.txt\n> +++ b/Documentation/technical/api-trace2.txt\n> @@ -717,6 +717,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n> {\n> \"event\":\"def_param\",\n> ...\n> + \"scope\":...\n> \"param\":\"core.abbrev\",\n> \"value\":\"7\"\n\nYes, that sound good, although I'd make that \"scope\" line be:\n\n\t\"scope\": \"global\" | \"worktree\" | \n\nOr just say:\n\n\t\"scope\": ,\n\nWhich covers all the possibilities, without hardcoding them there.\n\n> }\n>\n>> And that the addition to api-trace2.txt seems to partially be something\n>> that should just link to Documentation/config/trace2.txt, i.e. it's\n>> generally documenting an existing facility.\n>\n> Do you mean the modification about `trace2.configParams` and you suggest\n> to make a link to Documentation/config/trace2.txt(actually as git-config[1])\n> in /Documentation/technical/api-trace2.txt?\n\nI mean that part of what you're adding is about this new \"scope\"\nfeature, but another part just seems to be explaining how the\ntrace2.configParams works in general.\n\nFor the \"works in general\" let's either link to git-config(1), or if\nthat explanation is lacking improve it & link to it.\n\n>> I think it would be great in any case to have that 7/7 split into what\n>> we do now & docs for that, and then the minor addition of \"scope\".\n>\n> Let me try to understand this, it's better to split [7/7] into two commits.\n>\n> First commit is to add docs in /Documentation/technical/api-trace2.txt to let\n> reader to find the print-config ability, by the way link the doc about the\n> GIT_TRACE2_CONFIG_PARAM and trace2.configparami in git-config[1].\n>\n> Like:\n>\n> commit 2db47572d4462e3788a92fd355b97df13b9bcc39\n> Author: Teng Long \n> Date: Tue Jul 19 17:30:35 2022 +0800\n>\n> api-trace2.txt: add docs to print config\n>\n> It's supported to print \"interesting\" config value to tr2 log by\n> setting the \"GIT_TRACE2_CONFIG_PARAMS\" environment variable and\n> the \"trace2.configparam\" config, let's add the related docs.\n>\n> Signed-off-by: Teng Long \n>\n> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n> index bb13ca3db8..4e411f3306 100644\n> --- a/Documentation/technical/api-trace2.txt\n> +++ b/Documentation/technical/api-trace2.txt\n> @@ -1207,6 +1207,45 @@ at offset 508.\n> This example also shows that thread names are assigned in a racy manner\n> as each thread starts and allocates TLS storage.\n>\n> +Print Configs::\n> +\n> + Dump \"interesting\" config values to trace2 log.\n> ++\n> +The environment variable `GIT_TRACE2_CONFIG_PARAMS` and configuration\n> +`trace2.configparams` can be used to output config values which you care\n> +about(see linkgit:git-config[1). For example, assume that we want to\n> +config different `color.ui` values in multiple scopes, such as:\n> ++\n> +----------------\n> +$ git config --system color.ui never\n> +$ git config --global color.ui always\n> +$ git config --local color.ui auto\n> +$ git config --list --show-scope | grep 'color.ui'\n> +system color.ui=never\n> +global color.ui=always\n> +local color.ui=auto\n> +----------------\n> ++\n> +Then, mark the config `color.ui` as \"interesting\" config with\n> +`GIT_TRACE2_CONFIG_PARAMS`:\n> ++\n> +----------------\n> +$ export GIT_TRACE2_PERF_BRIEF=1\n> +$ export GIT_TRACE2_PERF=~/log.perf\n> +$ export GIT_TRACE2_CONFIG_PARAMS=color.ui\n> +$ git version\n> +...\n> +$ cat ~/log.perf\n> +d0 | main | version | | | | | ...\n> +d0 | main | start | | 0.000260 | | | /opt/git/master/bin/git version\n> +d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n> +d0 | main | cmd_name | | | | | version (version)\n> +d0 | main | def_param | | | | | color.ui:never\n> +d0 | main | def_param | | | | | color.ui:always\n> +d0 | main | def_param | | | | | color.ui:auto\n> +d0 | main | exit | | 0.000470 | | | code:0\n> +d0 | main | atexit | | 0.000477 | | | code:0\n> +----------------\n> == Future Work\n\nYes, something like that, although it's a bit odd to discuss \"scope\"\nhere and not have the trace show it yet, but that's fixed below.:\n\n> Second is actually what we do in [7/7], modify to support print scope names\n> and update the docs which based on what we just add in the first commit\n>\n> Like:\n>\n>\n> commit 615ab4864fce6b8042778aa78799ac2656785710 (HEAD -> tl/bitmap-append-trace2-outputs)\n> Author: Teng Long \n> Date: Thu Jul 7 21:48:32 2022 +0800\n>\n> tr2: dump names if config exist in multiple scopes\n>\n> When we specify GIT_TRACE2_CONFIG_PARAMS or trace2.configparams,\n> trace2 will prints \"interesting\" config values to log. Sometimes,\n> when a config set in multiple scope files, the following output\n> looks like (the irrelevant fields are omitted here as \"...\"):\n>\n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n> ...| def_param | ... | core.multipackindex:false\n>\n> As the log shows, even each config in different scope is dumped, but\n> we don't know which scope it comes from. Therefore, it's better to\n> add the scope names as well to make them be more recognizable. For\n> example, when execute:\n>\n> $ GIT_TRACE2_PERF=1 \\\n> > GIT_TRACE2_CONFIG_PARAMS=core.multipackIndex \\\n> > git rev-list --test-bitmap HEAD\"\n>\n> The following is the ouput (the irrelevant fields are omitted here\n> as \"...\"):\n>\n> Format normal:\n> ... git.c:461 ... def_param scope:system core.multipackindex=false\n> ... git.c:461 ... def_param scope:global core.multipackindex=false\n> ... git.c:461 ... def_param scope:local core.multipackindex=false\n>\n> Format perf:\n>\n> ... | def_param | ... | scope:system | core.multipackindex:false\n> ... | def_param | ... | scope:global | core.multipackindex:false\n> ... | def_param | ... | scope:local | core.multipackindex:false\n>\n> Format event:\n>\n> {\"event\":\"def_param\", ... ,\"scope\":\"system\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"global\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n> {\"event\":\"def_param\", ... ,\"scope\":\"local\",\"param\":\"core.multipackindex\",\"value\":\"false\"}\n>\n> Signed-off-by: Teng Long \n>\n> diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt\n> index 4e411f3306..50f1e0c259 100644\n> --- a/Documentation/technical/api-trace2.txt\n> +++ b/Documentation/technical/api-trace2.txt\n> @@ -717,6 +717,7 @@ The \"exec_id\" field is a command-unique id and is only useful if the\n> {\n> \"event\":\"def_param\",\n> ...\n> + \"scope\":...\n> \"param\":\"core.abbrev\",\n> \"value\":\"7\"\n> }\n> @@ -1240,9 +1241,9 @@ d0 | main | version | | | |\n> d0 | main | start | | 0.000260 | | | /opt/git/master/bin/git version\n> d0 | main | cmd_ancestry | | | | | ancestry:[bash sshd sshd sshd systemd]\n> d0 | main | cmd_name | | | | | version (version)\n> -d0 | main | def_param | | | | | color.ui:never\n> -d0 | main | def_param | | | | | color.ui:always\n> -d0 | main | def_param | | | | | color.ui:auto\n> +d0 | main | def_param | | | | scope:system | color.ui:never\n> +d0 | main | def_param | | | | scope:global | color.ui:always\n> +d0 | main | def_param | | | | scope:local | color.ui:auto\n> d0 | main | exit | | 0.000470 | | | code:0\n> d0 | main | atexit | | 0.000477 | | | code:0\n> ----------------\n> diff --git a/trace2/tr2_tgt_event.c b/trace2/tr2_tgt_event.c\n> index c5c8cfbbaa..37a3163be1 100644\n> --- a/trace2/tr2_tgt_event.c\n> +++ b/trace2/tr2_tgt_event.c\n> @@ -479,9 +479,12 @@ static void fn_param_fl(const char *file, int line, const char *param,\n> {\n> const char *event_name = \"def_param\";\n> struct json_writer jw = JSON_WRITER_INIT;\n> + enum config_scope scope = current_config_scope();\n> + const char *scope_name = config_scope_name(scope);\n>\n> jw_object_begin(&jw, 0);\n> event_fmt_prepare(event_name, file, line, NULL, &jw);\n> + jw_object_string(&jw, \"scope\", scope_name);\n> jw_object_string(&jw, \"param\", param);\n> jw_object_string(&jw, \"value\", value);\n> jw_end(&jw);\n> diff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c\n> index c42fbade7f..69f8033077 100644\n> --- a/trace2/tr2_tgt_normal.c\n> +++ b/trace2/tr2_tgt_normal.c\n> @@ -298,8 +298,11 @@ static void fn_param_fl(const char *file, int line, const char *param,\n> const char *value)\n> {\n> struct strbuf buf_payload = STRBUF_INIT;\n> + enum config_scope scope = current_config_scope();\n> + const char *scope_name = config_scope_name(scope);\n>\n> - strbuf_addf(&buf_payload, \"def_param %s=%s\", param, value);\n> + strbuf_addf(&buf_payload, \"def_param scope:%s %s=%s\", scope_name, param,\n> + value);\n> normal_io_write_fl(file, line, &buf_payload);\n> strbuf_release(&buf_payload);\n> }\n> diff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c\n> index a1eff8bea3..8cb792488c 100644\n> --- a/trace2/tr2_tgt_perf.c\n> +++ b/trace2/tr2_tgt_perf.c\n> @@ -441,12 +441,17 @@ static void fn_param_fl(const char *file, int line, const char *param,\n> {\n> const char *event_name = \"def_param\";\n> struct strbuf buf_payload = STRBUF_INIT;\n> + struct strbuf scope_payload = STRBUF_INIT;\n> + enum config_scope scope = current_config_scope();\n> + const char *scope_name = config_scope_name(scope);\n>\n> strbuf_addf(&buf_payload, \"%s:%s\", param, value);\n> + strbuf_addf(&scope_payload, \"%s:%s\", \"scope\", scope_name);\n>\n> - perf_io_write_fl(file, line, event_name, NULL, NULL, NULL, NULL,\n> - &buf_payload);\n> + perf_io_write_fl(file, line, event_name, NULL, NULL, NULL,\n> + scope_payload.buf, &buf_payload);\n> strbuf_release(&buf_payload);\n> + strbuf_release(&scope_payload);\n> }\n>\n> static void fn_repo_fl(const char *file, int line,\n>\n> Am I understand accurately?\n> Thanks.\n\nYes, exactly! That makes it much clearer what the functional change was\nabout, i.e. we can see what parts of the trace are now different (the\nscope is added to the trace).\n\n"},{"id":"459356","messageId":"20220719123416.51367-1-tenglong.tl@tenglongtldeMacBook-Pro.local","threadId":"57611","inReplyTo":"220719.86tu7duxl3.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v7 0/7] trace2: dump scope when print \"interesting\" config","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-19T12:34:16Z","receivedAt":"2022-07-19T13:18:11Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> We generally submit patches on a single \"topic\", what a \"topic\" is is\n> often fuzzy, and sometimes a topic that's mostly trying to do X will fix\n> or change some unrelated Y \"while at it\".\n>\n> But patch or patches at the end of a series don't depend on anything\n> that comes before them, and could be \"cherry-pick\"'d directly on top of\n> \"master\" that's generally a sign that you should be submitting two sets\n> of patches, not one.\n>\n> Per\n> https://lore.kernel.org/git/2016ef2e342c2ec6517afa8ec3e57035021fb965.1650547400.git.dyroneteng@gmail.com/\n> the \"let's log config\" is just something you happened to run into on\n> this topic, but it might have just as well been some other command.\n>\n> So I think it's better to split it up into its own topic.\n\nMake sense.\n\n> Yes, that sound good, although I'd make that \"scope\" line be:\n>\n> \t\"scope\": \"global\" | \"worktree\" | \n>\n> Or just say:\n>\n> \t\"scope\": ,\n>\n> Which covers all the possibilities, without hardcoding them there.\n\nI think I'd like to prefer the second way, thanks for the input.\n\n> I mean that part of what you're adding is about this new \"scope\"\n> feature, but another part just seems to be explaining how the\n> trace2.configParams works in general.\n>\n> For the \"works in general\" let's either link to git-config(1), or if\n> that explanation is lacking improve it & link to it.\n\nYes, I think the current explanation is OK for me in git-config[1], so\nI add the link as in previous reply (Search: 2db47572d4462e3788a92fd355b97df13b9bcc39) :\n\n+`trace2.configparams` can be used to output config values which you care\n+about(see linkgit:git-config[1).\n\n> Yes, something like that, although it's a bit odd to discuss \"scope\"\n> here and not have the trace show it yet, but that's fixed below.:\n\nYes, because I want the result to be more obvious, if a config only in single\nscope maybe it's a litter harder to remember \"Wo, what looks like if config is\nin multiple scopes?(although it's intentional)\"\n\n> Yes, exactly! That makes it much clearer what the functional change was\n> about, i.e. we can see what parts of the trace are now different (the\n> scope is added to the trace).\n\nYes.\n\nThanks.\n"},{"id":"459460","messageId":"xmqq4jzcak8g.fsf@gitster.g","threadId":"57611","inReplyTo":"a01ae8478d3a8545241c5b064b6d369a330ee59f.1658159746.git.dyroneteng@gmail.com","subject":"Re: [PATCH v7 7/7] tr2: dump names if config exist in multiple scopes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T21:03:11Z","receivedAt":"2022-07-19T21:03:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teng Long writes:\n\n> Subject: Re: [PATCH v7 7/7] tr2: dump names if config exist in multiple scopes\n\nThis make it sound as if we do not show the scope if a configuration\nvariable appears only in one scope, but I do not see how the updated\ncode achieves that. Instead, it seems that it shows the scope\nunconditionally in addition to the key-value pair.\n\nIf that is the case, a retitle is in order. Also this should\nprobably be a separate topic, unrelated from the other 6 patches.\n\n"},{"id":"459501","messageId":"20220720124847.65619-1-tenglong.tl@tenglongtldeMacBook-Pro.local","threadId":"57611","inReplyTo":"xmqq4jzcak8g.fsf@gitster.g","subject":"Re: [PATCH v7 7/7] tr2: dump names if config exist in multiple scopes","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-20T12:48:47Z","receivedAt":"2022-07-20T12:48:58Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"Junio C Hamano writes:\n\n> This make it sound as if we do not show the scope if a configuration\n> variable appears only in one scope, but I do not see how the updated\n> code achieves that. Instead, it seems that it shows the scope\n> unconditionally in addition to the key-value pair.\n\nYes. There is a problem with this statement. I will fix it, maybe like:\n\n tr2: print corresponding scope with config key-value\n\n> If that is the case, a retitle is in order. Also this should\n> probably be a separate topic, unrelated from the other 6 patches.\n\nThat's the case. I will send the [0-6] as the v8, and separate [7/7]\nas a new patch series.\n\nThanks.\n"},{"id":"459595","messageId":"cover.1658393855.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658159745.git.dyroneteng@gmail.com","subject":"[PATCH v8 0/6] pack-bitmap.c: optimize error messages","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:42Z","receivedAt":"2022-07-21T09:05:59Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nChanges since v7:\n\nRemove [7/7] and will move to another separate topic.\n\nThanks.\n\nTeng Long (6):\n pack-bitmap.c: fix formatting of error messages\n pack-bitmap.c: mark more strings for translations\n pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n pack-bitmap.c: do not ignore error when opening a bitmap file\n pack-bitmap.c: using error() instead of silently returning -1\n pack-bitmap.c: continue looping when first MIDX bitmap is found\n\n pack-bitmap.c | 103 ++++++++++++++++++++++++++++----------------------\n 1 file changed, 58 insertions(+), 45 deletions(-)\n\nRange-diff against v7:\n1: a01ae8478d < -: ---------- tr2: dump names if config exist in multiple scopes\n-: ---------- > 1: 7426b277ba pack-bitmap.c: fix formatting of error messages\n-: ---------- > 2: 4e493426a0 pack-bitmap.c: mark more strings for translations\n-: ---------- > 3: 263f45ba96 pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"\n-: ---------- > 4: d11ea092d5 pack-bitmap.c: do not ignore error when opening a bitmap file\n-: ---------- > 5: f60efe78d6 pack-bitmap.c: using error() instead of silently returning -1\n-: ---------- > 6: 83090308ad pack-bitmap.c: continue looping when first MIDX bitmap is found\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459596","messageId":"7426b277baed2df132fcb6bc75f0650769a5c7a1.1658393856.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"[PATCH v8 1/6] pack-bitmap.c: fix formatting of error messages","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:43Z","receivedAt":"2022-07-21T09:06:04Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nThere are some text output issues in 'pack-bitmap.c', they exist in\ndie(), error() etc. This includes issues with capitalization the\nfirst letter, newlines, error() instead of BUG(), and substitution\nthat don't have quotes around them.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 47 ++++++++++++++++++++++++-----------------------\n 1 file changed, 24 insertions(+), 23 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9c666cdb8b..0fdba5e8c5 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"strbuf.h\"\n #include \"tag.h\"\n #include \"diff.h\"\n #include \"revision.h\"\n@@ -138,7 +139,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"Failed to load bitmap index (corrupted?)\");\n+\t\terror(\"failed to load bitmap index (corrupted?)\");\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -160,14 +161,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"Corrupted bitmap index (too small)\");\n+\t\treturn error(\"corrupted bitmap index (too small)\");\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"Corrupted bitmap index file (wrong header)\");\n+\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"Unsupported version for bitmap index file (%d)\", index->version);\n+\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -176,7 +177,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \t\tunsigned char *index_end = index->map + index->map_size - the_hash_algo->rawsz;\n \n \t\tif ((flags & BITMAP_OPT_FULL_DAG) == 0)\n-\t\t\treturn error(\"Unsupported options for bitmap index file \"\n+\t\t\tBUG(\"unsupported options for bitmap index file \"\n \t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n@@ -215,7 +216,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"Duplicate entry in bitmap index: %s\", oid_to_hex(oid));\n+\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -274,13 +275,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"Corrupted bitmap pack index\");\n+\t\t\treturn error(\"corrupted bitmap pack index\");\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"Invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -330,7 +331,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", buf.buf);\n+\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -387,7 +388,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: %s\", packfile->pack_name);\n+\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -1628,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object %s not found in type bitmaps\",\n+\t\tdie(\"object '%s' not found in type bitmaps\",\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object %s does not have a unique type\",\n+\t\tdie(\"object '%s' does not have a unique type\",\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object %s: real type %s, expected: %s\",\n+\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1650,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&object->oid));\n+\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1665,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"Object not in bitmap: %s\\n\", oid_to_hex(&commit->object.oid));\n+\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1687,21 +1688,21 @@ void test_bitmap_walk(struct rev_info *revs)\n \tif (revs->pending.nr != 1)\n \t\tdie(\"you must specify exactly one commit to test\");\n \n-\tfprintf(stderr, \"Bitmap v%d test (%d entries loaded)\\n\",\n+\tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n \n \troot = revs->pending.objects[0].item;\n \tbm = bitmap_for_commit(bitmap_git, (struct commit *)root);\n \n \tif (bm) {\n-\t\tfprintf(stderr, \"Found bitmap for %s. %d bits / %08x checksum\\n\",\n+\t\tfprintf_ln(stderr, \"Found bitmap for '%s'. %d bits / %08x checksum\",\n \t\t\toid_to_hex(&root->oid), (int)bm->bit_size, ewah_checksum(bm));\n \n \t\tresult = ewah_to_bitmap(bm);\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"Commit %s doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1726,7 +1727,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstop_progress(&tdata.prg);\n \n \tif (bitmap_equals(result, tdata.base))\n-\t\tfprintf(stderr, \"OK!\\n\");\n+\t\tfprintf_ln(stderr, \"OK!\");\n \telse\n \t\tdie(\"mismatch in bitmap results\");\n \n@@ -1749,7 +1750,7 @@ int test_bitmap_commits(struct repository *r)\n \t\tdie(\"failed to load bitmap indexes\");\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n-\t\tprintf(\"%s\\n\", oid_to_hex(&oid));\n+\t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n \t});\n \n \tfree_bitmap_index(bitmap_git);\n@@ -1774,7 +1775,7 @@ int test_bitmap_hashes(struct repository *r)\n \n \t\tnth_bitmap_object_oid(bitmap_git, &oid, index_pos);\n \n-\t\tprintf(\"%s %\"PRIu32\"\\n\",\n+\t\tprintf_ln(\"%s %\"PRIu32\"\",\n \t\t oid_to_hex(&oid), get_be32(bitmap_git->hashes + index_pos));\n \t}\n \n@@ -1936,7 +1937,7 @@ static off_t get_disk_usage_for_type(struct bitmap_index *bitmap_git,\n \t\t\t\t\tstruct object_id oid;\n \t\t\t\t\tnth_midxed_object_oid(&oid, bitmap_git->midx, midx_pos);\n \n-\t\t\t\t\tdie(_(\"could not find %s in pack %s at offset %\"PRIuMAX),\n+\t\t\t\t\tdie(_(\"could not find '%s' in pack '%s' at offset %\"PRIuMAX),\n \t\t\t\t\t oid_to_hex(&oid),\n \t\t\t\t\t pack->pack_name,\n \t\t\t\t\t (uintmax_t)offset);\n@@ -1972,7 +1973,7 @@ static off_t get_disk_usage_for_extended(struct bitmap_index *bitmap_git)\n \t\t\tcontinue;\n \n \t\tif (oid_object_info_extended(the_repository, &obj->oid, &oi, 0) < 0)\n-\t\t\tdie(_(\"unable to get disk usage of %s\"),\n+\t\t\tdie(_(\"unable to get disk usage of '%s'\"),\n \t\t\t oid_to_hex(&obj->oid));\n \n \t\ttotal += object_size;\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459597","messageId":"4e493426a0049854c7eec64029928e10ad2b414c.1658393856.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"[PATCH v8 2/6] pack-bitmap.c: mark more strings for translations","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:44Z","receivedAt":"2022-07-21T09:06:07Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nIn pack-bitmap.c, some printed texts are translated, some are not.\nLet's support the translations of the bitmap related output.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 48 ++++++++++++++++++++++++------------------------\n 1 file changed, 24 insertions(+), 24 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 0fdba5e8c5..c970ab46dd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -139,7 +139,7 @@ static struct ewah_bitmap *read_bitmap_1(struct bitmap_index *index)\n \t\tindex->map_size - index->map_pos);\n \n \tif (bitmap_size < 0) {\n-\t\terror(\"failed to load bitmap index (corrupted?)\");\n+\t\terror(_(\"failed to load bitmap index (corrupted?)\"));\n \t\tewah_pool_free(b);\n \t\treturn NULL;\n \t}\n@@ -161,14 +161,14 @@ static int load_bitmap_header(struct bitmap_index *index)\n \tsize_t header_size = sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \n \tif (index->map_size < header_size + the_hash_algo->rawsz)\n-\t\treturn error(\"corrupted bitmap index (too small)\");\n+\t\treturn error(_(\"corrupted bitmap index (too small)\"));\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n-\t\treturn error(\"corrupted bitmap index file (wrong header)\");\n+\t\treturn error(_(\"corrupted bitmap index file (wrong header)\"));\n \n \tindex->version = ntohs(header->version);\n \tif (index->version != 1)\n-\t\treturn error(\"unsupported version '%d' for bitmap index file\", index->version);\n+\t\treturn error(_(\"unsupported version '%d' for bitmap index file\"), index->version);\n \n \t/* Parse known bitmap format options */\n \t{\n@@ -182,7 +182,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n \t\t\tif (cache_size > index_end - index->map - header_size)\n-\t\t\t\treturn error(\"corrupted bitmap index file (too short to fit hash cache)\");\n+\t\t\t\treturn error(_(\"corrupted bitmap index file (too short to fit hash cache)\"));\n \t\t\tindex->hashes = (void *)(index_end - cache_size);\n \t\t\tindex_end -= cache_size;\n \t\t}\n@@ -216,7 +216,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"duplicate entry in bitmap index: '%s'\", oid_to_hex(oid));\n+\t\terror(_(\"duplicate entry in bitmap index: '%s'\"), oid_to_hex(oid));\n \t\treturn NULL;\n \t}\n \n@@ -260,14 +260,14 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\tstruct object_id oid;\n \n \t\tif (index->map_size - index->map_pos < 6)\n-\t\t\treturn error(\"corrupt ewah bitmap: truncated header for entry %d\", i);\n+\t\t\treturn error(_(\"corrupt ewah bitmap: truncated header for entry %d\"), i);\n \n \t\tcommit_idx_pos = read_be32(index->map, &index->map_pos);\n \t\txor_offset = read_u8(index->map, &index->map_pos);\n \t\tflags = read_u8(index->map, &index->map_pos);\n \n \t\tif (nth_bitmap_object_oid(index, &oid, commit_idx_pos) < 0)\n-\t\t\treturn error(\"corrupt ewah bitmap: commit index %u out of range\",\n+\t\t\treturn error(_(\"corrupt ewah bitmap: commit index %u out of range\"),\n \t\t\t\t (unsigned)commit_idx_pos);\n \n \t\tbitmap = read_bitmap_1(index);\n@@ -275,13 +275,13 @@ static int load_bitmap_entries_v1(struct bitmap_index *index)\n \t\t\treturn -1;\n \n \t\tif (xor_offset > MAX_XOR_OFFSET || xor_offset > i)\n-\t\t\treturn error(\"corrupted bitmap pack index\");\n+\t\t\treturn error(_(\"corrupted bitmap pack index\"));\n \n \t\tif (xor_offset > 0) {\n \t\t\txor_bitmap = recent_bitmaps[(i - xor_offset) % MAX_XOR_OFFSET];\n \n \t\t\tif (xor_bitmap == NULL)\n-\t\t\t\treturn error(\"invalid XOR offset in bitmap pack index\");\n+\t\t\t\treturn error(_(\"invalid XOR offset in bitmap pack index\"));\n \t\t}\n \n \t\trecent_bitmaps[i % MAX_XOR_OFFSET] = store_bitmap(\n@@ -331,7 +331,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tget_midx_filename(&buf, midx->object_dir);\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: '%s'\", buf.buf);\n+\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), buf.buf);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t\treturn -1;\n@@ -388,7 +388,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\t/* ignore extra bitmap file; we can only handle one */\n-\t\twarning(\"ignoring extra bitmap file: '%s'\", packfile->pack_name);\n+\t\twarning(_(\"ignoring extra bitmap file: '%s'\"), packfile->pack_name);\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -819,7 +819,7 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,\n \t\trevs->include_check_data = &incdata;\n \n \t\tif (prepare_revision_walk(revs))\n-\t\t\tdie(\"revision walk setup failed\");\n+\t\t\tdie(_(\"revision walk setup failed\"));\n \n \t\tshow_data.bitmap_git = bitmap_git;\n \t\tshow_data.base = base;\n@@ -1629,15 +1629,15 @@ static void test_bitmap_type(struct bitmap_test_data *tdata,\n \t}\n \n \tif (bitmap_type == OBJ_NONE)\n-\t\tdie(\"object '%s' not found in type bitmaps\",\n+\t\tdie(_(\"object '%s' not found in type bitmaps\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmaps_nr > 1)\n-\t\tdie(\"object '%s' does not have a unique type\",\n+\t\tdie(_(\"object '%s' does not have a unique type\"),\n \t\t oid_to_hex(&obj->oid));\n \n \tif (bitmap_type != obj->type)\n-\t\tdie(\"object '%s': real type '%s', expected: '%s'\",\n+\t\tdie(_(\"object '%s': real type '%s', expected: '%s'\"),\n \t\t oid_to_hex(&obj->oid),\n \t\t type_name(obj->type),\n \t\t type_name(bitmap_type));\n@@ -1651,7 +1651,7 @@ static void test_show_object(struct object *object, const char *name,\n \n \tbitmap_pos = bitmap_position(tdata->bitmap_git, &object->oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&object->oid));\n+\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&object->oid));\n \ttest_bitmap_type(tdata, object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1666,7 +1666,7 @@ static void test_show_commit(struct commit *commit, void *data)\n \tbitmap_pos = bitmap_position(tdata->bitmap_git,\n \t\t\t\t &commit->object.oid);\n \tif (bitmap_pos < 0)\n-\t\tdie(\"object not in bitmap: '%s'\", oid_to_hex(&commit->object.oid));\n+\t\tdie(_(\"object not in bitmap: '%s'\"), oid_to_hex(&commit->object.oid));\n \ttest_bitmap_type(tdata, &commit->object, bitmap_pos);\n \n \tbitmap_set(tdata->base, bitmap_pos);\n@@ -1683,10 +1683,10 @@ void test_bitmap_walk(struct rev_info *revs)\n \tstruct ewah_bitmap *bm;\n \n \tif (!(bitmap_git = prepare_bitmap_git(revs->repo)))\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tif (revs->pending.nr != 1)\n-\t\tdie(\"you must specify exactly one commit to test\");\n+\t\tdie(_(\"you must specify exactly one commit to test\"));\n \n \tfprintf_ln(stderr, \"Bitmap v%d test (%d entries loaded)\",\n \t\tbitmap_git->version, bitmap_git->entry_count);\n@@ -1702,7 +1702,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \t}\n \n \tif (result == NULL)\n-\t\tdie(\"commit '%s' doesn't have an indexed bitmap\", oid_to_hex(&root->oid));\n+\t\tdie(_(\"commit '%s' doesn't have an indexed bitmap\"), oid_to_hex(&root->oid));\n \n \trevs->tag_objects = 1;\n \trevs->tree_objects = 1;\n@@ -1711,7 +1711,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tresult_popcnt = bitmap_popcount(result);\n \n \tif (prepare_revision_walk(revs))\n-\t\tdie(\"revision walk setup failed\");\n+\t\tdie(_(\"revision walk setup failed\"));\n \n \ttdata.bitmap_git = bitmap_git;\n \ttdata.base = bitmap_new();\n@@ -1729,7 +1729,7 @@ void test_bitmap_walk(struct rev_info *revs)\n \tif (bitmap_equals(result, tdata.base))\n \t\tfprintf_ln(stderr, \"OK!\");\n \telse\n-\t\tdie(\"mismatch in bitmap results\");\n+\t\tdie(_(\"mismatch in bitmap results\"));\n \n \tbitmap_free(result);\n \tbitmap_free(tdata.base);\n@@ -1747,7 +1747,7 @@ int test_bitmap_commits(struct repository *r)\n \tMAYBE_UNUSED void *value;\n \n \tif (!bitmap_git)\n-\t\tdie(\"failed to load bitmap indexes\");\n+\t\tdie(_(\"failed to load bitmap indexes\"));\n \n \tkh_foreach(bitmap_git->bitmaps, oid, value, {\n \t\tprintf_ln(\"%s\", oid_to_hex(&oid));\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459598","messageId":"263f45ba96c322665f102e22e1899c93541d87c4.1658393856.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"[PATCH v8 3/6] pack-bitmap.c: rename \"idx_name\" to \"bitmap_name\"","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:45Z","receivedAt":"2022-07-21T09:06:12Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nIn \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\" we use\na var named \"idx_name\" to represent the bitmap filename which\nis computed by \"midx_bitmap_filename()\" or \"pack_bitmap_filename()\"\nbefore we open it.\n\nThere may bring some confusion in this \"idx_name\" naming, which\nmight lead us to think of \".idx \"or\" multi-pack-index\" files,\nalthough bitmap is essentially can be understood as a kind of index,\nlet's define this name a little more accurate here.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex c970ab46dd..7d8cc063fc 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -314,10 +314,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t\t\t struct multi_pack_index *midx)\n {\n \tstruct stat st;\n-\tchar *idx_name = midx_bitmap_filename(midx);\n-\tint fd = git_open(idx_name);\n+\tchar *bitmap_name = midx_bitmap_filename(midx);\n+\tint fd = git_open(bitmap_name);\n \n-\tfree(idx_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n@@ -369,14 +369,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n {\n \tint fd;\n \tstruct stat st;\n-\tchar *idx_name;\n+\tchar *bitmap_name;\n \n \tif (open_pack_index(packfile))\n \t\treturn -1;\n \n-\tidx_name = pack_bitmap_filename(packfile);\n-\tfd = git_open(idx_name);\n-\tfree(idx_name);\n+\tbitmap_name = pack_bitmap_filename(packfile);\n+\tfd = git_open(bitmap_name);\n+\tfree(bitmap_name);\n \n \tif (fd < 0)\n \t\treturn -1;\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459599","messageId":"d11ea092d511f3be4601f65ec54d492474fe821c.1658393856.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"[PATCH v8 4/6] pack-bitmap.c: do not ignore error when opening a bitmap file","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:46Z","receivedAt":"2022-07-21T09:06:15Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nCalls to git_open() to open the pack bitmap file and\nmulti-pack bitmap file do not report any error when they\nfail. These files are optional and it is not an error if\nopen failed due to ENOENT, but we shouldn't be ignoring\nother kinds of errors.\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 7d8cc063fc..f8f9937c9e 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -317,10 +317,13 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n \tint fd = git_open(bitmap_name);\n \n-\tfree(bitmap_name);\n-\n-\tif (fd < 0)\n+\tif (fd < 0) {\n+\t\tif (errno != ENOENT)\n+\t\t\twarning_errno(\"cannot open '%s'\", bitmap_name);\n+\t\tfree(bitmap_name);\n \t\treturn -1;\n+\t}\n+\tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n@@ -376,10 +379,14 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \n \tbitmap_name = pack_bitmap_filename(packfile);\n \tfd = git_open(bitmap_name);\n-\tfree(bitmap_name);\n \n-\tif (fd < 0)\n+\tif (fd < 0) {\n+\t\tif (errno != ENOENT)\n+\t\t\twarning_errno(\"cannot open '%s'\", bitmap_name);\n+\t\tfree(bitmap_name);\n \t\treturn -1;\n+\t}\n+\tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n \t\tclose(fd);\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459600","messageId":"f60efe78d6a426f4a834680adf62a55937089aa9.1658393856.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"[PATCH v8 5/6] pack-bitmap.c: using error() instead of silently returning -1","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:47Z","receivedAt":"2022-07-21T09:06:21Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nIn \"open_pack_bitmap_1()\" and \"open_midx_bitmap_1()\", it's better to\nreturn error() instead of \"-1\" when some unexpected error occurs like\n\"stat bitmap file failed\", \"bitmap header is invalid\" or \"checksum\nmismatch\", etc.\n\nThere are places where we do not replace, such as when the bitmap\ndoes not exist (no bitmap in repository is allowed) or when another\nbitmap has already been opened (in which case it should be a warning\nrather than an error).\n\nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex f8f9937c9e..318edd8e0d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -326,6 +326,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n+\t\terror_errno(_(\"cannot fstat bitmap file\"));\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n@@ -350,8 +351,10 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \tif (load_bitmap_header(bitmap_git) < 0)\n \t\tgoto cleanup;\n \n-\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum))\n+\tif (!hasheq(get_midx_checksum(bitmap_git->midx), bitmap_git->checksum)) {\n+\t\terror(_(\"checksum doesn't match in MIDX and bitmap\"));\n \t\tgoto cleanup;\n+\t}\n \n \tif (load_midx_revindex(bitmap_git->midx) < 0) {\n \t\twarning(_(\"multi-pack bitmap is missing required reverse index\"));\n@@ -389,6 +392,7 @@ static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git\n \tfree(bitmap_name);\n \n \tif (fstat(fd, &st)) {\n+\t\terror_errno(_(\"cannot fstat bitmap file\"));\n \t\tclose(fd);\n \t\treturn -1;\n \t}\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459601","messageId":"83090308ad63543c9becfb4cec9c1cedcffb1fdc.1658393856.git.dyroneteng@gmail.com","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"[PATCH v8 6/6] pack-bitmap.c: continue looping when first MIDX bitmap is found","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-21T09:05:48Z","receivedAt":"2022-07-21T09:06:24Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"From: Teng Long \n\nIn \"open_midx_bitmap()\", we do a loop with the MIDX(es) in repo, when\nthe first one has been found, then will break out by a \"return\"\ndirectly.\n\nBut actually, it's better to continue the loop until we have visited\nboth the MIDX in our repository, as well as any alternates (along with\n_their_ alternates, recursively).\n\nThe reason for this is, there may exist more than one MIDX file in\na repo. The \"multi_pack_index\" struct is actually designed as a singly\nlinked list, and if a MIDX file has been already opened successfully,\nthen the other MIDX files will be skipped and left with a warning\n\"ignoring extra bitmap file.\" to the output.\n\nThe discussion link of community:\n\n https://public-inbox.org/git/YjzCTLLDCby+kJrZ@nand.local/\n\nHelped-by: Taylor Blau \nSigned-off-by: Teng Long \n---\n pack-bitmap.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 318edd8e0d..d013ea0277 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -506,15 +506,16 @@ static int open_pack_bitmap(struct repository *r,\n static int open_midx_bitmap(struct repository *r,\n \t\t\t struct bitmap_index *bitmap_git)\n {\n+\tint ret = -1;\n \tstruct multi_pack_index *midx;\n \n \tassert(!bitmap_git->map);\n \n \tfor (midx = get_multi_pack_index(r); midx; midx = midx->next) {\n \t\tif (!open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn -1;\n+\treturn ret;\n }\n \n static int open_bitmap(struct repository *r,\n-- \n2.35.0.rc0.679.gc613175da2\n\n"},{"id":"459740","messageId":"xmqq1quedq9p.fsf@gitster.g","threadId":"57611","inReplyTo":"cover.1658393855.git.dyroneteng@gmail.com","subject":"Re: [PATCH v8 0/6] pack-bitmap.c: optimize error messages","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T23:01:22Z","receivedAt":"2022-07-21T23:01:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"tenglong.tl\" writes:\n\n> From: Teng Long \n>\n> Changes since v7:\n>\n> Remove [7/7] and will move to another separate topic.\n\nIf that is the only change, then I am relieved, as I can safely\nignore this round ;-)\n\nThe previous round (without the last step) has been in 'next' since\nTuesday and cannot be replaced.\n\nIt seems that both of the topics are in good shape. Let's plan to\nmerge the other one about showing the scope of the configuration\nvariable in trace2 output down to 'next' soonish.\n\nThanks.\n\n"},{"id":"459755","messageId":"20220722061721.9803-1-tenglong.tl@alibaba-inc.com","threadId":"57611","inReplyTo":"xmqq1quedq9p.fsf@gitster.g","subject":"Re: [PATCH v8 0/6] pack-bitmap.c: optimize error messages","fromName":"tenglong.tl","fromEmail":"dyroneteng@gmail.com","sentAt":"2022-07-22T06:17:21Z","receivedAt":"2022-07-22T06:17:30Z","isPatch":true,"sender":{"key":"dyroneteng@gmail.com","avatar":"https://avatars.githubusercontent.com/u/7803958?v=4"},"body":"\nJunio C Hamano writes:\n\n> If that is the only change, then I am relieved, as I can safely\n> ignore this round ;-)\n\nYes, only removed 7/7, the others do not change.\n\n\n> The previous round (without the last step) has been in 'next' since\n> Tuesday and cannot be replaced.\n>\n> It seems that both of the topics are in good shape. Let's plan to\n> merge the other one about showing the scope of the configuration\n> variable in trace2 output down to 'next' soonish.\n\nOh, I see.\n\nThanks.\n"}]}