{"thread":{"id":"60420","subject":"ls-remote bug","startedAt":"2023-10-24T10:55:28Z","lastAt":"2023-10-31T10:00:54Z","messageCount":6,"participants":["Lior Zeltzer","Bagas Sanjaya","René Scharfe"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"483763","messageId":"BL0PR18MB2130A3CA5DEF0DD7199F2979BADFA@BL0PR18MB2130.namprd18.prod.outlook.com","threadId":"60420","inReplyTo":"BL0PR18MB2130C672D503E49F955E04E8BADFA@BL0PR18MB2130.namprd18.prod.outlook.com","subject":"ls-remote bug","fromName":"Lior Zeltzer","fromEmail":"liorz@marvell.com","sentAt":"2023-10-24T10:55:24Z","receivedAt":"2023-10-24T10:55:28Z","isPatch":false,"sender":{"key":"liorz@marvell.com","avatar":null},"body":"\n>uname -a\nLinux dc3lp-veld0045 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16 18:28:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux\n\nGerrit version :\n3.8.0\n\nBug description : \nWhen running ls-remote : sometime data gets cut in the middle\n\nReproducing :\nYou need a few files with a few repo names (I used 4 files with 10 repos each)\nCall then l1..l4\nAnd the code below just cd into each of them does ls-remote twice and compares the data\nDoing it in parallel on all lists.\nData received in both ls-remotes should be the same , if not, it prints ***\nRepos should contain a lot of tags and refs\n\nNote : \n1.  without stderr redirection (2>&1) all works well\n2. On local repos (not through gerrit) all works well\n\nI compared various git vers and found the bug to be between 2.31.8 and 2.32.0\nComparing ls-remote.c file between those vers gave me :\n\nLines :\nif (transport_disconnect(transport))\n\t\treturn 1;\n\nmoved to end of sub\n\ncopying ls-remote.c from 2.31.8 to 2.32.0 - fixed the bug\n\n\n\nCode reproducing bug :\n\n#!/proj/mislcad/areas/DAtools/tools/perl/5.10.1/bin/perl -w\nuse strict;\nuse Cwd qw(cwd);\n\nmy $count = 4;\nfor my $f (1..$count) {\n  my $child = fork();\n  if (!$child) {\n    my $curr = cwd();\n    \n    my @repos = `cat l$f`;\n    foreach my $repo (@repos) {\n      chomp $repo;\n      print \"$repo\\n\";\n      chdir($repo);\n      my $remote_tags_str = `git ls-remote  2>&1`;\n      my $remote_tags_str2 = `git ls-remote  2>&1 `;\n      chdir($curr);\n      if ( $remote_tags_str ne $remote_tags_str2) {\n         print \"***\\n\";\n      }\n    }\n  \n    exit(0);\n  }\n}\nwhile (wait != -1) {}\n1;\n\n"},{"id":"483948","messageId":"ZTsNfgJV1JN2y-Aw@debian.me","threadId":"60420","inReplyTo":"BL0PR18MB2130A3CA5DEF0DD7199F2979BADFA@BL0PR18MB2130.namprd18.prod.outlook.com","subject":"Re: ls-remote bug","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2023-10-27T01:08:14Z","receivedAt":"2023-10-27T01:08:22Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On Tue, Oct 24, 2023 at 10:55:24AM +0000, Lior Zeltzer wrote:\n> \n> >uname -a\n> Linux dc3lp-veld0045 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16 18:28:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux\n> \n> Gerrit version :\n> 3.8.0\n> \n> Bug description : \n> When running ls-remote : sometime data gets cut in the middle\n> \n> Reproducing :\n> You need a few files with a few repo names (I used 4 files with 10 repos each)\n> Call then l1..l4\n> And the code below just cd into each of them does ls-remote twice and compares the data\n> Doing it in parallel on all lists.\n> Data received in both ls-remotes should be the same , if not, it prints ***\n> Repos should contain a lot of tags and refs\n\nWhat repo did you find this regression? Did you mean linux.git (Linux kernel)?\n\n> \n> Note : \n> 1.  without stderr redirection (2>&1) all works well\n> 2. On local repos (not through gerrit) all works well\n> \n> I compared various git vers and found the bug to be between 2.31.8 and 2.32.0\n> Comparing ls-remote.c file between those vers gave me :\n> \n> Lines :\n> if (transport_disconnect(transport))\n> \t\treturn 1;\n> \n> moved to end of sub\n> \n> copying ls-remote.c from 2.31.8 to 2.32.0 - fixed the bug\n> \n> \n> \n> Code reproducing bug :\n> \n> #!/proj/mislcad/areas/DAtools/tools/perl/5.10.1/bin/perl -w\n> use strict;\n> use Cwd qw(cwd);\n> \n> my $count = 4;\n> for my $f (1..$count) {\n>   my $child = fork();\n>   if (!$child) {\n>     my $curr = cwd();\n>     \n>     my @repos = `cat l$f`;\n>     foreach my $repo (@repos) {\n>       chomp $repo;\n>       print \"$repo\\n\";\n>       chdir($repo);\n>       my $remote_tags_str = `git ls-remote  2>&1`;\n>       my $remote_tags_str2 = `git ls-remote  2>&1 `;\n>       chdir($curr);\n>       if ( $remote_tags_str ne $remote_tags_str2) {\n>          print \"***\\n\";\n>       }\n>     }\n>   \n>     exit(0);\n>   }\n> }\n> while (wait != -1) {}\n> 1;\n> \n\nI tried reproducing this regression by:\n\n```\n$ cd /path/to/git.git\n$ git ls-remote 2>&1 > /tmp/root.list\n$ cd builtin/\n$ git ls-remote 2>&1 > /tmp/builtin.list\n$ cd ../\n$ git diff --no-index /tmp/root.list /tmp/builtin.list\n```\n\nAnd indeed, the diff was empty (which meant that both listings are same).\n\nConfused...\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"483971","messageId":"BL0PR18MB213007901229C84D348D4CB8BADCA@BL0PR18MB2130.namprd18.prod.outlook.com","threadId":"60420","inReplyTo":"ZTsNfgJV1JN2y-Aw@debian.me","subject":"RE: [EXT] Re: ls-remote bug","fromName":"Lior Zeltzer","fromEmail":"liorz@marvell.com","sentAt":"2023-10-27T11:16:14Z","receivedAt":"2023-10-27T11:16:58Z","isPatch":false,"sender":{"key":"liorz@marvell.com","avatar":null},"body":"The reproduction ,as I wrote in the code, should be done with few threads in parallel \nEach working on a list of ~10 repos with each repo containing a lot of refs/tags (~1000)\nAll this should be against gerrit (my gerrit is 3.8.0)\n\nAlso read the notes below regarding the code that was moved between 2.31.8 and 2.32.0 in ls-remote.c file\nI can elaborate more, in a zoom meeting.\n\n10x\nLior.\n\n\n-----Original Message-----\nFrom: Bagas Sanjaya \nSent: Friday, October 27, 2023 4:08 AM\nTo: Lior Zeltzer ; Git Mailing List \nCc: Andrzej Hunt ; Ævar Arnfjörð Bjarmason ; Junio C Hamano \nSubject: [EXT] Re: ls-remote bug\n\nExternal Email\n\n----------------------------------------------------------------------\nOn Tue, Oct 24, 2023 at 10:55:24AM +0000, Lior Zeltzer wrote:\n> \n> >uname -a\n> Linux dc3lp-veld0045 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16 \n> 18:28:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux\n> \n> Gerrit version :\n> 3.8.0\n> \n> Bug description : \n> When running ls-remote : sometime data gets cut in the middle\n> \n> Reproducing :\n> You need a few files with a few repo names (I used 4 files with 10 \n> repos each) Call then l1..l4 And the code below just cd into each of \n> them does ls-remote twice and compares the data Doing it in parallel \n> on all lists.\n> Data received in both ls-remotes should be the same , if not, it \n> prints *** Repos should contain a lot of tags and refs\n\nWhat repo did you find this regression? Did you mean linux.git (Linux kernel)?\n\n> \n> Note : \n> 1.  without stderr redirection (2>&1) all works well 2. On local repos \n> (not through gerrit) all works well\n> \n> I compared various git vers and found the bug to be between 2.31.8 and \n> 2.32.0 Comparing ls-remote.c file between those vers gave me :\n> \n> Lines :\n> if (transport_disconnect(transport))\n> \t\treturn 1;\n> \n> moved to end of sub\n> \n> copying ls-remote.c from 2.31.8 to 2.32.0 - fixed the bug\n> \n> \n> \n> Code reproducing bug :\n> \n> #!/proj/mislcad/areas/DAtools/tools/perl/5.10.1/bin/perl -w use \n> strict; use Cwd qw(cwd);\n> \n> my $count = 4;\n> for my $f (1..$count) {\n>   my $child = fork();\n>   if (!$child) {\n>     my $curr = cwd();\n>     \n>     my @repos = `cat l$f`;\n>     foreach my $repo (@repos) {\n>       chomp $repo;\n>       print \"$repo\\n\";\n>       chdir($repo);\n>       my $remote_tags_str = `git ls-remote  2>&1`;\n>       my $remote_tags_str2 = `git ls-remote  2>&1 `;\n>       chdir($curr);\n>       if ( $remote_tags_str ne $remote_tags_str2) {\n>          print \"***\\n\";\n>       }\n>     }\n>   \n>     exit(0);\n>   }\n> }\n> while (wait != -1) {}\n> 1;\n> \n\nI tried reproducing this regression by:\n\n```\n$ cd /path/to/git.git\n$ git ls-remote 2>&1 > /tmp/root.list\n$ cd builtin/\n$ git ls-remote 2>&1 > /tmp/builtin.list $ cd ../ $ git diff --no-index /tmp/root.list /tmp/builtin.list ```\n\nAnd indeed, the diff was empty (which meant that both listings are same).\n\nConfused...\n\n--\nAn old man doll... just what I always wanted! - Clara\n"},{"id":"484054","messageId":"cc829a27-2580-4e07-a6ff-2c4992131420@web.de","threadId":"60420","inReplyTo":"BL0PR18MB213007901229C84D348D4CB8BADCA@BL0PR18MB2130.namprd18.prod.outlook.com","subject":"Re: [EXT] Re: ls-remote bug","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2023-10-29T18:56:43Z","receivedAt":"2023-10-29T18:56:56Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 27.10.23 um 13:16 schrieb Lior Zeltzer:\n> The reproduction ,as I wrote in the code, should be done with few threads in parallel\n> Each working on a list of ~10 repos with each repo containing a lot of refs/tags (~1000)\n> All this should be against gerrit (my gerrit is 3.8.0)\n>\n> Also read the notes below regarding the code that was moved between 2.31.8 and 2.32.0 in ls-remote.c file\n> I can elaborate more, in a zoom meeting.\n>\n> 10x\n> Lior.\n>\n>\n> -----Original Message-----\n> From: Bagas Sanjaya \n> Sent: Friday, October 27, 2023 4:08 AM\n> To: Lior Zeltzer ; Git Mailing List \n> Cc: Andrzej Hunt ; Ævar Arnfjörð Bjarmason ; Junio C Hamano \n> Subject: [EXT] Re: ls-remote bug\n>\n> External Email\n>\n> ----------------------------------------------------------------------\n> On Tue, Oct 24, 2023 at 10:55:24AM +0000, Lior Zeltzer wrote:\n>>\n>>> uname -a\n>> Linux dc3lp-veld0045 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16\n>> 18:28:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux\n>>\n>> Gerrit version :\n>> 3.8.0\n>>\n>> Bug description :\n>> When running ls-remote : sometime data gets cut in the middle\n>>\n>> Reproducing :\n>> You need a few files with a few repo names (I used 4 files with 10\n>> repos each) Call then l1..l4 And the code below just cd into each of\n>> them does ls-remote twice and compares the data Doing it in parallel\n>> on all lists.\n>> Data received in both ls-remotes should be the same , if not, it\n>> prints *** Repos should contain a lot of tags and refs\n>\n> What repo did you find this regression? Did you mean linux.git (Linux kernel)?\n>\n>>\n>> Note :\n>> 1.  without stderr redirection (2>&1) all works well 2. On local repos\n>> (not through gerrit) all works well\n>>\n>> I compared various git vers and found the bug to be between 2.31.8 and\n>> 2.32.0 Comparing ls-remote.c file between those vers gave me :\n>>\n>> Lines :\n>> if (transport_disconnect(transport))\n>> \t\treturn 1;\n>>\n>> moved to end of sub\n>>\n>> copying ls-remote.c from 2.31.8 to 2.32.0 - fixed the bug\n\nThis partly undoes 68ffe095a2 (transport: also free remote_refs in\ntransport_disconnect(), 2021-03-21). With that patch connections are\nkept open during ref sorting and printing.\n\nPerhaps the other side gets tired of waiting and aborts? Maybe\nsplitting transport_disconnect() into two functions -- one for\ndisconnecting and and for cleaning up -- would make sense? And\ndisconnecting as soon as possible? Just guessing -- didn't actually\nreproduce the bug. Still, demo patch below.\n\n>>\n>>\n>>\n>> Code reproducing bug :\n>>\n>> #!/proj/mislcad/areas/DAtools/tools/perl/5.10.1/bin/perl -w use\n>> strict; use Cwd qw(cwd);\n>>\n>> my $count = 4;\n>> for my $f (1..$count) {\n>>   my $child = fork();\n>>   if (!$child) {\n>>     my $curr = cwd();\n>>    \n>>     my @repos = `cat l$f`;\n>>     foreach my $repo (@repos) {\n>>       chomp $repo;\n>>       print \"$repo\\n\";\n>>       chdir($repo);\n>>       my $remote_tags_str = `git ls-remote  2>&1`;\n>>       my $remote_tags_str2 = `git ls-remote  2>&1 `;\n>>       chdir($curr);\n>>       if ( $remote_tags_str ne $remote_tags_str2) {\n>>          print \"***\\n\";\n>>       }\n>>     }\n>>  \n>>     exit(0);\n>>   }\n>> }\n>> while (wait != -1) {}\n>> 1;\n>>\n>\n> I tried reproducing this regression by:\n>\n> ```\n> $ cd /path/to/git.git\n> $ git ls-remote 2>&1 > /tmp/root.list\n> $ cd builtin/\n> $ git ls-remote 2>&1 > /tmp/builtin.list $ cd ../ $ git diff --no-index /tmp/root.list /tmp/builtin.list ```\n>\n> And indeed, the diff was empty (which meant that both listings are same).\n>\n> Confused...\n>\n> --\n> An old man doll... just what I always wanted! - Clara\n\n\n---\n builtin/ls-remote.c | 8 +++++---\n builtin/remote.c | 3 ++-\n transport.c | 15 +++++++++++++--\n transport.h | 2 ++\n 4 files changed, 22 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/ls-remote.c b/builtin/ls-remote.c\nindex fc76575430..4c1daa0f92 100644\n--- a/builtin/ls-remote.c\n+++ b/builtin/ls-remote.c\n@@ -128,6 +128,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\tint hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n \t\trepo_set_hash_algo(the_repository, hash_algo);\n \t}\n+\tif (transport_disconnect_raw(transport))\n+\t\tstatus = 1;\n\n \tif (!dest && !quiet)\n \t\tfprintf(stderr, \"From %s\\n\", *remote->url);\n@@ -154,12 +156,12 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\tif (show_symref_target && ref->symref)\n \t\t\tprintf(\"ref: %s\\t%s\\n\", ref->symref, ref->refname);\n \t\tprintf(\"%s\\t%s\\n\", oid_to_hex(&ref->objectname), ref->refname);\n-\t\tstatus = 0; /* we found something */\n+\t\tif (status != 1)\n+\t\t\tstatus = 0; /* we found something */\n \t}\n\n \tref_array_clear(&ref_array);\n-\tif (transport_disconnect(transport))\n-\t\tstatus = 1;\n+\ttransport_clear(transport);\n \ttransport_ls_refs_options_release(&transport_options);\n \treturn status;\n }\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex d91bbe728d..055a221942 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -1000,6 +1000,7 @@ static int get_remote_ref_states(const char *name,\n \t\ttransport = transport_get(states->remote, states->remote->url_nr > 0 ?\n \t\t\tstates->remote->url[0] : NULL);\n \t\tremote_refs = transport_get_remote_refs(transport, NULL);\n+\t\ttransport_disconnect_raw(transport);\n\n \t\tstates->queried = 1;\n \t\tif (query & GET_REF_STATES)\n@@ -1008,7 +1009,7 @@ static int get_remote_ref_states(const char *name,\n \t\t\tget_head_names(remote_refs, states);\n \t\tif (query & GET_PUSH_REF_STATES)\n \t\t\tget_push_ref_states(remote_refs, states);\n-\t\ttransport_disconnect(transport);\n+\t\ttransport_clear(transport);\n \t} else {\n \t\tfor_each_ref(append_ref_to_tracked_list, states);\n \t\tstring_list_sort(&states->tracked);\ndiff --git a/transport.c b/transport.c\nindex 219af8fd50..c71dab75e9 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1584,17 +1584,28 @@ int transport_connect(struct transport *transport, const char *name,\n \t\tdie(_(\"operation not supported by protocol\"));\n }\n\n-int transport_disconnect(struct transport *transport)\n+int transport_disconnect_raw(struct transport *transport)\n {\n \tint ret = 0;\n \tif (transport->vtable->disconnect)\n \t\tret = transport->vtable->disconnect(transport);\n+\treturn ret;\n+}\n+\n+void transport_clear(struct transport *transport)\n+{\n \tif (transport->got_remote_refs)\n \t\tfree_refs((void *)transport->remote_refs);\n \tclear_bundle_list(transport->bundles);\n \tfree(transport->bundles);\n \tfree(transport);\n-\treturn ret;\n+}\n+\n+int transport_disconnect(struct transport *transport)\n+{\n+\tint ret = transport_disconnect_raw(transport);\n+\ttransport_clear(transport);\n+\treturn 0;\n }\n\n /*\ndiff --git a/transport.h b/transport.h\nindex 6393cd9823..fd75905568 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -320,6 +320,8 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs);\n void transport_unlock_pack(struct transport *transport, unsigned int flags);\n\n int transport_disconnect(struct transport *transport);\n+void transport_clear(struct transport *transport);\n+int transport_disconnect_raw(struct transport *transport);\n char *transport_anonymize_url(const char *url);\n void transport_take_over(struct transport *transport,\n \t\t\t struct child_process *child);\n--\n2.42.0\n\n"},{"id":"484079","messageId":"BL0PR18MB2130C503D3608186AFC516D2BAA1A@BL0PR18MB2130.namprd18.prod.outlook.com","threadId":"60420","inReplyTo":"cc829a27-2580-4e07-a6ff-2c4992131420@web.de","subject":"RE: [EXT] Re: ls-remote bug","fromName":"Lior Zeltzer","fromEmail":"liorz@marvell.com","sentAt":"2023-10-30T08:17:38Z","receivedAt":"2023-10-30T08:17:52Z","isPatch":false,"sender":{"key":"liorz@marvell.com","avatar":null},"body":"But why when cancelling stderr redirect to stdout (2>&1) all works well ?\n\n-----Original Message-----\nFrom: René Scharfe \nSent: Sunday, October 29, 2023 8:57 PM\nTo: Lior Zeltzer ; Bagas Sanjaya ; Git Mailing List \nCc: Andrzej Hunt ; Ævar Arnfjörð Bjarmason ; Junio C Hamano \nSubject: Re: [EXT] Re: ls-remote bug\n\nAm 27.10.23 um 13:16 schrieb Lior Zeltzer:\n> The reproduction ,as I wrote in the code, should be done with few \n> threads in parallel Each working on a list of ~10 repos with each repo \n> containing a lot of refs/tags (~1000) All this should be against \n> gerrit (my gerrit is 3.8.0)\n>\n> Also read the notes below regarding the code that was moved between \n> 2.31.8 and 2.32.0 in ls-remote.c file I can elaborate more, in a zoom meeting.\n>\n> 10x\n> Lior.\n>\n>\n> -----Original Message-----\n> From: Bagas Sanjaya \n> Sent: Friday, October 27, 2023 4:08 AM\n> To: Lior Zeltzer ; Git Mailing List \n> \n> Cc: Andrzej Hunt ; Ævar Arnfjörð Bjarmason \n> ; Junio C Hamano \n> Subject: [EXT] Re: ls-remote bug\n>\n> External Email\n>\n> ----------------------------------------------------------------------\n> On Tue, Oct 24, 2023 at 10:55:24AM +0000, Lior Zeltzer wrote:\n>>\n>>> uname -a\n>> Linux dc3lp-veld0045 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16\n>> 18:28:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux\n>>\n>> Gerrit version :\n>> 3.8.0\n>>\n>> Bug description :\n>> When running ls-remote : sometime data gets cut in the middle\n>>\n>> Reproducing :\n>> You need a few files with a few repo names (I used 4 files with 10 \n>> repos each) Call then l1..l4 And the code below just cd into each of \n>> them does ls-remote twice and compares the data Doing it in parallel \n>> on all lists.\n>> Data received in both ls-remotes should be the same , if not, it \n>> prints *** Repos should contain a lot of tags and refs\n>\n> What repo did you find this regression? Did you mean linux.git (Linux kernel)?\n>\n>>\n>> Note :\n>> 1.  without stderr redirection (2>&1) all works well 2. On local \n>> repos (not through gerrit) all works well\n>>\n>> I compared various git vers and found the bug to be between 2.31.8 \n>> and\n>> 2.32.0 Comparing ls-remote.c file between those vers gave me :\n>>\n>> Lines :\n>> if (transport_disconnect(transport))\n>> \t\treturn 1;\n>>\n>> moved to end of sub\n>>\n>> copying ls-remote.c from 2.31.8 to 2.32.0 - fixed the bug\n\nThis partly undoes 68ffe095a2 (transport: also free remote_refs in transport_disconnect(), 2021-03-21). With that patch connections are kept open during ref sorting and printing.\n\nPerhaps the other side gets tired of waiting and aborts? Maybe splitting transport_disconnect() into two functions -- one for disconnecting and and for cleaning up -- would make sense? And disconnecting as soon as possible? Just guessing -- didn't actually reproduce the bug. Still, demo patch below.\n\n>>\n>>\n>>\n>> Code reproducing bug :\n>>\n>> #!/proj/mislcad/areas/DAtools/tools/perl/5.10.1/bin/perl -w use \n>> strict; use Cwd qw(cwd);\n>>\n>> my $count = 4;\n>> for my $f (1..$count) {\n>>   my $child = fork();\n>>   if (!$child) {\n>>     my $curr = cwd();\n>>    \n>>     my @repos = `cat l$f`;\n>>     foreach my $repo (@repos) {\n>>       chomp $repo;\n>>       print \"$repo\\n\";\n>>       chdir($repo);\n>>       my $remote_tags_str = `git ls-remote  2>&1`;\n>>       my $remote_tags_str2 = `git ls-remote  2>&1 `;\n>>       chdir($curr);\n>>       if ( $remote_tags_str ne $remote_tags_str2) {\n>>          print \"***\\n\";\n>>       }\n>>     }\n>>  \n>>     exit(0);\n>>   }\n>> }\n>> while (wait != -1) {}\n>> 1;\n>>\n>\n> I tried reproducing this regression by:\n>\n> ```\n> $ cd /path/to/git.git\n> $ git ls-remote 2>&1 > /tmp/root.list\n> $ cd builtin/\n> $ git ls-remote 2>&1 > /tmp/builtin.list $ cd ../ $ git diff \n> --no-index /tmp/root.list /tmp/builtin.list ```\n>\n> And indeed, the diff was empty (which meant that both listings are same).\n>\n> Confused...\n>\n> --\n> An old man doll... just what I always wanted! - Clara\n\n\n---\n builtin/ls-remote.c | 8 +++++---\n builtin/remote.c | 3 ++-\n transport.c | 15 +++++++++++++--\n transport.h | 2 ++\n 4 files changed, 22 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/ls-remote.c b/builtin/ls-remote.c index fc76575430..4c1daa0f92 100644\n--- a/builtin/ls-remote.c\n+++ b/builtin/ls-remote.c\n@@ -128,6 +128,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\tint hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));\n \t\trepo_set_hash_algo(the_repository, hash_algo);\n \t}\n+\tif (transport_disconnect_raw(transport))\n+\t\tstatus = 1;\n\n \tif (!dest && !quiet)\n \t\tfprintf(stderr, \"From %s\\n\", *remote->url); @@ -154,12 +156,12 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\tif (show_symref_target && ref->symref)\n \t\t\tprintf(\"ref: %s\\t%s\\n\", ref->symref, ref->refname);\n \t\tprintf(\"%s\\t%s\\n\", oid_to_hex(&ref->objectname), ref->refname);\n-\t\tstatus = 0; /* we found something */\n+\t\tif (status != 1)\n+\t\t\tstatus = 0; /* we found something */\n \t}\n\n \tref_array_clear(&ref_array);\n-\tif (transport_disconnect(transport))\n-\t\tstatus = 1;\n+\ttransport_clear(transport);\n \ttransport_ls_refs_options_release(&transport_options);\n \treturn status;\n }\ndiff --git a/builtin/remote.c b/builtin/remote.c index d91bbe728d..055a221942 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -1000,6 +1000,7 @@ static int get_remote_ref_states(const char *name,\n \t\ttransport = transport_get(states->remote, states->remote->url_nr > 0 ?\n \t\t\tstates->remote->url[0] : NULL);\n \t\tremote_refs = transport_get_remote_refs(transport, NULL);\n+\t\ttransport_disconnect_raw(transport);\n\n \t\tstates->queried = 1;\n \t\tif (query & GET_REF_STATES)\n@@ -1008,7 +1009,7 @@ static int get_remote_ref_states(const char *name,\n \t\t\tget_head_names(remote_refs, states);\n \t\tif (query & GET_PUSH_REF_STATES)\n \t\t\tget_push_ref_states(remote_refs, states);\n-\t\ttransport_disconnect(transport);\n+\t\ttransport_clear(transport);\n \t} else {\n \t\tfor_each_ref(append_ref_to_tracked_list, states);\n \t\tstring_list_sort(&states->tracked);\ndiff --git a/transport.c b/transport.c\nindex 219af8fd50..c71dab75e9 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1584,17 +1584,28 @@ int transport_connect(struct transport *transport, const char *name,\n \t\tdie(_(\"operation not supported by protocol\")); }\n\n-int transport_disconnect(struct transport *transport)\n+int transport_disconnect_raw(struct transport *transport)\n {\n \tint ret = 0;\n \tif (transport->vtable->disconnect)\n \t\tret = transport->vtable->disconnect(transport);\n+\treturn ret;\n+}\n+\n+void transport_clear(struct transport *transport) {\n \tif (transport->got_remote_refs)\n \t\tfree_refs((void *)transport->remote_refs);\n \tclear_bundle_list(transport->bundles);\n \tfree(transport->bundles);\n \tfree(transport);\n-\treturn ret;\n+}\n+\n+int transport_disconnect(struct transport *transport) {\n+\tint ret = transport_disconnect_raw(transport);\n+\ttransport_clear(transport);\n+\treturn 0;\n }\n\n /*\ndiff --git a/transport.h b/transport.h\nindex 6393cd9823..fd75905568 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -320,6 +320,8 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs);\n void transport_unlock_pack(struct transport *transport, unsigned int flags);\n\n int transport_disconnect(struct transport *transport);\n+void transport_clear(struct transport *transport);\n+int transport_disconnect_raw(struct transport *transport);\n char *transport_anonymize_url(const char *url);\n void transport_take_over(struct transport *transport,\n \t\t\t struct child_process *child);\n--\n2.42.0\n\n"},{"id":"484232","messageId":"4be3c37a-6a04-4d37-9b20-4199435e5cf7@web.de","threadId":"60420","inReplyTo":"BL0PR18MB2130C503D3608186AFC516D2BAA1A@BL0PR18MB2130.namprd18.prod.outlook.com","subject":"Re: [EXT] Re: ls-remote bug","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2023-10-31T10:00:36Z","receivedAt":"2023-10-31T10:00:54Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 30.10.23 um 09:17 schrieb Lior Zeltzer:\n> But why when cancelling stderr redirect to stdout (2>&1) all works well ?\n\nGood question. Maybe the output on stderr can give a hint? Any error\nmessages? I assume there is *some* output, otherwise the redirection\nshould have no effect.\n\nRené\n"}]}