{"thread":{"id":"45024","subject":"[PATCH] ls-remote: add \"--diff\" option to show only refs that differ","startedAt":"2017-02-02T19:49:59Z","lastAt":"2017-02-02T21:20:02Z","messageCount":5,"participants":["Linus Torvalds","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"310736","messageId":"alpine.LFD.2.20.1702021143470.21619@i7.lan","threadId":"45024","inReplyTo":null,"subject":"[PATCH] ls-remote: add \"--diff\" option to show only refs that differ","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-02-02T19:49:50Z","receivedAt":"2017-02-02T19:49:59Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nFrom: Linus Torvalds \nDate: Thu, 2 Feb 2017 11:37:49 -0800\nSubject: [PATCH] ls-remote: add \"--diff\" option to show only refs that differ\n\nMy main use of \"git ls-remote\" tends to be to check what the other end\nhas when some pull request goes wrong (they forgot to push, or they used\nthe wrong ref name or whatever), and it ends up being hard to see all\nthe relevant data from the noise of people just having the same basic\ntags etc from upstream.\n\nSo this adds a \"--diff\" option that shows only the refs that are\ndifferent from the local repository. So when somebody asks me to pull,\nI can now just trivially look at what they have that isn't already my\nbasic branches and tags.\n\nNote that \"--diff\" implies \"--refs\" (ie it also disables showing peeled\ntags).\n\nSigned-off-by: Linus Torvalds \n---\n\nThis is not a big deal, but maybe others have the same issues I've had. \nAnd maybe nobody else ever uses \"git ls-remote\". I dunno.\n\nAlso, I considered adding this feature as a more generic flag to \n\"check_ref_type()\" (ie add a REF_NONLOCAL option to complete the existing \nREF_NORMAL/REF_HEAD/etc flags), but that would have been a more involved \npatch and I'm not convinced it makes much sense for any other use, so I \nmade it a specific local hack to ls-remote instead.\n\nComments?\n\n builtin/ls-remote.c | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/builtin/ls-remote.c b/builtin/ls-remote.c\nindex 66cdd45cc..23469c3a6 100644\n--- a/builtin/ls-remote.c\n+++ b/builtin/ls-remote.c\n@@ -2,6 +2,7 @@\n #include \"cache.h\"\n #include \"transport.h\"\n #include \"remote.h\"\n+#include \"refs.h\"\n \n static const char * const ls_remote_usage[] = {\n \tN_(\"git ls-remote [--heads] [--tags] [--refs] [--upload-pack=]\\n\"\n@@ -31,6 +32,16 @@ static int tail_match(const char **pattern, const char *path)\n \treturn 0;\n }\n \n+static int has_ref_locally(const struct ref *ref)\n+{\n+\tunsigned char sha1[20];\n+\n+\tif (!resolve_ref_unsafe(ref->name, RESOLVE_REF_READING, sha1, NULL))\n+\t\treturn 0;\n+\n+\treturn !hashcmp(ref->old_oid.hash, sha1);\n+}\n+\n int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n {\n \tconst char *dest = NULL;\n@@ -39,6 +50,7 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \tint quiet = 0;\n \tint status = 0;\n \tint show_symref_target = 0;\n+\tint diff = 0;\n \tconst char *uploadpack = NULL;\n \tconst char **pattern = NULL;\n \n@@ -62,6 +74,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\t\t N_(\"exit with exit code 2 if no matching refs are found\"), 2),\n \t\tOPT_BOOL(0, \"symref\", &show_symref_target,\n \t\t\t N_(\"show underlying ref in addition to the object pointed by it\")),\n+\t\tOPT_BOOL(0, \"diff\", &diff,\n+\t\t\t N_(\"show only refs that differ from local refs\")),\n \t\tOPT_END()\n \t};\n \n@@ -98,6 +112,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \tif (transport_disconnect(transport))\n \t\treturn 1;\n \n+\tif (diff)\n+\t\tflags |= REF_NORMAL;\n \tif (!dest && !quiet)\n \t\tfprintf(stderr, \"From %s\\n\", *remote->url);\n \tfor ( ; ref; ref = ref->next) {\n@@ -105,6 +121,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n \t\t\tcontinue;\n \t\tif (!tail_match(pattern, ref->name))\n \t\t\tcontinue;\n+\t\tif (diff && has_ref_locally(ref))\n+\t\t\tcontinue;\n \t\tif (show_symref_target && ref->symref)\n \t\t\tprintf(\"ref: %s\\t%s\\n\", ref->symref, ref->name);\n \t\tprintf(\"%s\\t%s\\n\", oid_to_hex(&ref->old_oid), ref->name);\n"},{"id":"310737","messageId":"xmqqshnws6ma.fsf@gitster.mtv.corp.google.com","threadId":"45024","inReplyTo":"alpine.LFD.2.20.1702021143470.21619@i7.lan","subject":"Re: [PATCH] ls-remote: add \"--diff\" option to show only refs that differ","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-02T20:03:09Z","receivedAt":"2017-02-02T20:03:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> My main use of \"git ls-remote\" tends to be to check what the other end\n> has when some pull request goes wrong (they forgot to push, or they used\n> the wrong ref name or whatever), and it ends up being hard to see all\n> the relevant data from the noise of people just having the same basic\n> tags etc from upstream.\n>\n> So this adds a \"--diff\" option that shows only the refs that are\n> different from the local repository. So when somebody asks me to pull,\n> I can now just trivially look at what they have that isn't already my\n> basic branches and tags.\n\nMost downstream folks seem to care about refs/remotes/origin/$branch\nand I think in that context \"git ls-remote --diff [origin]\" that\ncompares their refs/heads/* and refs/remotes/origin/* would make\nsense. Your has_ref_locally() seems to return true by comparing\ntheir value with the value of the local ref without any the fetch\nrefspec mapping.\n\nWhen one contributor asks you to pull refs/heads/master you want to\ngo and see if it is different from refs/heads/master you have?\n\n> Comments?\n>\n> +static int has_ref_locally(const struct ref *ref)\n> +{\n> +\tunsigned char sha1[20];\n> +\n> +\tif (!resolve_ref_unsafe(ref->name, RESOLVE_REF_READING, sha1, NULL))\n> +\t\treturn 0;\n> +\n> +\treturn !hashcmp(ref->old_oid.hash, sha1);\n> +}\n> +\n> @@ -105,6 +121,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)\n> \t\t\tcontinue;\n> \t\tif (!tail_match(pattern, ref->name))\n> \t\t\tcontinue;\n> +\t\tif (diff && has_ref_locally(ref))\n> +\t\t\tcontinue;\n> \t\tif (show_symref_target && ref->symref)\n> \t\t\tprintf(\"ref: %s\\t%s\\n\", ref->symref, ref->name);\n> \t\tprintf(\"%s\\t%s\\n\", oid_to_hex(&ref->old_oid), ref->name);\n"},{"id":"310739","messageId":"CA+55aFw14UXMa6OJ6YLHjy3tzOD+VSNytw6kMpaxFEfyuO2hAw@mail.gmail.com","threadId":"45024","inReplyTo":"xmqqshnws6ma.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] ls-remote: add \"--diff\" option to show only refs that differ","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-02-02T20:32:45Z","receivedAt":"2017-02-02T20:32:51Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Thu, Feb 2, 2017 at 12:03 PM, Junio C Hamano wrote:\n>\n> Most downstream folks seem to care about refs/remotes/origin/$branch\n> and I think in that context \"git ls-remote --diff [origin]\" that\n> compares their refs/heads/* and refs/remotes/origin/* would make\n> sense.\n\nHmm. Maybe. The main target for noise reduction for me was actually\nall the shared tags.\n\nWhich doesn't have that issue.\n\nAlso, I've never ever used \"git ls-remote\" on origin. Do people\nactually do that? Why would a regular user ever use ls-remote in the\nfirst place?\n\nThe only reason I've ever had for using ls-remote is exactly because\nthe remote is somehow \"odd\", and the _normal_ flow didn't work, so you\nwant to start investigating. So by definition (at least for me),\nls-remote is not part of a good normal flow.\n\nSo I kind of see where you are coming from, but I don't really see\nthat as being a normal workflow for me - or really anybody.\n\nWhat I think *your* use case is would be more for a workflow along the lines of\n\n # update the remote data\n git fetch [origin]\n\n # have some way to just see what branches are not the same\n git status --all\n\nor something (\"git status\" already talks about the status of the\ncurrent branch vs the origin branch).\n\n> Your has_ref_locally() seems to return true by comparing\n> their value with the value of the local ref without any the fetch\n> refspec mapping.\n\nRight. Because I see the use of \"ls-remote\" being mostly for\nmaintainer pulls, and the \"origin\" in many ways would be the other way\naround (and you wouldn't even know what the name of said origin would\nbe locally).\n\nI basically don't see downstream contributor doing ls-remote, it's a\nupstream maintainer command.\n\nBut that may be a lack of imagination on my part.\n\n> When one contributor asks you to pull refs/heads/master you want to\n> go and see if it is different from refs/heads/master you have?\n\nNo. What happens is that people ask me to do something like\n\n git pull ..some-target.. tags/for-linus-3\n\nand the pull fails because there is no such tag. That's when I go \"ok,\nthey screwed up, let's see what they *meant* for me to pull\", and I go\n\"git ls-remote\".\n\nIn other words, I don't see anybody ever using git ls-remote if they\nalready know what the remote is. That's why I don't see \"origin\" to be\nan issue - origin is by definition somethinig you trust, and you just\nfetch and pull from.\n\nSo the only reason I've ever had for using ls-remote is literally \"ok,\nwhat the hell is going on at that remote repo\".\n\nAnd then it is generally a bare repository, and it generally does\n*not* have remote branches in it at all. But it *does* generally end\nup having all the basic branches and tags (not always, but it's very\ncommon).\n\nWhich is why I as a maintainer then want to just weed out anything\nthat is already my usual branches that everybody downstream already\nhas.\n\nI agree that it's a specialized case, but I also think it's the _main_\ncase for ls-remote in the first place (apart from some scripting to\ncheck for updates or whatever).\n\nBut maybe more people use ls-remote than I think they do (and in\ndifferent ways than what I envision).\n\n Linus\n"},{"id":"310740","messageId":"xmqqfujws3pl.fsf@gitster.mtv.corp.google.com","threadId":"45024","inReplyTo":"CA+55aFw14UXMa6OJ6YLHjy3tzOD+VSNytw6kMpaxFEfyuO2hAw@mail.gmail.com","subject":"Re: [PATCH] ls-remote: add \"--diff\" option to show only refs that differ","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-02T21:05:58Z","receivedAt":"2017-02-02T21:06:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> I basically don't see downstream contributor doing ls-remote, it's a\n> upstream maintainer command.\n>\n> But that may be a lack of imagination on my part.\n\nI actually share that perception. For the \"downstream wonders about\nthe state of the origin\" usecase, I would rather recommend \"fetch\",\neither without \"-n\" (when the downstream does not value the current\nstate of refs/remotes/*) or with \"-n\" (when it does for whatever\nreason).\n\n>> When one contributor asks you to pull refs/heads/master you want to\n>> go and see if it is different from refs/heads/master you have?\n>\n> No. What happens is that people ask me to do something like\n>\n> git pull ..some-target.. tags/for-linus-3\n>\n> and the pull fails because there is no such tag. That's when I go \"ok,\n> they screwed up, let's see what they *meant* for me to pull\", and I go\n> \"git ls-remote\".\n\nIn that context, I fully agree that \"--diff --tags\" would help. The\ncopies of your tags they have there would overwhelm what you are\nreally looking for in the output from the command.\n\nAnd if they asked you to pull \"for-linus-3\" branch, which is buried\nin many other branches (perhaps their publishing repository they ask\nyou to pull from is also serving as their back-up place, and the\nlocal branches they use before coming up with something pull-able\nare all there), then \"--diff --refs\" would still help by culling\ntags that originated from you.\n\n> I agree that it's a specialized case, but I also think it's the _main_\n> case for ls-remote in the first place (apart from some scripting to\n> check for updates or whatever).\n>\n> But maybe more people use ls-remote than I think they do (and in\n> different ways than what I envision).\n\nYou and I are not the only folks in the world, but I agree with you\nin thinking that \"ls-remote\" is not something you would use on\n'origin' as a downstream contributor or a consumer. \n\nAnother case I can think of that \"--diff\" would help is when you are\ninspecting your own mirror (but that can be seen as a special case\nof the \"they have copies of yours plus their own\", if you think of\nyour mirror as \"them\" and the difference is \"being stale\").\n"},{"id":"310741","messageId":"CA+55aFy27uEXX2FfQB7q5SWAaV-E0etCrBf0MnFRzARb1P2CaA@mail.gmail.com","threadId":"45024","inReplyTo":"xmqqfujws3pl.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] ls-remote: add \"--diff\" option to show only refs that differ","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-02-02T21:19:56Z","receivedAt":"2017-02-02T21:20:02Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Thu, Feb 2, 2017 at 1:05 PM, Junio C Hamano wrote:\n>\n> Another case I can think of that \"--diff\" would help is when you are\n> inspecting your own mirror (but that can be seen as a special case\n> of the \"they have copies of yours plus their own\", if you think of\n> your mirror as \"them\" and the difference is \"being stale\").\n\nYeah, that's actually what I did for some testing (not having stale\nbranches, but just to check the expected differences for my upstream\nkernel repo with my local pulls that I haven't pushed out yet).\n\nThe actual real use-case is something that only happens for me only\nvery occasionally. I end up sending out \"did you forget to push\"\nemails perhaps a couple of times every release, but every time I do I\nwill have gone and done a ls-remote on their repo..\n\n Linus\n"}]}