{"thread":{"id":"41145","subject":"how to fetch entire heirarchy of refs from a remote?","startedAt":"2016-01-07T23:06:00Z","lastAt":"2016-01-08T16:30:52Z","messageCount":8,"participants":["Jacob Keller","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"275509","messageId":"CA+P7+xqexsOma5c9NfS5ma333UJV=5i1a4GhnwZuU75eaxRd=Q@mail.gmail.com","threadId":"41145","inReplyTo":null,"subject":"how to fetch entire heirarchy of refs from a remote?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-01-07T23:06:00Z","receivedAt":"2016-01-07T23:06:00Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"Hi,\n\nSay I have a repository which has a series of refs in a heirarchy such\nas a gerrit repository with changes specified something like:\n\nrefs/changes/96/55596/3\nrefs/changes/96/55596/4\nrefs/changes/96/57496/1\nrefs/changes/96/57496/2\nrefs/changes/96/69796/1\nrefs/changes/96/69796/2\nrefs/changes/96/71696/1\nrefs/changes/96/71696/2\nrefs/changes/97/40197/1\nrefs/changes/97/40197/2\nrefs/changes/97/40197/3\nrefs/changes/97/40197/4\n...\nrefs/changes/97/71697/2\nrefs/changes/97/71697/3\nrefs/changes/98/47298/1\nrefs/changes/98/47298/2\nrefs/changes/98/47298/3\nrefs/changes/98/57298/1\nrefs/changes/98/57298/2\nrefs/changes/98/73598/1\nrefs/changes/99/44099/1\nrefs/changes/99/69299/1\n\n\nIs it possible to specify a refspec such that all of these will be\nfetched? I tried doing\n\n+refs/changes/*:refs/changes/*\n\nbut this doesn't work since the * can only be one portion of a refspec\n\nThe problem is that I would like to be able to pull all of these\nchanges in a git mirror so that I can reduce network traffic I use to\nthe gerrit server, by updating my local copy once and using the local\ncopy on the other tasks I need. This is necessary since the gerrit\nserver is (a) far away and (b) closes my connection when I have too\nmany requests.\n\nBut, git clone --mirror and git fetch have failed to pull every ref,\nand only end up with heads and tags.\n\nRegards,\nJake\n"},{"id":"275510","messageId":"xmqqpoxd2d74.fsf@gitster.mtv.corp.google.com","threadId":"41145","inReplyTo":"CA+P7+xqexsOma5c9NfS5ma333UJV=5i1a4GhnwZuU75eaxRd=Q@mail.gmail.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-07T23:15:59Z","receivedAt":"2016-01-07T23:15:59Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller writes:\n\n> refs/changes/98/47298/3\n> refs/changes/98/57298/1\n> refs/changes/98/57298/2\n> refs/changes/98/73598/1\n> refs/changes/99/44099/1\n> refs/changes/99/69299/1\n>\n>\n> Is it possible to specify a refspec such that all of these will be\n> fetched? I tried doing\n>\n> +refs/changes/*:refs/changes/*\n>\n> but this doesn't work since the * can only be one portion of a refspec\n\nI am not sure if I follow. With refs/heads/*:refs/remotes/origin/*\n(e.g. following git://github.com/gitster/git for example), I am\nreasonably sure you would get my refs/heads/jk/notes-dwim-doc and\nother topics you authored without the slash between jk and\nnotes-dwim-doc getting in the way.\n"},{"id":"275511","messageId":"CA+P7+xrZKqchLUwp7tPaydsm5cdcHx38oU9=kPjen-A2Pvkx0g@mail.gmail.com","threadId":"41145","inReplyTo":"xmqqpoxd2d74.fsf@gitster.mtv.corp.google.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-01-07T23:23:42Z","receivedAt":"2016-01-07T23:23:42Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Jan 7, 2016 at 3:15 PM, Junio C Hamano wrote:\n> Jacob Keller writes:\n>\n>> refs/changes/98/47298/3\n>> refs/changes/98/57298/1\n>> refs/changes/98/57298/2\n>> refs/changes/98/73598/1\n>> refs/changes/99/44099/1\n>> refs/changes/99/69299/1\n>>\n>>\n>> Is it possible to specify a refspec such that all of these will be\n>> fetched? I tried doing\n>>\n>> +refs/changes/*:refs/changes/*\n>>\n>> but this doesn't work since the * can only be one portion of a refspec\n>\n> I am not sure if I follow. With refs/heads/*:refs/remotes/origin/*\n> (e.g. following git://github.com/gitster/git for example), I am\n> reasonably sure you would get my refs/heads/jk/notes-dwim-doc and\n> other topics you authored without the slash between jk and\n> notes-dwim-doc getting in the way.\n>\n>\n\nThat appears to be correct, but.. why doesn't:\n\ngit fetch origin +refs/changes/*:refs/changes/*\n\nseem to anything at all?\n\nIdeally this owuld fetch all the remote refs in refs/changes and stick\nthem into my refs, but it appears to do nothing at all.\n\nRegards,\nJake\n"},{"id":"275512","messageId":"CA+P7+xpUW6as6aary+33V6sMo=BysFHpng6XeRRpXh2vXNKd=g@mail.gmail.com","threadId":"41145","inReplyTo":"xmqqpoxd2d74.fsf@gitster.mtv.corp.google.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-01-07T23:25:31Z","receivedAt":"2016-01-07T23:25:31Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Jan 7, 2016 at 3:15 PM, Junio C Hamano wrote:\n> Jacob Keller writes:\n>\n>> refs/changes/98/47298/3\n>> refs/changes/98/57298/1\n>> refs/changes/98/57298/2\n>> refs/changes/98/73598/1\n>> refs/changes/99/44099/1\n>> refs/changes/99/69299/1\n>>\n>>\n>> Is it possible to specify a refspec such that all of these will be\n>> fetched? I tried doing\n>>\n>> +refs/changes/*:refs/changes/*\n>>\n>> but this doesn't work since the * can only be one portion of a refspec\n>\n> I am not sure if I follow. With refs/heads/*:refs/remotes/origin/*\n> (e.g. following git://github.com/gitster/git for example), I am\n> reasonably sure you would get my refs/heads/jk/notes-dwim-doc and\n> other topics you authored without the slash between jk and\n> notes-dwim-doc getting in the way.\n>\n>\n\n\nInfact, even running\n\ngit fetch origin +refs/changes/98/73598/1:refs/changes/98/73598/1\n\ndoes not end up placing anything into refs/changes?\n\nIs it because it happens to not be known by git? That seems rather\nshort sighted...\n\nRegards,\nJake\n"},{"id":"275513","messageId":"CA+P7+xra4hjskOLg=VhLMGazTSDH=ky8H+fj2VMCw+HoY2Frrg@mail.gmail.com","threadId":"41145","inReplyTo":"CA+P7+xqexsOma5c9NfS5ma333UJV=5i1a4GhnwZuU75eaxRd=Q@mail.gmail.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-01-07T23:36:30Z","receivedAt":"2016-01-07T23:36:30Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Jan 7, 2016 at 3:06 PM, Jacob Keller wrote:\n>\n> But, git clone --mirror and git fetch have failed to pull every ref,\n> and only end up with heads and tags.\n>\n> Regards,\n> Jake\n\nIt turns out that it *does* pull the refs, but they don't appear to\nshow up inside the \"refs\" folder.. is there a reason for this?\n\nie:\n\nls .git/refs/changes\n\nshows nothing,but\n\ngit ls-remote file:///path/to/repo\n\ndoes show them? Any particular reason for why this is? I am confused\nwhy refs aren't showing up inside the refs folder...\n\nRegards,\nJake\n"},{"id":"275514","messageId":"CA+P7+xptfL2YfNR9g7bmnmfYPoJ0HW4gLEfvRbxmNZ1B=pU1RQ@mail.gmail.com","threadId":"41145","inReplyTo":"CA+P7+xqexsOma5c9NfS5ma333UJV=5i1a4GhnwZuU75eaxRd=Q@mail.gmail.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-01-07T23:39:25Z","receivedAt":"2016-01-07T23:39:25Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Jan 7, 2016 at 3:06 PM, Jacob Keller wrote:\n> Hi,\n>\n> Say I have a repository which has a series of refs in a heirarchy such\n> as a gerrit repository with changes specified something like:\n>\n> refs/changes/96/55596/3\n> refs/changes/96/55596/4\n> refs/changes/96/57496/1\n> refs/changes/96/57496/2\n> refs/changes/96/69796/1\n> refs/changes/96/69796/2\n> refs/changes/96/71696/1\n> refs/changes/96/71696/2\n> refs/changes/97/40197/1\n> refs/changes/97/40197/2\n> refs/changes/97/40197/3\n> refs/changes/97/40197/4\n> ...\n> refs/changes/97/71697/2\n> refs/changes/97/71697/3\n> refs/changes/98/47298/1\n> refs/changes/98/47298/2\n> refs/changes/98/47298/3\n> refs/changes/98/57298/1\n> refs/changes/98/57298/2\n> refs/changes/98/73598/1\n> refs/changes/99/44099/1\n> refs/changes/99/69299/1\n>\n>\n> Is it possible to specify a refspec such that all of these will be\n> fetched? I tried doing\n>\n> +refs/changes/*:refs/changes/*\n>\n> but this doesn't work since the * can only be one portion of a refspec\n>\n> The problem is that I would like to be able to pull all of these\n> changes in a git mirror so that I can reduce network traffic I use to\n> the gerrit server, by updating my local copy once and using the local\n> copy on the other tasks I need. This is necessary since the gerrit\n> server is (a) far away and (b) closes my connection when I have too\n> many requests.\n>\n> But, git clone --mirror and git fetch have failed to pull every ref,\n> and only end up with heads and tags.\n>\n> Regards,\n> Jake\n\nSo, sorry for the list spam. Things work correctly, it just is using\npacked-refs, which is a feature I wasn't aware of :) Git already did\nwhat I expected but in a way I did not, so I was very confused.\n\nRegards,\nJake\n"},{"id":"275516","messageId":"xmqqfuy92bt7.fsf@gitster.mtv.corp.google.com","threadId":"41145","inReplyTo":"CA+P7+xra4hjskOLg=VhLMGazTSDH=ky8H+fj2VMCw+HoY2Frrg@mail.gmail.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-07T23:45:56Z","receivedAt":"2016-01-07T23:45:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jacob Keller writes:\n\n> On Thu, Jan 7, 2016 at 3:06 PM, Jacob Keller wrote:\n>>\n>> But, git clone --mirror and git fetch have failed to pull every ref,\n>> and only end up with heads and tags.\n>>\n>> Regards,\n>> Jake\n>\n> It turns out that it *does* pull the refs, but they don't appear to\n> show up inside the \"refs\" folder.. is there a reason for this?\n>\n> ie:\n>\n> ls .git/refs/changes\n>\n> shows nothing,but\n>\n> git ls-remote file:///path/to/repo\n>\n> does show them? Any particular reason for why this is? I am confused\n> why refs aren't showing up inside the refs folder...\n\nDoes\n\n\tgit for-each-ref refs/changes/\n\nin your local resulting repository show them?\n\nThere is nothing that says \"ls -R .git/refs/\" is the way to\nenumerate available refs (in fact it is a _wrong_ way that is\nguaranteed to give you wrong results).\n\nHint: .git/packed-refs\n"},{"id":"275563","messageId":"CA+P7+xogcBaAfWi5nf85ki44p+fLHJiRWUu5iMMaFutenoA83g@mail.gmail.com","threadId":"41145","inReplyTo":"xmqqfuy92bt7.fsf@gitster.mtv.corp.google.com","subject":"Re: how to fetch entire heirarchy of refs from a remote?","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-01-08T16:30:52Z","receivedAt":"2016-01-08T16:30:52Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Jan 7, 2016 at 3:45 PM, Junio C Hamano wrote:\n>> does show them? Any particular reason for why this is? I am confused\n>> why refs aren't showing up inside the refs folder...\n>\n> Does\n>\n> git for-each-ref refs/changes/\n>\n> in your local resulting repository show them?\n>\n> There is nothing that says \"ls -R .git/refs/\" is the way to\n> enumerate available refs (in fact it is a _wrong_ way that is\n> guaranteed to give you wrong results).\n>\n> Hint: .git/packed-refs\n>\n\nYep I figured that out :) Now I know better!\n\nRegards,\nJake\n"}]}