# how to fetch entire heirarchy of refs from a remote? 8 messages from 2016-01-07 to 2016-01-08. Participants: Jacob Keller, Junio C Hamano. Thread: https://gitlist.dev/t/41145 ## Jacob Keller, 2016-01-07 23:06 Subject: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxqexsOma5c9NfS5ma333UJV%3D5i1a4GhnwZuU75eaxRd%3DQ%40mail.gmail.com ``` Hi, Say I have a repository which has a series of refs in a heirarchy such as a gerrit repository with changes specified something like: refs/changes/96/55596/3 refs/changes/96/55596/4 refs/changes/96/57496/1 refs/changes/96/57496/2 refs/changes/96/69796/1 refs/changes/96/69796/2 refs/changes/96/71696/1 refs/changes/96/71696/2 refs/changes/97/40197/1 refs/changes/97/40197/2 refs/changes/97/40197/3 refs/changes/97/40197/4 ... refs/changes/97/71697/2 refs/changes/97/71697/3 refs/changes/98/47298/1 refs/changes/98/47298/2 refs/changes/98/47298/3 refs/changes/98/57298/1 refs/changes/98/57298/2 refs/changes/98/73598/1 refs/changes/99/44099/1 refs/changes/99/69299/1 Is it possible to specify a refspec such that all of these will be fetched? I tried doing +refs/changes/*:refs/changes/* but this doesn't work since the * can only be one portion of a refspec The problem is that I would like to be able to pull all of these changes in a git mirror so that I can reduce network traffic I use to the gerrit server, by updating my local copy once and using the local copy on the other tasks I need. This is necessary since the gerrit server is (a) far away and (b) closes my connection when I have too many requests. But, git clone --mirror and git fetch have failed to pull every ref, and only end up with heads and tags. Regards, Jake ``` ## Junio C Hamano, 2016-01-07 23:15 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/xmqqpoxd2d74.fsf%40gitster.mtv.corp.google.com In-Reply-To: ``` Jacob Keller writes: > refs/changes/98/47298/3 > refs/changes/98/57298/1 > refs/changes/98/57298/2 > refs/changes/98/73598/1 > refs/changes/99/44099/1 > refs/changes/99/69299/1 > > > Is it possible to specify a refspec such that all of these will be > fetched? I tried doing > > +refs/changes/*:refs/changes/* > > but this doesn't work since the * can only be one portion of a refspec I am not sure if I follow. With refs/heads/*:refs/remotes/origin/* (e.g. following git://github.com/gitster/git for example), I am reasonably sure you would get my refs/heads/jk/notes-dwim-doc and other topics you authored without the slash between jk and notes-dwim-doc getting in the way. ``` ## Jacob Keller, 2016-01-07 23:23 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxrZKqchLUwp7tPaydsm5cdcHx38oU9%3DkPjen-A2Pvkx0g%40mail.gmail.com In-Reply-To: ``` On Thu, Jan 7, 2016 at 3:15 PM, Junio C Hamano wrote: > Jacob Keller writes: > >> refs/changes/98/47298/3 >> refs/changes/98/57298/1 >> refs/changes/98/57298/2 >> refs/changes/98/73598/1 >> refs/changes/99/44099/1 >> refs/changes/99/69299/1 >> >> >> Is it possible to specify a refspec such that all of these will be >> fetched? I tried doing >> >> +refs/changes/*:refs/changes/* >> >> but this doesn't work since the * can only be one portion of a refspec > > I am not sure if I follow. With refs/heads/*:refs/remotes/origin/* > (e.g. following git://github.com/gitster/git for example), I am > reasonably sure you would get my refs/heads/jk/notes-dwim-doc and > other topics you authored without the slash between jk and > notes-dwim-doc getting in the way. > > That appears to be correct, but.. why doesn't: git fetch origin +refs/changes/*:refs/changes/* seem to anything at all? Ideally this owuld fetch all the remote refs in refs/changes and stick them into my refs, but it appears to do nothing at all. Regards, Jake ``` ## Jacob Keller, 2016-01-07 23:25 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxpUW6as6aary%2B33V6sMo%3DBysFHpng6XeRRpXh2vXNKd%3Dg%40mail.gmail.com In-Reply-To: ``` On Thu, Jan 7, 2016 at 3:15 PM, Junio C Hamano wrote: > Jacob Keller writes: > >> refs/changes/98/47298/3 >> refs/changes/98/57298/1 >> refs/changes/98/57298/2 >> refs/changes/98/73598/1 >> refs/changes/99/44099/1 >> refs/changes/99/69299/1 >> >> >> Is it possible to specify a refspec such that all of these will be >> fetched? I tried doing >> >> +refs/changes/*:refs/changes/* >> >> but this doesn't work since the * can only be one portion of a refspec > > I am not sure if I follow. With refs/heads/*:refs/remotes/origin/* > (e.g. following git://github.com/gitster/git for example), I am > reasonably sure you would get my refs/heads/jk/notes-dwim-doc and > other topics you authored without the slash between jk and > notes-dwim-doc getting in the way. > > Infact, even running git fetch origin +refs/changes/98/73598/1:refs/changes/98/73598/1 does not end up placing anything into refs/changes? Is it because it happens to not be known by git? That seems rather short sighted... Regards, Jake ``` ## Jacob Keller, 2016-01-07 23:36 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2Bxra4hjskOLg%3DVhLMGazTSDH%3Dky8H%2Bfj2VMCw%2BHoY2Frrg%40mail.gmail.com In-Reply-To: ``` On Thu, Jan 7, 2016 at 3:06 PM, Jacob Keller wrote: > > But, git clone --mirror and git fetch have failed to pull every ref, > and only end up with heads and tags. > > Regards, > Jake It turns out that it *does* pull the refs, but they don't appear to show up inside the "refs" folder.. is there a reason for this? ie: ls .git/refs/changes shows nothing,but git ls-remote file:///path/to/repo does show them? Any particular reason for why this is? I am confused why refs aren't showing up inside the refs folder... Regards, Jake ``` ## Jacob Keller, 2016-01-07 23:39 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxptfL2YfNR9g7bmnmfYPoJ0HW4gLEfvRbxmNZ1B%3DpU1RQ%40mail.gmail.com In-Reply-To: ``` On Thu, Jan 7, 2016 at 3:06 PM, Jacob Keller wrote: > Hi, > > Say I have a repository which has a series of refs in a heirarchy such > as a gerrit repository with changes specified something like: > > refs/changes/96/55596/3 > refs/changes/96/55596/4 > refs/changes/96/57496/1 > refs/changes/96/57496/2 > refs/changes/96/69796/1 > refs/changes/96/69796/2 > refs/changes/96/71696/1 > refs/changes/96/71696/2 > refs/changes/97/40197/1 > refs/changes/97/40197/2 > refs/changes/97/40197/3 > refs/changes/97/40197/4 > ... > refs/changes/97/71697/2 > refs/changes/97/71697/3 > refs/changes/98/47298/1 > refs/changes/98/47298/2 > refs/changes/98/47298/3 > refs/changes/98/57298/1 > refs/changes/98/57298/2 > refs/changes/98/73598/1 > refs/changes/99/44099/1 > refs/changes/99/69299/1 > > > Is it possible to specify a refspec such that all of these will be > fetched? I tried doing > > +refs/changes/*:refs/changes/* > > but this doesn't work since the * can only be one portion of a refspec > > The problem is that I would like to be able to pull all of these > changes in a git mirror so that I can reduce network traffic I use to > the gerrit server, by updating my local copy once and using the local > copy on the other tasks I need. This is necessary since the gerrit > server is (a) far away and (b) closes my connection when I have too > many requests. > > But, git clone --mirror and git fetch have failed to pull every ref, > and only end up with heads and tags. > > Regards, > Jake So, sorry for the list spam. Things work correctly, it just is using packed-refs, which is a feature I wasn't aware of :) Git already did what I expected but in a way I did not, so I was very confused. Regards, Jake ``` ## Junio C Hamano, 2016-01-07 23:45 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/xmqqfuy92bt7.fsf%40gitster.mtv.corp.google.com In-Reply-To: ``` Jacob Keller writes: > On Thu, Jan 7, 2016 at 3:06 PM, Jacob Keller wrote: >> >> But, git clone --mirror and git fetch have failed to pull every ref, >> and only end up with heads and tags. >> >> Regards, >> Jake > > It turns out that it *does* pull the refs, but they don't appear to > show up inside the "refs" folder.. is there a reason for this? > > ie: > > ls .git/refs/changes > > shows nothing,but > > git ls-remote file:///path/to/repo > > does show them? Any particular reason for why this is? I am confused > why refs aren't showing up inside the refs folder... Does git for-each-ref refs/changes/ in your local resulting repository show them? There is nothing that says "ls -R .git/refs/" is the way to enumerate available refs (in fact it is a _wrong_ way that is guaranteed to give you wrong results). Hint: .git/packed-refs ``` ## Jacob Keller, 2016-01-08 16:30 Subject: Re: how to fetch entire heirarchy of refs from a remote? Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2BxogcBaAfWi5nf85ki44p%2BfLHJiRWUu5iMMaFutenoA83g%40mail.gmail.com In-Reply-To: ``` On Thu, Jan 7, 2016 at 3:45 PM, Junio C Hamano wrote: >> does show them? Any particular reason for why this is? I am confused >> why refs aren't showing up inside the refs folder... > > Does > > git for-each-ref refs/changes/ > > in your local resulting repository show them? > > There is nothing that says "ls -R .git/refs/" is the way to > enumerate available refs (in fact it is a _wrong_ way that is > guaranteed to give you wrong results). > > Hint: .git/packed-refs > Yep I figured that out :) Now I know better! Regards, Jake ```