{"thread":{"id":"28100","subject":"shallow clone not very shallow due to tags","startedAt":"2011-08-14T23:58:24Z","lastAt":"2011-08-17T14:35:52Z","messageCount":7,"participants":["Shawn Pearce","Nguyen Thai Ngoc Duy","Philip Oakley"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"173522","messageId":"CAJo=hJuyZMj+qwFr_=stbQtGh2SCCpjfsBxm+2wbfJK=i_VTdw@mail.gmail.com","threadId":"28100","inReplyTo":null,"subject":"shallow clone not very shallow due to tags","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2011-08-14T23:58:24Z","receivedAt":"2011-08-14T23:58:24Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"$ git clone --depth 1 git://repo.or.cz/alt-git.git tmp_cgit_5\nCloning into tmp_cgit_5...\nremote: Counting objects: 20234, done.\nremote: Compressing objects: 100% (9454/9454), done.\nremote: Total 20234 (delta 16355), reused 13587 (delta 10421)\nReceiving objects: 100% (20234/20234), 9.15 MiB | 1.17 MiB/s, done.\nResolving deltas: 100% (16355/16355), done.\n\n$ cd tmp_cgit_5\n$ git tag -l | grep 0.99\nv0.99\n\nUhm. That is not a very shallow clone. The clone copied 20234 objects\nat 9.15 MiB... so its ~20 MiB lighter than a full clone. But nearly\nall of the tags exist, because the clone client is declaring want\nlines for them, making the server generate up to 1 commit back from\nthe wanted tag. I know shallow support is the feature nobody wants to\nthink about, but this just seems broken to me. Clients performing a\nshallow clone shouldn't be asking for tags... but they should be using\nthe include-tag protocol option so that if they do happen to receive a\ntagged commit, the tag object will also be sent.\n\n-- \nShawn.\n"},{"id":"173532","messageId":"CACsJy8DF49PJtv41vZEOX0fLg4U1+m6JEW28+iYD6jJxGctEzg@mail.gmail.com","threadId":"28100","inReplyTo":"CAJo=hJuyZMj+qwFr_=stbQtGh2SCCpjfsBxm+2wbfJK=i_VTdw@mail.gmail.com","subject":"Re: shallow clone not very shallow due to tags","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-08-15T14:03:19Z","receivedAt":"2011-08-15T14:03:19Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Aug 15, 2011 at 6:58 AM, Shawn Pearce wrote:\n> Uhm. That is not a very shallow clone. The clone copied 20234 objects\n> at 9.15 MiB... so its ~20 MiB lighter than a full clone. But nearly\n> all of the tags exist, because the clone client is declaring want\n> lines for them, making the server generate up to 1 commit back from\n> the wanted tag. I know shallow support is the feature nobody wants to\n> think about, but this just seems broken to me. Clients performing a\n> shallow clone shouldn't be asking for tags... but they should be using\n> the include-tag protocol option so that if they do happen to receive a\n> tagged commit, the tag object will also be sent.\n\nThe same would apply if the repo in question has many branches. Should\nwe fetch only master (or a user-specified set of refs) in shallow\nclone?\n-- \nDuy\n"},{"id":"173548","messageId":"CAJo=hJv_NnZyGnedTQBL_mj3baS8O5Z2wiEbZCvv8305No4ehA@mail.gmail.com","threadId":"28100","inReplyTo":"CACsJy8DF49PJtv41vZEOX0fLg4U1+m6JEW28+iYD6jJxGctEzg@mail.gmail.com","subject":"Re: shallow clone not very shallow due to tags","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2011-08-15T17:16:52Z","receivedAt":"2011-08-15T17:16:52Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Mon, Aug 15, 2011 at 07:03, Nguyen Thai Ngoc Duy wrote:\n> On Mon, Aug 15, 2011 at 6:58 AM, Shawn Pearce wrote:\n>> Uhm. That is not a very shallow clone. The clone copied 20234 objects\n>> at 9.15 MiB... so its ~20 MiB lighter than a full clone. But nearly\n>> all of the tags exist, because the clone client is declaring want\n>> lines for them, making the server generate up to 1 commit back from\n>> the wanted tag. I know shallow support is the feature nobody wants to\n>> think about, but this just seems broken to me. Clients performing a\n>> shallow clone shouldn't be asking for tags... but they should be using\n>> the include-tag protocol option so that if they do happen to receive a\n>> tagged commit, the tag object will also be sent.\n>\n> The same would apply if the repo in question has many branches. Should\n> we fetch only master (or a user-specified set of refs) in shallow\n> clone?\n\nYes, I think a user who is making a shallow clone should only get the\nHEAD branch to the depth specified, nothing else. If they want\nadditional branches, they should either pass them on the command line\nto a new flag for clone, or modify their fetch configuration\nafter-the-fact and fetch those separately.\n\n>From what I can gather from some users I have talked to, the primary\nusage of shallow clone is to try and (sort of) quickly grab a branch,\nmake a change, and post that change to the maintainers for review and\nacceptance. E.g. correcting a spelling typo. Relatively simple changes\nthat can be built on a specific branch, and don't really require all\nof the history.\n\nAnd if a repository does have more than one branch, but is shallow\ncloned at say depth of 1, the user probably doesn't get the merge\nbases between them, so the value of the other branches is greatly\nreduced. You can't make a merge between these, and a new developer\ngetting involved in the project cannot see how the branches relate to\neach other. So there isn't a lot of value in sucking down those\nadditional branches during clone.\n\n-- \nShawn.\n"},{"id":"173559","messageId":"4EEB129543344D0AA1C3AD3696F8F5D5@PhilipOakley","threadId":"28100","inReplyTo":"CAJo=hJv_NnZyGnedTQBL_mj3baS8O5Z2wiEbZCvv8305No4ehA@mail.gmail.com","subject":"Re: shallow clone not very shallow due to tags","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2011-08-15T23:00:09Z","receivedAt":"2011-08-15T23:00:09Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"Monday, August 15, 2011 6:16 PM \"Shawn Pearce\" \n> On Mon, Aug 15, 2011 at 07:03, Nguyen Thai Ngoc Duy \n> wrote:\n>> On Mon, Aug 15, 2011 at 6:58 AM, Shawn Pearce \n>> wrote:\n>>> Uhm. That is not a very shallow clone. The clone copied 20234 objects\n>>> at 9.15 MiB... so its ~20 MiB lighter than a full clone. But nearly\n>>> all of the tags exist, because the clone client is declaring want\n>>> lines for them, making the server generate up to 1 commit back from\n>>> the wanted tag. I know shallow support is the feature nobody wants to\n>>> think about, but this just seems broken to me. Clients performing a\n>>> shallow clone shouldn't be asking for tags... but they should be using\n>>> the include-tag protocol option so that if they do happen to receive a\n>>> tagged commit, the tag object will also be sent.\n>>\n>> The same would apply if the repo in question has many branches. Should\n>> we fetch only master (or a user-specified set of refs) in shallow\n>> clone?\n>\n> Yes, I think a user who is making a shallow clone should only get the\n> HEAD branch to the depth specified, nothing else. If they want\n> additional branches, they should either pass them on the command line\n> to a new flag for clone, or modify their fetch configuration\n> after-the-fact and fetch those separately.\n>\n> From what I can gather from some users I have talked to, the primary\n> usage of shallow clone is to try and (sort of) quickly grab a branch,\n> make a change, and post that change to the maintainers for review and\n> acceptance. E.g. correcting a spelling typo. Relatively simple changes\n> that can be built on a specific branch, and don't really require all\n> of the history.\n>\n> And if a repository does have more than one branch, but is shallow\n> cloned at say depth of 1, the user probably doesn't get the merge\n> bases between them, so the value of the other branches is greatly\n> reduced. You can't make a merge between these, and a new developer\n> getting involved in the project cannot see how the branches relate to\n> each other. So there isn't a lot of value in sucking down those\n> additional branches during clone.\n>\nThe shallow clone problem came up a few times recently on StackOverflow. One \nusage is to create an orphan branch, but the manual doesn't point the user \nto that option. When I first read the man page I expected that any clone \nwould be from a fixed point in history, rather than a point that may move \n(as extra commits are made to the remote). It maybe that what is needed is \nan option that will clone from a fixed commit point, so that there is no \nconfusion as to the 'current depth'. \n"},{"id":"173661","messageId":"CACsJy8CXiZ8uGruAk5XGFhY8MJsTHAPv3Mvh0euQdFE3U43MCg@mail.gmail.com","threadId":"28100","inReplyTo":"CAJo=hJv_NnZyGnedTQBL_mj3baS8O5Z2wiEbZCvv8305No4ehA@mail.gmail.com","subject":"Re: shallow clone not very shallow due to tags","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-08-17T02:49:59Z","receivedAt":"2011-08-17T02:49:59Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Aug 16, 2011 at 12:16 AM, Shawn Pearce > Yes, I think a user\nwho is making a shallow clone should only get the\n> HEAD branch to the depth specified, nothing else. If they want\n> additional branches, they should either pass them on the command line\n> to a new flag for clone, or modify their fetch configuration\n> after-the-fact and fetch those separately.\n\nI'd like maintain all refs of the repo, even though we though we only\nhave objects for one branch. If I do a shallow clone and want to\nchange branch, I could just list available branch locally instead of\ngoing to gitweb.somewhere for the branch name.\n\nDo we have someway to represent a ref placeholder?\n-- \nDuy\n"},{"id":"173673","messageId":"CAJo=hJvySfuWmz6vGUveC5KoR1NZGC4ji0saByVX40UxLqHqdg@mail.gmail.com","threadId":"28100","inReplyTo":"CACsJy8CXiZ8uGruAk5XGFhY8MJsTHAPv3Mvh0euQdFE3U43MCg@mail.gmail.com","subject":"Re: shallow clone not very shallow due to tags","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2011-08-17T14:19:31Z","receivedAt":"2011-08-17T14:19:31Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Tue, Aug 16, 2011 at 19:49, Nguyen Thai Ngoc Duy wrote:\n> On Tue, Aug 16, 2011 at 12:16 AM, Shawn Pearce > Yes, I think a user\n> who is making a shallow clone should only get the\n>> HEAD branch to the depth specified, nothing else. If they want\n>> additional branches, they should either pass them on the command line\n>> to a new flag for clone, or modify their fetch configuration\n>> after-the-fact and fetch those separately.\n>\n> I'd like maintain all refs of the repo, even though we though we only\n> have objects for one branch. If I do a shallow clone and want to\n> change branch, I could just list available branch locally instead of\n> going to gitweb.somewhere for the branch name.\n>\n> Do we have someway to represent a ref placeholder?\n\nNot currently, no. I haven't checked refs.c, but you may be able to\ncreate an empty (0 byte) ref file for each remote reference we didn't\nobtain. These should be ignored since they aren't a valid reference..\nThough they might cause warnings to be sent to stderr on every\ncommand, since they exist in the refs directory but aren't a valid\nreference.\n\nOf course a placeholder ref cannot be used as a starting point to\ncheckout something, can it? It has no commit object available. So if\nyou did decide to checkout a placeholder, you would need to first\nfetch that placeholder. So why not use git ls-remote to list the\nremotely available branches?\n\n-- \nShawn.\n"},{"id":"173676","messageId":"CACsJy8BGf1uV13PrPHi621N_aeXzamOFv_yZQoL0gxUxqv-_hQ@mail.gmail.com","threadId":"28100","inReplyTo":"CAJo=hJvySfuWmz6vGUveC5KoR1NZGC4ji0saByVX40UxLqHqdg@mail.gmail.com","subject":"Re: shallow clone not very shallow due to tags","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-08-17T14:35:52Z","receivedAt":"2011-08-17T14:35:52Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Aug 17, 2011 at 9:19 PM, Shawn Pearce wrote:\n> Of course a placeholder ref cannot be used as a starting point to\n> checkout something, can it? It has no commit object available. So if\n> you did decide to checkout a placeholder, you would need to first\n> fetch that placeholder. So why not use git ls-remote to list the\n> remotely available branches?\n\nAh.. ls-remote is the answer. Thanks, placeholders are gone (for me).\n-- \nDuy\n"}]}