"git clone --shallow-exclude ...", fatal: the remote end hung up unexpectedly

  probably doing something idiotic but i'm enumerating variations of
shallow cloning, and tried the following:

$ git clone --shallow-exclude=master https://github.com/django/django.git Cloning into 'django'... fatal: the remote end hung up unexpectedly $

  it is entirely reproducible, and some googling suggests that this
represents an error at the *other* end, which in some weird way does
not support that clone option. that seems strange ... should this
option work? am i using it incorrectly?

  wait, hang on ... i just picked one of django's topic branches at
random, and this did succeed:

$ git clone --shallow-exclude=stable/2.0.x https://github.com/django/django.git Cloning into 'django'... remote: Enumerating objects: 33112, done. remote: Counting objects: 100% (33112/33112), done. ... etc etc ... $

but all this gave me was the master branch. i clearly don't understand what this option is supposed to do.

rday

On Fri, Mar 15, 2019 at 7:17 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:

Yeah i think when you request shallow clone, by default it only gets one branch (see --single-branch, often 'master'). So when you specify --shallow-exclude you essentially say 'give me master branch but exclude everything from master'.

I should probably make it print a friendlier message than simply terminateing like that (it's still a guess, I haven't tried it out)

You're saying 'give me master branch, exclude everything that is reachable from stable/2.0.x'. If there's nothing in common between master and this branch, you get full 'master' branch. Otherwise you get a shallow 'master' branch.

This option is meant to give you another way to cut the history. Instead of saying 'give me X latest commits' with --shallow, you could say 'give me commits since this cut point' and the cut point is often a tag, on the same branch. Using an unrelated bracnh or tag could give some surprising result.

On Fri, 15 Mar 2019, Duy Nguyen wrote:

  this is the first time i've played with this feature, so i'm still
working my way through the man page, trying to figure out the various
valid combinations for shallow cloning.

  i notice that the SYNOPSIS for "man git-clone" does not contain all
of the supported options (eg., --shallow-exclude is missing, among
others). is that deliberate?

rday

On Fri, Mar 15, 2019 at 8:19 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:

No. It's either laziness or giving up on adding every option in the SYNOPSIS. Improvements are welcome. I can see now that --single-branch is mentioned in --depth (the original option to make a shallow clone) but not on the newer ones. My bad.

On Fri, 15 Mar 2019, Duy Nguyen wrote:

  it's fairly obvious that, when you have a git command with a
bazillion options, it's pointless to try to include everything in the
SYNOPSIS part of the man page -- best to just go with something like:

  SYNOPSIS
       git log [<options>] [<revision range>] [[--] <path>...]

what should (IMHO) be avoided are *incomplete* synopses, as those are exactly the ones that can mislead the reader into thinking that's the full set of options. "man git-clone" might be at the point of just reducing the synopsis.

  also, i think "man git-clone" could really use a set of examples for
shallow cloning. i'd offer to write it but i'm still figuring it out.

rday

On Fri, Mar 15, 2019 at 9:31 PM Robert P. J. Day <rpjday@crashcourse.ca> wrote:

Cool. I'll give you a quick introduction to help with those examples :D

Side note: there's also narrow/lazy clone (or something with the promisor) but I still haven't caught up with that. So none of that here, but I think we should eventually have some examples/description.

A shallow clone is basically a repository with commit history cut short. If you have a commit though, you have full content. It's not cut by paths or anything.

"git clone --depth=X" the the normal way to do this. You get a repository where all branches and tags have maximum X commits deep. But since the main purpose of shallow clone is redude download, --single-branch is made default when --depth is present, so you get the default branch instead of all the branches (you would need --no-single-branch for that). You could see these cut points with "git log --decorate". I think they are marked "crafted" or something.

Once you have made a shallow clone, you could do everything like usual. Local operations of course can only reach as far as the cut points. Pull and push to a full clone are possible (it's even possible to do so to another shallow clone).

When you "git fetch", the cut points remain the same, so you get more recent commits and the history depth increases. If you do "git fetch --depth=X" then the cut points are adjusted so that you only have maximum X commits deep for all select branches, the same way a shallow clone is made initially. "git fetch --unshallow" can be used to turn a shallow clone into a complete one. And I'm pretty sure you could turn a complete one to shallow with "git fetch --depth".

While --depth works fine most of the time, sometimes you want a different way to define these shallow/cut points. --shallow-since and --shallow-exclude are added as the result. --shallow-since cut the history by a specific date (for all select branches). --shallow-exclude cuts the history by a ref (often a tag). As I mentioned before, it's mostly used to say "I want a shallow clone from (but not including) v2.9.0".

And I think that's pretty much it. There's --update-shallow option but I can't remember when that's needed. Shallow clones may also prevent you from using some optional features. Pack bitmap is one of those and I think probably commit graph too. It's not technical limitation, just lack of time/effort to support them in shallow clones.

On Sat, 16 Mar 2019, Duy Nguyen wrote:

  ... snip ...

  i can see today is going to be busy ... thanks for all that, will
start reading shortly.

rday