threads / patch / 51077

patchdoc: explain why file: URLs and bundles don't mix

Subject: [PATCH] doc: explain why file: URLs and bundles don't mix

## tl;dr

5 messages between May 13, 2019 and May 14, 2019. Diffs are folded; open one to read it.

replies: 4people: 3as markdown or json

Alyssa Ross· May 13, 2019, 00:23 UTC · lore

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
 Documentation/urls.txt | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

Show changes to Documentation/urls.txt +15 −4

diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index b05da95788..c83d9f859e 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -43,14 +43,24 @@ syntaxes may be used:
 - \file:///path/to/repo.git/
 
 ifndef::git-clone[]
-These two syntaxes are mostly equivalent, except when cloning, when
-the former implies --local option. See linkgit:git-clone[1] for
-details.
+These two syntaxes are mostly equivalent, with some exceptions:
+
+- When cloning, the former implies --local option. See
+  linkgit:git-clone[1] for details.
+
+- The latter is implemented using linkgit:git-upload-pack[1], which
+  expects its repository to be a directory, and therefore does not
+  work for bundles (see linkgit:git-bundle[1]).
 endif::git-clone[]
 
 ifdef::git-clone[]
-These two syntaxes are mostly equivalent, except the former implies
---local option.
+These two syntaxes are mostly equivalent, with some exceptions:
+
+- The former implies --local option.
+
+- The latter is implemented using linkgit:git-upload-pack[1], which
+  expects its repository to be a directory, and therefore does not
+  work for bundles (see linkgit:git-bundle[1]).
 endif::git-clone[]
 
 When Git doesn't know how to handle a certain transport protocol, it

-- 
2.21.0

Junio C Hamano· May 13, 2019, 08:05 UTC · re: Alyssa Ross · lore

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

Alyssa Ross <hi@alyssa.is> writes:

Show 24 quoted lines

> Signed-off-by: Alyssa Ross <hi@alyssa.is>
> ---
>  Documentation/urls.txt | 20 +++++++++++++++-----
>  1 file changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/urls.txt b/Documentation/urls.txt
> index b05da95788..c83d9f859e 100644
> --- a/Documentation/urls.txt
> +++ b/Documentation/urls.txt
> @@ -43,14 +43,24 @@ syntaxes may be used:
>  - \file:///path/to/repo.git/
>  
>  ifndef::git-clone[]
> -These two syntaxes are mostly equivalent, except when cloning, when
> -the former implies --local option. See linkgit:git-clone[1] for
> -details.
> +These two syntaxes are mostly equivalent, with some exceptions:
> +
> +- When cloning, the former implies --local option. See
> +  linkgit:git-clone[1] for details.
> +
> +- The latter is implemented using linkgit:git-upload-pack[1], which
> +  expects its repository to be a directory, and therefore does not
> +  work for bundles (see linkgit:git-bundle[1]).

Hmm, I do not think this is quite true.

If "git clone /path/to/repo.bndl" implied --local, we would end up trying to hardlink into /path/to/repo.bndl/objects and would fail.

I think what is closer to the reaility is that we check if the source is a bundle when the local filesystem path is used and try to clone from the bundle, before using the local filesystem path as a directory we can "clone --local" from. On the other hand, when the <scheme>://<path> syntax is used, we do not even bother seeing if the named resource is a bundle, or if --local optimization is possible (because we do not bother seeing if the named resource is a local filesystem entity, either).

A possibly interesting tangent to think about is what would happen if we slightly tweak the above design. What it would require for the code to take "git clone https://site/repo.bndl", realize that the named resource is a bundle file, curl/wget it and clone from that downloaded bundle? And if it is feasible to implement, would it even be a good idea to begin with? I do not have a ready answer to either of these questions myself.

Alyssa Ross· May 13, 2019, 16:54 UTC · re: Junio C Hamano · lore

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

Show 20 quoted lines

> > +- When cloning, the former implies --local option. See
> > +  linkgit:git-clone[1] for details.
> > +
> > +- The latter is implemented using linkgit:git-upload-pack[1], which
> > +  expects its repository to be a directory, and therefore does not
> > +  work for bundles (see linkgit:git-bundle[1]).
>
> Hmm, I do not think this is quite true.
>
> If "git clone /path/to/repo.bndl" implied --local, we would end up
> trying to hardlink into /path/to/repo.bndl/objects and would fail.
>
> I think what is closer to the reaility is that we check if the
> source is a bundle when the local filesystem path is used and try to
> clone from the bundle, before using the local filesystem path as a
> directory we can "clone --local" from.  On the other hand, when the
> <scheme>://<path> syntax is used, we do not even bother seeing if
> the named resource is a bundle, or if --local optimization is
> possible (because we do not bother seeing if the named resource is a
> local filesystem entity, either).

Would the following work better for you?

When cloning, the former will check to see if the source is a bundle. If it is, it will clone from the bundle, otherwise it will behave as if given --local. The latter performs neither of these checks, and therefore does not support bundles.

Show 7 quoted lines

> A possibly interesting tangent to think about is what would happen
> if we slightly tweak the above design.  What it would require for
> the code to take "git clone https://site/repo.bndl", realize that
> the named resource is a bundle file, curl/wget it and clone from
> that downloaded bundle?  And if it is feasible to implement, would
> it even be a good idea to begin with?  I do not have a ready answer
> to either of these questions myself.

I was very surprised to find that cloning from a bundle didn't even work with a file URL, especially since the documentation seemed to imply that it should. I assumed that file URLs not supporting bundles was an oversight, and made some attempt to read the code with a view to fixing it. Once I realised that file URLs went through a completely different code path to paths, I decided it was better to just document things as they were.

I think it would be nice if, at least for file URLs, bundles were supported. I doubt supporting cloning a bundle over a network would inherently cause problems (although I'm by no means qualified to make such an assessment) -- if it's not clear whether it's a good idea to do so, it could always be implemented without being advertised.

Jeff King· May 14, 2019, 09:30 UTC · re: Alyssa Ross · lore

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

On Mon, May 13, 2019 at 04:54:55PM +0000, Alyssa Ross wrote:

Show 15 quoted lines

> > I think what is closer to the reaility is that we check if the
> > source is a bundle when the local filesystem path is used and try to
> > clone from the bundle, before using the local filesystem path as a
> > directory we can "clone --local" from.  On the other hand, when the
> > <scheme>://<path> syntax is used, we do not even bother seeing if
> > the named resource is a bundle, or if --local optimization is
> > possible (because we do not bother seeing if the named resource is a
> > local filesystem entity, either).
> 
> Would the following work better for you?
> 
> When cloning, the former will check to see if the source is a bundle.
> If it is, it will clone from the bundle, otherwise it will behave as if
> given --local. The latter performs neither of these checks, and
> therefore does not support bundles.

FWIW, I like that much better than the original proposal. I think it would also be OK to say something like: ...does not support bundles (but may in the future).

-Peff

Jeff King· May 14, 2019, 09:29 UTC · re: Junio C Hamano · lore

Re: [PATCH] doc: explain why file: URLs and bundles don't mix

On Mon, May 13, 2019 at 05:05:14PM +0900, Junio C Hamano wrote:

Show 8 quoted lines

> I think what is closer to the reaility is that we check if the
> source is a bundle when the local filesystem path is used and try to
> clone from the bundle, before using the local filesystem path as a
> directory we can "clone --local" from.  On the other hand, when the
> <scheme>://<path> syntax is used, we do not even bother seeing if
> the named resource is a bundle, or if --local optimization is
> possible (because we do not bother seeing if the named resource is a
> local filesystem entity, either).

Yeah. My concern on reading Alyssa's patch is that it goes into too much detail about "why", when this is basically just "it's a bug that nobody has yet fixed". I think it makes sense to warn people about the behavior, but we should probably be vague about the reasons and up-front that there is really no good reason that it doesn't work (and it might in the future).

Of course fixing the issue is better still. ;)

Show 7 quoted lines

> A possibly interesting tangent to think about is what would happen
> if we slightly tweak the above design.  What it would require for
> the code to take "git clone https://site/repo.bndl", realize that
> the named resource is a bundle file, curl/wget it and clone from
> that downloaded bundle?  And if it is feasible to implement, would
> it even be a good idea to begin with?  I do not have a ready answer
> to either of these questions myself.

I've been meaning to come back to this for, oh, going on 8 years now:

  https://public-inbox.org/git/20111110074330.GA27925@sigill.intra.peff.net/

(though it was meant to work with a CDN "try this first then top-off" strategy; with Jonathan Tan's more flexible CDN work which uses raw packfiles, I doubt I'd come back to this).

-Peff