threads / discuss / 57429

Supporting --depth when cloning from a bundle

Subject: Supporting --depth when cloning from a bundle

## tl;dr

4 messages between Feb 16, 2022 and Feb 17, 2022.

replies: 3people: 2as markdown or json

Konstantin Ryabitsev· Feb 16, 2022, 22:07 UTC · lore

Hello, all:

It would be pretty handy if it was possible to do shallow clones from a bundle file. For example, this would be nice for CI environments that first download a clone.bundle file from a locally cached location, clone from it, and then fetch any remaining objects from remote.

E.g., in my mind this outta work:

$ wget https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/clone.bundle $ git clone --depth 1 clone.bundle linux -b master

Right now, this operation succeeds as-is without any errors, but the resulting git repository isn't actually shallow.

Oh, it would be even awesomer if the bundle files could be used directly with --reference, e.g.:

$ git clone --reference clone.bundle --dissociate \
  https://git.kernel.org/.../torvalds/linux.git linux

Any reason that can't work?

-K

Junio C Hamano· Feb 16, 2022, 22:37 UTC · re: Konstantin Ryabitsev · lore

Re: Supporting --depth when cloning from a bundle

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

Show 22 quoted lines

> Hello, all:
>
> It would be pretty handy if it was possible to do shallow clones from a bundle
> file. For example, this would be nice for CI environments that first download
> a clone.bundle file from a locally cached location, clone from it, and then
> fetch any remaining objects from remote.
>
> E.g., in my mind this outta work:
>
> $ wget https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/clone.bundle
> $ git clone --depth 1 clone.bundle linux -b master
>
> Right now, this operation succeeds as-is without any errors, but the resulting
> git repository isn't actually shallow.
>
> Oh, it would be even awesomer if the bundle files could be used directly with
> --reference, e.g.:
>
> $ git clone --reference clone.bundle --dissociate \
>   https://git.kernel.org/.../torvalds/linux.git linux
>
> Any reason that can't work?

The "depth" is used to negitiate what objects to include in the packfile that is transmit from the other side. That would not fundamentally work on bundles, because a bundle is pregenerated and there is nobody on the other side for you to negotiate with to affect what objects are included in the packfile which is contained in the bundle file.

You _should_ be able to make the resulting full clone into a shallow one, though.

Konstantin Ryabitsev· Feb 17, 2022, 14:19 UTC · re: Junio C Hamano · lore

Re: Supporting --depth when cloning from a bundle

On Wed, Feb 16, 2022 at 02:37:20PM -0800, Junio C Hamano wrote:

Show 6 quoted lines

> The "depth" is used to negitiate what objects to include in the
> packfile that is transmit from the other side.  That would not
> fundamentally work on bundles, because a bundle is pregenerated and
> there is nobody on the other side for you to negotiate with to
> affect what objects are included in the packfile which is contained
> in the bundle file.

Junio:

Sorry for the potentially stupid follow-up question. I know that git makes a distinction when cloning locally between a direct path and a file:// URL (e.g. it's only possible to do a shallow clone from a local repository when using the file:// scheme). When that happens, doesn't that same negotiation happen locally (perhaps in a simulated way)? If that so, then theoretically the same approach could be taken with the bundle file?

(I'm not actually that interested in that feature, to be honest -- I'd much rather see further work on git-native CDN offload of packfiles, so we stop using kludges like bundle files for this purpose.)

-K

Junio C Hamano· Feb 17, 2022, 21:05 UTC · re: Konstantin Ryabitsev · lore

Re: Supporting --depth when cloning from a bundle

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> Sorry for the potentially stupid follow-up question. I know that git makes a
> distinction when cloning locally between a direct path and a file:// URL (e.g.
> it's only possible to do a shallow clone from a local repository when using
> the file:// scheme).

The local optimization uses hardlinks into an existing repository bypassing most of what makes Git Git, and there is no negotiation.

"clone --no-local $PATH" or "clone file://$PATH" runs the "server side" inside $PATH and talks with it, which is ONLY possible because $PATH is a real repository. A bundle file is not a repository, and nobody has written the "server side" to workin inside it, so...