Re: Some feedback on 'git clone create'

From: Justin Tobler <jltobler@gmail.com>
Date: Dec 5, 2024, 15:27 UTC
Message-ID: <6okfwtbgcve77xwjdqjnvsa4yigo5tymlsrypjgztbrmabjtrz@o7dfhaltih35>
In-Reply-To: <Z1Fma7OLPOkxRncd@pks.im>

On 24/12/05 09:38AM, Patrick Steinhardt wrote:

Show 23 quoted lines

> On Wed, Dec 04, 2024 at 05:19:53PM -0600, Justin Tobler wrote:
> > On 24/12/04 05:28AM, Sainan wrote:
> > > Hi, I hope this email finds you well.
> > > 
> > > I think Git bundles/packfiles are an exceptional compression format, but I find there are some rough edges with the tool to create them:
> > > 
> > > 1. There is no way to specify that you want a shallow bundle, instead you are only able to a) pack the entire tree at a given head or b) pack new/updated objects in a specified range. Anecdotally, this could store data in ~67% of the size of an equivalent .zip file.
> > 
> > You can create an incremental bundle covering a specified range.
> > Something like the following example might help you achieve what you are
> > looking for:
> > 
> >   $ git bundle create inc.bundle main~10..main
> 
> Yup. The thing that might not be immediately obvious is that
> git-bundle(1) accepts git-rev-list(1) arguments, so you can influence
> what is and isn't included via that. You can for example even generate
> partial bundles without blobs:
> 
>     $ git bundle create partial.bundle main~10..main \
>         --filter=blob:none
> 
> What you can do with the resulting bundle might be a different story.

When trying to unbundle an incremental bundle into a repository that lacks the prerequisite objects, Git fails. These prerequisite objects are also listed in the bundle header. Maybe it would be nice if we were able to create a shallow repository from this bundle.

Show 26 quoted lines

> 
> > > 2. It seems that when specifiying a commit hash, it raises an error:
> > > $ git bundle create repo.bundle $(git rev-list HEAD | head -n 1)
> > > > fatal: Refusing to create empty bundle.
> > > This confuses me slightly because I thought a commit hash should also be a valid head _pointer_. 'git rev-list' also seems to agree with me on this.
> > 
> > A bundle is essentially a pack file with a header indicating the
> > references contained within the bundle. If no reference is provided, the
> > bundle is considered empty and git refuses to create it. I think this
> > makes sense in the context of unbundling as you probably would not want
> > to add new objects without updating references in the target repository.
> > 
> > From the git-bundle(1) docs for "create", the usage does say it accepts
> > <git-rev-list-args> which may be a bit misleading because
> > git-rev-list(1) does consider the commit hash as valid. Maybe that
> > should be updated to indicate that proper references are expected.
> 
> That's somewhat weird indeed. I don't see a strong reason why the first
> of the following commands works while the second one doesn't:
> 
>     $ git bundle create inc.bundle master~..master
>     $ git bundle create inc.bundle $(git rev-parse master~)..$(git rev-parse master)
> 
> It's not like the bundle has "master" in its header in the first command
> anyway, it only lists HEAD in there. So I'd claim that we could do the
> same for the second command, as well.

I'm not quite sure I follow. According to gitformat-bundle(5), we should see "obj-id SP refname LF" in the header. Inspecting the header of a bundle created from `git bundle create inc.bundle master~..master` also shows "refs/heads/master" in the header.

Looking at git-bundle(1) docs more closely it also does mention this limitation:

       A revision name or a range whose right-hand-side cannot be
       resolved to a reference is not accepted:

           $ git bundle create HEAD.bundle $(git rev-parse HEAD)
           fatal: Refusing to create empty bundle.
           $ git bundle create master-yesterday.bundle master~10..master~5
           fatal: Refusing to create empty bundle.

It looks like when creating a bundle in `bundle.c:create_bundle()`, if the call to `write_bundle_refs()` returns a reference count of 0, git dies with the error seen. When a commit hash is used for the rev-list-arg, it is not able to determine a reference from it.

-Justin