threads / discuss / 53884

Pushing tag from a partial clone

Subject: Pushing tag from a partial clone

## tl;dr

4 messages between Jul 20, 2020 and Jul 20, 2020.

replies: 3people: 3as markdown or json

Son Luong Ngoc· Jul 20, 2020, 11:44 UTC · lore

Hi folks,

At $day_job, we are trying to push tags to a repo from a partial clone copy. However it seems like this push would requires the partial clone copy to download more objects? Is this intended?

Reproduce:

	mkdir repo && cd repo
	git init
	git remote add origin git@domain.com:path/repo.git
	git fetch --filter=tree:0 --no-tags --prune origin <commit-id>
	git sparse-checkout init --cone
	git checkout --force <commit-id>
	git tag -a sluongng-test -m "Test push from partial clone"
	git push HEAD:refs/tags/sluongng-test
	<git starts to download objects>

Ideally we would like to be able to push tag from a shallow + partial clone repo without 
having to download extra objects if possible.
We would like to keep the required repo to the absolute minimum.
	git fetch --depth 1 --filter=tree:0 --no-tags --prune origin <commit-id>

Creating and pushing tags should not require local repo to have trees/blobs in it?

Git version: 2.27.0

Cheers, Son Luong.

Derrick Stolee· Jul 20, 2020, 12:18 UTC · re: Son Luong Ngoc · lore

Re: Pushing tag from a partial clone

On 7/20/2020 7:44 AM, Son Luong Ngoc wrote:

Show 26 quoted lines

> Hi folks,
> 
> At $day_job, we are trying to push tags to a repo from a partial clone copy.
> However it seems like this push would requires the partial clone copy to download more objects?
> Is this intended?
> 
> Reproduce:
> 
> 	mkdir repo && cd repo
> 	git init
> 	git remote add origin git@domain.com:path/repo.git
> 	git fetch --filter=tree:0 --no-tags --prune origin <commit-id>
> 	git sparse-checkout init --cone
> 	git checkout --force <commit-id>
> 	git tag -a sluongng-test -m "Test push from partial clone"
> 	git push HEAD:refs/tags/sluongng-test
> 	<git starts to download objects>
> 
> Ideally we would like to be able to push tag from a shallow + partial clone repo without 
> having to download extra objects if possible.
> We would like to keep the required repo to the absolute minimum.
> 	git fetch --depth 1 --filter=tree:0 --no-tags --prune origin <commit-id>
> 
> Creating and pushing tags should not require local repo to have trees/blobs in it?
> 
> Git version: 2.27.0

Could you try this again with 2.28.0-rc1? I think Jonathan Tan added the "no-fetch" flag in more places since 2.27.0, and this might already be fixed.

Thanks, -Stolee

Son Luong Ngoc· Jul 20, 2020, 13:47 UTC · re: Derrick Stolee · lore

Re: Pushing tag from a partial clone

Hi,

Show 33 quoted lines

> On Jul 20, 2020, at 14:18, Derrick Stolee <stolee@gmail.com> wrote:
> 
> On 7/20/2020 7:44 AM, Son Luong Ngoc wrote:
>> Hi folks,
>> 
>> At $day_job, we are trying to push tags to a repo from a partial clone copy.
>> However it seems like this push would requires the partial clone copy to download more objects?
>> Is this intended?
>> 
>> Reproduce:
>> 
>> 	mkdir repo && cd repo
>> 	git init
>> 	git remote add origin git@domain.com:path/repo.git
>> 	git fetch --filter=tree:0 --no-tags --prune origin <commit-id>
>> 	git sparse-checkout init --cone
>> 	git checkout --force <commit-id>
>> 	git tag -a sluongng-test -m "Test push from partial clone"
>> 	git push HEAD:refs/tags/sluongng-test
>> 	<git starts to download objects>
>> 
>> Ideally we would like to be able to push tag from a shallow + partial clone repo without 
>> having to download extra objects if possible.
>> We would like to keep the required repo to the absolute minimum.
>> 	git fetch --depth 1 --filter=tree:0 --no-tags --prune origin <commit-id>
>> 
>> Creating and pushing tags should not require local repo to have trees/blobs in it?
>> 
>> Git version: 2.27.0
> 
> Could you try this again with 2.28.0-rc1? I think Jonathan
> Tan added the "no-fetch" flag in more places since 2.27.0,
> and this might already be fixed.

I just freshly compiled from 'next' branch:

	> git version
	git version 2.28.0.rc1.139.gd6b33fda9d

And the problem still occurring:
	> mkdir scalar
	> cd scalar
	> git init
	Initialized empty Git repository in /Users/sluongngoc/work/booking/core/scalar/.git/
	# use my own fork here so that i have push permission
	> git remote add origin git@github.com:sluongng/scalar.git
	> git sparse-checkout init --cone
	> git fetch --filter=tree:0 --no-tags --prune origin 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
	remote: Enumerating objects: 2553, done.
	remote: Total 2553 (delta 0), reused 0 (delta 0), pack-reused 2553
	Receiving objects: 100% (2553/2553), 957.85 KiB | 1.06 MiB/s, done.
	Resolving deltas: 100% (74/74), done.
	From github.com:sluongng/scalar
	 * branch            4ba6c1c090e6e5a413e3ac2fc094205bd78f761e -> FETCH_HEAD
	> git tag -a test-tag -m 'test tag message' 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
	> git push origin refs/tags/test-tag:refs/tags/test-tag
	...<download start>

> 
> Thanks,
> -Stolee
>

Thanks, Son Luong.

Jonathan Tan· Jul 20, 2020, 17:54 UTC · re: Son Luong Ngoc · lore

Re: Pushing tag from a partial clone

Show 23 quoted lines

> I just freshly compiled from 'next' branch:
> 
> 	> git version
> 	git version 2.28.0.rc1.139.gd6b33fda9d
> 
> And the problem still occurring:
> 	> mkdir scalar
> 	> cd scalar
> 	> git init
> 	Initialized empty Git repository in /Users/sluongngoc/work/booking/core/scalar/.git/
> 	# use my own fork here so that i have push permission
> 	> git remote add origin git@github.com:sluongng/scalar.git
> 	> git sparse-checkout init --cone
> 	> git fetch --filter=tree:0 --no-tags --prune origin 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
> 	remote: Enumerating objects: 2553, done.
> 	remote: Total 2553 (delta 0), reused 0 (delta 0), pack-reused 2553
> 	Receiving objects: 100% (2553/2553), 957.85 KiB | 1.06 MiB/s, done.
> 	Resolving deltas: 100% (74/74), done.
> 	From github.com:sluongng/scalar
> 	 * branch            4ba6c1c090e6e5a413e3ac2fc094205bd78f761e -> FETCH_HEAD
> 	> git tag -a test-tag -m 'test tag message' 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
> 	> git push origin refs/tags/test-tag:refs/tags/test-tag
> 	...<download start>

Thanks for the reproduction steps. Is 4ba6c1c advertised as a ref by the remote? If not, what is probably happening is that the client doesn't realize that the server already has 4ba6c1c, so the client needs to fetch 4ba6c1c's objects to send it to the server.

I am planning to see if I can add batch prefetching to pack-objects to reduce the severity of similar situations (just one batch prefetch instead of many one-by-one fetches), although that would work better with a blob filter instead of a tree filter.