threads / discuss / 5644

git-push of new, tiny branch doing more work than necessary?

Subject: git-push of new, tiny branch doing more work than necessary?

## tl;dr

5 messages between Sep 21, 2006 and Sep 28, 2006.

replies: 4people: 4as markdown or json

Carl Worth· Sep 21, 2006, 22:29 UTC · lore

I've run through something quite like the following scenario a few times lately:

	git pull origin
	git checkout -b experiment
	# hack a file or two
	git commit -a -m "new experiment"
	git push origin experiment

What I expect at this point is for git to push the few newly created objects out to the repository. Instead it talks about generating, deltifying, transferring, and unpacking thousands of objects (see below).

Shouldn't the same WANT/HAVE hand shaking that makes git-fetch efficient be doing something similar here? Or am I just doing something wrong?

-Carl
$ git --version
git version 1.4.2.rc2.gef1d9
$ git push cworth 8379
updating 'refs/heads/8379'
  from 0000000000000000000000000000000000000000
  to   44379202da96faf632b73cdcdd6c0a74fb4b54f4
Generating pack...
Done counting 4900 objects.
Result has 4622 objects.
Deltifying 4622 objects.
 100% (4622/4622) done
Unpacking 4622 objects
Total 4622, written 4622 (delta 3411), reused 1259 (delta 946)
refs/heads/8379: 0000000000000000000000000000000000000000 ->
44379202da96faf632b73cdcdd6c0a74fb4b54f4
Johannes Schindelin· Sep 21, 2006, 22:34 UTC · re: Carl Worth · lore

Re: git-push of new, tiny branch doing more work than necessary?

Hi,
On Thu, 21 Sep 2006, Carl Worth wrote:
Show 13 quoted lines
> I've run through something quite like the following scenario a few
> times lately:
> 
> 	git pull origin
> 	git checkout -b experiment
> 	# hack a file or two
> 	git commit -a -m "new experiment"
> 	git push origin experiment
> 
> What I expect at this point is for git to push the few newly created
> objects out to the repository. Instead it talks about generating,
> deltifying, transferring, and unpacking thousands of objects (see
> below).

I experienced something like this, too, but did not have the time to debug it. But alas, some time (I think last week) it started working as expected again.

> git version 1.4.2.rc2.gef1d9
Please try a more recent version (yours is from Jul 27).

Ciao, Dscho

Andy Whitcroft· Sep 22, 2006, 07:40 UTC · re: Johannes Schindelin · lore

Re: git-push of new, tiny branch doing more work than necessary?

Johannes Schindelin wrote:
Show 21 quoted lines
> Hi,
> 
> On Thu, 21 Sep 2006, Carl Worth wrote:
> 
>> I've run through something quite like the following scenario a few
>> times lately:
>>
>> 	git pull origin
>> 	git checkout -b experiment
>> 	# hack a file or two
>> 	git commit -a -m "new experiment"
>> 	git push origin experiment
>>
>> What I expect at this point is for git to push the few newly created
>> objects out to the repository. Instead it talks about generating,
>> deltifying, transferring, and unpacking thousands of objects (see
>> below).
> 
> I experienced something like this, too, but did not have the time to debug 
> it. But alas, some time (I think last week) it started working as expected 
> again.

I hope this started working again because we fixed it this time. Two bugs were fixed in this area. First any branch names over about 40 characters in length were being ignored completely due to a validation bug. Second in an attempt to ensure we didn't bust the 900 entry command line limit for rev-list we were restricting ourselves to 16 remote refs as possible bases. master now has these changes in it and I am no longer seeing this behavior.

> 
>> git version 1.4.2.rc2.gef1d9
> 
> Please try a more recent version (yours is from Jul 27).
-apw
Carl Worth· Sep 28, 2006, 18:10 UTC · re: Andy Whitcroft · lore

Re: git-push of new, tiny branch doing more work than necessary?

On Fri, 22 Sep 2006 08:40:41 +0100, Andy Whitcroft wrote:
Show 7 quoted lines
> I hope this started working again because we fixed it this time.  Two
> bugs were fixed in this area.  First any branch names over about 40
> characters in length were being ignored completely due to a validation
> bug.  Second in an attempt to ensure we didn't bust the 900 entry
> command line limit for rev-list we were restricting ourselves to 16
> remote refs as possible bases.  master now has these changes in it and I
> am no longer seeing this behavior.
Aha! Thanks.

It doesn't appear I had the 40-character ref problem, nor was I exceeding the 900 ref limit. But I was definitely well over the 16 ref limit.

> >> git version 1.4.2.rc2.gef1d9
> >
> > Please try a more recent version (yours is from Jul 27).

I updated as soon as I got your reply, but just waited until I hit a repeat of the use case today, (I was just too lazy to try to roll things back and duplicate the exact same conditions).

Sure enough, things seem to be much better now (git version 1.4.2.1.g16fd), so thanks for the fix.

-Carl
Junio C Hamano· Sep 22, 2006, 02:31 UTC · re: Carl Worth · lore

Re: git-push of new, tiny branch doing more work than necessary?

Carl Worth <cworth@cworth.org> writes:
Show 17 quoted lines
> I've run through something quite like the following scenario a few
> times lately:
>
> 	git pull origin
> 	git checkout -b experiment
> 	# hack a file or two
> 	git commit -a -m "new experiment"
> 	git push origin experiment
>
> What I expect at this point is for git to push the few newly created
> objects out to the repository. Instead it talks about generating,
> deltifying, transferring, and unpacking thousands of objects (see
> below).
>
> Shouldn't the same WANT/HAVE hand shaking that makes git-fetch
> efficient be doing something similar here? Or am I just doing
> something wrong?

Depending on the version of git you are running, there are two possibilities.

There has been a change to send-pack last week pushed out to "master", in which Andy Whitcroft fixed a problem when there are tons of refs on either end. Earlier we were limiting ourselves to use 900 or so refs when generating the pack, but the new code uses all refs it knows about (so it excludes commits pointed by all remote refs, not just the ones that fits in the arbitrary 900 limit).

If you are seeing problem in a pair of repository with tons of refs, it may be the problem Andy fixed with the above change, so using the recent "master" version would help you.

If you are on the other han seeing the problem to start after installing the recent "master", it could be that the above change broke things for you. If you do not see the problem with v1.4.2.1 but do see the problem with the current "master" that would be an indication this is the case.

Another thing to watch out for is that send-pack/receive-pack protocol does NOT do the full "want/have" conversation like fetch-pack/upload-pack protocol does. It merely say "receiving end has these at tips of branches and tags", "sending side has these at tips of branches and tags", and does not ask about what's befind the tips. So if you are in this situation, with or without Andy's fix I can imagine you would see some redundant sending of things.

 - The origin repository has an ancient tag and the master
   branch.  Nothing else.
 - Clone origin to victim repository. The victim repository
   would now have origin and master (the same), and the tag.
 - Somebody creates a commit in the "origin" repository on the
   master branch.
 - You create a commit on the "master" in the victim
   repository.
 - You run send-pack in the victim repository and try to push
   your master to the master in the origin repository.

In this case, the update is not a fast forward, so you may need to force it. When you do force, send-pack running on your end learns from the other end about the ancient tag and its "master", the latter of which it does not know about at all (because you haven't pulled from the origin). So all it has to work with as the common commit is the ancient tag -- which means you would end up sending everything since that ancient tag til your tip of "master".

I do not know if that is what is happening to you in this case, though. While I think not doing the discovery harder like upload-pack is a bug in send-pack, it is not an issue when you are not forcing a non-fast forward, so it probably is a lower priority bug than other things. To fix it properly I think we would need a minor protocol update.

← back to recent threads