threads / discuss / 7524

How can git pull be up-to-date and git push fail?

Subject: How can git pull be up-to-date and git push fail?

## tl;dr

13 messages between Apr 5, 2007 and Apr 7, 2007.

replies: 12people: 3as markdown or json

Bill Lear· Apr 5, 2007, 12:15 UTC · lore

I'm trying to explain this behavior to a co-worker, and how to solve it:

% git pull Already up-to-date.

% git push error: remote 'refs/remotes/origin/flexify' is not a strict subset of localref 'refs/remotes/origin/flexify'. maybe you are not up-to-date and need to pull first?

I believe he is using git 1.5.0 and that the above was tried from the master branch.

So, I'm confused: I would have thought the way you solve the problem in step 2 is by executing step 1. Would he need to do this:

% git checkout flexify % git pull flexify:flexify % git checkout master % git push

Bill
Jeff King· Apr 5, 2007, 13:49 UTC · re: Bill Lear · lore

Re: How can git pull be up-to-date and git push fail?

On Thu, Apr 05, 2007 at 07:15:54AM -0500, Bill Lear wrote:
Show 10 quoted lines
> I'm trying to explain this behavior to a co-worker, and how to solve
> it:
> 
> % git pull
> Already up-to-date.
> 
> % git push
> error: remote 'refs/remotes/origin/flexify' is not a strict subset of
> localref 'refs/remotes/origin/flexify'. maybe you are not up-to-date
> and need to pull first?

That ref name is a bit suspect...why are you pushing your your remote tracking branches? If you are simply pushing to a repository that is a pure mirror, maybe your 'push' line in the config file should have a '+' at the front?

At any rate, the reason why the pull doesn't fix it is that it is not pulling onto flexify, but onto master, which it looks like you guessed:

Show 7 quoted lines
> So, I'm confused: I would have thought the way you solve the problem
> in step 2 is by executing step 1.  Would he need to do this:
> 
> % git checkout flexify
> % git pull flexify:flexify
> % git checkout master
> % git push
It would need to be:
  git pull $remote flexify:flexify

I don't think that works, though, because it's _not_ the flexify branch that's the problem. It's the refs/remotes/origin/flexify, which generally shouldn't be getting pushed at all. Can you show us the .git/config?

-Peff
Bill Lear· Apr 5, 2007, 14:07 UTC · re: Jeff King · lore

Re: How can git pull be up-to-date and git push fail?

On Thursday, April 5, 2007 at 09:49:54 (-0400) Jeff King writes:
Show 36 quoted lines
>On Thu, Apr 05, 2007 at 07:15:54AM -0500, Bill Lear wrote:
>
>> I'm trying to explain this behavior to a co-worker, and how to solve
>> it:
>> 
>> % git pull
>> Already up-to-date.
>> 
>> % git push
>> error: remote 'refs/remotes/origin/flexify' is not a strict subset of
>> localref 'refs/remotes/origin/flexify'. maybe you are not up-to-date
>> and need to pull first?
>
>That ref name is a bit suspect...why are you pushing your your remote
>tracking branches? If you are simply pushing to a repository that is a
>pure mirror, maybe your 'push' line in the config file should have a '+'
>at the front?
>
>At any rate, the reason why the pull doesn't fix it is that it is not
>pulling onto flexify, but onto master, which it looks like you guessed:
>
>> So, I'm confused: I would have thought the way you solve the problem
>> in step 2 is by executing step 1.  Would he need to do this:
>> 
>> % git checkout flexify
>> % git pull flexify:flexify
>> % git checkout master
>> % git push
>
>It would need to be:
>  git pull $remote flexify:flexify
>
>I don't think that works, though, because it's _not_ the flexify branch
>that's the problem. It's the refs/remotes/origin/flexify, which
>generally shouldn't be getting pushed at all. Can you show us the
>.git/config?
Here is the config for the remote repository:
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
[remote "origin"]
        url = git://source/fusion
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master
Here is the local:
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = ssh://poire/home/jml/repos/new/fusion
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

Hmm.... I wonder if his remote repo is non-bare... I'll try to find out.

Bill
Bill Lear· Apr 5, 2007, 14:25 UTC · re: Bill Lear · lore

Re: How can git pull be up-to-date and git push fail?

On Thursday, April 5, 2007 at 09:07:11 (-0500) Bill Lear writes:
>...
>Hmm.... I wonder if his remote repo is non-bare... I'll try to find
>out.
Yes, it is.  Here is the reply I got from my friend:
Show 18 quoted lines
>Bill Lear writes:
> > So, the remote repo to which you pushed originally is non-bare?  That
> > is, it has working files checked out?
>
>Correct.
>
> > If so, that is a problem, but
> > perhaps not the cause of your problems.  Never push to a non-bare
> > repo: you need to go that repo and do a pull instead.
>
>I've been able to work around such problems.  I know well about the
>double-commit reversal whammy that can happen (since it happened to me).
>If I push to a non-bare repo that's otherwise got no file modifications
>in it, I can do a "git status" to see what files it thinks are out of
>date, which tells me (by the file list) that the push succeeded.  Then
>I just do a "git checkout -f" to sync up.  Up til now, has worked like
>a charm.  Alternately, I switch to some bogus temporary branch in the
>non-bare repo I'm pushing to & switch back after.
Bill
Jeff King· Apr 5, 2007, 20:46 UTC · re: Bill Lear · lore

Re: How can git pull be up-to-date and git push fail?

On Thu, Apr 05, 2007 at 09:07:11AM -0500, Bill Lear wrote:
Show 13 quoted lines
> Here is the local:
> 
> [core]
>         repositoryformatversion = 0
>         filemode = true
>         bare = false
>         logallrefupdates = true
> [remote "origin"]
>         url = ssh://poire/home/jml/repos/new/fusion
>         fetch = +refs/heads/*:refs/remotes/origin/*
> [branch "master"]
>         remote = origin
>         merge = refs/heads/master

I don't see anything there that should cause branches under refs/remotes to be pushed. Was he using 'git-push --all' by any chance?

-Peff
Bill Lear· Apr 5, 2007, 20:55 UTC · re: Jeff King · lore

Re: How can git pull be up-to-date and git push fail?

On Thursday, April 5, 2007 at 16:46:04 (-0400) Jeff King writes:
Show 18 quoted lines
>On Thu, Apr 05, 2007 at 09:07:11AM -0500, Bill Lear wrote:
>
>> Here is the local:
>> 
>> [core]
>>         repositoryformatversion = 0
>>         filemode = true
>>         bare = false
>>         logallrefupdates = true
>> [remote "origin"]
>>         url = ssh://poire/home/jml/repos/new/fusion
>>         fetch = +refs/heads/*:refs/remotes/origin/*
>> [branch "master"]
>>         remote = origin
>>         merge = refs/heads/master
>
>I don't see anything there that should cause branches under refs/remotes
>to be pushed. Was he using 'git-push --all' by any chance?

No, just plain ol' git push. One possibility is that in mediating between him and the list, there has been a loss of information. Perhaps he omitted details in his account of how things progressed --- who knows, perhaps he was on the wrong branch (though I doubt this, he only works on the master branch, and the other branch in question was someone else's entirely).

Regardless, does my assumption --- a 'git pull' should rectify the 'you are not up to date' problem --- hold in general?

Thanks for helping.
Bill
Jeff King· Apr 7, 2007, 16:16 UTC · re: Bill Lear · lore

Re: How can git pull be up-to-date and git push fail?

On Thu, Apr 05, 2007 at 03:55:18PM -0500, Bill Lear wrote:
Show 6 quoted lines
> No, just plain ol' git push.  One possibility is that in mediating
> between him and the list, there has been a loss of information.
> Perhaps he omitted details in his account of how things progressed ---
> who knows, perhaps he was on the wrong branch (though I doubt this, he
> only works on the master branch, and the other branch in question was
> someone else's entirely).

No, I think you gave the correct information. See the other part of the thread between me and Junio. What's happening is that 'git-push' tries to push every ref that both you and the remote have. Because he's pushing into a non-bare repository, both sides have refs/remotes/origin/flexify. But they're not necessarily at the same point, since they're both fetched from whatever your origin is, thus the problem. In this case, his flexify tracking branch for origin is _behind_ the remote's, so the push doesn's work.

> Regardless, does my assumption --- a 'git pull' should rectify
> the 'you are not up to date' problem --- hold in general?

Generally, yes. But the remotes/ hierarchy is not something you pull into. So in this case, it won't help. The real solution is that git-push should not be pushing remotes/ at all: it should be purely a local thing (the patch I posted elsewhere in the thread fixes this, but will not be the final behavior -- however, it's enough to fix this problem).

As a temporary solution, fetching from origin should put him at the same place (or ahead of) the remote, which should allow the push to go through.

-Peff
Junio C Hamano· Apr 5, 2007, 21:18 UTC · re: Jeff King · lore

Re: How can git pull be up-to-date and git push fail?

Jeff King <peff@peff.net> writes:
Show 18 quoted lines
> On Thu, Apr 05, 2007 at 09:07:11AM -0500, Bill Lear wrote:
>
>> Here is the local:
>> 
>> [core]
>>         repositoryformatversion = 0
>>         filemode = true
>>         bare = false
>>         logallrefupdates = true
>> [remote "origin"]
>>         url = ssh://poire/home/jml/repos/new/fusion
>>         fetch = +refs/heads/*:refs/remotes/origin/*
>> [branch "master"]
>>         remote = origin
>>         merge = refs/heads/master
>
> I don't see anything there that should cause branches under refs/remotes
> to be pushed. Was he using 'git-push --all' by any chance?

IIRC "git push" without explicit refspecs push the matching refs, but I am a bit under the weather and feverish, so don't take my word literally but look at git-push manual page please.

Jeff King· Apr 6, 2007, 02:52 UTC · re: Junio C Hamano · lore

Re: How can git pull be up-to-date and git push fail?

On Thu, Apr 05, 2007 at 02:18:58PM -0700, Junio C Hamano wrote:
> IIRC "git push" without explicit refspecs push the matching
> refs, but I am a bit under the weather and feverish, so don't
> take my word literally but look at git-push manual page please.

Ah, yes you're right. It really doesn't make sense to push refs/remotes/* in most cases, since they're just tracking branches, and if the destination _does_ have them, then it is unlikely to be in sync with you (leading to Bill's problem). OTOH, you might want to be able to push them explicitly if you are doing a strict mirror of a repository.

The patch below turns off refs/remotes/ sending for "git-push" and "git-push --all", but still allows "git-push origin remotes/origin/master". I'm not sure about the semantics; maybe --all should imply even remotes?

It also only impacts send-pack; I suspect pushing over dumb transports now has different behavior, but I haven't looked. My testing was light, so I may have totally broken something else, too. Input from more clueful people would be helpful.

Does this seem like a sane direction to take? It just seems silly to be pushing refs/remotes, which 99% of the time should be a purely local thing.

-Peff
---
diff --git a/send-pack.c b/send-pack.c
index d5b5162..39829e3 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -131,6 +131,8 @@ static int one_local_ref(const char *refname, const unsigned char *sha1, int fla
 {
 	struct ref *ref;
 	int len = strlen(refname) + 1;
+	if (!prefixcmp(refname, "refs/remotes/"))
+		return 0;
 	ref = xcalloc(1, sizeof(*ref) + len);
 	hashcpy(ref->new_sha1, sha1);
 	memcpy(ref->name, refname, len);
Junio C Hamano· Apr 6, 2007, 21:44 UTC · re: Jeff King · lore

Re: How can git pull be up-to-date and git push fail?

Jeff King <peff@peff.net> writes:
Show 17 quoted lines
> On Thu, Apr 05, 2007 at 02:18:58PM -0700, Junio C Hamano wrote:
>
>> IIRC "git push" without explicit refspecs push the matching
>> refs, but I am a bit under the weather and feverish, so don't
>> take my word literally but look at git-push manual page please.
>
> Ah, yes you're right. It really doesn't make sense to push
> refs/remotes/* in most cases, since they're just tracking branches, and
> if the destination _does_ have them, then it is unlikely to be in sync
> with you (leading to Bill's problem).  OTOH, you might want to be able
> to push them explicitly if you are doing a strict mirror of a
> repository.
>
> The patch below turns off refs/remotes/ sending for "git-push" and
> "git-push --all", but still allows "git-push origin
> remotes/origin/master". I'm not sure about the semantics; maybe --all
> should imply even remotes?

The behaviour of git-send-pack that pushes all the matching refs made sense when there was no separate-remotes layout (and there was no "git push" wrapper). Back then, the expected use was really:

 * For integrators who own public repositories, send-pack to
   their public repositories without refspecs updated what they
   already decided to publish in refs/heads and refs/tags.
   Updating refs/tags hierarchy means moving tags which should
   have done with caution (since it changes the general public's
   view of what the version v2.6.12 really is), but no one can
   update tag without first forcing with "git tag -f" in one's
   own repository to begin with, it was not a problem in
   practice.
 * For people who used pull/push with a central repository as
   the CVS-style synchronization point, the central repository,
   being the mother of all participant repositories, was
   supposed to lack the 'origin' branch, and participant
   repositories had 'origin' that tracked 'master' of the
   central repository.  So matching push never needed to worry
   about 'origin', and 'master' of participant was pushed to
   'master' of the central repository, undergoing the usual
   fast-forward checks and all.
   Even here, we had a few trouble reports from people who
   primed their central repository by cloning from somewhere
   (and cloning without --bare) and left the 'origin' branch
   around.  We might have added instruction to our documentation
   that tells people to remove 'origin' branch from their
   central repositories to avoid confusion (but I am too lazy to
   look).
 * For anybody who has more than one private repositories
   (e.g. mothership and notebook) and uses push between them
   when switching repositories to work in, new development might
   have been on a new branch, in which case it needed an
   explicit refspec to send the new branch or tag.  But
   "matching refs" rule was useful for heads most of the time,
   as long as the two repositories were reasonably in sync
   (e.g. after supper you sit on your couch working on notebook,
   and you finish the day by pushing matching refs to your
   mothership to prepare for tomorrow, which you would start on
   your mothership machine).

In all these cases, what we meant by "matching refs" rule is to update matching branch heads. Even pushing matching tags is debatable, as consciously replacing a tag that was already pushed out with a new one is already a special case, and I would suspect that replacing a public tag should be done via an explicit "git send-pack $url tag v2.6.12", even though the "matching refs" rule would silently update it.

People who built git-send-pack did not use separate remotes layout (which has not been even invented yet), nor StGIT, so refs/remotes, refs/bases nor refs/patches were not even on the radar. Speaking of StGIT, I do not think it makes sense to sync only .git/refs/{bases,patches} without sending .git/patches.

> Does this seem like a sane direction to take? It just seems silly to be
> pushing refs/remotes, which 99% of the time should be a purely local
> thing.

So I'd suspect "git push" (not limited to git-send-pack) without explicit refspecs should only do "matching branch heads" these days, to keep the original spirit but yet to adjust to today's reality. In other words, instead of treating remotes/ specially (in a negative sense), like your patch does, I think we should do only "matching refs in heads/" hierarchy.

Jeff King· Apr 7, 2007, 17:07 UTC · re: Junio C Hamano · lore

Re: How can git pull be up-to-date and git push fail?

On Fri, Apr 06, 2007 at 02:44:02PM -0700, Junio C Hamano wrote:
> was no "git push" wrapper).  Back then, the expected use was
> really:
Thanks for this for the historical information.
Show 6 quoted lines
> So I'd suspect "git push" (not limited to git-send-pack) without
> explicit refspecs should only do "matching branch heads" these
> days, to keep the original spirit but yet to adjust to today's
> reality.  In other words, instead of treating remotes/ specially
> (in a negative sense), like your patch does, I think we should
> do only "matching refs in heads/" hierarchy.

Agreed. That was actually my initial feeling, but it meant changing the semantics of all of the other parts of the refs/* hierarchy with respect to publishing, which I was a bit nervous about.

A patch is attached below for comment; it's the same as my previous
patch, except:
  - matching only refs/heads/
  - rename local_ref to local_head for consistency (one of the functions
    was already get_local_heads, but the other's didn't match)
  - avoid calling strlen if we're just going to return immediately
  - add a test to simulate Bill's problem
  - update documentation for git-push

But it looks like I have broken pushing of existing tags using --tags, so I think I might have to put the ref-culling in a different spot. Let me look in that.

BTW, my gut feeling is that send-pack is plumbing, and git-push is porcelain, and therefore the decisions about what to push should be made at the push layer, and send-pack should only push what it is explicitly told to push. But changing the defaults of send-pack at this point is probably a bad idea.

---
 Documentation/git-push.txt      |    4 ++--
 Documentation/git-send-pack.txt |    4 ++--
 send-pack.c                     |   15 +++++++++------
 t/t5400-send-pack.sh            |   18 ++++++++++++++++++
 4 files changed, 31 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-push.txt b/Documentation/git-push.txt
index f8cc2b5..0b21306 100644
--- a/Documentation/git-push.txt
+++ b/Documentation/git-push.txt
@@ -46,7 +46,7 @@ even if it does not result in a fast forward update.
 Note: If no explicit refspec is found, (that is neither
 on the command line nor in any Push line of the
 corresponding remotes file---see below), then all the
-refs that exist both on the local side and on the remote
+heads that exist both on the local side and on the remote
 side are updated.
 +
 `tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`.
@@ -60,7 +60,7 @@ the remote repository.
 
 \--all::
 	Instead of naming each ref to push, specifies that all
-	refs be pushed.
+	refs under `$GIT_DIR/refs/heads/` be pushed.
 
 \--tags::
 	All refs under `$GIT_DIR/refs/tags` are pushed, in
diff --git a/Documentation/git-send-pack.txt b/Documentation/git-send-pack.txt
index 205bfd2..3271e88 100644
--- a/Documentation/git-send-pack.txt
+++ b/Documentation/git-send-pack.txt
@@ -32,7 +32,7 @@ OPTIONS
 
 \--all::
 	Instead of explicitly specifying which refs to update,
-	update all refs that locally exist.
+	update all heads that locally exist.
 
 \--force::
 	Usually, the command refuses to update a remote ref that
@@ -70,7 +70,7 @@ With '--all' flag, all refs that exist locally are transferred to
 the remote side.  You cannot specify any '<ref>' if you use
 this flag.
 
-Without '--all' and without any '<ref>', the refs that exist
+Without '--all' and without any '<ref>', the heads that exist
 both on the local side and on the remote side are updated.
 
 When one or more '<ref>' are specified explicitly, it can be either a
diff --git a/send-pack.c b/send-pack.c
index d5b5162..b2b6886 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -124,13 +124,16 @@ static int ref_newer(const unsigned char *new_sha1,
 	return found;
 }
 
-static struct ref *local_refs, **local_tail;
+static struct ref *local_heads, **local_tail;
 static struct ref *remote_refs, **remote_tail;
 
-static int one_local_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
+static int one_local_head(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
 {
 	struct ref *ref;
-	int len = strlen(refname) + 1;
+	int len;
+	if (prefixcmp(refname, "refs/heads/"))
+		return 0;
+	len = strlen(refname) + 1;
 	ref = xcalloc(1, sizeof(*ref) + len);
 	hashcpy(ref->new_sha1, sha1);
 	memcpy(ref->name, refname, len);
@@ -141,8 +144,8 @@ static int one_local_ref(const char *refname, const unsigned char *sha1, int fla
 
 static void get_local_heads(void)
 {
-	local_tail = &local_refs;
-	for_each_ref(one_local_ref, NULL);
+	local_tail = &local_heads;
+	for_each_ref(one_local_head, NULL);
 }
 
 static int receive_status(int in)
@@ -198,7 +201,7 @@ static int send_pack(int in, int out, int nr_refspec, char **refspec)
 	/* match them up */
 	if (!remote_tail)
 		remote_tail = &remote_refs;
-	if (match_refs(local_refs, remote_refs, &remote_tail,
+	if (match_refs(local_heads, remote_refs, &remote_tail,
 		       nr_refspec, refspec, send_all))
 		return -1;
 
diff --git a/t/t5400-send-pack.sh b/t/t5400-send-pack.sh
index 477b267..ca4485a 100755
--- a/t/t5400-send-pack.sh
+++ b/t/t5400-send-pack.sh
@@ -113,4 +113,22 @@ test_expect_success \
 	! git diff .git/refs/heads/master victim/.git/refs/heads/master
 '
 
+test_expect_success \
+	'pushing does not include non-head refs' '
+	mkdir parent && cd parent &&
+	  git-init && touch file && git-add file && git-commit -m add &&
+	cd .. &&
+	git-clone parent child1 &&
+	git-clone parent child2 &&
+	cd parent &&
+	  echo parent >file && git-commit -a -m change &&
+	cd .. &&
+	cd child1 && git-pull && cd .. &&
+	cd child2 &&
+	  touch >file2 git-add file2 && git-add file2 && git-commit -m add2 &&
+	  git-pull ../child1 &&
+	  git-push ../child1 2>err &&
+	  ! grep ^error err
+'
+
 test_done
Junio C Hamano· Apr 7, 2007, 21:26 UTC · re: Jeff King · lore

Re: How can git pull be up-to-date and git push fail?

Jeff King <peff@peff.net> writes:
Show 21 quoted lines
> On Fri, Apr 06, 2007 at 02:44:02PM -0700, Junio C Hamano wrote:
>
>> was no "git push" wrapper).  Back then, the expected use was
>> really:
>
> Thanks for this for the historical information.
>
>> So I'd suspect "git push" (not limited to git-send-pack) without
>> explicit refspecs should only do "matching branch heads" these
>> days, to keep the original spirit but yet to adjust to today's
>> reality....
>
> Agreed. That was actually my initial feeling, but it meant changing the
> semantics of all of the other parts of the refs/* hierarchy with respect
> to publishing, which I was a bit nervous about.
> ...
> BTW, my gut feeling is that send-pack is plumbing, and git-push is
> porcelain, and therefore the decisions about what to push should be made
> at the push layer, and send-pack should only push what it is explicitly
> told to push. But changing the defaults of send-pack at this point is
> probably a bad idea.

Yes, that is exactly why I suggested "git push" without explicit refspecs to drive the plumbings to push only matching branches. So if you are worried about breaking backward compatibility for real men (iow people who use git-send-pack directly), it would be a less impact change to (1) add --matching-heads option to send-pack and perhaps http-push, and change their behaviour only in that case, and (2) pass that option when git-push did not get explicit refspecs to send from the user, either from the command line nor from the config.

Jeff King· Apr 7, 2007, 21:32 UTC · re: Junio C Hamano · lore

Re: How can git pull be up-to-date and git push fail?

On Sat, Apr 07, 2007 at 02:26:44PM -0700, Junio C Hamano wrote:
Show 9 quoted lines
> Yes, that is exactly why I suggested "git push" without explicit
> refspecs to drive the plumbings to push only matching branches.
> So if you are worried about breaking backward compatibility for
> real men (iow people who use git-send-pack directly), it would
> be a less impact change to (1) add --matching-heads option to
> send-pack and perhaps http-push, and change their behaviour only
> in that case, and (2) pass that option when git-push did not get
> explicit refspecs to send from the user, either from the command
> line nor from the config.

Ah, I (obviously) didn't pick that up from your original message. Let me rework my patch and see if I can get something more reasonable.

-Peff

← back to recent threads