threads / discuss / 2163

rsync update appears broken now

Subject: rsync update appears broken now

## tl;dr

21 messages between Oct 20, 2005 and Oct 21, 2005.

replies: 20people: 9as markdown or json

Randal L. Schwartz· Oct 20, 2005, 12:47 UTC · lore
Doing my daily git-pull now broke in this way (using yesterday's git version):
sent 1196 bytes  received 155984 bytes  4555.94 bytes/sec
total size is 4511741  speedup is 28.70
* committish: 6e1c6c103c522d01829f3a63992a023ff031e851
  branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
* refs/heads/origin: does not fast forward to branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git;
  not updating.
Trying really trivial in-index merge...
fatal: Merge requires file-level merging
Nope.
Trying simple merge.
Simple merge failed, trying Automatic merge.
Auto-merging fetch-pack.c.
merge: warning: conflicts during merge
ERROR: Merge conflict in fetch-pack.c.
fatal: merge program failed
Automatic merge failed; fix up by hand
-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Alex Riesen· Oct 20, 2005, 13:08 UTC · re: Randal L. Schwartz · lore

Re: rsync update appears broken now

On 20 Oct 2005 05:47:42 -0700, Randal L. Schwartz <merlyn@stonehenge.com> wrote:
Show 19 quoted lines
>
> Doing my daily git-pull now broke in this way (using yesterday's git version):
>
> sent 1196 bytes  received 155984 bytes  4555.94 bytes/sec
> total size is 4511741  speedup is 28.70
> * committish: 6e1c6c103c522d01829f3a63992a023ff031e851
>   branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
> * refs/heads/origin: does not fast forward to branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git;
>   not updating.
> Trying really trivial in-index merge...
> fatal: Merge requires file-level merging
> Nope.
> Trying simple merge.
> Simple merge failed, trying Automatic merge.
> Auto-merging fetch-pack.c.
> merge: warning: conflicts during merge
> ERROR: Merge conflict in fetch-pack.c.
> fatal: merge program failed
> Automatic merge failed; fix up by hand

Absolutely normal pull into a changed repository. Just fix the conflict (in fetch-pack.c, look for >>>), git-update-index the file and commit. Doesn't look like a problem at all.

Randal L. Schwartz· Oct 20, 2005, 14:12 UTC · re: Alex Riesen · lore

Re: rsync update appears broken now

>>>>> "Alex" == Alex Riesen <raa.lkml@gmail.com> writes:

Alex> Absolutely normal pull into a changed repository. Just fix the Alex> conflict (in fetch-pack.c, look for >>>), git-update-index the file Alex> and commit. Doesn't look like a problem at all.

What do you mean "changed repository"? This is my git image, and I'm not working on git. I made no changes.

Thus, something broken?
-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Alex Riesen· Oct 20, 2005, 14:24 UTC · re: Randal L. Schwartz · lore

Re: rsync update appears broken now

On 20 Oct 2005 07:12:53 -0700, Randal L. Schwartz <merlyn@stonehenge.com> wrote:
Show 9 quoted lines
> Alex> Absolutely normal pull into a changed repository. Just fix the
> Alex> conflict (in fetch-pack.c, look for >>>), git-update-index the file
> Alex> and commit. Doesn't look like a problem at all.
>
> What do you mean "changed repository"?  This is my git image, and I'm
> not working on git.  I made no changes.
>
> Thus, something broken?
>

Not necessarily (I see this every morning, but I know precisely I touched the sources). Maybe local damage? What does git-status show?

Linus Torvalds· Oct 20, 2005, 17:48 UTC · re: Alex Riesen · lore

Re: rsync update appears broken now

On Thu, 20 Oct 2005, Alex Riesen wrote:
> 
> Absolutely normal pull into a changed repository. Just fix the
> conflict (in fetch-pack.c, look for >>>), git-update-index the file
> and commit. Doesn't look like a problem at all.
No, that's wrong.

The fact is, Junio has screwed up his repository, and if you merge, you'll never have something that matches _his_ repository.

To fix this, _Junio_ needs to fix his repository.

For those of us who have a separate branch to track the original (I call mine "parent), this tells the story:

	* refs/heads/parent: does not fast forward to branch 'master' of master.kernel.org:/pub/scm/git/git;
	  not updating.

IOW, it looks like Junio has screwed up his "master" branch on kernel.org and it no longer contains what it used to contain (and this is not a mirroring problem - I'm using "master.kernel.org" with no mirrors in between).

Doing a
	git fetch parent master:new-junio
followed by a
	gitk parent new-junio

shows that Junio seems to have re-based his "master" branch by removing his old top-most entry, and restarted the "master" branch, which is wrong, wrong, wrong.

Junio, please don't do that. It really screws people up. Now people can't fetch your head any more, and can't track you, because your branch isn't stable any more. I know you're very used to doing so in your "pu" branch, but it's _wrong_. It's wrong in "pu" too, but at least there you have an excuse.

Reparenting things is ok in _private_ branches, but it's not ok in public ones, since _others_ will have worked or at least downloaded the previous state.

The fact is, "distributed" fundamentally means "mistakes cannot be undone". Mistakes have to be _fixed_, not removed. Because the mistakes have already percolated to others.

			Linus
Junio C Hamano· Oct 20, 2005, 20:48 UTC · re: Linus Torvalds · lore

Re: rsync update appears broken now

Linus Torvalds <torvalds <at> osdl.org> writes:
> For those of us who have a separate branch to track the original (I call 
> mine "parent), this tells the story:
> 
> 	* refs/heads/parent: does not fast forward to branch 'master' of
master.kernel.org:/pub/scm/git/git;
> 	  not updating.
Yes, I screwed up.  One honest mistake and another stupid one.

Note that "those of us" above include everybody who did "git-clone" to set up the repository and left the default "Pull: master:origin" line in .git/refs/remotes/origin.

> Junio, please don't do that. It really screws people up. Now people can't 
> fetch your head any more, and can't track you, because your branch isn't 
> stable any more.

Sorry, and yes that was the "honest mistake" part. I know I should never rewind beyond what I already have pushed out, and usually I carefully follow that rule.

But this time I forgot that I pushed something out and also failed to re-check what I pushed out before rewinding (the check is done by fetching from master.kernel.org myself).

And another stupid mistake part was when I pushed out the result. send-pack correctly refused to update the master side, but I forced it without much thinking.

Sorry about these mistakes.

Having said that, all is not lost. Your message made it sound like the new head was completely re-rooted and there is no common commit (which almost gave me a heart attack), but I do not think it is that bad. It is more like pulling from two places.

               -----(A) head merlyn and everybody
             /          pulled from kernel.org previously
  --- common ------------------------------------(B) head rebased and pushed
                                                     out by mistake

The "parent" head you described in your message is A above, and what is on kernel.org is B. A is known as "origin" in the default setup git-clone makes.

If you have *no* further development on top of (A), the recovery is simple.
   $ git checkout master ;# make sure you are on your "master" branch.
   $ git fetch origin +master:origin ;# force "origin" to be (B)
   $ git reset --hard origin ;# reset your "master" branch to (B)
If you had any further development on top of (A), then the graph would look
like:
               -----(A)---------------(X) your tip
             /
  --- common ----------------------(B)

The merge conflict message you got when you pulled is result of trying to merge (X) and (B).

There are two things we could do.
One is to forward port your changes between (A) and (X) on top of (B):
   $ git checkout master ;# make sure you are on your "master" branch.
   $ git tag anchor-a origin ;# stash (A) away
   $ git fetch origin +master:origin ;# force "origin" to be (B)
   $ git checkout -b rescue-stupid-junio origin
   $ rm -fr .dotest
   $ git format-patch -k --stdout anchor-a..master | git am -k -3

Depending on the changes between A and X, forward porting may result in conflicting/unapplicable patches. The individual changes are found in .dotest/ directory and you can re-run 'git am -i' after fixing up the conflicts.

Then validate the tip of the result to make sure forward porting did not lose your changes:

   $ git diff origin..rescue-stupid-junio
If the result look OK, then we can make this your master branch.
   $ git checkout master
   $ git reset --hard rescue-stupid-junio
and clean things up by
   $ rm -f .git/refs/tags/anchor-a
   $ git branch -d rescue-stupid-junio

Another thing we could do is to treat as if this stupid maintainer briefly had a twin who did concurrent development, and merge (X) and (B).

   $ git checkout master
   $ git pull origin +master:origin

This, as Merlyn and you saw, probably would result in merge conflicts which need to be resolved manually.

Unfortunately I do not have access right now to commits between "common" and (A) in the above picture (I am at work), so I cannot try any of the above out myself.

Linus Torvalds· Oct 20, 2005, 21:32 UTC · re: Junio C Hamano · lore

Re: rsync update appears broken now

On Thu, 20 Oct 2005, Junio C Hamano wrote:
Show 5 quoted lines
> 
> Having said that, all is not lost.  Your message made it sound like the
> new head was completely re-rooted and there is no common commit (which
> almost gave me a heart attack), but I do not think it is that bad.  It is
> more like pulling from two places.
No. 

What you _must_ do to fix things up is that _you_ merge your current head with your old head, and then you push out the result.

At that point:
>                -----(A) head merlyn and everybody
>              /          pulled from kernel.org previously
>   --- common ------------------------------------(B) head rebased and pushed
>                                                      out by mistake

You should have a (C) that is the merge of both A and B (or a later commit, if you have other commits merged).

Now you can push out that merge, and everybody will be happy again: they'll see somethign that is a proper superset of whichever HEAD they happened to pull.

Don't make everybody else try to fix up your mistakes for you. It's easy enough for you to just fix it up, and everybody else can just pull.

Of course, anybody who did a pull and then did the merge by hand will always have their own private merge, and they should probably just undo that. But undoing _private_ stuff is ok.

		Linus
Junio Hamano· Oct 20, 2005, 23:37 UTC · re: Linus Torvalds · lore

Re: rsync update appears broken now

Linus Torvalds <torvalds@osdl.org> writes:
> What you _must_ do to fix things up is that _you_ merge your current head 
> with your old head, and then you push out the result.
Yes.  That is what I will do when able (not right now).

I posted a workaround because I am at work and I do not have access to kernel.org machine from here, and I did not pull yesterday at work so I do not have an access to that lost side branch until later today (I think I still have it in my private repo, either on my home machine or my ~/git on hera).

Show 6 quoted lines
> At that point:
>
>>                -----(A) head merlyn and everybody
>>              /          pulled from kernel.org previously
>>   --- common ------------------------------------(B) head rebased and pushed
>>                                                      out by mistake
Mind telling me the (A) commit ID if you know it?
Johannes Schindelin· Oct 20, 2005, 23:41 UTC · re: Junio Hamano · lore

Re: rsync update appears broken now

Hi,
On Thu, 20 Oct 2005, Junio Hamano wrote:
Show 8 quoted lines
> > At that point:
> >
> >>                -----(A) head merlyn and everybody
> >>              /          pulled from kernel.org previously
> >>   --- common ------------------------------------(B) head rebased and pushed
> >>                                                      out by mistake
> 
> Mind telling me the (A) commit ID if you know it?
ea5a65a59916503d2a14369c46b1023384d51645

Also, people having merged with (A) would not have to undo that merge as was suggested. The new (C) commit would contain the revert of (A).

Ciao, Dscho

Linus Torvalds· Oct 20, 2005, 23:49 UTC · re: Johannes Schindelin · lore

Re: rsync update appears broken now

On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> 
> Also, people having merged with (A) would not have to undo that merge as 
> was suggested. The new (C) commit would contain the revert of (A).

Oh, they'll need to revert if they want to match Junios history. Otherwise they'll always generate yet another merge when they pull.

Doesn't matter that the _tree_ state may be the same, the only thing that matters is that their history will be different, and thus they will never be able to fast-forwared to Junio's tree unless they revert their local merge.

		Linus
Johannes Schindelin· Oct 21, 2005, 00:01 UTC · re: Linus Torvalds · lore

Re: rsync update appears broken now

Hi,
On Thu, 20 Oct 2005, Linus Torvalds wrote:
Show 7 quoted lines
> On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> > 
> > Also, people having merged with (A) would not have to undo that merge as 
> > was suggested. The new (C) commit would contain the revert of (A).
> 
> Oh, they'll need to revert if they want to match Junios history. Otherwise 
> they'll always generate yet another merge when they pull.
So?
> Doesn't matter that the _tree_ state may be the same, the only thing that 
> matters is that their history will be different, and thus they will never 
> be able to fast-forwared to Junio's tree unless they revert their local 
> merge.

It does not have to be a fast-forward. After all, what is another merge? Since that merge does not have Junio as committer, close inspection of the commit will reveal that.

Of course, if you did not have local changes and merged, you might want to do a clean "git-fetch kernel.org +master:parent".

So, no need for a revert.

Ciao, Dscho

Linus Torvalds· Oct 21, 2005, 00:19 UTC · re: Johannes Schindelin · lore

Re: rsync update appears broken now

On Fri, 21 Oct 2005, Johannes Schindelin wrote:
Show 5 quoted lines
> > 
> > Oh, they'll need to revert if they want to match Junios history. Otherwise 
> > they'll always generate yet another merge when they pull.
> 
> So?
History matters. History matters as much as the data does.
> It does not have to be a fast-forward. After all, what is another merge? 
> Since that merge does not have Junio as committer, close inspection of 
> the commit will reveal that.

It _does_ have to be a fast-forward, if you expect the tree to have the same content as Junio's.

Even if it merges everything automatically, if the history is different, it could in theory at least merge _differently_ than what Junio had. Plus you'll have a really ugly version history for no good reason.

But yes, it will _work_. It just won't work the way people expect it to, so they'll make bug-reports about not being able to merge automatically, and others will say that they can't re-create it and blame the wrong person.

		Linus
Petr Baudis· Oct 21, 2005, 00:34 UTC · re: Linus Torvalds · lore

Re: rsync update appears broken now

Dear diary, on Fri, Oct 21, 2005 at 02:19:42AM CEST, I got a letter where Linus Torvalds <torvalds@osdl.org> told me that...

Show 11 quoted lines
> On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> > It does not have to be a fast-forward. After all, what is another merge? 
> > Since that merge does not have Junio as committer, close inspection of 
> > the commit will reveal that.
> 
> It _does_ have to be a fast-forward, if you expect the tree to have the 
> same content as Junio's.
> 
> Even if it merges everything automatically, if the history is different, 
> it could in theory at least merge _differently_ than what Junio had. Plus 
> you'll have a really ugly version history for no good reason.

And if Junio will want to merge with many of those people, you will better have to get a dualhead for your gitk.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Linus Torvalds· Oct 20, 2005, 23:47 UTC · re: Junio Hamano · lore

Re: rsync update appears broken now

On Thu, 20 Oct 2005, Junio Hamano wrote:
> 
> Mind telling me the (A) commit ID if you know it?

The latest one I have is ea5a65a59916503d2a14369c46b1023384d51645, but if you had more pushed out at some point that I just didn't happen to pick up, that may not be the top-most (A).

		Linus
Junio C Hamano· Oct 21, 2005, 00:26 UTC · re: Linus Torvalds · lore

Re: rsync update appears broken now

Linus Torvalds <torvalds <at> osdl.org> writes:
> The latest one I have is ea5a65a59916503d2a14369c46b1023384d51645, but if 
> you had more pushed out at some point that I just didn't happen to pick 
> up, that may not be the top-most (A).

I am reasonably sure that the screw-up was only rewinding one commit too much. I've done the merge so things should look better once mirrors catch up.

Thanks for your help.
Linus Torvalds· Oct 21, 2005, 00:52 UTC · re: Junio C Hamano · lore

Re: rsync update appears broken now

On Fri, 21 Oct 2005, Junio C Hamano wrote:
> 
> I am reasonably sure that the screw-up was only rewinding one commit too much.
> I've done the merge so things should look better once mirrors catch up.
Yup, works at least for me. Thx,
		Linus
Randal L. Schwartz· Oct 21, 2005, 13:49 UTC · re: Linus Torvalds · lore

Re: rsync update appears broken now

>>>>> "Linus" == Linus Torvalds <torvalds@osdl.org> writes:
Linus> On Fri, 21 Oct 2005, Junio C Hamano wrote:
>> 
>> I am reasonably sure that the screw-up was only rewinding one commit too much.
>> I've done the merge so things should look better once mirrors catch up.
Linus> Yup, works at least for me. Thx,
Doesn't yet work for me.  What do I need to do now?
    * committish: 4ae22d96fe9248dac4f26b1fc91154ba5e879799
      branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
    * refs/heads/origin: same as branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
    Updating from ea5a65a59916503d2a14369c46b1023384d51645 to 4ae22d96fe9248dac4f26b1fc91154ba5e879799.
    fetch-pack.c: needs update
    fatal: Entry 'daemon.c' would be overwritten by merge. Cannot merge.
-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Linus Torvalds· Oct 21, 2005, 15:26 UTC · re: Randal L. Schwartz · lore

Re: rsync update appears broken now

On Fri, 21 Oct 2005, Randal L. Schwartz wrote:
Show 8 quoted lines
> 
> Doesn't yet work for me.  What do I need to do now?
> 
>     * committish: 4ae22d96fe9248dac4f26b1fc91154ba5e879799
>       branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
>     * refs/heads/origin: same as branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
>     Updating from ea5a65a59916503d2a14369c46b1023384d51645 to 4ae22d96fe9248dac4f26b1fc91154ba5e879799.
>     fetch-pack.c: needs update

Your previous failed pull had left the unfinished merge around, so now when you try to pull and it tries to update, it can't.

Do a 
	git checkout -f

first to reset your tree to your old HEAD, and then try again. It should work then.

(Or do "git reset --hard", which should end up doing the same thing)
		Linus
dave morgan· Oct 20, 2005, 18:03 UTC · re: Morten Welinder · lore

Re: rsync update appears broken now

On Thu, 20 Oct 2005 10:15:58 -0400, Morten Welinder <mwelinder@gmail.com> wrote:

Show 7 quoted lines
>I see the very same with an http pull.
>
>Morten
>-
>To unsubscribe from this list: send the line "unsubscribe git" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
I got the same with a git:// update
origin  git://git.kernel.org/pub/scm/git/git.git	

david@tower2:~/git$ cg-update Up to date.

Applying changes...
Merging f8765797a41a39f4dfc7030098c38283e6461a83 ->
6e1c6c103c522d01829f3a63992a023ff031e851
        to ea5a65a59916503d2a14369c46b1023384d51645...
... Auto-merging fetch-pack.c
merge: warning: conflicts during merge
        Conflicts during merge. Do cg-commit after resolving them.
here is status after pull
david@tower2:~/git$ cg-status
Heads:
   >master      ea5a65a59916503d2a14369c46b1023384d51645
  R origin      6e1c6c103c522d01829f3a63992a023ff031e851

? git-build-rev-cache ? git-checkout-cache ? git-convert-cache ? git-diff-cache ? git-diff-helper ? git-export ? git-fsck-cache ? git-http-pull ? git-local-pull ? git-merge-cache ? git-rev-tree ? git-show-rev-cache ? git-update-cache

Dave
Stephen C. Tweedie· Oct 20, 2005, 15:03 UTC · re: Randal L. Schwartz · lore

Re: rsync update appears broken now

Hi,
On Thu, 2005-10-20 at 05:47 -0700, Randal L. Schwartz wrote:
Show 6 quoted lines
> Doing my daily git-pull now broke in this way (using yesterday's git version):
> ...
> * committish: 6e1c6c103c522d01829f3a63992a023ff031e851
>   branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git
> * refs/heads/origin: does not fast forward to branch 'master' of rsync://rsync.kernel.org/pub/scm/git/git;
>   not updating.
Seen here too.  My HEAD and HEAD^ as of yesterday's pull was:
commit ea5a65a59916503d2a14369c46b1023384d51645
Author: Junio C Hamano <junkio@cox.net>
Date:   Tue Oct 18 18:42:19 2005 -0700
    Do not ask for objects known to be complete.
    On top of optimization by Linus not to ask refs that already match, we
    can walk our refs and not issue "want" for things that are known to be
    reachable from them.
    Signed-off-by: Junio C Hamano <junkio@cox.net>
commit f8765797a41a39f4dfc7030098c38283e6461a83
Author: Junio C Hamano <junkio@cox.net>
Date:   Tue Oct 18 18:42:14 2005 -0700
    Even when overwriting tags, report if they are changed or not.
    Signed-off-by: Junio C Hamano <junkio@cox.net>

Today, looking at git-web I can see that that HEAD^ commit is still there, but yesterday's HEAD is simply not in the commit chain any more: there are a couple of other commits and then the "Do not ask for objects known to be complete" commit appears with SHA 49bb805e69f97e75472e54a68e9eb24e08dee011.

Somebody broke this by rsyncing a non-superset tree to kernel.org, maybe?

"git reset --hard HEAD^" got rid of the missing HEAD commit and allowed me to pull from the new commit chain, but that really shouldn't be necessary.

--Stephen

← back to recent threads