threads / discuss / 36599
Subject: Beginner question on "Pull is mostly evil"
During my initial self-education I came across the maxim "don't pull, fetch+merge instead" and have been doing that. I think I followed most of the "pull is (mostly) evil" discussion but one facet still puzzles me: the idea that pull will do a merge "in the wrong direction" sometimes.
Do I understand correctly that this occurs only in the presence of multiple remotes? Can someone provide a simple example of a situation where pull would do the "wrong" thing?
Thanks
David KastrupRe: Beginner question on "Pull is mostly evil"
Jim Garrison <jim.garrison@nwea.org> writes:
> During my initial self-education I came across the maxim "don't pull, > fetch+merge instead" and have been doing that. I think I followed > most of the "pull is (mostly) evil" discussion but one facet still > puzzles me: the idea that pull will do a merge "in the wrong > direction" sometimes. > > Do I understand correctly that this occurs only in the presence of > multiple remotes? > Can someone provide a simple example of a situation where pull would > do the "wrong" thing?
That's basically unavoidable. Two opposing directions are actually part of the same workflow usually handled by "git pull":
"Codeveloper X sends a pull request to Y who maintains the mainline. Y executes git pull to merge X' sidebranch into the mainline."
"Codeveloper X executes git pull in order to merge the mainline from Y back into his private sidebranch."
-- David Kastrup
Jeff KingRe: Beginner question on "Pull is mostly evil"
On Wed, May 07, 2014 at 03:40:28PM +0000, Jim Garrison wrote:
> During my initial self-education I came across the maxim "don't pull, > fetch+merge instead" and have been doing that. I think I followed > most of the "pull is (mostly) evil" discussion but one facet still > puzzles me: the idea that pull will do a merge "in the wrong > direction" sometimes. > > Do I understand correctly that this occurs only in the presence of > multiple remotes?
No, it does not have to do with multiple remotes. It is about "X merged into Y" versus "Y merged into X". The ordering of parents in a merge doesn't matter for the merge result, but git must choose some order, and it always uses your current HEAD first, and then the commit you are merging second (and so on, in an octopus merge).
As a result, you can use "git log --first-parent" to follow the line of development that always got merged into. In a strict topic-branch workflow like git.git, this will show you just what happened on master: a linear sequence of merges of topic branches, with occasional direct-to-master commits like version bumps.
For an integrator who is pulling from other people, "git pull bob topic" from "master" does the right thing: master is the first parent, and topic is the second parent.
For somebody with a centralized repo who follows the "push was a non-fastforward, so pull then push" advice, the merge between their work and master will be "backwards". The merge commit will have upstream's work (i.e., "master") merged into their topic. Following --first-parent will walk down their work instead of the merge commits on master.
Does that explain it?
-Peff
Junio C HamanoRe: Beginner question on "Pull is mostly evil"
Jim Garrison <jim.garrison@nwea.org> writes:
> During my initial self-education I came across the maxim "don't > pull, fetch+merge instead" and have been doing that. I think I > followed most of the "pull is (mostly) evil" discussion but one > facet still puzzles me: the idea that pull will do a merge "in the > wrong direction" sometimes.
[administrivia: wrap your lines to reasonable length like ~70 cols, please]
> Do I understand correctly that this occurs only in the presence of > multiple remotes?
No. This is most often true for people who use a single repository as a place for everybody to meet, in the same way as SVN.
Suppose that that central repository has this history:
---o---o---A
which ends at commit A (time flows from left to right and each node in the graph is a commit, lines between them indicating parent-child relationship).
Then you clone it and work on your own commits, which leads you to have this in *your* repository:
---o---o---A---B---C
Imagine your coworker did the same and built on top of A in *his* repository this history in the meantime, and then pushed it to the central repository:
---o---o---A---X---Y---Z
Now, if you "git push" at this point, beause your history that leads to C lack X, Y and Z, it will fail. You need to somehow make the tip of your history a descendant of Z.
One way that "mostly evil" thread discusses is to "pull", which is "fetch and then merge" (note that I am saying "don't pull, instead fetch and merge" is not an advice to solve "pull is mostly evil" issue at all). If you fetch, your repository will have a history like this:
---o---o---A---B---C
\
X---Y---ZAnd then if you did merge after that, while still on *your* branch, i.e. C, you will create a merge M and make the history look like this:
---o---o---A---B---C---M
\ /
X---Y---ZM is a descendant of Z, so you can push to update the central repository. Such a merge M does not lose any commit in both histories, so in that sense it may not be wrong, but when people would want to talk about "the authoritative canonical history that is shared among the project participants", i.e. "the trunk", the way they often use is to do:
$ git log --first-parent
For all other people who observed the central repository after your coworker pushed Z but before you pushed M, the commit on the trunk used to be "o-o-A-X-Y-Z". But because you made M while you were on C, M's first parent is C, so by pushing M to advance the central repository, you made X-Y-Z a side branch, not on the trunk.
You would rather want to have a history of this shape:
---o---o---A---X---Y---Z---M'
\ /
B-----------Cso that in the first-parent chain, it is clear that the project first did X and then Y and then Z and merged a change that consists of two commits B and C that achieves a single goal. You may have worked on fixing the bug #12345 with these two patches, and the merge M' with swapped parents can say in its log message "Merge 'fix-bug-12345'".
Note that I said "achieves a single goal" above, because this is important. "swapping the merge order" only covers a special case where the project does not care too much about having unrelated things done on a single merge but cares a lot about first-parent chain.
There are multiple schools of thought about the "trunk" management.
1. Some projects want to keep a completely linear history without
any merges. Obviously, swapping the merge order would not help
their taste. You would need to flatten your history on top of
the updated upstream to result in a history of this shape
instead:---o---o---A---X---Y---Z---B---C
with "git pull --rebase" or something.
2. Some projects tolerate merges in their history, but do not worry
too much about the first-parent order, and allows fast-forward
merges. To them, swapping the merge order does not hurt, but
it is unnecessary. 3. Some projects want each commit on the "trunk" to do one single
thing. The output of "git log --first-parent" in such a project
would show either a merge of a side branch that completes a
single theme, or a single commit that completes a single theme
by itself. If your two commits B and C (or they may even be two
groups of commits) were solving two independent issues, then the
merge M' we made in the earlier example by swapping the merge
order is still not up to the project standard. It merges two
unrelated efforts B and C at the same time.For projects in the last category (git itself is one of them), individual developers would want to prepare a history more like this:
C0--C1--C2 topic-c
/
---o---o---A master
\
B0--B1--B2 topic-bThat is, keeping separate topics on separate branches, perhaps like so:
$ git clone $URL work && cd work
$ git checkout -b topic-b master
$ ... work to create B0, B1 and B2 to complete one theme
$ git checkout -b topic-c master
$ ... same for the theme of topic-cAnd then
$ git checkout master
$ git pull --ff-onlywould grab X, Y and Z from the upstream and advance your master branch:
C0--C1--C2
/
---o---o---A---X---Y---Z
\
B0--B1--B2And then you would merge these two branches separately:
$ git merge topic-b
$ git merge topic-cto result in
C0--C1---------C2
/ \
---o---o---A---X---Y---Z---M---N
\ /
B0--B1-----B2and push it back to the central repository.
It is very much possible that while you are merging topic-b and topic-c, somebody again advanced the history in the central repository to put W on top of Z, and make your "git push" fail.
In such a case, you would rewind to discard M and N, update the tip of your 'master' again and redo the two merges:
$ git reset --hard origin/master
$ git pull --ff-only
$ git merge topic-b
$ git merge topic-c C0--C1--------------C2
/ \
---o---o---A---X---Y---Z---W---M'--N
\ /
B0--B1---------B2So one part of the solution to "pull is mostly evil" has to involve making this "recreating your work on top of the updated upstream" easier for users. Otherwise, even if people *know* that rewinding and rebuilding is the right thing to do, they will find it too cumbersome and end up pushing merges in random order.
For another way to put this, see
http://git-blame.blogspot.com/2012/03/fun-with-first-parent.html
HTH.
RE: Beginner question on "Pull is mostly evil"
> -----Original Message----- > From: Junio C Hamano > Sent: Wednesday, May 07, 2014 1:16 PM > Subject: Re: Beginner question on "Pull is mostly evil" > > No. This is most often true for people who use a single repository as a > place for everybody to meet, in the same way as SVN.
[snip lots of excellent detail]
> HTH.
Wow. That helps tremendously, and should be incorporated somewhere in the Git documentation. Thank you for your immensely detailed response.
Apologies about not breaking lines... I'll remember that in future.
Junio C HamanoRe: Beginner question on "Pull is mostly evil"
Jim Garrison <jim.garrison@nwea.org> writes:
>> -----Original Message----- >> From: Junio C Hamano >> Sent: Wednesday, May 07, 2014 1:16 PM >> Subject: Re: Beginner question on "Pull is mostly evil" >> >> No. This is most often true for people who use a single repository as a >> place for everybody to meet, in the same way as SVN. > [snip lots of excellent detail] >> HTH. > > Wow. That helps tremendously, and should be incorporated somewhere in the > Git documentation. Thank you for your immensely detailed response.
We used to collect useful list postings in Documentation/howto/; perhaps somebody wants to do the minimum copyediting of the message and send a patch?
Re: Beginner question on "Pull is mostly evil"
I'll create a patch.
On Wednesday, May 07, 2014 01:51:04 PM Junio C Hamano wrote:
> Jim Garrison <jim.garrison@nwea.org> writes: > > >> -----Original Message----- > >> From: Junio C Hamano > >> Sent: Wednesday, May 07, 2014 1:16 PM > >> Subject: Re: Beginner question on "Pull is mostly evil" > >> > >> No. This is most often true for people who use a single repository as a > >> place for everybody to meet, in the same way as SVN. > > [snip lots of excellent detail] > >> HTH. > > > > Wow. That helps tremendously, and should be incorporated somewhere in the > > Git documentation. Thank you for your immensely detailed response. > > We used to collect useful list postings in Documentation/howto/; > perhaps somebody wants to do the minimum copyediting of the message > and send a patch? > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] How to keep a project's canonical history correct.
During the mail thread about "Pull is mostly evil" a user asked how the first parent could become reversed.
This howto explains how the first parent can get reversed when viewed by the project and then explains a method to keep the history correct.
Signed-off-by: Stephen P. Smith <ischis2@cox.net> --- Documentation/Makefile | 1 + .../howto/keep-canonical-history-correct.txt | 213 +++++++++++++++++++++ 2 files changed, 214 insertions(+) create mode 100644 Documentation/howto/keep-canonical-history-correct.txt
diff --git a/Documentation/Makefile b/Documentation/Makefile index fc6b2cf..cea0e7a 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -59,6 +59,7 @@ SP_ARTICLES += howto/recover-corrupted-blob-object SP_ARTICLES += howto/recover-corrupted-object-harder SP_ARTICLES += howto/rebuild-from-update-hook SP_ARTICLES += howto/rebase-from-internal-branch +SP_ARTICLES += howto/keep-canonical-history-correct SP_ARTICLES += howto/maintain-git API_DOCS = $(patsubst %.txt,%,$(filter-out technical/api-index-skel.txt technical/api-index.txt, $(wildcard technical/api-*.txt))) SP_ARTICLES += $(API_DOCS) diff --git a/Documentation/howto/keep-canonical-history-correct.txt b/Documentation/howto/keep-canonical-history-correct.txt new file mode 100644 index 0000000..927154c --- /dev/null +++ b/Documentation/howto/keep-canonical-history-correct.txt @@ -0,0 +1,213 @@ +From: Junio C Hamano <gitster@pobox.com> +Date: Wed, 07 May 2014 13:15:39 -0700 +Subject: Beginner question on "Pull is mostly evil" +Abstract: This how-to explains a method for keeping a + project's history correct when using git pull. +Content-type: text/asciidoc + +Keep authoritative canonical history correct with git pull +========================================================== + +Sometimes a new project integrator will end up with project history +that appears to be "backwards" from what other project developers +expect. This howto presents a suggested integration workflow for +maintaining a central repository. + +Suppose that that central repository has this history: + +------------ + ---o---o---A +------------ + +which ends at commit `A` (time flows from left to right and each node +in the graph is a commit, lines between them indicating parent-child +relationship). + +Then you clone it and work on your own commits, which leads you to +have this history in *your* repository: + +------------ + ---o---o---A---B---C +------------ + +Imagine your coworker did the same and built on top of `A` in *his* +repository in the meantime, and then pushed it to the +central repository: + +------------ + ---o---o---A---X---Y---Z +------------ + +Now, if you "git push" at this point, beause your history that leads +to `C` lacks `X`, `Y` and `Z`, it will fail. You need to somehow make +the tip of your history a descendant of `Z`. + +One suggested way to solve the problem is "fetch and then merge", aka +"git pull". When you fetch, your repository will have a history like +this: + +------------ + ---o---o---A---B---C + \ + X---Y---Z +------------ + +Once you run merge after that, while still on *your* branch, i.e. `C`, +you will create a merge `M` and make the history look like this: + +------------ + ---o---o---A---B---C---M + \ / + X---Y---Z +------------ + +`M` is a descendant of `Z`, so you can push to update the central +repository. Such a merge `M` does not lose any commit in both +histories, so in that sense it may not be wrong, but when people want +to talk about "the authoritative canonical history that is shared +among the project participants", i.e. "the trunk", the way they often +use is to do: + +------------ + $ git log --first-parent +------------ + +For all other people who observed the central repository after your +coworker pushed `Z` but before you pushed `M`, the commit on the trunk +used to be `o-o-A-X-Y-Z`. But because you made `M` while you were on +`C`, `M`'s first parent is `C`, so by pushing `M` to advance the +central repository, you made `X-Y-Z` a side branch, not on the trunk. + +You would rather want to have a history of this shape: + +------------ + ---o---o---A---X---Y---Z---M' + \ / + B-----------C +------------ + +so that in the first-parent chain, it is clear that the project first +did `X` and then `Y` and then `Z` and merged a change that consists of +two commits `B` and `C` that achieves a single goal. You may have +worked on fixing the bug #12345 with these two patches, and the merge +`M'` with swapped parents can say in its log message "Merge +'fix-bug-12345'". Having a way to tell "git pull" to create a merge +but record the parents in reverse order may be a way to do so. + +Note that I said "achieves a single goal" above, because this is +important. "swapping the merge order" only covers a special case +where the project does not care too much about having unrelated +things done on a single merge but cares a lot about first-parent +chain. + +There are multiple schools of thought about the "trunk" management. + + 1. Some projects want to keep a completely linear history without any + merges. Obviously, swapping the merge order would not match their + taste. You would need to flatten your history on top of the + updated upstream to result in a history of this shape instead: ++ +------------ + ---o---o---A---X---Y---Z---B---C +------------ ++ + with `git pull --rebase` or something. + + 2. Some projects tolerate merges in their history, but do not worry + too much about the first-parent order, and allow fast-forward + merges. To them, swapping the merge order does not hurt, but + it is unnecessary. + + 3. Some projects want each commit on the "trunk" to do one single + thing. The output of `git log --first-parent` in such a project + would show either a merge of a side branch that completes a single + theme, or a single commit that completes a single theme by itself. + If your two commits `B` and `C` (or they may even be two groups of + commits) were solving two independent issues, then the merge `M'` + we made in the earlier example by swapping the merge order is + still not up to the project standard. It merges two unrelated + efforts `B` and `C` at the same time. + +For projects in the last category (Git itself is one of them), +individual developers would want to prepare a history more like +this: + +------------ + C0--C1--C2 topic-c + / + ---o---o---A master + \ + B0--B1--B2 topic-b +------------ + +That is, keeping separate topics on separate branches, perhaps like +so: + +------------ + $ git clone $URL work && cd work + $ git checkout -b topic-b master + $ ... work to create B0, B1 and B2 to complete one theme + $ git checkout -b topic-c master + $ ... same for the theme of topic-c +------------ + +And then + +------------ + $ git checkout master + $ git pull --ff-only +------------ + +would grab `X`, `Y` and `Z` from the upstream and advance your master +branch: + +------------ + C0--C1--C2 topic-c + / + ---o---o---A---X---Y---Z master + \ + B0--B1--B2 topic-b +------------ + +And then you would merge these two branches separately: + +------------ + $ git merge topic-b + $ git merge topic-c +------------ + +to result in + +------------ + C0--C1---------C2 + / \ + ---o---o---A---X---Y---Z---M---N + \ / + B0--B1-----B2 +------------ + +and push it back to the central repository. + +It is very much possible that while you are merging topic-b and +topic-c, somebody again advanced the history in the central repository +to put `W` on top of `Z`, and make your "git push" fail. + +In such a case, you would rewind to discard `M` and `N`, update the +tip of your 'master' again and redo the two merges: + +------------ + $ git reset --hard origin/master + $ git pull --ff-only + $ git merge topic-b + $ git merge topic-c +------------ + +------------ + C0--C1--------------C2 + / \ + ---o---o---A---X---Y---Z---W---M'--N' + \ / + B0--B1---------B2 +------------ + +See http://git-blame.blogspot.com/2012/03/fun-with-first-parent.html
-- 2.0.0.rc1
Re: [PATCH] How to keep a project's canonical history correct.
> On May 8, 2014, at 11:08 PM, "Stephen P. Smith" <ischis2@cox.net> wrote: > > During the mail thread about "Pull is mostly evil" a user asked how > the first parent could become reversed. > > This howto explains how the first parent can get reversed when viewed > by the project and then explains a method to keep the history correct. > > Signed-off-by: Stephen P. Smith <ischis2@cox.net> > ---
The reason I resubmitted as a brand new patch was because I was using the message ID from the original topic as requested.
In my repository I did a "git merge --squash" then the format patch.
Junio C HamanoRe: [PATCH] How to keep a project's canonical history correct.
"Stephen P. Smith" <ischis2@cox.net> writes:
> During the mail thread about "Pull is mostly evil" a user asked how > the first parent could become reversed. > > This howto explains how the first parent can get reversed when viewed > by the project and then explains a method to keep the history correct. > > Signed-off-by: Stephen P. Smith <ischis2@cox.net> > ---
I needed a few tweaks on top while queuing. You will find the result on 'pu' after I push it out.
In addition to one typofix ("because" lacking "c"), here are what I did:
- Typeset concrete command e.g. `git pull` in monospace.
- The second and subsequent paragraphs continued with "+" need to be flushed to the left; leaving them indented will format them in monospace (see "with `git pull --rebase` or something").
- Be more explicit in describing 'trunk' being 'the first-parent chain' in the text.
- Refer to a newer article that discusses this exact topic.
- De-emphasize 'fix-bug-12345' in "Merge fix-bug-12345" log message.
- Describe what the final history illustration shows.
Unless you have objections to the below (or suggestions for better alternatives), there is no need to resend the patch.
Thanks.
diff --git a/Documentation/howto/keep-canonical-history-correct.txt b/Documentation/howto/keep-canonical-history-correct.txt index 5979a79..35d48ef 100644 --- a/Documentation/howto/keep-canonical-history-correct.txt +++ b/Documentation/howto/keep-canonical-history-correct.txt @@ -38,12 +38,12 @@ central repository: ---o---o---A---X---Y---Z ------------ -Now, if you "git push" at this point, beause your history that leads +Now, if you `git push` at this point, because your history that leads to `C` lacks `X`, `Y` and `Z`, it will fail. You need to somehow make the tip of your history a descendant of `Z`. One suggested way to solve the problem is "fetch and then merge", aka -"git pull". When you fetch, your repository will have a history like +`git pull`. When you fetch, your repository will have a history like this: ------------ @@ -65,8 +65,9 @@ you will create a merge `M` and make the history look like this: repository. Such a merge `M` does not lose any commit in both histories, so in that sense it may not be wrong, but when people want to talk about "the authoritative canonical history that is shared -among the project participants", i.e. "the trunk", the way they often -use is to do: +among the project participants", i.e. "the trunk", they often view +it as "commits you see by following the first-parent chain", and use +this command to view it: ------------ $ git log --first-parent @@ -91,11 +92,11 @@ did `X` and then `Y` and then `Z` and merged a change that consists of two commits `B` and `C` that achieves a single goal. You may have worked on fixing the bug #12345 with these two patches, and the merge `M'` with swapped parents can say in its log message "Merge -'fix-bug-12345'". Having a way to tell "git pull" to create a merge +fix-bug-12345". Having a way to tell `git pull` to create a merge but record the parents in reverse order may be a way to do so. Note that I said "achieves a single goal" above, because this is -important. "swapping the merge order" only covers a special case +important. "Swapping the merge order" only covers a special case where the project does not care too much about having unrelated things done on a single merge but cares a lot about first-parent chain. @@ -111,7 +112,7 @@ There are multiple schools of thought about the "trunk" management. ---o---o---A---X---Y---Z---B---C ------------ + - with `git pull --rebase` or something. +with `git pull --rebase` or something. 2. Some projects tolerate merges in their history, but do not worry too much about the first-parent order, and allow fast-forward @@ -190,7 +191,7 @@ and push it back to the central repository. It is very much possible that while you are merging topic-b and topic-c, somebody again advanced the history in the central repository -to put `W` on top of `Z`, and make your "git push" fail. +to put `W` on top of `Z`, and make your `git push` fail. In such a case, you would rewind to discard `M` and `N`, update the tip of your 'master' again and redo the two merges: @@ -202,6 +203,8 @@ tip of your 'master' again and redo the two merges: $ git merge topic-c ------------ +The procedure will result in a history that looks like this: + ------------ C0--C1--------------C2 / \ @@ -210,4 +213,4 @@ tip of your 'master' again and redo the two merges: B0--B1---------B2 ------------ -See http://git-blame.blogspot.com/2012/03/fun-with-first-parent.html +See also http://git-blame.blogspot.com/2013/09/fun-with-first-parent-history.html
Re: [PATCH] How to keep a project's canonical history correct.
On Friday, May 09, 2014 02:05:44 PM Junio C Hamano wrote:
> I needed a few tweaks on top while queuing. You will find the
> result on 'pu' after I push it out.
>
> In addition to one typofix ("because" lacking "c"), here are what I
> did:
>
> - Typeset concrete command e.g. `git pull` in monospace.
>
> - The second and subsequent paragraphs continued with "+" need to
> be flushed to the left; leaving them indented will format them in
> monospace (see "with `git pull --rebase` or something").
>
> - Be more explicit in describing 'trunk' being 'the first-parent
> chain' in the text.
>
> - Refer to a newer article that discusses this exact topic.
>
> - De-emphasize 'fix-bug-12345' in "Merge fix-bug-12345" log message.
>
> - Describe what the final history illustration shows.
>
>
> Unless you have objections to the below (or suggestions for better
> alternatives), there is no need to resend the patch.
> I like the changes.