How does Git's maintenance policy handle topics that don't start from "master?"

I've read the /Addendum to "MaintNotes"/ document¹ several times in the last few years, but in the process of trying to employ the policy with my current team, our progress is stuck on a case that isn't addressed by the policy -- directly, anyway.

In the policy section "Handle the remaining patches," the first clause reads as follows:

,----[ First case for remaining patches ]
| Anything unobvious that is applicable to 'master' (in other
| words, does not depend on anything that is still in 'next'
| and not in 'master') is applied to a new topic branch that
| is forked from the tip of 'master'.  This includes both
| enhancements and unobvious fixes to 'master'.
`----

It addresses topics that can be built on top of the "master" branch, these topics not depending on anything only available outside the "master" branch, such as in the "next" branch. This policy is focusing on the receiver and integrator of patches, rather than the author, but it's not hard to infer that an author should start his work from the "master" branch in order for his patches to be eligible for treatment by this clause.

What about the case where an author started his work from the "next" branch instead? He may have submitted an earlier batch of work that's still cooking in "next," and now he needs to build something else that can take advantage of that earlier work. It's clear that if he starts from "next" and relies on that earlier work, then his later work is not independent and cannot possibly graduate to the "master" branch unless and until his earlier work graduates too.

Is the Git policy on such dependency simply, "Don't do that?"

Consider a situation where the earlier topic branch's contribution cooking in "next" is looking good and everyone is feeling confident that it's going to graduate, and our poor author /needs/ to get started on his next task that would make use of the earlier work. If he does start his new topic branch from "next" -- or maybe starts it from his earlier topic branch instead -- what will go wrong later? Is there a part of the policy that addresses this case that I missed?

Footnotes: 
¹ http://www.kernel.org/pub/software/scm/git/docs/v1.7.10.1/howto/maintain-git.txt

"Steven E. Harris" <seh@panix.com> writes:

After posting I discovered the "SubmittingPatches" document¹, which does address part of my inquiry:

,----[ Decide what to base your work on ]
| - A new feature should be based on 'master' in general. If the new
|    feature depends on a topic that is in 'pu', but not in 'master',
|    base your work on the tip of that topic.
| 
| - Corrections and enhancements to a topic not yet in 'master' should
|    be based on the tip of that topic. If the topic has not been merged
|    to 'next', it's alright to add a note to squash minor corrections
|    into the series.
| 
| - In the exceptional case that a new feature depends on several topics
|    not in 'master', start working on 'next' or 'pu' privately and send
|    out patches for discussion. Before the final merge, you may have to
|    wait until some of the dependent topics graduate to 'master', and
|    rebase your work.
`----

The last one in particular implies that the topic branch will not be fit for serious consideration until its dependencies have themselves graduated to "master."

Footnotes: 
¹ https://raw.github.com/git/git/master/Documentation/SubmittingPatches

"Steven E. Harris" <seh@panix.com> writes:

It often is clear that the follow-on topic depends on an earlier topic branch (mostly because the contributor is aware of it and state it in the message). An obvious thing to do in such a case is to create a new branch to queue that topic starting at the tip of the earlier topic. Note that this is never from the tip of "next", as it is very unlikely that such a follow-on topic depends on everything that is not in "master" yet.

Sometimes a new work depends on multiple topics that are still cooking, and *all* of these topics that the new work depends on are in good shape. In such a case, I create a new branch by merging these prerequisite topics and then queue new work there. Obviously the new work is taken hostage to *all* of its dependent topics, and cannot graduate until all of the base topics graduate.

Sometimes a new work depends on one topic that is still cooking in "next", *and* also needs updates made by other topics that are already in "master". You can guess what should happen---take the tip of that topic that is still cooking in "next", merge the commit on "master" that adds other necessary bits, and then that becomes the base of the new topic. The "commit that adds other necessary bits" could be the tip of "master" (easiest for me, but it makes the new topic unmergeable to "maint" later) or the tip of an old topic that was merged to "master" (more work for me, but it is worth if both the other old topic and the topic that is cooking in "next" are meant to be merged to "maint" later, and the new work is also meant to eventually be merged to "maint").

In a rare cases where a new work depends on millions of uncooked topics, we simply reject the follow-on series and tell the submitter to wait until the dust settles, but in practice it does not happen very often.

In other words, the "policy" is not a mechanical recipe to be followed by brainless monkeys; the integrator needs to follow the common sense of keeping the resulting topic branch mergeable to as many relevant contexts as necessary.

And the contributor can help in this process, as well.

Junio C Hamano <gitster@pobox.com> writes:

Thank you for the thorough reply.

I want to make sure that I understand the main hazard that the policy aims to avoid. If an author bases a topic branch on the tip of "next," then, later, it will not be possible to merge his work to "master" without implicitly accepting all of "next" (or, at least the part of "next" the precedes this topic branch) into "master." We are trying to avoid merging "upward" from "next" to "master" like that, and prefer to take the topics to "master" directly rather than implicitly by way of their inclusion in "next."

What isn't so clear to me, though, is /why/ this don't-merge-from-"next" rule is so important. Say that we had one topic "t1" depart from "next," and then another topic "t2" depart from "t1," and both have been cooking in "next," with good results.

  ---o---o---o---o  master
                  \
                   o---o---o---o---o---M---o---o next
                        \     /       /
                         o---o t1    /
                              \     /
                               o---o t2

If we wanted to graduate these two topics to "master," we /could/ merge from commit M back to "master," though here I deliberately included the nefarious commit X, which shows other interleaved contributions along "next" that are also part of the M commit.

What about this case, where topics "t1" and "t2" did depart from "master," and are doing well along "next" together as of commit M.

  ---o---o---o---o  master
      \   \       \
       \   o---o---o---M---o---o next
        \     /       /
         o---o t1    /
          \         /
           o---o---o t2

The Git policy as I understand it prescribes that we merge from the tips of "t1" and "t2" back to master, not from a commit like M. What harm would come from merging from M in this case? Future archaeology of topic provenance?

"Steven E. Harris" <seh@panix.com> writes:

I do not see any X above, but I think you meant the commits in master..M that are not reachable from either t1 and t2.

And I think you answered your own question. These "master..M ^t1 ^t2" commits are topics that are *not* part of t1 nor t2. If you deem all of them are good enough for your master, it is perfectly fine to merge M to master. In reality, it is more cumbersome to think about what is and what is not yet in M and decide if the set of changes that happen to be in M match exactly what you want to merge, than knowing that you exactly want to have t1 and t2 and nothing else in your master during this integration run and merge only these two topics.

The same answer to the other picture. If you can figure out an appropriate commit M that has what you want, go right ahead and merge that to 'master'; I do not see any harm there. I personally do not think it is worth the effort to figure out which commit between master..next that M is, and verify master..M contains everything you want and nothing you don't. Merging t1 and t2 explicitly, when you know they are the only thing you want to merge, is much simpler and less error prone.

Junio C Hamano <gitster@pobox.com> writes:

You would also have to think about what you write to explain that merge of M into 'master' in the commit log message. In the above special case where 'next' happened to have had only t1 and t2 when you decided to merge to 'master', you could say "Merge t1 and t2 to achieve X", but in the earlier example (elided), it is not clear what random bits you are merging with it.

Compared to that, if you merge t1 and t2 separately, you can say "I am merging t1 topic that does X", and readers of "git log master" (better yet "git log --first-parent master") would understand that the master branch after that commit can do X (and everything before that commit does not).