threads / discuss / 27983

Branch dependencies

Subject: Branch dependencies

## tl;dr

8 messages between Aug 1, 2011 and Aug 4, 2011.

replies: 7people: 3as markdown or json

martin f krafft· Aug 1, 2011, 12:19 UTC · lore
Dear list,

We are trying to approach a functionality I call "branch dependencies". Essentially, the idea is rooted in distro development, but probably applies to normal development too.

For instance, you might have a feature branch off upstream. Once upstream advances, you should merge upstream into your feature branch (or rebase your feature branch) to ensure that you are working on the right assumptions.

Another case might be a feature you want to write, which depends on two (or more) feature branches of other people, e.g. say you need a new configuration option introduced in feature branch "conf-option", and you also base your work on the "speedup" branch, because otherwise the software is too slow for your new feature. If one or both feature branches advance, you should merge, as before.

It would be useful if Git could help you keep track of what needs to be merged, especially as the number of feature branches and dependencies increases. TopGit was Petr's answer to this challenge, and it works fine, albeit it's a bit too complex and we find it scaring new contributors, rather than making their lives easier.

Therefore I am investigating ways in which to simplify/improve TopGit. In doing so, I discovered that you guys made a lot of progress in Git since the last time I had time to really dive into your tool.

If you would permit me, then please let me ask if you can think of Git functionality that could be useful in achieving what we're trying to do.

For instance, there is git-branch --set-upstream, which could be useful, but it only seems to support one "dependency" (which is usually the remote ref being tracked by a branch).

One challenge seems to be that a branching point has no information about which of the children continues as mainline — this information is only available in a project's workflow policy. For instance:

  o--o--o--●    upstream
      \
       o--●     feature
But his is actually just the same as
      ,o--●     feature
  o--o
      `o--●     upstream

and Git has no way to find out whether it is now "feature" that needs a merge of "upstream" or vice versa.

It is thus necessary somehow to store the (project-specific) dependency information, to be able to (automatically) determine that "feature" needs an update in the above.

TopGit does this using a file in the worktree, but many of us find this suboptimal.

I have had the following alternative ideas:
  1. a separate DAG, like Git notes. The problem is that this
     requires additional refspecs to be set up for merges and
     fetches;
  2. like (1.), but a ref in refs/heads/* (like pristine-tar). This
     could be considered ugly as it exposes too much implementation
     detail;
  3. information stored in the Git commit messages. Again, too much
     implementation detail exposed and ugly;
  4. additional Git commit headers — this is not supported at the
     moment (cf. commit generation discussion);
  5. orphan parent nodes to certain commits, in which these data can
     be stored, e.g.
       o--o--o--●
            /
           o
     To fetch these data, one would walk up the DAG until one finds
     a multi-parent commit with a parent having a specific format
     (somewhat brittle…)
     To me, this is the least offensive, but it does expose
     implementation details in the commit history.

How else could I store the dependency information, keeping in mind that I might have more than one dependency?

And the original question remains: given such dependency information, which Git tools could I harness for the purpose, trying to reduce the amount of additional code needed?

Thanks,
-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
it is better to have loft and lost
than to never have loft at all.
                                                       -- groucho marx
 
spamtraps: madduck.bogus@madduck.net
Bert Wesarg· Aug 2, 2011, 13:06 UTC · re: martin f krafft · lore

Re: Branch dependencies

Hi,
On Mon, Aug 1, 2011 at 14:19, martin f krafft <madduck@madduck.net> wrote:
> Dear list,

while I appreciate, that you dig this topic up. I think you are trying to solve the wrong problem first. My main problem with the TopGit approach is, that you can't freely change the dependencies of a topic. This may be not the most common case in distro development. But in my eyes more problematic than maintaining the meta data.

Please note, that I'm more than aware the the TopGit approach for handling the meta data is awful and we need a new way here. My personal impression is, that the git notes is the best we can have.

For my first mentioned problem, I think a new 'system' needs to be 'rebase' based, not merge based like TopGit.

Bert
martin f krafft· Aug 2, 2011, 19:08 UTC · re: Bert Wesarg · lore

Re: Branch dependencies

also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.02.1506 +0200]:
Show 5 quoted lines
> while I appreciate, that you dig this topic up. I think you are trying
> to solve the wrong problem first. My main problem with the TopGit
> approach is, that you can't freely change the dependencies of a topic.
> This may be not the most common case in distro development. But in my
> eyes more problematic than maintaining the meta data.
Hello Bert, thank you for taking the time to respond!

Could you please try to illuminate me a bit on a use-case of changing dependencies? I am aware that TopGit has had a problem with changing dependencies due to renamed branches, and I think I have a solution to that (encode the dependent ref, not the branch head), but I cannot come up with a use case for freely changing dependencies just like that.

> For my first mentioned problem, I think a new 'system' needs to be
> 'rebase' based, not merge based like TopGit.
The problem with rebasing is that you cannot publish the branches.

However, maybe I am simply not seeing the light here. Do you have some further ideas about what this would be like? Please keep in mind that what I seek is not just a way to bring feature branches up-to-date with upstream, but also to have those branches be shared among developers.

Thanks,
-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"gott ist tot! und wir haben ihn getötet."
                                                 - friedrich nietzsche
 
spamtraps: madduck.bogus@madduck.net
Bert Wesarg· Aug 2, 2011, 23:25 UTC · re: martin f krafft · lore

Re: Branch dependencies

Hi,
On Tue, Aug 2, 2011 at 21:08, martin f krafft <madduck@madduck.net> wrote:
Show 15 quoted lines
> also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.02.1506 +0200]:
>> while I appreciate, that you dig this topic up. I think you are trying
>> to solve the wrong problem first. My main problem with the TopGit
>> approach is, that you can't freely change the dependencies of a topic.
>> This may be not the most common case in distro development. But in my
>> eyes more problematic than maintaining the meta data.
>
> Hello Bert, thank you for taking the time to respond!
>
> Could you please try to illuminate me a bit on a use-case of
> changing dependencies? I am aware that TopGit has had a problem with
> changing dependencies due to renamed branches, and I think I have
> a solution to that (encode the dependent ref, not the branch head),
> but I cannot come up with a use case for freely changing
> dependencies just like that.

Not each feature branch may end up in master. And there does not need to be one feature branch which depends on all other features. For the latter you have probably an empty feature branch which just depends on all features. I call this branch mostly 'tip'. Removing a feature branch from the tips dependency list only with merges can't be done right now, and the proposed solutions never reached a usable state.

My second usecase is to convert a big quilt patch series into TopGit. Such big Quilt patches have mostly an artificial dependency to its predecessors. Removing these artifical dependencies makes it necessary to remove dependencies from patches.

Show 5 quoted lines
>
>> For my first mentioned problem, I think a new 'system' needs to be
>> 'rebase' based, not merge based like TopGit.
>
> The problem with rebasing is that you cannot publish the branches.

That doesn't hold you back to publish them. But the other side need to know how to deal with them.

Saying that doesn't mean I know a good way to deal with them. I mostly end up using plumbing commands to deal with this.

Show 6 quoted lines
>
> However, maybe I am simply not seeing the light here. Do you have
> some further ideas about what this would be like? Please keep in
> mind that what I seek is not just a way to bring feature branches
> up-to-date with upstream, but also to have those branches be shared
> among developers.

I think that having the TopGit philosophy of one feature branch is one patch, you can handle an rebased upstream. Thinking of a feature branch as a series of patches makes this way harder. But I would like to have this philosophy.

Bert
>
> Thanks,
>
martin f krafft· Aug 3, 2011, 09:51 UTC · re: Bert Wesarg · lore

changing the set of dependencies (was: Branch dependencies)

also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.03.0125 +0200]:
Show 11 quoted lines
> Not each feature branch may end up in master. And there does not need
> to be one feature branch which depends on all other features. For the
> latter you have probably an empty feature branch which just depends on
> all features. I call this branch mostly 'tip'. Removing a feature
> branch from the tips dependency list only with merges can't be done
> right now, and the proposed solutions never reached a usable state.
> 
> My second usecase is to convert a big quilt patch series into TopGit.
> Such big Quilt patches have mostly an artificial dependency to its
> predecessors. Removing these artifical dependencies makes it necessary
> to remove dependencies from patches.
Dear Bert, thank you for your reply.

Okay, understood. Yes, I agree entirely, the set of dependencies needs to be mutable. And they must not be invalidated by branch renames. Therefore, we really need some sort of other way to identify branches.

Is there a way other than lamenting that refs did not get UUIDs assigned to them from the early days onwards? ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"if one cannot enjoy reading a book over and over again,
 there is no use in reading it at all."
                                                        -- oscar wilde
 
spamtraps: madduck.bogus@madduck.net
martin f krafft· Aug 3, 2011, 10:01 UTC · re: Bert Wesarg · lore

TopGit with rebased branches, problems with publishing (was: Branch dependencies)

also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.03.0125 +0200]:
Show 10 quoted lines
> >> For my first mentioned problem, I think a new 'system' needs to be
> >> 'rebase' based, not merge based like TopGit.
> >
> > The problem with rebasing is that you cannot publish the branches.
> 
> That doesn't hold you back to publish them. But the other side need to
> know how to deal with them.
> 
> Saying that doesn't mean I know a good way to deal with them. I mostly
> end up using plumbing commands to deal with this.

Let me address this in terms of patch queue branches. Obviously, you are talking topic branches, but wrt rebase-publish, the two are identical. Patch queue branches are essentially quilt imports, e.g. one-commit-one-patch, usually edited with git rebase -i (which some people consider to be the better quilt).

The problem that led me away from patch queue branches is that — albeit unlikely in our context (distro packaging) — it is possible that you and I both rewrite the patch queue at the same time. The first one to push will then have his changes overwritten by the second push — in order to push the rebased ref, receive.denyNonFastForwards needs to be off.

Unfortunately, I can't see a remedy to this, apart from tagging each and every patch queue branch, but that does not solve the problem of having to merge changes made simultaneously. It could go something like this:

  1. ensure that noone has pushed an updated ref since you last
     fetched.
  2. if a new ref is found, download it and rebase your own patch
     queue on top of it, consolidating the changes.
  3. tag and push your new ref.

But there's a race condition in this, and so we're back to square one.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
uʍop ǝpısdn sı ɹoʇıuoɯ ɹnoʎ
 
spamtraps: madduck.bogus@madduck.net
martin f krafft· Aug 3, 2011, 10:10 UTC · re: Bert Wesarg · lore

Tracking topic branches with rebases vs. merges (was: Branch dependencies)

also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.03.0125 +0200]:
> I think that having the TopGit philosophy of one feature branch is
> one patch, you can handle an rebased upstream. Thinking of
> a feature branch as a series of patches makes this way harder. But
> I would like to have this philosophy.

Let me just make sure I understand you right: you do not like the following branch management strategy:

       o--o--o--+--o--o--+--o-.
      /        /        /      \
  o--o--o--o--o--o--o--o--o--o--o--●

and you would prefer if the topic branch was rebased all along (like what git.git advocates), and then transferred to mainline with git-format-patch/git-send-e-mail/git-am (or ff-merged).

I am torn on this issue. On the one hand, I do not find the above to be so problematic, especially not if the downstream merges happen only infrequently (undo merges that do not produce conflicts, as advocated by gitworkflows(7)).

On the other hand, I completely agree with you that rebasing is much nicer, and it certainly works without publishing the branch. In git.git, people seem to maintain their own branches and send patch sets to the mailing list for review.

But how could you and I truly cooperate on a feature branch without using merges, and without establishing a lock-unlock protocol to ensure only sequential updates of the refs?

Thanks,
-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"alles gackert, aber wer will noch still
 auf dem nest sitzen und eier zu brüten?"
                                      -- friedrich wilhelm nietzsche
 
spamtraps: madduck.bogus@madduck.net
Nicolas Sebrecht· Aug 4, 2011, 17:38 UTC · re: martin f krafft · lore

Re: Branch dependencies

The 02/08/11, martin f krafft wrote:
> also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.02.1506 +0200]:
> > For my first mentioned problem, I think a new 'system' needs to be
> > 'rebase' based, not merge based like TopGit.
> 
> The problem with rebasing is that you cannot publish the branches.

But you may make public the way to set up the builded branch. Say you share A and B branches and you build M branch on top of them. You are able to build M as long as others know M rely on A and B.

In a situation like
      a--+       (A)
         M--y--z (M)
      b--+       (B)
or
   a--+          (A)
      b--+       (B)
         M--y--z (M)
M is the builded branch. Commits y and z should not be public, IMHO.
-- 
Nicolas Sebrecht

← back to recent threads