Questions about branches in git

Hi, my company switched to git a few months ago because the way it handles submodules seems safer to us than our previous scm tool and because our ruby developers wanted to take advantage of the community on github. However, I'm having some problems getting git's representations of branches to show me what change sets they contain. It seems that after a topic or bug branch is merged back into its parent, especially if it was fast forwarded, it becomes hard to determine what changes were made in it, to resolve the problem that it was created to address. This is fairly important to me since I need to be able to backport fixes to older revisions on occasion, and to perform development on multiple releases for multiple platforms in parallel, so it seems really handy for a branch to show us just the changes that were made in it, between the time it was spawned from its parent and the time the parent accepted its changes.

I've looked through as much documentation as I can find about git show and git log, and I've played around with git rebase to try to apply changes to multiple places, but I have not been able to find a way to display the commits relevant to a particular bug/topic branch. Rebasing from a branch that has already had the changeset merged back in, to another branch, seems to actually wipe out the contents of the branch completely.

I understand that there are mechanism kind of available to address this problem. If we (all developers in my company) remember always to rebase -i before they merge their topic branches back in, then it could be squashed making it easier to identify and cherry pick onto other branches, or *if* we always remember to rebase before we merge and then create a patch set and store that on the topic branch, we could kind of organize our change sets that way. But it seems that it should be easier than that, shouldn't it? If I look at the git log for a branch, I really feel that I should see some distinction between that changes that were originally made on a branch, and the ones that were inherited from some other branch or merged in from some other branch. Simply because mistakes get made and if you we don't realize that a fix may need to be applied to some older revision when we first develop it, it seems that this could really cost a lot of time and money to try to identify the individual commits that were useful in addressing the problem.

What am I missing here?

Any help is appreciated.

Thank you,

Michael Linck

On Thu, Jan 28, 2010 at 12:44 PM, Mike Linck <mgl@absolute-performance.com> wrote:

For now, you should probably rely on graphical tools like gitk in order to visualize the various branches. There's also `git log --graph'. You could also just keep your branches around for reference and use `git merge-base' as necessary.

However, I've been thinking for a while that it would be useful to have übercommits (they don't exist) that are treated like single commits but that actually encapsulate multiple continguous commits.

For your case, you could tell git to make merges by creating such übercommits, which would then be easily identified, referenced, and manipulated for your backporting purposes; such übercommits would encapsulate the relevant commits.

These übercommits would also allow developers to make a string of commits that by themselves break things but together formulate a complete solution; because the übercommits encapsulate the breakage, bisection would still be simple (no fear of dealing with broken commits), but the small manageable commits would still be available for references and manipulation.

Perhaps trees could be reappropriated for the implementation of übercommits.

Sincerely, Michael Witten

On Thu, Jan 28, 2010 at 2:03 PM, Michael Witten <mfwitten@gmail.com> wrote:

In fact, the commit message body is already being used to create unofficial übercommits. Consider a common merge commit from a clone of Linus's Linux repo:

    commit e80b1359858df17b0034bdf7d1b6f3e0d5b97257
    Merge: 341031c b27d515
    Author: Linus Torvalds <torvalds@linux-foundation.org>
    Date:   Thu Jan 21 08:50:04 2010 -0800
    
        Merge branch 'perf-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
        
        * 'perf-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
          perf: x86: Add support for the ANY bit
          perf: Change the is_software_event() definition
          perf: Honour event state for aux stream data
          perf: Fix perf_event_do_pending() fallback callsite
          perf kmem: Print usage help for unknown commands
          perf kmem: Increase "Hit" column length
          hw-breakpoints, perf: Fix broken mmiotrace due to dr6 by reference change
          perf timechart: Use tid not pid for COMM change

It seems like this kind of useful information should be a more integral part of the metadata.

Indeed, it seems like commit messages are often used for metadata that git perhaps *should* handle natively, like sign-offs and multiple Authors, etc.

Of course, I'm betting that git doesn't handle such things officially because it would require more general data structures (especially for variable numbers of Authors) and thus slower algorithms.

Sincerely, Michael Witten

On Thu, Jan 28, 2010 at 2:03 PM, Michael Witten <mfwitten@gmail.com> wrote:

As a corollary to this, developers can maintain patch integrity.

Quite often, I've sent a patch off to some project only to have the maintainer `tweak' the result before making a commit. However, I frankly don't want my name attached to someone else's work, because I may disagree with what has been done.

Were übercommits available, the maintainer could commit my original work and then make a new `tweak' commit and then bundle the 2 together as an übercommit in order to encapsulate this series of events.

On Thu, Jan 28, 2010 at 1:03 PM, Michael Witten <mfwitten@gmail.com> wrote:

Well, even gitk can't show me the information I'm looking for if the parent branch ended up fast-forwarding to include the changes made in the topic branch. As far as I can tell there is *no way* to tell what changes were made in a particular branch after a fast-forward has taken place, which seems to make it hard to organize fixes for specific topics/bugs/tickets.

Yeah, what concerns me is that there seems to be no point in keeping your branches around "for reference" because even when you're looking at them you can't tell what changes were made in them after they were spawned, unless they haven't been merged into another branch yet. So it seems that a branch is only useful for merging once and unless the branch was squashed in the process of mergin, good luck identifying your change set for a particular topic.

I don't know if an uebercommit is necessary. But it would help seem like it would help if a branch knew about it's spawn point from its parent, and if you could use that to get git log to only show you the commits made to a branch after it was spawned. And if there were forms of operations like merge, or rebase that could act intelligently based on that information like "merge this branch into that branch, and by this branch I don't mean anything from this branch's parent branch" so that a fix that was developed on edge or master could be safely merged into an old branch without importing every other change on edge or master, and also the other way around.

I just looked at merge-base. It doesn't seem to address the problem. I grabbed an old topic branch from our repo which I knew was created from master and at some point merged back into master via fast-forward. I checked it out, I called "git merge base topic-id master", hoping that it would "output a commit which is reachable from both A and B through the parent relationship." Instead it seems to have modified the topic branch by fast forwarding it to the include all the changes up to the tip of master. Clearly not what I'm looking for.

Michael Linck

Am 28.01.2010 22:17, schrieb Mike Linck:

You could disable fast forward merges using the --no-ff option. Then git will always create a merge commit even if it could have done a fast forward. This can be enabled permanently for a branch with 'git config branch.master.mergeoptions "--no-ff"'. We use that at my dayjob to preserve the branches after merging.

On Thu, Jan 28, 2010 at 2:29 PM, Jens Lehmann <Jens.Lehmann@web.de> wrote:

OK, so what I'm getting is that if a developer forgot to disable fast-forward when they created a topic branch, and if the parent branch has been fast forwarded to include it, then you might as well just throw away the topic branch, is that correct?

Could anyone point me to a good book that actually describes the style of code management that git was intended to support? Because I'm finding this a bit baffling, to be honest. I thought it was intended to make the developers' side of code management easier to do, but it seems to me that they have to think a lot harder about what they're trying to accomplish, at least in this sort of case. I'm not trying to be rude, but I just feel that if I want to keep working with this tool, I have to rethink how the code is organized in a pretty fundamental way and I'd like to get as comprehensive of a guide as possible from someone who has adopted their tactics to it.

Thanks

Michael Linck

On Thu, Jan 28, 2010 at 02:38:14PM -0700, Mike Linck wrote:

If you want to enforce this you can use an update hook on the receivers side and check that a branch update can only be made to real merge commits.

The practise we use at $dayjob is that we prepackage git installations containing default values in /etc/gitconfig so its not easily forgotten.

As stated in a later message there is no such thing as the design goal for git. Its designed by practise which has made it so flexible that you can design you own.

cheers Heiko

Quoting Mike Linck <mgl@absolute-performance.com>

You may want to add the result of googling 

  "Fun with" site:gitster.livejournal.com

to the list of Git documents you read. "Fork from the oldest branch" is one of the techniques Junio teaches often and many of his other techiniques are built upon.

He not just teaches useful techniques but explains a lot about the reasoning behind them in his Git book. His blog articles have similar explanations on many topics I saw in his book but not in other places. It is a useful substitute until his book gets translated to English for people who don't read Japanese.

Nanako Shiraishi <nanako3@lavabit.com> writes:

Quite honestly, I don't think my blog articles are all that good; certainly not as good as the book, as I don't draw pictures.

For this particular topic, I would instead recommend:

    http://nvie.com/archives/323 (A successful Git branching model)

What it teaches isn't anything earth-shattering (it's the same old "how to use topic branch effectively" and I don't necessarily agree with the choice of fork points of topic branches depicted in the article), but people seem to like its graphics very much and it is scoring quite high in delicio.us.

On Thu, 28 Jan 2010, Mike Linck wrote:

You should consider using tags in conjunction with your bugs/tickets system. The fork point for a bug fix may be tagged, as well as the last commit representing the bugfix completion (not the merge point though). This way you can always retrieve the exact set of commits forming up that bugfix, regardless if it was merged back into the main branch with a fast forward or not.

Nicolas

I agree with Nicolas here - i was also thinking about using lightweight tags in this case. If you really, really need to port changes to multiple branches, tags would show you exactly which commits you should work with.

git log tag1(branched)..tag2(brnach_ready)

In addition to that i shold say that most of the time branches are supposed to be created from the latest stable master, i.e. released code. Each release should be tagged. So, in this case you don't need to have to have the first tag, as you branching from a tagged commit. As soon as you have first point, second tag may be not necessary until you can operate with the last commit. As soon as it is not possible - you can create lightweight tag.

On Thu, Jan 28, 2010 at 5:04 PM, Nicolas Pitre <nico@fluxnic.net> wrote:

On Thu, Jan 28, 2010 at 05:04:36PM -0500, Nicolas Pitre wrote:

Tags, combined with --no-ff, should help you out a bit. If you're worried about devs forgetting to configure the no-ff then you might be able to help them out if you have any control over /etc/gitconfig on their systems. That gives you a standard, global way to set defaults.

This table gives a great summary of 'git log' commands for inspecting branches.

http://book.git-scm.com/3_reviewing_history_-_git_log.html

As far as "what's the way to do branches right in git" then there is no "one single way" because git is a framework upon which you can build your ideal workflow. That said, there are some very good examples to follow. For example, there is much that can be learned by studying how git.git's branches are managed.

http://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html

This webcast covers a few more workflows and is a very good crash course:

http://www.gitcasts.com/posts/railsconf-git-talk

On Thu, Jan 28, 2010 at 3:17 PM, Mike Linck <mgl@absolute-performance.com> wrote:

As Jens Lehmann pointed out, use something like:

    git checkout master
    git pull --no-ff . topic

I would think that you'd only care about the contiguous commits between merges anyway.

You incorrectly used `git merge' rather than `git merge-base'.

This is kind of off the top of my head. Try something like this:

    merged_commit_0=$(git merge-base master topic-id)
    merged_commit_1=$(git merge-base master ${merged_commit_0}^)

I think that should give you the range of commits between the last 2 merges (for at least simple cases). Then:

    git log $merged_commit_1^..$merged_commit_0

or

    gitk $merged_commit_1..$merged_commit_0

to see them.

You could, I suppose, keep looping until you find the oldest merge-base that is still in the topic-id branch. To do so, the following information may be of use:

    http://marc.info/?l=git&m=126457707700573&w=2

Anyway, it's probably best to use Nicolas Pitre's suggestion to use tags to mark commits yourself, but the above might be useful if you haven't.

On Thu, Jan 28, 2010 at 3:18 PM, Michael Witten <mfwitten@gmail.com> wrote:

Well, I'll be using tags more in the future and I'll look through some more documentation about recommended workflows and see if I can work out something that will do what we need to do.

Your example will work on non-FF branches if you make the second merge-base use master~1 (else it just gives you the same commit sha twice)

The tough thing to identify is the change sets that got fast-forwarded in though. I guess I'll just go ahead and erase those branches since they won't do any of us any good anyway and I'll read up on the work flow, and try to avoid having the same mistakes recur.

thanks again,

mike

On Thu, Jan 28, 2010 at 4:56 PM, Mike Linck <mgl@absolute-performance.com> wrote:

That's the point of the '^' character that I used in my example; it wasn't a mistake.

Michael Witten:

Your "übercommits" can easily be faked by wrapping the project up in a git submodule. The supermodule (which would just contain one entry -- the project you are "übercommit-tracking") would then contain one single commit for each set of commits you wish to publish.

I'm not sure such a work-flow makes much sense, though. Annotated tags are probably a better idea.

On Thu, Jan 28, 2010 at 7:44 PM, Mike Linck <mgl@absolute-performance.com> wrote:

I've done a similar job for quite a while, and kernel hackers need that all the time too. If the branches merge a lot, what you need to know is what patches are only on one side, and not on the other. Two things to the rescue:

 - the "3 dots" "..." separator. try with gitk and git log:
    git log mybranch...hisbranch
    git log hisbranch...mybranch

 - git am (which is part of the rebase machinery) does the same as the
"..." operator, but has additional tricks to try and spot commits that
have been cherry picked. So I would often export patches with git am
just to review what' s on one side.

hth,

m

Mike Linck <mgl@absolute-performance.com> writes:

On the quite contrary, probably an easier way is to pick a the oldest commit that a fix or enhancement would apply, build a topic that deals with only the fix or enhancement in question without doing anything else on top of it, and merge the resulting topic. The choice of the fork point needs to be made wisely in such a way that the resulting topic would not cause too much undue conflicts when merged to a modern mainline but old enough that you _could_ merge the result to any older maintenance branch if you choose to.

One implication is that you do _not_ rebase the series to newer codebase because doing so would make the result unmergeable to older releases even if you later realize that the fix/enhancement would be suitable to them.

And if you fork from older commit than tip, you will automatically get a non-ff merge when you merge it back to the mainline, which would delineate the history of the side branch from the integration branches rather nicely.

On Thu, Jan 28, 2010 at 4:33 PM, Junio C Hamano <gitster@pobox.com> wrote:

nice!

Mike Linck:

If you keep the branch name somewhere (either pushed to the master repository, or to a side-repository used to store old "dead" branches), then you at least have the pointer to the last commit on that particular branch.

You can then backtrack through the commits to the previous merge-point, or branch head, in the history to find the point where it, most likely, was branched off from.