Subversion-style incrementing revision numbers

Hello, all.

I'm considering adopting Git for a medium-sized project which is currently managed using Subversion. I've used Git for a few smaller projects already, and the thing I've missed most from Subversion is the convenience of incrementing revision numbers. The following is a proposal to add this feature to Git.

Rationale:

Incrementing revision numbers (IRNs - an acronym I just made up) are useful in that they can be treated as auto-generated tags which are easier to remember and communicate than SHA hashes, yet do not require extra effort to create like real tags. Also, they have the advantage of being chronologically ordered, so if I assert that a bug was fixed in revision 42 of a shared repository, everyone may assume that revision 45 has that fix as well.

Proposal:

As with Subversion, the IRN state in Git would be specific to a given repository and have no significance beyond that repository. Also like Subversion, IRN state would be global across a repository, so that a commit to any branch would increment the current IRN value. Every Git command taking a revision parameter would accept an IRN using a syntax such as "r$IRN". Every commit would report the IRN to the user as well as the SHA ID. The IRN feature could be enabled or disabled via a configuration option.

Implementation:

A simple, efficient implementation of this feature would be based on a single file, $GIT_DIR/history, which would contain a newline-delimited list of SHA commit IDs in chronological order, oldest first. The current repository IRN would be calculated as the size of that file divided by the SHA+newline length, and the commit ID of any IRN could be determined by seeking to the correct offset in that file. Every commit would cause a new line to be appended to the history file with that commit's ID. Finally, a history file could be generated for an existing repository by serializing the commit history based on chronological order.

I'd be happy to put together a patch that implements this, but first I'd like to get some feedback. If something like this has already been proposed, please point me to the discussion. Thanks.

  - Joel

Dear diary, on Tue, Sep 19, 2006 at 11:07:45PM CEST, I got a letter where Joel Dice <dicej@mailsnare.net> said that...

We already have support for recording something similar, it's called a revlog. You would just need to modify it to aggregate all the branches in a single file.

Also, multiple IRNs could refer to a single real commit if you do e.g. cg-admin-uncommit, since revlog logs revision updates, not new revisions created. This may or may not be considered a good thing. If you rather want to just create a new IRN at commit object creation time, also note that some tools _might_ validly create commit objects and then throw them away, which would generate non-sensical (and after prune, invalid) IRNs.

On Tue, 19 Sep 2006, Petr Baudis wrote:

Thanks - I'll look at that.

I'm not too worried about cg-admin-uncommit or git-reset, since the IRN feature is intended mainly for shared repositories. I would suggest that such commands simply be disallowed for such repositories.

The problem of temporary commits certainly needs to be addressed. In this case, may I assume nothing under $GIT_DIR/refs is ever modified? If so, perhaps I could somehow hook into the git-update-ref step. Is that what the revlog code does?

  - Joel

Dear diary, on Tue, Sep 19, 2006 at 11:42:20PM CEST, I got a letter where Joel Dice <dicej@mailsnare.net> said that...

  What kind of shared repositories? You yourself said that IRNs are
local to a repository, thus they are not preserved over cloning/fetching
from a repository, if you mean that.

  Yes. But not every commit is always recorded to something in refs/.
The simplest case is if you fetch from a remote repository (or push to
your repository), only the latest commit is recorded.

On Wed, 20 Sep 2006, Petr Baudis wrote:

The scenario I envision is several developers, each with a clone of a shared repository. The clones would not have IRNs turned on, only the shared repository. So, when I do a "git push", I get an IRN back, and I am not confused, because I know that IRN only applies to the shared repository. Then, when I mark a Bugzilla bug as fixed and attach the IRN to it, everybody knows that IRN refers to the shared repository. After all, I wouldn't mark the bug fixed if I had only committed it to my own private repository.

I could also turn on IRNs on my clone if I really wanted, but not if I thought it would confuse myself or others.

That's what I figured. I should be able to walk the commit chain to get at all the commits in a push or fetch, right?

  - Joel

Joel Dice <dicej@mailsnare.net> wrote:

How about recording the Bugzilla bug id in the commit message (such as by simply mentioning "bug 123") and having an update hook on the central repository extract that from the commits being pushed and posting an update to the Bugzilla database? The update can include the branch name the commit(s) were pushed into and the final commit SHA1 (or say 8 character abbreviation).

Its automatic and its easy for the developers.  I'm (sort of) doing
this for a system that's a lot more difficult to use than Bugzilla.
My fellow developers are very happy.
 

In the update hook you get 3 args:  refname oldsha1 newsha1

You can get the list of commits you are "receiving" during the single push operation from:

	git log --pretty=raw $2..$3 

a quick parse for "bug (\d)+" on that text, make a set of of IDs to update, then go update then storing the first 8 characters of $3 and $1 (the branch name) into your bug tracker.

I said "receiving" in air quotes as you may already have some or all of those commits in the current repository; they just are part of other branches other than the one being pushed into.

Later another developer can ask if the fix for bug 123 is in any given branch by looking at

	git log master..$idfrombug

where 'master' is any branch and $idfrombug is the 8 character abbreviation you stored into the bug record in the update hook.

If this returns no output then the bug fix is contained in the branch master; if it returns output then those are the commits which were considered to be part of the bug fix that aren't in the branch master.

Hi,

On Tue, 19 Sep 2006, Joel Dice wrote:

Why not just tag the commit with something like "bugzilla/#bugid"? Granted, it makes for lots of tags. But we actually handle lots of tags.

BTW if your IRN just wants to serve to be less cryptic than SHA1s, you can try git-name-rev (of course, it is only immutable if you have a recent tag and use the --tags option).

Ciao, Dscho

Joel Dice <dicej@mailsnare.net> wrote:

$GIT_DIR/refs is always modified. Its probably the most heavily modified part of a GIT_DIR, aside from the index. Simply because the ref files must be modified every time a commit, fetch or merge completes. Its also a directory you don't want to delete; I once did an `rm -rf .git/refs` in a repository with a many branches. That was no fun to recover.

git-update-ref is used by pretty much all non-C code to update a ref file. The APIs it calls invoke the reflog code to record the update being made to the ref if the user has enabled logging on that ref (not all users want all refs logged apparently). I don't think its a great idea to plug more complex logic into that part of the system. The reflog code already made it more complex then it needed to be; Linus apparently just found out how heavily we use static buffers down there...

Petr Baudis wrote:
 

_reflog_, not revlog.

On Tue, 19 Sep 2006, Joel Dice wrote:

I really don't think you can do it sanely.

The thing is, the reason subversion can do it is simply that SVN is terminally broken.

Really. SVN is crap. It may be prettier crap than CVS, but it's literaly no different from putting confectioners sugar on top of a cowpatty. It's not really "fixing" anything. And the revision numbering is actually very much part of the problem.

The thing is, you can limit yourself to a single repository, and once you have branches, you _still_ cannot sanely number the commits with anything like "r1.3.4.1".

Here's an example. Let's say that you start out with just a regular "master" branch, and you number the commits "r1.1", "r1.2" and so on (of course, that extra "1." at the beginning doesn't actually make any _sense_, but let's keep it just to not confuse CVS/SVN users any more than they are already confused).

Now, you've done five commits, and are at "r1.5", and everything looks beautiful, and you're getting comfy, so you start and check out a branch called "test". What do you call it?

The thing is, you need to call it "r1.5", because the _commit_ is still exactly the same as the master one. Creation of a branch in git doesn't actually create any new commits, it just creates a new pointer.

So far so fine. We can ignore that, and make the rules be:

 - when creating a commit, we look at the branch name. If the branch name 
   is "master", we increment the second number, and drop all other 
   numbers.

 - if the branch name is anything else, we look whether the current state 
   is the same any other branch, in which case we just add a ".1" at the 
   end relative to the branch (or a ".2" if a ".1" already exists and so 
   on). Otherwise we increment the last number.

that's not a wonderful set of rules, but it might work. So what happens is that when we create the next commit (on the "test" branch), that commit will be called "r1.5.1" (which is what you'd expect). Go along a bit, and you get "r1.5.2" and "r1.5.3" too. Wonderful.

Now you decide to merge this back into "master". Which, btw, hasn't had any actual development happen, so it's stil at 1.5. You check out "master" and do a "git pull . test" (or "git pull . r1.5.3", which is obviously the same thing), and what happens?

That's right, we now fast-forward master to that revision, since nothing happened on master. Which means that your master branch is now comprised of revisions "r1.1" .. "r1.5" "r1.5.1" .. "r1.5.3". That makes no sense at all, but it works. The next commit would be called "r1.6".

In other words, the SVN numbering is just insane in a git repo, although you can "make it work" in a limited sense (but the numbering would jump in odd ways. The thing is, all the things that are true in a distributed environment are true in a single repository too, so limiting yourself to a single repo doesn't really make them go away.

Another thing you CAN do, is to just number them in time in a single repo. Every time you do a commit, you can create a "r1.<n+1>" revision, and that would work. It wouldn't look like the SVN numbers do, and it would only work _within_ that repository, but it would work.

But it would mean that "r1.57" is _not_ necessarily the child of "r1.56". It might be that "r1.56" was done on another branch, and is totally unrelated to "r1.57" (other than they sharing some common ancestor far back).

See? The SVN (and CVS) revision numbering really only makes sense because it's fundamentally a non-distributed model.

Git is so _fundamentally_ distributed that even within one single repository, each branch is totally independent ("distributed"). In CVS/SVN, branches aren't independent of each other, and that's why they show up in the revision numbering.

So you can really only number things as long as you never do any branches at all, and just do a linear development. In that case, and in that case only, does numbering work. Or you'd have to have a very "non-git" approach to branches (the above silly rules _can_ be used - possibly with some fixes - to give you essentially SVN-like revision numbers, if you _always_ generate a commit when you merge).

It doesn't seem to make a whole lot of sense, does it?

Btw, along the same issues, BK actually had RCS/CVS/SVN-like revision numbers, but to solve the distributed problem (BK didn't have branches) BK had a different approach.

So BK actually _did_ number things as "1.1" and "1.4.6" etc, and was obviously distributed, and the revision numbers actually matched the shape of the development history the same way CVS/SVN does. But if you do that, what happens is that a merge will _change_ the numbers: you may have started with two different branches that both were at "r1.2", but at the end of it one of them got renamed to "r1.1.1", and the final result was called "r1.3".

So if you allow malleable revision numbers, you can definitely do it. That ends up being _really_ confusing. I know a number of people who otherwise were smart really never understood why "r1.576" might not stay as "r1.576" forever, but might suddenly become "1.401.3.55" when merged into another repository.

Anyway, what I'm trying to say is that you're probably better off trying to get used to the git numbering than trying to force the SVN model. You're going to hit a few confusing issues if you really want to call things "r1.x.y.z"

			Linus

Dear diary, on Tue, Sep 19, 2006 at 11:51:49PM CEST, I got a letter where Linus Torvalds <torvalds@osdl.org> said that...

This is actually exactly how SVN revision numbering works. There's just a single number (no '1.') and it indeed jumps randomly if you have several concurrent branches in your (ok, Linus does not have any, just someone's) repository.

Noone does, that indeed would be horrible. But having the commits numbered inside a repository would indeed make for simple usage if you need to type in commit ids frequently, and could make Git a bit friendlier to newcomers.

On Wed, 20 Sep 2006, Petr Baudis wrote:

Oh, ok, if it's just a single numbering, then that's easy to do. It won't _mean_ anything, and you're seriously screwed if you ever merge anything else (or use a git that doesn't update the refcache or whatever), but it is simple and stable within a single repo.

		Linus

On Tue, 19 Sep 2006, Linus Torvalds wrote:

Well, what it means is "this is the order in which commits were applied to this repository". I suggest that this information is useful for the most common development style - the kind which relies on a central repository as the canonical source for a project's code. "gcc-trunk-r117064" means a lot more to me than "39282037d7cc39829f1d56bf8307b8e5430d585f", and is no less precise.

I do believe that distributed VCSs such as Git can improve the productivity of these kinds of projects without forcing the developers to suddenly and dramatically alter their workflow. I think ICNs would help make this possible.

  - Joel

Joel Dice wrote:

What about "v1.4.2.1-gf7f93e7", or "tags/v1.4.2-rc4^0~19", or just "39282037"? Or "next@{2006-09-19 22:44:33 +0000}"?

On Wed, 20 Sep 2006, Jakub Narebski wrote:

The last one is closest to what I want in that it gives me some sense of the order in which commits appeared in the repository.

Anyway, after some reflection, I've come to the following conclusions:

  1. Although the IRN feature would be useful to people like me, it doesn't 
fit _naturally_ into Git in particular or DVCSs in general due to the 
fact that IRNs are tied to repositories.

  2. There are easier, more elegant ways to solve the problem of tying 
commits to bug numbers, as Shawn and Johannes pointed out.

So, I'm shelving the IRN idea until and unless I can reconcile it with the spirit of distributed version control. In the meantime, I plan to pursue a solution along the lines of Shawn's update hook strategy for bug tracking, perhaps with the additional step of automatically tagging each such update.

Thanks for everyone's comments.

  - Joel

Petr Baudis <pasky@suse.cz> writes, responding to Linus:

SVN increments the number every time anything changes in the whole-tree. Typically (1) when you make a commit, anywhere, in the tree or (2) when you tag or branch. An interesting property of SVN version number is that you can say "at revision 1423, frotz and nitfol branches existed (filfre branch did not exist back then) and their states were such and such".

So if Pasky wants to help Joel to emulate it in git, it is not sufficient to number commits because there are fast-forwards and it happens rather often in reality. A closer emulation at the conceptual level is to run pack-refs every time you change any ref in your repository and version control the resulting packed-ref file with serial number.

Then when people talk about git.git repository at kernel.org people can say "the repository at revision 144 had master at 019ba86 and next at 832e76a; tag v1.4.3 did not exist back then."

Is it useful? In general I think it is useful only as a local matter, just like ref-log is useful locally. In fact, I think it is a natural extension of the ref-log (and it can even capture where the tags were, which I sometimes find sorely lacking in the current ref-log scheme -- anybody want to update git-tag.sh to have it use ref-log? hint hint...).

But I am not sure how you would propagate it sanely in the distributed environment.

I do not often pull from others, but I occasionally pull from Paul's gitk tree. When I see gitk's tip advance and I pull from him, I may end up getting a string of two commits. In other words, in Joel's serial numbered revision scheme (emulated by packed-ref snapshots versioned serially), there are two gitk commits between my rN and r(N+1).

Somebody who hasn't thought about merges might be tempted to say that we should use 2 revision numbers when a pull ends up integrating 2 commits from outside, but that is not really feasible. When you make a merge, it is not like you are cherry-picking individual commits from somebody else one by one. In state rN, I did not have either of these two commits, and in state r(N+1) I now have both, and that is the only definition of rN that makes any sense.

    Side note: SVN does not have this "one state change but with
    multiple revisions" uneasiness problem, because it does not
    have a merge in distributed SCM's sense.  What it has is a
    convenient way to side-port changes that happened elsewhere
    to your state and create a new commit (so it is not that
    different from "cvs update -j").  After a "merge" happens
    you are just one commit ahead from previous state, just like
    a normal commit.

    The difference from that and us is that "log ORIG_HEAD.."
    will say we got new two commits integrated with the merge
    which makes us wonder why there are multiple events between
    two states, while SVN does not record what got cherry-picked
    from the side so you cannot even worry about it if you
    wanted to ;-).

But that definition stops making sense immediately when you look at things from Paul's point of view. He had these two commits done separately, and in his repository they are recorded as two separate events, even under Joel's serial numbered revision scheme. So for this serially numbered revisioning scheme to make any sense to you, you have to refuse the everybody-is-equal distributed model of git, and instead treat one repository (mine) as more important than all others. If that central repository says these two commits did not exist in rN and they appeared in r(N+1), that's the reality as far as you are concerned, and what Paul says does not count to you at all. That makes Paul unhappy.

In the same sense, you have to realize that the number "144" above that came from kernel.org would mean absolutely nothing to me. I work on one commit at a time unless applying sequence of patches, and typically ref-log knows that the last time _my_ tip of "master" and "next" advanced were more than 30 minutes apart. But I push four branches (maint/master/next/pu) out at the same time to kernel.org, so between revision 143 and revision 144 four refs would have changed at kernel.org. But in the repository I do the real work with ref-log knows that that is not the way how these branches advanced at all. So it is not a convenient number to work with from my point of view either.

onsdag 20 september 2006 00:06 skrev Petr Baudis: [...]

Typing those numbers is really insane, so the idea never occurred to me. Most environments have simple methods to copy and paste strings using a clipboard, even in a console. So typically you can left-doubleclick on a string and paste with the middle button in *nix like systems (right button in cygwin/windows command line).

-- robin

Dear diary, on Wed, Sep 20, 2006 at 07:28:45PM CEST, I got a letter where Robin Rosenberg <robin.rosenberg.lists@dewire.com> said that...

  ...which is certainly appealing for the Plan9 fans, but I'm not so
excited from having to grab my mouse all the time. :-) That said, typing
just the few leading digits is not really that painful.

Petr Baudis <pasky@suse.cz> writes:

I am so used to live always in "screen" so that is not much of a problem for me.

Sorry for the noise ;-).

Joel Dice wrote:

That is true _only_ if you have linear history. If you have multiple concurrent branches, revision 42 can be in branch 'next', revision '45' in topic branch 'xx/topic' which forked before revision 42, and do not have the fix.

Unfortunately, one cannot (as of now) use result of git-describe as <commit-ish>. I'd rather have it fixed, than port idea from _centralized_ SCM do distributed SCM.

This of course limits IRN much. Tags are valid across repositories. I'm not sure if many repositories are managed using shared repositories (centralized approach).

Dear diary, on Wed, Sep 20, 2006 at 12:07:08AM CEST, I got a letter where Jakub Narebski <jnareb@gmail.com> said that...

Oops, I've completely overlooked that bit of the rationale. Of course IRNs cannot assure this.

Jakub Narebski wrote:

Additionally, what does _chronological_ mean? Time the commit is recorded? Remember that in distributed development commits can be fetched from other repository, or arrive via email and applied using git-am. In git you can also rebase branch. Git records merges, and said merges are sometimes just fast-forward. All those difficulties have to be solved for IRNs.

By the way, there was similar proposal to add either hidden field ('note' like) to commit object denoting _generation_ of commit, or add cache of commits' generation numbers. Generation number of root (parentless) commit is 0, generation number of a commit is maximum of generation numbers of its parents plus 1. But this proposal was for easier and faster generation of topological order.

On Wed, 20 Sep 2006, Jakub Narebski wrote:

Yes.

All of these boil down to pointing the head of a branch to a new commit object, right? I'm simply proposing that every time this happens, that new commit object (which may not really be *new*), should be appended to the IRN history file, along with any commits attached to it (as in the cases of fetch, etc.).

  - Joel

Jakub Narebski <jnareb@gmail.com> wrote:

This doesn't work?

	git log $(git describe | cut -d- -f2 | sed 's/^g//')

Its obviously so clear and easy to type!  :-)

That should be pretty easy to teach sha1_name.c:get_sha1_basic to look for the pattern commonly used by git-describe, break on "-g", verify the tag, then unabbreviate the ID.

Jakub Narebski <jnareb@gmail.com> wrote:

Not only that but its somewhat difficult to execute `git log v1.4.0..v1.4.1` on a central repository; if IRNs are only valid within that repository then that's the only way they could even be used.

Note that somewhat difficult here means you need to:

	- gain direct read-only access to the repository's files (not
	  all systems may be able to offer this to all users);

	- set GIT_DIR environment variable OR pass --git-dir before
	  the subcommand;

	- do all of that without doing something stupid in the
	  repository directory such as `rm *` without realizing
	  where you are;

	- but still maintain write access so you can push.

and that latter GIT_DIR/--git-dir part users are bound to forgot as its not as natural as just assuming the repository is the directory you are in. Of course that could be probably fixed...

On Wed, 20 Sep 2006, Jakub Narebski wrote:

In practice, this is not a problem. The branch referred to is usually understood from the context (ex. a Bugzilla bug for a particular version of a product implies a particular branch). Otherwise, yes, you must also explictly state which branch the fix was applied to.

I would suggest that most repositories are centralized. They're just not using Git yet :)

  - Joel

Joel Dice <dicej@mailsnare.net> wrote:

I can't say I miss that particular feature.  Abbrevations of
SHA1 IDs tend to work very well for the projects I use Git on;
6-8 characters is easily more than enough to uniquely identify
a revision.  Heck half the time even 4 hex characters is enough.
 

But with respect to what branch?

Branches in a Git repository are very common:

    o------o-o-o-o-o-o---  B1
   /
  -o--o--o-----o--------o  B2

Assuming the branch point on the left was r60, what is the latest commit on B1? On B2? What about if I merge B1 and B2 together?

Or are you proposing that the commits are ordered by the time that they arrive into the repository?

But even if that's the case lets say I fix the bug on B1 in r42 but a commit on B2 gets assigned r45. The bug isn't fixed in r45; not unless B1 was merged into B2 in r43, r44 or r45. But there's no way to know that from just an IRN.

I fail to see how IRNs simplify commit naming or determining where
a feature or bug fix is within the branching structure.
 

How about just using bare tags under the namespace 'refs/tags/r/'? With the new packed refs implemention these would cost you a tiny amount of overhead over what you propose (due to the ref name also being stored) but its already implemented and does the job just as well.

On Tue, 19 Sep 2006, Shawn Pearce wrote:

Branches in Subversion are also very common. The "last changed" IRN for a branch is the index of the head of the branch in the history.

Yes.

I'm not suggesting that an IRN stand on it's own in this way. Additional context may be necessary depending on the situation. It's intended as a convenience, not a replacement for the SHA IDs.

Good idea.  I'll try that.

  - Joel