merge time

Sorry to bring up the time issue again [that I am perhaps still confused about] but I have been playing around with git more and I think I can phrase my question/observation better.

he later merges changes but some subset of those changes/commits show up in the list in time order as taking place _before_ the rc1 tag was made even though they were merged after. Do I describe a real or possible phenomenon? And does this happen because the developer that made the subset of changes in question commit them to his/her local repository in time order before the rc1 tag was made? So an external repository had the change before the rc1 tag was made but Linus' repository didn't? But internally git on Linus' machine knows that the gitweb.cgi displayed time order is wrong as far as the state is concerned because each repository's index file keeps local track of the true local state [just time isn't reconcilable], or am I missing something(s)?

Is it possible for gitweb.cgi to have a new view mode that sorts/displays the list based on merge time for commits (the time merged into Linus' or whatever repository) so the above situation doesn't happen? The actual time of a local commit should be the time it was merged locally not the time it was created externally/originally, right? Where can I find the gitweb.cgi source/package? I could maybe hack gitweb.cgi myself.

Please CC me on any replies since I am not subscribed to the list.

-Matt

       
____________________________________________________________________________________
Get the Yahoo! toolbar and be alerted to new email wherever you're surfing.
http://new.toolbar.yahoo.com/toolbar/features/mail/index.php

[Cc: Matthew L Foster <mfoster167@yahoo.com>, git@vger.kernel.org]

Please, word wrap lines around 70 - 76 column.

Matthew L Foster wrote:

Of course. The time ordering is by the time commits were _created_, not when they were merged.

In git, you can try tu use --topo-order parameter to git-log. In gitweb, not yet.

Real phenomenon.

That's correct. Well, Linus' repository might have had change, but just not merged in, but that is just small detail.

Git does not know, and git proper has no utility to show for each commit when it was merged to given branch. And that is even more complicated by the fast-forward merges. Reflog (local matter, not displayed in gitweb, available as "git reflog show <branch>" or "git log --show-reflog [<branch>]" in git) keeps track of the branch tip history; it doesn't offer information [you think] you want.

I don't think that it is possible, but you are welcome to try...

Gitweb is now in the git.git repository, in gitweb/gitweb.perl
(see also gitweb/README and gitweb/INSTALL).
 

Please Cc: list.

On Sun, 29 Jul 2007, Matthew L Foster wrote:

Absolutely. This is very common indeed. It's even more common with not the -rc1 tag, but a release.

When I cut a full release, that "opens the floodgates" for the merge window, and a lot of people who have committed their changes (maybe weeks or *months* before) but where the changes weren't appropriate to be merged before the merge window, will now ask me to pull.

So you may have the situation that 2.6.22 was released, but then a few days later I'll merge stuff that was actually committed two weeks before the 2.6.22 release, but was not _in_ the release.

Yes. I would seriously suggest you not use "gitweb" as your way to look at the repository, because you'll never see all the interactions that way.

Cloning a git repository (not necessarily the kernel, but it needs to be something with concurrent developement), and exploring it locally with "gitk" or "qgit" is a _lot_ more informative. When you see the actual history chains graphically, something that might look "odd" in gitweb (commits that look old but weren't there a few days ago) suddenly makes tons of sense.

Well, there i sno "wrong" time. There are just "different" times. The only thing git really tracks is not actually the time (that's purely for human consumption), but the *relationship* between commits. So git really very fundmanetally just tracks things like "commit X was the parent of commit Y", and the time is really immaterial.

The time, to git, is not really different from authorship: it's very important to track when something was done, but it's really purely informational, exactly the same way the _author_ is purely informational. It has no "meaning" for git itself.

The public repositories don't even know what the merge time was for me. That's a purely local feature, and while I can see it in my private repository that I actually did the merge in, I don't publish that information. It's incidental, and quite frankly, it's "wrong" to care: because "Linus' tree" is really not even supposed to be special.

		Linus

--- Linus Torvalds <torvalds@linux-foundation.org> wrote:

Is it possible for git and/or gitweb to know that commits X and Y are descendents of merge C and use the time merge C happened locally for both instead of using the time commits X and Y were created? It seems to me changes showing up as being made long before they really were merged is a very serious problem verification wise but if everyone is using git then perhaps it's not as bad as I think. What happens when security bug fix Z errantly seems to be in v2.6.22 but in reality its not?

Thanks for the responses, -Matt

      ____________________________________________________________________________________
Park yourself in front of a world of choices in alternative vehicles. Visit the Yahoo! Auto Green Center.
http://autos.yahoo.com/green_center/ 

On Sun, 29 Jul 2007, Matthew L Foster wrote:

git knows what's a decendent of what, but gitweb doesn't show it well. that's why Linus suggested you look at gitk or qgit.

by the way, you probably mean that commits X and Y are parents of merge C, not decendants.

but if git did what you wanted it would show every commit with the time of the merge, and that wouldn't help you anyway.

you don't look at the dates to see if the bugfix is in 2.6.22 you look at the graph or ask git to tell you

remember that in git you don't have one-true-trunk of the project, you have a mesh of interconnected points, some of which are pointed to by tags that tell you that other people thought that they are particularly interesting.

David Lang

On Sun, 29 Jul 2007, Matthew L Foster wrote:

The data exists, but there is nothing to make "C" special. Or rather, the two "sides" of C that got merged are 100% equivalent.

You'd probably have to read a lot of old emails (from very early in the git process) to see the whole picture. But the fact is, in a distributed environment, the parents to a merge C are totally equivalent.

So thus when you talk about merge new code into an old repository (and giving those new commits the same date as the merge), that is actually technically 100% equivalent to merging the other way around, and thus you'd have to give all the *old* commits that new merge date too!

And yes, this does happen. It's not at all the case that I always merge other peoples tree: quite often other people merge _my_ tree, and the end result really is that they pulled in the changes _I_ had done.

[ Side note: we do actually try to avoid that, just because it makes the 
  history harder to read, and the resulting criss-cross merges, while 
  technically not a problem at all, can confuse people. But when I say 
  "try to avoid that", I mean just that: it's not a hard rule, and the 
  reverse merging _does_ happen, and there are good reasons for why it 
  happens.

  So *most* of the time, the history looks like it's me merging other 
  peoples code, but then every once in a while, the merge is done by the 
  other side. And from a pure technical standpoint, the two cases are 
  totally equivalent, and git is very fundamentally designed to *not* 
  care. ]

So there is never really any way to say that one side of a merge is special. The closest you can get is saying

 - the first parent is special.

   This is "see merges from the viewpoint of the merger", but as 
   mentioned, the person who actually did the merge isn't necessarily me, 
   so while this is a totally self-consistent view, it's not really the 
   view you are looking for.

   You can get some of this view by using "git log --first-parent", which 
   basically follows commits preferentially using the first parent, and 
   thus "prefers" history as seen by whoever did the merge.

or

 - the *local* repository is special (but this is a purely repository- 
   local viewpoint).

   This is what "reflogs" give you, and what "git log -g" shows. It shows 
   the history not as it pertains to parenthood, but as the tip-of-tree 
   has changed in *that* repository.

but neither of those are really "sensible" (the first one will give *wildly* different results if the repository is ever "switched around" because it's not merged by a single person, and the second one is a purely local thing and has no meaning for anybody else.

The fact is, distributed history isn't one-dimensional. You *cannot* linearize it as some one-dimensional time. Impossible. Any system that tries is broken. Fundamentally.

		Linus

--- Linus Torvalds <torvalds@linux-foundation.org> wrote:

I don't want distributed history, I want what local time changes were merged locally. That is why I described a separate view for this feature, this feature request is not meant to replace how time is currently not being used. :)

-Matt

       
____________________________________________________________________________________
Pinpoint customers who are looking for what you sell. 
http://searchmarketing.yahoo.com/

On Sun, 29 Jul 2007, Matthew L Foster wrote:

The point is, there is no "locally".

Do you mean locally on my machine? That's actually *different* from the locally on the public machines, and no, I wouldn't give you that information anyway (since that information would include the mistakes that I fixed up ;)

And in fact, even on the public machines, the "locally" would be different depending on things like mirroring delays, although that is currently hidden by the fact that kernel.org uses rsync for mirroring rather than using git natively.

So in theory, we could pick one particular public kernel.org machine, and use the times as _that_ machine sees it, but the fact is, that isn't how git works. No normal git command will ever show you such a senseless ordering.

I suspect that the closest you could get to what you want would be to actually run git-cvsserver on kernel.org to export the git data as "CVS" data, and then you could use a CVS client that gets a linearized model of history. That is, afaik, the only way to give you what you want.

And quite frankly, I'd never ask the kernel.org maintainers to do something that perverse. You could ask them, and maybe they would do so out of some really perverse self-destructive death-wish, but quite frankly, you'd probably be better off setting up such a git->CVS gateway on some local machine yourself.

			Linus

On Sun, 29 Jul 2007, Linus Torvalds wrote:

So to be constructive, and just tell you what the *sensible* ordering is:

 - get a kernel git clone

 - do "git pull" to update it.

 - do

	gitk ORIG_HEAD..

   to show what the new stuff is after each update, or do something like

	gitk v2.6.23-rc1..

  to show what is new after -rc1 (or "gitk @{2.days.ago}.." to see what 
  is new in _your_ tree in the last two days or whatever).

No commit dates anywhere. Just commit relationships and your *local* views of time.

(Sure, gitk will show you the commit dates too, but they aren't important, since they have no meaning as to whether a commit got merged into 2.6.23-rc1 or not).

		Linus

--- Linus Torvalds <torvalds@linux-foundation.org> wrote:

       
____________________________________________________________________________________
Be a better Heartthrob. Get better relationship answers from someone who knows. Yahoo! Answers - Check it out. 
http://answers.yahoo.com/dir/?link=list&sid=396545433

On Sun, 29 Jul 2007 20:16:01 -0700 (PDT) Linus Torvalds <torvalds@linux-foundation.org> wrote:

It sounds like that would be the proper way to address the issue within Gitweb as well. Assuming the issue is users wanting to see changes since a specific release, offering a "Commits since tag" option or some such should resolve it.

Sean

--- Linus Torvalds <torvalds@linux-foundation.org> wrote:
 

Ok that explains it. How about locally on gitX.kernel.org [would have to be real git repository]? I'd want gitweb.cgi running on my local repository to use the local time of local merges to ensure the problem I described at the start of this thread doesn't happen. Git reset --hard can be used to fix mistakes and that wouldn't show?

Yes, it should be different, everyone's local commit/merge order is not the same.

Is local commit order really senseless? Using local commit order and/or the local time of local merges would solve the problem I mentioned in the start of this thread? I don't think this info has to be exported to CVS, a web interface to git and/or git itself should be able to tell us what local time order and/or what local commit order merges were made in.

-Matt

       
____________________________________________________________________________________
Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. 
http://mobile.yahoo.com/go?refer=1GNXIC

On Jul 30, 2007, at 4:29 AM, Linus Torvalds wrote:

In general the first parent is not special, agreed. But could one deliberately built a history by following certain rules that make the first parent _always_ a special one?

I think of a quite simple example. Topic branches are always branched off the master, and later merged back. The master, which is the branch an official release is created from, must always be the first parent during a merge. It doesn't matter who does the merges or who does the releases, as long as he follows the rule to start from the last release point and merge in all changes in the appropriate way, such that the first parent rule is followed.

Here are two applications that I think would be interesting:

1) The kernel history. If you went from a current release back
along the first parents you'd see the changes that entered the
official kernel sorted backwards in time. I believe the history of
the kernel is already an approximation of this rule. But maybe I'm
wrong.

2) Developing using topic branches. You start topic branches and
start developing new features. Only later you fix problems on all
supported platforms, polish and review the changes. Not every
commit on the topic branch has the same quality, e.g. the early
commits may not even compile on all supported platforms. But the
tip of the topic branch passes all required tests. After a merge
of the topic branch to a stable branch it would be nice to
distinguish the stable history from the less polished commits on
the topic branch. I think this could be achieved if the first
parent always is linked to a stable commit.

I'm sure some will propose that instead of (2) I should rewrite a topic branch such that every single commit passes all quality requirements. But I strongly believe it is a viable choice not to do so. It may just be too much work and often it's sufficient and easier to polish the tip of a topic branch.

Fast-forward merges would break both of the scenarios. But would they be possible without fast-forward merges?

Would it be possible to ensure that all commits along the first-parent-path have 'release' quality and all release tags are located along this path? What would be the right rules to achieve this objective?

	Steffen

This is an old topic rehashed from time to time. The last time was March this year if I recall correctly.

    http://thread.gmane.org/gmane.comp.version-control.git/41931/focus=41983
 
I tend to agree that it would make sense to allow something like:

    $ git merge --no-fast-forward -m 'Merge topic XYZ' xyz

but at the same time Linus is pretty clear that he would not use that for his kernel merges, so even if we add that feature, gitweb at k.org that shows linux-2.6 history would not see it.

On Jul 30, 2007, at 8:48 AM, Junio C Hamano wrote:

Another option could be to force a commit before the merge, even if there are no changes to commit, something like

     $ git commit --force -m 'MARK'

[to my knowledge, --force or similar is not yet available]

	Steffen

Steffen Prohaska <prohaska@zib.de> wrote:

git-force-commit:

  git config --global alias.force-commit \
  '! git update-ref HEAD $(echo MARK | git commit-tree HEAD -p HEAD)'

But you didn't hear it from me.  ;-)

Yes, I've actually done this once in a while. But only to create some temporary throw-away state that I need for some diff operation or whatnot. Usually because I make it in one repository, but want to fetch it into another and fetch likes moving refs to commits better than refs to trees. Oh, and I'm usually passing in more than one -p, and usually not HEAD as the tree. But whatever.

On Jul 30, 2007, at 9:49 AM, Shawn O. Pearce wrote:

I needed to slightly modify it

     git config --global alias.force-commit \
     '! git update-ref HEAD $(echo MARK | git commit-tree $(git cat- 
file commit HEAD | grep ^tree | cut -b 6-) -p HEAD)'

I'll tell no-one ;)

	Steffen

On Mon, Jul 30, 2007 at 10:09:16AM +0200, Steffen Prohaska wrote:

How about "git commit-tree HEAD^{tree} -p HEAD"?

But I think making a "fake" commit to force non-fast-forward is the wrong thing. You really want to make your "extra" commit be the merge that wouldn't have happened (which unfortunately is not as simple as just creating a commit by hand, since you have to actually _do_ the merge to get the tree).

-Peff

Jeff King <peff@peff.net> writes:

I do agree that if you really want to create a commit instead of allowing a fast forward, you really should create a proper merge commit.

But it is not hard. If it is a fast forward in reality but you are marking it as a real merge by creating a merge commit, then:

 - The tree is obviously the merged branch that is a fast
   forward of your old HEAD;

 - The first parent of the resulting merge is your old HEAD; and

 - The second parent of the resulting merge is the merged
   branch.

So:

	git merge --no-fast-forward other

when other is a fast forward of HEAD would do:

	git commit-tree other^{tree} -p HEAD -p other

On Mon, Jul 30, 2007 at 01:24:43AM -0700, Junio C Hamano wrote:

Of course, I was not thinking right. I was considering the difficulty of invoking the merge machinery, but of course by definition for a fast-forward you don't have to.

Yes, that makes sense.

-Peff

On Jul 30, 2007, at 10:14 AM, Jeff King wrote:

Thanks, I wasn't aware of this syntax.

I agree. But I think except for being 'fake' there shouldn't be any problems with the extra commit.

	Steffen

On Mon, Jul 30, 2007 at 10:25:25AM +0200, Steffen Prohaska wrote:

Problems, no. But it will look ugly when browsing history. :)

-Peff

On Mon, 30 Jul 2007, Jeff King wrote:

yes, but this entire thread was an attempt to avoid browsing the history and instead figure out the relationships between commits by dates. so the people wanting this don't care how ugly it makes the history (but since they want to do this in other people's repositories this won't work for them either)

David Lang

On Mon, Jul 30, 2007 at 01:34:55AM -0700, david@lang.hm wrote:

I think we drifted a bit from that with Steffen's original message...

If you followed a strict policy of always merging topics to a "base" branch as your first parent, then never allowing fast forwards should allow a very easy-to-read history in gitk. The left-most line of commits would simply be a record of topics getting merged in, and would never contain any "non-base" commits.

-Peff

On Mon, 30 Jul 2007, Jeff King wrote:

Only if only *one* person ever does any merges.

Immediately when you have other people merging code, you're now back in the same boat.

This is why I personally think the whole policy of "no fast forward" is totally broken. It's only usable in a non-distributed environment, where there is one central person who does everythng (a so-called "star topology").

			Linus

On Jul 30, 2007, at 7:42 PM, Linus Torvalds wrote:

I think no fast forward merges can always provide useful information.

Public releases are often created from a public, linear branch that is not arbitrarily jumping around. Even if more than one person creates such releases there must be some agreement upon the responsibility for the release and thus for the branch the releases are created from.

If all agreed to merge topics following the "no fast forward" policy and to use the release branch as a first parent, the commits along the first parents would document how commits came into the release branch. Non fast forward merges would always document that a series of commits was added to the release branch at once at the (local time) of the merge commit.

If fast forwards are allowed it depends on whether the release branch can be fast-forwarded over a series of commits on a topic branch or not. In case of fast-forward, the history makes you believe each of the commits of the topic branch entered the release branch separately. It hides that they were all merged at once.

I think, avoiding fast forwards may also be used to document a different level of quality required from commits to a topic branch and from commits to the release branch. A the time of the merge the tip of a topic branch must fulfill all quality requirements, for example compile on all platforms and pass all tests. But this need not be the case for every single commit on the topic branch. If you avoid fast forwards you still get a chain of commits along the first parent that would fulfill your quality requirements. If you allowed a fast forward over a low quality topic branch your chain might be broken and you can't be sure, which commits will have high quality and which one have a lower quality.

You may argue that this is all crap because every single commit should be of highest quality, but I think this is unrealistic. Especially if more than one person is jointly working on a topic branch and less experienced and more experienced developers work together. It may also be rational to first get the basic functionality right and later polish work for several platforms and usage scenarios.

Obviously other techniques, like rewriting a topic branch to a perfect commit series may be used as well. But from my understanding this may be hard if more than a single person is involved. You could also squash the complete topic branch into a single commit but then valuable history might get lost.

I don't see why what I have in mind would break if other people are merging code, too. They may or may not follow my local rules. But at the time of merging their changes, I can enforce my local quality policy and document this by create a merge commit.

This could even be applied recursively. You could start to do a couple of merges but only thoroughly test the final result of the last merge. You could then enforce a link between the last very stable commit before you did all the merges and the very well tested result after the final merge by enforcing a "no fast forward" merge.

I'm not quite sure if such a history would be useful in the long term or may just become to complex. I'm also not sure if the artificial links would cause any trouble. I'm wondering, for example, if there are any cases in which the additional links would make branching and merging harder.

	Steffen

main trunk is at commit X

developer clones Y and adds a patch to it

the merge from Y's tree to the main trunk is a fast-forward

David Lang

On Tue, 31 Jul 2007, Steffen Prohaska wrote:

Matthew L Foster wrote:

The last time this topic came up, folks either created (or pointed to) reflogs as a way to determine the changes to the local state of a repo.

i.e. by checking the reflog for a ref, you could say that that ref was changed from v2.6.20 to v2.6.21 at such and such date and time.

That would allow you to determine what commits were available via that ref at a particular time, regardless of the actual commit dates.

To the best of my knowledge, though, reflogs are not enabled by default on bare repos, and gitweb makes no use of the reflogs anyway.

Rogan

Maybe I am the only one that thinks the web interface should be just as feature rich as the command line interface?

-Matt

       
____________________________________________________________________________________
Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games.
http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow  

Hi,

On Mon, 30 Jul 2007, Matthew L Foster wrote:

*lol* I would be _nice_. But frankly, how do you want to get the power of the command line into a website, _without_ reproducing a command line? I can _script_ the command line, for example.

Ciao, Dscho

--- Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
 

Creating a webified command line is a perfectly fine solution towards making gitweb.cgi as full featured as git is.

-Matt

      ____________________________________________________________________________________
Shape Yahoo! in your own image.  Join our Network Research Panel today!   http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 

Matthew L Foster wrote:

Well, having said that "Use reflogs" was the general response, I don't think that anyone ever really followed through and made it a readily accessible feature, even on the command line.

That is, the reflogs are readily accessible, but noone made it possible to link them to the commits in such a way that the dates/times change as you seem to desire.

Of course, you can do things like:

$ git show HEAD@{2.days.ago}

to see what HEAD looked like 2 days ago, but you can't do things like get gitk to show you that this tag only appeared in your repo 2 days ago.

(Keep in mind, of course, that reflogs are referring to YOUR local copy of the git repo, not what HEAD was on the server that you cloned from.)

And also keep in mind that on the command line you can invoke a lot of "plumbing commands" that you certainly wouldn't expect to be exposed in a web interface.

Rogan

--- Rogan Dawes <lists@dawes.za.net> wrote:

If the web interface requires logins over https why can't plumbing commands be exposed to the web? Though I agree not everything needs to be webified. What I envision is a wikipedia style interface front end with git remaining the backend so you can more easily browse the file system and see history and diff the way you can on Wikipedia. But that idea is very separate from my concern that right now gitweb.cgi effectively has a bug in it because it sorts using external/superset commit order/time rather than local commit order which causes changes to appear as if they were made before they were really merged locally.

-Matt

       
____________________________________________________________________________________
Pinpoint customers who are looking for what you sell. 
http://searchmarketing.yahoo.com/

On Mon, 30 Jul 2007, Matthew L Foster wrote:

what you are asking for would be a very useful piece of software, but that's very different from the current gitweb. you are fileing a bug about the software doing what it was designed to do. yes, it could be done differently. yes, that would be useful. but the software to do so is mostly a new product, not a simple tweak to the existing gitweb software.

right now gitweb isn't designed for figuring out what was merged when, it's just for retreiving specific versions.

if someone really wanted to do this, the right answer may be to take the concept of gitk and webify it (think SVG for the graphics and AJAX interfaces to retreive the info as needed). I think this would be a very useful tool, but it would be a lot of work to implement.

but without the graph showing the commits and how they are related to each other, you really are crippled in your ability to figure out how things are related to each other. Date order just doesn't cut it.

David Lang

david@lang.hm wrote:

By the way, gitweb at repo.or.cz has graphical log (a la gitk) using git-browser by Arteem Khodush, which uses JavaScript library for graphics (creating lines box by box) and a bit of AJAX-ism.

I was thinking about using "template" PNG with transparency and colored boxes to have lighter than git-browser graphical history in gitweb, but...

By the way, you can try to add --topo-order support to gitweb, although I'm not sure if it would do what you want.