threads / discuss / 28191

How to check out the repository at a particular point in time

Subject: How to check out the repository at a particular point in time

## tl;dr

12 messages between Aug 22, 2011 and Aug 24, 2011.

replies: 11people: 8as markdown or json

R. Diez· Aug 22, 2011, 12:25 UTC · lore
Hi all:
I'm writing a daily build script for all the OpenRISC components, so every day I need to check out several git repositories with the source code of many tools that depend on each other.
The hole process takes hours. In order to minimize the risk of repository skew, I thought I could just take the current date and time, clone all repositories to my local PC and check them all out at that particular timestamp.
I figured out that something like git "checkout HEAD@{2011-08-21 10:00:00}" does not really cut it. I'm getting this warning:
  warning: Log for 'HEAD' only goes back to Sun, 21 Aug 2011 10:00:02 +0200.
The reason is, the timestamp was taken at 10:00, and the repository was cloned 2 seconds later, which means that 10:00 is earlier than the repository. That was totally unexpected, but then I found this in the documentation for "gitrevisions":
  Note that this looks up the state of your local ref at
  a given time; e.g., what was in your local master branch last week.
  If you want to look at commits made during certain times,
  see --since and --until. 
I guess that means the HEAD@{date} syntax does not do what I expected. But hey, it's not the first time I find the git docs hard to follow... }8-)
By the way, it would be nice if the gitrevisions documentation could be improved, as I still don't understand what that really means. Say, for example, an hour ago I had temporarily checked out last year's versions, but half an hour ago I went back to this year's versions. If I check out at HEAD@{1 hour ago}, will I get then last year's version, or this year's?
Anyway, my real problem is with the mentioned --until option. "git checkout" does not understand that option, so I guess I need to feed the date to some other git command in order to get the commit ID for "git checkout", right? Can someone help me here?
Or even better, can someone add this kind of explanation to the "git checkout" documentation? If you are used to other version control systems, and wish to checkout the versions at a particular date, that's the documentation page you first look at.
For extra karma points, git checkout could understand the --until option itself.
Many thanks in advance,
  R. Diez
Thomas Rast· Aug 22, 2011, 13:25 UTC · re: R. Diez · lore

Re: How to check out the repository at a particular point in time

I'll basically reply from bottom up so you can see the motivation and then my suggestions for the solution.

R. Diez wrote:
Show 5 quoted lines
> 
>   Note that this looks up the state of your local ref at
>   a given time; e.g., what was in your local master branch last week.
>   If you want to look at commits made during certain times,
>   see --since and --until. 
[...]
> Say, for example, an hour ago I had temporarily checked out last
> year's versions, but half an hour ago I went back to this year's
> versions. If I check out at HEAD@{1 hour ago}, will I get then last
> year's version, or this year's?

The @{date} and @{n} syntax refers to the reflog, which as the name tries to imply, is a log of where *your local ref* was at that time/step. Since the HEAD ref is by definition what you have checked out at the moment, HEAD@{1 hour ago} indeed refers to last year's version.

> Anyway, my real problem is with the mentioned --until option. "git
> checkout" does not understand that option, so I guess I need to feed
> the date to some other git command in order to get the commit ID for
> "git checkout", right? Can someone help me here?

It is a git-log option (or more precisely, revision walker option). The main problem is that your request is not very well-defined: in nonlinear history there will in general be more than one commit at the time requested.

    ---a----b----c----M----       (M is a merge)
        \            /
         d-----e----f
             ^---- April 1st

Suppose you ask git for "the newest commit as of April 1st" in this history. Is it supposed to give you b or d? [If you think nonlinear history is easy, try to figure out a good rule in the presence of time skew, where misconfigured clocks/timezones resulted in parents being younger than children.]

Hence:
> For extra karma points, git checkout could understand the --until option itself.
It probably never will, because that is an ill-defined request.
You can indeed say
  git log -1 --until="april 1"

to get *one* commit that happened before April 1st, but which one is up to the order internally used by git. You can also say

  git log -1 --first-parent --until="april 1"

to get the first such commit along the first-parent ancestry, which might suit the ticket.

But there is a more fundamental issue.  Let me explain.
Show 6 quoted lines
> I'm writing a daily build script for all the OpenRISC components, so
> every day I need to check out several git repositories with the
> source code of many tools that depend on each other.
> 
> The hole process takes hours. In order to minimize the risk of
> repository skew

Step back and consider the real problem here. In the simplest case you are getting two components A and B which depend on each other, e.g., A depends on B. But there is a race condition in the case where a user updates an API between them in a backward-incompatible way: she has to update both A and B, and an unfortunate coworker/buildbot may pull old-A and new-B (or vice versa) and get a broken build. [Incidentally this seems to be a frequent problem with SVN externals.]

You might say: if only we had a way to record the fact that the "blessed" version of B to go with old-A is old-B, and for new-A it's new-B.

And indeed we do. Submodules were invented to allow B to be "linked" into A's repository, such that a checkout of any commit of A "knows" the correct corresponding version of B. A user who updates the API can record the update to B inside the API-changing commit in A.

So while you can kludge your way around the problem with clever use of 'git log --until', submodules would be the "correct" solution.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch
R. Diez· Aug 22, 2011, 15:18 UTC · re: Thomas Rast · lore

Re: How to check out the repository at a particular point in time

Hallo Thomas Rast:
Thanks for your quick answer. Please see mine below.
Show 8 quoted lines
> The @{date} and @{n} syntax refers to the reflog, which as
> the name
> tries to imply, is a log of where *your local ref* was at
> that
> time/step.  Since the HEAD ref is by definition what
> you have checked
> out at the moment, HEAD@{1 hour ago} indeed refers to last
> year's version.
OK, thanks. That kind of example would be nice to have in the "git checkout" documentation page. In the meantime, I've seen on the Internet that other people also got caught by this... let's say... 'unintuitive' behaviour or documentation. 8-)
> It is a git-log option (or more precisely, revision walker
> option).
In the meantime, I've seen this done with "git rev-list" instead, like this:
  git rev-list -n 1 --before="2010-11-01 11:45:16 +0000" master
Is that the same as with git-log ?
Show 15 quoted lines
> The main problem is that your request is not very
> well-defined: in
> nonlinear history there will in general be more than one
> commit at the
> time requested.
> 
>     ---a----b----c----M----  (M is a merge)
>         \            /
>          d-----e----f
> 
>             ^----
> April 1st
> 
> Suppose you ask git for "the newest commit as of April 1st"
> in this history.  Is it supposed to give you b or d?
I still don't quite understand how git works, but let me risk a naive statement here. If "a-b-c-M" were 'master', and "d-e-f" were 'new-feature', then on April 1st the current version on 'master' is 'b', because I merged the 'new-feature' branch at a later point in time. Does that make sense?
> Step back and consider the real problem here.  In the
You're right in saying there is a race condition here between developers, and the right solution would be of course to tag which versions work well with each other.
But that problem the daily build is trying to solve is precisely that it's too hard to keep track of all component versions in all repositories. Things just move too fast, and as far as I understand it, git submodules require manual intervention. If I ever tag anything manually, it must have already passed the daily build!
The development model looks like this: the latest HEAD versions of all components should always work well with each other. If something breaks, the daily build will let the developer know by the next day. If two developers make incompatible changes, they'll speak to each other and commit their changes within a few hours. During that time, they will be trouble, but that's quite alright (at least for the moment).
I just didn't want the daily build to add to the uncertainty of what went wrong by introducing a few hours' worth of random time skew to the mix.
The daily build server needs to check out from git the head status at say 02:00 am on all repositories, as if the server had so many CPUs that it had ran a "git pull" for all of them simultaneously. That's close enough for my purposes. Like stated above, if someone merges some old branch at 02:01 am, a user that did a "git pull" on master at 02:00 am would not have seen that merge, that's the effect I would like to achieve.
In the future there will probably be a stable HEAD branch and a development branch with the same name across all git repositories, and the daily build can do both every day. Or maybe the stable versions will not come from git any more, but from .tar.gz files. Another solution to automate releases without so much human intervention would be as follows: if the automated build and automated testing succeeded 3 hours ago, then that timestamp can be entered in the database of "pretty sure it works" versions. The timestamp becomes effectively the version number. Manually coordinating all participants is hard if you don't have so many human resources.
Thanks again,
  R. Diez
Jens Lehmann· Aug 22, 2011, 16:56 UTC · re: R. Diez · lore

Re: How to check out the repository at a particular point in time

Am 22.08.2011 17:18, schrieb R. Diez:
Show 5 quoted lines
> But that problem the daily build is trying to solve is precisely that it's
> too hard to keep track of all component versions in all repositories. Things
> just move too fast, and as far as I understand it, git submodules require
> manual intervention. If I ever tag anything manually, it must have already
> passed the daily build!

Submodules can easily be scripted too. Why don't you let your buildsystem automatically create a commit with the current HEADs in the superproject when building and testing all repositories was successful? Then each developer can use one of those superproject commits as starting point and happily hack away on a repository. And as a bonus he can see in the superproject how many changes he did from the nightly build he started with. And if he doesn't care, he can forget about the superproject until he needs to sync again.

Show 6 quoted lines
> The development model looks like this: the latest HEAD versions of all
> components should always work well with each other. If something breaks,
> the daily build will let the developer know by the next day. If two
> developers make incompatible changes, they'll speak to each other and
> commit their changes within a few hours. During that time, they will be
> trouble, but that's quite alright (at least for the moment).

You can decide later if you want to use the superproject to coordinate such possibly conflicting changes, but that would mean your developers would have to commit their changes in the superproject too.

Michael Witten· Aug 23, 2011, 15:54 UTC · re: R. Diez · lore

Re: How to check out the repository at a particular point in time

On Mon, Aug 22, 2011 at 15:18, R. Diez <rdiezmail-temp2@yahoo.de> wrote:
Show 21 quoted lines
>> The main problem is that your request is not very
>> well-defined: in
>> nonlinear history there will in general be more than one
>> commit at the
>> time requested.
>>
>>     ---a----b----c----M----  (M is a merge)
>>         \            /
>>          d-----e----f
>>
>>              ^----April 1st
>>
>> Suppose you ask git for "the newest commit as of April 1st"
>> in this history.  Is it supposed to give you b or d?
>
> I still don't quite understand how git works, but let me
> risk a naive statement here. If "a-b-c-M" were 'master',
> and "d-e-f" were 'new-feature', then on April 1st the
> current version on 'master' is 'b', because I merged the
> 'new-feature' branch at a later point in time. Does that
> make sense?
O! for the love all that is Holy!
You see, guys? The term `branch' was a TERRIBLE choice.

What git calls `branch master' in your example is just a pointer to the commit object `M'; it has nothing to do with particular lineages like `a-b-c-M'.

Please see my discussion with Hilco, starting here:
  http://marc.info/?l=git&m=131364675708355&w=2
  Message-ID: CAMOZ1BsZvXsnnWAPXR7UGKdqOMwuGB-ffaAPk55U_1dcjZUcDw@mail.gmail.com
and this email in particular:
  http://marc.info/?l=git&m=131396006222173&w=2
  Message-ID: CAMOZ1BvpnP_729YOHrrPW3B8wa5c4cLyD_qAQ5rTuy0JqNiiXg@mail.gmail.com
which also includes the following very germane link:
  http://slashdot.org/comments.pl?sid=2350536&cid=36903136
Jonathan Nieder· Aug 23, 2011, 16:05 UTC · re: Michael Witten · lore

Re: How to check out the repository at a particular point in time

Michael Witten wrote:
> On Mon, Aug 22, 2011 at 15:18, R. Diez <rdiezmail-temp2@yahoo.de> wrote:
Show 8 quoted lines
>> I still don't quite understand how git works, but let me
>> risk a naive statement here. If "a-b-c-M" were 'master',
>> and "d-e-f" were 'new-feature', then on April 1st the
>> current version on 'master' is 'b', because I merged the
>> 'new-feature' branch at a later point in time. Does that
>> make sense?
>
> O! for the love all that is Holy!

Wait, what's wrong with what R. Diez said? It's exactly what --first-parent gives you.

Matthieu Moy· Aug 23, 2011, 16:09 UTC · re: Jonathan Nieder · lore

Re: How to check out the repository at a particular point in time

Jonathan Nieder <jrnieder@gmail.com> writes:
Show 14 quoted lines
> Michael Witten wrote:
>> On Mon, Aug 22, 2011 at 15:18, R. Diez <rdiezmail-temp2@yahoo.de> wrote:
>
>>> I still don't quite understand how git works, but let me
>>> risk a naive statement here. If "a-b-c-M" were 'master',
>>> and "d-e-f" were 'new-feature', then on April 1st the
>>> current version on 'master' is 'b', because I merged the
>>> 'new-feature' branch at a later point in time. Does that
>>> make sense?
>>
>> O! for the love all that is Holy!
>
> Wait, what's wrong with what R. Diez said?  It's exactly what
> --first-parent gives you.
Not really. Suppose, on April 1st, I have
A--B--C <-master
    \
     D--E <-new-feature
Then, I merge from upstream
A--B-----C <-master
    \     \
     D--E--F <-new-feature
and then I push to master, or master fast-forward-pulls from me:
A--B-----C
    \     \
     D--E--F <-new-feature, master

Then, what used to be in master on April 1st is C, but --first-parent will give you E instead.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
Michael Witten· Aug 23, 2011, 16:23 UTC · re: Matthieu Moy · lore

Re: How to check out the repository at a particular point in time

On Tue, Aug 23, 2011 at 16:09, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:

Show 37 quoted lines
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Michael Witten wrote:
>>> On Mon, Aug 22, 2011 at 15:18, R. Diez <rdiezmail-temp2@yahoo.de> wrote:
>>
>>>> I still don't quite understand how git works, but let me
>>>> risk a naive statement here. If "a-b-c-M" were 'master',
>>>> and "d-e-f" were 'new-feature', then on April 1st the
>>>> current version on 'master' is 'b', because I merged the
>>>> 'new-feature' branch at a later point in time. Does that
>>>> make sense?
>>>
>>> O! for the love all that is Holy!
>>
>> Wait, what's wrong with what R. Diez said?  It's exactly what
>> --first-parent gives you.
>
> Not really. Suppose, on April 1st, I have
>
> A--B--C <-master
>    \
>     D--E <-new-feature
>
> Then, I merge from upstream
>
> A--B-----C <-master
>    \     \
>     D--E--F <-new-feature
>
> and then I push to master, or master fast-forward-pulls from me:
>
> A--B-----C
>    \     \
>     D--E--F <-new-feature, master
>
> Then, what used to be in master on April 1st is C, but --first-parent
> will give you E instead.

Indeed. That example makes perfect sense when so-called `branches' master and new-feature are rightfully thought of as the `pointers' (or, `references') that they are.

Moreover, Jonathan, can't you see that your more knowledgeable mind has automatically smoothed over the inaccuracies presented by R. Diez's description? In particular, a `branch' has nothing to do with walking a commit object's ancestry.

Jonathan Nieder· Aug 23, 2011, 16:53 UTC · re: Michael Witten · lore

Re: How to check out the repository at a particular point in time

Michael Witten wrote:
> Indeed. That example makes perfect sense when so-called `branches'
> master and new-feature are rightfully thought of as the `pointers'
> (or, `references') that they are.

You are right to point out there is potential for confusion, because different VCSes model the practice of branching and merging in so many different ways:

 - a single text field in each commit, as in Mercurial
 - a list of "sticky" keywords attached to each commit, as in CVS
 - special revision numbers, as in CVS
 - special directory names, as in Subversion
 - pointers to the tip of a line of history, as in Git

Each model has its strengths and weaknesses in terms of supporting different operations one might to perform on a branch:

 - creating a branch
 - advancing a branch
 - studying the history of development on a single branch
 - fetching and pushing
 - renaming a branch
 - deleting
 - comparing the history of two branches
 - merging and deleting the now-merged-in branch
 - merging and not deleting the now-merged-in branch
 - "context switch" from working on one branch to another

In particular, you are right to emphasize that in git's model, "what was on the foo branch at 12:00pm on Saturday" is just not a well defined question. The foo branch tip might have been at commit A on my machine, commit B on a server we share, and commit C on your machine, so Git doesn't even bother --- branch refs are pointers into history rather than participating in the history themselves.

Instead, we can content ourselves with the answers to questions like "what was in the foo branch at 12pm on Saturday on the server used as a rendezvous point", by logging in and examining the reflog on the server.

A nice side-benefit is that branch refs are considered to be _private_ unless they are pushed or mentioned in a commit message. So there is no pressure to come up with good names for them.

Hope that helps, Jonathan

Andreas Ericsson· Aug 24, 2011, 15:41 UTC · re: Michael Witten · lore

Re: How to check out the repository at a particular point in time

On 08/23/2011 05:54 PM, Michael Witten wrote:
Show 31 quoted lines
> On Mon, Aug 22, 2011 at 15:18, R. Diez<rdiezmail-temp2@yahoo.de>  wrote:
>>> The main problem is that your request is not very
>>> well-defined: in
>>> nonlinear history there will in general be more than one
>>> commit at the
>>> time requested.
>>>
>>>      ---a----b----c----M----  (M is a merge)
>>>          \            /
>>>           d-----e----f
>>>
>>>               ^----April 1st
>>>
>>> Suppose you ask git for "the newest commit as of April 1st"
>>> in this history.  Is it supposed to give you b or d?
>>
>> I still don't quite understand how git works, but let me
>> risk a naive statement here. If "a-b-c-M" were 'master',
>> and "d-e-f" were 'new-feature', then on April 1st the
>> current version on 'master' is 'b', because I merged the
>> 'new-feature' branch at a later point in time. Does that
>> make sense?
> 
> O! for the love all that is Holy!
> 
> You see, guys? The term `branch' was a TERRIBLE choice.
> 
> What git calls `branch master' in your example is just a pointer to
> the commit object `M'; it has nothing to do with particular lineages
> like `a-b-c-M'.
> 

Back in 2005 when git was young and fresh, there was a discussion about what to call things. If memory serves (which it might not), I think the consensus was that "branch" works just fine, and when someone who doesn't like it comes along we can just tell them that it's short for "tip-of-branch pointer", which is far more accurate. A "ref" is always local though, which is why the reflog (which is used for such date resolving problems) is never even considered to work on remote refs. They *can* work on remotes' refs though, which is a slightly different thing.

Whatever, really. The fact that pretty much everyone seems to know what a branch is and how it works in git after a (very) brief intro to it means it's either right on target or that people are so used to the fact that branch means something different in every scm that they don't even bother loading the word with some preconceived notion that used to be right in cvs.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.
Randal L. Schwartz· Aug 24, 2011, 15:48 UTC · re: Andreas Ericsson · lore

Re: How to check out the repository at a particular point in time

>>>>> "Andreas" == Andreas Ericsson <ae@op5.se> writes:

Andreas> Whatever, really. The fact that pretty much everyone seems to know Andreas> what a branch is and how it works in git after a (very) brief intro Andreas> to it means it's either right on target or that people are so used to Andreas> the fact that branch means something different in every scm that they Andreas> don't even bother loading the word with some preconceived notion that Andreas> used to be right in cvs.

It does lead to confusion though, and once you *really* understand that a branch is a point, not a line, you can do a lot more cool things with git, and rebase and stash make a lot more sense.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion
Michael Witten· Aug 24, 2011, 16:18 UTC · re: Randal L. Schwartz · lore

Re: How to check out the repository at a particular point in time

On Wed, Aug 24, 2011 at 15:48, Randal L. Schwartz <merlyn@stonehenge.com> wrote:
Show 12 quoted lines
>>>>>> "Andreas" == Andreas Ericsson <ae@op5.se> writes:
>
> Andreas> Whatever, really. The fact that pretty much everyone seems to know
> Andreas> what a branch is and how it works in git after a (very) brief intro
> Andreas> to it means it's either right on target or that people are so used to
> Andreas> the fact that branch means something different in every scm that they
> Andreas> don't even bother loading the word with some preconceived notion that
> Andreas> used to be right in cvs.
>
> It does lead to confusion though, and once you *really* understand that
> a branch is a point, not a line, you can do a lot more cool things with
> git, and rebase and stash make a lot more sense.

Indeed. I don't think Andreas is correct in the assertion that "everyone seems to know what a branch is and how it works in git after a (very) brief intro". Rather, I think it is just the case that everyone seems to know how to use the basic public interface for branches after a (very) brief intro, and that many tasks allow everyone to get away with having wrongheaded ideas.

← back to recent threads