threads / discuss / 139

naive question

Subject: naive question

## tl;dr

6 messages between Apr 19, 2005 and Apr 19, 2005.

replies: 5people: 5as markdown or json

Paul Mackerras· Apr 19, 2005, 13:00 UTC · lore

Is there a way to check out a tree without changing the mtime of any files that you have already checked out and which are the same as the version you are checking out? It seems that checkout-cache -a doesn't overwrite any existing files, and checkout-cache -f -a overwrites all files and gives them the current mtime. This is a pain if you are using make and your tree is large (like, for instance, the linux kernel :), because it means that after a checkout-cache -f -a you get to recompile everything.

Paul.
David Woodhouse· Apr 19, 2005, 13:19 UTC · re: Paul Mackerras · lore

Re: naive question

On Tue, 2005-04-19 at 23:00 +1000, Paul Mackerras wrote:
Show 8 quoted lines
> Is there a way to check out a tree without changing the mtime of any
> files that you have already checked out and which are the same as the
> version you are checking out?  It seems that checkout-cache -a doesn't
> overwrite any existing files, and checkout-cache -f -a overwrites all
> files and gives them the current mtime.  This is a pain if you are
> using make and your tree is large (like, for instance, the linux
> kernel :), because it means that after a checkout-cache -f -a you get
> to recompile everything.
Corollary: why aren't we storing mtime in the tree objects?
-- 
dwmw2
Ingo Molnar· Apr 19, 2005, 13:51 UTC · re: David Woodhouse · lore

Re: naive question

* David Woodhouse <dwmw2@infradead.org> wrote:
Show 11 quoted lines
> On Tue, 2005-04-19 at 23:00 +1000, Paul Mackerras wrote:
> > Is there a way to check out a tree without changing the mtime of any
> > files that you have already checked out and which are the same as the
> > version you are checking out?  It seems that checkout-cache -a doesn't
> > overwrite any existing files, and checkout-cache -f -a overwrites all
> > files and gives them the current mtime.  This is a pain if you are
> > using make and your tree is large (like, for instance, the linux
> > kernel :), because it means that after a checkout-cache -f -a you get
> > to recompile everything.
> 
> Corollary: why aren't we storing mtime in the tree objects?

Check the "[bug] git: check-files mtime problem?" thread - i noticed this problem before and gave a few suggestions but the discussion got nowhere. But the problem is still very much present.

	Ingo
Petr Baudis· Apr 19, 2005, 17:15 UTC · re: Paul Mackerras · lore

Re: naive question

Dear diary, on Tue, Apr 19, 2005 at 03:00:14PM CEST, I got a letter where Paul Mackerras <paulus@samba.org> told me that...

Show 8 quoted lines
> Is there a way to check out a tree without changing the mtime of any
> files that you have already checked out and which are the same as the
> version you are checking out?  It seems that checkout-cache -a doesn't
> overwrite any existing files, and checkout-cache -f -a overwrites all
> files and gives them the current mtime.  This is a pain if you are
> using make and your tree is large (like, for instance, the linux
> kernel :), because it means that after a checkout-cache -f -a you get
> to recompile everything.

Actually, to then get sensible show-diff output, you need to also update-cache --refresh to compensate for the changes. I personally really hate update-cache --refresh; sure, 0.1s with hot cache, but easily eats 5 minutes (!) with cold cache.

I'd actually prefer, if:

(i) checkout-cache simply wouldn't touch files whose stat matches with what is in the cache; it updates the cache with the stat informations of touched files

(ii) read-tree would take over the stat information from the matching files in previous cache.

This way, doing update-cache --refresh would become a rather rare event. Stuff would become swifter, faster, less I/O bound and you would get rid of problems as the one described above.

What do you think?
-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
Linus Torvalds· Apr 19, 2005, 17:41 UTC · re: Petr Baudis · lore

Re: naive question

On Tue, 19 Apr 2005, Petr Baudis wrote:
Show 6 quoted lines
> 
> I'd actually prefer, if:
> 
> (i) checkout-cache simply wouldn't touch files whose stat matches with
> what is in the cache; it updates the cache with the stat informations
> of touched files

Run "update-cache --refresh" _before_ doing the "checkout-cache", and that is exactly what will happen.

But yes, if you want to make checkout-cache update the stat info (Ingo wanted to do that too), it should be possible. The end result is a combination of "update-cache" and "checkout-cache", though: you'll effectively need to both (just in one pass).

With the current setup, you have to do
	update-cache --refresh
	checkout-cache -f -a
	update-cache --refresh
which is admittedly fairly inefficient.

The real expense right now of a merge is that we always forget all the stat information when we do a merge (since it does a read-tree). I have a cunning way to fix that, though, which is to make "read-tree -m" read in the old index state like it used to, and then at the end just throw it away except for the stat information.

		Linus
Linus Torvalds· Apr 19, 2005, 18:27 UTC · re: Linus Torvalds · lore

Re: naive question

On Tue, 19 Apr 2005, Linus Torvalds wrote:
Show 6 quoted lines
> 
> The real expense right now of a merge is that we always forget all the
> stat information when we do a merge (since it does a read-tree). I have a
> cunning way to fix that, though, which is to make "read-tree -m" read in
> the old index state like it used to, and then at the end just throw it
> away except for the stat information.

Ok, done. That was really the plan all along, it just got dropped in the excitement of trying to get the dang thing to _work_ in the first place ;)

The current version only does
	read-tree -m <orig> <branch1> <branch2>

which now reads the old stat cache information, and then applies that to the end result of any trivial merges in case the merge result matches the old file stats. It really boils down to this littel gem;

            /*
             * See if we can re-use the old CE directly?
             * That way we get the uptodate stat info.
             */
            if (path_matches(result, old) && same(result, old))
                    *result = *old;
and it seems to work fine.
HOWEVER, I'll also make it do the same for a "single-tree merge":
	read-tree -m <newtree>

so that you can basically say "read a new tree, and merge the stat information from the current cache". That means that if you do a "read-tree -m <newtree>" followed by a "checkout-cache -f -a", the checkout-cache only checks out the stuff that really changed.

You'll still need to do an "update-cache --refresh" for the actual new stuff. We could make "checkout-cache" update the cache too, but I really do prefer a "checkout-cache only reads the index, never changes it" world-view. It's nice to be able to have a read-only git tree.

Final note: just doing a plain "read-tree <newtree>" will still throw all the stat info away, and you'll have to refresh it all...

		Linus

← back to recent threads