Re: [PATCH/RFC v3 01/13] Move index v2 specific functions to their own file

From: Thomas Gummerer <t.gummerer@gmail.com>
Date: Aug 10, 2012, 15:40 UTC
Message-ID: <20120810154015.GF5127@tommy-fedora.scientificnet.net>
In-Reply-To: <7vobmilro1.fsf@alter.siamese.dyndns.org>

On 08/10, Junio C Hamano wrote:

Show 27 quoted lines

> Thomas Rast <trast@student.ethz.ch> writes:
> 
> > But I think the idea always was that any write that changes the basic
> > layout of the file (so that you would read something wrong) will need a
> > full rewrite.  Otherwise we're too far in DB land.
> >
> > Most updates will be
> > of the "update the stat and/or sha1 of a file" kind, anyway.
> 
> Yes, I agree the v5 format documented in the series does not let you
> do anything other than the kind of updates without rewriting [*1*]
> 
> But that does not fundamentally change the story that a new format
> and a new way to access the index to cope with larger projects would
> want to come up with a solution to address the competing read/write
> issue, or at least help to make it easier to solve the issue in the
> future.
> 
> "That problem is not new" is not an answer when the question is "We
> still have the problem".
> 
> 
> [Footnote]
> 
> *1* While my gut feeling matches your guess that the kind of updates
> would be the majority, I do not think anybody did numbers to
> substanticate it, which we may want to see happen.

Hrm anther way to solve this may be the following. The idea would be to just check if the index_file changed basically using the same heuristic we already use to detect file changes. (use the stat data, mtime, size, etc.)

With this code we do not rely on the crc sums to check if the index needs to be re-read anymore and don't have a problem if part of the index changes, after we read it (we know the index changed from its mtime and can just re-read it). Another thing that would have to change is that we can't use die if a crc is wrong, but some return code, but that shouldn't be a problem. I'm not sure I'm not missing something here though.

do {
	fd = open()
	fstat(fd, &st_old);
	mmap = xmmap(fd);
	verify_various_fields(mmap);
	istate->ops->read_index(istate,
				mmap,
				mmap_size));
	fstat(fd, &st_new);
	close(fd);
} while (stat_data_doesnt_match(st_old, st_new));