Re: [StGit PATCH 03/14] Write to a stack log when stack is modified

From: Catalin Marinas <catalin.marinas@gmail.com>
Date: Jun 18, 2008, 16:16 UTC
Message-ID: <b0943d9e0806180916m4af3970ck347408661e95663@mail.gmail.com>
In-Reply-To: <20080618143633.GB30540@diana.vm.bytemark.co.uk>

2008/6/18 Karl Hasselström <kha@treskal.com>:

Show 12 quoted lines

> On 2008-06-18 14:03:51 +0100, Catalin Marinas wrote:
>
>> 2008/6/17 Karl Hasselström <kha@treskal.com>:
>>
>> > The log actually _contains_ those trees, so there is no problem.
>>
>> OK, I begin to understand. It is a generic solution for storing
>> metadata
>
> I don't know about "generic" -- it's made specifically for storing an
> StGit stack. You could certainly store just about anything in a git
> tree, but that's also true for a git commit message ...

By generic I meant that it is easily extensible to store other blobs of whatever you need. As you say, a commit message could be extensible as well.

Show 6 quoted lines

>> but IMHO it is too overweight when we only need a list of applied
>> and unapplied files (we can remove hidden) that could be easily
>> stored in a commit log.
>
> Do you mean remove hidden patches from StGit altogether, or just not
> store them in the log?

Remove them altogether. Is there anyone using them apart from me? I could create a "rotten" branch and pick those patches with --unapplied.

Show 8 quoted lines

>> It would be useful to run a quick benchmark with many (hundreds) of
>> patches and compare it with no logging variant (or the current patch
>> logging, which isn't as advanced).
>
> Will do, as soon as I've done some basic optimization. (A simple
> "dirty" flag on each patch will enable us to only write out the log
> for the patches that have actually changed, and reuse the rest from
> the previous log entry.)

It gets too complicated, really. A single commit with the proper parents could do the job. We could also easily use the commit message directly for metadata instead of .git/patches/<branch>/.

Show 21 quoted lines

>> Could we not make this (much) simpler? I.e. <branch>.stgit is a
>> commit object whose tree is <branch>^{tree} and the message contains
>> the command followed by list of patches in the form "<commit>
>> <patch>"? This commit can have two parents - the previous
>> <branch>.stgit and current <branch> head. All the patches are
>> referred via the <branch> head or the previous <branch> heads if
>> unapplied (assuming that when a patch is created it is first applied
>> and then popped, maybe this needs a bit of thinking).
>
> I considered a scheme much like this, but besides the problem with
> (the very few) patches that are created unapplied, it fails badly in a
> very important corner case: when you start logging in a pre-existing
> stack. A similar failure will occur if we ever build some sort of log
> pruning (without which the log will grow without limit).
>
> I suppose it would be possible to special-case the very first log
> entry: let it have all patches as ancestors. But I don't really see
> this format as being simpler than the one I use now: all you've done,
> basically, is cram all the data into the commit message instead of
> storing it in the tree. (Benchmarking could certainly prove your
> design superior to mine, however.)

The time to create that tree and blobs worries me a bit, plus the (in my view) complicated structure.

Making the first log entry special gets difficult with log pruning (unless you prune the whole log rather than entries older than a chosen time or number) since you might have to re-create all the chained log entries as the first log's sha1 will probably change.

The applied patches are chained automatically via HEAD. For unapplied patches, we could add the correponding commits as parents of the logging commit (starting with the third parent as the first two are used for log chaining and applied patches). Do we hit any OS limit with the number of arguments?

Since we only need the unapplied commits tracking for gc and pull reasons, we could only create that commit that references them when we prune the stack log and link it from the top one (but it won't be used by stgit).

-- 
Catalin