From: david@lang.hm
Date: Mon, 20 Dec 2010 05:52:09 GMT
Subject: Re: [PATCH] convert filter: supply path to external driver
Message-ID: <alpine.DEB.2.00.1012192151090.20181@asgard.lang.hm>
In-Reply-To: <20101220022409.GA4625@sigill.intra.peff.net>

On Sun, 19 Dec 2010, Jeff King wrote:

> On Sun, Dec 19, 2010 at 01:59:50PM -0800, Junio C Hamano wrote:
>
>> Modulo one worry.  Don't we have, or don't we at least plant to allow us
>> to have, a facility to cache expensive blob conversion result, similar to
>> the textconv caching?  How would this change interact with two blobs that
>> live in different paths?
>
> Yeah, it has been talked about, but I don't think anyone is working on
> it (I don't personally use clean/smudge at all, so it is not something I
> have thought _that_ much about).
>
> This does definitely complicate matters, as the filtering is no longer a
> pure mapping of sha1->sha1. However, I think in practice we could do
> just fine by using a multi-level lookup. I.e., mapping a sha1 to be
> filtered into a tree. Each tree entry would represent the remaining
> cache parameters. In this case, the only other parameter we have is the
> path given to the filter (but it could easily be extended to include
> other parameters, if they existed, in this or other caching cases).
>
> We only get a high-performance lookup for the first part of the
> multi-level (i.e., the sha1), but that's OK if we assume the number of
> second-level items is going to be small. Which I think is the case here
> (a sha1 will tend to be found only under one or a few names).
>
> An alternative would be to combine all parts of the filter under a
> single lookup key. E.g., calculate and store under sha1(sha1(blob) +
> filename)). But that means the notes keys are not actual object sha1s,
> which throws off pruning.
>
> Anyway, that's just my quick thinking on the subject. I don't see any
> reason to restrict a feature just because we might want to cache it in
> the future. At the very worst, we could always cache filters which do
> not use %s, and make only %s users pay the penalty.

or you could cache the stats of the filtered file, including any changes 
that are made.

David Lang