Re: [RFC] Long running Git clean/smudge filter

From: Lars Schneider <larsxschneider@gmail.com>
Date: Jul 12, 2016, 09:30 UTC
Message-ID: <4947BBA2-6260-4546-9641-E639A8C70174@gmail.com>
In-Reply-To: <20160710151046.GA7306@kitenet.net>

Show 10 quoted lines

> On 10 Jul 2016, at 17:10, Joey Hess <id@joeyh.name> wrote:
> 
> larsxschneider@gmail.com wrote:
>> (2) Joey's topic, which is the base for my patch, looks stalled for more than
>> 2 weeks:
>> http://thread.gmane.org/gmane.comp.version-control.git/297994/focus=298006
>> I would be happy to address Junio's comments and post a reroll. However,
>> I don't want to interfere with Joey's plans.
> 
> I've been on vacation and plan to get back to that in the upcoming week.

Good to hear :-) ! I hope you had a great vacation!

Show 16 quoted lines

>> @Joey (in case you are reading this):
>> My patch changes your initial idea quite a bit. However, I believe it is an
>> improvement that could be beneficial for git-annex, too. Would you prefer to
>> work with me on the combination of our ideas (file clean/smudge + long running
>> clean/smudge processes) or would you prefer to keep your interface?
> 
> Long running filters mean that you need a more complicated protocol to
> speak over the pipes. Seems that such a protocol could be designed to work
> with the original smudge/clean filters as well as with my
> smudgeToFile/cleanFromFile filters. Assuming that there's a way to
> tell whether the filters support being long-running or not.
> 
> Note that the interface we arrived at for smudgeToFile/cleanFromFile is as
> similar as possible to smudge/clean, so the filter developer only has to
> change one thing. That's a big plus, and so I don't like diverging the
> two interfaces.

I understand, thanks for the clarification. My plan is to implement long running filters only for smudgeToFile/cleanFromFile (at least initially) as this should solve the major pain point already and is relatively straight forward.

What do you think about this kind of config? Any idea for a better config name?

[filter "bar"]
    clean = foo1 %f
    smudge = foo2 %f
    cleanFromFile foo3
    smudgeToFile foo4
    longRunning = true

I think it would be easy to support both of our interfaces in parallel. Do you understand why the "async" API is necessary? https://github.com/larsxschneider/git/blob/74e22bd4e0b505785fa8ffc2ef15721909635d1c/convert.c#L585-L599

Would you be OK if I implement smudgeToFile/cleanFromFile as separate "apply_filter" function as I have prototyped here: https://github.com/larsxschneider/git/blob/74e22bd4e0b505785fa8ffc2ef15721909635d1c/convert.c#L1143-L1146 https://github.com/larsxschneider/git/blob/74e22bd4e0b505785fa8ffc2ef15721909635d1c/convert.c#L402-L488

Thanks, Lars