Re: [PATCH 1/2] add setup step to filter-branch

From: Jeff King <peff@peff.net>
Date: Jun 9, 2017, 05:49 UTC
Message-ID: <20170609054955.y52cro2c3bwhl2sh@sigill.intra.peff.net>
In-Reply-To: <xmqq1sqzuqmh.fsf@gitster.mtv.corp.google.com>

On Mon, Jun 05, 2017 at 11:15:18AM +0900, Junio C Hamano wrote:

Show 19 quoted lines

> Andreas Heiduk <asheiduk@gmail.com> writes:
> 
> > A specific `--setup` step in `git filter-branch` makes it much easier
> > to define the initial values of variables used in the real filters.
> > Also sourcing/defining utility functions here instead of
> > `--env-filter` improves performance and minimizes clogging the output
> > in case of errors.
> >
> > Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
> > ---
> 
> I was placed on To: line, but I do not have a strong opinion on this
> change, either for or against.
> 
> "filter-branch" program itself may probably already be hard to port
> to C, but I need to point out that this makes it even harder than it
> currently is [*1*], and it is likely that it has to stay implemented
> in shell forever, though.  I do not mind that future myself, but
> those on platforms with weaker implementation of shells might.

I have a feeling that if we were ever to rewrite filter-branch, it would probably be worth allowing people to write snippets in a better language (possibly even a domain-specific language). I'm sure that most of the program being written in shell doesn't help, but if we're spawning one or more shell instances per commit (plus the Git programs they spawn!), it's always going to be slow.

But I suspect that would be an uphill battle, as our only stable API involves starting external processes anyway. You'd probably do better to pick a language you like and rewrite it in using libgit2's bindings to that language. It's not feature complete, but basic stuff like "put this entry in the tree" is certainly mature.

Show 5 quoted lines

>  *1* The issue is *not* that these individual filter commands expect
>      <command> written as a shell scriptlet; it is that these
>      scriptlets expect to be evaled inside a single shell process,
>      making an update to a shell variable in one command visible to
>      the next command that runs.

I think you'd need a shell "helper" that's a single long-running process and just reads "eval the index snippet now" instructions from the C controller. At which point I don't think Andreas's "setup" feature is any harder to support. We just send an "eval the setup snippet" instruction first.

I guess one way to think of that is that the C program is itself generating a shell script.

-Peff