Re: [PATCH v1 3/3] convert: add filter.<driver>.useProtocol option

From: Duy Nguyen <pclouds@gmail.com>
Date: Jul 25, 2016, 15:48 UTC
Message-ID: <CACsJy8D_21pY5FRs4VCpDUuUJ56OZT8DZ1ZKMiwWKeeSMiLSdA@mail.gmail.com>
In-Reply-To: <121F8453-3A12-47AE-AE10-021EADC4D995@gmail.com>

On Sun, Jul 24, 2016 at 9:11 PM, Lars Schneider <larsxschneider@gmail.com> wrote:

Show 12 quoted lines

>
> On 23 Jul 2016, at 10:14, Eric Wong <e@80x24.org> wrote:
>
>> larsxschneider@gmail.com wrote:
>>> Please note that the protocol filters do not support stream processing
>>> with this implemenatation because the filter needs to know the length of
>>> the result in advance. A protocol version 2 could address this in a
>>> future patch.
>>
>> Would it be prudent to reuse pkt-line for this?
>
> Peff suggested that, too, in $gmane/299902.

And I was about to suggest the same too, until I saw his patch then stopped. Having a common way to split a byte stream to a packet stream could be a good thing.

> However, this would make the protocol a bit more complicated

For high level scripting languages, pkt-line is dead simple. If your scripts are in sh then it could get a bit ugly, but I'm thinking of a small utility to make shell scripting pkt-line easier anyway (it goes back to the idea of rewriting index-helper as a script, which I might do).

> and it wouldn't buy us anything for Git
> large file processing filters (my main motivation for this patch) as these
> filters can't leverage streaming anyways.

This is a good point. How are you planning to do it? Unless streaming is done entirely in kernel (sendfie() and friends, which is not all positive), I think you can still stream and wrap/unwrap pkt-line just fine.

-- 
Duy