Re: [PATCH 0/4] Run hooks in parallel

From: Adrian Ratiu <adrian.ratiu@collabora.com>
Date: Feb 12, 2026, 14:24 UTC
Message-ID: <87jywiox9f.fsf@collabora.com>
In-Reply-To: <eedde9c3-2aff-433b-9dc3-f8593d53dec0@gmail.com>

On Thu, 12 Feb 2026, Phillip Wood <phillip.wood123@gmail.com> wrote:

Show 30 quoted lines

> Hi Adrian
>
> On 04/02/2026 17:33, Adrian Ratiu wrote:
>> Hello everyone,
>> 
>> This enables running hook commands in parallel and is based on the patch
>> series enabling config hooks [1], which added the ability to run a list
>> of hooks for each hook event.
>> 
>> For context, hooks used to run sequentially due to hardcoded .jobs == 1
>> in hook.c, leading to .processes == 1 in run-command.c. We're removing
>> that restriction for hooks known to be safe to parallelize.
>> 
>> The parallelism enabled here is to run multiple hook commands/scripts
>> in parallel for a single event, for example the pre-push event might
>> trigger linters / spell checkers / unit tests to run at the same time.
>> 
>> Another kind of parallelism is to split the hook input to multiple
>> child processes, running the same command in parallel on subsets of
>> the hook input. This series does not do that. It might be a future
>> addition on top of this, since it's kind of a lower-level parallelism.
>
> There's quite a lot of prior-art on parallelization from the various 
> hook managers - is there anything we can learn from them? For example I 
> know some of them serialize the pre-commit hook by default as it may 
> update the index but allow the user to configure a subset of scripts 
> that can be parallelized. They also allow for parallelization where 
> different scripts update different files (e.g. code formatters for 
> python and C can run in parallel). We don't need to implement all that 
> now but we should design our config so that we can support it in the future.

Yes, all the prior-art is very useful and it is possible to do finer-grained (or lower-level? :-) ) parallelism further with the new run-command parallelization design, APIs and config.

Obviously that will require more work and doing careful analysis on each hook-by-hook case. I'm just adding the basic buliding blocks and enabling the "highest-level" (most... independent?... level between tasks) of parallelism here.

My approach to this big problem was to simplify and break it down into smaller / easier-to-manage chunks. I'm still splitting up commits and untangling logic to ensure each part is done properly (also easier to review) and can work independently (no regresions etc) before building on top of it.

Thank you, and everyone else, so much for all the help and patience.

Show 9 quoted lines

>> The pre-push hook is special because it is the only known hook to break
>> backward compatibility when running in parallel, due to run-command
>> collating its outputs via a pipe, so I added an extension for it.
>> Users can opt-in to this extension with a runtime config.
>
> In the past we had a regression report [1] when the pre-commit hook 
> stopped having access to the terminal. I've not been following the hook 
> changes, is this series (or any of your preparatory series) in danger of 
> reintroducing that regression?

Thank you for raising this, it is a very valuable data point!

The preparatory series 100% will not reintroduce it.

This series might reintroduce it, depending how we set the defaults.

By that I mean:
    -j1 will keep all hooks connected to the tty, just like before.
    -jN with N>1 will disconnect the hooks from the tty and their
        outputs will get buffered through run-command's pipes.

The design I followed (which to be transparent is Peff's design, I just implemented his ideas :), is the keep the original, serialized behavior exactly how it was before, to not introduce any regressions and to also keep identical "real-time" performance.

Taking this into account, together with Patrick's feedback on this series, I do intend keep the jobs == 1 default for all hooks in v2.

>
> Thanks for working on this - both config based hooks and parallel 
> execution are really nice improvements.

Thank you for the kind words. :)