Automated reviews by AI (was Re: [PATCH 0/5] Duplicate entry hardening)
- From
Christian Couder <christian.couder@gmail.com>
- Date
- Jun 12, 2026, 13:29 UTC
- Message-ID
- <CAP8UFD35cLP6FcEuPr+SghKae1ew4JWLWYAoMQ-fuEOu-JmZdg@mail.gmail.com>
- In-Reply-To
- <ah2PLBluBFy44AQI@pks.im>
On Tue, Jun 2, 2026 at 8:16 AM Patrick Steinhardt <ps@pks.im> wrote:
> > On Mon, Jun 01, 2026 at 09:33:10PM +0900, Junio C Hamano wrote:
Show 32 quoted lines
> > This is a fix to an important corner of our system, but somehow left > > in "Needs review" state for much longer than I would have liked, so > > even though I am officially on vacation ;-), I took some time to > > read these through (by the way it was a pleasant read, thank you). > > Honestly, I always shy away from the merge-related subsystems. It has a > lot of subtleties that I don't have any experience with, so I never > really consider my input to be helpful here. > > > I wonder if we create a rule like > > > > Those of you who have more than 30 commits in our project are > > expected to review one topic (or more) from other contributors > > for every three patches you send and ask for reviews by others. > > Heh, that would make me condense patch series into fewer patches ;) > > > it would help balance the patch vs review ratio, perhaps? > > It's a good question. I typically try to aim for reviewing series on the > mailing list at least every second day, and I always encourage other > folks in my team to do the same. But recently I (well, rather we) > haven't really been able to due to the current situation at GitLab, > which forces us to put almost all of our focus towards a different > project for a while. > > Overall I agree that everyone who is a core contributor should also make > reviews part of their regular worflow. At least for corporate > contributors that might also make it easier to communicate this to their > respective employers. Regardless of that, my expectation is that there > will be times where it works well, and other times where it works less > well.
Sashiko (https://github.com/sashiko-dev/sashiko) is used these days by Linux kernel developers and seems to work well for them.
At GitLab and probably in other companies, some of us also use AI to review our work before sending it to the mailing list. And yeah, it helps find issues before our patches reach the mailing list.
In the same way as we require that patches must pass CI, do we want to require that patches "pass" an AI review before they get accepted?
The benefit would be that it would hopefully catch a lot of trivial things like indentation, typos/grammos, etc, and a lot of things a bit more difficult to spot like memory issues. Perhaps with some amount of prompting/configuration (for example pointing it at our CodingGuidelines and SubmittingPatches) it could also catch issues like style issues, commits that do too many things, refactoring opportunities, etc.
We would likely still require at least one human review (by someone who is not the maintainer) to validate architectural decisions, to make sure it goes in the same direction as other efforts, and perhaps also to make sure that AI suggestions were properly handled by the patch author.
If we decide to require it, then there are a lot of questions that we will have to answer.
Do we want to have our own system somehow managed by us or would we be happy to use existing systems already in place in some companies as long as we can still tweak them in some ways, like the current CI systems we use?
If we use existing systems likely at GitLab and GitHub, it might be more difficult to get coherent results as they might use different LLMs, but maybe it could help tighten our docs to make sure everyone is aligned, and we could get better reviews by using multiple systems because an LLM might find an issue that the other LLM missed.
Do we want an AI review right after a patch is posted or only if there is no human review in the next X days?
Also what if the AI makes a long concrete suggestion to improve on the patches? Could that be incompatible with our AI policy to apply it? Should we try to prevent the AI from making such a suggestion in the first place?
I haven't looked at how Sashiko is used for the kernel, but maybe there will need to be some kinds of restrictions/authentications to avoid potential abuse.