threads / discuss / 61592

RFC: indicating diff strategy in format-patch message headers

Subject: RFC: indicating diff strategy in format-patch message headers

## tl;dr

6 messages between Jun 5, 2024 and Jun 16, 2024.

replies: 5people: 3as markdown or json

Konstantin Ryabitsev· Jun 5, 2024, 18:01 UTC · lore
Hello, all:

When developing tooling that attempts to map commits to patches, we often don't have more than just the git-patch-id to go by. The problem is, there is any number of ways to generate patches from commits:

- using a different strategy (--histogram, --patience, etc)
- using a different number of context lines (-U5)

Without knowing what options were used by the original author, we cannot be certain that we'll create a patch with the same git-patch-id, even if it's from the exact same commit.

Would it make sense to have git-format-patch (and friends) include an additional header hinting at the options used to generate the patch? E.g.:

    X-git-diff-options: algo=myers; context=3;

It won't help for all cases where we need to make a match (e.g. when we match from git commits to a patch query), but it will help matching the other way.

Any thoughts?
-K
Junio C Hamano· Jun 5, 2024, 18:22 UTC · re: Konstantin Ryabitsev · lore

Re: RFC: indicating diff strategy in format-patch message headers

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:
> Would it make sense to have git-format-patch (and friends) include an
> additional header hinting at the options used to generate the patch? E.g.:
>
>     X-git-diff-options: algo=myers; context=3;

I doubt it. And newer version of Git _will_ try to improve how patch text appears to be more useful to human users, so you have more moving parts than you'd want to even think about enumerating.

If you were to add a new e-mail header, wouldn't it make more sense to add a patch-id header and agree on the set of options to be used to generate that patch-id (which might be different from the setting used to format the real patch for human and "git am" consumption)?

Konstantin Ryabitsev· Jun 5, 2024, 18:46 UTC · re: Junio C Hamano · lore

Re: RFC: indicating diff strategy in format-patch message headers

On Wed, Jun 05, 2024 at 11:22:08AM GMT, Junio C Hamano wrote:
Show 8 quoted lines
> > Would it make sense to have git-format-patch (and friends) include an
> > additional header hinting at the options used to generate the patch? E.g.:
> >
> >     X-git-diff-options: algo=myers; context=3;
> 
> I doubt it.  And newer version of Git _will_ try to improve how
> patch text appears to be more useful to human users, so you have
> more moving parts than you'd want to even think about enumerating.
OK, I was afraid that was the case.
> If you were to add a new e-mail header, wouldn't it make more sense
> to add a patch-id header and agree on the set of options to be used
> to generate that patch-id (which might be different from the setting
> used to format the real patch for human and "git am" consumption)?

That would be redundant with the message-id. Unfortunately, this doesn't solve the problem of how to reliably map a commit to the patch from which it originated, other than using the Message-ID: or Link: trailers.

I'm always happy to see a Link: trailer pointing at the patch submission, e.g.:

    | Link: https://msgid.link/[msgid]

However, not all maintainers welcome them in commits (e.g. Linus was vocal about it in the past [1]). It could be the case of "use them anyway," but we'll still have a large enough subset of commits without this info and increasingly without a reliable mechanism to map commits to original submissions and code review systems.

I'm not sure if there's a reliable way to solve this.
-K
[1]: https://lore.kernel.org/lkml/CAHk-=wgAk3NEJ2PHtb0jXzCUOGytiHLq=rzjkFKfpiuH-SROgA@mail.gmail.com/
Eric Wong· Jun 5, 2024, 22:26 UTC · re: Konstantin Ryabitsev · lore

Re: RFC: indicating diff strategy in format-patch message headers

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
Show 5 quoted lines
> On Wed, Jun 05, 2024 at 11:22:08AM GMT, Junio C Hamano wrote:
> > > Would it make sense to have git-format-patch (and friends) include an
> > > additional header hinting at the options used to generate the patch? E.g.:
> > >
> > >     X-git-diff-options: algo=myers; context=3;

Fwiw, I use format-patch command-line + options in the --signature= switch when generating patches in WIP git repo viewer:

https://yhbt.net/lore/pub/scm/git/git.git/6549c41e/s/0001.patch
Show 8 quoted lines
> > If you were to add a new e-mail header, wouldn't it make more sense
> > to add a patch-id header and agree on the set of options to be used
> > to generate that patch-id (which might be different from the setting
> > used to format the real patch for human and "git am" consumption)?
> 
> That would be redundant with the message-id. Unfortunately, this doesn't solve
> the problem of how to reliably map a commit to the patch from which it
> originated, other than using the Message-ID: or Link: trailers.
dfpre:, dfblob:, dfpost: search queries on lore seem to work...

On my lore + gko mirror, the #related search off a commit mostly works for a single extindex, but mapping hundreds of forks to hundreds of inboxes with a presentable UI for w3m||lynx users is a different matter I haven't been able to solve:

https://yhbt.net/lore/pub/scm/git/git.git/6549c41e/s/#related
> I'm not sure if there's a reliable way to solve this.

I started working on a scoring mechanism a while back but got distracted with personal matters. UI is hard, and dealing with my gko mirror has been a source of pain due to various memory+performance problems.

Konstantin Ryabitsev· Jun 7, 2024, 19:53 UTC · re: Eric Wong · lore

Re: RFC: indicating diff strategy in format-patch message headers

On Wed, Jun 05, 2024 at 10:26:58PM GMT, Eric Wong wrote:
Show 5 quoted lines
> > That would be redundant with the message-id. Unfortunately, this doesn't solve
> > the problem of how to reliably map a commit to the patch from which it
> > originated, other than using the Message-ID: or Link: trailers.
> 
> dfpre:, dfblob:, dfpost: search queries on lore seem to work...

I've thought about that, but this is also not very reliable, at least not when patch series are applied as fast-forwards, not merges. Unfortunately, some projects enforce a flat history (glibc, gcc), with merges being specifically not allowed -- which means dfblob matching is not going to match a lot of commits.

Show 13 quoted lines
> On my lore + gko mirror, the #related search off a commit mostly
> works for a single extindex, but mapping hundreds of forks to
> hundreds of inboxes with a presentable UI for w3m||lynx users is
> a different matter I haven't been able to solve:
> 
> https://yhbt.net/lore/pub/scm/git/git.git/6549c41e/s/#related
> 
> > I'm not sure if there's a reliable way to solve this.
> 
> I started working on a scoring mechanism a while back but got
> distracted with personal matters.  UI is hard, and dealing with
> my gko mirror has been a source of pain due to various
> memory+performance problems.

I can totally relate to the pain of memory+performance problems. :) Especially when a ton of "AI" bots decide to aggressively crawl your sites. :/

-K
Eric Wong· Jun 16, 2024, 23:47 UTC · re: Konstantin Ryabitsev · lore

Re: RFC: indicating diff strategy in format-patch message headers

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
Show 12 quoted lines
> On Wed, Jun 05, 2024 at 10:26:58PM GMT, Eric Wong wrote:
> > > That would be redundant with the message-id. Unfortunately, this doesn't solve
> > > the problem of how to reliably map a commit to the patch from which it
> > > originated, other than using the Message-ID: or Link: trailers.
> > 
> > dfpre:, dfblob:, dfpost: search queries on lore seem to work...
> 
> I've thought about that, but this is also not very reliable, at least not when
> patch series are applied as fast-forwards, not merges. Unfortunately, some
> projects enforce a flat history (glibc, gcc), with merges being specifically
> not allowed -- which means dfblob matching is not going to match a lot of
> commits.

Yeha, my initial plan (way back around 2016 or 2017) was to use Subject; but I also forgot why I stopped with that idea :x

Anyways, trying to add bs: (body+subject) to queries in addition to dfblob may prove useful:

https://public-inbox.org/meta/20240616233532.574646-1-e@80x24.org/
Not sure if author + date is necessary...

← back to recent threads