Storing Maintainers info around the kernel tree

Merging a couple related threads here:

On Aug 16, 2007, at 07:57:23, Rene Herman wrote:

 From a practical standpoint we don't want to duplicate someone's  
maintainer information in the attributes of every file they  
maintain.  It would be much easier to put in the "kernel/somesubsys"  
directory a Maintainers file which has:

[SOME RANDOM SUBSYSTEM]
P: J. Random Hacker
M: j.random.hacker@localhost
L: random-subsys-devel@vger.kernel.org
F: *

Anywhere else you had files that you wanted to associate with J. Random Hacker's maintainership, you would just use:

[SOME RANDOM SUBSYSTEM]
F: somesubsys.h

I posted a comment describing a mechanism like this a couple days ago:
   http://lkml.org/lkml/2007/8/14/488

Executive overview: On Aug 15, 2007, at 07:21:04, Rafael J. Wysocki wrote:

Well, to back up this idea with some code, I'm attaching a little  
perl script which does part (b).  Basically you call it as:
   ./maint-combine $(find . -name Maintainers)

It will print any syntax errors on stderr during parsing. Once it's done it will dump to stdout its combined "MAINTAINERS" text. A couple notes:

*  It uses a little "priority" system to figure out what order to  
print the data from each origin in.  For example, the "F:" tag is  
given a score of 0, to force data consisting of just files towards  
the end.  The "P:", "M:", and "L:" tags are given scores of 5, since  
people are generally interesting to know about.  Everything else is  
given a score of "1".  The scores are added up per ($file,  
$subsystem) pair and then during printing each subsystem's data is  
ordered by score (highest comes first).

*  It generally allows any field at all; eventually we might want to  
limit it to a fixed list to help avoid typos.

*  It has a little bit of magic logic for the "F:" field so that it  
figures out the relative directory for each field when generating the  
output.  For example, an entry of "asm-*/suspend.h" in a file  
"include/Maintainers" will produce the output file entry: "F: include/ 
asm-*/suspend.h"

*  The format isn't quite the same as the current MAINTAINERS file,  
to make parsing easier and more dummy-proof I changed the syntax for  
a subsystem-name to use square brackets (IE: "[SUSPEND TO RAM]").   
The samples I gave in my previous email are what I used to test it  
with, plus a little dummy file with some syntax errors to check out  
the error messages:

Maintainers:

kernel/power/Maintainers:

include/Maintainers:

If you have any other questions, the perl script is pretty self- explanatory and I'll be completely back online this weekend. With any luck I'll have some time in a hotel tomorrow (mmm, slow-as-dirt hotel wireless, what fun) to work on parts (c) and (d).

Cheers, Kyle Moffett

#! /usr/bin/perl

use strict; use warnings;

## This table is for determining what order to show entries from different
## source "Maintainers" files.  They are prioritized based on the number and
## significance of each field, as listed in this table.  Any field not found
## here will be assumed to have a significance of "1".
my %fieldprio = (
	p => 5,  ## Person
	m => 5,  ## Email
	l => 5,  ## Mailing List
	f => 0,  ## File
);

my %mprio; my %maint; my $section; my $origin; my $pathprefix; my $printorigin;

while (<>) {
	unless (defined $origin and $origin eq $ARGV) {
		undef $section;

		## Get rid of extra slashes, unnecessary "./" entries
		$ARGV =~ s{//+}{/}g;
		$ARGV =~ s{(^|/)\./}{$1};

		$origin = $ARGV;
	}

	chomp; s/\s+/ /g; s/^ //; s/ $//;
	/\S+/ or next;

	if (/^\[ ?([^]]+) ?\]$/) {
		$section = uc $1;

		$printorigin = 1 if $origin ne '-';

		$maint{$section}{$origin} ||= [];
		$mprio{$section}{$origin} ||= 0;

		## Figure out a useful path prefix if there is one
		$pathprefix = ($origin =~ m{(^.*/)(?!\.\.?$)[^/]+$})?$1:"";

		next;
	}

	unless (/^([^:]+?) ?: ?(.*)$/) {
		print STDERR "$ARGV:$.: Invalid line: $_\n";
		next;
	}

	my($field,$value) = (lc($1), $2);

	unless (defined $section) {
		print STDERR "$ARGV:$.: Found field '\u$field' before ",
				"any subsystem declaration: $_\n";
		next;
	}

	## Preprocess paths to make them absolute
	$value = $pathprefix.$value if $field eq 'f';

	## Update the sorting order
	if (exists $fieldprio{$field}) {
		$mprio{$section}{$origin} += $fieldprio{$field};
	} else {
		$mprio{$section}{$origin}++;
	}

	push @{$maint{$section}{$origin}}, [$field, $value];
}

sub priosort ( $@ )
{
	my $section = shift;
	return sort {$mprio{$section}{$b} <=> $mprio{$section}{$a}} @_;
}

for $section (sort {$a cmp $b} keys %maint) {
	print "[$section]\n";
	for my $origin (priosort $section, keys %{$maint{$section}}) {
		print "\nOrigin: $origin\n" if $printorigin;
		print "\u$_->[0]: $_->[1]\n" for @{$maint{$section}{$origin}};
	}
	print "\n\n";
}

# vim:set ft=perl:

On 08/16/2007 03:04 PM, Kyle Moffett wrote:

In a tree structure, you don't have to. As described earlier, the tool (git-prop) looks for the requested property being set first on the file itself, then on the directory in which it resides, then its parent, and so on. If I read things right, this is also how properties work in subversion in fact.

So after

$ git prop --set --name maintainer --value \
	"Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>" drivers/ide/
and

$ git prop --set --name maintainer --value \
	"Alan Cox <alan@lxorguk.ukuu.org.uk>" drivers/ide/ide-cd.*

we get:

$ git prop --get --name maintainer drivers/ide/ide-cd.c Alan Cox <alan@lxorguk.ukuu.org.uk>

$ git prop --get --name maintainer drivers/ide/ide-generic.c Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Now, this override behaviour needs a tree structure ofcourse, but notice I set the "maintainer" property only to the name/address. The other information from the MAINTAINERS file would be using their own properties:

$ git prop --set --name tree --value \
     "quilt kernel.org/pub/linux/kernel/people/bart/pata-2.6/" drivers/ide/

and nothing under drivers/ide/ would override this value nor would it be repeated anywhere. Alan takes care of more than ide-cd but only the actual "maintainer" value string would be set on the others as well and repeating that much for different "maintenance units" is no different from the current MAINTAINERS file where it also is (well, would be, Alan is in fact only listed for ide-cd it seems...) repeated in different entries.

(as a slight difference -- in the above example, Alan's information _is_ repeated over ide-cd.c and ide-cd.h where the current MAINTAINERS file just says "IDE/ATAPI CDROM DRIVER", but that's a bit of an oddbal situation since you normally have either single files or a tree that make up a "maintenance unit" -- and is in fact just a human versus tool difference).

It's ofcourse possible, but note that if we want this stuff to be minimally manual, moving files around (and deleting them) then requires editing these actual in-tree files via a tool.

With the properties deleting files just requires deleting any file-specific properties alongside which is trivial since those are linked from the file.

Moving stuff works by building a list of all properties that are set on the source starting at the source and destination's highest shared parent directory and then reconstructing this list at the destination, striking properties off the list that are already set at the destination.

Adding properties, alongside added files or after the fact, could be done via standard patch submissals via the kind of "meta-diff" that already exists for "git move".

I really believe this stuff should be meta-data -- and these properties as outlined work well it seems.

$ git prop --set --name git.ignore -V ./.gitignore . $ rm .gitignore

This is something I saw subversion also uses properties for. Takes the value from a file instead of the command line. .gitattributes are also easily incorporated into the property-system directly.

$ git prop --set --name git.executable scripts/Lindent

Must say I'm not particularly sure if this one has much value over the current executable bit storage,but also from svn and example of a boolean property.

$ git prop --set --name license --value "GPL v2" . $ git prop --set --name license --value "GPL" sound/alsa/

and so on. The GPL v2 on the source root only works if you set the property on everything that's not, so you may not want to but as a "wouldn't it be nice if" kinda thing. Makes for easy license analysis at least...

$ git prop --set --name FIXME drivers/block/floppy.c

Okay, that's probably overdoing it a bit, but as long as I'm having fun here...

Note -- the properties would be versioned themselves ofcourse so that you'd always have a tree where data and meta-data matched. Basically, I believe you'd view the properties as just more data files, one per property, with the exception that they'd not actually live in the working tree and are linked from data (files/directories) that do.

Long "letters of intent" this, but I'm by now in love with these things. More comments (or implementations obviously :-) welcome. Any significant misses in this?

Rene.

On Thu, 16 Aug 2007, Rene Herman wrote:

Please remember that not everybody uses git. The MAINTAINERS data should be available in the kernel source itself.

(Maybe your suggestion is consistent with this -- I simply wanted to raise the point.)

Alan Stern

On 08/16/2007 05:31 PM, Alan Stern wrote:

It may be useful to generate a MAINTAINERS file into releases yes.

I must say though that "why?" would also be a question. I personally don't think there's a whole lot wrong with more and more expecting people who submit patches (for whom this automation is intended) to be using git. Back in the BK days there were lots of reasons for resisting any and all dependency on the source code management tool but there don't seem to be too many left today as far as I'm concerned.

If it's about non-developer users, I suspect it would to a fairly large degree be an "in theory" thing to expect that said user does want the information in a downloaded releases, but not in git, and not online where git-web could also easily display all the information right alongside the files.

But yes, sure, anything can be generated...

Rene.

Rene Herman wrote:

You mean "people who frequently submit patches for various different subsystems".

On 08/16/2007 11:39 PM, Stefan Richter wrote:

Erm, I guess. Is that agreeing or disagreeing with me?

Rene.

On Fri, 17 Aug 2007, Rene Herman wrote:

Don't forget also that the MAINTAINERS information is (or should be!) used by people who want to submit bug reports, not just by people who submit patches. Bug reporters shouldn't need to use Git.

Alan Stern

On 08/17/2007 03:58 AM, Alan Stern wrote:

Like I said:

And again, generating the MAINTAINERS file/info into releases is fine as well.

Rene.

Rene Herman wrote:

Yes, problem reporters and people who infrequently (or for their first time) submit patches, and even people who frequently submit patches but most of the time only to the same one or two subsystems need an obvious, tool-independent way to get contact information.

Good. This generated data will be used by almost everyone except for a certain special group of submitters.