threads / rfc / 16231

[RFC] Configuring (future) committags support in gitweb

Subject: [RFC] Configuring (future) committags support in gitweb

## tl;dr

20 messages between Nov 8, 2008 and Feb 24, 2009.

replies: 19people: 5as markdown or json

Jakub Narebski· Nov 8, 2008, 19:07 UTC · lore

Francis Galiegue <fg@one2team.net> writes in "Need help for migration from CVS to git in one go..."

Show 5 quoted lines
> * third: also Bonsai-related; Bonsai can link to Bugzilla by
> matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
> http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
> this built-in? (haven't looked yet) Is this planned, or has it been
> discussed and been considered not worth the hassle?

Here below there is proposal how the committags support could look like for gitweb _user_, which means how to configure gitweb to use (or do not use) committags, how to configure committags, and how to define new committags.

Committags are "tags" in commit messages, expanded when rendering commit message, like gitweb now does for (shortened) SHA-1, converting them to 'object' view link. It should be done in a way to make it easy configurable, preferably having to configure only variable part, and not having to write whole replacement rule.

Possible committags include: _BUG(n)_, bug _#n_, _FEATURE(n), Message-Id, plain text URL e.g. _http://repo.or.cz_, spam protecting of email addresses, "rich text formatting" like *bold* and _underline_, syntax highlighting of signoff lines.

I think it would be good idea to use repository config file for setting-up repository-specific committags, and use whatever Perl structure for global configuration. The config language can be borrowed from "drivers" in gitattributes (`diff' and `merge' drivers).

So the example configuration could look like this:
  [gitweb]
  	committags = sha1 signoff bugzilla
  [committag "bugzilla"]
  	match = "\\b(?:#?)(\\d+)\\b"
  	link  = "http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1"
where 'sha1' and 'signoff' are built-in committags, committags are
applied in the order they are put in gitweb.committags; possible actions
for committag driver include:
 * link: replace $match by '_<a href="$link">_$match_</a>_'
 * html: replace $match by '_$html_'
 * text: replace $match by '$text'
where '_a_' means that 'a' is treated as HTML, and is not expanded
further, and 'b' means that it can be further expanded by later
committags, and finally is HTML-escaped (esc_html).
What do you think about this?
-- 
Jakub Narebski
Poland
Francis Galiegue· Nov 8, 2008, 20:02 UTC · re: Jakub Narebski · lore

Re: [RFC] Configuring (future) committags support in gitweb

Le Saturday 08 November 2008 20:07:53 Jakub Narebski, vous avez écrit :
Show 14 quoted lines
> Francis Galiegue <fg@one2team.net> writes
> in "Need help for migration from CVS to git in one go..."
>
> > * third: also Bonsai-related; Bonsai can link to Bugzilla by
> > matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
> > http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
> > this built-in? (haven't looked yet) Is this planned, or has it been
> > discussed and been considered not worth the hassle?
>
> Here below there is proposal how the committags support could look like
> for gitweb _user_, which means how to configure gitweb to use (or do not
> use) committags, how to configure committags, and how to define new
> committags.
>

Your proposal goes much further than my initial question, but I thought I'd jump in anyway :p

Show 11 quoted lines
> Committags are "tags" in commit messages, expanded when rendering commit
> message, like gitweb now does for (shortened) SHA-1, converting them to
> 'object' view link.  It should be done in a way to make it easy
> configurable, preferably having to configure only variable part, and not
> having to write whole replacement rule.
>
> Possible committags include: _BUG(n)_, bug _#n_, _FEATURE(n),
> Message-Id, plain text URL e.g. _http://repo.or.cz_, spam protecting
> of email addresses, "rich text formatting" like *bold* and _underline_,
> syntax highlighting of signoff lines.
>
What do you mean with "not having to write whole replacement rule"?
Show 16 quoted lines
> I think it would be good idea to use repository config file for
> setting-up repository-specific committags, and use whatever Perl
> structure for global configuration. The config language can be
> borrowed from "drivers" in gitattributes (`diff' and `merge' drivers).
>
> So the example configuration could look like this:
>
>   [gitweb]
>   	committags = sha1 signoff bugzilla
>
>   [committag "bugzilla"]
>   	match = "\\b(?:#?)(\\d+)\\b"
>   	link  = "http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1"
>
> where 'sha1' and 'signoff' are built-in committags, committags are
> applied in the order they are put in gitweb.committags;

I don't understand what the "signoff" builtin is : is that a link to see only commits "Signed-off-by:" a particular person?

If so, might I suggest that an "alt" tells "Only show commits signed off by this person"?

And also, what about the sha1 builtin? AFAIK, a SHA1 can point to a commit, a tree, and others... In fact, it points to any of these right now, but how would you tell apart these different SHA1s in a commit message? The only obvious use I see for it is the builtin "Revert ..." commit message, that the commiter _can_ override...

Or would that be:
my $sha1_re = qr/[a-z[0-9]{40}/;

/(?:(?i:commit\s+))?\b($sha1_re)\b/ => [link to commit $1] /(?:(?i:tree\s+))\b($sha1_re)\b)/ => [link to tree $1] /(?:(?:tag\s+))\b($sha1_re)\b)/ => [link to tag $1]

Finally, is there any reason to think that a sha1 or signoff committag will ever need to be overriden in some way?

Show 9 quoted lines
> possible actions 
> for committag driver include:
>  * link: replace $match by '_<a href="$link">_$match_</a>_'
>  * html: replace $match by '_$html_'
>  * text: replace $match by '$text'
> where '_a_' means that 'a' is treated as HTML, and is not expanded
> further, and 'b' means that it can be further expanded by later
> committags, and finally is HTML-escaped (esc_html).
>
What use do you see for the html match? Just asking...
And I don't see what you '_a_' and '_b_' are about...
-- 
fge
Jakub Narebski· Nov 8, 2008, 22:35 UTC · re: Francis Galiegue · lore

Re: [RFC] Configuring (future) committags support in gitweb

Dnia sobota 8. listopada 2008 21:02, Francis Galiegue napisał:
Show 9 quoted lines
> Le Saturday 08 November 2008 20:07:53 Jakub Narebski, vous avez écrit :
>> Francis Galiegue <fg@one2team.net> writes
>> in "Need help for migration from CVS to git in one go..."
>>
>>> * third: also Bonsai-related; Bonsai can link to Bugzilla by
>>> matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
>>> http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
>>> this built-in? (haven't looked yet) Is this planned, or has it been
>>> discussed and been considered not worth the hassle?
[...]
Show 13 quoted lines
>> Committags are "tags" in commit messages, expanded when rendering commit
>> message, like gitweb now does for (shortened) SHA-1, converting them to
>> 'object' view link.  It should be done in a way to make it easy
>> configurable, preferably having to configure only variable part, and not
>> having to write whole replacement rule.
>>
>> Possible committags include: _BUG(n)_, bug _#n_, _FEATURE(n),
>> Message-Id, plain text URL e.g. _http://repo.or.cz_, spam protecting
>> of email addresses, "rich text formatting" like *bold* and _underline_,
>> syntax highlighting of signoff lines.
>>
> 
> What do you mean with "not having to write whole replacement rule"?

Like in example with 'link' rule, not having to write whole <a href="http://example.com/bugzilla.php?id=$1">$&</a> (or something like that).

Show 19 quoted lines
>> I think it would be good idea to use repository config file for
>> setting-up repository-specific committags, and use whatever Perl
>> structure for global configuration. The config language can be
>> borrowed from "drivers" in gitattributes (`diff' and `merge' drivers).
>>
>> So the example configuration could look like this:
>>
>>   [gitweb]
>>   	committags = sha1 signoff bugzilla
>>
>>   [committag "bugzilla"]
>>   	match = "\\b(?:#?)(\\d+)\\b"
>>   	link  = "http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1"
>>
>> where 'sha1' and 'signoff' are built-in committags, committags are
>> applied in the order they are put in gitweb.committags;
> 
> I don't understand what the "signoff" builtin is : is that a link to see only 
> commits "Signed-off-by:" a particular person?

Committags doesn't need to be replaced by links. In this case I meant here using 'signoff' class for Signed-off-by: (and the like) lines, by wrapping it in '<span class="signoff">' ... '</a>'.

Show 5 quoted lines
> And also, what about the sha1 builtin? AFAIK, a SHA1 can point to a commit, a 
> tree, and others... In fact, it points to any of these right now, but how 
> would you tell apart these different SHA1s in a commit message? The only 
> obvious use I see for it is the builtin "Revert ..." commit message, that the 
> commiter _can_ override...

SHA1 (or shortened SHA1 from 8 charasters to 40 characters, or to be even more exact something that looks like SHA1) is replaced by link to 'object' view, which in turn finds type of object and _redirect_ to proper view, be it 'commit' (most frequent), 'tag', 'blob' or 'tree'.

We could have used instead gitweb link with 'h' (hash) parameter, but
without 'a' (action) parameter, which currently finds type of object
and _uses_ correct view...
 
> Finally, is there any reason to think that a sha1 or signoff committag will 
> ever need to be overriden in some way?
One might not want to link SHA1, for example if there are lots of false
positives because of commit message conventions or something, or refine
'signoff' committag to use different styles for different types of
signoff: Signed-off-by, Acked-by, Tested-by, other.  Having explicit
'signoff' committag allows us also to put some committags _after_ it,
for example SPAM-protection of emails, or add some committag before
'sha1' to filter out some SHA1 match false positives.
 
Show 11 quoted lines
>> possible actions 
>> for committag driver include:
>>  * link: replace $match by '_<a href="$link">_$match_</a>_'
>>  * html: replace $match by '_$html_'
>>  * text: replace $match by '$text'
>> where '_a_' means that 'a' is treated as HTML, and is not expanded
>> further, and 'b' means that it can be further expanded by later
>> committags, and finally is HTML-escaped (esc_html).
>>
> 
> What use do you see for the html match? Just asking...

For example 'signoff' committag... well, it is not exactly pure "html" but rather something like template.

  [committag "signoff"]
  	match = "(?i)^ *(signed[ \\-]off[ \\-]by[ :]|acked[ \\-]by[ :]|cc[ :])"
  	templ = "{<span class=\"signoff\">}$1{</span>}"
Or simpler
  [committag "signoff"]
  	match = "(?i)^ *(signed[ \\-]off[ \\-]by[ :]|acked[ \\-]by[ :]|cc[ :])"
  	class = signoff
> And I don't see what you '_a_' and '_b_' are about...

For example in link match, the text of the link can be further refined by committags later in sequence.

-- 
Jakub Narebski
Poland
Francis Galiegue· Nov 8, 2008, 23:27 UTC · re: Jakub Narebski · lore

Re: [RFC] Configuring (future) committags support in gitweb

Le Saturday 08 November 2008 23:35:48 Jakub Narebski, vous avez écrit :
Show 7 quoted lines
> >
> > What do you mean with "not having to write whole replacement rule"?
>
> Like in example with 'link' rule, not having to write whole
> <a href="http://example.com/bugzilla.php?id=$1">$&</a>
> (or something like that).
>
OK, good one.
Show 24 quoted lines
> >> I think it would be good idea to use repository config file for
> >> setting-up repository-specific committags, and use whatever Perl
> >> structure for global configuration. The config language can be
> >> borrowed from "drivers" in gitattributes (`diff' and `merge' drivers).
> >>
> >> So the example configuration could look like this:
> >>
> >>   [gitweb]
> >>   	committags = sha1 signoff bugzilla
> >>
> >>   [committag "bugzilla"]
> >>   	match = "\\b(?:#?)(\\d+)\\b"
> >>   	link  = "http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1"
> >>
> >> where 'sha1' and 'signoff' are built-in committags, committags are
> >> applied in the order they are put in gitweb.committags;
> >
> > I don't understand what the "signoff" builtin is : is that a link to see
> > only commits "Signed-off-by:" a particular person?
>
> Committags doesn't need to be replaced by links. In this case I meant
> here using 'signoff' class for Signed-off-by: (and the like) lines, by
> wrapping it in '<span class="signoff">' ... '</a>'.
>
Well, this would also mean to update gitweb.css, wouldn't it?
Show 15 quoted lines
> > And also, what about the sha1 builtin? AFAIK, a SHA1 can point to a
> > commit, a tree, and others... In fact, it points to any of these right
> > now, but how would you tell apart these different SHA1s in a commit
> > message? The only obvious use I see for it is the builtin "Revert ..."
> > commit message, that the commiter _can_ override...
>
> SHA1 (or shortened SHA1 from 8 charasters to 40 characters, or to be
> even more exact something that looks like SHA1) is replaced by link
> to 'object' view, which in turn finds type of object and _redirect_
> to proper view, be it 'commit' (most frequent), 'tag', 'blob' or 'tree'.
>
> We could have used instead gitweb link with 'h' (hash) parameter, but
> without 'a' (action) parameter, which currently finds type of object
> and _uses_ correct view...
>
OK, you lost me somewhat.

What I understand is that right now, the SHA1 links are "pre-processed" by gitweb so that the 'a' parameter is correct, right?

Out of curiosity, I just went to the kernel git repository (I don't know the git version that git.kernel.org uses offhand) and altered the 'a' parameter to something which is not even an 'a' command at all: 500...

However, if I try a VALID 'a' command with an "irrelevant" 'h' parameter, it acts quite funny: it just looks like it wants to try the closest match, but takes some time figuring it out... Sometimes to something relevant, sometimes to nothing really relevant. See for instance [1], in which 'a' was originally "commit".

Ow.
 [1]
http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.27.y.git;a=tag;h=788a5f3f70e2a9c46020bdd3a195f2a866441c5d
Show 11 quoted lines
> > Finally, is there any reason to think that a sha1 or signoff committag
> > will ever need to be overriden in some way?
>
> One might not want to link SHA1, for example if there are lots of false
> positives because of commit message conventions or something, or refine
> 'signoff' committag to use different styles for different types of
> signoff: Signed-off-by, Acked-by, Tested-by, other.  Having explicit
> 'signoff' committag allows us also to put some committags _after_ it,
> for example SPAM-protection of emails, or add some committag before
> 'sha1' to filter out some SHA1 match false positives.
>

Hmmm, so this means you'd want to make styles customizable somewhat (signoff). In fact, what you really want is span for CSS! Then why not, just, making a document to say "This is what you can do with CSS for gitweb", and say "these are the available CSS tags", and then be done with it?

I mean, when comes the day that someone will WANT other spans to be defined, badly, it's not like it will be unheard of, won't it?

Show 5 quoted lines
>
> > And I don't see what you '_a_' and '_b_' are about...
>
> For example in link match, the text of the link can be further refined
> by committags later in sequence.
I still don't get it. Can you give an example?

[personal thoughts: it would be really, really nice if, somewhat, gitweb.perl were splitted somewhat into different modules, and ideally use more of "what's out there on CPAN". I'm convinced that some CPAN modules would be of GREAT help to gitweb, as well as I'm convinced that not many people out there use Windows to run gitweb anyway :p]

-- 
fge
Jakub Narebski· Nov 9, 2008, 00:25 UTC · re: Francis Galiegue · lore

Re: [RFC] Configuring (future) committags support in gitweb

Dnia niedziela 9. listopada 2008 00:27, Francis Galiegue napisał:
> Le Saturday 08 November 2008 23:35:48 Jakub Narebski, vous avez écrit :
Show 8 quoted lines
>>> I don't understand what the "signoff" builtin is : is that a link to see
>>> only commits "Signed-off-by:" a particular person?
>>
>> Committags doesn't need to be replaced by links. In this case I meant
>> here using 'signoff' class for Signed-off-by: (and the like) lines, by
>> wrapping it in '<span class="signoff">' ... '</a>'.
> 
> Well, this would also mean to update gitweb.css, wouldn't it?
Not necessary. Please remember that you can configure gitweb to either
use _alternate_ stylesheet (instead of provided gitweb.css), or use
_additional_ stylesheet (for example gitweb-commit.css in addition to
gitweb.css)
 
Show 16 quoted lines
>>> And also, what about the sha1 builtin? AFAIK, a SHA1 can point to a
>>> commit, a tree, and others... In fact, it points to any of these right
>>> now, but how would you tell apart these different SHA1s in a commit
>>> message? The only obvious use I see for it is the builtin "Revert ..."
>>> commit message, that the commiter _can_ override...
>>
>> SHA1 (or shortened SHA1 from 8 charasters to 40 characters, or to be
>> even more exact something that looks like SHA1) is replaced by link
>> to 'object' view, which in turn finds type of object and _redirect_
>> to proper view, be it 'commit' (most frequent), 'tag', 'blob' or 'tree'.
>>
>> We could have used instead gitweb link with 'h' (hash) parameter, but
>> without 'a' (action) parameter, which currently finds type of object
>> and _uses_ correct view...
> 
> OK, you lost me somewhat.
I'm sorry about that. Perhaps I should use only one mechanism.
 
> What I understand is that right now, the SHA1 links are "pre-processed" by 
> gitweb so that the 'a' parameter is correct, right?

No, they are 'post-processed': finding correct action is left to _after_ you click on the link (it is more natural, and helps performance).

If you don't know action for given SHA-1 you can use either
  http://example.com/gitweb.cgi?a=object;h=deadbeef

which finds correct type (for example 'commit') and redirects using HTTP 302 Found redirection to

  http://example.com/gitweb.cgi?a=commit;h=deadbeef
This way is used bu SHA-1 committag in commit messages.

Alternate solution (meant for bugtrackers), done by another author, is to simply skip action ('a') parameter:

  http://example.com/gitweb.cgi?h=deadbeef

and then gitweb finds type of object and act accordingly (without redirect to correct view).

[...]
Show 18 quoted lines
>>> Finally, is there any reason to think that a sha1 or signoff committag
>>> will ever need to be overriden in some way?
>>
>> One might not want to link SHA1, for example if there are lots of false
>> positives because of commit message conventions or something, or refine
>> 'signoff' committag to use different styles for different types of
>> signoff: Signed-off-by, Acked-by, Tested-by, other.  Having explicit
>> 'signoff' committag allows us also to put some committags _after_ it,
>> for example SPAM-protection of emails, or add some committag before
>> 'sha1' to filter out some SHA1 match false positives.
> 
> Hmmm, so this means you'd want to make styles customizable somewhat (signoff). 
> In fact, what you really want is span for CSS! Then why not, just, making a 
> document to say "This is what you can do with CSS for gitweb", and say "these 
> are the available CSS tags", and then be done with it?
> 
> I mean, when comes the day that someone will WANT other spans to be defined, 
> badly, it's not like it will be unheard of, won't it? 
Errr... I don't understand.

The examples perhaps are not the best. One would be for example to use different class (different style) for Signed-off-by signoff (which denotes signing Certificate of Origin), and the rest of (informative) signoff:

  [gitweb]
  	committags = sha1 signoff_signed signoff

Another example would be to add SPAM-protection of emails, for example replacing 'user@example.com' by 'user AT example DOT com', or something more advanced. This have to be used _after_ signoff, because otherwise regexp could have difficulties matching mangled email

  [gitweb]
  	committags = sha1 signoff mailto
Show 7 quoted lines
>>
>>> And I don't see what you '_a_' and '_b_' are about...
>>
>> For example in link match, the text of the link can be further refined
>> by committags later in sequence.
> 
> I still don't get it. Can you give an example?
For example signoff line
  Signed-off-by: A U Thor <author@example.com>
would be replaced by
  {<span class="signoff">}Signed-off-by: A U Thor <author@example.com>{</a>}
 
where parts inside {...} is HTML code, and should be not expanded
further, and parts outside it could be expanded further by following
lower priority committags (like anti-SPAM for emails), and have to be
finally HTML escaped (like '<' and '>' in email in signoff).
.....................................................................
Show 5 quoted lines
> [personal thoughts: it would be really, really nice if, somewhat, gitweb.perl 
> were splitted somewhat into different modules, and ideally use more 
> of "what's out there on CPAN". I'm convinced that some CPAN modules would be 
> of GREAT help to gitweb, as well as I'm convinced that not many people out 
> there use Windows to run gitweb anyway :p]

First, having gitweb in (almost) one piece makes for easier installation. But there are plans to have gitweb use Git.pm or future Got::Repo and friends. I'm not sure about further splitting...

Second, we cannot in good faith use CPAN modules which cannot be found in standard Perl distributions, or at least in some trusted extras package (application) repositories, as gitweb is sometimes run on machines with tight security (think kernel.org for example) where you cannot simply ask admin to install some third-party alpha-version CPAN module.

-- 
Jakub Narebski
Poland
Marcel M. Cary· Feb 17, 2009, 15:32 UTC · re: Jakub Narebski · lore

Re: [RFC] Configuring (future) committags support in gitweb, especially bug linking

I'm interested in cross-linking bug references in commit messages to a bug tracking system. I started tinkering a couple weeks ago and am finally understanding that committags encompass this functionality. (From the subject line I first understood "tags" to mean git tags rather than commit message munging.)

Is the committags idea still under active development?
I'd be happy to share what I have, which is for bug linking only, but
very little code would apply to the more general concept of committags.
 Here are some ideas that might apply...

Two regexes would make it easier to configure a driver without needing look-ahead and look-behind assertions. For example, if you want to match non-negative integers but only in the context of a Resolves-bug header:

    Resolves-bug: 1234, 1235
With two regexes you can write:
    /^Resolves-bug: \d+(, \d+)*/
    /\d+/
But with only one, you'd have to write:
    /(?<=^Resolves-bug: (\d+, )*)(\d+)/

The need for a lookbehind assertion means I need to stop at perlreref to lookup syntax. Hrm... and with testing I see that it's worse than that:

    $ perl -wpe 's/(?<=^Resolves-bug: (\d+, )*)(\d+)/[$2]/g'
    Variable length lookbehind not implemented in regex; marked by <--
    HERE in m/(?<=^Resolves-bug: (\d+, )*)(\d+) <-- HERE / at -e line 1.
I guess it can't be done even with the look-behind assertion.

I got the two-regex idea from a spec I ran across while evaluating Subversion:

http://guest:@tortoisesvn.tigris.org/svn/tortoisesvn/trunk/doc/issuetrackers.txt

If there is interest in BTS integration beyond gitweb, for example in git-gui, gitk, or the Windows UIs, then perhaps this spec would be worth considering. It covers more than just hyperlinking. It also considers issues like how to draw the form field for a bug ID as part of a commit message form, how to validate that form field, and then how to munge the log message to include the entered ID. Some details, like using a newline to separate the two regexes, might be more awkward for Git than Subversion.

I like the idea of allowing a regex writer -- a gitweb admin or a repository owner -- to ignore issues regarding HTML escaping. For example, I'd rather not have &nbsp; in the regex. And I don't want the replacement to have to escape "&" in a query string. That's a strength of not having to write the whole link replacement rule. And I think hyperlinking will be one of the most common uses of this committag feature, so it's worth special support.

In the case of false positives, it might also be helpful to have a title attribute that explains the committag's interpretation of the text.

I also like the idea of giving the admin full control to specify a Perl function of some sort, which might go as far as looking up bug summaries for the "title" attribute or adding JS to fetch it via AJAX on mouse-over. But I doubt I would bother with that myself.

Appealing as it is, the use of '$1' in my replacements didn't work for me:
    $ perl -wpe '$reg = "(\\d+)"; $rep = ".\$1."; s/$reg/$rep/g'
    123
    .$1.

I think usage of capturing parenthesis is important, even with two regexes, because it makes it easier to specify link text that's broader than the data that goes in the URL. Specifically, I wanted to be able to produce HTML like this, with the hash mark hyperlinked but not used in the URL:

    <a href="...bug=123">#123</a>

I guess that's just my aesthetic. To support that, my code calls sprintf with $&, $1, $2, ... $9, and that particualr replacement URL uses %2$s.

I'm concerned about the composition of these committag drivers. In other words, will it be hard for the configurer to manage interactions between committag drivers? To choose a sane order, will I have to understand the implementation details of each committag driver?

Perhaps a simpler alternative would be to let at most one driver process a given snippet of text, forbidding nesting of replacements. (If I understand Junio's suggestion to use a list of strings and refs, non-nesting overlaps are already not supported.) If all replacements were hyperlinks -- and I expect that to be the common case -- they wouldn't be nestable anyway. I wouldn't see it as a huge loss for the nesting examples I can think of: Separate rules for span around S-o-b and linking or obfuscation of email could be combined into one... A rule to shade text quoted email-style with leading angle brackets could just clobber any further processing of that text. And it might simplify the code and testing of it quite a bit.

If committags do turn out to support nesting, perhaps it would make sense to stratify the ordering so that it's clear whether a particular driver takes as input HTML vs. text and outputs HTML vs. text. (For example, weak email obfuscation might be text -> text.) I guess to strictly honor the input and output types of a driver, the text -> html drivers still have to be expanded in a single pass.

A few ideas for drivers that I don't think have been mentioned yet:
* Wiki page names, like to [[Feature Documentation]].  These are notable
because they tend to contain punctuation that get HTML-escaped, like
quotes and ampersands.
* Links to gitweb itself, such as 123abc:file.txt and HEAD:file.txt.  I
guess the current hash linking sort of does the first case except that
you have to get the hash of the blob instead of using the commit hash,
and the current hash linking wouldn't reveal the filename until after
you click, nor when viewing textual log messages.  I'm not sure whether
special support for linking to multi-commit diffs or other object types
would be as helpful.
Marcel
Jakub Narebski wrote:
Show 114 quoted lines
> Dnia sobota 8. listopada 2008 21:02, Francis Galiegue napisał:
>> Le Saturday 08 November 2008 20:07:53 Jakub Narebski, vous avez écrit :
>>> Francis Galiegue <fg@one2team.net> writes
>>> in "Need help for migration from CVS to git in one go..."
>>>
>>>> * third: also Bonsai-related; Bonsai can link to Bugzilla by
>>>> matching (wild guess) /\b(?:#?)(\d+)\b/ and transforming this into
>>>> http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1. Does gitweb have
>>>> this built-in? (haven't looked yet) Is this planned, or has it been
>>>> discussed and been considered not worth the hassle?
> [...]
>
>>> Committags are "tags" in commit messages, expanded when rendering commit
>>> message, like gitweb now does for (shortened) SHA-1, converting them to
>>> 'object' view link.  It should be done in a way to make it easy
>>> configurable, preferably having to configure only variable part, and not
>>> having to write whole replacement rule.
>>>
>>> Possible committags include: _BUG(n)_, bug _#n_, _FEATURE(n),
>>> Message-Id, plain text URL e.g. _http://repo.or.cz_, spam protecting
>>> of email addresses, "rich text formatting" like *bold* and _underline_,
>>> syntax highlighting of signoff lines.
>>>
>> What do you mean with "not having to write whole replacement rule"?
>
> Like in example with 'link' rule, not having to write whole
> <a href="http://example.com/bugzilla.php?id=$1">$&</a>
> (or something like that).
>
>>> I think it would be good idea to use repository config file for
>>> setting-up repository-specific committags, and use whatever Perl
>>> structure for global configuration. The config language can be
>>> borrowed from "drivers" in gitattributes (`diff' and `merge' drivers).
>>>
>>> So the example configuration could look like this:
>>>
>>>   [gitweb]
>>>      committags = sha1 signoff bugzilla
>>>
>>>   [committag "bugzilla"]
>>>      match = "\\b(?:#?)(\\d+)\\b"
>>>      link  = "http://your.bugzilla.fqdn.here/show_bug.cgi?id=$1"
>>>
>>> where 'sha1' and 'signoff' are built-in committags, committags are
>>> applied in the order they are put in gitweb.committags;
>> I don't understand what the "signoff" builtin is : is that a link to see only
>> commits "Signed-off-by:" a particular person?
>
> Committags doesn't need to be replaced by links. In this case I meant
> here using 'signoff' class for Signed-off-by: (and the like) lines, by
> wrapping it in '<span class="signoff">' ... '</a>'.
>
>> And also, what about the sha1 builtin? AFAIK, a SHA1 can point to a commit, a
>> tree, and others... In fact, it points to any of these right now, but how
>> would you tell apart these different SHA1s in a commit message? The only
>> obvious use I see for it is the builtin "Revert ..." commit message, that the
>> commiter _can_ override...
>
> SHA1 (or shortened SHA1 from 8 charasters to 40 characters, or to be
> even more exact something that looks like SHA1) is replaced by link
> to 'object' view, which in turn finds type of object and _redirect_
> to proper view, be it 'commit' (most frequent), 'tag', 'blob' or 'tree'.
>
> We could have used instead gitweb link with 'h' (hash) parameter, but
> without 'a' (action) parameter, which currently finds type of object
> and _uses_ correct view...
>
>> Finally, is there any reason to think that a sha1 or signoff committag will
>> ever need to be overriden in some way?
>
> One might not want to link SHA1, for example if there are lots of false
> positives because of commit message conventions or something, or refine
> 'signoff' committag to use different styles for different types of
> signoff: Signed-off-by, Acked-by, Tested-by, other.  Having explicit
> 'signoff' committag allows us also to put some committags _after_ it,
> for example SPAM-protection of emails, or add some committag before
> 'sha1' to filter out some SHA1 match false positives.
>
>>> possible actions
>>> for committag driver include:
>>>  * link: replace $match by '_<a href="$link">_$match_</a>_'
>>>  * html: replace $match by '_$html_'
>>>  * text: replace $match by '$text'
>>> where '_a_' means that 'a' is treated as HTML, and is not expanded
>>> further, and 'b' means that it can be further expanded by later
>>> committags, and finally is HTML-escaped (esc_html).
>>>
>> What use do you see for the html match? Just asking...
>
> For example 'signoff' committag... well, it is not exactly pure "html"
> but rather something like template.
>
>   [committag "signoff"]
>         match = "(?i)^ *(signed[ \\-]off[ \\-]by[ :]|acked[ \\-]by[ :]|cc[ :])"
>         templ = "{<span class=\"signoff\">}$1{</span>}"
>
> Or simpler
>
>   [committag "signoff"]
>         match = "(?i)^ *(signed[ \\-]off[ \\-]by[ :]|acked[ \\-]by[ :]|cc[ :])"
>         class = signoff
>
>> And I don't see what you '_a_' and '_b_' are about...
>
> For example in link match, the text of the link can be further refined
> by committags later in sequence.
>
> --
> Jakub Narebski
> Poland
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Marcel M. Cary· Feb 18, 2009, 03:00 UTC · re: Marcel M. Cary · lore

[PATCH RFC 1/2] gitweb: Fix warnings with override permitted but no repo override

When a feature like "blame" is permitted to be overridden in the repository configuration but it is not actually set in the repository, a warning is emitted due to the undefined value of the repository configuration, even though it's a perfectly normal condition.

The warning is grounds for test failure in the gitweb test script, so it causes some new feature tests of mine to fail.

This patch prevents warning and adds a test case to exercise it.
Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
---

Here's a small patch I put together while tinkering with bug hyperlinking. Does this look reasonable?

Marcel
 gitweb/gitweb.perl                     |    8 +++++---
 t/t9500-gitweb-standalone-no-errors.sh |    5 +++++
 2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7c48181..653f0be 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -402,13 +402,13 @@ sub feature_bool {
 	my $key = shift;
 	my ($val) = git_get_project_config($key, '--bool');
 
-	if ($val eq 'true') {
+	if (!defined $val) {
+		return ($_[0]);
+	} elsif ($val eq 'true') {
 		return (1);
 	} elsif ($val eq 'false') {
 		return (0);
 	}
-
-	return ($_[0]);
 }
 
 sub feature_snapshot {
@@ -1978,6 +1978,8 @@ sub git_get_project_config {
 		$config_file = "$git_dir/config";
 	}
 
+	return undef if (!defined $config{"gitweb.$key"});
+
 	# ensure given type
 	if (!defined $type) {
 		return $config{"gitweb.$key"};
diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 7c6f70b..559045e 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -662,6 +662,11 @@ cat >>gitweb_config.perl <<EOF
 EOF
 
 test_expect_success \
+	'config override: tree view, features not overridden in repo config' \
+	'gitweb_run "p=.git;a=tree"'
+test_debug 'cat gitweb.log'
+
+test_expect_success \
 	'config override: tree view, features disabled in repo config' \
 	'git config gitweb.blame no &&
 	 git config gitweb.snapshot none &&
-- 
1.6.1
Marcel M. Cary· Feb 18, 2009, 03:00 UTC · re: Marcel M. Cary · lore

[PATCH RFC 2/2] gitweb: Hyperlink multiple git hashes on the same commit message line

The current implementation only hyperlinks the first hash on a given line of the commit message. It seems sensible to highlight all of them if there are multiple, and it seems plausible that there would be multiple even with a tidy line length limit, because they can be abbreviated as short as 8 characters.

Benchmark:

I wanted to make sure that using the 'e' switch to the Perl regex wasn't going to kill performance, since this is called once per commit message line displayed.

In all three A/B scenarios I tried, the A and B yielded the same results within 2%, where A is the version of code before this patch and B is the version after.

1: View a commit message containing the last 1000 commit hashes
2: View a commit message containing 1000 lines of 40 dots to avoid
   hyperlinking at the same message length
3: View a short merge commit message with a few lines of text and
   no hashes

All were run in CGI mode on my sub-production hardware on a recent clone of git.git. Numbers are the average of 10 reqests per second with the first request discarded, since I expect this change to affect primarily CPU usage. Measured with ApacheBench.

Note that the web page rendered was the same; while the new code supports multiple hashes per line, there was at most one per line.

The primary purpose of scenarios 2 and 3 were to verify that the addition of 1000 commit messages had an impact on how much of the time was spent rendering commit messages. They were all within 2% of 0.80 requests per second (much faster).

So I think the patch has no noticeable effect on performance.
Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
---
And here's another.
Marcel
 gitweb/gitweb.perl |   12 +++++-------
 1 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 653f0be..51b7f56 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1384,13 +1384,11 @@ sub format_log_line_html {
 	my $line = shift;
 
 	$line = esc_html($line, -nbsp=>1);
-	if ($line =~ m/\b([0-9a-fA-F]{8,40})\b/) {
-		my $hash_text = $1;
-		my $link =
-			$cgi->a({-href => href(action=>"object", hash=>$hash_text),
-			        -class => "text"}, $hash_text);
-		$line =~ s/$hash_text/$link/;
-	}
+	$line =~ s{\b([0-9a-fA-F]{8,40})\b}{
+		return $cgi->a({-href => href(action=>"object", hash=>$1),
+					   -class => "text"}, $1);
+	}eg;
+
 	return $line;
 }
 
-- 
1.6.1
Jakub Narebski· Feb 18, 2009, 21:55 UTC · re: Marcel M. Cary · lore

Re: [PATCH RFC 2/2] gitweb: Hyperlink multiple git hashes on the same commit message line

On Wed, 18 Feb 2009, Marcel M. Cary wrote:
Show 6 quoted lines
> The current implementation only hyperlinks the first hash on
> a given line of the commit message.  It seems sensible to
> highlight all of them if there are multiple, and it seems
> plausible that there would be multiple even with a tidy line
> length limit, because they can be abbreviated as short as 8
> characters.

That is a good catch. Code simply was not modified since we required fill-length 40-characters SHA-1 id.

Show 16 quoted lines
> 
> Benchmark:
> 
> I wanted to make sure that using the 'e' switch to the Perl regex
> wasn't going to kill performance, since this is called once per commit
> message line displayed.
> 
> In all three A/B scenarios I tried, the A and B yielded the same
> results within 2%, where A is the version of code before this patch
> and B is the version after.
> 
> 1: View a commit message containing the last 1000 commit hashes
> 2: View a commit message containing 1000 lines of 40 dots to avoid
>    hyperlinking at the same message length
> 3: View a short merge commit message with a few lines of text and
>    no hashes

I don't think we should worry about that; after all esc_path and unescape subroutines also use 'e' switch to Perl regexp.

So the benchmark is nice addition, but I don't think it is really necessary, especially that the change results in shorter and easier (I think) to maintain code.

[...]
Show 8 quoted lines
> So I think the patch has no noticeable effect on performance.
> 
> Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
> ---
> 
> And here's another.
> 
> Marcel

Do I understand correctly that those patches are not related at all semantically or textually, only in that you have them one after other (and blob sha-1 in the index line reflects state after former), isn't it?

Show 25 quoted lines
> 
> 
>  gitweb/gitweb.perl |   12 +++++-------
>  1 files changed, 5 insertions(+), 7 deletions(-)
> 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 653f0be..51b7f56 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1384,13 +1384,11 @@ sub format_log_line_html {
>  	my $line = shift;
>  
>  	$line = esc_html($line, -nbsp=>1);
> -	if ($line =~ m/\b([0-9a-fA-F]{8,40})\b/) {
> -		my $hash_text = $1;
> -		my $link =
> -			$cgi->a({-href => href(action=>"object", hash=>$hash_text),
> -			        -class => "text"}, $hash_text);
> -		$line =~ s/$hash_text/$link/;
> -	}
> +	$line =~ s{\b([0-9a-fA-F]{8,40})\b}{
> +		return $cgi->a({-href => href(action=>"object", hash=>$1),
> +					   -class => "text"}, $1);
> +	}eg;
> +

Almost correct... but for this unnecessary 'return' statement. Without it: ACK.

Show 7 quoted lines
>  	return $line;
>  }
>  
> -- 
> 1.6.1
> 
> 

P.S. Why bare emails (without user names), e.g. "pasky@suse.cz" and not "Petr Baudis <pasky@suse.cz>"? Just curious...

-- 
Jakub Narebski
Poland
Junio C Hamano· Feb 20, 2009, 08:35 UTC · re: Jakub Narebski · lore

Re: [PATCH RFC 2/2] gitweb: Hyperlink multiple git hashes on the same commit message line

Jakub Narebski <jnareb@gmail.com> writes:
Show 8 quoted lines
>> +	$line =~ s{\b([0-9a-fA-F]{8,40})\b}{
>> +		return $cgi->a({-href => href(action=>"object", hash=>$1),
>> +					   -class => "text"}, $1);
>> +	}eg;
>> +
>
> Almost correct... but for this unnecessary 'return' statement.
> Without it: ACK.

I've applied this directly on 'master' without the return from inside s///e with your Ack. Please check the result.

Thanks.
Jakub Narebski· Feb 20, 2009, 11:46 UTC · re: Junio C Hamano · lore

Re: [PATCH RFC 2/2] gitweb: Hyperlink multiple git hashes on the same commit message line

Junio C Hamano wrote:
Show 13 quoted lines
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> >> +	$line =~ s{\b([0-9a-fA-F]{8,40})\b}{
> >> +		return $cgi->a({-href => href(action=>"object", hash=>$1),
> >> +					   -class => "text"}, $1);
> >> +	}eg;
> >> +
> >
> > Almost correct... but for this unnecessary 'return' statement.
> > Without it: ACK.
> 
> I've applied this directly on 'master' without the return from inside s///e
> with your Ack.  Please check the result.

I did quick test by installing newest gitweb (with above commit applied), doing in gitweb searching commit message for '[a-f0-9]{8,40}' with regexp search; everything looks all right. But I didn't do extensive tests.

-- 
Jakub Narebski
Poland
Marcel M. Cary· Feb 24, 2009, 15:38 UTC · re: Jakub Narebski · lore

Addresses with full names in patch emails

Thanks for the two patch tweaks.
Jakub Narebski wrote:
> P.S. Why bare emails (without user names), e.g. "pasky@suse.cz"
> and not "Petr Baudis <pasky@suse.cz>"? Just curious...

I've been using "git send-email" for patches, and have Thunderbird as my MUA otherwise. (I'd use (al)pine if I could make it work with Exchange/NTLM at work, but that's another story...) I've been transfering recipients (--to and --cc) from Thunderbird to the commandline with copy/paste.

In Thurderbird, copying an email address from a message only gets you the user@domain part, not the "Full Name" <user@domain>. To get the "Full Name" <user@domain> I would have to View Message Source and pickout the CC line, which is marginally harder.

And on the commandline, instead of just pasting an email as a shell word, I'd have to add single quotes (I think) to keep the whole "Full Name" <user@domain> as one word and quote the shell meta characters.

Neither piece is all that onerous, I guess. Sounds like you would see some value in the full names. Maybe I'll try including them on my next patch. Looks like --cc-cmd or sendemail.aliasesfile might make it easier, but I'd have to set them up.

Marcel
Jakub Narebski· Feb 24, 2009, 15:58 UTC · re: Marcel M. Cary · lore

Re: Addresses with full names in patch emails

On Tue, 24 Feb 2009, Marcel M. Cary wrote:
Show 11 quoted lines
> Thanks for the two patch tweaks.
> 
> Jakub Narebski wrote:
> > P.S. Why bare emails (without user names), e.g. "pasky@suse.cz"
> > and not "Petr Baudis <pasky@suse.cz>"? Just curious...
> 
> I've been using "git send-email" for patches, and have Thunderbird as my
> MUA otherwise.  (I'd use (al)pine if I could make it work with
> Exchange/NTLM at work, but that's another story...)  I've been
> transfering recipients (--to and --cc) from Thunderbird to the
> commandline with copy/paste.
[...]
Well, 'technical reasons with copy'n'paste' would be enough for me.
P.S. But '"pasky@suse.cz" <pasky@suse.cz>' looks very silly...
-- 
Jakub Narebski
Poland
Marcel M. Cary· Feb 24, 2009, 16:33 UTC · re: Jakub Narebski · lore

Re: [PATCH RFC 2/2] gitweb: Hyperlink multiple git hashes on the same commit message line

Jakub Narebski wrote:
> Do I understand correctly that those patches are not related at all
> semantically or textually, only in that you have them one after other
> (and blob sha-1 in the index line reflects state after former), isn't
> it?

Yes, I think they were independent. They are related only in that they are small tweaks to gitweb and products of my bug hyperlinking patch (not submitted). I submitted them as a series mainly to group them. Are you suggesting that I reserve threading patches together for when one builds upon the previous one?

Marcel
Giuseppe Bilotta· Feb 18, 2009, 07:41 UTC · re: Marcel M. Cary · lore

Re: [PATCH RFC 1/2] gitweb: Fix warnings with override permitted but no repo override

On Wed, Feb 18, 2009 at 4:00 AM, Marcel M. Cary <marcel@oak.homeunix.org> wrote:
Show 16 quoted lines
> When a feature like "blame" is permitted to be overridden in the
> repository configuration but it is not actually set in the
> repository, a warning is emitted due to the undefined value
> of the repository configuration, even though it's a perfectly
> normal condition.
>
> The warning is grounds for test failure in the gitweb test script,
> so it causes some new feature tests of mine to fail.
>
> This patch prevents warning and adds a test case to exercise it.
>
> Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
> ---
>
> Here's a small patch I put together while tinkering with bug hyperlinking.
> Does this look reasonable?
My only perplexity is about this:
Show 6 quoted lines
> @@ -1978,6 +1978,8 @@ sub git_get_project_config {
>                $config_file = "$git_dir/config";
>        }
>
> +       return undef if (!defined $config{"gitweb.$key"});
> +

I'm no Perl expert, so I have no idea: how do non-bool config checks (which expect arrays) cope with an undef? Also, you may want to add a non-bool override test in the test suite.

-- 
Giuseppe "Oblomov" Bilotta
Junio C Hamano· Feb 18, 2009, 08:40 UTC · re: Marcel M. Cary · lore

Re: [PATCH RFC 1/2] gitweb: Fix warnings with override permitted but no repo override

"Marcel M. Cary" <marcel@oak.homeunix.org> writes:
Show 16 quoted lines
> When a feature like "blame" is permitted to be overridden in the
> repository configuration but it is not actually set in the
> repository, a warning is emitted due to the undefined value
> of the repository configuration, even though it's a perfectly
> normal condition.
>
> The warning is grounds for test failure in the gitweb test script,
> so it causes some new feature tests of mine to fail.
>
> This patch prevents warning and adds a test case to exercise it.
>
> Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
> ---
>
> Here's a small patch I put together while tinkering with bug hyperlinking.
> Does this look reasonable?

To my cursory look, it doesn't, and it is not entirely your fault, but if we applied this patch, it would not improve things very much. It just would shift the same problem around.

Show 16 quoted lines
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 7c48181..653f0be 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -402,13 +402,13 @@ sub feature_bool {
>  	my $key = shift;
>  	my ($val) = git_get_project_config($key, '--bool');
>  
> -	if ($val eq 'true') {
> +	if (!defined $val) {
> +		return ($_[0]);
> +	} elsif ($val eq 'true') {
>  		return (1);
>  	} elsif ($val eq 'false') {
>  		return (0);
>  	}

I think the warning you are talking about is to compare $val with 'true' with 'eq' operator when $val could be undef. The check to see if $val is undefined ato avoid that 'eq' comparison is fine, and the intent to return false is also good, but I think feature_bool is meant to say "yes" or "no", and existing code for 'false' is returning (0). I'd rather see your new codepath normalize incoming undef the same way string 'false' is normalized to return (0). Granted, the caller should be prepared to take the answer as boolean and treat the usual Perl false values (numeric zero, a string with single "0", or an undef) without barfing, so returning (undef) from here ought to be safe (otherwise the callers are broken), but I'd rather see this function play safe.

But it certainly is not my main complaint.
Show 7 quoted lines
>  sub feature_snapshot {
> @@ -1978,6 +1978,8 @@ sub git_get_project_config {
>  		$config_file = "$git_dir/config";
>  	}
>  
> +	return undef if (!defined $config{"gitweb.$key"});
> +

I think this change is missing a lot of necessary fixes associated with it. Have you actually audited all the callers of this function you are modifying? For example, feature_bool does this:

        sub feature_bool {
                my $key = shift;
                my ($val) = git_get_project_config($key, '--bool');
                if ($val eq 'true') {
                        return (1);
                } elsif ($val eq 'false') {
	...

With your above change, I think a missing configuration variable will stuff undef in $val, and trigger the same "$val eq 'true'" comparison warning here.

Granted, without your change the existing code triggers the same error in another way, by calling config_to_bool sub with undef here:

	# ensure given type
	if (!defined $type) {
		return $config{"gitweb.$key"};
	} elsif ($type eq 'bool') {
		# backward compatibility: 'git config --bool' returns true/false
		return config_to_bool($config{"gitweb.$key"}) ? 'true' : 'false';
and config_to_bool sub is written in the same carelessness like so:
        sub config_to_bool {
                my $val = shift;
                # strip leading and trailing whitespace
                $val =~ s/^\s+//;

and triggers the same error here in the s/// operation. I think the right fix for this part would look like this:

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7c48181..2b140cc 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1920,6 +1920,8 @@ sub git_parse_project_config {
 sub config_to_bool {
 	my $val = shift;
 
+	return 1 if (!defined $val);
+
 	# strip leading and trailing whitespace
 	$val =~ s/^\s+//;
 	$val =~ s/\s+$//;

Because

	[gitweb]
        	variable

parsed by git_parse_project_config('gitweb') should return a hash that
maps "gitweb.variable" to undef it must be fed as undef to
config_to_bool.  This variable should be reported as "true".

There are tons of undef unsafeness in this file from a very cursory look.

Unrelated to any of these, I think the following is wrong:

        sub feature_patches {
                my @val = (git_get_project_config('patches', '--int'));

                if (@val) {
                        return @val;
                }

                return ($_[0]);
        }

As git_get_project_config() always returns something, hence "if (@val)"
can never be false.
Jakub Narebski· Feb 18, 2009, 13:09 UTC · re: Junio C Hamano · lore

Re: [PATCH RFC 1/2] gitweb: Fix warnings with override permitted but no repo override

Fixed up patch at the bottom.
Junio C Hamano wrote:
Show 18 quoted lines
> "Marcel M. Cary" <marcel@oak.homeunix.org> writes:
> 
> > When a feature like "blame" is permitted to be overridden in the
> > repository configuration but it is not actually set in the
> > repository, a warning is emitted due to the undefined value
> > of the repository configuration, even though it's a perfectly
> > normal condition.
> >
> > The warning is grounds for test failure in the gitweb test script,
> > so it causes some new feature tests of mine to fail.
> >
> > This patch prevents warning and adds a test case to exercise it.
> >
> > Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
> > ---
> >
> > Here's a small patch I put together while tinkering with bug hyperlinking.
> > Does this look reasonable?
Somewhat.
Corrected patch at the bottom.
Show 29 quoted lines
> 
> To my cursory look, it doesn't, and it is not entirely your fault, but if
> we applied this patch, it would not improve things very much.  It just
> would shift the same problem around.
> 
> > diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> > index 7c48181..653f0be 100755
> > --- a/gitweb/gitweb.perl
> > +++ b/gitweb/gitweb.perl
> > @@ -402,13 +402,13 @@ sub feature_bool {
> >  	my $key = shift;
> >  	my ($val) = git_get_project_config($key, '--bool');
> >  
> > -	if ($val eq 'true') {
> > +	if (!defined $val) {
> > +		return ($_[0]);
> > +	} elsif ($val eq 'true') {
> >  		return (1);
> >  	} elsif ($val eq 'false') {
> >  		return (0);
> >  	}
> 
> I think the warning you are talking about is to compare $val with 'true'
> with 'eq' operator when $val could be undef.  The check to see if $val is
> undefined to avoid that 'eq' comparison is fine, and the intent to return
> false is also good, but I think feature_bool is meant to say "yes" or
> "no", and existing code for 'false' is returning (0).  I'd rather see your
> new codepath normalize incoming undef the same way string 'false' is
> normalized to return (0).
Actually git_get_project_config($key, '--bool') can return only three
values: 
 * 'true' if gitweb.$key config variable is 'true', 'yes', 1, or (after
   fixes in the fixed up patch at the bottom) is undefined, i.e.
     [gitweb]
     	blame
   case
 * 'false' if gitweb.$key exists and has other value (that includes
   empty string value: "[section] val" is git-config --bool true, while
   "[section] val = " is --bool false).
 * undef if gitweb.$key does not exist in the config;
   earlier version which used "git config --bool <variable>" returned
   empty string ('') here.

We want to fall back to %feature value (i.e. do not override) if variable is not set in config. Variable not set was '', and now is undef, therefore need for this (correct) change.

Show 7 quoted lines
> Granted, the caller should be prepared to take 
> the answer as boolean and treat the usual Perl false values (numeric zero,
> a string with single "0", or an undef) without barfing, so returning (undef)
> from here ought to be safe (otherwise the callers are broken), but I'd
> rather see this function play safe.
> 
> But it certainly is not my main complaint.

Well, I think we can now get rid of backwards compatibility (which is not complete anyway: '' for not existent variable for old version, undef for new version) with the old version which ran git-config once for each variable, and do not go through 'true'/'false' to imitate calling "git config --bool", which has to be converted back to Perl boolean anyway.

Show 8 quoted lines
> 
> >  sub feature_snapshot {
> > @@ -1978,6 +1978,8 @@ sub git_get_project_config {
> >  		$config_file = "$git_dir/config";
> >  	}
> >  
> > +	return undef if (!defined $config{"gitweb.$key"});
> > +

It should be !exists, and not !defined here, see my fixed up patch below.

Show 17 quoted lines
> 
> I think this change is missing a lot of necessary fixes associated with
> it.  Have you actually audited all the callers of this function you are
> modifying?  For example, feature_bool does this:
> 
>         sub feature_bool {
>                 my $key = shift;
>                 my ($val) = git_get_project_config($key, '--bool');
> 
>                 if ($val eq 'true') {
>                         return (1);
>                 } elsif ($val eq 'false') {
> 	...
> 
> With your above change, I think a missing configuration variable will
> stuff undef in $val, and trigger the same "$val eq 'true'" comparison
> warning here.
Errr... Junio, Marcel DID fix feature_bool, see above:
Show 12 quoted lines
> > @@ -402,13 +402,13 @@ sub feature_bool {
> >  	my $key = shift;
> >  	my ($val) = git_get_project_config($key, '--bool');
> >  
> > -	if ($val eq 'true') {
> > +	if (!defined $val) {
> > +		return ($_[0]);
> > +	} elsif ($val eq 'true') {
> >  		return (1);
> >  	} elsif ($val eq 'false') {
> >  		return (0);
> >  	}
Show 44 quoted lines
> 
> Granted, without your change the existing code triggers the same error in
> another way, by calling config_to_bool sub with undef here:
> 
> 	# ensure given type
> 	if (!defined $type) {
> 		return $config{"gitweb.$key"};
> 	} elsif ($type eq 'bool') {
> 		# backward compatibility: 'git config --bool' returns true/false
> 		return config_to_bool($config{"gitweb.$key"}) ? 'true' : 'false';
> 
> and config_to_bool sub is written in the same carelessness like so:
> 
>         sub config_to_bool {
>                 my $val = shift;
> 
>                 # strip leading and trailing whitespace
>                 $val =~ s/^\s+//;
> 
> and triggers the same error here in the s/// operation.  I think the right
> fix for this part would look like this:
> 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 7c48181..2b140cc 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1920,6 +1920,8 @@ sub git_parse_project_config {
>  sub config_to_bool {
>  	my $val = shift;
>  
> +	return 1 if (!defined $val);
> +
>  	# strip leading and trailing whitespace
>  	$val =~ s/^\s+//;
>  	$val =~ s/\s+$//;
> 
> Because
> 
> 	[gitweb]
>         	variable
> 
> parsed by git_parse_project_config('gitweb') should return a hash that
> maps "gitweb.variable" to undef it must be fed as undef to
> config_to_bool.  This variable should be reported as "true".
Right (but not complete).

We do check !defined $val, but too late: _after_ trying to strip leading and trailing whitespace. When going through various versions of config_to_bool I have somehow forgot about this issue...

Show 17 quoted lines
> 
> There are tons of undef unsafeness in this file from a very cursory look.
> 
> Unrelated to any of these, I think the following is wrong:
> 
>         sub feature_patches {
>                 my @val = (git_get_project_config('patches', '--int'));
> 
>                 if (@val) {
>                         return @val;
>                 }
> 
>                 return ($_[0]);
>         }
> 
> As git_get_project_config() always returns something, hence "if (@val)"
> can never be false.

Actually after the patch below I think that git_get_project_config returns empty list () in the list (array) context now if variable does not exist in the config thanks to "return ;" magic. And empty list evaluates to false in scalar context: "if (@val)" would be false if variable does not exist in the config... in which case we would not override 'default' value in %feature.

Alternate solution would be to use
         sub feature_patches {
                 my $val = git_get_project_config('patches', '--int');
 
                 if (defined $val) {
                         return ($val);
                 }
 
                 return ($_[0]);
         }
-- >8 --
From: Marcel M. Cary <marcel@oak.homeunix.org>
Subject: [PATCH] gitweb: Fix warnings with override permitted but no repo override

When a feature like "blame" is permitted to be overridden in the repository configuration but it is not actually set in the repository, a warning is emitted due to the undefined value of the repository configuration, even though it's a perfectly normal condition. Emitting warning is grounds for test failure in the gitweb test script.

This error was caused by rewrite of git_get_project_config from using "git config [<type>] <name>" for each individual configuration variable checked to parsing "git config --list --null" output in commit b201927 (gitweb: Read repo config using 'git config -z -l'). Earlier version of git_get_project_config was returning empty string if variable do not exist in config; newer version is meant to return undef in this case, therefore change in feature_bool was needed.

Additionally config_to_* subroutines were meant to be invoked only if configuration variable exists; therefore we added early return to git_get_project_config: it now returns no value if variable does not exists in config. Otherwise config_to_* subroutines (config_to_bool in paryicular) wouldn't be able to distinguish between the case where variable does not exist and the case where variable doesn't have value (the "[section] noval" case, which evaluates to true for boolean).

While at it fix bug in config_to_bool, where checking if $val is defined (if config variable has value) was done _after_ stripping leading and trailing whitespace, which lead to 'Use of uninitialized value' warning.

Add test case for features overridable but not overriden in repo config, and case for no value boolean configuration variable.

Signed-off-by: Marcel M. Cary <marcel@oak.homeunix.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
 gitweb/gitweb.perl                     |   16 ++++++++++------
 t/t9500-gitweb-standalone-no-errors.sh |   18 +++++++++++++++++-
 2 files changed, 27 insertions(+), 7 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7c48181..83858fb 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -402,13 +402,13 @@ sub feature_bool {
 	my $key = shift;
 	my ($val) = git_get_project_config($key, '--bool');
 
-	if ($val eq 'true') {
+	if (!defined $val) {
+		return ($_[0]);
+	} elsif ($val eq 'true') {
 		return (1);
 	} elsif ($val eq 'false') {
 		return (0);
 	}
-
-	return ($_[0]);
 }
 
 sub feature_snapshot {
@@ -1914,18 +1914,19 @@ sub git_parse_project_config {
 	return %config;
 }
 
-# convert config value to boolean, 'true' or 'false'
+# convert config value to boolean: 'true' or 'false'
 # no value, number > 0, 'true' and 'yes' values are true
 # rest of values are treated as false (never as error)
 sub config_to_bool {
 	my $val = shift;
 
+	return 1 if !defined $val;             # section.key
+
 	# strip leading and trailing whitespace
 	$val =~ s/^\s+//;
 	$val =~ s/\s+$//;
 
-	return (!defined $val ||               # section.key
-	        ($val =~ /^\d+$/ && $val) ||   # section.key = 1
+	return (($val =~ /^\d+$/ && $val) ||   # section.key = 1
 	        ($val =~ /^(?:true|yes)$/i));  # section.key = true
 }
 
@@ -1978,6 +1979,9 @@ sub git_get_project_config {
 		$config_file = "$git_dir/config";
 	}
 
+	# check if config variable (key) exists
+	return unless exists $config{"gitweb.$key"};
+
 	# ensure given type
 	if (!defined $type) {
 		return $config{"gitweb.$key"};
diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 7c6f70b..6ed10d0 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -662,6 +662,11 @@ cat >>gitweb_config.perl <<EOF
 EOF
 
 test_expect_success \
+	'config override: tree view, features not overridden in repo config' \
+	'gitweb_run "p=.git;a=tree"'
+test_debug 'cat gitweb.log'
+
+test_expect_success \
 	'config override: tree view, features disabled in repo config' \
 	'git config gitweb.blame no &&
 	 git config gitweb.snapshot none &&
@@ -669,12 +674,23 @@ test_expect_success \
 test_debug 'cat gitweb.log'
 
 test_expect_success \
-	'config override: tree view, features enabled in repo config' \
+	'config override: tree view, features enabled in repo config (1)' \
 	'git config gitweb.blame yes &&
 	 git config gitweb.snapshot "zip,tgz, tbz2" &&
 	 gitweb_run "p=.git;a=tree"'
 test_debug 'cat gitweb.log'
 
+cat >.git/config <<\EOF
+# testing noval and alternate separator
+[gitweb]
+	blame
+	snapshot = zip tgz
+EOF
+test_expect_success \
+	'config override: tree view, features enabled in repo config (2)' \
+	'gitweb_run "p=.git;a=tree"'
+test_debug 'cat gitweb.log'
+
 # ----------------------------------------------------------------------
 # non-ASCII in README.html
 
-- 
1.6.1
Jakub Narebski· Feb 18, 2009, 03:38 UTC · re: Marcel M. Cary · lore

Re: [RFC] Configuring (future) committags support in gitweb, especially bug linking

On Tue, 17 Feb 2009, Marcel M. Cary wrote:
Show 5 quoted lines
> I'm interested in cross-linking bug references in commit messages to a
> bug tracking system.  I started tinkering a couple weeks ago and am
> finally understanding that committags encompass this functionality.
> (From the subject line I first understood "tags" to mean git tags rather
> than commit message munging.)
What would you name this feature, then?
> 
> Is the committags idea still under active development?
Well, it is in my todo list, rather further on...
[...]
Show 6 quoted lines
> Two regexes would make it easier to configure a driver without needing
> look-ahead and look-behind assertions.  For example, if you want to
> match non-negative integers but only in the context of a Resolves-bug
> header:
> 
>     Resolves-bug: 1234, 1235
[...]
> I got the two-regex idea from a spec I ran across while evaluating
> Subversion:
> 
> http://guest:@tortoisesvn.tigris.org/svn/tortoisesvn/trunk/doc/issuetrackers.txt

You don't need multiple regexps for that, and in above example it is used _single_ regexp; only with more than one catching group.

Show 7 quoted lines
> I like the idea of allowing a regex writer -- a gitweb admin or a
> repository owner -- to ignore issues regarding HTML escaping.  For
> example, I'd rather not have &nbsp; in the regex.  And I don't want the
> replacement to have to escape "&" in a query string.  That's a strength
> of not having to write the whole link replacement rule.  And I think
> hyperlinking will be one of the most common uses of this committag
> feature, so it's worth special support.
[...]
> I'm concerned about the composition of these committag drivers.  In
> other words, will it be hard for the configurer to manage interactions
> between committag drivers?  To choose a sane order, will I have to
> understand the implementation details of each committag driver?

In current proposal the order of running committags drivers is specified in configuration...

Show 12 quoted lines
> 
> Perhaps a simpler alternative would be to let at most one driver process
> a given snippet of text, forbidding nesting of replacements.  (If I
> understand Junio's suggestion to use a list of strings and refs,
> non-nesting overlaps are already not supported.)  If all replacements
> were hyperlinks -- and I expect that to be the common case -- they
> wouldn't be nestable anyway.  I wouldn't see it as a huge loss for the
> nesting examples I can think of:  Separate rules for span around S-o-b
> and linking or obfuscation of email could be combined into one...  A
> rule to shade text quoted email-style with leading angle brackets could
> just clobber any further processing of that text.  And it might simplify
> the code and testing of it quite a bit.

... but I guess that at first attempt we could support non-overlapping committags only, i.e. replacement is always as whole not passed to later committags.

Still there is a problem how to specify which parts of replacement for committags have to be HTML escaped, and which are HTML and should not be (and which are attributes, and have to be escaped too).

[...]
Show 5 quoted lines
> A few ideas for drivers that I don't think have been mentioned yet:
> 
> * Wiki page names, like to [[Feature Documentation]].  These are notable
> because they tend to contain punctuation that get HTML-escaped, like
> quotes and ampersands.

Well, I think if it would be supported, it would be a very special case, so I don't think generic support for this is needed nor required.

Show 8 quoted lines
> 
> * Links to gitweb itself, such as 123abc:file.txt and HEAD:file.txt.  I
> guess the current hash linking sort of does the first case except that
> you have to get the hash of the blob instead of using the commit hash,
> and the current hash linking wouldn't reveal the filename until after
> you click, nor when viewing textual log messages.  I'm not sure whether
> special support for linking to multi-commit diffs or other object types
> would be as helpful.

Also 'v1.5.4' etc linking to tag; both would be a good idea. At this point I think we have already list of all references (for ref markers) so it wouldn't require additional call to git command.

P.S. I understand that this post is an exception (send after long, long time), but please do not toppost in replies. It goes against natural reading order.

-- 
Jakub Narebski
Poland
Marcel M. Cary· Feb 19, 2009, 17:08 UTC · re: Jakub Narebski · lore

Re: [RFC] Configuring (future) committags support in gitweb, especially bug linking

Jakub Narebski wrote:
Show 9 quoted lines
> On Tue, 17 Feb 2009, Marcel M. Cary wrote:
>
>> I'm interested in cross-linking bug references in commit messages to
>> a bug tracking system.  I started tinkering a couple weeks ago and am
>> finally understanding that committags encompass this functionality.
>> (From the subject line I first understood "tags" to mean git tags
>> rather than commit message munging.)
>
> What would you name this feature, then?

Heh, I'm not sure. It's like a filter in the unix pipeline sense, but "commit message filter" sounds to me like some messages might be rejected. Most of the drivers markup static text with HTML tags, but not all of them. Maybe "commit message embellishment".

Perhaps a more important question is: will people find the feature once it's implemented? I think that won't be a problem provided that it's listed in the gitweb docs like the other configs.

>> Is the committags idea still under active development?
>
> Well, it is in my todo list, rather further on...

Is any code for it published in a repository anywhere? I see a branch jn/gitweb-committag merged into master that looks relevant, but it only has the sha1 regex improvement.

Show 12 quoted lines
>> Two regexes would make it easier to configure a driver without
>> needing look-ahead and look-behind assertions.  For example, if you
>> want to match non-negative integers but only in the context of a
>> Resolves-bug header:
>>
>>     Resolves-bug: 1234, 1235
>
> [...]
>> I got the two-regex idea from a spec I ran across while evaluating
>> Subversion:
>>
>>
http://guest:@tortoisesvn.tigris.org/svn/tortoisesvn/trunk/doc/issuetrackers.txt
>
> You don't need multiple regexps for that, and in above example it is
> used _single_ regexp; only with more than one catching group.

I'm not sure what exactly you propose. In the second example in the bugtraq spec, there are two regexes. Maybe you mean something like this, but it breaks with three bugs:

    $ perl -MData::Dumper -wne '
        m/^Resolves-bug: (\d+)(?:, (\d+))*/;
        print Dumper([$1, $2, $3, $4]);
    '
    Resolves-bug: 123
    $VAR1 = [
          '123',
          undef,
          undef,
          undef
        ];
    Resolves-bug: 123, 124
    $VAR1 = [
          '123',
          '124',
          undef,
          undef
        ];
    Resolves-bug: 123, 124, 125
    $VAR1 = [
          '123',
          '125',
          undef,
          undef
        ];

Maybe something like this? But it's limited to an arbitrary number of bug matches. Maybe it's good enough for pratical purposes, but it's prone to unexpected breakage when the user exceeds the threshold of, in this case, four bugs.

    /^Resolves-bug: (\d+)(?:, (\d+))?(?:, (\d+))?(?:, (\d+))?/
Marcel

← back to recent threads