threads / patch / 48961

patchRFC Highlight keywords in remote sideband output.

Subject: [PATCH] RFC Highlight keywords in remote sideband output.

## tl;dr

4 messages between Jul 26, 2018 and Jul 30, 2018. Diffs are folded; open one to read it.

replies: 3people: 3as markdown or json

Han-Wen Nienhuys· Jul 26, 2018, 17:17 UTC · lore
Supported keywords are "error", "warning", "hint" and "success".
TODO:
 * make the coloring optional? What variable to use?
 * doc for the coloring option.
 * how to test?
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I090412a1288bc2caef0916447e28c2d0199da47d
---
 sideband.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 60 insertions(+), 9 deletions(-)
Show changes to sideband.c +60 −9
diff --git a/sideband.c b/sideband.c
index 325bf0e97..c8b7cb6dd 100644
--- a/sideband.c
+++ b/sideband.c
@@ -1,6 +1,53 @@
 #include "cache.h"
 #include "pkt-line.h"
 #include "sideband.h"
+#include "color.h"
+
+/*
+ * Optionally highlight some keywords in remote output if they appear at the
+ * start of the line.
+ */
+void emit_sideband(struct strbuf *dest, const char *src, int n) {
+        // NOSUBMIT - maybe use transport.color property?
+        int want_color = want_color_stderr(GIT_COLOR_AUTO);
+
+        if (!want_color) {
+                strbuf_add(dest, src, n);
+                return;
+        }
+
+        struct kwtable {
+                const char* keyword;
+                const char* color;
+        } keywords[] = {
+                {"hint", GIT_COLOR_YELLOW},
+                {"warning", GIT_COLOR_BOLD_YELLOW},
+                {"success", GIT_COLOR_BOLD_GREEN},
+                {"error", GIT_COLOR_BOLD_RED},
+                {},
+        };
+
+        while (isspace(*src)) {
+                strbuf_addch(dest, *src);
+                src++;
+                n--;
+        }
+
+        for (struct kwtable* p = keywords; p->keyword; p++) {
+                int len = strlen(p->keyword);
+                if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
+                        strbuf_addstr(dest, p->color);
+                        strbuf_add(dest, src, len);
+                        strbuf_addstr(dest, GIT_COLOR_RESET);
+                        n -= len;
+                        src += len;
+                        break;
+                }
+        }
+
+        strbuf_add(dest, src, n);
+}
+
 
 /*
  * Receive multiplexed output stream over git native protocol.
@@ -48,8 +95,10 @@ int recv_sideband(const char *me, int in_stream, int out)
 		len--;
 		switch (band) {
 		case 3:
-			strbuf_addf(&outbuf, "%s%s%s", outbuf.len ? "\n" : "",
-				    DISPLAY_PREFIX, buf + 1);
+			strbuf_addf(&outbuf, "%s%s", outbuf.len ? "\n" : "",
+				    DISPLAY_PREFIX);
+                        emit_sideband(&outbuf, buf+1, len);
+
 			retval = SIDEBAND_REMOTE_ERROR;
 			break;
 		case 2:
@@ -69,20 +118,22 @@ int recv_sideband(const char *me, int in_stream, int out)
 				if (!outbuf.len)
 					strbuf_addstr(&outbuf, DISPLAY_PREFIX);
 				if (linelen > 0) {
-					strbuf_addf(&outbuf, "%.*s%s%c",
-						    linelen, b, suffix, *brk);
-				} else {
-					strbuf_addch(&outbuf, *brk);
+                                        emit_sideband(&outbuf, b, linelen);
+                                        strbuf_addstr(&outbuf, suffix);
 				}
+
+                                strbuf_addch(&outbuf, *brk);
 				xwrite(2, outbuf.buf, outbuf.len);
 				strbuf_reset(&outbuf);
 
 				b = brk + 1;
 			}
 
-			if (*b)
-				strbuf_addf(&outbuf, "%s%s", outbuf.len ?
-					    "" : DISPLAY_PREFIX, b);
+			if (*b) {
+				strbuf_addstr(&outbuf, outbuf.len ?
+					    "" : DISPLAY_PREFIX);
+                                emit_sideband(&outbuf, b, strlen(b));
+                        }
 			break;
 		case 1:
 			write_or_die(out, buf + 1, len);
-- 
2.18.0.233.g985f88cf7e-goog
Stefan Beller· Jul 26, 2018, 17:53 UTC · re: Han-Wen Nienhuys · lore

Re: [PATCH] RFC Highlight keywords in remote sideband output.

On Thu, Jul 26, 2018 at 10:18 AM Han-Wen Nienhuys <hanwen@google.com> wrote:
>
> Supported keywords are "error", "warning", "hint" and "success".
Thanks for taking this upstream. :-)
>
> TODO:
>  * make the coloring optional? What variable to use?

This is the natural extension of the topic merged at a56fb3dcc09 (Merge branch 'js/colored-push-errors', 2018-05-08), which was merged rather recently, so I'd think extending the color.transport option would be useful here. (cc'd the authors of that series)

>  * doc for the coloring option.
>  * how to test?

I think the best way to get started with a test is to be inspired by 8301266afa4 (push: test to verify that push errors are colored, 2018-04-21) from that series.

> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Change-Id: I090412a1288bc2caef0916447e28c2d0199da47d

We do not do Change-Ids in git upstream. :/ As different workflows have different edges, everyone has their own little script to deal with those. For example Brandon has https://github.com/bmwill/dotfiles/blob/master/bin/check-patch that contains a section to remove change ids

    # Remove Change-Id from patch
    sed -i "/Change-Id:/d" "$f"
> +void emit_sideband(struct strbuf *dest, const char *src, int n) {

Coding style: we start the brace in the next line for new functions (but not after if/while/for)

Also I did not think hard enough in the internal review, as this function is not emitting to the sideband. that is solely done by the xwrite call in recv_sideband. So maybe prepare_sideband would be a better name?

> +        // NOSUBMIT - maybe use transport.color property?
Yes, that would be my suggestion (note that we do not use // comments)
Show 88 quoted lines
> +        int want_color = want_color_stderr(GIT_COLOR_AUTO);
> +
> +        if (!want_color) {
> +                strbuf_add(dest, src, n);
> +                return;
> +        }
> +
> +        struct kwtable {
> +                const char* keyword;
> +                const char* color;
> +        } keywords[] = {
> +                {"hint", GIT_COLOR_YELLOW},
> +                {"warning", GIT_COLOR_BOLD_YELLOW},
> +                {"success", GIT_COLOR_BOLD_GREEN},
> +                {"error", GIT_COLOR_BOLD_RED},
> +                {},
> +        };
> +
> +        while (isspace(*src)) {
> +                strbuf_addch(dest, *src);
> +                src++;
> +                n--;
> +        }
> +
> +        for (struct kwtable* p = keywords; p->keyword; p++) {
> +                int len = strlen(p->keyword);
> +                if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
> +                        strbuf_addstr(dest, p->color);
> +                        strbuf_add(dest, src, len);
> +                        strbuf_addstr(dest, GIT_COLOR_RESET);
> +                        n -= len;
> +                        src += len;
> +                        break;
> +                }
> +        }
> +
> +        strbuf_add(dest, src, n);
> +}
> +
>
>  /*
>   * Receive multiplexed output stream over git native protocol.
> @@ -48,8 +95,10 @@ int recv_sideband(const char *me, int in_stream, int out)
>                 len--;
>                 switch (band) {
>                 case 3:
> -                       strbuf_addf(&outbuf, "%s%s%s", outbuf.len ? "\n" : "",
> -                                   DISPLAY_PREFIX, buf + 1);
> +                       strbuf_addf(&outbuf, "%s%s", outbuf.len ? "\n" : "",
> +                                   DISPLAY_PREFIX);
> +                        emit_sideband(&outbuf, buf+1, len);
> +
>                         retval = SIDEBAND_REMOTE_ERROR;
>                         break;
>                 case 2:
> @@ -69,20 +118,22 @@ int recv_sideband(const char *me, int in_stream, int out)
>                                 if (!outbuf.len)
>                                         strbuf_addstr(&outbuf, DISPLAY_PREFIX);
>                                 if (linelen > 0) {
> -                                       strbuf_addf(&outbuf, "%.*s%s%c",
> -                                                   linelen, b, suffix, *brk);
> -                               } else {
> -                                       strbuf_addch(&outbuf, *brk);
> +                                        emit_sideband(&outbuf, b, linelen);
> +                                        strbuf_addstr(&outbuf, suffix);
>                                 }
> +
> +                                strbuf_addch(&outbuf, *brk);
>                                 xwrite(2, outbuf.buf, outbuf.len);
>                                 strbuf_reset(&outbuf);
>
>                                 b = brk + 1;
>                         }
>
> -                       if (*b)
> -                               strbuf_addf(&outbuf, "%s%s", outbuf.len ?
> -                                           "" : DISPLAY_PREFIX, b);
> +                       if (*b) {
> +                               strbuf_addstr(&outbuf, outbuf.len ?
> +                                           "" : DISPLAY_PREFIX);
> +                                emit_sideband(&outbuf, b, strlen(b));
> +                        }
>                         break;
>                 case 1:
>                         write_or_die(out, buf + 1, len);
> --
> 2.18.0.233.g985f88cf7e-goog
>
Junio C Hamano· Jul 26, 2018, 18:50 UTC · re: Han-Wen Nienhuys · lore

Re: [PATCH] RFC Highlight keywords in remote sideband output.

Han-Wen Nienhuys <hanwen@google.com> writes:
Show 6 quoted lines
> Supported keywords are "error", "warning", "hint" and "success".
>
> TODO:
>  * make the coloring optional? What variable to use?
>  * doc for the coloring option.
>  * how to test?

At the sideband layer, the sender only has the same level of information as the receiver has regarding what the random bytes that happens to be sent on the sideband, and that shows from the need to guess with "keywords" in this code. So if you are operating on the sideband layer, I think the right way to do so would be to add the coloring on the receiving end. That way, talking to the same sender, each receiver can decide if s/he prefers or capable of color.

    Hold your objection a bit.  I'll come back to it soon ;-)

It theoretically may make more sense to color on the sender side, but that is true only if done at a higher layer that prepares a string and calls into the sideband code to send. That code must know what the bytes _mean_ a lot better than the code at the sideband layer, so we do not have to guess.

Having written all the above, I think you are doing this at the receiving end, so this actually makes quite a lot of sense. I was fooled greatly by "EMIT_sideband", which in reality does NOT emit at all. That function is badly misnamed.

The function is more like "color sideband payload"; actual "emitting" is still done at the places the code originally "emitted" them to the receiving user.

> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Change-Id: I090412a1288bc2caef0916447e28c2d0199da47d
That's an unusual trailer we do not use in this project.
Show 19 quoted lines
> ---
>  sideband.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++-------
>  1 file changed, 60 insertions(+), 9 deletions(-)
>
> diff --git a/sideband.c b/sideband.c
> index 325bf0e97..c8b7cb6dd 100644
> --- a/sideband.c
> +++ b/sideband.c
> @@ -1,6 +1,53 @@
>  #include "cache.h"
>  #include "pkt-line.h"
>  #include "sideband.h"
> +#include "color.h"
> +
> +/*
> + * Optionally highlight some keywords in remote output if they appear at the
> + * start of the line.
> + */
> +void emit_sideband(struct strbuf *dest, const char *src, int n) {
Open brace on its own line.
> +        // NOSUBMIT - maybe use transport.color property?
Avoid // comment.
Show 10 quoted lines
> +        int want_color = want_color_stderr(GIT_COLOR_AUTO);
> +
> +        if (!want_color) {
> +                strbuf_add(dest, src, n);
> +                return;
> +        }
> +
> +        struct kwtable {
> +                const char* keyword;
> +                const char* color;
In our codebase in C, asterisk sticks to the variable not the type.
Show 6 quoted lines
> +        } keywords[] = {
> +                {"hint", GIT_COLOR_YELLOW},
> +                {"warning", GIT_COLOR_BOLD_YELLOW},
> +                {"success", GIT_COLOR_BOLD_GREEN},
> +                {"error", GIT_COLOR_BOLD_RED},
> +                {},

Drop the last sentinel element, and instead stop iteration over the array using (i < ARRAY_SIZE(keywords)).

Show 9 quoted lines
> +        };
> +
> +        while (isspace(*src)) {
> +                strbuf_addch(dest, *src);
> +                src++;
> +                n--;
> +        }
> +
> +        for (struct kwtable* p = keywords; p->keyword; p++) {

Does anybody know if we already use the variable decl inside the "for (...)" construct like this? I know we discussed the idea of using it somewhere as a weather-balloon to see if people with exotic environment would mind, and I certainly do not mind making this patch serve as such a weather-baloon, but if that is what we are doing, I want the commit log message clearly marked as such, so that we can later "git log --grep=C99" to see how long ago such an experiment started.

Note that we did attempt to start such an experiment once
cf. https://public-inbox.org/git/20170719181956.15845-1-sbeller@google.com/
but failed.
Show 14 quoted lines
> +                int len = strlen(p->keyword);
> +                if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
> +                        strbuf_addstr(dest, p->color);
> +                        strbuf_add(dest, src, len);
> +                        strbuf_addstr(dest, GIT_COLOR_RESET);
> +                        n -= len;
> +                        src += len;
> +                        break;
> +                }
> +        }
> +
> +        strbuf_add(dest, src, n);
> +}
> +

I wonder how this interacts with exotics like the progress output, which is essentially a single long line with many CRs sprinkled in, sent on many packets? I offhand do not think of a possible bad interaction, but others may be able to think of some.

Show 12 quoted lines
>  /*
>   * Receive multiplexed output stream over git native protocol.
> @@ -48,8 +95,10 @@ int recv_sideband(const char *me, int in_stream, int out)
>  		len--;
>  		switch (band) {
>  		case 3:
> -			strbuf_addf(&outbuf, "%s%s%s", outbuf.len ? "\n" : "",
> -				    DISPLAY_PREFIX, buf + 1);
> +			strbuf_addf(&outbuf, "%s%s", outbuf.len ? "\n" : "",
> +				    DISPLAY_PREFIX);
> +                        emit_sideband(&outbuf, buf+1, len);
> +
Let's not lose SP around "+" on both sides.

Also you seem to be indenting some lines with all SP and some with mixture of HT and SP. We prefer to use as many 8-column HT and then fill the remainder with SP if needed to align with the opening parenthesis on line above it (imitate the way strbuf_addf() is split into two lines in the original in this hunk).

Show 10 quoted lines
> @@ -69,20 +118,22 @@ int recv_sideband(const char *me, int in_stream, int out)
>  				if (!outbuf.len)
>  					strbuf_addstr(&outbuf, DISPLAY_PREFIX);
>  				if (linelen > 0) {
> -					strbuf_addf(&outbuf, "%.*s%s%c",
> -						    linelen, b, suffix, *brk);
> -				} else {
> -					strbuf_addch(&outbuf, *brk);
> +                                        emit_sideband(&outbuf, b, linelen);
> +                                        strbuf_addstr(&outbuf, suffix);

Here, suffix is the "clear to the end of the line" sequence (used mostly for progress display), so it is correct to exclude that from the call to "color this sideband payload" helper. Good.

Thanks. While there are need for mostly minor fix-ups, the logic seems quite sane. I think we can start without configuration and then "fix" it later.

While I am OK with calling that variable "transport.<something>", we should not define/explain it as "color output coming from the other end over the wire transport". Those who want to see messages emitted remotely during "git fetch" in color would want to see the messages generated by "git fetch" locally painted in the same color scheme, so it makes sense to let "git fetch" pay attention and honor that variable even for its own locally generated messages. The variable instead means "color any message, either generated locally or remotely, during an operation that has something to do with object transport", or something like that.

Thanks.
Han-Wen Nienhuys· Jul 30, 2018, 12:24 UTC · re: Junio C Hamano · lore

Re: [PATCH] RFC Highlight keywords in remote sideband output.

On Thu, Jul 26, 2018 at 8:50 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 12 quoted lines
>     Hold your objection a bit.  I'll come back to it soon ;-)
>
> It theoretically may make more sense to color on the sender side,
> but that is true only if done at a higher layer that prepares a
> string and calls into the sideband code to send.  That code must
> know what the bytes _mean_ a lot better than the code at the
> sideband layer, so we do not have to guess.
>
> Having written all the above, I think you are doing this at the
> receiving end, so this actually makes quite a lot of sense.  I was
> fooled greatly by "EMIT_sideband", which in reality does NOT emit at
> all.  That function is badly misnamed.
fixed.
Show 8 quoted lines
> The function is more like "color sideband payload"; actual
> "emitting" is still done at the places the code originally "emitted"
> them to the receiving user.
>
> > Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> > Change-Id: I090412a1288bc2caef0916447e28c2d0199da47d
>
> That's an unusual trailer we do not use in this project.
Yes, I know. I forgot to strip it from v2 again, though :-(
> > +void emit_sideband(struct strbuf *dest, const char *src, int n) {
>
> Open brace on its own line.
Done.
> > +        // NOSUBMIT - maybe use transport.color property?
>
> Avoid // comment.
Done
> In our codebase in C, asterisk sticks to the variable not the type.
Done.
Show 9 quoted lines
> > +        } keywords[] = {
> > +                {"hint", GIT_COLOR_YELLOW},
> > +                {"warning", GIT_COLOR_BOLD_YELLOW},
> > +                {"success", GIT_COLOR_BOLD_GREEN},
> > +                {"error", GIT_COLOR_BOLD_RED},
> > +                {},
>
> Drop the last sentinel element, and instead stop iteration over the
> array using (i < ARRAY_SIZE(keywords)).
Done.
Show 10 quoted lines
> > +        for (struct kwtable* p = keywords; p->keyword; p++) {
>
> Does anybody know if we already use the variable decl inside the
> "for (...)" construct like this?  I know we discussed the idea of
> using it somewhere as a weather-balloon to see if people with exotic
> environment would mind, and I certainly do not mind making this
> patch serve as such a weather-baloon, but if that is what we are
> doing, I want the commit log message clearly marked as such, so that
> we can later "git log --grep=C99" to see how long ago such an
> experiment started.

I elided this. (I had expected for the compile to enforce restrictions like these using --std=c99.)

Show 19 quoted lines
> >   * Receive multiplexed output stream over git native protocol.
> > @@ -48,8 +95,10 @@ int recv_sideband(const char *me, int in_stream, int out)
> >               len--;
> >               switch (band) {
> >               case 3:
> > -                     strbuf_addf(&outbuf, "%s%s%s", outbuf.len ? "\n" : "",
> > -                                 DISPLAY_PREFIX, buf + 1);
> > +                     strbuf_addf(&outbuf, "%s%s", outbuf.len ? "\n" : "",
> > +                                 DISPLAY_PREFIX);
> > +                        emit_sideband(&outbuf, buf+1, len);
> > +
>
> Let's not lose SP around "+" on both sides.
>
> Also you seem to be indenting some lines with all SP and some with
> mixture of HT and SP.  We prefer to use as many 8-column HT and then
> fill the remainder with SP if needed to align with the opening
> parenthesis on line above it (imitate the way strbuf_addf() is split
> into two lines in the original in this hunk).
Fixed these, I think.
> Thanks.  While there are need for mostly minor fix-ups, the logic
> seems quite sane.  I think we can start without configuration and
> then "fix" it later.
I need the configuration to be able to test this, though.
Show 10 quoted lines
> While I am OK with calling that variable "transport.<something>", we
> should not define/explain it as "color output coming from the other
> end over the wire transport".  Those who want to see messages
> emitted remotely during "git fetch" in color would want to see the
> messages generated by "git fetch" locally painted in the same color
> scheme, so it makes sense to let "git fetch" pay attention and honor
> that variable even for its own locally generated messages.  The
> variable instead means "color any message, either generated locally
> or remotely, during an operation that has something to do with
> object transport", or something like that.

I used color.remote for the property, but I'm happy to colorize the bikeshed with another color.

--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

← back to recent threads