threads / patch / 31618

patch, 3 partsnicer receive-pack errors over http

Subject: [PATCH 0/3] nicer receive-pack errors over http

## tl;dr

8 messages between Sep 21, 2012 and Sep 21, 2012. Diffs are folded; open one to read it.

replies: 7people: 2as markdown or json

Jeff King· Sep 21, 2012, 05:30 UTC · lore

While we are on the subject of http user experience, I thought I'd mention this patch to route more errors from index-pack back to the user. We're not doing it yet at GitHub, but I plan to apply it soon.

The first patch is a cleanup and minor bug fix. The second is the interesting one. The third is purely cosmetic, and doesn't need to be tied to the others.

  [1/3]: receive-pack: redirect unpack-objects stdout to /dev/null
  [2/3]: receive-pack: send pack-processing stderr over sideband
  [3/3]: receive-pack: drop "n/a" on unpacker errors
-Peff
Jeff King· Sep 21, 2012, 05:32 UTC · re: Jeff King · lore

[PATCH 1/3] receive-pack: redirect unpack-objects stdout to /dev/null

The unpack-objects command should not generally produce any output on stdout. However, if it's given extra input after the packfile, it will spew the remainder to stdout. When called by receive-pack, this means we will break protocol, since our stdout is connected to the remote send-pack.

Signed-off-by: Jeff King <peff@peff.net>
---
I've never actually seen this bug in practice, but I noticed it while
auditing the outputs of receive-pack's children recently.
 builtin/receive-pack.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Show changes to builtin/receive-pack.c +6 −1
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 9145f1a..5ba0c98 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -815,6 +815,7 @@ static const char *unpack(void)
 
 	if (ntohl(hdr.hdr_entries) < unpack_limit) {
 		int code, i = 0;
+		struct child_process child;
 		const char *unpacker[5];
 		unpacker[i++] = "unpack-objects";
 		if (quiet)
@@ -823,7 +824,11 @@ static const char *unpack(void)
 			unpacker[i++] = "--strict";
 		unpacker[i++] = hdr_arg;
 		unpacker[i++] = NULL;
-		code = run_command_v_opt(unpacker, RUN_GIT_CMD);
+		memset(&child, 0, sizeof(child));
+		child.argv = unpacker;
+		child.no_stdout = 1;
+		child.git_cmd = 1;
+		code = run_command(&child);
 		if (!code)
 			return NULL;
 		return "unpack-objects abnormal exit";
-- 
1.7.11.7.15.g085c6bd
Jeff King· Sep 21, 2012, 05:34 UTC · re: Jeff King · lore

[PATCH 2/3] receive-pack: send pack-processing stderr over sideband

Receive-pack invokes either unpack-objects or index-pack to handle the incoming pack. However, we do not redirect the stderr of the sub-processes at all, so it is never seen by the client. From the initial thread adding sideband support, which is here:

  http://thread.gmane.org/gmane.comp.version-control.git/139471

it is clear that some messages are specifically kept off the sideband (with the assumption that they are of interest only to an administrator, not the client). The stderr of the subprocesses is mentioned in the thread, but it's unclear if they are included in that group, or were simply forgotten.

However, there are a few good reasons to show them to the client:

  1. In many cases, they are directly about the incoming
     packfile (e.g., fsck warnings with --strict, corruption
     in the packfile, etc). Without these messages, the
     client just gets "unpacker error" with no extra useful
     diagnosis.
  2. No matter what the cause, we are probably better off
     showing the errors to the client. If the client and the
     server admin are not the same entity, it is probably
     much easier for the client to cut-and-paste the errors
     they see than for the admin to try to dig them out of a
     log and correlate them with a particular session.
  3. Users of the ssh transport typically already see these
     stderr messages, as the remote's stderr is copied
     literally by ssh. This brings other transports (http,
     and push-over-git if you are crazy enough to enable it)
     more in line with ssh. As a bonus for ssh users,
     because the messages are now fed through the sideband
     and printed by the local git, they will have "remote:"
     prepended and be properly interleaved with any local
     output to stderr.
Signed-off-by: Jeff King <peff@peff.net>
---
This one is logically independent of the first patch, but relies
textually on the conversion of unpack-objects to use a separate
child_process.
 builtin/receive-pack.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)
Show changes to builtin/receive-pack.c +24 −2
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5ba0c98..ac679ab 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -795,7 +795,7 @@ static const char *pack_lockfile;
 
 static const char *pack_lockfile;
 
-static const char *unpack(void)
+static const char *unpack(int err_fd)
 {
 	struct pack_header hdr;
 	const char *hdr_err;
@@ -827,6 +827,7 @@ static const char *unpack(void)
 		memset(&child, 0, sizeof(child));
 		child.argv = unpacker;
 		child.no_stdout = 1;
+		child.err = err_fd;
 		child.git_cmd = 1;
 		code = run_command(&child);
 		if (!code)
@@ -853,6 +854,7 @@ static const char *unpack(void)
 		memset(&ip, 0, sizeof(ip));
 		ip.argv = keeper;
 		ip.out = -1;
+		ip.err = err_fd;
 		ip.git_cmd = 1;
 		status = start_command(&ip);
 		if (status) {
@@ -869,6 +871,26 @@ static const char *unpack(void)
 	}
 }
 
+static const char *unpack_with_sideband(void)
+{
+	struct async muxer;
+	const char *ret;
+
+	if (!use_sideband)
+		return unpack(0);
+
+	memset(&muxer, 0, sizeof(muxer));
+	muxer.proc = copy_to_sideband;
+	muxer.in = -1;
+	if (start_async(&muxer))
+		return NULL;
+
+	ret = unpack(muxer.in);
+
+	finish_async(&muxer);
+	return ret;
+}
+
 static void report(struct command *commands, const char *unpack_status)
 {
 	struct command *cmd;
@@ -966,7 +988,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 		const char *unpack_status = NULL;
 
 		if (!delete_only(commands))
-			unpack_status = unpack();
+			unpack_status = unpack_with_sideband();
 		execute_commands(commands, unpack_status);
 		if (pack_lockfile)
 			unlink_or_warn(pack_lockfile);
-- 
1.7.11.7.15.g085c6bd
Junio C Hamano· Sep 21, 2012, 16:49 UTC · re: Jeff King · lore

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

Jeff King <peff@peff.net> writes:
Show 29 quoted lines
> Receive-pack invokes either unpack-objects or index-pack to
> handle the incoming pack. However, we do not redirect the
> stderr of the sub-processes at all, so it is never seen by
> the client. From the initial thread adding sideband support,
> which is here:
>
>   http://thread.gmane.org/gmane.comp.version-control.git/139471
>
> it is clear that some messages are specifically kept off the
> sideband (with the assumption that they are of interest only
> to an administrator, not the client). The stderr of the
> subprocesses is mentioned in the thread, but it's unclear if
> they are included in that group, or were simply forgotten.
>
> However, there are a few good reasons to show them to the
> client:
>
>   1. In many cases, they are directly about the incoming
>      packfile (e.g., fsck warnings with --strict, corruption
>      in the packfile, etc). Without these messages, the
>      client just gets "unpacker error" with no extra useful
>      diagnosis.
>
>   2. No matter what the cause, we are probably better off
>      showing the errors to the client. If the client and the
>      server admin are not the same entity, it is probably
>      much easier for the client to cut-and-paste the errors
>      they see than for the admin to try to dig them out of a
>      log and correlate them with a particular session.

I agree with the "probably" above (and also with points 1 and 3), but it at the same time feel a bit iffy. The server side would lose log entries to check when the operator observes higher error rate and starts suspecting something recently broke, and the lost clue cannot be recovered without contacting the pushers, no?

Show 12 quoted lines
>   3. Users of the ssh transport typically already see these
>      stderr messages, as the remote's stderr is copied
>      literally by ssh. This brings other transports (http,
>      and push-over-git if you are crazy enough to enable it)
>      more in line with ssh. As a bonus for ssh users,
>      because the messages are now fed through the sideband
>      and printed by the local git, they will have "remote:"
>      prepended and be properly interleaved with any local
>      output to stderr.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
Will queue; thanks.
Jeff King· Sep 21, 2012, 17:05 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

On Fri, Sep 21, 2012 at 09:49:40AM -0700, Junio C Hamano wrote:
Show 12 quoted lines
> >   2. No matter what the cause, we are probably better off
> >      showing the errors to the client. If the client and the
> >      server admin are not the same entity, it is probably
> >      much easier for the client to cut-and-paste the errors
> >      they see than for the admin to try to dig them out of a
> >      log and correlate them with a particular session.
> 
> I agree with the "probably" above (and also with points 1 and 3),
> but it at the same time feel a bit iffy.  The server side would lose
> log entries to check when the operator observes higher error rate
> and starts suspecting something recently broke, and the lost clue
> cannot be recovered without contacting the pushers, no?

Yeah, that is true, although that is already the case with ssh pushes. Conversely, it also means that servers using the ssh transport have lost the option of redirecting the server-side stderr (e.g., with a wrapper around git-receive-pack) to a log if they were already doing so.

However, this does make things more consistent with upload-pack, which connects the stderr of pack-objects to sideband (which it must to handle progress). Furthermore, many of the messages from receive-pack are handled by rp_error, which sends to the sideband. So if you were monitoring your git purely by trying to capture stderr, you were already only getting a fraction of the real data.

If a server side really did want to capture the error output, I think the right way to do it would be:

  1. Modify send_sideband to send a copy of all band-2 data to stderr.
  2. Redirect stderr from all processes to a log processor (it's
     tempting to just send it somewhere besides stderr in (1) above, but
     you may also get clients which do not claim to support sidebands,
     in which case we will just spew to stderr).
  3. Do immediate culling on the output before storage, because some of
     what you get will be junk like progress reports (and they all come
     down the same fd, since it is typically stderr from a subprocess).

So I think this patch is not really making anything _worse_ if somebody wanted to do that. It just moves the messages from being caught by step (2) to being caught by step (1). But you have to do both either way.

-Peff
Junio C Hamano· Sep 21, 2012, 17:25 UTC · re: Jeff King · lore

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

Jeff King <peff@peff.net> writes:
Show 19 quoted lines
> On Fri, Sep 21, 2012 at 09:49:40AM -0700, Junio C Hamano wrote:
>
>> >   2. No matter what the cause, we are probably better off
>> >      showing the errors to the client. If the client and the
>> >      server admin are not the same entity, it is probably
>> >      much easier for the client to cut-and-paste the errors
>> >      they see than for the admin to try to dig them out of a
>> >      log and correlate them with a particular session.
>> 
>> I agree with the "probably" above (and also with points 1 and 3),
>> but it at the same time feel a bit iffy.  The server side would lose
>> log entries to check when the operator observes higher error rate
>> and starts suspecting something recently broke, and the lost clue
>> cannot be recovered without contacting the pushers, no?
>
> Yeah, that is true, although that is already the case with ssh pushes.
> Conversely, it also means that servers using the ssh transport have lost
> the option of redirecting the server-side stderr (e.g., with a wrapper
> around git-receive-pack) to a log if they were already doing so.
Yes.
Show 6 quoted lines
> However, this does make things more consistent with upload-pack, which
> connects the stderr of pack-objects to sideband (which it must to handle
> progress). Furthermore, many of the messages from receive-pack are
> handled by rp_error, which sends to the sideband. So if you were
> monitoring your git purely by trying to capture stderr, you were already
> only getting a fraction of the real data.

The comments were not meant as a rejection notice ;-) Just to see if some server operators have input on the matter.

I personally do not think tee-ing the error output is worth it; it would be reasonably simple to arrange, and the server operators who want it can ask later if that is need.

Thanks.
Jeff King· Sep 21, 2012, 17:40 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

On Fri, Sep 21, 2012 at 10:25:24AM -0700, Junio C Hamano wrote:
Show 9 quoted lines
> > However, this does make things more consistent with upload-pack, which
> > connects the stderr of pack-objects to sideband (which it must to handle
> > progress). Furthermore, many of the messages from receive-pack are
> > handled by rp_error, which sends to the sideband. So if you were
> > monitoring your git purely by trying to capture stderr, you were already
> > only getting a fraction of the real data.
> 
> The comments were not meant as a rejection notice ;-) Just to see if
> some server operators have input on the matter.

I know. But your comment made me second-guess a little whether anybody would be inadvertently hurt, but thinking it through and writing it out helped convince myself that it's the right thing to do.

One of the hardest parts of working on a mature software project is not just thinking about what you want to do, but thinking about what everyone else wants to do (or is doing). So it never hurts to double-check your assumptions in such a case, and I don't mind working through these "what ifs" even if they end in us doing the original thing. I hope you do not mind reading them too much. :)

> I personally do not think tee-ing the error output is worth it; it
> would be reasonably simple to arrange, and the server operators who
> want it can ask later if that is need.
Agreed.
-Peff
Jeff King· Sep 21, 2012, 05:38 UTC · re: Jeff King · lore

[PATCH 3/3] receive-pack: drop "n/a" on unpacker errors

The output from git push currently looks like this:
  $ git push dest HEAD
  fatal: [some message from index-pack]
  error: unpack failed: index-pack abnormal exit
  To dest
   ! [remote rejected] HEAD -> master (n/a (unpacker error))

That n/a is meant to be "the per-ref status is not available" but the nested parentheses just make it look ugly. Let's turn the final line into just:

   ! [remote rejected] HEAD -> master (unpacker error)
Signed-off-by: Jeff King <peff@peff.net>
---
Maybe it is just me, but I have always found the "n/a" and extra
parentheses ugly and unnecessary. But obviously others may differ.
It doesn't really come up that often, since index-pack failing usually
implies a git bug. But with transfer.fsckObjects turn on, it is more
common.

I don't think there should be any backwards compatibility issues with changing this. The "reason" field sent back by receive-pack has always been a free-form human-readable string.

I also dislike the "index-pack abnormal exit" message. Again, when index-pack really crashes, it's fine, but it can die due to bogus objects, too, in which case it might be nice to have a more human-readable message.

 builtin/receive-pack.c          | 2 +-
 t/t5504-fetch-receive-strict.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Show changes to 2 files +2 −2

builtin/receive-pack.c, t/t5504-fetch-receive-strict.sh

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ac679ab..ff781fe 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -695,7 +695,7 @@ static void execute_commands(struct command *commands, const char *unpacker_erro
 
 	if (unpacker_error) {
 		for (cmd = commands; cmd; cmd = cmd->next)
-			cmd->error_string = "n/a (unpacker error)";
+			cmd->error_string = "unpacker error";
 		return;
 	}
 
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index 35ec294..69ee13c 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -89,7 +89,7 @@ To dst
 
 cat >exp <<EOF
 To dst
-!	refs/heads/master:refs/heads/test	[remote rejected] (n/a (unpacker error))
+!	refs/heads/master:refs/heads/test	[remote rejected] (unpacker error)
 EOF
 
 test_expect_success 'push with receive.fsckobjects' '
-- 
1.7.11.7.15.g085c6bd

← back to recent threads