{"thread":{"id":"36646","subject":"Returning error message from custom smart http server","startedAt":"2014-05-13T07:39:59Z","lastAt":"2014-05-19T15:09:42Z","messageCount":5,"participants":["Ákos, Tajti","brian m. carlson","Bryan Turner","Carlos Martín Nieto","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"241338","messageId":"5371CC4F.80602@intland.com","threadId":"36646","inReplyTo":null,"subject":"Returning error message from custom smart http server","fromName":"Ákos, Tajti","fromEmail":"akos.tajti@intland.com","sentAt":"2014-05-13T07:39:59Z","receivedAt":"2014-05-13T07:39:59Z","isPatch":false,"sender":{"key":"akos.tajti@intland.com","avatar":null},"body":"Dear List,\n\nwe implemented our own git smart http server to be able to check \npermissions and other thing before pushes. It works fine, however, the \nerror messages we generate on the server side are not displayed by the \ncommand line client. On the server we generate error messages like this:\n\n response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);\n response.getWriter().write(msg);\n\nOn the command line we get this:\n\nTotal 0 (delta 0), reused 0 (delta 0)\nPOST git-receive-pack (290 bytes)\nefrror: RPC failed; result=22, HTTP code = 401\natal: The remote end hung up unexpectedly\nfatal: The remote end hung up unexpectedly\n\nThe server message is completely missing. Is there a solution for this?\n\nThanks,\nÁkos Tajti\n\n\n---\nA levél vírus, és rosszindulatú kód mentes, mert az avast! Antivirus védelme ellenőrizte azt.\nhttp://www.avast.com\n"},{"id":"242011","messageId":"20140516230116.GC180798@vauxhall.crustytoothpaste.net","threadId":"36646","inReplyTo":"5371CC4F.80602@intland.com","subject":"Re: Returning error message from custom smart http server","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-05-16T23:01:16Z","receivedAt":"2014-05-16T23:01:16Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, May 13, 2014 at 09:39:59AM +0200, \"Ákos, Tajti\" wrote:\n> Dear List,\n> \n> we implemented our own git smart http server to be able to check permissions\n> and other thing before pushes. It works fine, however, the error messages we\n> generate on the server side are not displayed by the command line client. On\n> the server we generate error messages like this:\n> \n> response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);\n> response.getWriter().write(msg);\n> \n> On the command line we get this:\n> \n> Total 0 (delta 0), reused 0 (delta 0)\n> POST git-receive-pack (290 bytes)\n> efrror: RPC failed; result=22, HTTP code = 401\n> atal: The remote end hung up unexpectedly\n> fatal: The remote end hung up unexpectedly\n> \n> The server message is completely missing. Is there a solution for this?\n\nIt does look that way. Does the following patch work for you?\n\n-- >8 --\nSubject: [PATCH] http: provide server's error message on RPC failure\n\nThe server might provide a custom error message that is useful to the user.\nProvide this message to the user if HTTP RPC fails.\n\nSigned-off-by: brian m. carlson \n---\n remote-curl.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 52c2d96..5984d35 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -426,8 +426,8 @@ static int run_slot(struct active_request_slot *slot,\n \terr = run_one_slot(slot, results);\n \n \tif (err != HTTP_OK && err != HTTP_REAUTH) {\n-\t\terror(\"RPC failed; result=%d, HTTP code = %ld\",\n-\t\t results->curl_result, results->http_code);\n+\t\terror(\"RPC failed; result=%d, HTTP code = %ld (%s)\",\n+\t\t results->curl_result, results->http_code, curl_errorstr);\n \t}\n \n \treturn err;\n-- >8 --\n\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"},{"id":"242146","messageId":"CAGyf7-EoSgTxZzReFArOgcrBaARv7fRiZTMPZX+Loy9dec23aQ@mail.gmail.com","threadId":"36646","inReplyTo":"20140516230116.GC180798@vauxhall.crustytoothpaste.net","subject":"Re: Returning error message from custom smart http server","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2014-05-19T08:12:10Z","receivedAt":"2014-05-19T08:12:10Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"On Sat, May 17, 2014 at 9:01 AM, brian m. carlson\n wrote:\n> On Tue, May 13, 2014 at 09:39:59AM +0200, \"Ákos, Tajti\" wrote:\n>> Dear List,\n>>\n>> we implemented our own git smart http server to be able to check permissions\n>> and other thing before pushes. It works fine, however, the error messages we\n>> generate on the server side are not displayed by the command line client. On\n>> the server we generate error messages like this:\n>>\n>> response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);\n>> response.getWriter().write(msg);\n>>\n>> On the command line we get this:\n>>\n>> Total 0 (delta 0), reused 0 (delta 0)\n>> POST git-receive-pack (290 bytes)\n>> efrror: RPC failed; result=22, HTTP code = 401\n>> atal: The remote end hung up unexpectedly\n>> fatal: The remote end hung up unexpectedly\n>>\n>> The server message is completely missing. Is there a solution for this?\n\nYou should not need a patched git; the wire protocol itself has a\nmechanism for sending \"smart\" error messages. It's not particularly\n_obvious_, but it's there.\n\nFor starters, to return an error message, your status must be 200 OK.\nYou can't return any other status code or Git will interpret your\nerror as some form of _HTTP_ error rather than a _git_ error.\n\nIn the smart protocol the client sends a service to the server as a\nquery parameter, like \"?service=git-receive-pack\". For such a request,\nyou need to:\n- Set the content type to \"application/x--advertisement\"\n(e.g. \"application/x-git-receive-pack-advertisement\") (Not all command\nline Git versions require this, but JGit does)\n- Set the status code as 200 OK\n- Write back a payload where the first 4 bytes are the hex-encoded\nlength of the text (where \"FFFF\" is max length for a single packet).\nNote that the 4 bytes for the size are _part_ of that length, so if\nyou're writing \"Test\" the length is 8, not 4\n- After the size, you write \"# service=\" (e.g. \"#\nservice=git-receive-pack\"; note the space after the #) This is the\nmetadata. For an error, you don't really have much to say.\n- After that, an empty packet, which is \"0000\" (four zeros) This\nseparates the metadata from the ref advertisement\n- After that you can write your message, beginning with \"ERR \" (note\nthe trailing space there). The \"ERR \" tells Git what you're writing\nisn't a ref, it's an error. I'd recommend appending a newline (and add\n1 more to your length for it), because when Git echoes your error\nmessage it doesn't seem to do that\n\nI'm not sure whether there's a document that describes all of this; I\nfound it by digging into the Git source code (you can find the \"ERR\"\nhandling in connect.c, get_remote_heads). This may be exploiting the\nprotocol, I'll leave that to someone more knowledgeable on how they\n_intended_ this all to be used, but it works for us.\n\nA full example looks something like this: \"0036#\nservice=git-receive-pack0000ERR This is a test\\n\"\n\nHope this helps,\nBryan Turner\n\n>\n> It does look that way. Does the following patch work for you?\n>\n> -- >8 --\n> Subject: [PATCH] http: provide server's error message on RPC failure\n>\n> The server might provide a custom error message that is useful to the user.\n> Provide this message to the user if HTTP RPC fails.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> remote-curl.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/remote-curl.c b/remote-curl.c\n> index 52c2d96..5984d35 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -426,8 +426,8 @@ static int run_slot(struct active_request_slot *slot,\n> err = run_one_slot(slot, results);\n>\n> if (err != HTTP_OK && err != HTTP_REAUTH) {\n> - error(\"RPC failed; result=%d, HTTP code = %ld\",\n> - results->curl_result, results->http_code);\n> + error(\"RPC failed; result=%d, HTTP code = %ld (%s)\",\n> + results->curl_result, results->http_code, curl_errorstr);\n> }\n>\n> return err;\n> -- >8 --\n>\n> --\n> brian m. carlson / brian with sandals: Houston, Texas, US\n> +1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\n> OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"},{"id":"242149","messageId":"1400492420.2595.60.camel@centaur.cmartin.tk","threadId":"36646","inReplyTo":"CAGyf7-EoSgTxZzReFArOgcrBaARv7fRiZTMPZX+Loy9dec23aQ@mail.gmail.com","subject":"Re: Returning error message from custom smart http server","fromName":"Carlos Martín Nieto","fromEmail":"cmn@elego.de","sentAt":"2014-05-19T09:40:20Z","receivedAt":"2014-05-19T09:40:20Z","isPatch":false,"sender":{"key":"cmn@elego.de","avatar":"https://avatars.githubusercontent.com/u/335443?v=4"},"body":"On Mon, 2014-05-19 at 18:12 +1000, Bryan Turner wrote:\n> On Sat, May 17, 2014 at 9:01 AM, brian m. carlson\n> wrote:\n> > On Tue, May 13, 2014 at 09:39:59AM +0200, \"Ákos, Tajti\" wrote:\n> >> Dear List,\n> >>\n> >> we implemented our own git smart http server to be able to check permissions\n> >> and other thing before pushes. It works fine, however, the error messages we\n> >> generate on the server side are not displayed by the command line client. On\n> >> the server we generate error messages like this:\n> >>\n> >> response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);\n> >> response.getWriter().write(msg);\n> >>\n> >> On the command line we get this:\n> >>\n> >> Total 0 (delta 0), reused 0 (delta 0)\n> >> POST git-receive-pack (290 bytes)\n> >> efrror: RPC failed; result=22, HTTP code = 401\n> >> atal: The remote end hung up unexpectedly\n> >> fatal: The remote end hung up unexpectedly\n> >>\n> >> The server message is completely missing. Is there a solution for this?\n> \n> You should not need a patched git; the wire protocol itself has a\n> mechanism for sending \"smart\" error messages. It's not particularly\n> _obvious_, but it's there.\n> \n> For starters, to return an error message, your status must be 200 OK.\n> You can't return any other status code or Git will interpret your\n> error as some form of _HTTP_ error rather than a _git_ error.\n> \n> In the smart protocol the client sends a service to the server as a\n> query parameter, like \"?service=git-receive-pack\". For such a request,\n> you need to:\n> - Set the content type to \"application/x--advertisement\"\n> (e.g. \"application/x-git-receive-pack-advertisement\") (Not all command\n> line Git versions require this, but JGit does)\n> - Set the status code as 200 OK\n> - Write back a payload where the first 4 bytes are the hex-encoded\n> length of the text (where \"FFFF\" is max length for a single packet).\n> Note that the 4 bytes for the size are _part_ of that length, so if\n> you're writing \"Test\" the length is 8, not 4\n> - After the size, you write \"# service=\" (e.g. \"#\n> service=git-receive-pack\"; note the space after the #) This is the\n> metadata. For an error, you don't really have much to say.\n> - After that, an empty packet, which is \"0000\" (four zeros) This\n> separates the metadata from the ref advertisement\n> - After that you can write your message, beginning with \"ERR \" (note\n> the trailing space there). The \"ERR \" tells Git what you're writing\n> isn't a ref, it's an error. I'd recommend appending a newline (and add\n> 1 more to your length for it), because when Git echoes your error\n> message it doesn't seem to do that\n> \n> I'm not sure whether there's a document that describes all of this; I\n> found it by digging into the Git source code (you can find the \"ERR\"\n> handling in connect.c, get_remote_heads). This may be exploiting the\n> protocol, I'll leave that to someone more knowledgeable on how they\n> _intended_ this all to be used, but it works for us.\n> \n> A full example looks something like this: \"0036#\n> service=git-receive-pack0000ERR This is a test\\n\"\n\nThis is indeed documented, namely in \n\n Documentation/technical/pack-protocol.txt\n\nI guess it could do with an example, but your usage seems correct. There\nare two different places where things could go wrong, either in HTTP,\nsuch as authentication, or in the Git part of the request. If you return\nan HTTP 404, then all you're telling the client is that you couldn't\nfind what it asked for, but that could mean either the\nreceice-pack/upload-pack program or the repository itself. If something\nwent wrong at the Git level, whether it's a resource problem in the\nserver or simply that the repo doesn't exist, then ERR is the right\nthing to use.\n\nParticularly, we can't rely on the HTTP 404 response being anything\nmeaningful, as it could simply be the host's default 404 page, and you\ndon't want html flying through your terminal.\n\nCheers,\n cmn\n"},{"id":"242164","messageId":"20140519150942.GC20289@sigill.intra.peff.net","threadId":"36646","inReplyTo":"CAGyf7-EoSgTxZzReFArOgcrBaARv7fRiZTMPZX+Loy9dec23aQ@mail.gmail.com","subject":"Re: Returning error message from custom smart http server","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-05-19T15:09:42Z","receivedAt":"2014-05-19T15:09:42Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 19, 2014 at 06:12:10PM +1000, Bryan Turner wrote:\n\n> For starters, to return an error message, your status must be 200 OK.\n> You can't return any other status code or Git will interpret your\n> error as some form of _HTTP_ error rather than a _git_ error.\n\nAs of git v1.8.3, git will show text/plain content sent along with a\na non-200 HTTP code.\n\nHowever, it does this _only_ for the initial refs fetch (along with\nseveral other error-reporting niceties, including specifically handling\nHTTP 401s). The thinking was that the interesting smart-http errors\nhappen on that initial contact (e.g., failure to login, access denied,\netc). Errors at the HTTP level that happen later during POST requests\nmean that the server is misconfigured or broken somehow, and should be\nrare. That's the theory anyway.\n\nIn the original poster's example, it looks like the server is rejecting\nthe push with an HTTP 401 during the POST call, after the initial ref\nadvertisement. This is non-ideal, because it means the client may have\ngone to significant work to generate the packfile. It should instead\nreject it as soon as it sees a request for\n\".../info/refs?service=git-receive-pack\". Current git clients will\nprompt for errors, and will also show the text/plain content.\n\n> - Set the content type to \"application/x--advertisement\"\n> (e.g. \"application/x-git-receive-pack-advertisement\") (Not all command\n> line Git versions require this, but JGit does)\n\nA side note, but command-line Git cares about the content-type since\nv1.8.1.5.\n\n> [...how git's ERR lines work...]\n\nYour description seemed accurate from my brief read. Sending ERR lines\ngoes back much further. However, for a 401, I think they really want to\nsend the HTTP code (and at the right time), so that the client can\nrecognize this, gather credentials from the user, and try again.\n\n-Peff\n"}]}