# [PATCH 2/2] Use xmmap_gently instead of xmmap in use_pack 5 messages from 2019-05-16 to 2019-05-16. Participants: Mike Hommey, Jeff King. Thread: https://gitlist.dev/t/51107 ## Mike Hommey, 2019-05-16 00:37 Subject: [PATCH 2/2] Use xmmap_gently instead of xmmap in use_pack Message-ID: <20190516003736.25544-2-mh@glandium.org> URL: https://gitlist.dev/e/20190516003736.25544-2-mh%40glandium.org In-Reply-To: <20190516003736.25544-1-mh@glandium.org> ``` use_pack has its own error message on mmap error, but it can't be reached when using xmmap, which dies with its own error. Signed-off-by: Mike Hommey --- packfile.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packfile.c b/packfile.c index 16bcb75262..6a66b605e9 100644 --- a/packfile.c +++ b/packfile.c @@ -630,7 +630,7 @@ unsigned char *use_pack(struct packed_git *p, while (packed_git_limit < pack_mapped && unuse_one_window(p)) ; /* nothing */ - win->base = xmmap(NULL, win->len, + win->base = xmmap_gently(NULL, win->len, PROT_READ, MAP_PRIVATE, p->pack_fd, win->offset); if (win->base == MAP_FAILED) -- 2.21.0.dirty ``` ## Mike Hommey, 2019-05-16 00:37 Subject: [PATCH 1/2] dup() the input fd for fast-import used for remote helpers Message-ID: <20190516003736.25544-1-mh@glandium.org> URL: https://gitlist.dev/e/20190516003736.25544-1-mh%40glandium.org In-Reply-To: <20190516003130.racexnofe7cxra3t@glandium.org> ``` When a remote helper exposes the "import" capability, stdout of the helper is sent to stdin of a new fast-import process. This is done by setting the corresponding child_process's in field to the value of the out field of the helper child_process. The child_process API is defined to close the file descriptors it's given when calling start_command. This means when start_command is called for the fast-import process, its input fd (the output fd of the helper), is closed. But when the transport helper is later destroyed, in disconnect_helper, its input and output are closed, which means close() is called with an invalid fd (since it was already closed as per above). Or worse, with a valid fd owned by something else (since fd numbers can be reused). Signed-off-by: Mike Hommey --- transport-helper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/transport-helper.c b/transport-helper.c index 1f52c95fd8..29787b749e 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -421,7 +421,7 @@ static int get_importer(struct transport *transport, struct child_process *fasti struct helper_data *data = transport->data; int cat_blob_fd, code; child_process_init(fastimport); - fastimport->in = helper->out; + fastimport->in = xdup(helper->out); argv_array_push(&fastimport->args, "fast-import"); argv_array_push(&fastimport->args, debug ? "--stats" : "--quiet"); -- 2.21.0.dirty ``` ## Mike Hommey, 2019-05-16 00:31 Subject: Re: Closing fds twice when using remote helpers Message-ID: <20190516003130.racexnofe7cxra3t@glandium.org> URL: https://gitlist.dev/e/20190516003130.racexnofe7cxra3t%40glandium.org In-Reply-To: <20190515105609.sucfjvuumeyyrmjb@glandium.org> ``` On Wed, May 15, 2019 at 07:56:09PM +0900, Mike Hommey wrote: > Hi, > > I started getting a weird error message during some test case involving > git-cinnabar, which is a remote-helper to access mercurial > repositories. > > The error says: > fatal: mmap failed: Bad file descriptor > > ... which was not making much sense. Some debugging later, and it turns > out this is what happens: > > - start_command is called for fast-import > - start_command is called again for git-remote-hg, passing the > fast_import->out as cmd->in. My mistake, it's the opposite: start_command for fast-import is called with the out of git-remote-hg as in. Anyways, patches incoming. Mike ``` ## Jeff King, 2019-05-16 03:28 Subject: Re: [PATCH 1/2] dup() the input fd for fast-import used for remote helpers Message-ID: <20190516032838.GC4596@sigill.intra.peff.net> URL: https://gitlist.dev/e/20190516032838.GC4596%40sigill.intra.peff.net In-Reply-To: <20190516003736.25544-1-mh@glandium.org> ``` On Thu, May 16, 2019 at 09:37:35AM +0900, Mike Hommey wrote: > When a remote helper exposes the "import" capability, stdout of the > helper is sent to stdin of a new fast-import process. This is done by > setting the corresponding child_process's in field to the value of the > out field of the helper child_process. > > The child_process API is defined to close the file descriptors it's > given when calling start_command. This means when start_command is > called for the fast-import process, its input fd (the output fd of the > helper), is closed. > > But when the transport helper is later destroyed, in disconnect_helper, > its input and output are closed, which means close() is called with > an invalid fd (since it was already closed as per above). Or worse, with > a valid fd owned by something else (since fd numbers can be reused). I think this strategy is OK, as explained in my other email. -Peff ``` ## Jeff King, 2019-05-16 03:34 Subject: Re: [PATCH 2/2] Use xmmap_gently instead of xmmap in use_pack Message-ID: <20190516033422.GD4596@sigill.intra.peff.net> URL: https://gitlist.dev/e/20190516033422.GD4596%40sigill.intra.peff.net In-Reply-To: <20190516003736.25544-2-mh@glandium.org> ``` On Thu, May 16, 2019 at 09:37:36AM +0900, Mike Hommey wrote: > use_pack has its own error message on mmap error, but it can't be > reached when using xmmap, which dies with its own error. Makes sense. Amusingly this xmmap comes from c4712e4553 (Replace mmap with xmmap, better handling MAP_FAILED., 2006-12-24), which specifically moved everything to xmmap for its error handling, and dropped all of the MAP_FAILED checks. But it forgot to remove this one. So we could also go the other way, and just remove the unreachable check. I do think the error from use_pack() is better though, so I like the direction you went. -Peff ```