threads / patch / 51107

patch, 2 partsUse xmmap_gently instead of xmmap in use_pack

Subject: [PATCH 2/2] Use xmmap_gently instead of xmmap in use_pack

## tl;dr

5 messages between May 16, 2019 and May 16, 2019. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

Mike Hommey· May 16, 2019, 00:31 UTC · lore

Re: Closing fds twice when using remote helpers

On Wed, May 15, 2019 at 07:56:09PM +0900, Mike Hommey wrote:
Show 15 quoted lines
> Hi,
> 
> I started getting a weird error message during some test case involving
> git-cinnabar, which is a remote-helper to access mercurial
> repositories.
> 
> The error says:
> fatal: mmap failed: Bad file descriptor
> 
> ... which was not making much sense. Some debugging later, and it turns
> out this is what happens:
> 
> - start_command is called for fast-import
> - start_command is called again for git-remote-hg, passing the
>   fast_import->out as cmd->in.

My mistake, it's the opposite: start_command for fast-import is called with the out of git-remote-hg as in.

Anyways, patches incoming.
Mike
Mike Hommey· May 16, 2019, 00:37 UTC · re: Mike Hommey · lore

[PATCH 1/2] dup() the input fd for fast-import used for remote helpers

When a remote helper exposes the "import" capability, stdout of the helper is sent to stdin of a new fast-import process. This is done by setting the corresponding child_process's in field to the value of the out field of the helper child_process.

The child_process API is defined to close the file descriptors it's given when calling start_command. This means when start_command is called for the fast-import process, its input fd (the output fd of the helper), is closed.

But when the transport helper is later destroyed, in disconnect_helper, its input and output are closed, which means close() is called with an invalid fd (since it was already closed as per above). Or worse, with a valid fd owned by something else (since fd numbers can be reused).

Signed-off-by: Mike Hommey <mh@glandium.org>
---
 transport-helper.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to transport-helper.c +1 −1
diff --git a/transport-helper.c b/transport-helper.c
index 1f52c95fd8..29787b749e 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -421,7 +421,7 @@ static int get_importer(struct transport *transport, struct child_process *fasti
 	struct helper_data *data = transport->data;
 	int cat_blob_fd, code;
 	child_process_init(fastimport);
-	fastimport->in = helper->out;
+	fastimport->in = xdup(helper->out);
 	argv_array_push(&fastimport->args, "fast-import");
 	argv_array_push(&fastimport->args, debug ? "--stats" : "--quiet");
 

-- 
2.21.0.dirty
Mike Hommey· May 16, 2019, 00:37 UTC · re: Mike Hommey · lore

use_pack has its own error message on mmap error, but it can't be reached when using xmmap, which dies with its own error.

Signed-off-by: Mike Hommey <mh@glandium.org>
---
 packfile.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to packfile.c +1 −1
diff --git a/packfile.c b/packfile.c
index 16bcb75262..6a66b605e9 100644
--- a/packfile.c
+++ b/packfile.c
@@ -630,7 +630,7 @@ unsigned char *use_pack(struct packed_git *p,
 			while (packed_git_limit < pack_mapped
 				&& unuse_one_window(p))
 				; /* nothing */
-			win->base = xmmap(NULL, win->len,
+			win->base = xmmap_gently(NULL, win->len,
 				PROT_READ, MAP_PRIVATE,
 				p->pack_fd, win->offset);
 			if (win->base == MAP_FAILED)

-- 
2.21.0.dirty
Jeff King· May 16, 2019, 03:34 UTC · re: Mike Hommey · lore

Re: [PATCH 2/2] Use xmmap_gently instead of xmmap in use_pack

On Thu, May 16, 2019 at 09:37:36AM +0900, Mike Hommey wrote:
> use_pack has its own error message on mmap error, but it can't be
> reached when using xmmap, which dies with its own error.

Makes sense. Amusingly this xmmap comes from c4712e4553 (Replace mmap with xmmap, better handling MAP_FAILED., 2006-12-24), which specifically moved everything to xmmap for its error handling, and dropped all of the MAP_FAILED checks. But it forgot to remove this one.

So we could also go the other way, and just remove the unreachable check. I do think the error from use_pack() is better though, so I like the direction you went.

-Peff
Jeff King· May 16, 2019, 03:28 UTC · re: Mike Hommey · lore

Re: [PATCH 1/2] dup() the input fd for fast-import used for remote helpers

On Thu, May 16, 2019 at 09:37:35AM +0900, Mike Hommey wrote:
Show 14 quoted lines
> When a remote helper exposes the "import" capability, stdout of the
> helper is sent to stdin of a new fast-import process. This is done by
> setting the corresponding child_process's in field to the value of the
> out field of the helper child_process.
> 
> The child_process API is defined to close the file descriptors it's
> given when calling start_command. This means when start_command is
> called for the fast-import process, its input fd (the output fd of the
> helper), is closed.
> 
> But when the transport helper is later destroyed, in disconnect_helper,
> its input and output are closed, which means close() is called with
> an invalid fd (since it was already closed as per above). Or worse, with
> a valid fd owned by something else (since fd numbers can be reused).
I think this strategy is OK, as explained in my other email.
-Peff

← back to recent threads