git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] merge_blobs: use strbuf instead of manually-sized mmfile_t

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Feb 16, 2016, 05:09 UTC
Message-ID
<20160216050915.GA5765@flurp.local>
In-Reply-To
<20160216011258.GA11961@sigill.intra.peff.net>
On Mon, Feb 15, 2016 at 08:12:58PM -0500, Jeff King wrote:
Show 64 quoted lines
> On Mon, Feb 15, 2016 at 10:39:39PM +0100, Stefan Frühwirth wrote:
> > in one specific circumstance, git-merge-tree exits with a segfault caused by
> > "*** Error in `git': malloc(): memory corruption (fast)":
> > 
> > There is a test case[1] kindly provided by chrisrossi, which he crafted
> > after I discovered the problem[2] in the context of Pylons/acidfs.
> 
> -- >8 --
> Subject: merge_blobs: use strbuf instead of manually-sized mmfile_t
> 
> [...]
> It does so by calling xdiff with XDIFF_EMIT_COMMON, and
> stores the result in a buffer that is as big as the smaller
> of "ours" and "theirs".
> 
> In theory, this is right; we cannot have more common content
> than is in the smaller of the two blobs. But in practice,
> xdiff may give us more: if neither file ends in a newline,
> we get the "\nNo newline at end of file" marker.
> [...]
> 
> Reported-by: Stefan Frühwirth <stefan.fruehwirth@uni-graz.at>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> diff --git a/merge-blobs.c b/merge-blobs.c
> @@ -51,19 +51,16 @@ static void *three_way_filemerge(const char *path, mmfile_t *base, mmfile_t *our
>  static int common_outf(void *priv_, mmbuffer_t *mb, int nbuf)
>  {
>  	int i;
> -	mmfile_t *dst = priv_;
> +	struct strbuf *dst = priv_;
>  
> -	for (i = 0; i < nbuf; i++) {
> -		memcpy(dst->ptr + dst->size, mb[i].ptr, mb[i].size);
> -		dst->size += mb[i].size;
> -	}
> +	for (i = 0; i < nbuf; i++)
> +		strbuf_add(dst, mb[i].ptr, mb[i].size);
>  	return 0;
>  }
>  
>  static int generate_common_file(mmfile_t *res, mmfile_t *f1, mmfile_t *f2)
>  {
> -	unsigned long size = f1->size < f2->size ? f1->size : f2->size;
> -	void *ptr = xmalloc(size);
> +	struct strbuf out = STRBUF_INIT;
>  	xpparam_t xpp;
>  	xdemitconf_t xecfg;
>  	xdemitcb_t ecb;
> @@ -75,11 +72,15 @@ static int generate_common_file(mmfile_t *res, mmfile_t *f1, mmfile_t *f2)
>  	xecfg.flags = XDL_EMIT_COMMON;
>  	ecb.outf = common_outf;
>  
> -	res->ptr = ptr;
> -	res->size = 0;
> +	ecb.priv = &out;
> +	if (xdi_diff(f1, f2, &xpp, &xecfg, &ecb) < 0) {
> +		strbuf_release(&out);
> +		return -1;
> +	}
>  
> -	ecb.priv = res;
> -	return xdi_diff(f1, f2, &xpp, &xecfg, &ecb);
> +	res->size = out.len; /* avoid long/size_t pointer mismatch below */

It took a minute or two for me to realize that "mismatch below" was talking about the second argument to strbuf_detach(). I tried rewriting the comment to mention the second argument explicitly, but couldn't come up with one sufficiently succinct. Oh well.

> +	res->ptr = strbuf_detach(&out, NULL);
> +	return 0;
>  }

My reviewed-by may not be worth much since this code is new to me too, but this patch looks "obviously correct" to me, so:

    Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>

Perhaps squash in the following test which I adapted from the reproduction recipe provided by Chris Rossi[1]?

[1] https://gist.github.com/chrisrossi/f09c8bed70b364f9f12e
--- 8< ---
diff --git a/t/t4300-merge-tree.sh b/t/t4300-merge-tree.sh
index 9015e47..1f2aa74 100755
--- a/t/t4300-merge-tree.sh
+++ b/t/t4300-merge-tree.sh
@@ -352,4 +352,22 @@ test_expect_success 'turn tree to file' '
 	test_cmp expect actual
 '
 
+test_expect_success "don't underallocate result buffer" '
+	test_when_finished "git checkout master" &&
+	git checkout --orphan some &&
+	git rm -rf . &&
+	printf "b\n" >a &&
+	git add a &&
+	git commit -m "first commit" &&
+	printf "\na" >b &&
+	git add b &&
+	git commit -m "second commit, first branch" &&
+	git checkout @^ &&
+	git checkout -b other &&
+	printf "a" >b &&
+	git add b &&
+	git commit -m "second commit, second branch" &&
+	git merge-tree @^ some other
+'
+
 test_done
--- 8< ---
Previous: Jeff KingNext: Jeff King
Message 4 of 10 in “malloc memory corruption on merge-tree with leading newline”
  1. Stefan FrühwirthFeb 15, 2016
  2. Stefan FrühwirthFeb 15, 2016
  3. merge_blobs: use strbuf instead of manually-sized mmfile_tJeff King, Feb 16, 2016
  4. Eric SunshineFeb 16, 2016
  5. Jeff KingFeb 16, 2016
  6. Stefan FrühwirthFeb 16, 2016
  7. Jeff KingFeb 16, 2016
  8. Stefan FrühwirthFeb 19, 2016
  9. Junio C HamanoFeb 16, 2016
  10. Stefan FrühwirthFeb 19, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.