threads / patch / 42743

patch, 9 partsUse merge_recursive() directly in the builtin am

Subject: [PATCH 0/9] Use merge_recursive() directly in the builtin am

## tl;dr

262 messages between Jun 29, 2016 and Aug 10, 2016. Diffs are folded; open one to read it.

replies: 261people: 15as markdown or json

Johannes Schindelin· Jun 29, 2016, 11:36 UTC · lore

This is the long-awaited re-roll of the attempt to avoid spawning merge-recursive from the builtin am and use merge_recursive() directly instead.

As indicated in the message of the final commit, the performance improvement is modest, if noticable.

The *real* reason for the reroll is that I need a libified recursive merge to accelerate the interactive rebase by teaching the sequencer to do rebase -i's grunt work.

In other words, this is one of those 13 (now 14) patch series leading up to a faster interactive rebase.

It did take quite a while to go through the code "with a fine-toothed comb", and it did turn up a few surprises.

For example, the recursive merge calls remove_file() a couple of times but actually does not always care about the return value, and rightfully so. When updating a working tree, for example, where a file was replaced by a directory, the code may create the directory first (implicitly deleting the file) and only then attempt to remove the file, failing because it is a directory already.

One might argue that this logic is flawed, then, and I would agree. Right now my focus is on rebase -i and I want to avoid getting side-tracked by the recursive merge logic. So the clean-up will need to wait for another day (or week).

We also need to be extra careful to retain backwards-compatibility. The test script t6022-merge-rename.sh, for example, verifies that `git pull` exits with status 128 in case of a fatal error. To that end, we need to make sure that fatal errors are handled by existing (builtin) users via exit(128) (or die(), which calls exit(128) at the end). New users (such as a builtin helper doing rebase -i's grunt work) may want to print some helpful advice what happened and how to get out of this mess before erroring out.

In contrast to the original attempt at libifying merge_recursive() (as part of fixing a regression in the builtin am which wanted to print some advice, but could not, because the merge machinery die()d before it could), I no longer use the "gently" flag. Better to get it right to begin with: fatal errors are indicated by a negative return value. No dying without proper cause.

In an earlier iteration of this patch series which was not sent to the mailing list, I used the special return vale -128 to indicate "real fatal errors". This turned out to be unnecessary: returning -1 always, to indicate that the operation could not complete successfully, is the appropriate way to handle all errors.

As this patch series touches rather important code, I would really appreciate thorough reviews, with a particular focus on what regressions this patch series might introduce.

Johannes Schindelin (8):
  Report bugs consistently
  merge-recursive: clarify code in was_tracked()
  Prepare the builtins for a libified merge_recursive()
  merge_recursive: abort properly upon errors
  merge-recursive: avoid returning a wholesale struct
  merge-recursive: allow write_tree_from_memory() to error out
  merge-recursive: handle return values indicating errors
  merge-recursive: switch to returning errors instead of dying
Junio C Hamano (1):
  am: make a direct call to merge_recursive
 builtin/am.c           |  27 ++--
 builtin/checkout.c     |   4 +-
 builtin/ls-files.c     |   3 +-
 builtin/merge.c        |   4 +
 builtin/update-index.c |   2 +-
 grep.c                 |   8 +-
 imap-send.c            |   2 +-
 merge-recursive.c      | 397 ++++++++++++++++++++++++++++++-------------------
 sequencer.c            |   4 +
 sha1_file.c            |   4 +-
 trailer.c              |   2 +-
 transport.c            |   2 +-
 wt-status.c            |   4 +-
 13 files changed, 279 insertions(+), 184 deletions(-)
Published-As: https://github.com/dscho/git/releases/tag/am-3-merge-recursive-direct-v1
-- 
2.9.0.268.gcabc8b0

base-commit: cf4c2cfe52be5bd973a4838f73a35d3959ce2f43
Johannes Schindelin· Jun 29, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH 1/9] Report bugs consistently

The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:".

As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/ls-files.c     |  3 ++-
 builtin/update-index.c |  2 +-
 grep.c                 |  8 ++++----
 imap-send.c            |  2 +-
 merge-recursive.c      | 13 ++++++-------
 sha1_file.c            |  4 ++--
 trailer.c              |  2 +-
 transport.c            |  2 +-
 wt-status.c            |  4 ++--
 9 files changed, 20 insertions(+), 20 deletions(-)
Show changes to 9 files +20 −20

builtin/ls-files.c, builtin/update-index.c, grep.c, imap-send.c, merge-recursive.c, sha1_file.c, trailer.c, transport.c, wt-status.c

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f02e3d2..00ea91a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
 				 */
 				pos = cache_name_pos(ent->name, ent->len);
 				if (0 <= pos)
-					die("bug in show-killed-files");
+					die("BUG: killed-file %.*s not found",
+						ent->len, ent->name);
 				pos = -pos - 1;
 				while (pos < active_nr &&
 				       ce_stage(active_cache[pos]))
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 6cdfd5f..ba04b19 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1146,7 +1146,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		report(_("Untracked cache enabled for '%s'"), get_git_work_tree());
 		break;
 	default:
-		die("Bug: bad untracked_cache value: %d", untracked_cache);
+		die("BUG: bad untracked_cache value: %d", untracked_cache);
 	}
 
 	if (active_cache_changed) {
diff --git a/grep.c b/grep.c
index 1e15b62..f1ca0a0 100644
--- a/grep.c
+++ b/grep.c
@@ -643,10 +643,10 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 	for (p = opt->header_list; p; p = p->next) {
 		if (p->token != GREP_PATTERN_HEAD)
-			die("bug: a non-header pattern in grep header list.");
+			die("BUG: a non-header pattern in grep header list.");
 		if (p->field < GREP_HEADER_FIELD_MIN ||
 		    GREP_HEADER_FIELD_MAX <= p->field)
-			die("bug: unknown header field %d", p->field);
+			die("BUG: unknown header field %d", p->field);
 		compile_regexp(p, opt);
 	}
 
@@ -659,7 +659,7 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 		h = compile_pattern_atom(&pp);
 		if (!h || pp != p->next)
-			die("bug: malformed header expr");
+			die("BUG: malformed header expr");
 		if (!header_group[p->field]) {
 			header_group[p->field] = h;
 			continue;
@@ -1464,7 +1464,7 @@ static int grep_source_1(struct grep_opt *opt, struct grep_source *gs, int colle
 		case GREP_BINARY_TEXT:
 			break;
 		default:
-			die("bug: unknown binary handling mode");
+			die("BUG: unknown binary handling mode");
 		}
 	}
 
diff --git a/imap-send.c b/imap-send.c
index 938c691..cd39805 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -511,7 +511,7 @@ static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 
 	va_start(va, fmt);
 	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
-		die("Fatal: buffer too small. Please report a bug.");
+		die("BUG: buffer too small (%d < %d)", ret, blen);
 	va_end(va);
 	return ret;
 }
diff --git a/merge-recursive.c b/merge-recursive.c
index 65cb5d6..98f4632 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -259,7 +259,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
-		die("Bug in merge-recursive.c");
+		die("BUG: unmerged index entries in merge-recursive.c");
 	}
 
 	if (!active_cache_tree)
@@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 
 			if (!sha_eq(a->sha1, b->sha1))
 				result.clean = 0;
-		} else {
-			die(_("unsupported object type in the tree"));
-		}
+		} else
+			die(_("BUG: unsupported object type in the tree"));
 	}
 
 	return result;
@@ -1343,7 +1342,7 @@ static int process_renames(struct merge_options *o,
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1_src != ren2_src");
+				die("BUG: ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
@@ -1377,7 +1376,7 @@ static int process_renames(struct merge_options *o,
 			ren2 = lookup->util;
 			ren2_dst = ren2->pair->two->path;
 			if (strcmp(ren1_dst, ren2_dst) != 0)
-				die("ren1_dst != ren2_dst");
+				die("BUG: ren1_dst != ren2_dst");
 
 			clean_merge = 0;
 			ren2->processed = 1;
@@ -1853,7 +1852,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("Unprocessed path??? %s"),
+				die(_("BUG: unprocessed path??? %s"),
 				    entries->items[i].string);
 		}
 
diff --git a/sha1_file.c b/sha1_file.c
index d5e1121..aa7006c 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -795,7 +795,7 @@ void close_all_packs(void)
 
 	for (p = packed_git; p; p = p->next)
 		if (p->do_not_close)
-			die("BUG! Want to close pack marked 'do-not-close'");
+			die("BUG: Want to close pack marked 'do-not-close'");
 		else
 			close_pack(p);
 }
@@ -2330,7 +2330,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 	case OBJ_OFS_DELTA:
 	case OBJ_REF_DELTA:
 		if (data)
-			die("BUG in unpack_entry: left loop at a valid delta");
+			die("BUG: unpack_entry: left loop at a valid delta");
 		break;
 	case OBJ_COMMIT:
 	case OBJ_TREE:
diff --git a/trailer.c b/trailer.c
index 8e48a5c..c6ea9ac 100644
--- a/trailer.c
+++ b/trailer.c
@@ -562,7 +562,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
-		die("internal bug in trailer.c");
+		die("BUG: trailer.c: unhandled type %d", type);
 	}
 	return 0;
 }
diff --git a/transport.c b/transport.c
index 095e61f..52bf997 100644
--- a/transport.c
+++ b/transport.c
@@ -563,7 +563,7 @@ void transport_take_over(struct transport *transport,
 	struct git_transport_data *data;
 
 	if (!transport->smart_options)
-		die("Bug detected: Taking over transport requires non-NULL "
+		die("BUG: taking over transport requires non-NULL "
 		    "smart_options field.");
 
 	data = xcalloc(1, sizeof(*data));
diff --git a/wt-status.c b/wt-status.c
index 4ce4e35..311ae7c 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die(_("bug: unhandled unmerged status %x"), stagemask);
+		die(_("BUG: unhandled unmerged status %x"), stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die(_("bug: unhandled diff status %c"), status);
+		die(_("BUG: unhandled diff status %c"), status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.268.gcabc8b0
Johannes Schindelin· Jun 29, 2016, 15:11 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

Hi,
On Wed, 29 Jun 2016, Johannes Schindelin wrote:
Show 13 quoted lines
> diff --git a/imap-send.c b/imap-send.c
> index 938c691..cd39805 100644
> --- a/imap-send.c
> +++ b/imap-send.c
> @@ -511,7 +511,7 @@ static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
>  
>  	va_start(va, fmt);
>  	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
> -		die("Fatal: buffer too small. Please report a bug.");
> +		die("BUG: buffer too small (%d < %d)", ret, blen);
>  	va_end(va);
>  	return ret;
>  }

Oy vey. Travis CI found a bug here, thanks to clang's quite smart checking. This here fixup is needed (will make that part of v2, but wait for comments):

-- snipsnap --
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: [PATCH] fixup! Report bugs consistently
---
 imap-send.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to imap-send.c +1 −1
diff --git a/imap-send.c b/imap-send.c
index cd39805..369f72a 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -506,7 +506,7 @@ static char *next_arg(char **s)
 
 static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 {
-	int ret;
+	int ret = -1;
 	va_list va;
 
 	va_start(va, fmt);
-- 
2.9.0.270.g810e421
Eric Sunshine· Jun 29, 2016, 18:12 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

On Wed, Jun 29, 2016 at 7:36 AM, Johannes Schindelin <johannes.schindelin@gmx.de> wrote:

Show 9 quoted lines
> The vast majority of error messages in Git's source code which report a
> bug use the convention to prefix the message with "BUG:".
> [...]
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/merge-recursive.c b/merge-recursive.c
> @@ -1853,7 +1852,7 @@ int merge_trees(struct merge_options *o,
> -                               die(_("Unprocessed path??? %s"),
> +                               die(_("BUG: unprocessed path??? %s"),

This and others downcase the first word (which is consistent with modern practice)...

> diff --git a/sha1_file.c b/sha1_file.c
> @@ -795,7 +795,7 @@ void close_all_packs(void)
> -                       die("BUG! Want to close pack marked 'do-not-close'");
> +                       die("BUG: Want to close pack marked 'do-not-close'");
...but this one neglects to.
Johannes Schindelin· Jun 30, 2016, 08:41 UTC · re: Eric Sunshine · lore

Re: [PATCH 1/9] Report bugs consistently

Hi Eric,
On Wed, 29 Jun 2016, Eric Sunshine wrote:
Show 21 quoted lines
> On Wed, Jun 29, 2016 at 7:36 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > The vast majority of error messages in Git's source code which report a
> > bug use the convention to prefix the message with "BUG:".
> > [...]
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > @@ -1853,7 +1852,7 @@ int merge_trees(struct merge_options *o,
> > -                               die(_("Unprocessed path??? %s"),
> > +                               die(_("BUG: unprocessed path??? %s"),
> 
> This and others downcase the first word (which is consistent with
> modern practice)...
> 
> > diff --git a/sha1_file.c b/sha1_file.c
> > @@ -795,7 +795,7 @@ void close_all_packs(void)
> > -                       die("BUG! Want to close pack marked 'do-not-close'");
> > +                       die("BUG: Want to close pack marked 'do-not-close'");
> 
> ...but this one neglects to.
Thanks! Will be fixed as part of v2.

Ciao, Dscho

Junio C Hamano· Jun 29, 2016, 20:50 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> The vast majority of error messages in Git's source code which report a
> bug use the convention to prefix the message with "BUG:".
Good thing to do.

But if we were to review and apply a 200+ line patch, I wonder if we want to go one step further to allow us to write

    BUG("killed-file %s not found", name);
instead.
Johannes Schindelin· Jun 30, 2016, 08:42 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/9] Report bugs consistently

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 13 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The vast majority of error messages in Git's source code which report a
> > bug use the convention to prefix the message with "BUG:".
> 
> Good thing to do.
> 
> But if we were to review and apply a 200+ line patch, I wonder if we
> want to go one step further to allow us to write
> 
>     BUG("killed-file %s not found", name);
> 
> instead.

If the idea is to make it easier to find, I would wager a guess that 'die("BUG:' would be just as good a search term. Even better, I think, because 'BUG' would also match comments.

Ciao, Dscho

Jeff King· Jun 30, 2016, 09:23 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

On Thu, Jun 30, 2016 at 10:42:37AM +0200, Johannes Schindelin wrote:
Show 15 quoted lines
> > > The vast majority of error messages in Git's source code which report a
> > > bug use the convention to prefix the message with "BUG:".
> > 
> > Good thing to do.
> > 
> > But if we were to review and apply a 200+ line patch, I wonder if we
> > want to go one step further to allow us to write
> > 
> >     BUG("killed-file %s not found", name);
> > 
> > instead.
> 
> If the idea is to make it easier to find, I would wager a guess that
> 'die("BUG:' would be just as good a search term. Even better, I think,
> because 'BUG' would also match comments.

I have been tempted to switch to BUG(), because it would make it easy to call abort() and get a coredump (and therefore a stack trace). On the other hand:

  - we could always trigger such behavior in die() by looking for "BUG:" in
    the output string. :)
  - it's also sometimes useful to get a stack trace from a regular
    non-bug die(). So maybe something optional like:
      if (git_env_bool("GIT_ABORT_ON_DIE", 0))
              abort();
    would be helpful (since you have to turn it on ahead of time, you
    could also just run the program under gdb, of course; however, I
    sometimes find that it's hard to get gdb where you want it because
    git spawns so many sub-programs. Or maybe I just need to get better
    at using gdb's child-following options).

The other thing BUG() would get us is that we could turn it into a macro (on systems with vararg macros) and report things like __FILE__ and __LINE__. In practice, though our BUG messages are unique enough that there is no problem finding the source.

-Peff
Johannes Schindelin· Jul 1, 2016, 13:51 UTC · re: Jeff King · lore

Re: [PATCH 1/9] Report bugs consistently

Hi Peff,
On Thu, 30 Jun 2016, Jeff King wrote:
Show 20 quoted lines
> On Thu, Jun 30, 2016 at 10:42:37AM +0200, Johannes Schindelin wrote:
> 
> > > > The vast majority of error messages in Git's source code which report a
> > > > bug use the convention to prefix the message with "BUG:".
> > > 
> > > Good thing to do.
> > > 
> > > But if we were to review and apply a 200+ line patch, I wonder if we
> > > want to go one step further to allow us to write
> > > 
> > >     BUG("killed-file %s not found", name);
> > > 
> > > instead.
> > 
> > If the idea is to make it easier to find, I would wager a guess that
> > 'die("BUG:' would be just as good a search term. Even better, I think,
> > because 'BUG' would also match comments.
> 
> I have been tempted to switch to BUG(), because it would make it easy to
> call abort() and get a coredump (and therefore a stack trace).

Please keep in mind that abort() does not produce stackdumps with MinGW. So at least Windows developers would not be better off.

Show 16 quoted lines
> On the other hand:
> 
>   - we could always trigger such behavior in die() by looking for "BUG:" in
>     the output string. :)
> 
>   - it's also sometimes useful to get a stack trace from a regular
>     non-bug die(). So maybe something optional like:
> 
>       if (git_env_bool("GIT_ABORT_ON_DIE", 0))
>               abort();
> 
>     would be helpful (since you have to turn it on ahead of time, you
>     could also just run the program under gdb, of course; however, I
>     sometimes find that it's hard to get gdb where you want it because
>     git spawns so many sub-programs. Or maybe I just need to get better
>     at using gdb's child-following options).

Heh. I still find myself using that good old trick where I set a variable, loop while it is set and print out the pid, waiting for a debugger to attach and re-set that variable.

> The other thing BUG() would get us is that we could turn it into a macro
> (on systems with vararg macros) and report things like __FILE__ and
> __LINE__.  In practice, though our BUG messages are unique enough that
> there is no problem finding the source.

That would be very nice *also* for error() messages. But I guess we cannot have it, vararg macros being a feature rather than a standard.

Ciao, Dscho

Jeff King· Jul 1, 2016, 18:16 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

On Fri, Jul 01, 2016 at 03:51:33PM +0200, Johannes Schindelin wrote:
Show 9 quoted lines
> > > If the idea is to make it easier to find, I would wager a guess that
> > > 'die("BUG:' would be just as good a search term. Even better, I think,
> > > because 'BUG' would also match comments.
> > 
> > I have been tempted to switch to BUG(), because it would make it easy to
> > call abort() and get a coredump (and therefore a stack trace).
> 
> Please keep in mind that abort() does not produce stackdumps with MinGW.
> So at least Windows developers would not be better off.

It doesn't in Linux either. But you can use the core file to get a backtrace. Do you not get cores at all in Windows? Yuck. :)

> Heh. I still find myself using that good old trick where I set a variable,
> loop while it is set and print out the pid, waiting for a debugger to
> attach and re-set that variable.
That's a new one to me. Gross. :)

Usually I can coax git into running "gdbserver ... git foo" instead of git foo" via config. But sometimes the values are hard-coded (for instance, the way we call pack-objects, which I recently made a bit more flexible).

Show 7 quoted lines
> > The other thing BUG() would get us is that we could turn it into a macro
> > (on systems with vararg macros) and report things like __FILE__ and
> > __LINE__.  In practice, though our BUG messages are unique enough that
> > there is no problem finding the source.
> 
> That would be very nice *also* for error() messages. But I guess we cannot
> have it, vararg macros being a feature rather than a standard.

It is a standard, it's just C99, which we've been rather slow to embrace.

However we have prior art in the trace code. If your system supports variadic macros, then we define the macros to pass the file/line info. Otherwise, we skip the macros, and those names point directly to functions.

So it would not be too hard to implement. I've never bothered, though, because in my experience the interesting thing is almost never "on what line was error() called", but rather "what line called the function that called error()". I.e., the stack trace.

I looked a while ago at trying to auto-generate stack traces in git using something like GNU's backtrace (obviously not portable, but it's a start). My conclusion was that the simplest thing is to just generate a core and run "gdb -ex bt" on it.

-Peff
Johannes Sixt· Jun 30, 2016, 05:38 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

Am 29.06.2016 um 13:36 schrieb Johannes Schindelin:
Show 9 quoted lines
> @@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
>
>   			if (!sha_eq(a->sha1, b->sha1))
>   				result.clean = 0;
> -		} else {
> -			die(_("unsupported object type in the tree"));
> -		}
> +		} else
> +			die(_("BUG: unsupported object type in the tree"));

Would it perhaps make sense to remove the _() markup (here and a few more instances in this patch)? It's simpler for developers to find the code location when a "BUG:" is reported untranslated.

-- Hannes
Johannes Schindelin· Jun 30, 2016, 08:45 UTC · re: Johannes Sixt · lore

Re: [PATCH 1/9] Report bugs consistently

Hi Hannes,
On Thu, 30 Jun 2016, Johannes Sixt wrote:
Show 15 quoted lines
> Am 29.06.2016 um 13:36 schrieb Johannes Schindelin:
> > @@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct
> > merge_options *o,
> >
> >      if (!sha_eq(a->sha1, b->sha1))
> >   				result.clean = 0;
> > -		} else {
> > -			die(_("unsupported object type in the tree"));
> > -		}
> > +		} else
> > +			die(_("BUG: unsupported object type in the tree"));
> 
> Would it perhaps make sense to remove the _() markup (here and a few more
> instances in this patch)? It's simpler for developers to find the code
> location when a "BUG:" is reported untranslated.

I would agree, but the purpose of this patch was not to fix that, but to fix the inconsistency of the message.

Maybe as an add-on patch, with *all* 'die(_("BUG:' instances converted? That would be even more outside the purview of my patch series than touching the bug reports outside merge-recursive.c, though.

Ciao, Dscho

Duy Nguyen· Jul 2, 2016, 05:11 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

On Wed, Jun 29, 2016 at 1:36 PM, Johannes Schindelin <johannes.schindelin@gmx.de> wrote:

Show 9 quoted lines
> @@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
>
>                         if (!sha_eq(a->sha1, b->sha1))
>                                 result.clean = 0;
> -               } else {
> -                       die(_("unsupported object type in the tree"));
> -               }
> +               } else
> +                       die(_("BUG: unsupported object type in the tree"));

As a message targeting developers, we do not need to mark this for translation. There are a couple other _() in this patch that should be removed as well.

-- 
Duy
Johannes Schindelin· Jul 2, 2016, 07:25 UTC · re: Duy Nguyen · lore

Re: [PATCH 1/9] Report bugs consistently

Hi Duy,
On Sat, 2 Jul 2016, Duy Nguyen wrote:
Show 15 quoted lines
> On Wed, Jun 29, 2016 at 1:36 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > @@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
> >
> >                         if (!sha_eq(a->sha1, b->sha1))
> >                                 result.clean = 0;
> > -               } else {
> > -                       die(_("unsupported object type in the tree"));
> > -               }
> > +               } else
> > +                       die(_("BUG: unsupported object type in the tree"));
> 
> As a message targeting developers, we do not need to mark this for
> translation. There are a couple other _() in this patch that should be
> removed as well.
Yes, Hannes already pointed that out.

My answer is the same: it is not the purpose of this patch series to fix this, and therefore it retains the previous behavior.

Ciao, Dscho

Duy Nguyen· Jul 2, 2016, 08:01 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/9] Report bugs consistently

On Sat, Jul 2, 2016 at 9:25 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 21 quoted lines
> Hi Duy,
>
> On Sat, 2 Jul 2016, Duy Nguyen wrote:
>
>> On Wed, Jun 29, 2016 at 1:36 PM, Johannes Schindelin
>> <johannes.schindelin@gmx.de> wrote:
>> > @@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
>> >
>> >                         if (!sha_eq(a->sha1, b->sha1))
>> >                                 result.clean = 0;
>> > -               } else {
>> > -                       die(_("unsupported object type in the tree"));
>> > -               }
>> > +               } else
>> > +                       die(_("BUG: unsupported object type in the tree"));
>>
>> As a message targeting developers, we do not need to mark this for
>> translation. There are a couple other _() in this patch that should be
>> removed as well.
>
> Yes, Hannes already pointed that out.
Ah.. sorry I didn't read the whole thread.
> My answer is the same: it is not the purpose of this patch series to fix
> this, and therefore it retains the previous behavior.

You're changing the string and adding more work to translators. So either leave the string untouched, or drop _().

-- 
Duy
Johannes Schindelin· Jul 5, 2016, 11:32 UTC · re: Duy Nguyen · lore

Re: [PATCH 1/9] Report bugs consistently

Hi Duy,
On Sat, 2 Jul 2016, Duy Nguyen wrote:
> You're changing the string and adding more work to translators. So
> either leave the string untouched, or drop _().

Thanks. I addressed that concern in v2. Could you please now have a look at the parts of the patch series which could possibly regress Git's functionality? I am quite a bit more interested in having extra pairs of eyes look over those.

Thanks, Dscho

Johannes Schindelin· Jun 29, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH 2/9] merge-recursive: clarify code in was_tracked()

It can be puzzling to see that was_tracked() tries to match an index entry by name even if cache_name_pos() returned a negative value. Let's clarify that cache_name_pos() implicitly looks for stage 0, while we are also okay with finding other stages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 1 +
 1 file changed, 1 insertion(+)
Show changes to merge-recursive.c +1 −0
diff --git a/merge-recursive.c b/merge-recursive.c
index 98f4632..bcb53f0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -658,6 +658,7 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
+	/* cache_name_pos() looks for stage == 0, so pos may be < 0 */
 	if (pos < 0)
 		pos = -1 - pos;
 	while (pos < active_nr &&
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 18:35 UTC · re: Johannes Schindelin · lore

Re: [PATCH 2/9] merge-recursive: clarify code in was_tracked()

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 19 quoted lines
> It can be puzzling to see that was_tracked() tries to match an index
> entry by name even if cache_name_pos() returned a negative value. Let's
> clarify that cache_name_pos() implicitly looks for stage 0, while we are
> also okay with finding other stages.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  merge-recursive.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 98f4632..bcb53f0 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -658,6 +658,7 @@ static int was_tracked(const char *path)
>  {
>  	int pos = cache_name_pos(path, strlen(path));
>  
> +	/* cache_name_pos() looks for stage == 0, so pos may be < 0 */

It returns >= if found at stage #0, or a negative (counting from -1) to indicate where the path would be inserted if it were to be added at stage #0.

The new comment does not explain how "pos may be < 0" leads to "hence pos = -1 - pos is the right thing to do here". It is misleading and we probably are better off without.

>  	if (pos < 0)
>  		pos = -1 - pos;
>  	while (pos < active_nr &&
Johannes Schindelin· Jul 1, 2016, 09:23 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/9] merge-recursive: clarify code in was_tracked()

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 25 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > It can be puzzling to see that was_tracked() tries to match an index
> > entry by name even if cache_name_pos() returned a negative value. Let's
> > clarify that cache_name_pos() implicitly looks for stage 0, while we are
> > also okay with finding other stages.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  merge-recursive.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index 98f4632..bcb53f0 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -658,6 +658,7 @@ static int was_tracked(const char *path)
> >  {
> >  	int pos = cache_name_pos(path, strlen(path));
> >  
> > +	/* cache_name_pos() looks for stage == 0, so pos may be < 0 */
> 
> It returns >= if found at stage #0, or a negative (counting from -1)
> to indicate where the path would be inserted if it were to be added
> at stage #0.

Right. Please note that nothing in the function call to cache_name_pos() specifies that we are looking for any particular stage. You are probably too intimate with the implementation details, so naturally you assume that cache_name_pos() looks for a precise match *with stage 0*. I am not that familiar with that part of the implementation, so I *was* puzzled.

> The new comment does not explain how "pos may be < 0" leads to
> "hence pos = -1 - pos is the right thing to do here".  It is
> misleading and we probably are better off without.

I agree that the comment is not very good currently. But I disagree that we are better off without any comment here.

So maybe a comment is not really enough here. In case the entry is in stage 0, we will have an exact match, so there is no need to go to the lengths of a while() loop.

I would like to propose this diff instead (it is larger, but with a net savings of one line):

-- snipsnap --
Show changes to merge-recursive.c +9 −10
diff --git a/merge-recursive.c b/merge-recursive.c
index d5a593c..0eda51a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -658,24 +658,22 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
-	if (pos < 0)
-		pos = -1 - pos;
-	while (pos < active_nr &&
-	       !strcmp(path, active_cache[pos]->name)) {
+	if (pos >= 0)
+		return pos < active_nr;
+	/*
+	 * cache_name_pos() looks for stage == 0, even if we did not ask for
+	 * it. Let's look for stage == 2 now.
+	 */
+	for (pos = -1 - pos; pos < active_nr &&
+	     !strcmp(path, active_cache[pos]->name); pos++)
 		/*
 		 * If stage #0, it is definitely tracked.
 		 * If it has stage #2 then it was tracked
 		 * before this merge started.  All other
 		 * cases the path was not tracked.
 		 */
-		switch (ce_stage(active_cache[pos])) {
-		case 0:
-		case 2:
+		if (ce_stage(active_cache[pos]) == 2)
 			return 1;
-		}
-		pos++;
-	}
 	return 0;
 }
 
Junio C Hamano· Jul 1, 2016, 15:31 UTC · re: Johannes Schindelin · lore

Re: [PATCH 2/9] merge-recursive: clarify code in was_tracked()

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> I agree that the comment is not very good currently. But I disagree that
> we are better off without any comment here.

I meant we are better off without your particular version of comment which is misleading. I am all for a better comment to help those who are new to the codepath.

Show 22 quoted lines
> I would like to propose this diff instead (it is larger, but with a net
> savings of one line):
>
> -- snipsnap --
> diff --git a/merge-recursive.c b/merge-recursive.c
> index d5a593c..0eda51a 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -658,24 +658,22 @@ static int was_tracked(const char *path)
>  {
>  	int pos = cache_name_pos(path, strlen(path));
>  
> -	if (pos < 0)
> -		pos = -1 - pos;
> -	while (pos < active_nr &&
> -	       !strcmp(path, active_cache[pos]->name)) {
> +	if (pos >= 0)
> +		return pos < active_nr;
> +	/*
> +	 * cache_name_pos() looks for stage == 0, even if we did not ask for
> +	 * it. Let's look for stage == 2 now.
> +	 */

I think this keeps the same phrasing from the original that makes the comment misleading. It "looks for stage == 0" is not the whole story but only half. It looks for a place to insert the path at stage #0" is. Your half is used by the "if (0 <= pos)" you split out into a separate statement above already, and the untold half is needed to explain why this loop is correct.

It returns the place to insert stage #0 entry, so if you are looking for stage #1 or higher, you only have to loop while the path matches, because the entries are sorted by <path, stage>.

And with that understanding, there is no strong reason to special case "ah, we found stage #0 entry" at all.

Show 19 quoted lines
> +	for (pos = -1 - pos; pos < active_nr &&
> +	     !strcmp(path, active_cache[pos]->name); pos++)
>  		/*
>  		 * If stage #0, it is definitely tracked.
>  		 * If it has stage #2 then it was tracked
>  		 * before this merge started.  All other
>  		 * cases the path was not tracked.
>  		 */
> -		switch (ce_stage(active_cache[pos])) {
> -		case 0:
> -		case 2:
> +		if (ce_stage(active_cache[pos]) == 2)
>  			return 1;
> -		}
> -		pos++;
> -	}
>  	return 0;
>  }
>  
Johannes Schindelin· Jul 2, 2016, 07:20 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/9] merge-recursive: clarify code in was_tracked()

Hi Junio,
On Fri, 1 Jul 2016, Junio C Hamano wrote:
Show 28 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > I would like to propose this diff instead (it is larger, but with a net
> > savings of one line):
> >
> > -- snipsnap --
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index d5a593c..0eda51a 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -658,24 +658,22 @@ static int was_tracked(const char *path)
> >  {
> >  	int pos = cache_name_pos(path, strlen(path));
> >  
> > -	if (pos < 0)
> > -		pos = -1 - pos;
> > -	while (pos < active_nr &&
> > -	       !strcmp(path, active_cache[pos]->name)) {
> > +	if (pos >= 0)
> > +		return pos < active_nr;
> > +	/*
> > +	 * cache_name_pos() looks for stage == 0, even if we did not ask for
> > +	 * it. Let's look for stage == 2 now.
> > +	 */
> 
> I think this keeps the same phrasing from the original that makes
> the comment misleading.  It "looks for stage == 0" is not the whole
> story but only half.

Yes, it is the relevant part of the story to explain why we're not done when pos < 0.

To understand why we're not done yet, the crucial point is *not* that the return value encodes the insert position. The crucial point is that despite asking for an index entry matching a specific name, we might not find one, *even if there is one*. And the reason is that cache_name_pos() does not quite do what the name suggests.

I am sorry to disagree with you here: I really find it important to document this potential misunderstanding.

> It looks for a place to insert the path at stage #0" is.  Your half is
> used by the "if (0 <= pos)" you split out into a separate statement
> above already, and the untold half is needed to explain why this loop is
> correct.

True. I did not bother to document that part. Because even if I was puzzled by the logic handling a negative return value, the assignment "pos = -1 - pos" made it very clear to me what was happening. I am not *that* easily puzzled.

Show 6 quoted lines
> It returns the place to insert stage #0 entry, so if you are looking
> for stage #1 or higher, you only have to loop while the path
> matches, because the entries are sorted by <path, stage>.
> 
> And with that understanding, there is no strong reason to special
> case "ah, we found stage #0 entry" at all.

There is one, and I mentioned it. In the common case (i.e. we found an entry right away because it is in stage 0), the loop unnecessarily compares the name *again*: index_name_pos() already performed that comparison and if it returned a non-negative value, we know that that comparison was successful.

I find the combination of clarification, code reduction, and separation between conflated cases (stage 0 does not need that loop at all) compelling enough to state that my patch is an improvement overall. We can discuss about wording and what details to mention in the comments, of course.

Ciao, Dscho

Junio C Hamano· Jul 6, 2016, 15:30 UTC · re: Johannes Schindelin · lore

Re: [PATCH 2/9] merge-recursive: clarify code in was_tracked()

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> To understand why we're not done yet, the crucial point is *not* that the
> return value encodes the insert position. The crucial point is that
> despite asking for an index entry matching a specific name, we might not
> find one, *even if there is one*.

I've been wondering why you keep saying "even though we didn't ask, we look for stage#0", and now I see why. The cache_pos() interface *is* about finding the stage#0 entry for the given path.

When it finds none, it indicates where a stage#0 entry of that path would be inserted, which by the sort-order would give us where higher stage entries for the path would be found (if there is any). There is no parameter for you to tell it to find stage#2, and "even though we didn't ask" is showing (and being the source of) the confusion.

And I did not want a misleading comment to spread the confusion; that is why I was reacting strongly.

As you pointed out, we can return early without falling into the generic "we are still looking at the same path" codepath when we find thestage#0 entry, so I wouldn't mind doing something like the following.

static int was_tracked(const char *path)
{
	int pos = cache_name_pos(path, strlen(path));
        if (0 <= pos)
	        /* we have been tracking this path */
        	return 1;
	/*
         * Look for an unmerged entry for the path,
         * specifically stage #2, which would indicate
         * that "our" side before the merge started
         * had the path tracked (and resulted in a conflict).
         */
	for (pos = -1 - pos;
             pos < active_nr && !strcmp(path, active_cache[pos]->name);
	     pos++)
		if (ce_stage(active_cache[pos]) == 2)
			return 1;
	return 0;
}
Johannes Schindelin· Jul 7, 2016, 11:17 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/9] merge-recursive: clarify code in was_tracked()

Hi Junio,
On Wed, 6 Jul 2016, Junio C Hamano wrote:
Show 10 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > To understand why we're not done yet, the crucial point is *not* that the
> > return value encodes the insert position. The crucial point is that
> > despite asking for an index entry matching a specific name, we might not
> > find one, *even if there is one*.
> 
> I've been wondering why you keep saying "even though we didn't ask,
> we look for stage#0", and now I see why.  The cache_pos() interface
> *is* about finding the stage#0 entry for the given path.
Good that this is clarified now.
Show 6 quoted lines
> [...]
> As you pointed out, we can return early without falling into the
> generic "we are still looking at the same path" codepath when we
> find thestage#0 entry, so I wouldn't mind doing something like the
> following.
> [...]

As this is essentially what I wrote with some minor touch-ups, I just replaced my version with yours.

Will resend, Dscho

Johannes Schindelin· Jun 29, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH 4/9] merge_recursive: abort properly upon errors

There are a couple of places where return values indicating errors are ignored. Let's teach them manners.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +8 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index bcb53f0..c4ece96 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1944,8 +1944,9 @@ int merge_recursive(struct merge_options *o,
 		saved_b2 = o->branch2;
 		o->branch1 = "Temporary merge branch 1";
 		o->branch2 = "Temporary merge branch 2";
-		merge_recursive(o, merged_common_ancestors, iter->item,
-				NULL, &merged_common_ancestors);
+		if (merge_recursive(o, merged_common_ancestors, iter->item,
+				NULL, &merged_common_ancestors) < 0)
+			return -1;
 		o->branch1 = saved_b1;
 		o->branch2 = saved_b2;
 		o->call_depth--;
@@ -1961,6 +1962,8 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	if (clean < 0)
+		return clean;
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
@@ -2017,6 +2020,9 @@ int merge_recursive_generic(struct merge_options *o,
 	hold_locked_index(lock, 1);
 	clean = merge_recursive(o, head_commit, next_commit, ca,
 			result);
+	if (clean < 0)
+		return clean;
+
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 		return error(_("Unable to write index."));
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 20:08 UTC · re: Johannes Schindelin · lore

Re: [PATCH 4/9] merge_recursive: abort properly upon errors

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 24 quoted lines
> There are a couple of places where return values indicating errors
> are ignored. Let's teach them manners.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  merge-recursive.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index bcb53f0..c4ece96 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1944,8 +1944,9 @@ int merge_recursive(struct merge_options *o,
>  		saved_b2 = o->branch2;
>  		o->branch1 = "Temporary merge branch 1";
>  		o->branch2 = "Temporary merge branch 2";
> -		merge_recursive(o, merged_common_ancestors, iter->item,
> -				NULL, &merged_common_ancestors);
> +		if (merge_recursive(o, merged_common_ancestors, iter->item,
> +				NULL, &merged_common_ancestors) < 0)
> +			return -1;
>  		o->branch1 = saved_b1;
>  		o->branch2 = saved_b2;
>  		o->call_depth--;

OK, this early return (and others in this patch) are only for negative (i.e. error) cases, and "attempted a merge, resulted in conflicts" cases are handled as before.

Which is good.

I wonder if o->branch[12] need to be restored, though. The only sensible thing the caller can do is to punt, but would it expect to be able to do some error reporting based on these fields, e.g. printf("merge of %s and %s failed", o->branch1, o->branch2) or something? In addition to that kind of "state restoration", we may need to watch out for resource leaks, but I think there is none at least these three early returns.

Thanks.
Show 19 quoted lines
> @@ -1961,6 +1962,8 @@ int merge_recursive(struct merge_options *o,
>  	o->ancestor = "merged common ancestors";
>  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
>  			    &mrtree);
> +	if (clean < 0)
> +		return clean;
>  
>  	if (o->call_depth) {
>  		*result = make_virtual_commit(mrtree, "merged tree");
> @@ -2017,6 +2020,9 @@ int merge_recursive_generic(struct merge_options *o,
>  	hold_locked_index(lock, 1);
>  	clean = merge_recursive(o, head_commit, next_commit, ca,
>  			result);
> +	if (clean < 0)
> +		return clean;
> +
>  	if (active_cache_changed &&
>  	    write_locked_index(&the_index, lock, COMMIT_LOCK))
>  		return error(_("Unable to write index."));
Johannes Schindelin· Jul 1, 2016, 10:16 UTC · re: Junio C Hamano · lore

Re: [PATCH 4/9] merge_recursive: abort properly upon errors

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 40 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > There are a couple of places where return values indicating errors
> > are ignored. Let's teach them manners.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  merge-recursive.c | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index bcb53f0..c4ece96 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -1944,8 +1944,9 @@ int merge_recursive(struct merge_options *o,
> >  		saved_b2 = o->branch2;
> >  		o->branch1 = "Temporary merge branch 1";
> >  		o->branch2 = "Temporary merge branch 2";
> > -		merge_recursive(o, merged_common_ancestors, iter->item,
> > -				NULL, &merged_common_ancestors);
> > +		if (merge_recursive(o, merged_common_ancestors, iter->item,
> > +				NULL, &merged_common_ancestors) < 0)
> > +			return -1;
> >  		o->branch1 = saved_b1;
> >  		o->branch2 = saved_b2;
> >  		o->call_depth--;
> 
> OK, this early return (and others in this patch) are only for
> negative (i.e. error) cases, and "attempted a merge, resulted in
> conflicts" cases are handled as before.
> 
> Which is good.
> 
> I wonder if o->branch[12] need to be restored, though.  The only
> sensible thing the caller can do is to punt, but would it expect to
> be able to do some error reporting based on these fields, e.g.
> printf("merge of %s and %s failed", o->branch1, o->branch2) or
> something?  In addition to that kind of "state restoration", we may
> need to watch out for resource leaks, but I think there is none at
> least these three early returns.

I do not think that the caller can do anything sensible with *o after we return an error: it means that we failed *somewhere* in that operation, but does not say exactly where.

Ciao, Dscho

Junio C Hamano· Jul 1, 2016, 15:56 UTC · re: Johannes Schindelin · lore

Re: [PATCH 4/9] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 17 quoted lines
>> >  		saved_b2 = o->branch2;
>> >  		o->branch1 = "Temporary merge branch 1";
>> >  		o->branch2 = "Temporary merge branch 2";
>> > -		merge_recursive(o, merged_common_ancestors, iter->item,
>> > -				NULL, &merged_common_ancestors);
>> > +		if (merge_recursive(o, merged_common_ancestors, iter->item,
>> > +				NULL, &merged_common_ancestors) < 0)
>> > +			return -1;
>> >  		o->branch1 = saved_b1;
>> >  		o->branch2 = saved_b2;
>> >  		o->call_depth--;
>> 
>> I wonder if o->branch[12] need to be restored, though.  The only
>> sensible thing the caller can do is to punt,...
>
> I do not think that the caller can do anything sensible with *o after we
> return an error...
That is totally up to what this patch does, isn't it?

By deliberately keeping o->branch[12] to point at the temporary names and not restoring, this patch declares "the caller cannot do anything sensible with *o". If it restores, the caller still can. Even with this step as-is, the caller can tell at which recursion level the merge failed by looking at o->call_depth, for example.

I do not think the current set of callers, and a new one you will be introducing, would prefer to be able to do something with *o after the failure return from this function, so in that sense, I do not care deeply either way.

But if the patch is making a policy decision "*o is undefined upon error return from this function", it would help people who want to build on top of this codebase to add that to the comment before the function, wouldn't it?

Johannes Schindelin· Jul 2, 2016, 11:30 UTC · re: Junio C Hamano · lore

Re: [PATCH 4/9] merge_recursive: abort properly upon errors

Hi Junio,
On Fri, 1 Jul 2016, Junio C Hamano wrote:
Show 21 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> >  		saved_b2 = o->branch2;
> >> >  		o->branch1 = "Temporary merge branch 1";
> >> >  		o->branch2 = "Temporary merge branch 2";
> >> > -		merge_recursive(o, merged_common_ancestors, iter->item,
> >> > -				NULL, &merged_common_ancestors);
> >> > +		if (merge_recursive(o, merged_common_ancestors, iter->item,
> >> > +				NULL, &merged_common_ancestors) < 0)
> >> > +			return -1;
> >> >  		o->branch1 = saved_b1;
> >> >  		o->branch2 = saved_b2;
> >> >  		o->call_depth--;
> >> 
> >> I wonder if o->branch[12] need to be restored, though.  The only
> >> sensible thing the caller can do is to punt,...
> >
> > I do not think that the caller can do anything sensible with *o after we
> > return an error...
> 
> That is totally up to what this patch does, isn't it?

No, not really. We do not really know *where* in the recursive merge the failure happened.

All we know is that it happened while trying to merge the temporary branches.

> By deliberately keeping o->branch[12] to point at the temporary
> names and not restoring, this patch declares "the caller cannot do
> anything sensible with *o".  If it restores, the caller still can.

But would restoring the branch names not give the false impression that the error occurred during a different phase of the recursive merge?

> Even with this step as-is, the caller can tell at which recursion
> level the merge failed by looking at o->call_depth, for example.
Well, not really:
1) please note the o->call_depth-- that is *also* skipped in case of an
   error after my patch, and
2) what use is the recursion level if an arbitrary number of merges could
   happen at the same recursion depth?

In short, I do not think that resetting those values in *o could bring any benefit to the caller, short of introducing those fine-grained error values I mentioned elsewhere.

Ciao, Dscho

Johannes Schindelin· Jun 29, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH 3/9] Prepare the builtins for a libified merge_recursive()

A truly libified function does not die() just for fun. As such, the recursive merge will convert all die() calls to return -1 instead in the next commits, giving the caller a chance at least to print some helpful message.

Let's prepare the builtins for this fatal error condition, even if we do not really do more than imitating the previous die()'s exit(128): this is what callers of e.g. `git merge` have come to expect.

Note that the callers of the sequencer (revert and cherry-pick) already fail fast even for the return value -1; The only difference is that they now get a chance to say "<command> failed".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 4 +++-
 builtin/merge.c    | 4 ++++
 sequencer.c        | 4 ++++
 3 files changed, 11 insertions(+), 1 deletion(-)
Show changes to 3 files +11 −1

builtin/checkout.c, builtin/merge.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index c3486bd..14312f7 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			o.ancestor = old->name;
 			o.branch1 = new->name;
 			o.branch2 = "local";
-			merge_trees(&o, new->commit->tree, work,
+			ret = merge_trees(&o, new->commit->tree, work,
 				old->commit->tree, &result);
+			if (ret < 0)
+				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
 			if (ret)
diff --git a/builtin/merge.c b/builtin/merge.c
index b555a1b..133b853 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -682,6 +682,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
+		if (clean < 0)
+			exit(128);
 		if (active_cache_changed &&
 		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 			die (_("unable to write %s"), get_index_file());
@@ -1550,6 +1552,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
 		ret = try_merge_strategy(use_strategies[i]->name,
 					 common, remoteheads,
 					 head_commit, head_arg);
+		if (ret < 0)
+			exit(128);
 		if (!option_commit && !ret) {
 			merge_was_ok = 1;
 			/*
diff --git a/sequencer.c b/sequencer.c
index c6362d6..13b794a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (clean < 0)
+		return clean;
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
@@ -561,6 +563,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
+		if (res < 0)
+			return res;
 		write_message(&msgbuf, git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 18:56 UTC · re: Johannes Schindelin · lore

Re: [PATCH 3/9] Prepare the builtins for a libified merge_recursive()

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> A truly libified function does not die() just for fun.

The sentence is wasting bits. After all, a helper function in run-once-and-exit program does not die() just for fun, either.

So what's more interesting to know for the readers?
Show 8 quoted lines
> As such, the
> recursive merge will convert all die() calls to return -1 instead in the
> next commits, giving the caller a chance at least to print some helpful
> message.
>
> Let's prepare the builtins for this fatal error condition, even if we do
> not really do more than imitating the previous die()'s exit(128): this is
> what callers of e.g. `git merge` have come to expect.
One thing missing is your design decision and justification.

For example, the above explanation hints that the original code in this hunk expected merge_trees() to die() with some useful message when there is an error condition, but merge_trees() is going to be improved not to die() itself and return -1 instead to signal an error, so that the caller can react more flexibly, and this is a step to prepare for the version of merge_trees() that no longer dies.

Show 5 quoted lines
> -			merge_trees(&o, new->commit->tree, work,
> +			ret = merge_trees(&o, new->commit->tree, work,
>  				old->commit->tree, &result);
> +			if (ret < 0)
> +				exit(128);

The postimage of the patch tells us that the caller is now responsible for exiting with status 128, but neither the proposed log message nor the above hunk tells us where the message the original code must have given to the end user from die() inside merge_trees(). The updated caller just exits, so a natural guess is that the calls to die() have been changed to fprintf(stderr) with the patch.

But that does not mesh very well with the stated objective of the patch. The callers want flexibility to do their own error handling, including giving their own message, so letting merge_trees() to still write the same message to the standard error stream would not work well for them. A caller may want to do merge_trees() just to see if it succeeds, without wanting to give _any_ indication of that is happening to the user, because it has an alternate/fallback code if merge_trees() fails, for example (analogy: "am -3" first tries a straight patch application before fallking back to 3-way merge; it may not want to show the error from the first attempt).

The reader _can_ guess that this step ignores the error-message issue, and improving it later (or keep ignoring that issue) might be OK in the context of this patch series, but it is necessary to be upfront to the readers what the design choices were and which one of those choices the proposed patch adopted as its design for them to be able to evaluate the patch series correctly.

One design alternative we've seen used in our code to help callers who want no default messages is to pass struct strbuf &err down the callchain and collect the default messages without emitting them directly to the standard error stream when an error is diagnosed. I do not know if that pattern is applicable or should be applied to this codepath.

Show 71 quoted lines
> Note that the callers of the sequencer (revert and cherry-pick) already
> fail fast even for the return value -1; The only difference is that they
> now get a chance to say "<command> failed".
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  builtin/checkout.c | 4 +++-
>  builtin/merge.c    | 4 ++++
>  sequencer.c        | 4 ++++
>  3 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/builtin/checkout.c b/builtin/checkout.c
> index c3486bd..14312f7 100644
> --- a/builtin/checkout.c
> +++ b/builtin/checkout.c
> @@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
>  			o.ancestor = old->name;
>  			o.branch1 = new->name;
>  			o.branch2 = "local";
> -			merge_trees(&o, new->commit->tree, work,
> +			ret = merge_trees(&o, new->commit->tree, work,
>  				old->commit->tree, &result);
> +			if (ret < 0)
> +				exit(128);
>  			ret = reset_tree(new->commit->tree, opts, 0,
>  					 writeout_error);
>  			if (ret)
> diff --git a/builtin/merge.c b/builtin/merge.c
> index b555a1b..133b853 100644
> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -682,6 +682,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
>  		hold_locked_index(&lock, 1);
>  		clean = merge_recursive(&o, head,
>  				remoteheads->item, reversed, &result);
> +		if (clean < 0)
> +			exit(128);
>  		if (active_cache_changed &&
>  		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
>  			die (_("unable to write %s"), get_index_file());
> @@ -1550,6 +1552,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
>  		ret = try_merge_strategy(use_strategies[i]->name,
>  					 common, remoteheads,
>  					 head_commit, head_arg);
> +		if (ret < 0)
> +			exit(128);
>  		if (!option_commit && !ret) {
>  			merge_was_ok = 1;
>  			/*
> diff --git a/sequencer.c b/sequencer.c
> index c6362d6..13b794a 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
>  	clean = merge_trees(&o,
>  			    head_tree,
>  			    next_tree, base_tree, &result);
> +	if (clean < 0)
> +		return clean;
>  
>  	if (active_cache_changed &&
>  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
> @@ -561,6 +563,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
>  	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
>  		res = do_recursive_merge(base, next, base_label, next_label,
>  					 head, &msgbuf, opts);
> +		if (res < 0)
> +			return res;
>  		write_message(&msgbuf, git_path_merge_msg());
>  	} else {
>  		struct commit_list *common = NULL;
Johannes Schindelin· Jul 1, 2016, 10:14 UTC · re: Junio C Hamano · lore

Re: [PATCH 3/9] Prepare the builtins for a libified merge_recursive()

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 6 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > A truly libified function does not die() just for fun.
> 
> The sentence is wasting bits.  After all, a helper function in
> run-once-and-exit program does not die() just for fun, either.

Given that I had a hard time when reviewing Christian's apply patches to drive home the message that it is not okay for library functions to call die() or exit(), I happen to disagree with your notion that this sentence is wasting bits.

This sentence does not so much target *you* personally as audience, but the occasional reader of the log who wonders: "Why don't we just call die()? We would not have to worry about passing back the return value through all those long call chains..."

Show 17 quoted lines
> > As such, the recursive merge will convert all die() calls to return -1
> > instead in the next commits, giving the caller a chance at least to
> > print some helpful message.
> >
> > Let's prepare the builtins for this fatal error condition, even if we do
> > not really do more than imitating the previous die()'s exit(128): this is
> > what callers of e.g. `git merge` have come to expect.
> 
> One thing missing is your design decision and justification.
> 
> For example, the above explanation hints that the original code in
> this hunk expected merge_trees() to die() with some useful message
> when there is an error condition, but merge_trees() is going to be
> improved not to die() itself and return -1 instead to signal an
> error, so that the caller can react more flexibly, and this is a
> step to prepare for the version of merge_trees() that no longer
> dies.

Okay, I will replace the "As such..." paragraph with a modified version of your paraphrased explanation.

Show 13 quoted lines
> > -			merge_trees(&o, new->commit->tree, work,
> > +			ret = merge_trees(&o, new->commit->tree, work,
> >  				old->commit->tree, &result);
> > +			if (ret < 0)
> > +				exit(128);
> 
> The postimage of the patch tells us that the caller is now
> responsible for exiting with status 128, but neither the proposed
> log message nor the above hunk tells us where the message the
> original code must have given to the end user from die() inside
> merge_trees().  The updated caller just exits, so a natural guess is
> that the calls to die() have been changed to fprintf(stderr) with
> the patch.

Even more natural is it to guess that the code will call error(), just like we do almost everywhere else.

But you are right, I do not have to leave the reader guessing. Better to err on the side of being slightly verbose than to be so concise that nobody understands what I mean.

Show 17 quoted lines
> But that does not mesh very well with the stated objective of the
> patch.  The callers want flexibility to do their own error handling,
> including giving their own message, so letting merge_trees() to
> still write the same message to the standard error stream would not
> work well for them.  A caller may want to do merge_trees() just to
> see if it succeeds, without wanting to give _any_ indication of that
> is happening to the user, because it has an alternate/fallback code
> if merge_trees() fails, for example (analogy: "am -3" first tries a
> straight patch application before fallking back to 3-way merge; it
> may not want to show the error from the first attempt).
> 
> The reader _can_ guess that this step ignores the error-message
> issue, and improving it later (or keep ignoring that issue) might be
> OK in the context of this patch series, but it is necessary to be
> upfront to the readers what the design choices were and which one of
> those choices the proposed patch adopted as its design for them to
> be able to evaluate the patch series correctly.

To be honest, I did not even think about the error message issue because my primary concern is to teach the sequencer to perform rebase -i's grunt work. And while we usually suppress the output of the commands in rebase -i, we do show them in case of errors.

It will make things more complex, unfortunately, even if it will be straight-forward: there is already a strbuf and a flag in struct merge_options to collect output. The merge_options are simply not passed through to all of the previously die()ing functions yet.

I won't have time to get this implemented this week, unfortunately. So please do not expect the next iteration of this patch series before next week.

I could imagine that you wanted even more fine-grained control, where we have a range of return values indicating different error conditions. However, I already spent two weeks' worth of work to get this far, and would like to defer that task to the developer who will actually need these fine-grained return values (if we ever will need them).

Ciao, Dscho

P.S.: For the future, would you mind deleting the quoted remainder of my patches when there are no further comments? I deleted a footer of 73 unnecessary lines in this mail. It's no big deal if this is too tedious, but it would make my life easier.

Junio C Hamano· Jul 1, 2016, 15:43 UTC · re: Johannes Schindelin · lore

Re: [PATCH 3/9] Prepare the builtins for a libified merge_recursive()

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 9 quoted lines
>> > A truly libified function does not die() just for fun.
>> 
>> The sentence is wasting bits.  After all, a helper function in
>> run-once-and-exit program does not die() just for fun, either.
>
> This sentence does not so much target *you* personally as audience, but
> the occasional reader of the log who wonders: "Why don't we just call
> die()? We would not have to worry about passing back the return value
> through all those long call chains..."
I was (and I am still) reacting mostly to "just for fun".
Show 8 quoted lines
> Even more natural is it to guess that the code will call error(), just
> like we do almost everywhere else.
> ...
>> But that does not mesh very well with the stated objective of the
>> patch.
> ...
> I could imagine that you wanted even more fine-grained control, where we
> have a range of return values indicating different error conditions.

I personally don't. I was pointing out the discrepancy between what the introduction says, i.e. "this way is way more flexible for the callers when they want to do their own error handling", and what the code actually does. If the explanation said "This series does not give the full flexibility potential callers may desire yet, but at least gives enough flexibility to do 'I do not want the called function to die, but append my own error message before I die myself'.", that is certainly an understandable stance to take, I would say.

Johannes Schindelin· Jul 2, 2016, 07:24 UTC · re: Junio C Hamano · lore

Re: [PATCH 3/9] Prepare the builtins for a libified merge_recursive()

Hi Junio,
On Fri, 1 Jul 2016, Junio C Hamano wrote:
Show 13 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> > A truly libified function does not die() just for fun.
> >> 
> >> The sentence is wasting bits.  After all, a helper function in
> >> run-once-and-exit program does not die() just for fun, either.
> >
> > This sentence does not so much target *you* personally as audience, but
> > the occasional reader of the log who wonders: "Why don't we just call
> > die()? We would not have to worry about passing back the return value
> > through all those long call chains..."
> 
> I was (and I am still) reacting mostly to "just for fun".
Yeah, sorry, that part was lost on me.
Show 18 quoted lines
> > Even more natural is it to guess that the code will call error(), just
> > like we do almost everywhere else.
> > ...
> >> But that does not mesh very well with the stated objective of the
> >> patch.
> > ...
> > I could imagine that you wanted even more fine-grained control, where we
> > have a range of return values indicating different error conditions.
> 
> I personally don't.  I was pointing out the discrepancy between what
> the introduction says, i.e. "this way is way more flexible for the
> callers when they want to do their own error handling", and what the
> code actually does.  If the explanation said "This series does not
> give the full flexibility potential callers may desire yet, but at
> least gives enough flexibility to do 'I do not want the called
> function to die, but append my own error message before I die
> myself'.", that is certainly an understandable stance to take, I
> would say.

Ah, but the message did not say "error message handling", but "error handling". With my limited command of the English language, I tried to convey that this patch allows the callers to do something when the called operation reported an error. Previously they did not get the chance.

Ciao, Dscho

Johannes Schindelin· Jun 29, 2016, 11:37 UTC · re: Johannes Schindelin · lore

[PATCH 6/9] merge-recursive: allow write_tree_from_memory() to error out

It is possible that a tree cannot be written (think: disk full). We will want to give the caller a chance to clean up instead of letting the program die() in such a case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +2 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index d56651c9..6ab7dfc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1875,8 +1875,8 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	if (o->call_depth)
-		*result = write_tree_from_memory(o);
+	if (o->call_depth && !(*result = write_tree_from_memory(o)))
+		return -1;
 
 	return clean;
 }
-- 
2.9.0.268.gcabc8b0
Johannes Schindelin· Jun 29, 2016, 11:37 UTC · re: Johannes Schindelin · lore

[PATCH 7/9] merge-recursive: handle return values indicating errors

We are about to libify the recursive merge machinery, where we only die() in case of a bug or memory contention. To that end, we must heed negative return values as indicating errors.

This requires our functions to be careful to pass through error conditions in call chains, and for quite a few functions this means that they have to return values to begin with.

The next step will be to convert the places where we currently die() to return negative values (read: -1) instead.

Note that we ignore errors reported by make_room_for_path(), consistent with the previous behavior (update_file_flags() used the return value of make_room_for_path() only to indicate an early return, but not a fatal error): if the error is really a fatal error, we will notice later; If not, it was not that serious a problem to begin with. (Witnesses in favor of this reasoning are t4151-am-abort and t7610-mergetool, which would start failing if we stopped on errors reported by make_room_for_path()).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 251 ++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 158 insertions(+), 93 deletions(-)
Show changes to merge-recursive.c +158 −93
diff --git a/merge-recursive.c b/merge-recursive.c
index 6ab7dfc..bb075e3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -266,8 +266,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		active_cache_tree = cache_tree();
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
-	    cache_tree_update(&the_index, 0) < 0)
-		die(_("error building trees"));
+	    cache_tree_update(&the_index, 0) < 0) {
+		error(_("error building trees"));
+		return NULL;
+	}
 
 	result = lookup_tree(active_cache_tree->sha1);
 
@@ -548,19 +550,17 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 	 */
 	int clear = 1;
 	int options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_SKIP_DFCHECK;
+	int ret = 0;
+
 	if (clear)
-		if (remove_file_from_cache(path))
-			return -1;
-	if (o)
-		if (add_cacheinfo(o->mode, o->sha1, path, 1, 0, options))
-			return -1;
-	if (a)
-		if (add_cacheinfo(a->mode, a->sha1, path, 2, 0, options))
-			return -1;
-	if (b)
-		if (add_cacheinfo(b->mode, b->sha1, path, 3, 0, options))
-			return -1;
-	return 0;
+		ret = remove_file_from_cache(path);
+	if (!ret && o)
+		ret = add_cacheinfo(o->mode, o->sha1, path, 1, 0, options);
+	if (!ret && a)
+		ret = add_cacheinfo(a->mode, a->sha1, path, 2, 0, options);
+	if (!ret && b)
+		ret = add_cacheinfo(b->mode, b->sha1, path, 3, 0, options);
+	return ret;
 }
 
 static void update_entry(struct stage_data *entry,
@@ -736,7 +736,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	return error(msg, path, _(": perhaps a D/F conflict?"));
 }
 
-static void update_file_flags(struct merge_options *o,
+static int update_file_flags(struct merge_options *o,
 			      const unsigned char *sha,
 			      unsigned mode,
 			      const char *path,
@@ -777,8 +777,7 @@ static void update_file_flags(struct merge_options *o,
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
-			free(buf);
-			goto update_index;
+			goto free_buf;
 		}
 		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
 			int fd;
@@ -801,20 +800,22 @@ static void update_file_flags(struct merge_options *o,
 		} else
 			die(_("do not know what to do with %06o %s '%s'"),
 			    mode, sha1_to_hex(sha), path);
+free_buf:
 		free(buf);
 	}
  update_index:
 	if (update_cache)
 		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+	return 0;
 }
 
-static void update_file(struct merge_options *o,
+static int update_file(struct merge_options *o,
 			int clean,
 			const unsigned char *sha,
 			unsigned mode,
 			const char *path)
 {
-	update_file_flags(o, sha, mode, path, o->call_depth || clean, !o->call_depth);
+	return update_file_flags(o, sha, mode, path, o->call_depth || clean, !o->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1010,7 +1011,7 @@ static int merge_file_one(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
-static void handle_change_delete(struct merge_options *o,
+static int handle_change_delete(struct merge_options *o,
 				 const char *path,
 				 const unsigned char *o_sha, int o_mode,
 				 const unsigned char *a_sha, int a_mode,
@@ -1018,6 +1019,7 @@ static void handle_change_delete(struct merge_options *o,
 				 const char *change, const char *change_past)
 {
 	char *renamed = NULL;
+	int ret = 0;
 	if (dir_in_way(path, !o->call_depth)) {
 		renamed = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
 	}
@@ -1028,21 +1030,23 @@ static void handle_change_delete(struct merge_options *o,
 		 * correct; since there is no true "middle point" between
 		 * them, simply reuse the base version for virtual merge base.
 		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_sha, o_mode, renamed ? renamed : path);
+		ret = remove_file_from_cache(path);
+		if (!ret)
+			ret = update_file(o, 0, o_sha, o_mode,
+					  renamed ? renamed : path);
 	} else if (!a_sha) {
 		if (!renamed) {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path);
-			update_file(o, 0, b_sha, b_mode, path);
+			ret = update_file(o, 0, b_sha, b_mode, path);
 		} else {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path, renamed);
-			update_file(o, 0, b_sha, b_mode, renamed);
+			ret = update_file(o, 0, b_sha, b_mode, renamed);
 		}
 	} else {
 		if (!renamed) {
@@ -1055,7 +1059,7 @@ static void handle_change_delete(struct merge_options *o,
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch2, change_past,
 			       o->branch1, o->branch1, path, renamed);
-			update_file(o, 0, a_sha, a_mode, renamed);
+			ret = update_file(o, 0, a_sha, a_mode, renamed);
 		}
 		/*
 		 * No need to call update_file() on path when !renamed, since
@@ -1065,9 +1069,11 @@ static void handle_change_delete(struct merge_options *o,
 		 */
 	}
 	free(renamed);
+
+	return ret;
 }
 
-static void conflict_rename_delete(struct merge_options *o,
+static int conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
 				   const char *other_branch)
@@ -1078,6 +1084,7 @@ static void conflict_rename_delete(struct merge_options *o,
 	const unsigned char *b_sha = NULL;
 	int a_mode = 0;
 	int b_mode = 0;
+	int ret = 0;
 
 	if (rename_branch == o->branch1) {
 		a_sha = dest->sha1;
@@ -1087,21 +1094,22 @@ static void conflict_rename_delete(struct merge_options *o,
 		b_mode = dest->mode;
 	}
 
-	handle_change_delete(o,
+	ret = handle_change_delete(o,
 			     o->call_depth ? orig->path : dest->path,
 			     orig->sha1, orig->mode,
 			     a_sha, a_mode,
 			     b_sha, b_mode,
 			     _("rename"), _("renamed"));
-
-	if (o->call_depth) {
-		remove_file_from_cache(dest->path);
-	} else {
-		update_stages(dest->path, NULL,
+	if (ret < 0)
+		return ret;
+	if (o->call_depth)
+		ret = remove_file_from_cache(dest->path);
+	else
+		ret = update_stages(dest->path, NULL,
 			      rename_branch == o->branch1 ? dest : NULL,
 			      rename_branch == o->branch1 ? NULL : dest);
-	}
 
+	return ret;
 }
 
 static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
@@ -1117,7 +1125,7 @@ static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
-static void handle_file(struct merge_options *o,
+static int handle_file(struct merge_options *o,
 			struct diff_filespec *rename,
 			int stage,
 			struct rename_conflict_info *ci)
@@ -1127,6 +1135,7 @@ static void handle_file(struct merge_options *o,
 	const char *cur_branch, *other_branch;
 	struct diff_filespec other;
 	struct diff_filespec *add;
+	int ret;
 
 	if (stage == 2) {
 		dst_entry = ci->dst_entry1;
@@ -1141,7 +1150,8 @@ static void handle_file(struct merge_options *o,
 	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
 	if (add) {
 		char *add_name = unique_path(o, rename->path, other_branch);
-		update_file(o, 0, add->sha1, add->mode, add_name);
+		if ((ret = update_file(o, 0, add->sha1, add->mode, add_name)))
+			return ret;
 
 		remove_file(o, 0, rename->path, 0);
 		dst_name = unique_path(o, rename->path, cur_branch);
@@ -1152,23 +1162,27 @@ static void handle_file(struct merge_options *o,
 			       rename->path, other_branch, dst_name);
 		}
 	}
-	update_file(o, 0, rename->sha1, rename->mode, dst_name);
-	if (stage == 2)
-		update_stages(rename->path, NULL, rename, add);
+	if ((ret = update_file(o, 0, rename->sha1, rename->mode, dst_name)))
+		; /* fall through, do allow dst_name to be released */
+	else if (stage == 2)
+		ret = update_stages(rename->path, NULL, rename, add);
 	else
-		update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
+
+	return ret;
 }
 
-static void conflict_rename_rename_1to2(struct merge_options *o,
+static int conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
 	struct diff_filespec *one = ci->pair1->one;
 	struct diff_filespec *a = ci->pair1->two;
 	struct diff_filespec *b = ci->pair2->two;
+	int ret = 0;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename \"%s\"->\"%s\" in branch \"%s\" "
@@ -1181,12 +1195,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct diff_filespec other;
 		struct diff_filespec *add;
 
-		if (merge_file_one(o, one->path,
+		if ((ret = merge_file_one(o, one->path,
 				 one->sha1, one->mode,
 				 a->sha1, a->mode,
 				 b->sha1, b->mode,
-				 ci->branch1, ci->branch2, &mfi) < 0)
-			return;
+				 ci->branch1, ci->branch2, &mfi)))
+			return ret;
 
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
@@ -1194,7 +1208,8 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, mfi.sha, mfi.mode, one->path);
+		if ((ret = update_file(o, 0, mfi.sha, mfi.mode, one->path)))
+			return ret;
 
 		/*
 		 * Above, we put the merged content at the merge-base's
@@ -1205,22 +1220,31 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * resolving the conflict at that path in its favor.
 		 */
 		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
-		if (add)
-			update_file(o, 0, add->sha1, add->mode, a->path);
+		if (add) {
+			if ((ret = update_file(o, 0, add->sha1, add->mode,
+					a->path)))
+				return ret;
+		}
 		else
 			remove_file_from_cache(a->path);
 		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
-		if (add)
-			update_file(o, 0, add->sha1, add->mode, b->path);
+		if (add) {
+			if ((ret = update_file(o, 0, add->sha1, add->mode,
+					b->path)))
+				return ret;
+		}
 		else
 			remove_file_from_cache(b->path);
 	} else {
-		handle_file(o, a, 2, ci);
-		handle_file(o, b, 3, ci);
+		if ((ret = handle_file(o, a, 2, ci)) ||
+		    (ret = handle_file(o, b, 3, ci)))
+			return ret;
 	}
+
+	return ret;
 }
 
-static void conflict_rename_rename_2to1(struct merge_options *o,
+static int conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* Two files, a & b, were renamed to the same thing, c. */
@@ -1231,6 +1255,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	char *path = c1->path; /* == c2->path */
 	struct merge_file_info mfi_c1;
 	struct merge_file_info mfi_c2;
+	int ret;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1241,13 +1266,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
+	if ((ret = merge_file_special_markers(o, a, c1, &ci->ren1_other,
 					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path, &mfi_c1) < 0 ||
-	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
+					    o->branch2, ci->ren1_other.path, &mfi_c1)) ||
+	    (ret = merge_file_special_markers(o, b, &ci->ren2_other, c2,
 					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path, &mfi_c2) < 0)
-		return;
+					    o->branch2, c2->path, &mfi_c2)))
+		return ret;
 
 	if (o->call_depth) {
 		/*
@@ -1258,26 +1283,32 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 		 * again later for the non-recursive merge.
 		 */
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, mfi_c1.sha, mfi_c1.mode, a->path);
-		update_file(o, 0, mfi_c2.sha, mfi_c2.mode, b->path);
+		ret = update_file(o, 0, mfi_c1.sha, mfi_c1.mode, a->path);
+		if (!ret)
+			ret = update_file(o, 0, mfi_c2.sha, mfi_c2.mode,
+				b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, _("Renaming %s to %s and %s to %s instead"),
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, mfi_c1.sha, mfi_c1.mode, new_path1);
-		update_file(o, 0, mfi_c2.sha, mfi_c2.mode, new_path2);
+		ret = update_file(o, 0, mfi_c1.sha, mfi_c1.mode, new_path1);
+		if (!ret)
+			ret = update_file(o, 0, mfi_c2.sha, mfi_c2.mode,
+				new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
+
+	return ret;
 }
 
 static int process_renames(struct merge_options *o,
 			   struct string_list *a_renames,
 			   struct string_list *b_renames)
 {
-	int clean_merge = 1, i, j;
+	int clean_merge = 1, i, j, ret;
 	struct string_list a_by_dst = STRING_LIST_INIT_NODUP;
 	struct string_list b_by_dst = STRING_LIST_INIT_NODUP;
 	const struct rename *sre;
@@ -1453,12 +1484,14 @@ static int process_renames(struct merge_options *o,
 				 * update_file_flags() instead of
 				 * update_file().
 				 */
-				update_file_flags(o,
+				ret = update_file_flags(o,
 						  ren1->pair->two->sha1,
 						  ren1->pair->two->mode,
 						  ren1_dst,
 						  1, /* update_cache */
 						  0  /* update_wd    */);
+				if (ret)
+					clean_merge = ret;
 			} else if (!sha_eq(dst_other.sha1, null_sha1)) {
 				clean_merge = 0;
 				try_merge = 1;
@@ -1468,23 +1501,29 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					if (merge_file_one(o, ren1_dst, null_sha1, 0,
+					if ((ret = merge_file_one(o, ren1_dst, null_sha1, 0,
 							 ren1->pair->two->sha1, ren1->pair->two->mode,
 							 dst_other.sha1, dst_other.mode,
-							 branch1, branch2, &mfi) < 0)
-						return -1;
+							 branch1, branch2, &mfi))) {
+						clean_merge = ret;
+						goto cleanup_and_return;
+					}
 					output(o, 1, _("Adding merged %s"), ren1_dst);
-					update_file(o, 0, mfi.sha, mfi.mode, ren1_dst);
+					if ((ret = update_file(o, 0, mfi.sha, mfi.mode, ren1_dst)))
+						clean_merge = ret;
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, _("Adding as %s instead"), new_path);
-					update_file(o, 0, dst_other.sha1, dst_other.mode, new_path);
+					if ((ret = update_file(o, 0, dst_other.sha1, dst_other.mode, new_path)))
+						clean_merge = ret;
 					free(new_path);
 				}
 			} else
 				try_merge = 1;
 
+			if (clean_merge < 0)
+				goto cleanup_and_return;
 			if (try_merge) {
 				struct diff_filespec *one, *a, *b;
 				src_other.path = (char *)ren1_src;
@@ -1511,6 +1550,7 @@ static int process_renames(struct merge_options *o,
 			}
 		}
 	}
+cleanup_and_return:
 	string_list_clear(&a_by_dst, 0);
 	string_list_clear(&b_by_dst, 0);
 
@@ -1573,13 +1613,13 @@ error_return:
 	return ret;
 }
 
-static void handle_modify_delete(struct merge_options *o,
+static int handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 unsigned char *o_sha, int o_mode,
 				 unsigned char *a_sha, int a_mode,
 				 unsigned char *b_sha, int b_mode)
 {
-	handle_change_delete(o,
+	return handle_change_delete(o,
 			     path,
 			     o_sha, o_mode,
 			     a_sha, a_mode,
@@ -1599,6 +1639,7 @@ static int merge_content(struct merge_options *o,
 	struct merge_file_info mfi;
 	struct diff_filespec one, a, b;
 	unsigned df_conflict_remains = 0;
+	int ret;
 
 	if (!o_sha) {
 		reason = _("add/add");
@@ -1628,10 +1669,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	if (merge_file_special_markers(o, &one, &a, &b,
+	if ((ret = merge_file_special_markers(o, &one, &a, &b,
 					 o->branch1, path1,
-					 o->branch2, path2, &mfi) < 0)
-		return -1;
+					 o->branch2, path2, &mfi)))
+		return ret;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode) {
@@ -1658,7 +1699,8 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			update_stages(path, &one, &a, &b);
+			if ((ret = update_stages(path, &one, &a, &b)))
+				return ret;
 	}
 
 	if (df_conflict_remains) {
@@ -1666,27 +1708,33 @@ static int merge_content(struct merge_options *o,
 		if (o->call_depth) {
 			remove_file_from_cache(path);
 		} else {
-			if (!mfi.clean)
-				update_stages(path, &one, &a, &b);
-			else {
+			if (!mfi.clean) {
+				if ((ret = update_stages(path, &one, &a, &b)))
+					return ret;
+			} else {
 				int file_from_stage2 = was_tracked(path);
 				struct diff_filespec merged;
 				hashcpy(merged.sha1, mfi.sha);
 				merged.mode = mfi.mode;
 
-				update_stages(path, NULL,
+				if ((ret = update_stages(path, NULL,
 					      file_from_stage2 ? &merged : NULL,
-					      file_from_stage2 ? NULL : &merged);
+					      file_from_stage2 ? NULL : &merged)))
+					return ret;
 			}
 
 		}
 		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, _("Adding as %s instead"), new_path);
-		update_file(o, 0, mfi.sha, mfi.mode, new_path);
+		if ((ret = update_file(o, 0, mfi.sha, mfi.mode, new_path))) {
+			free(new_path);
+			return ret;
+		}
 		free(new_path);
 		mfi.clean = 0;
 	} else {
-		update_file(o, mfi.clean, mfi.sha, mfi.mode, path);
+		if ((ret = update_file(o, mfi.clean, mfi.sha, mfi.mode, path)))
+			return ret;
 	}
 	return mfi.clean;
 
@@ -1696,7 +1744,7 @@ static int merge_content(struct merge_options *o,
 static int process_entry(struct merge_options *o,
 			 const char *path, struct stage_data *entry)
 {
-	int clean_merge = 1;
+	int clean_merge = 1, ret;
 	int normalize = o->renormalize;
 	unsigned o_mode = entry->stages[1].mode;
 	unsigned a_mode = entry->stages[2].mode;
@@ -1717,17 +1765,23 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
-			conflict_rename_delete(o, conflict_info->pair1,
+			if ((ret = conflict_rename_delete(o,
+					       conflict_info->pair1,
 					       conflict_info->branch1,
-					       conflict_info->branch2);
+					       conflict_info->branch2)))
+				clean_merge = ret;
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info);
+			if ((ret = conflict_rename_rename_1to2(o,
+					conflict_info)))
+				clean_merge = ret;
 			break;
 		case RENAME_TWO_FILES_TO_ONE:
 			clean_merge = 0;
-			conflict_rename_rename_2to1(o, conflict_info);
+			if ((ret = conflict_rename_rename_2to1(o,
+					conflict_info)))
+				clean_merge = ret;
 			break;
 		default:
 			entry->processed = 0;
@@ -1747,8 +1801,9 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_modify_delete(o, path, o_sha, o_mode,
-					     a_sha, a_mode, b_sha, b_mode);
+			if ((ret = handle_modify_delete(o, path, o_sha, o_mode,
+					a_sha, a_mode, b_sha, b_mode)))
+				clean_merge = ret;
 		}
 	} else if ((!o_sha && a_sha && !b_sha) ||
 		   (!o_sha && !a_sha && b_sha)) {
@@ -1780,14 +1835,18 @@ static int process_entry(struct merge_options *o,
 			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s"),
 			       conf, path, other_branch, path, new_path);
-			update_file(o, 0, sha, mode, new_path);
-			if (o->call_depth)
+			ret = update_file(o, 0, sha, mode, new_path);
+			if (ret)
+				clean_merge = ret;
+			else if (o->call_depth)
 				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, _("Adding %s"), path);
 			/* do not overwrite file if already present */
-			update_file_flags(o, sha, mode, path, 1, !a_sha);
+			if ((ret = update_file_flags(o, sha, mode, path, 1,
+					!a_sha)))
+				clean_merge = ret;
 		}
 	} else if (a_sha && b_sha) {
 		/* Case C: Added in both (check for same permissions) and */
@@ -1850,12 +1909,18 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
+		if (clean < 0)
+			return clean;
 		for (i = entries->nr-1; 0 <= i; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_entry(o, path, e))
-				clean = 0;
+			if (!e->processed) {
+				int ret = process_entry(o, path, e);
+				if (!ret)
+					clean = 0;
+				else if (ret < 0)
+					return ret;
+			}
 		}
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 21:06 UTC · re: Johannes Schindelin · lore

Re: [PATCH 7/9] merge-recursive: handle return values indicating errors

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 14 quoted lines
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 6ab7dfc..bb075e3 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -266,8 +266,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
>  		active_cache_tree = cache_tree();
>  
>  	if (!cache_tree_fully_valid(active_cache_tree) &&
> -	    cache_tree_update(&the_index, 0) < 0)
> -		die(_("error building trees"));
> +	    cache_tree_update(&the_index, 0) < 0) {
> +		error(_("error building trees"));
> +		return NULL;
> +	}

This actually is a BUG(), isn't it? We have already verified that the cache is merged, so cache_tree_update() ought to be able to come up with the whole-tree hash.

Show 28 quoted lines
> @@ -548,19 +550,17 @@ static int update_stages(const char *path, const struct diff_filespec *o,
>  	 */
>  	int clear = 1;
>  	int options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_SKIP_DFCHECK;
> +	int ret = 0;
> +
>  	if (clear)
> -		if (remove_file_from_cache(path))
> -			return -1;
> -	if (o)
> -		if (add_cacheinfo(o->mode, o->sha1, path, 1, 0, options))
> -			return -1;
> -	if (a)
> -		if (add_cacheinfo(a->mode, a->sha1, path, 2, 0, options))
> -			return -1;
> -	if (b)
> -		if (add_cacheinfo(b->mode, b->sha1, path, 3, 0, options))
> -			return -1;
> -	return 0;
> +		ret = remove_file_from_cache(path);
> +	if (!ret && o)
> +		ret = add_cacheinfo(o->mode, o->sha1, path, 1, 0, options);
> +	if (!ret && a)
> +		ret = add_cacheinfo(a->mode, a->sha1, path, 2, 0, options);
> +	if (!ret && b)
> +		ret = add_cacheinfo(b->mode, b->sha1, path, 3, 0, options);
> +	return ret;
>  }

Aren't the preimage and the postimage doing the same thing? The only two differences I spot are (1) it is clear in the original that the returned value is -1 in the error case, even if the error convention of remove_file_from_cache() and add_cacheinfo() were "0 is good, others are bad"; and (2) the control flow is easier to follow in the original.

Show 25 quoted lines
> @@ -736,7 +736,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
>  	return error(msg, path, _(": perhaps a D/F conflict?"));
>  }
>  
> -static void update_file_flags(struct merge_options *o,
> +static int update_file_flags(struct merge_options *o,
>  			      const unsigned char *sha,
>  			      unsigned mode,
>  			      const char *path,
> @@ -777,8 +777,7 @@ static void update_file_flags(struct merge_options *o,
>  
>  		if (make_room_for_path(o, path) < 0) {
>  			update_wd = 0;
> -			free(buf);
> -			goto update_index;
> +			goto free_buf;
>  		}
>  		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
>  			int fd;
> @@ -801,20 +800,22 @@ static void update_file_flags(struct merge_options *o,
>  		} else
>  			die(_("do not know what to do with %06o %s '%s'"),
>  			    mode, sha1_to_hex(sha), path);
> +free_buf:
>  		free(buf);
I somehow find the above change harder to follow than the original.
Show 7 quoted lines
>  	}
>   update_index:
>  	if (update_cache)
>  		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
> +	return 0;
>  }
>  
This one is in line with the stated goal of the patch.
Show 10 quoted lines
> @@ -1028,21 +1030,23 @@ static void handle_change_delete(struct merge_options *o,
>  		 * correct; since there is no true "middle point" between
>  		 * them, simply reuse the base version for virtual merge base.
>  		 */
> -		remove_file_from_cache(path);
> -		update_file(o, 0, o_sha, o_mode, renamed ? renamed : path);
> +		ret = remove_file_from_cache(path);
> +		if (!ret)
> +			ret = update_file(o, 0, o_sha, o_mode,
> +					  renamed ? renamed : path);

As you noted in the log message, this does change the behaviour. If remove returns non-zero for whatever reason, we still did update() in the original, but we no longer do. Does this have negative effect to the overall codeflow?

Or, assuming that everybody returns -1 for errors, perhaps
	ret = remove();
        ret |= update();
may be a more faithful and safe conversion?
Show 25 quoted lines
> @@ -1087,21 +1094,22 @@ static void conflict_rename_delete(struct merge_options *o,
>  		b_mode = dest->mode;
>  	}
>  
> -	handle_change_delete(o,
> +	ret = handle_change_delete(o,
>  			     o->call_depth ? orig->path : dest->path,
>  			     orig->sha1, orig->mode,
>  			     a_sha, a_mode,
>  			     b_sha, b_mode,
>  			     _("rename"), _("renamed"));
> -
> -	if (o->call_depth) {
> -		remove_file_from_cache(dest->path);
> -	} else {
> -		update_stages(dest->path, NULL,
> +	if (ret < 0)
> +		return ret;
> +	if (o->call_depth)
> +		ret = remove_file_from_cache(dest->path);
> +	else
> +		ret = update_stages(dest->path, NULL,
>  			      rename_branch == o->branch1 ? dest : NULL,
>  			      rename_branch == o->branch1 ? NULL : dest);
> -	}

Similarly, if handle_change_delete() returns non-zero, we no longer call remove() or update(). Is that a good behaviour change?

Show 5 quoted lines
> -static void handle_file(struct merge_options *o,
> +static int handle_file(struct merge_options *o,
>  			struct diff_filespec *rename,
>  			int stage,
>  			struct rename_conflict_info *ci)
Likewise.
Show 14 quoted lines
> -static void conflict_rename_rename_1to2(struct merge_options *o,
> +static int conflict_rename_rename_1to2(struct merge_options *o,
>  					struct rename_conflict_info *ci)
>  {
> ...
> -		if (merge_file_one(o, one->path,
> +		if ((ret = merge_file_one(o, one->path,
>  				 one->sha1, one->mode,
>  				 a->sha1, a->mode,
>  				 b->sha1, b->mode,
> -				 ci->branch1, ci->branch2, &mfi) < 0)
> -			return;
> +				 ci->branch1, ci->branch2, &mfi)))
> +			return ret;
This does not change behaviour.
Show 7 quoted lines
> @@ -1194,7 +1208,8 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
>  		 * pathname and then either rename the add-source file to that
>  		 * unique path, or use that unique path instead of src here.
>  		 */
> -		update_file(o, 0, mfi.sha, mfi.mode, one->path);
> +		if ((ret = update_file(o, 0, mfi.sha, mfi.mode, one->path)))
> +			return ret;
But this does.
Show 11 quoted lines
> @@ -1205,22 +1220,31 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
>  		 * resolving the conflict at that path in its favor.
>  		 */
>  		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
> -		if (add)
> -			update_file(o, 0, add->sha1, add->mode, a->path);
> +		if (add) {
> +			if ((ret = update_file(o, 0, add->sha1, add->mode,
> +					a->path)))
> +				return ret;
> +		}
So does this.
Show 10 quoted lines
>  		else
>  			remove_file_from_cache(a->path);
>  		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
> -		if (add)
> -			update_file(o, 0, add->sha1, add->mode, b->path);
> +		if (add) {
> +			if ((ret = update_file(o, 0, add->sha1, add->mode,
> +					b->path)))
> +				return ret;
> +		}
And this.
Show 6 quoted lines
>  	} else {
> -		handle_file(o, a, 2, ci);
> -		handle_file(o, b, 3, ci);
> +		if ((ret = handle_file(o, a, 2, ci)) ||
> +		    (ret = handle_file(o, b, 3, ci)))
> +			return ret;
And this.
Johannes Schindelin· Jul 1, 2016, 11:08 UTC · re: Junio C Hamano · lore

Re: [PATCH 7/9] merge-recursive: handle return values indicating errors

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 20 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index 6ab7dfc..bb075e3 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -266,8 +266,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
> >  		active_cache_tree = cache_tree();
> >  
> >  	if (!cache_tree_fully_valid(active_cache_tree) &&
> > -	    cache_tree_update(&the_index, 0) < 0)
> > -		die(_("error building trees"));
> > +	    cache_tree_update(&the_index, 0) < 0) {
> > +		error(_("error building trees"));
> > +		return NULL;
> > +	}
> 
> This actually is a BUG(), isn't it?  We have already verified that
> the cache is merged, so cache_tree_update() ought to be able to come
> up with the whole-tree hash.
I am not so sure. What if the disk is full?
Show 35 quoted lines
> > @@ -548,19 +550,17 @@ static int update_stages(const char *path, const struct diff_filespec *o,
> >  	 */
> >  	int clear = 1;
> >  	int options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_SKIP_DFCHECK;
> > +	int ret = 0;
> > +
> >  	if (clear)
> > -		if (remove_file_from_cache(path))
> > -			return -1;
> > -	if (o)
> > -		if (add_cacheinfo(o->mode, o->sha1, path, 1, 0, options))
> > -			return -1;
> > -	if (a)
> > -		if (add_cacheinfo(a->mode, a->sha1, path, 2, 0, options))
> > -			return -1;
> > -	if (b)
> > -		if (add_cacheinfo(b->mode, b->sha1, path, 3, 0, options))
> > -			return -1;
> > -	return 0;
> > +		ret = remove_file_from_cache(path);
> > +	if (!ret && o)
> > +		ret = add_cacheinfo(o->mode, o->sha1, path, 1, 0, options);
> > +	if (!ret && a)
> > +		ret = add_cacheinfo(a->mode, a->sha1, path, 2, 0, options);
> > +	if (!ret && b)
> > +		ret = add_cacheinfo(b->mode, b->sha1, path, 3, 0, options);
> > +	return ret;
> >  }
> 
> Aren't the preimage and the postimage doing the same thing?  The
> only two differences I spot are (1) it is clear in the original that
> the returned value is -1 in the error case, even if the error
> convention of remove_file_from_cache() and add_cacheinfo() were "0
> is good, others are bad"; and (2) the control flow is easier to
> follow in the original.

Ah crap. This is a late left-over from the time when I thought I had to introduce a "really fatal error" condition, i.e. return -128. I thought that there might be errors that need to be handled differently, and therefore it was essential that the return value -128 would not be converted into a -1.

Will revert this hunk.
Show 27 quoted lines
> > @@ -736,7 +736,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
> >  	return error(msg, path, _(": perhaps a D/F conflict?"));
> >  }
> >  
> > -static void update_file_flags(struct merge_options *o,
> > +static int update_file_flags(struct merge_options *o,
> >  			      const unsigned char *sha,
> >  			      unsigned mode,
> >  			      const char *path,
> > @@ -777,8 +777,7 @@ static void update_file_flags(struct merge_options *o,
> >  
> >  		if (make_room_for_path(o, path) < 0) {
> >  			update_wd = 0;
> > -			free(buf);
> > -			goto update_index;
> > +			goto free_buf;
> >  		}
> >  		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
> >  			int fd;
> > @@ -801,20 +800,22 @@ static void update_file_flags(struct merge_options *o,
> >  		} else
> >  			die(_("do not know what to do with %06o %s '%s'"),
> >  			    mode, sha1_to_hex(sha), path);
> > +free_buf:
> >  		free(buf);
> 
> I somehow find the above change harder to follow than the original.

Yeah, you noticed when commenting on the next patch why I did it. I added a comment to the commit message.

Show 25 quoted lines
> >  	}
> >   update_index:
> >  	if (update_cache)
> >  		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
> > +	return 0;
> >  }
> >  
> 
> This one is in line with the stated goal of the patch.
> 
> > @@ -1028,21 +1030,23 @@ static void handle_change_delete(struct merge_options *o,
> >  		 * correct; since there is no true "middle point" between
> >  		 * them, simply reuse the base version for virtual merge base.
> >  		 */
> > -		remove_file_from_cache(path);
> > -		update_file(o, 0, o_sha, o_mode, renamed ? renamed : path);
> > +		ret = remove_file_from_cache(path);
> > +		if (!ret)
> > +			ret = update_file(o, 0, o_sha, o_mode,
> > +					  renamed ? renamed : path);
> 
> As you noted in the log message, this does change the behaviour.  If
> remove returns non-zero for whatever reason, we still did update()
> in the original, but we no longer do.  Does this have negative
> effect to the overall codeflow?

Please note that remove_file_from_index() does not, in fact, anything but 0 for the moment. So the practical use of testing the return value is nil with Git's current source code.

It is more about future-proofing than anything else.

As to this incantation specifically, if we ever fail in remove_file_from_cache(), I would expect the code to fail fast. That is how I implemented it in this patch.

Show 28 quoted lines
> > @@ -1087,21 +1094,22 @@ static void conflict_rename_delete(struct merge_options *o,
> >  		b_mode = dest->mode;
> >  	}
> >  
> > -	handle_change_delete(o,
> > +	ret = handle_change_delete(o,
> >  			     o->call_depth ? orig->path : dest->path,
> >  			     orig->sha1, orig->mode,
> >  			     a_sha, a_mode,
> >  			     b_sha, b_mode,
> >  			     _("rename"), _("renamed"));
> > -
> > -	if (o->call_depth) {
> > -		remove_file_from_cache(dest->path);
> > -	} else {
> > -		update_stages(dest->path, NULL,
> > +	if (ret < 0)
> > +		return ret;
> > +	if (o->call_depth)
> > +		ret = remove_file_from_cache(dest->path);
> > +	else
> > +		ret = update_stages(dest->path, NULL,
> >  			      rename_branch == o->branch1 ? dest : NULL,
> >  			      rename_branch == o->branch1 ? NULL : dest);
> > -	}
> 
> Similarly, if handle_change_delete() returns non-zero, we no longer
> call remove() or update().  Is that a good behaviour change?

Please note that handle_change_delete() did not return any value before this patch series. In those cases that now return -1, it previously die()d.

My patch ensures that the latter functions are not called in the error case. Just like before.

Show 7 quoted lines
> > -static void handle_file(struct merge_options *o,
> > +static int handle_file(struct merge_options *o,
> >  			struct diff_filespec *rename,
> >  			int stage,
> >  			struct rename_conflict_info *ci)
> 
> Likewise.
Ditto.
Show 26 quoted lines
> > -static void conflict_rename_rename_1to2(struct merge_options *o,
> > +static int conflict_rename_rename_1to2(struct merge_options *o,
> >  					struct rename_conflict_info *ci)
> >  {
> > ...
> > -		if (merge_file_one(o, one->path,
> > +		if ((ret = merge_file_one(o, one->path,
> >  				 one->sha1, one->mode,
> >  				 a->sha1, a->mode,
> >  				 b->sha1, b->mode,
> > -				 ci->branch1, ci->branch2, &mfi) < 0)
> > -			return;
> > +				 ci->branch1, ci->branch2, &mfi)))
> > +			return ret;
> 
> This does not change behaviour.
> 
> > @@ -1194,7 +1208,8 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
> >  		 * pathname and then either rename the add-source file to that
> >  		 * unique path, or use that unique path instead of src here.
> >  		 */
> > -		update_file(o, 0, mfi.sha, mfi.mode, one->path);
> > +		if ((ret = update_file(o, 0, mfi.sha, mfi.mode, one->path)))
> > +			return ret;
> 
> But this does.

As stated above: it does not. In case of an error in update_file() (which is now reported via a return value, in contrast to the previous behavior), just like before, we skip the rest of the function.

The difference is that we now clean up our resources and return an error instead of die()ing.

Show 13 quoted lines
> > @@ -1205,22 +1220,31 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
> >  		 * resolving the conflict at that path in its favor.
> >  		 */
> >  		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
> > -		if (add)
> > -			update_file(o, 0, add->sha1, add->mode, a->path);
> > +		if (add) {
> > +			if ((ret = update_file(o, 0, add->sha1, add->mode,
> > +					a->path)))
> > +				return ret;
> > +		}
> 
> So does this.
Same as above (i.e. same behavior as previously).
Show 12 quoted lines
> >  		else
> >  			remove_file_from_cache(a->path);
> >  		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
> > -		if (add)
> > -			update_file(o, 0, add->sha1, add->mode, b->path);
> > +		if (add) {
> > +			if ((ret = update_file(o, 0, add->sha1, add->mode,
> > +					b->path)))
> > +				return ret;
> > +		}
> 
> And this.
Yep. Same behavior.
Show 8 quoted lines
> >  	} else {
> > -		handle_file(o, a, 2, ci);
> > -		handle_file(o, b, 3, ci);
> > +		if ((ret = handle_file(o, a, 2, ci)) ||
> > +		    (ret = handle_file(o, b, 3, ci)))
> > +			return ret;
> 
> And this.
Yep. Same behavior.

Ciao, Dscho

Johannes Schindelin· Jun 29, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH 5/9] merge-recursive: avoid returning a wholesale struct

It is technically allowed, as per C89, for functions' return type to be complete structs (i.e. *not* just pointers to structs), but it is bad practice.

This is a very late attempt to contain the damage done by this developer in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which introduced such a return type.

It will also help the current effort to libify merge-recursive.c, as it will allow us to return proper error codes later.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 93 ++++++++++++++++++++++++++++++-------------------------
 1 file changed, 50 insertions(+), 43 deletions(-)
Show changes to merge-recursive.c +50 −43
diff --git a/merge-recursive.c b/merge-recursive.c
index c4ece96..d56651c9 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -888,47 +888,47 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file_1(struct merge_options *o,
+static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
 					   const struct diff_filespec *b,
 					   const char *branch1,
-					   const char *branch2)
+					   const char *branch2,
+					   struct merge_file_info *result)
 {
-	struct merge_file_info result;
-	result.merge = 0;
-	result.clean = 1;
+	result->merge = 0;
+	result->clean = 1;
 
 	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
-		result.clean = 0;
+		result->clean = 0;
 		if (S_ISREG(a->mode)) {
-			result.mode = a->mode;
-			hashcpy(result.sha, a->sha1);
+			result->mode = a->mode;
+			hashcpy(result->sha, a->sha1);
 		} else {
-			result.mode = b->mode;
-			hashcpy(result.sha, b->sha1);
+			result->mode = b->mode;
+			hashcpy(result->sha, b->sha1);
 		}
 	} else {
 		if (!sha_eq(a->sha1, one->sha1) && !sha_eq(b->sha1, one->sha1))
-			result.merge = 1;
+			result->merge = 1;
 
 		/*
 		 * Merge modes
 		 */
 		if (a->mode == b->mode || a->mode == one->mode)
-			result.mode = b->mode;
+			result->mode = b->mode;
 		else {
-			result.mode = a->mode;
+			result->mode = a->mode;
 			if (b->mode != one->mode) {
-				result.clean = 0;
-				result.merge = 1;
+				result->clean = 0;
+				result->merge = 1;
 			}
 		}
 
 		if (sha_eq(a->sha1, b->sha1) || sha_eq(a->sha1, one->sha1))
-			hashcpy(result.sha, b->sha1);
+			hashcpy(result->sha, b->sha1);
 		else if (sha_eq(b->sha1, one->sha1))
-			hashcpy(result.sha, a->sha1);
+			hashcpy(result->sha, a->sha1);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
 			int merge_status;
@@ -940,62 +940,63 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 				die(_("Failed to execute internal merge"));
 
 			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result.sha))
+					    blob_type, result->sha))
 				die(_("Unable to add %s to database"),
 				    a->path);
 
 			free(result_buf.ptr);
-			result.clean = (merge_status == 0);
+			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result.clean = merge_submodule(result.sha,
+			result->clean = merge_submodule(result->sha,
 						       one->path, one->sha1,
 						       a->sha1, b->sha1,
 						       !o->call_depth);
 		} else if (S_ISLNK(a->mode)) {
-			hashcpy(result.sha, a->sha1);
+			hashcpy(result->sha, a->sha1);
 
 			if (!sha_eq(a->sha1, b->sha1))
-				result.clean = 0;
+				result->clean = 0;
 		} else
 			die(_("BUG: unsupported object type in the tree"));
 	}
 
-	return result;
+	return 0;
 }
 
-static struct merge_file_info
-merge_file_special_markers(struct merge_options *o,
+static int merge_file_special_markers(struct merge_options *o,
 			   const struct diff_filespec *one,
 			   const struct diff_filespec *a,
 			   const struct diff_filespec *b,
 			   const char *branch1,
 			   const char *filename1,
 			   const char *branch2,
-			   const char *filename2)
+			   const char *filename2,
+			   struct merge_file_info *mfi)
 {
 	char *side1 = NULL;
 	char *side2 = NULL;
-	struct merge_file_info mfi;
+	int ret;
 
 	if (filename1)
 		side1 = xstrfmt("%s:%s", branch1, filename1);
 	if (filename2)
 		side2 = xstrfmt("%s:%s", branch2, filename2);
 
-	mfi = merge_file_1(o, one, a, b,
-			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	ret = merge_file_1(o, one, a, b,
+		side1 ? side1 : branch1, side2 ? side2 : branch2, mfi);
 	free(side1);
 	free(side2);
-	return mfi;
+	return ret;
 }
 
-static struct merge_file_info merge_file_one(struct merge_options *o,
+static int merge_file_one(struct merge_options *o,
 					 const char *path,
 					 const unsigned char *o_sha, int o_mode,
 					 const unsigned char *a_sha, int a_mode,
 					 const unsigned char *b_sha, int b_mode,
 					 const char *branch1,
-					 const char *branch2)
+					 const char *branch2,
+					 struct merge_file_info *mfi)
 {
 	struct diff_filespec one, a, b;
 
@@ -1006,7 +1007,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
 	a.mode = a_mode;
 	hashcpy(b.sha1, b_sha);
 	b.mode = b_mode;
-	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
 static void handle_change_delete(struct merge_options *o,
@@ -1179,11 +1180,14 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct merge_file_info mfi;
 		struct diff_filespec other;
 		struct diff_filespec *add;
-		mfi = merge_file_one(o, one->path,
+
+		if (merge_file_one(o, one->path,
 				 one->sha1, one->mode,
 				 a->sha1, a->mode,
 				 b->sha1, b->mode,
-				 ci->branch1, ci->branch2);
+				 ci->branch1, ci->branch2, &mfi) < 0)
+			return;
+
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1237,12 +1241,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
+	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
 					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path);
-	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
+					    o->branch2, ci->ren1_other.path, &mfi_c1) < 0 ||
+	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path);
+					    o->branch2, c2->path, &mfi_c2) < 0)
+		return;
 
 	if (o->call_depth) {
 		/*
@@ -1463,10 +1468,11 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					mfi = merge_file_one(o, ren1_dst, null_sha1, 0,
+					if (merge_file_one(o, ren1_dst, null_sha1, 0,
 							 ren1->pair->two->sha1, ren1->pair->two->mode,
 							 dst_other.sha1, dst_other.mode,
-							 branch1, branch2);
+							 branch1, branch2, &mfi) < 0)
+						return -1;
 					output(o, 1, _("Adding merged %s"), ren1_dst);
 					update_file(o, 0, mfi.sha, mfi.mode, ren1_dst);
 					try_merge = 0;
@@ -1622,9 +1628,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_special_markers(o, &one, &a, &b,
+	if (merge_file_special_markers(o, &one, &a, &b,
 					 o->branch1, path1,
-					 o->branch2, path2);
+					 o->branch2, path2, &mfi) < 0)
+		return -1;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode) {
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 20:21 UTC · re: Johannes Schindelin · lore

Re: [PATCH 5/9] merge-recursive: avoid returning a wholesale struct

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> It is technically allowed, as per C89, for functions' return type to
> be complete structs (i.e. *not* just pointers to structs), but it is
> bad practice.
Not necessarily.
Show 6 quoted lines
> This is a very late attempt to contain the damage done by this developer
> in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which
> introduced such a return type.
>
> It will also help the current effort to libify merge-recursive.c, as
> it will allow us to return proper error codes later.

But this part of the motivation does make sense. Having to pass an extra int &error_code field, only because the return value is already used for something else, is a lot more weird.

Show 224 quoted lines
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  merge-recursive.c | 93 ++++++++++++++++++++++++++++++-------------------------
>  1 file changed, 50 insertions(+), 43 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index c4ece96..d56651c9 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -888,47 +888,47 @@ static int merge_3way(struct merge_options *o,
>  	return merge_status;
>  }
>  
> -static struct merge_file_info merge_file_1(struct merge_options *o,
> +static int merge_file_1(struct merge_options *o,
>  					   const struct diff_filespec *one,
>  					   const struct diff_filespec *a,
>  					   const struct diff_filespec *b,
>  					   const char *branch1,
> -					   const char *branch2)
> +					   const char *branch2,
> +					   struct merge_file_info *result)
>  {
> -	struct merge_file_info result;
> -	result.merge = 0;
> -	result.clean = 1;
> +	result->merge = 0;
> +	result->clean = 1;
>  
>  	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
> -		result.clean = 0;
> +		result->clean = 0;
>  		if (S_ISREG(a->mode)) {
> -			result.mode = a->mode;
> -			hashcpy(result.sha, a->sha1);
> +			result->mode = a->mode;
> +			hashcpy(result->sha, a->sha1);
>  		} else {
> -			result.mode = b->mode;
> -			hashcpy(result.sha, b->sha1);
> +			result->mode = b->mode;
> +			hashcpy(result->sha, b->sha1);
>  		}
>  	} else {
>  		if (!sha_eq(a->sha1, one->sha1) && !sha_eq(b->sha1, one->sha1))
> -			result.merge = 1;
> +			result->merge = 1;
>  
>  		/*
>  		 * Merge modes
>  		 */
>  		if (a->mode == b->mode || a->mode == one->mode)
> -			result.mode = b->mode;
> +			result->mode = b->mode;
>  		else {
> -			result.mode = a->mode;
> +			result->mode = a->mode;
>  			if (b->mode != one->mode) {
> -				result.clean = 0;
> -				result.merge = 1;
> +				result->clean = 0;
> +				result->merge = 1;
>  			}
>  		}
>  
>  		if (sha_eq(a->sha1, b->sha1) || sha_eq(a->sha1, one->sha1))
> -			hashcpy(result.sha, b->sha1);
> +			hashcpy(result->sha, b->sha1);
>  		else if (sha_eq(b->sha1, one->sha1))
> -			hashcpy(result.sha, a->sha1);
> +			hashcpy(result->sha, a->sha1);
>  		else if (S_ISREG(a->mode)) {
>  			mmbuffer_t result_buf;
>  			int merge_status;
> @@ -940,62 +940,63 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
>  				die(_("Failed to execute internal merge"));
>  
>  			if (write_sha1_file(result_buf.ptr, result_buf.size,
> -					    blob_type, result.sha))
> +					    blob_type, result->sha))
>  				die(_("Unable to add %s to database"),
>  				    a->path);
>  
>  			free(result_buf.ptr);
> -			result.clean = (merge_status == 0);
> +			result->clean = (merge_status == 0);
>  		} else if (S_ISGITLINK(a->mode)) {
> -			result.clean = merge_submodule(result.sha,
> +			result->clean = merge_submodule(result->sha,
>  						       one->path, one->sha1,
>  						       a->sha1, b->sha1,
>  						       !o->call_depth);
>  		} else if (S_ISLNK(a->mode)) {
> -			hashcpy(result.sha, a->sha1);
> +			hashcpy(result->sha, a->sha1);
>  
>  			if (!sha_eq(a->sha1, b->sha1))
> -				result.clean = 0;
> +				result->clean = 0;
>  		} else
>  			die(_("BUG: unsupported object type in the tree"));
>  	}
>  
> -	return result;
> +	return 0;
>  }
>  
> -static struct merge_file_info
> -merge_file_special_markers(struct merge_options *o,
> +static int merge_file_special_markers(struct merge_options *o,
>  			   const struct diff_filespec *one,
>  			   const struct diff_filespec *a,
>  			   const struct diff_filespec *b,
>  			   const char *branch1,
>  			   const char *filename1,
>  			   const char *branch2,
> -			   const char *filename2)
> +			   const char *filename2,
> +			   struct merge_file_info *mfi)
>  {
>  	char *side1 = NULL;
>  	char *side2 = NULL;
> -	struct merge_file_info mfi;
> +	int ret;
>  
>  	if (filename1)
>  		side1 = xstrfmt("%s:%s", branch1, filename1);
>  	if (filename2)
>  		side2 = xstrfmt("%s:%s", branch2, filename2);
>  
> -	mfi = merge_file_1(o, one, a, b,
> -			   side1 ? side1 : branch1, side2 ? side2 : branch2);
> +	ret = merge_file_1(o, one, a, b,
> +		side1 ? side1 : branch1, side2 ? side2 : branch2, mfi);
>  	free(side1);
>  	free(side2);
> -	return mfi;
> +	return ret;
>  }
>  
> -static struct merge_file_info merge_file_one(struct merge_options *o,
> +static int merge_file_one(struct merge_options *o,
>  					 const char *path,
>  					 const unsigned char *o_sha, int o_mode,
>  					 const unsigned char *a_sha, int a_mode,
>  					 const unsigned char *b_sha, int b_mode,
>  					 const char *branch1,
> -					 const char *branch2)
> +					 const char *branch2,
> +					 struct merge_file_info *mfi)
>  {
>  	struct diff_filespec one, a, b;
>  
> @@ -1006,7 +1007,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
>  	a.mode = a_mode;
>  	hashcpy(b.sha1, b_sha);
>  	b.mode = b_mode;
> -	return merge_file_1(o, &one, &a, &b, branch1, branch2);
> +	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
>  }
>  
>  static void handle_change_delete(struct merge_options *o,
> @@ -1179,11 +1180,14 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
>  		struct merge_file_info mfi;
>  		struct diff_filespec other;
>  		struct diff_filespec *add;
> -		mfi = merge_file_one(o, one->path,
> +
> +		if (merge_file_one(o, one->path,
>  				 one->sha1, one->mode,
>  				 a->sha1, a->mode,
>  				 b->sha1, b->mode,
> -				 ci->branch1, ci->branch2);
> +				 ci->branch1, ci->branch2, &mfi) < 0)
> +			return;
> +
>  		/*
>  		 * FIXME: For rename/add-source conflicts (if we could detect
>  		 * such), this is wrong.  We should instead find a unique
> @@ -1237,12 +1241,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
>  	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
>  	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
>  
> -	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
> +	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
>  					    o->branch1, c1->path,
> -					    o->branch2, ci->ren1_other.path);
> -	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
> +					    o->branch2, ci->ren1_other.path, &mfi_c1) < 0 ||
> +	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
>  					    o->branch1, ci->ren2_other.path,
> -					    o->branch2, c2->path);
> +					    o->branch2, c2->path, &mfi_c2) < 0)
> +		return;
>  
>  	if (o->call_depth) {
>  		/*
> @@ -1463,10 +1468,11 @@ static int process_renames(struct merge_options *o,
>  				       ren1_dst, branch2);
>  				if (o->call_depth) {
>  					struct merge_file_info mfi;
> -					mfi = merge_file_one(o, ren1_dst, null_sha1, 0,
> +					if (merge_file_one(o, ren1_dst, null_sha1, 0,
>  							 ren1->pair->two->sha1, ren1->pair->two->mode,
>  							 dst_other.sha1, dst_other.mode,
> -							 branch1, branch2);
> +							 branch1, branch2, &mfi) < 0)
> +						return -1;
>  					output(o, 1, _("Adding merged %s"), ren1_dst);
>  					update_file(o, 0, mfi.sha, mfi.mode, ren1_dst);
>  					try_merge = 0;
> @@ -1622,9 +1628,10 @@ static int merge_content(struct merge_options *o,
>  		if (dir_in_way(path, !o->call_depth))
>  			df_conflict_remains = 1;
>  	}
> -	mfi = merge_file_special_markers(o, &one, &a, &b,
> +	if (merge_file_special_markers(o, &one, &a, &b,
>  					 o->branch1, path1,
> -					 o->branch2, path2);
> +					 o->branch2, path2, &mfi) < 0)
> +		return -1;
>  
>  	if (mfi.clean && !df_conflict_remains &&
>  	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode) {
Johannes Schindelin· Jul 1, 2016, 13:48 UTC · re: Junio C Hamano · lore

Re: [PATCH 5/9] merge-recursive: avoid returning a wholesale struct

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 7 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > It is technically allowed, as per C89, for functions' return type to
> > be complete structs (i.e. *not* just pointers to structs), but it is
> > bad practice.
> 
> Not necessarily.
Okay, I fixed it.
Show 10 quoted lines
> > This is a very late attempt to contain the damage done by this developer
> > in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which
> > introduced such a return type.
> >
> > It will also help the current effort to libify merge-recursive.c, as
> > it will allow us to return proper error codes later.
> 
> But this part of the motivation does make sense.  Having to pass an
> extra int &error_code field, only because the return value is
> already used for something else, is a lot more weird.

While I still think it is inelegant to return a whole struct (understanding the machine code aspects of it), I toned it down to say that we just do not use that construct in Git's source code. And I kept the paragraph that you found more convincing, of course.

Ciao, Dscho

P.S.: If it is not too much of a problem, may I ask you to simply delete remainders of my patches when replying and not commenting on them? I just deleted 226 lines after verifying that you really did not respond to any part of it in the unhelpful Alpine client (which I still use because it *still* fits much better with my workflow than anything else, by a lot). Again, not a big deal if it would make your life more painful.

Eric Wong· Jul 1, 2016, 15:02 UTC · re: Johannes Schindelin · lore

Re: [PATCH 5/9] merge-recursive: avoid returning a wholesale struct

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 6 quoted lines
> P.S.: If it is not too much of a problem, may I ask you to simply delete
> remainders of my patches when replying and not commenting on them? I just
> deleted 226 lines after verifying that you really did not respond to any
> part of it in the unhelpful Alpine client (which I still use because it
> *still* fits much better with my workflow than anything else, by a lot).
> Again, not a big deal if it would make your life more painful.
I find the over-quoting annoying, too.  Fwiw, my muttrc has:
bind pager , skip-quoted

Which allows me to hit the comma key to skip over quoted text. Not sure if Alpine or other clients have something similar. But I'd rather not waste the keystroke/bandwidth/storage.

Johannes Schindelin· Jun 29, 2016, 11:37 UTC · re: Johannes Schindelin · lore

[PATCH 8/9] merge-recursive: switch to returning errors instead of dying

The recursive merge machinery is supposed to be a library function, i.e. it should return an error when it fails. Originally the functions were part of the builtin "merge-recursive", though, where it was simpler to call die() and be done with error handling.

The existing callers were already prepared to detect negative return values to indicate errors and to behave as previously: exit with code 128 (which is the same thing that die() does, after printing the message).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 55 ++++++++++++++++++++++++++++++-------------------------
 1 file changed, 30 insertions(+), 25 deletions(-)
Show changes to merge-recursive.c +30 −25
diff --git a/merge-recursive.c b/merge-recursive.c
index bb075e3..d5a593c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -710,12 +710,10 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
-		if (status == SCLD_EXISTS) {
+		if (status == SCLD_EXISTS)
 			/* something else exists */
-			error(msg, path, _(": perhaps a D/F conflict?"));
-			return -1;
-		}
-		die(msg, path, "");
+			return error(msg, path, _(": perhaps a D/F conflict?"));
+		return error(msg, path, "");
 	}
 
 	/*
@@ -743,6 +741,8 @@ static int update_file_flags(struct merge_options *o,
 			      int update_cache,
 			      int update_wd)
 {
+	int ret = 0;
+
 	if (o->call_depth)
 		update_wd = 0;
 
@@ -763,9 +763,11 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(sha, &type, &size);
 		if (!buf)
-			die(_("cannot read object %s '%s'"), sha1_to_hex(sha), path);
-		if (type != OBJ_BLOB)
-			die(_("blob expected for %s '%s'"), sha1_to_hex(sha), path);
+			return error(_("cannot read object %s '%s'"), sha1_to_hex(sha), path);
+		if (type != OBJ_BLOB) {
+			ret = error(_("blob expected for %s '%s'"), sha1_to_hex(sha), path);
+			goto free_buf;
+		}
 		if (S_ISREG(mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
 			if (convert_to_working_tree(path, buf, size, &strbuf)) {
@@ -786,8 +788,10 @@ static int update_file_flags(struct merge_options *o,
 			else
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-			if (fd < 0)
-				die_errno(_("failed to open '%s'"), path);
+			if (fd < 0) {
+				ret = error_errno(_("failed to open '%s'"), path);
+				goto free_buf;
+			}
 			write_in_full(fd, buf, size);
 			close(fd);
 		} else if (S_ISLNK(mode)) {
@@ -795,18 +799,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				die_errno(_("failed to symlink '%s'"), path);
+				ret = error_errno(_("failed to symlink '%s'"), path);
 			free(lnk);
 		} else
-			die(_("do not know what to do with %06o %s '%s'"),
-			    mode, sha1_to_hex(sha), path);
+			ret = error_errno(_("do not know what to do with %06o %s '%s'"),
+				mode, sha1_to_hex(sha), path);
 free_buf:
 		free(buf);
 	}
  update_index:
-	if (update_cache)
+	if (!ret && update_cache)
 		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
-	return 0;
+	return ret;
 }
 
 static int update_file(struct merge_options *o,
@@ -932,20 +936,22 @@ static int merge_file_1(struct merge_options *o,
 			hashcpy(result->sha, a->sha1);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
-			int merge_status;
+			int ret = 0, merge_status;
 
 			merge_status = merge_3way(o, &result_buf, one, a, b,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				die(_("Failed to execute internal merge"));
+				ret = error(_("Failed to execute internal merge"));
 
-			if (write_sha1_file(result_buf.ptr, result_buf.size,
+			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 					    blob_type, result->sha))
-				die(_("Unable to add %s to database"),
-				    a->path);
+				ret = error(_("Unable to add %s to database"),
+					a->path);
 
 			free(result_buf.ptr);
+			if (ret)
+				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
 			result->clean = merge_submodule(result->sha,
@@ -1861,7 +1867,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("Fatal merge failure, shouldn't happen."));
+		return error(_("Fatal merge failure, shouldn't happen."));
 
 	return clean_merge;
 }
@@ -1889,11 +1895,10 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			die(_("merging of trees %s and %s failed"),
+			error(_("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
-		else
-			exit(128);
+		return -1;
 	}
 
 	if (unmerged_cache()) {
@@ -2024,7 +2029,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			die(_("merge returned no commit"));
+			return error(_("merge returned no commit"));
 	}
 
 	discard_cache();
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 21:19 UTC · re: Johannes Schindelin · lore

Re: [PATCH 8/9] merge-recursive: switch to returning errors instead of dying

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 7 quoted lines
> @@ -743,6 +741,8 @@ static int update_file_flags(struct merge_options *o,
>  			      int update_cache,
>  			      int update_wd)
> ...
> +			ret = error_errno(_("do not know what to do with %06o %s '%s'"),
> +				mode, sha1_to_hex(sha), path);
>  free_buf:

OK, with a few more users of this label, it no longer looks so strange to me to have this label here.

At least, match the indentation level to the existing one we see below, though?

Show 6 quoted lines
>  		free(buf);
>  	}
>   update_index:
> -	if (update_cache)
> +	if (!ret && update_cache)
>  		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);

Unlike my reaction to "if (!ret) do this" in an earlier patch, I do think this change is sensible. We wouldn't have reached here in the original code if we saw errors, and some of the variable used to call add_cacheinfo() at this point may be garbage when ret is non-zero, i.e. we know we earlier saw an error.

Show 11 quoted lines
> -	return 0;
> +	return ret;
>  }
> ...
>  			if ((merge_status < 0) || !result_buf.ptr)
> -				die(_("Failed to execute internal merge"));
> +				ret = error(_("Failed to execute internal merge"));
>  
> -			if (write_sha1_file(result_buf.ptr, result_buf.size,
> +			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
>  					    blob_type, result->sha))
Likewise.
Show 6 quoted lines
> @@ -1861,7 +1867,7 @@ static int process_entry(struct merge_options *o,
>  		 */
>  		remove_file(o, 1, path, !a_mode);
>  	} else
> -		die(_("Fatal merge failure, shouldn't happen."));
> +		return error(_("Fatal merge failure, shouldn't happen."));
Isn't this BUG()?
Johannes Schindelin· Jul 1, 2016, 11:14 UTC · re: Junio C Hamano · lore

Re: [PATCH 8/9] merge-recursive: switch to returning errors instead of dying

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 15 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -743,6 +741,8 @@ static int update_file_flags(struct merge_options *o,
> >  			      int update_cache,
> >  			      int update_wd)
> > ...
> > +			ret = error_errno(_("do not know what to do with %06o %s '%s'"),
> > +				mode, sha1_to_hex(sha), path);
> >  free_buf:
> 
> OK, with a few more users of this label, it no longer looks so
> strange to me to have this label here.
> 
> At least, match the indentation level to the existing one we see
> below, though?
Okay, fixed.

Although I have to say that I do not quite understand the meaning of the one-space indentation, in particular since the later "error_return" label lacks it.

Show 12 quoted lines
> >  		free(buf);
> >  	}
> >   update_index:
> > [...]
> > @@ -1861,7 +1867,7 @@ static int process_entry(struct merge_options *o,
> >  		 */
> >  		remove_file(o, 1, path, !a_mode);
> >  	} else
> > -		die(_("Fatal merge failure, shouldn't happen."));
> > +		return error(_("Fatal merge failure, shouldn't happen."));
> 
> Isn't this BUG()?
I guess it is! Fixed.

Ciao, Dscho

Johannes Schindelin· Jun 29, 2016, 11:38 UTC · re: Johannes Schindelin · lore

[PATCH 9/9] am: make a direct call to merge_recursive

From: Junio C Hamano <gitster@pobox.com>

Instead of spawning merge-recursive via run_command() in run_fallback_merge_recursive(), make a direct call to the internal merge_recursive_generic().

Here is a quick benchmark result, applying a patch for b4391657 (merge: drop 'git merge <message> HEAD <commit>' syntax, 2015-03-25) that was still cooking in 'next' on 4b1fd356 (git-multimail: update to release 1.2.0, 2015-10-11) which was the tip of 'master' at some stage, on an x86-64 running Ubuntu:

      real    0m0.169s                      real    0m0.163s
      user    0m0.108s                      user    0m0.134s
      sys     0m0.068s                      sys     0m0.033s
      real    0m0.175s                      real    0m0.161s
      user    0m0.110s                      user    0m0.120s
      sys     0m0.066s                      sys     0m0.047s
      real    0m0.168s                      real    0m0.162s
      user    0m0.124s                      user    0m0.114s
      sys     0m0.045s                      sys     0m0.051s
      real    0m0.167s                      real    0m0.152s
      user    0m0.124s                      user    0m0.122s
      sys     0m0.045s                      sys     0m0.031s
      real    0m0.169s                      real    0m0.164s
      user    0m0.131s                      user    0m0.129s
      sys     0m0.043s                      sys     0m0.041s

Left-hand side shows the original, right-hand side shows the result of this optimization.

Timings on Windows:

original: 0.00user 0.01system 0:00.29elapsed 0.00user 0.00system 0:00.25elapsed 0.01user 0.00system 0:00.24elapsed 0.01user 0.00system 0:00.26elapsed 0.00user 0.01system 0:00.23elapsed

with optimization: 0.00user 0.01system 0:00.22elapsed 0.00user 0.00system 0:00.25elapsed 0.00user 0.01system 0:00.22elapsed 0.00user 0.00system 0:00.22elapsed 0.01user 0.00system 0:00.21elapsed

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
	It feels *slightly* wrong to submit your own patch to review,
	however, please keep in mind that
	1) I changed the patch (o.gently does not exist anymore, so I do
	   not set it), and
	2) I added my own timings performed on Windows.
 builtin/am.c | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)
Show changes to builtin/am.c +14 −13
diff --git a/builtin/am.c b/builtin/am.c
index 3dfe70b..dd41154 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1587,25 +1587,26 @@ static int run_fallback_merge_recursive(const struct am_state *state,
 					unsigned char *our_tree,
 					unsigned char *his_tree)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
+	const unsigned char *bases[1] = {orig_tree};
+	struct merge_options o;
+	struct commit *result;
+	char *his_tree_name;
 	int status;
 
-	cp.git_cmd = 1;
+	init_merge_options(&o);
+
+	o.branch1 = "HEAD";
+	his_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
+	o.branch2 = his_tree_name;
 
-	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(his_tree), linelen(state->msg), state->msg);
 	if (state->quiet)
-		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
+		o.verbosity = 0;
 
-	argv_array_push(&cp.args, "merge-recursive");
-	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
-	argv_array_push(&cp.args, "--");
-	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(his_tree));
+	status = merge_recursive_generic(&o, our_tree, his_tree, 1, bases, &result);
+	if (status < 0)
+		exit(128);
+	free(his_tree_name);
 
-	status = run_command(&cp) ? (-1) : 0;
-	discard_cache();
-	read_cache();
 	return status;
 }
 
-- 
2.9.0.268.gcabc8b0
Junio C Hamano· Jun 29, 2016, 17:45 UTC · re: Johannes Schindelin · lore

Re: [PATCH 9/9] am: make a direct call to merge_recursive

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> From: Junio C Hamano <gitster@pobox.com>
Did I write this thing?

Having two sets of numbers that illustrated that this is not really a useful optimization in the bigger picture looks vaguely familiar (e.g. $gmane/279417), but the numbers are different.

Show 7 quoted lines
> 	It feels *slightly* wrong to submit your own patch to review,
> 	however, please keep in mind that
>
> 	1) I changed the patch (o.gently does not exist anymore, so I do
> 	   not set it), and
>
> 	2) I added my own timings performed on Windows.

It probably is much less confusing if you take the authorship, possibly with a passing reference to whatever I wrote as the source of inspiration in the log message, or something. I do not think I deserve a credit in this 9-patch series.

I haven't read the other 8 patches, so I cannot comment on it yet.
Show 46 quoted lines
>  builtin/am.c | 27 ++++++++++++++-------------
>  1 file changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/builtin/am.c b/builtin/am.c
> index 3dfe70b..dd41154 100644
> --- a/builtin/am.c
> +++ b/builtin/am.c
> @@ -1587,25 +1587,26 @@ static int run_fallback_merge_recursive(const struct am_state *state,
>  					unsigned char *our_tree,
>  					unsigned char *his_tree)
>  {
> -	struct child_process cp = CHILD_PROCESS_INIT;
> +	const unsigned char *bases[1] = {orig_tree};
> +	struct merge_options o;
> +	struct commit *result;
> +	char *his_tree_name;
>  	int status;
>  
> -	cp.git_cmd = 1;
> +	init_merge_options(&o);
> +
> +	o.branch1 = "HEAD";
> +	his_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
> +	o.branch2 = his_tree_name;
>  
> -	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
> -			 sha1_to_hex(his_tree), linelen(state->msg), state->msg);
>  	if (state->quiet)
> -		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
> +		o.verbosity = 0;
>  
> -	argv_array_push(&cp.args, "merge-recursive");
> -	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
> -	argv_array_push(&cp.args, "--");
> -	argv_array_push(&cp.args, sha1_to_hex(our_tree));
> -	argv_array_push(&cp.args, sha1_to_hex(his_tree));
> +	status = merge_recursive_generic(&o, our_tree, his_tree, 1, bases, &result);
> +	if (status < 0)
> +		exit(128);
> +	free(his_tree_name);
>  
> -	status = run_command(&cp) ? (-1) : 0;
> -	discard_cache();
> -	read_cache();
>  	return status;
>  }
Johannes Schindelin· Jun 30, 2016, 08:38 UTC · re: Junio C Hamano · lore

Re: [PATCH 9/9] am: make a direct call to merge_recursive

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 5 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > From: Junio C Hamano <gitster@pobox.com>
> 
> Did I write this thing?

Yes, you did. It was db05d6194d3f9ea9e64163944961d5f6e85302be as part of pu@{2016-06-15}.

> Having two sets of numbers that illustrated that this is not really
> a useful optimization in the bigger picture looks vaguely familiar
> (e.g. $gmane/279417), but the numbers are different.

I beg to differ. While the performance improvement is not huge, even your toy experiment shows it is significant, i.e. noticeable.

Show 12 quoted lines
> > 	It feels *slightly* wrong to submit your own patch to review,
> > 	however, please keep in mind that
> >
> > 	1) I changed the patch (o.gently does not exist anymore, so I do
> > 	   not set it), and
> >
> > 	2) I added my own timings performed on Windows.
> 
> It probably is much less confusing if you take the authorship,
> possibly with a passing reference to whatever I wrote as the source
> of inspiration in the log message, or something.  I do not think I
> deserve a credit in this 9-patch series.

The patch is an almost verbatim copy of db05d6194, I only removed the now-obsolete "gently" setting, is all.

That means that I feel really, really uneasy about claiming authorship because I did not write it.

If you really want me to, I will take custody of this patch and rewrite the commit message as well using --reset--author, of course.

Ciao, Dscho

Junio C Hamano· Jul 1, 2016, 16:03 UTC · re: Johannes Schindelin · lore

Re: [PATCH 9/9] am: make a direct call to merge_recursive

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 10 quoted lines
> On Wed, 29 Jun 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > From: Junio C Hamano <gitster@pobox.com>
>> 
>> Did I write this thing?
>
> Yes, you did. It was db05d6194d3f9ea9e64163944961d5f6e85302be as part of
> pu@{2016-06-15}.

Ah, OK, that is quite old, dating from Oct 2015. Thanks for a pointer.

Junio C Hamano· Jun 29, 2016, 21:23 UTC · re: Johannes Schindelin · lore

Re: [PATCH 9/9] am: make a direct call to merge_recursive

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 28 quoted lines
> -	cp.git_cmd = 1;
> +	init_merge_options(&o);
> +
> +	o.branch1 = "HEAD";
> +	his_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
> +	o.branch2 = his_tree_name;
>  
> -	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
> -			 sha1_to_hex(his_tree), linelen(state->msg), state->msg);
>  	if (state->quiet)
> -		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
> +		o.verbosity = 0;
>  
> -	argv_array_push(&cp.args, "merge-recursive");
> -	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
> -	argv_array_push(&cp.args, "--");
> -	argv_array_push(&cp.args, sha1_to_hex(our_tree));
> -	argv_array_push(&cp.args, sha1_to_hex(his_tree));
> +	status = merge_recursive_generic(&o, our_tree, his_tree, 1, bases, &result);
> +	if (status < 0)
> +		exit(128);
> +	free(his_tree_name);
>  
> -	status = run_command(&cp) ? (-1) : 0;
> -	discard_cache();
> -	read_cache();
>  	return status;
>  }
Is this a correct conversion?

We used to prepare the command line and called run_command() and without dying returned status with the error status from the merge-recursive that was spawned by run_command().

The new code does not report failure to the caller and instead dies.
Johannes Schindelin· Jul 1, 2016, 12:46 UTC · re: Junio C Hamano · lore

Re: [PATCH 9/9] am: make a direct call to merge_recursive

Hi Junio,
On Wed, 29 Jun 2016, Junio C Hamano wrote:
Show 38 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > -	cp.git_cmd = 1;
> > +	init_merge_options(&o);
> > +
> > +	o.branch1 = "HEAD";
> > +	his_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
> > +	o.branch2 = his_tree_name;
> >  
> > -	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
> > -			 sha1_to_hex(his_tree), linelen(state->msg), state->msg);
> >  	if (state->quiet)
> > -		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
> > +		o.verbosity = 0;
> >  
> > -	argv_array_push(&cp.args, "merge-recursive");
> > -	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
> > -	argv_array_push(&cp.args, "--");
> > -	argv_array_push(&cp.args, sha1_to_hex(our_tree));
> > -	argv_array_push(&cp.args, sha1_to_hex(his_tree));
> > +	status = merge_recursive_generic(&o, our_tree, his_tree, 1, bases, &result);
> > +	if (status < 0)
> > +		exit(128);
> > +	free(his_tree_name);
> >  
> > -	status = run_command(&cp) ? (-1) : 0;
> > -	discard_cache();
> > -	read_cache();
> >  	return status;
> >  }
> 
> Is this a correct conversion?
> 
> We used to prepare the command line and called run_command() and
> without dying returned status with the error status from the
> merge-recursive that was spawned by run_command().
> 
> The new code does not report failure to the caller and instead dies.
True, this is incorrect.

I took a step back and realized that the most appropriate course of action would be to revert the commit that calls run_command() to begin with. This also solves the authorship issue.

And while at it, I also noticed another sore to my eye that I had noticed repeatedly and now fix as part of this patch series.

Thanks, Dscho

Johannes Schindelin· Jul 5, 2016, 11:22 UTC · re: Johannes Schindelin · lore

[PATCH v2 00/17] Use merge_recursive() directly in the builtin am

This is the second iteration of the long-awaited re-roll of the attempt to avoid spawning merge-recursive from the builtin am and use merge_recursive() directly instead.

The *real* reason for the reroll is that I need a libified recursive merge to accelerate the interactive rebase by teaching the sequencer to do rebase -i's grunt work.

In this endeavor, we need to be extra careful to retain backwards compatibility. The test script t6022-merge-rename.sh, for example, verifies that `git pull` exits with status 128 in case of a fatal error. To that end, we need to make sure that fatal errors are handled by existing (builtin) users via exit(128) (or die(), which calls exit(128) at the end). New users (such as a builtin helper doing rebase -i's grunt work) may want to print some helpful advice what happened and how to get out of this mess before erroring out.

The changes relative to the first iteration of this patch series:
- a variable that could be used uninitialized is now initialized (thanks,
  Travis & clang!)
- several commit messages were touched up (and I hope y'all agree, improved)
- an unnecessary hunk was reverted (this was a left-over from an
  unpublished iteration that needed to retain return values faithfully, i.e.
  it made a difference between -1 and -128 as error value)
- Junio's patch to use recursive_merge() directly in the builtin am was
  replaced by a different solution
- an error message was identified as, and converted into, a bug report
  instead
- the code in was_tracked() now avoids a loop when it is unnecessary,
  and further clarifies why we keep looking when cache_name_pos() did
  not find the entry we asked for
- die("BUG: ...") statements are no longer translated
- one die("BUG: ...") report that continued in upper-case after the "BUG:"
  prefix was fixed
- I addressed a gender bias that has been bugging me ever since I noticed it
- recursive merge's error messages are now printed after flushing the
  output buffer (instead of swallowing that output)
- callers of the recursive merge can now ask that the output buffer not be
  flushed, but retained without printing it instead. This gives the caller the
  control about handling errors which Junio asked for.
- some long-standing bugs have been recognized and addressed:
  - when the recursive merge failed, it lost the buffered output
  - the output buffer of the recursive merge was never released
  - some stdout/stderr interference that we tried to address in 2007 is
    now fully addressed (the progress output could be printed in the
    middle of the commit title because the latter was still directly printed
    to stdout, which is buffered, instead of being buffered and flushed)
- a lot of unnecessary 'ret' variables are gone now: originally, I wanted to
  retain the *exact* return value, but now errors are indicated by -1,
  always
- lastly, I remembered that my original attempt at fixing the pull --rebase
  issue contained a test case, and I forward-ported that, and augmented it

So while I addressed all comments, I also went through the patch series a couple of times myself and whatever bugged me, I tried to resolve, too.

This patch series touches rather important code. Now that I addressed concerns such as fixing translated bug reports, I would appreciate thorough reviews with a focus on the critical parts of the code, those that could result in regressions.

Johannes Schindelin (17):
  Verify that `git pull --rebase` shows the helpful advice when failing
  Report bugs consistently
  Avoid translating bug messages
  merge-recursive: clarify code in was_tracked()
  Prepare the builtins for a libified merge_recursive()
  merge_recursive: abort properly upon errors
  merge-recursive: avoid returning a wholesale struct
  merge-recursive: allow write_tree_from_memory() to error out
  merge-recursive: handle return values indicating errors
  merge-recursive: switch to returning errors instead of dying
  am: counteract gender bias
  am -3: use merge_recursive() directly again
  merge-recursive: flush output buffer before printing error messages
  merge-recursive: write the commit title in one go
  merge-recursive: offer an option to retain the output in 'obuf'
  Ensure that the output buffer is released after calling merge_trees()
  merge-recursive: flush output buffer even when erroring out
 builtin/am.c           |  55 ++----
 builtin/checkout.c     |   5 +-
 builtin/ls-files.c     |   3 +-
 builtin/merge.c        |   2 +
 builtin/update-index.c |   2 +-
 grep.c                 |   8 +-
 imap-send.c            |   4 +-
 merge-recursive.c      | 490 +++++++++++++++++++++++++++++--------------------
 merge-recursive.h      |   2 +-
 sequencer.c            |   5 +
 sha1_file.c            |   4 +-
 t/t5520-pull.sh        |  30 +++
 trailer.c              |   2 +-
 transport.c            |   2 +-
 wt-status.c            |   4 +-
 15 files changed, 369 insertions(+), 249 deletions(-)
Published-As: https://github.com/dscho/git/releases/tag/am-3-merge-recursive-direct-v2
Interdiff vs v1:
 diff --git a/builtin/am.c b/builtin/am.c
 index dd41154..be652f9 100644
 --- a/builtin/am.c
 +++ b/builtin/am.c
 @@ -1578,45 +1578,16 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
  }
  
  /**
 - * Do the three-way merge using fake ancestor, his tree constructed
 - * from the fake ancestor and the postimage of the patch, and our
 - * state.
 - */
 -static int run_fallback_merge_recursive(const struct am_state *state,
 -					unsigned char *orig_tree,
 -					unsigned char *our_tree,
 -					unsigned char *his_tree)
 -{
 -	const unsigned char *bases[1] = {orig_tree};
 -	struct merge_options o;
 -	struct commit *result;
 -	char *his_tree_name;
 -	int status;
 -
 -	init_merge_options(&o);
 -
 -	o.branch1 = "HEAD";
 -	his_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
 -	o.branch2 = his_tree_name;
 -
 -	if (state->quiet)
 -		o.verbosity = 0;
 -
 -	status = merge_recursive_generic(&o, our_tree, his_tree, 1, bases, &result);
 -	if (status < 0)
 -		exit(128);
 -	free(his_tree_name);
 -
 -	return status;
 -}
 -
 -/**
   * Attempt a threeway merge, using index_path as the temporary index.
   */
  static int fall_back_threeway(const struct am_state *state, const char *index_path)
  {
 -	unsigned char orig_tree[GIT_SHA1_RAWSZ], his_tree[GIT_SHA1_RAWSZ],
 +	unsigned char orig_tree[GIT_SHA1_RAWSZ], her_tree[GIT_SHA1_RAWSZ],
  		      our_tree[GIT_SHA1_RAWSZ];
 +	const unsigned char *bases[1] = {orig_tree};
 +	struct merge_options o;
 +	struct commit *result;
 +	char *her_tree_name;
  
  	if (get_sha1("HEAD", our_tree) < 0)
  		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
 @@ -1652,7 +1623,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  		return error(_("Did you hand edit your patch?\n"
  				"It does not apply to blobs recorded in its index."));
  
 -	if (write_index_as_tree(his_tree, &the_index, index_path, 0, NULL))
 +	if (write_index_as_tree(her_tree, &the_index, index_path, 0, NULL))
  		return error("could not write tree");
  
  	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
 @@ -1662,17 +1633,28 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  
  	/*
  	 * This is not so wrong. Depending on which base we picked, orig_tree
 -	 * may be wildly different from ours, but his_tree has the same set of
 +	 * may be wildly different from ours, but her_tree has the same set of
  	 * wildly different changes in parts the patch did not touch, so
  	 * recursive ends up canceling them, saying that we reverted all those
  	 * changes.
  	 */
  
 -	if (run_fallback_merge_recursive(state, orig_tree, our_tree, his_tree)) {
 +	init_merge_options(&o);
 +
 +	o.branch1 = "HEAD";
 +	her_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
 +	o.branch2 = her_tree_name;
 +
 +	if (state->quiet)
 +		o.verbosity = 0;
 +
 +	if (merge_recursive_generic(&o, our_tree, her_tree, 1, bases, &result)) {
  		rerere(state->allow_rerere_autoupdate);
 +		free(her_tree_name);
  		return error(_("Failed to merge in the changes."));
  	}
  
 +	free(her_tree_name);
  	return 0;
  }
  
 diff --git a/builtin/checkout.c b/builtin/checkout.c
 index 14312f7..ced4ac4 100644
 --- a/builtin/checkout.c
 +++ b/builtin/checkout.c
 @@ -573,6 +573,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
  				exit(128);
  			ret = reset_tree(new->commit->tree, opts, 0,
  					 writeout_error);
 +			strbuf_release(&o.obuf);
  			if (ret)
  				return ret;
  		}
 diff --git a/builtin/merge.c b/builtin/merge.c
 index 133b853..7b898db 100644
 --- a/builtin/merge.c
 +++ b/builtin/merge.c
 @@ -1552,8 +1552,6 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
  		ret = try_merge_strategy(use_strategies[i]->name,
  					 common, remoteheads,
  					 head_commit, head_arg);
 -		if (ret < 0)
 -			exit(128);
  		if (!option_commit && !ret) {
  			merge_was_ok = 1;
  			/*
 diff --git a/imap-send.c b/imap-send.c
 index cd39805..369f72a 100644
 --- a/imap-send.c
 +++ b/imap-send.c
 @@ -506,7 +506,7 @@ static char *next_arg(char **s)
  
  static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
  {
 -	int ret;
 +	int ret = -1;
  	va_list va;
  
  	va_start(va, fmt);
 diff --git a/merge-recursive.c b/merge-recursive.c
 index d5a593c..d94f853 100644
 --- a/merge-recursive.c
 +++ b/merge-recursive.c
 @@ -23,6 +23,37 @@
  #include "dir.h"
  #include "submodule.h"
  
 +static void flush_output(struct merge_options *o)
 +{
 +	if (o->buffer_output < 2 && o->obuf.len) {
 +		fputs(o->obuf.buf, stdout);
 +		strbuf_reset(&o->obuf);
 +	}
 +}
 +
 +static int err(struct merge_options *o, const char *err, ...)
 +{
 +	va_list params;
 +
 +	va_start(params, err);
 +	if (o->buffer_output < 2)
 +		flush_output(o);
 +	else {
 +		strbuf_complete(&o->obuf, '\n');
 +		strbuf_addstr(&o->obuf, "error: ");
 +	}
 +	strbuf_vaddf(&o->obuf, err, params);
 +	if (o->buffer_output > 1)
 +		strbuf_addch(&o->obuf, '\n');
 +	else {
 +		error("%s", o->obuf.buf);
 +		strbuf_reset(&o->obuf);
 +	}
 +	va_end(params);
 +
 +	return -1;
 +}
 +
  static struct tree *shift_tree_object(struct tree *one, struct tree *two,
  				      const char *subtree_shift)
  {
 @@ -148,14 +179,6 @@ static int show(struct merge_options *o, int v)
  	return (!o->call_depth && o->verbosity >= v) || o->verbosity >= 5;
  }
  
 -static void flush_output(struct merge_options *o)
 -{
 -	if (o->obuf.len) {
 -		fputs(o->obuf.buf, stdout);
 -		strbuf_reset(&o->obuf);
 -	}
 -}
 -
  __attribute__((format (printf, 3, 4)))
  static void output(struct merge_options *o, int v, const char *fmt, ...)
  {
 @@ -177,28 +200,30 @@ static void output(struct merge_options *o, int v, const char *fmt, ...)
  
  static void output_commit_title(struct merge_options *o, struct commit *commit)
  {
 -	int i;
 -	flush_output(o);
 -	for (i = o->call_depth; i--;)
 -		fputs("  ", stdout);
 +	strbuf_addchars(&o->obuf, ' ', o->call_depth * 2);
  	if (commit->util)
 -		printf("virtual %s\n", merge_remote_util(commit)->name);
 +		strbuf_addf(&o->obuf, "virtual %s\n",
 +			merge_remote_util(commit)->name);
  	else {
 -		printf("%s ", find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV));
 +		strbuf_addf(&o->obuf, "%s ",
 +			find_unique_abbrev(commit->object.oid.hash,
 +				DEFAULT_ABBREV));
  		if (parse_commit(commit) != 0)
 -			printf(_("(bad commit)\n"));
 +			strbuf_addf(&o->obuf, _("(bad commit)\n"));
  		else {
  			const char *title;
  			const char *msg = get_commit_buffer(commit, NULL);
  			int len = find_commit_subject(msg, &title);
  			if (len)
 -				printf("%.*s\n", len, title);
 +				strbuf_addf(&o->obuf, "%.*s\n", len, title);
  			unuse_commit_buffer(commit, msg);
  		}
  	}
 +	flush_output(o);
  }
  
 -static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,
 +static int add_cacheinfo(struct merge_options *o,
 +		unsigned int mode, const unsigned char *sha1,
  		const char *path, int stage, int refresh, int options)
  {
  	struct cache_entry *ce;
 @@ -206,7 +231,7 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,
  			      (refresh ? (CE_MATCH_REFRESH |
  					  CE_MATCH_IGNORE_MISSING) : 0 ));
  	if (!ce)
 -		return error(_("addinfo_cache failed for path '%s'"), path);
 +		return err(o, _("addinfo_cache failed for path '%s'"), path);
  	return add_cache_entry(ce, options);
  }
  
 @@ -267,7 +292,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
  
  	if (!cache_tree_fully_valid(active_cache_tree) &&
  	    cache_tree_update(&the_index, 0) < 0) {
 -		error(_("error building trees"));
 +		err(o, _("error building trees"));
  		return NULL;
  	}
  
 @@ -535,7 +560,8 @@ static struct string_list *get_renames(struct merge_options *o,
  	return renames;
  }
  
 -static int update_stages(const char *path, const struct diff_filespec *o,
 +static int update_stages(struct merge_options *opt, const char *path,
 +			 const struct diff_filespec *o,
  			 const struct diff_filespec *a,
  			 const struct diff_filespec *b)
  {
 @@ -550,17 +576,19 @@ static int update_stages(const char *path, const struct diff_filespec *o,
  	 */
  	int clear = 1;
  	int options = ADD_CACHE_OK_TO_ADD | ADD_CACHE_SKIP_DFCHECK;
 -	int ret = 0;
 -
  	if (clear)
 -		ret = remove_file_from_cache(path);
 -	if (!ret && o)
 -		ret = add_cacheinfo(o->mode, o->sha1, path, 1, 0, options);
 -	if (!ret && a)
 -		ret = add_cacheinfo(a->mode, a->sha1, path, 2, 0, options);
 -	if (!ret && b)
 -		ret = add_cacheinfo(b->mode, b->sha1, path, 3, 0, options);
 -	return ret;
 +		if (remove_file_from_cache(path))
 +			return -1;
 +	if (o)
 +		if (add_cacheinfo(opt, o->mode, o->sha1, path, 1, 0, options))
 +			return -1;
 +	if (a)
 +		if (add_cacheinfo(opt, a->mode, a->sha1, path, 2, 0, options))
 +			return -1;
 +	if (b)
 +		if (add_cacheinfo(opt, b->mode, b->sha1, path, 3, 0, options))
 +			return -1;
 +	return 0;
  }
  
  static void update_entry(struct stage_data *entry,
 @@ -658,24 +686,22 @@ static int was_tracked(const char *path)
  {
  	int pos = cache_name_pos(path, strlen(path));
  
 -	/* cache_name_pos() looks for stage == 0, so pos may be < 0 */
 -	if (pos < 0)
 -		pos = -1 - pos;
 -	while (pos < active_nr &&
 -	       !strcmp(path, active_cache[pos]->name)) {
 +	if (pos >= 0)
 +		return pos < active_nr;
 +	/*
 +	 * cache_name_pos() looks for stage == 0, even if we did not ask for
 +	 * it. Let's look for stage == 2 now.
 +	 */
 +	for (pos = -1 - pos; pos < active_nr &&
 +	     !strcmp(path, active_cache[pos]->name); pos++)
  		/*
  		 * If stage #0, it is definitely tracked.
  		 * If it has stage #2 then it was tracked
  		 * before this merge started.  All other
  		 * cases the path was not tracked.
  		 */
 -		switch (ce_stage(active_cache[pos])) {
 -		case 0:
 -		case 2:
 +		if (ce_stage(active_cache[pos]) == 2)
  			return 1;
 -		}
 -		pos++;
 -	}
  	return 0;
  }
  
 @@ -712,8 +738,8 @@ static int make_room_for_path(struct merge_options *o, const char *path)
  	if (status) {
  		if (status == SCLD_EXISTS)
  			/* something else exists */
 -			return error(msg, path, _(": perhaps a D/F conflict?"));
 -		return error(msg, path, "");
 +			return err(o, msg, path, _(": perhaps a D/F conflict?"));
 +		return err(o, msg, path, "");
  	}
  
  	/*
 @@ -721,7 +747,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
  	 * tracking it.
  	 */
  	if (would_lose_untracked(path))
 -		return error(_("refusing to lose untracked file at '%s'"),
 +		return err(o, _("refusing to lose untracked file at '%s'"),
  			     path);
  
  	/* Successful unlink is good.. */
 @@ -731,7 +757,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
  	if (errno == ENOENT)
  		return 0;
  	/* .. but not some other error (who really cares what?) */
 -	return error(msg, path, _(": perhaps a D/F conflict?"));
 +	return err(o, msg, path, _(": perhaps a D/F conflict?"));
  }
  
  static int update_file_flags(struct merge_options *o,
 @@ -763,9 +789,9 @@ static int update_file_flags(struct merge_options *o,
  
  		buf = read_sha1_file(sha, &type, &size);
  		if (!buf)
 -			return error(_("cannot read object %s '%s'"), sha1_to_hex(sha), path);
 +			return err(o, _("cannot read object %s '%s'"), sha1_to_hex(sha), path);
  		if (type != OBJ_BLOB) {
 -			ret = error(_("blob expected for %s '%s'"), sha1_to_hex(sha), path);
 +			ret = err(o, _("blob expected for %s '%s'"), sha1_to_hex(sha), path);
  			goto free_buf;
  		}
  		if (S_ISREG(mode)) {
 @@ -789,7 +815,8 @@ static int update_file_flags(struct merge_options *o,
  				mode = 0666;
  			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
  			if (fd < 0) {
 -				ret = error_errno(_("failed to open '%s'"), path);
 +				ret = err(o, _("failed to open '%s': %s"),
 +					path, strerror(errno));
  				goto free_buf;
  			}
  			write_in_full(fd, buf, size);
 @@ -799,17 +826,18 @@ static int update_file_flags(struct merge_options *o,
  			safe_create_leading_directories_const(path);
  			unlink(path);
  			if (symlink(lnk, path))
 -				ret = error_errno(_("failed to symlink '%s'"), path);
 +				ret = err(o, _("failed to symlink '%s': %s"),
 +					path, strerror(errno));
  			free(lnk);
  		} else
 -			ret = error_errno(_("do not know what to do with %06o %s '%s'"),
 +			ret = err(o, _("do not know what to do with %06o %s '%s'"),
  				mode, sha1_to_hex(sha), path);
 -free_buf:
 + free_buf:
  		free(buf);
  	}
   update_index:
  	if (!ret && update_cache)
 -		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
 +		add_cacheinfo(o, mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
  	return ret;
  }
  
 @@ -942,11 +970,11 @@ static int merge_file_1(struct merge_options *o,
  						  branch1, branch2);
  
  			if ((merge_status < 0) || !result_buf.ptr)
 -				ret = error(_("Failed to execute internal merge"));
 +				ret = err(o, _("Failed to execute internal merge"));
  
  			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
  					    blob_type, result->sha))
 -				ret = error(_("Unable to add %s to database"),
 +				ret = err(o, _("Unable to add %s to database"),
  					a->path);
  
  			free(result_buf.ptr);
 @@ -964,7 +992,7 @@ static int merge_file_1(struct merge_options *o,
  			if (!sha_eq(a->sha1, b->sha1))
  				result->clean = 0;
  		} else
 -			die(_("BUG: unsupported object type in the tree"));
 +			die("BUG: unsupported object type in the tree");
  	}
  
  	return 0;
 @@ -1090,7 +1118,6 @@ static int conflict_rename_delete(struct merge_options *o,
  	const unsigned char *b_sha = NULL;
  	int a_mode = 0;
  	int b_mode = 0;
 -	int ret = 0;
  
  	if (rename_branch == o->branch1) {
  		a_sha = dest->sha1;
 @@ -1100,22 +1127,19 @@ static int conflict_rename_delete(struct merge_options *o,
  		b_mode = dest->mode;
  	}
  
 -	ret = handle_change_delete(o,
 +	if (handle_change_delete(o,
  			     o->call_depth ? orig->path : dest->path,
  			     orig->sha1, orig->mode,
  			     a_sha, a_mode,
  			     b_sha, b_mode,
 -			     _("rename"), _("renamed"));
 -	if (ret < 0)
 -		return ret;
 +			     _("rename"), _("renamed")))
 +		return -1;
  	if (o->call_depth)
 -		ret = remove_file_from_cache(dest->path);
 +		return remove_file_from_cache(dest->path);
  	else
 -		ret = update_stages(dest->path, NULL,
 +		return update_stages(o, dest->path, NULL,
  			      rename_branch == o->branch1 ? dest : NULL,
  			      rename_branch == o->branch1 ? NULL : dest);
 -
 -	return ret;
  }
  
  static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 @@ -1156,8 +1180,8 @@ static int handle_file(struct merge_options *o,
  	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
  	if (add) {
  		char *add_name = unique_path(o, rename->path, other_branch);
 -		if ((ret = update_file(o, 0, add->sha1, add->mode, add_name)))
 -			return ret;
 +		if (update_file(o, 0, add->sha1, add->mode, add_name))
 +			return -1;
  
  		remove_file(o, 0, rename->path, 0);
  		dst_name = unique_path(o, rename->path, cur_branch);
 @@ -1171,9 +1195,9 @@ static int handle_file(struct merge_options *o,
  	if ((ret = update_file(o, 0, rename->sha1, rename->mode, dst_name)))
  		; /* fall through, do allow dst_name to be released */
  	else if (stage == 2)
 -		ret = update_stages(rename->path, NULL, rename, add);
 +		ret = update_stages(o, rename->path, NULL, rename, add);
  	else
 -		ret = update_stages(rename->path, NULL, add, rename);
 +		ret = update_stages(o, rename->path, NULL, add, rename);
  
  	if (dst_name != rename->path)
  		free(dst_name);
 @@ -1188,7 +1212,6 @@ static int conflict_rename_rename_1to2(struct merge_options *o,
  	struct diff_filespec *one = ci->pair1->one;
  	struct diff_filespec *a = ci->pair1->two;
  	struct diff_filespec *b = ci->pair2->two;
 -	int ret = 0;
  
  	output(o, 1, _("CONFLICT (rename/rename): "
  	       "Rename \"%s\"->\"%s\" in branch \"%s\" "
 @@ -1201,12 +1224,12 @@ static int conflict_rename_rename_1to2(struct merge_options *o,
  		struct diff_filespec other;
  		struct diff_filespec *add;
  
 -		if ((ret = merge_file_one(o, one->path,
 +		if (merge_file_one(o, one->path,
  				 one->sha1, one->mode,
  				 a->sha1, a->mode,
  				 b->sha1, b->mode,
 -				 ci->branch1, ci->branch2, &mfi)))
 -			return ret;
 +				 ci->branch1, ci->branch2, &mfi))
 +			return -1;
  
  		/*
  		 * FIXME: For rename/add-source conflicts (if we could detect
 @@ -1214,8 +1237,8 @@ static int conflict_rename_rename_1to2(struct merge_options *o,
  		 * pathname and then either rename the add-source file to that
  		 * unique path, or use that unique path instead of src here.
  		 */
 -		if ((ret = update_file(o, 0, mfi.sha, mfi.mode, one->path)))
 -			return ret;
 +		if (update_file(o, 0, mfi.sha, mfi.mode, one->path))
 +			return -1;
  
  		/*
  		 * Above, we put the merged content at the merge-base's
 @@ -1227,27 +1250,22 @@ static int conflict_rename_rename_1to2(struct merge_options *o,
  		 */
  		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
  		if (add) {
 -			if ((ret = update_file(o, 0, add->sha1, add->mode,
 -					a->path)))
 -				return ret;
 +			if (update_file(o, 0, add->sha1, add->mode, a->path))
 +				return -1;
  		}
  		else
  			remove_file_from_cache(a->path);
  		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
  		if (add) {
 -			if ((ret = update_file(o, 0, add->sha1, add->mode,
 -					b->path)))
 -				return ret;
 +			if (update_file(o, 0, add->sha1, add->mode, b->path))
 +				return -1;
  		}
  		else
  			remove_file_from_cache(b->path);
 -	} else {
 -		if ((ret = handle_file(o, a, 2, ci)) ||
 -		    (ret = handle_file(o, b, 3, ci)))
 -			return ret;
 -	}
 +	} else if (handle_file(o, a, 2, ci) || handle_file(o, b, 3, ci))
 +		return -1;
  
 -	return ret;
 +	return 0;
  }
  
  static int conflict_rename_rename_2to1(struct merge_options *o,
 @@ -1272,13 +1290,13 @@ static int conflict_rename_rename_2to1(struct merge_options *o,
  	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
  	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
  
 -	if ((ret = merge_file_special_markers(o, a, c1, &ci->ren1_other,
 +	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
  					    o->branch1, c1->path,
 -					    o->branch2, ci->ren1_other.path, &mfi_c1)) ||
 -	    (ret = merge_file_special_markers(o, b, &ci->ren2_other, c2,
 +					    o->branch2, ci->ren1_other.path, &mfi_c1) ||
 +	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
  					    o->branch1, ci->ren2_other.path,
 -					    o->branch2, c2->path, &mfi_c2)))
 -		return ret;
 +					    o->branch2, c2->path, &mfi_c2))
 +		return -1;
  
  	if (o->call_depth) {
  		/*
 @@ -1314,7 +1332,7 @@ static int process_renames(struct merge_options *o,
  			   struct string_list *a_renames,
  			   struct string_list *b_renames)
  {
 -	int clean_merge = 1, i, j, ret;
 +	int clean_merge = 1, i, j;
  	struct string_list a_by_dst = STRING_LIST_INIT_NODUP;
  	struct string_list b_by_dst = STRING_LIST_INIT_NODUP;
  	const struct rename *sre;
 @@ -1490,14 +1508,13 @@ static int process_renames(struct merge_options *o,
  				 * update_file_flags() instead of
  				 * update_file().
  				 */
 -				ret = update_file_flags(o,
 +				if (update_file_flags(o,
  						  ren1->pair->two->sha1,
  						  ren1->pair->two->mode,
  						  ren1_dst,
  						  1, /* update_cache */
 -						  0  /* update_wd    */);
 -				if (ret)
 -					clean_merge = ret;
 +						  0  /* update_wd    */))
 +					clean_merge = -1;
  			} else if (!sha_eq(dst_other.sha1, null_sha1)) {
  				clean_merge = 0;
  				try_merge = 1;
 @@ -1507,22 +1524,22 @@ static int process_renames(struct merge_options *o,
  				       ren1_dst, branch2);
  				if (o->call_depth) {
  					struct merge_file_info mfi;
 -					if ((ret = merge_file_one(o, ren1_dst, null_sha1, 0,
 +					if (merge_file_one(o, ren1_dst, null_sha1, 0,
  							 ren1->pair->two->sha1, ren1->pair->two->mode,
  							 dst_other.sha1, dst_other.mode,
 -							 branch1, branch2, &mfi))) {
 -						clean_merge = ret;
 +							 branch1, branch2, &mfi)) {
 +						clean_merge = -1;
  						goto cleanup_and_return;
  					}
  					output(o, 1, _("Adding merged %s"), ren1_dst);
 -					if ((ret = update_file(o, 0, mfi.sha, mfi.mode, ren1_dst)))
 -						clean_merge = ret;
 +					if (update_file(o, 0, mfi.sha, mfi.mode, ren1_dst))
 +						clean_merge = -1;
  					try_merge = 0;
  				} else {
  					char *new_path = unique_path(o, ren1_dst, branch2);
  					output(o, 1, _("Adding as %s instead"), new_path);
 -					if ((ret = update_file(o, 0, dst_other.sha1, dst_other.mode, new_path)))
 -						clean_merge = ret;
 +					if (update_file(o, 0, dst_other.sha1, dst_other.mode, new_path))
 +						clean_merge = -1;
  					free(new_path);
  				}
  			} else
 @@ -1568,23 +1585,25 @@ static unsigned char *stage_sha(const unsigned char *sha, unsigned mode)
  	return (is_null_sha1(sha) || mode == 0) ? NULL: (unsigned char *)sha;
  }
  
 -static int read_sha1_strbuf(const unsigned char *sha1, struct strbuf *dst)
 +static int read_sha1_strbuf(struct merge_options *o,
 +	const unsigned char *sha1, struct strbuf *dst)
  {
  	void *buf;
  	enum object_type type;
  	unsigned long size;
  	buf = read_sha1_file(sha1, &type, &size);
  	if (!buf)
 -		return error(_("cannot read object %s"), sha1_to_hex(sha1));
 +		return err(o, _("cannot read object %s"), sha1_to_hex(sha1));
  	if (type != OBJ_BLOB) {
  		free(buf);
 -		return error(_("object %s is not a blob"), sha1_to_hex(sha1));
 +		return err(o, _("object %s is not a blob"), sha1_to_hex(sha1));
  	}
  	strbuf_attach(dst, buf, size, size + 1);
  	return 0;
  }
  
 -static int blob_unchanged(const unsigned char *o_sha,
 +static int blob_unchanged(struct merge_options *opt,
 +			  const unsigned char *o_sha,
  			  unsigned o_mode,
  			  const unsigned char *a_sha,
  			  unsigned a_mode,
 @@ -1602,7 +1621,7 @@ static int blob_unchanged(const unsigned char *o_sha,
  		return 0;
  
  	assert(o_sha && a_sha);
 -	if (read_sha1_strbuf(o_sha, &o) || read_sha1_strbuf(a_sha, &a))
 +	if (read_sha1_strbuf(opt, o_sha, &o) || read_sha1_strbuf(opt, a_sha, &a))
  		goto error_return;
  	/*
  	 * Note: binary | is used so that both renormalizations are
 @@ -1645,7 +1664,6 @@ static int merge_content(struct merge_options *o,
  	struct merge_file_info mfi;
  	struct diff_filespec one, a, b;
  	unsigned df_conflict_remains = 0;
 -	int ret;
  
  	if (!o_sha) {
  		reason = _("add/add");
 @@ -1675,10 +1693,10 @@ static int merge_content(struct merge_options *o,
  		if (dir_in_way(path, !o->call_depth))
  			df_conflict_remains = 1;
  	}
 -	if ((ret = merge_file_special_markers(o, &one, &a, &b,
 +	if (merge_file_special_markers(o, &one, &a, &b,
  					 o->branch1, path1,
 -					 o->branch2, path2, &mfi)))
 -		return ret;
 +					 o->branch2, path2, &mfi))
 +		return -1;
  
  	if (mfi.clean && !df_conflict_remains &&
  	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode) {
 @@ -1692,7 +1710,7 @@ static int merge_content(struct merge_options *o,
  		 */
  		path_renamed_outside_HEAD = !path2 || !strcmp(path, path2);
  		if (!path_renamed_outside_HEAD) {
 -			add_cacheinfo(mfi.mode, mfi.sha, path,
 +			add_cacheinfo(o, mfi.mode, mfi.sha, path,
  				      0, (!o->call_depth), 0);
  			return mfi.clean;
  		}
 @@ -1705,8 +1723,8 @@ static int merge_content(struct merge_options *o,
  		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
  				reason, path);
  		if (rename_conflict_info && !df_conflict_remains)
 -			if ((ret = update_stages(path, &one, &a, &b)))
 -				return ret;
 +			if (update_stages(o, path, &one, &a, &b))
 +				return -1;
  	}
  
  	if (df_conflict_remains) {
 @@ -1715,42 +1733,39 @@ static int merge_content(struct merge_options *o,
  			remove_file_from_cache(path);
  		} else {
  			if (!mfi.clean) {
 -				if ((ret = update_stages(path, &one, &a, &b)))
 -					return ret;
 +				if (update_stages(o, path, &one, &a, &b))
 +					return -1;
  			} else {
  				int file_from_stage2 = was_tracked(path);
  				struct diff_filespec merged;
  				hashcpy(merged.sha1, mfi.sha);
  				merged.mode = mfi.mode;
  
 -				if ((ret = update_stages(path, NULL,
 +				if (update_stages(o, path, NULL,
  					      file_from_stage2 ? &merged : NULL,
 -					      file_from_stage2 ? NULL : &merged)))
 -					return ret;
 +					      file_from_stage2 ? NULL : &merged))
 +					return -1;
  			}
  
  		}
  		new_path = unique_path(o, path, rename_conflict_info->branch1);
  		output(o, 1, _("Adding as %s instead"), new_path);
 -		if ((ret = update_file(o, 0, mfi.sha, mfi.mode, new_path))) {
 +		if (update_file(o, 0, mfi.sha, mfi.mode, new_path)) {
  			free(new_path);
 -			return ret;
 +			return -1;
  		}
  		free(new_path);
  		mfi.clean = 0;
 -	} else {
 -		if ((ret = update_file(o, mfi.clean, mfi.sha, mfi.mode, path)))
 -			return ret;
 -	}
 +	} else if (update_file(o, mfi.clean, mfi.sha, mfi.mode, path))
 +		return -1;
  	return mfi.clean;
 -
  }
  
  /* Per entry merge function */
  static int process_entry(struct merge_options *o,
  			 const char *path, struct stage_data *entry)
  {
 -	int clean_merge = 1, ret;
 +	int clean_merge = 1;
  	int normalize = o->renormalize;
  	unsigned o_mode = entry->stages[1].mode;
  	unsigned a_mode = entry->stages[2].mode;
 @@ -1771,23 +1786,21 @@ static int process_entry(struct merge_options *o,
  			break;
  		case RENAME_DELETE:
  			clean_merge = 0;
 -			if ((ret = conflict_rename_delete(o,
 +			if (conflict_rename_delete(o,
  					       conflict_info->pair1,
  					       conflict_info->branch1,
 -					       conflict_info->branch2)))
 -				clean_merge = ret;
 +					       conflict_info->branch2))
 +				clean_merge = -1;
  			break;
  		case RENAME_ONE_FILE_TO_TWO:
  			clean_merge = 0;
 -			if ((ret = conflict_rename_rename_1to2(o,
 -					conflict_info)))
 -				clean_merge = ret;
 +			if (conflict_rename_rename_1to2(o, conflict_info))
 +				clean_merge = -1;
  			break;
  		case RENAME_TWO_FILES_TO_ONE:
  			clean_merge = 0;
 -			if ((ret = conflict_rename_rename_2to1(o,
 -					conflict_info)))
 -				clean_merge = ret;
 +			if (conflict_rename_rename_2to1(o, conflict_info))
 +				clean_merge = -1;
  			break;
  		default:
  			entry->processed = 0;
 @@ -1796,8 +1809,8 @@ static int process_entry(struct merge_options *o,
  	} else if (o_sha && (!a_sha || !b_sha)) {
  		/* Case A: Deleted in one */
  		if ((!a_sha && !b_sha) ||
 -		    (!b_sha && blob_unchanged(o_sha, o_mode, a_sha, a_mode, normalize, path)) ||
 -		    (!a_sha && blob_unchanged(o_sha, o_mode, b_sha, b_mode, normalize, path))) {
 +		    (!b_sha && blob_unchanged(o, o_sha, o_mode, a_sha, a_mode, normalize, path)) ||
 +		    (!a_sha && blob_unchanged(o, o_sha, o_mode, b_sha, b_mode, normalize, path))) {
  			/* Deleted in both or deleted in one and
  			 * unchanged in the other */
  			if (a_sha)
 @@ -1807,9 +1820,9 @@ static int process_entry(struct merge_options *o,
  		} else {
  			/* Modify/delete; deleted side may have put a directory in the way */
  			clean_merge = 0;
 -			if ((ret = handle_modify_delete(o, path, o_sha, o_mode,
 -					a_sha, a_mode, b_sha, b_mode)))
 -				clean_merge = ret;
 +			if (handle_modify_delete(o, path, o_sha, o_mode,
 +						 a_sha, a_mode, b_sha, b_mode))
 +				clean_merge = -1;
  		}
  	} else if ((!o_sha && a_sha && !b_sha) ||
  		   (!o_sha && !a_sha && b_sha)) {
 @@ -1841,18 +1854,16 @@ static int process_entry(struct merge_options *o,
  			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
  			       "Adding %s as %s"),
  			       conf, path, other_branch, path, new_path);
 -			ret = update_file(o, 0, sha, mode, new_path);
 -			if (ret)
 -				clean_merge = ret;
 +			if (update_file(o, 0, sha, mode, new_path))
 +				clean_merge = -1;
  			else if (o->call_depth)
  				remove_file_from_cache(path);
  			free(new_path);
  		} else {
  			output(o, 2, _("Adding %s"), path);
  			/* do not overwrite file if already present */
 -			if ((ret = update_file_flags(o, sha, mode, path, 1,
 -					!a_sha)))
 -				clean_merge = ret;
 +			if (update_file_flags(o, sha, mode, path, 1, !a_sha))
 +				clean_merge = -1;
  		}
  	} else if (a_sha && b_sha) {
  		/* Case C: Added in both (check for same permissions) and */
 @@ -1867,7 +1878,7 @@ static int process_entry(struct merge_options *o,
  		 */
  		remove_file(o, 1, path, !a_mode);
  	} else
 -		return error(_("Fatal merge failure, shouldn't happen."));
 +		die("BUG: fatal merge failure, shouldn't happen.");
  
  	return clean_merge;
  }
 @@ -1895,7 +1906,7 @@ int merge_trees(struct merge_options *o,
  
  	if (code != 0) {
  		if (show(o, 4) || o->call_depth)
 -			error(_("merging of trees %s and %s failed"),
 +			err(o, _("merging of trees %s and %s failed"),
  			    oid_to_hex(&head->object.oid),
  			    oid_to_hex(&merge->object.oid));
  		return -1;
 @@ -1930,7 +1941,7 @@ int merge_trees(struct merge_options *o,
  		for (i = 0; i < entries->nr; i++) {
  			struct stage_data *e = entries->items[i].util;
  			if (!e->processed)
 -				die(_("BUG: unprocessed path??? %s"),
 +				die("BUG: unprocessed path??? %s",
  				    entries->items[i].string);
  		}
  
 @@ -2029,7 +2040,7 @@ int merge_recursive(struct merge_options *o,
  		o->call_depth--;
  
  		if (!merged_common_ancestors)
 -			return error(_("merge returned no commit"));
 +			return err(o, _("merge returned no commit"));
  	}
  
  	discard_cache();
 @@ -2039,6 +2050,7 @@ int merge_recursive(struct merge_options *o,
  	o->ancestor = "merged common ancestors";
  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
  			    &mrtree);
 +	flush_output(o);
  	if (clean < 0)
  		return clean;
  
 @@ -2047,7 +2059,8 @@ int merge_recursive(struct merge_options *o,
  		commit_list_insert(h1, &(*result)->parents);
  		commit_list_insert(h2, &(*result)->parents->next);
  	}
 -	flush_output(o);
 +	if (o->buffer_output < 2)
 +		strbuf_release(&o->obuf);
  	if (show(o, 2))
  		diff_warn_rename_limit("merge.renamelimit",
  				       o->needed_rename_limit, 0);
 @@ -2088,7 +2101,7 @@ int merge_recursive_generic(struct merge_options *o,
  		for (i = 0; i < num_base_list; ++i) {
  			struct commit *base;
  			if (!(base = get_ref(base_list[i], sha1_to_hex(base_list[i]))))
 -				return error(_("Could not parse object '%s'"),
 +				return err(o, _("Could not parse object '%s'"),
  					sha1_to_hex(base_list[i]));
  			commit_list_insert(base, &ca);
  		}
 @@ -2102,7 +2115,7 @@ int merge_recursive_generic(struct merge_options *o,
  
  	if (active_cache_changed &&
  	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 -		return error(_("Unable to write index."));
 +		return err(o, _("Unable to write index."));
  
  	return clean ? 0 : 1;
  }
 diff --git a/merge-recursive.h b/merge-recursive.h
 index 52f0201..407d4fc 100644
 --- a/merge-recursive.h
 +++ b/merge-recursive.h
 @@ -13,7 +13,7 @@ struct merge_options {
  		MERGE_RECURSIVE_THEIRS
  	} recursive_variant;
  	const char *subtree_shift;
 -	unsigned buffer_output : 1;
 +	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
  	unsigned renormalize : 1;
  	long xdl_opts;
  	int verbosity;
 diff --git a/sequencer.c b/sequencer.c
 index 13b794a..8ceeb1b 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -293,6 +293,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
  	clean = merge_trees(&o,
  			    head_tree,
  			    next_tree, base_tree, &result);
 +	strbuf_release(&o.obuf);
  	if (clean < 0)
  		return clean;
  
 diff --git a/sha1_file.c b/sha1_file.c
 index aa7006c..5085fe0 100644
 --- a/sha1_file.c
 +++ b/sha1_file.c
 @@ -795,7 +795,7 @@ void close_all_packs(void)
  
  	for (p = packed_git; p; p = p->next)
  		if (p->do_not_close)
 -			die("BUG: Want to close pack marked 'do-not-close'");
 +			die("BUG: want to close pack marked 'do-not-close'");
  		else
  			close_pack(p);
  }
 diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
 index 3159956..b281d6f 100755
 --- a/t/t5520-pull.sh
 +++ b/t/t5520-pull.sh
 @@ -255,6 +255,36 @@ test_expect_success '--rebase' '
  	test new = "$(git show HEAD:file2)"
  '
  
 +test_expect_success '--rebase with conflicts shows advice' '
 +	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
 +	git checkout -b seq &&
 +	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
 +	git add seq.txt &&
 +	test_tick &&
 +	git commit -m "Add seq.txt" &&
 +	printf "6\\n" >>seq.txt &&
 +	test_tick &&
 +	git commit -m "Append to seq.txt" seq.txt &&
 +	git checkout -b with-conflicts HEAD^ &&
 +	printf "conflicting\\n" >>seq.txt &&
 +	test_tick &&
 +	git commit -m "Create conflict" seq.txt &&
 +	test_must_fail git pull --rebase . seq 2>err >out &&
 +	grep "When you have resolved this problem" out
 +'
 +test_expect_success 'failed --rebase shows advice' '
 +	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
 +	git checkout -b diverging &&
 +	test_commit attributes .gitattributes "* text=auto" attrs &&
 +	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
 +	git update-index --cacheinfo 0644 $sha1 file &&
 +	git commit -m v1-with-cr &&
 +	git checkout -f -b fails-to-rebase HEAD^ &&
 +	test_commit v2-without-cr file "2" file2-lf &&
 +	test_must_fail git pull --rebase . diverging 2>err >out &&
 +	grep "When you have resolved this problem" out
 +'
 +
  test_expect_success '--rebase fails with multiple branches' '
  	git reset --hard before-rebase &&
  	test_must_fail git pull --rebase . copy master 2>err &&
 diff --git a/wt-status.c b/wt-status.c
 index 311ae7c..75c1162 100644
 --- a/wt-status.c
 +++ b/wt-status.c
 @@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
  	case 7:
  		return _("both modified:");
  	default:
 -		die(_("BUG: unhandled unmerged status %x"), stagemask);
 +		die("BUG: unhandled unmerged status %x", stagemask);
  	}
  }
  
 @@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
  	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
  	what = wt_status_diff_status_string(status);
  	if (!what)
 -		die(_("BUG: unhandled diff status %c"), status);
 +		die("BUG: unhandled diff status %c", status);
  	len = label_width - utf8_strwidth(what);
  	assert(len >= 0);
  	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.280.g32e2a70

base-commit: cf4c2cfe52be5bd973a4838f73a35d3959ce2f43
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 01/17] Verify that `git pull --rebase` shows the helpful advice when failing

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t5520-pull.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
Show changes to t/t5520-pull.sh +30 −0
diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
index 3159956..b281d6f 100755
--- a/t/t5520-pull.sh
+++ b/t/t5520-pull.sh
@@ -255,6 +255,36 @@ test_expect_success '--rebase' '
 	test new = "$(git show HEAD:file2)"
 '
 
+test_expect_success '--rebase with conflicts shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b seq &&
+	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
+	git add seq.txt &&
+	test_tick &&
+	git commit -m "Add seq.txt" &&
+	printf "6\\n" >>seq.txt &&
+	test_tick &&
+	git commit -m "Append to seq.txt" seq.txt &&
+	git checkout -b with-conflicts HEAD^ &&
+	printf "conflicting\\n" >>seq.txt &&
+	test_tick &&
+	git commit -m "Create conflict" seq.txt &&
+	test_must_fail git pull --rebase . seq 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+test_expect_success 'failed --rebase shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b diverging &&
+	test_commit attributes .gitattributes "* text=auto" attrs &&
+	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
+	git update-index --cacheinfo 0644 $sha1 file &&
+	git commit -m v1-with-cr &&
+	git checkout -f -b fails-to-rebase HEAD^ &&
+	test_commit v2-without-cr file "2" file2-lf &&
+	test_must_fail git pull --rebase . diverging 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
 test_expect_success '--rebase fails with multiple branches' '
 	git reset --hard before-rebase &&
 	test_must_fail git pull --rebase . copy master 2>err &&
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 02/17] Report bugs consistently

The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:".

As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/ls-files.c     |  3 ++-
 builtin/update-index.c |  2 +-
 grep.c                 |  8 ++++----
 imap-send.c            |  4 ++--
 merge-recursive.c      | 15 +++++++--------
 sha1_file.c            |  4 ++--
 trailer.c              |  2 +-
 transport.c            |  2 +-
 wt-status.c            |  4 ++--
 9 files changed, 22 insertions(+), 22 deletions(-)
Show changes to 9 files +22 −22

builtin/ls-files.c, builtin/update-index.c, grep.c, imap-send.c, merge-recursive.c, sha1_file.c, trailer.c, transport.c, wt-status.c

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f02e3d2..00ea91a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
 				 */
 				pos = cache_name_pos(ent->name, ent->len);
 				if (0 <= pos)
-					die("bug in show-killed-files");
+					die("BUG: killed-file %.*s not found",
+						ent->len, ent->name);
 				pos = -pos - 1;
 				while (pos < active_nr &&
 				       ce_stage(active_cache[pos]))
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 6cdfd5f..ba04b19 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1146,7 +1146,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		report(_("Untracked cache enabled for '%s'"), get_git_work_tree());
 		break;
 	default:
-		die("Bug: bad untracked_cache value: %d", untracked_cache);
+		die("BUG: bad untracked_cache value: %d", untracked_cache);
 	}
 
 	if (active_cache_changed) {
diff --git a/grep.c b/grep.c
index 1e15b62..f1ca0a0 100644
--- a/grep.c
+++ b/grep.c
@@ -643,10 +643,10 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 	for (p = opt->header_list; p; p = p->next) {
 		if (p->token != GREP_PATTERN_HEAD)
-			die("bug: a non-header pattern in grep header list.");
+			die("BUG: a non-header pattern in grep header list.");
 		if (p->field < GREP_HEADER_FIELD_MIN ||
 		    GREP_HEADER_FIELD_MAX <= p->field)
-			die("bug: unknown header field %d", p->field);
+			die("BUG: unknown header field %d", p->field);
 		compile_regexp(p, opt);
 	}
 
@@ -659,7 +659,7 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 		h = compile_pattern_atom(&pp);
 		if (!h || pp != p->next)
-			die("bug: malformed header expr");
+			die("BUG: malformed header expr");
 		if (!header_group[p->field]) {
 			header_group[p->field] = h;
 			continue;
@@ -1464,7 +1464,7 @@ static int grep_source_1(struct grep_opt *opt, struct grep_source *gs, int colle
 		case GREP_BINARY_TEXT:
 			break;
 		default:
-			die("bug: unknown binary handling mode");
+			die("BUG: unknown binary handling mode");
 		}
 	}
 
diff --git a/imap-send.c b/imap-send.c
index 938c691..369f72a 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -506,12 +506,12 @@ static char *next_arg(char **s)
 
 static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 {
-	int ret;
+	int ret = -1;
 	va_list va;
 
 	va_start(va, fmt);
 	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
-		die("Fatal: buffer too small. Please report a bug.");
+		die("BUG: buffer too small (%d < %d)", ret, blen);
 	va_end(va);
 	return ret;
 }
diff --git a/merge-recursive.c b/merge-recursive.c
index 65cb5d6..9849439 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -259,7 +259,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
-		die("Bug in merge-recursive.c");
+		die("BUG: unmerged index entries in merge-recursive.c");
 	}
 
 	if (!active_cache_tree)
@@ -955,9 +955,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 
 			if (!sha_eq(a->sha1, b->sha1))
 				result.clean = 0;
-		} else {
-			die(_("unsupported object type in the tree"));
-		}
+		} else
+			die(_("BUG: unsupported object type in the tree"));
 	}
 
 	return result;
@@ -1343,7 +1342,7 @@ static int process_renames(struct merge_options *o,
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1_src != ren2_src");
+				die("BUG: ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
@@ -1377,7 +1376,7 @@ static int process_renames(struct merge_options *o,
 			ren2 = lookup->util;
 			ren2_dst = ren2->pair->two->path;
 			if (strcmp(ren1_dst, ren2_dst) != 0)
-				die("ren1_dst != ren2_dst");
+				die("BUG: ren1_dst != ren2_dst");
 
 			clean_merge = 0;
 			ren2->processed = 1;
@@ -1795,7 +1794,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("Fatal merge failure, shouldn't happen."));
+		die(_("BUG: fatal merge failure, shouldn't happen."));
 
 	return clean_merge;
 }
@@ -1853,7 +1852,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("Unprocessed path??? %s"),
+				die(_("BUG: unprocessed path??? %s"),
 				    entries->items[i].string);
 		}
 
diff --git a/sha1_file.c b/sha1_file.c
index d5e1121..5085fe0 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -795,7 +795,7 @@ void close_all_packs(void)
 
 	for (p = packed_git; p; p = p->next)
 		if (p->do_not_close)
-			die("BUG! Want to close pack marked 'do-not-close'");
+			die("BUG: want to close pack marked 'do-not-close'");
 		else
 			close_pack(p);
 }
@@ -2330,7 +2330,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 	case OBJ_OFS_DELTA:
 	case OBJ_REF_DELTA:
 		if (data)
-			die("BUG in unpack_entry: left loop at a valid delta");
+			die("BUG: unpack_entry: left loop at a valid delta");
 		break;
 	case OBJ_COMMIT:
 	case OBJ_TREE:
diff --git a/trailer.c b/trailer.c
index 8e48a5c..c6ea9ac 100644
--- a/trailer.c
+++ b/trailer.c
@@ -562,7 +562,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
-		die("internal bug in trailer.c");
+		die("BUG: trailer.c: unhandled type %d", type);
 	}
 	return 0;
 }
diff --git a/transport.c b/transport.c
index 095e61f..52bf997 100644
--- a/transport.c
+++ b/transport.c
@@ -563,7 +563,7 @@ void transport_take_over(struct transport *transport,
 	struct git_transport_data *data;
 
 	if (!transport->smart_options)
-		die("Bug detected: Taking over transport requires non-NULL "
+		die("BUG: taking over transport requires non-NULL "
 		    "smart_options field.");
 
 	data = xcalloc(1, sizeof(*data));
diff --git a/wt-status.c b/wt-status.c
index 4ce4e35..311ae7c 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die(_("bug: unhandled unmerged status %x"), stagemask);
+		die(_("BUG: unhandled unmerged status %x"), stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die(_("bug: unhandled diff status %c"), status);
+		die(_("BUG: unhandled diff status %c"), status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.280.g32e2a70
Jakub Narębski· Jul 5, 2016, 13:05 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 02/17] Report bugs consistently

W dniu 2016-07-05 o 13:23, Johannes Schindelin pisze:
Show 14 quoted lines
> diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> index f02e3d2..00ea91a 100644
> --- a/builtin/ls-files.c
> +++ b/builtin/ls-files.c
> @@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
>  				 */
>  				pos = cache_name_pos(ent->name, ent->len);
>  				if (0 <= pos)
> -					die("bug in show-killed-files");
> +					die("BUG: killed-file %.*s not found",
> +						ent->len, ent->name);
>  				pos = -pos - 1;
>  				while (pos < active_nr &&
>  				       ce_stage(active_cache[pos]))

This has an additional improvement (not mentioned in the commit message, but probably not worth it) in that it shows which file was not found, not only that there was some bug, isn't it?

-- 
Jakub Narębski
Johannes Schindelin· Jul 5, 2016, 13:38 UTC · re: Jakub Narębski · lore

Re: [PATCH v2 02/17] Report bugs consistently

Hi Kuba,
On Tue, 5 Jul 2016, Jakub Narębski wrote:
Show 19 quoted lines
> W dniu 2016-07-05 o 13:23, Johannes Schindelin pisze:
> > diff --git a/builtin/ls-files.c b/builtin/ls-files.c
> > index f02e3d2..00ea91a 100644
> > --- a/builtin/ls-files.c
> > +++ b/builtin/ls-files.c
> > @@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
> >  				 */
> >  				pos = cache_name_pos(ent->name, ent->len);
> >  				if (0 <= pos)
> > -					die("bug in show-killed-files");
> > +					die("BUG: killed-file %.*s not found",
> > +						ent->len, ent->name);
> >  				pos = -pos - 1;
> >  				while (pos < active_nr &&
> >  				       ce_stage(active_cache[pos]))
> 
> This has an additional improvement (not mentioned in the commit
> message, but probably not worth it) in that it shows which file
> was not found, not only that there was some bug, isn't it?

Sure, it improves that report. In the unlikely event that a bug is encountered :-)

Is it really worth mentioning in the commit message?

Looking at it again, however, I think there is a bug in my patch. It says that the file was not found, but pos was non-negative, so it was found unexpectedly. So I think I should strike the "not" part. Would you concur?

Ciao, Dscho

Duy Nguyen· Jul 6, 2016, 15:30 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 02/17] Report bugs consistently

On Tue, Jul 5, 2016 at 1:23 PM, Johannes Schindelin <johannes.schindelin@gmx.de> wrote:

Show 6 quoted lines
> The vast majority of error messages in Git's source code which report a
> bug use the convention to prefix the message with "BUG:".
>
> As part of cleaning up merge-recursive to stop die()ing except in case of
> detected bugs, let's just make the remainder of the bug reports consistent
> with the de facto rule.

If you search 'die(_("bug:' in this patch, you'll find 5 instances where _() should be gone too.

-- 
Duy
Johannes Schindelin· Jul 7, 2016, 11:23 UTC · re: Duy Nguyen · lore

Re: [PATCH v2 02/17] Report bugs consistently

Hi Duy,
On Wed, 6 Jul 2016, Duy Nguyen wrote:
Show 11 quoted lines
> On Tue, Jul 5, 2016 at 1:23 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > The vast majority of error messages in Git's source code which report a
> > bug use the convention to prefix the message with "BUG:".
> >
> > As part of cleaning up merge-recursive to stop die()ing except in case of
> > detected bugs, let's just make the remainder of the bug reports consistent
> > with the de facto rule.
> 
> If you search 'die(_("bug:' in this patch,  you'll find 5 instances
> where _() should be gone too.

I should have known better. Vasco's i18n work already conflicts seriously with what I have in this patch series.

This proves once again, beyond any doubt, that one should refrain from introducing patches that have little to do with the purpose of the patch series.

So as far as this here patch series goes, let's re-focus on the recursive merge code again, okay?

Ciao, Dscho

Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 04/17] merge-recursive: clarify code in was_tracked()

It can be puzzling to see that was_tracked() asks to get an index entry by name, but does not take a negative return value for an answer.

The reason we have to do this is that cache_name_pos() only looks for entries in stage 0, even if nobody asked for any stage in particular.

Let's rewrite the logic a little bit, to handle the easy case early: if cache_name_pos() returned a non-negative position, we know it is a match, and we do not even have to compare the name again (cache_name_pos() did that for us already). We can say right away: yes, this file was tracked.

Only if there was no exact match do we need to look harder for any matching entry in stage 2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
Show changes to merge-recursive.c +9 −10
diff --git a/merge-recursive.c b/merge-recursive.c
index e51f8fc..66ce27c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -658,23 +658,22 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
-	if (pos < 0)
-		pos = -1 - pos;
-	while (pos < active_nr &&
-	       !strcmp(path, active_cache[pos]->name)) {
+	if (pos >= 0)
+		return pos < active_nr;
+	/*
+	 * cache_name_pos() looks for stage == 0, even if we did not ask for
+	 * it. Let's look for stage == 2 now.
+	 */
+	for (pos = -1 - pos; pos < active_nr &&
+	     !strcmp(path, active_cache[pos]->name); pos++)
 		/*
 		 * If stage #0, it is definitely tracked.
 		 * If it has stage #2 then it was tracked
 		 * before this merge started.  All other
 		 * cases the path was not tracked.
 		 */
-		switch (ce_stage(active_cache[pos])) {
-		case 0:
-		case 2:
+		if (ce_stage(active_cache[pos]) == 2)
 			return 1;
-		}
-		pos++;
-	}
 	return 0;
 }
 
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 03/17] Avoid translating bug messages

While working on the patch series that avoids die()ing in recursive merges, the issue came up that bug reports (i.e. die("BUG: ...") constructs) should never be translated, as the target audience is the Git developer community, not necessarily the current user, and hence a translated message would make it *harder* to address the problem.

So let's stop translating the obvious ones. As it is really, really outside the purview of this patch series to see whether there are more die() statements that report bugs and are currently translated, that task is left for another day and patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 6 +++---
 wt-status.c       | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)
Show changes to 2 files +5 −5

merge-recursive.c, wt-status.c

diff --git a/merge-recursive.c b/merge-recursive.c
index 9849439..e51f8fc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -956,7 +956,7 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 			if (!sha_eq(a->sha1, b->sha1))
 				result.clean = 0;
 		} else
-			die(_("BUG: unsupported object type in the tree"));
+			die("BUG: unsupported object type in the tree");
 	}
 
 	return result;
@@ -1794,7 +1794,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("BUG: fatal merge failure, shouldn't happen."));
+		die("BUG: fatal merge failure, shouldn't happen.");
 
 	return clean_merge;
 }
@@ -1852,7 +1852,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("BUG: unprocessed path??? %s"),
+				die("BUG: unprocessed path??? %s",
 				    entries->items[i].string);
 		}
 
diff --git a/wt-status.c b/wt-status.c
index 311ae7c..75c1162 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die(_("BUG: unhandled unmerged status %x"), stagemask);
+		die("BUG: unhandled unmerged status %x", stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die(_("BUG: unhandled diff status %c"), status);
+		die("BUG: unhandled diff status %c", status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 06/17] merge_recursive: abort properly upon errors

There are a couple of places where return values indicating errors are ignored. Let's teach them manners.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +8 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index 66ce27c..716488b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1942,8 +1942,9 @@ int merge_recursive(struct merge_options *o,
 		saved_b2 = o->branch2;
 		o->branch1 = "Temporary merge branch 1";
 		o->branch2 = "Temporary merge branch 2";
-		merge_recursive(o, merged_common_ancestors, iter->item,
-				NULL, &merged_common_ancestors);
+		if (merge_recursive(o, merged_common_ancestors, iter->item,
+				NULL, &merged_common_ancestors) < 0)
+			return -1;
 		o->branch1 = saved_b1;
 		o->branch2 = saved_b2;
 		o->call_depth--;
@@ -1959,6 +1960,8 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	if (clean < 0)
+		return clean;
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
@@ -2015,6 +2018,9 @@ int merge_recursive_generic(struct merge_options *o,
 	hold_locked_index(lock, 1);
 	clean = merge_recursive(o, head_commit, next_commit, ca,
 			result);
+	if (clean < 0)
+		return clean;
+
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 		return error(_("Unable to write index."));
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 07/17] merge-recursive: avoid returning a wholesale struct

It is technically allowed, as per C89, for functions' return type to be complete structs (i.e. *not* just pointers to structs).

However, it was just an oversight of this developer when converting Python code to C code in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which introduced such a return type.

Besides, by converting this construct to pass in the struct, we can now start returning a value that can indicate errors in future patches. This will help the current effort to libify merge-recursive.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 93 ++++++++++++++++++++++++++++++-------------------------
 1 file changed, 50 insertions(+), 43 deletions(-)
Show changes to merge-recursive.c +50 −43
diff --git a/merge-recursive.c b/merge-recursive.c
index 716488b..3e3667f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -886,47 +886,47 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file_1(struct merge_options *o,
+static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
 					   const struct diff_filespec *b,
 					   const char *branch1,
-					   const char *branch2)
+					   const char *branch2,
+					   struct merge_file_info *result)
 {
-	struct merge_file_info result;
-	result.merge = 0;
-	result.clean = 1;
+	result->merge = 0;
+	result->clean = 1;
 
 	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
-		result.clean = 0;
+		result->clean = 0;
 		if (S_ISREG(a->mode)) {
-			result.mode = a->mode;
-			hashcpy(result.sha, a->sha1);
+			result->mode = a->mode;
+			hashcpy(result->sha, a->sha1);
 		} else {
-			result.mode = b->mode;
-			hashcpy(result.sha, b->sha1);
+			result->mode = b->mode;
+			hashcpy(result->sha, b->sha1);
 		}
 	} else {
 		if (!sha_eq(a->sha1, one->sha1) && !sha_eq(b->sha1, one->sha1))
-			result.merge = 1;
+			result->merge = 1;
 
 		/*
 		 * Merge modes
 		 */
 		if (a->mode == b->mode || a->mode == one->mode)
-			result.mode = b->mode;
+			result->mode = b->mode;
 		else {
-			result.mode = a->mode;
+			result->mode = a->mode;
 			if (b->mode != one->mode) {
-				result.clean = 0;
-				result.merge = 1;
+				result->clean = 0;
+				result->merge = 1;
 			}
 		}
 
 		if (sha_eq(a->sha1, b->sha1) || sha_eq(a->sha1, one->sha1))
-			hashcpy(result.sha, b->sha1);
+			hashcpy(result->sha, b->sha1);
 		else if (sha_eq(b->sha1, one->sha1))
-			hashcpy(result.sha, a->sha1);
+			hashcpy(result->sha, a->sha1);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
 			int merge_status;
@@ -938,62 +938,63 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 				die(_("Failed to execute internal merge"));
 
 			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result.sha))
+					    blob_type, result->sha))
 				die(_("Unable to add %s to database"),
 				    a->path);
 
 			free(result_buf.ptr);
-			result.clean = (merge_status == 0);
+			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result.clean = merge_submodule(result.sha,
+			result->clean = merge_submodule(result->sha,
 						       one->path, one->sha1,
 						       a->sha1, b->sha1,
 						       !o->call_depth);
 		} else if (S_ISLNK(a->mode)) {
-			hashcpy(result.sha, a->sha1);
+			hashcpy(result->sha, a->sha1);
 
 			if (!sha_eq(a->sha1, b->sha1))
-				result.clean = 0;
+				result->clean = 0;
 		} else
 			die("BUG: unsupported object type in the tree");
 	}
 
-	return result;
+	return 0;
 }
 
-static struct merge_file_info
-merge_file_special_markers(struct merge_options *o,
+static int merge_file_special_markers(struct merge_options *o,
 			   const struct diff_filespec *one,
 			   const struct diff_filespec *a,
 			   const struct diff_filespec *b,
 			   const char *branch1,
 			   const char *filename1,
 			   const char *branch2,
-			   const char *filename2)
+			   const char *filename2,
+			   struct merge_file_info *mfi)
 {
 	char *side1 = NULL;
 	char *side2 = NULL;
-	struct merge_file_info mfi;
+	int ret;
 
 	if (filename1)
 		side1 = xstrfmt("%s:%s", branch1, filename1);
 	if (filename2)
 		side2 = xstrfmt("%s:%s", branch2, filename2);
 
-	mfi = merge_file_1(o, one, a, b,
-			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	ret = merge_file_1(o, one, a, b,
+		side1 ? side1 : branch1, side2 ? side2 : branch2, mfi);
 	free(side1);
 	free(side2);
-	return mfi;
+	return ret;
 }
 
-static struct merge_file_info merge_file_one(struct merge_options *o,
+static int merge_file_one(struct merge_options *o,
 					 const char *path,
 					 const unsigned char *o_sha, int o_mode,
 					 const unsigned char *a_sha, int a_mode,
 					 const unsigned char *b_sha, int b_mode,
 					 const char *branch1,
-					 const char *branch2)
+					 const char *branch2,
+					 struct merge_file_info *mfi)
 {
 	struct diff_filespec one, a, b;
 
@@ -1004,7 +1005,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
 	a.mode = a_mode;
 	hashcpy(b.sha1, b_sha);
 	b.mode = b_mode;
-	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
 static void handle_change_delete(struct merge_options *o,
@@ -1177,11 +1178,14 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct merge_file_info mfi;
 		struct diff_filespec other;
 		struct diff_filespec *add;
-		mfi = merge_file_one(o, one->path,
+
+		if (merge_file_one(o, one->path,
 				 one->sha1, one->mode,
 				 a->sha1, a->mode,
 				 b->sha1, b->mode,
-				 ci->branch1, ci->branch2);
+				 ci->branch1, ci->branch2, &mfi))
+			return;
+
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1235,12 +1239,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
+	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
 					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path);
-	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
+					    o->branch2, ci->ren1_other.path, &mfi_c1) ||
+	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path);
+					    o->branch2, c2->path, &mfi_c2))
+		return;
 
 	if (o->call_depth) {
 		/*
@@ -1461,10 +1466,11 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					mfi = merge_file_one(o, ren1_dst, null_sha1, 0,
+					if (merge_file_one(o, ren1_dst, null_sha1, 0,
 							 ren1->pair->two->sha1, ren1->pair->two->mode,
 							 dst_other.sha1, dst_other.mode,
-							 branch1, branch2);
+							 branch1, branch2, &mfi))
+						return -1;
 					output(o, 1, _("Adding merged %s"), ren1_dst);
 					update_file(o, 0, mfi.sha, mfi.mode, ren1_dst);
 					try_merge = 0;
@@ -1620,9 +1626,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_special_markers(o, &one, &a, &b,
+	if (merge_file_special_markers(o, &one, &a, &b,
 					 o->branch1, path1,
-					 o->branch2, path2);
+					 o->branch2, path2, &mfi))
+		return -1;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    sha_eq(mfi.sha, a_sha) && mfi.mode == a_mode) {
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 05/17] Prepare the builtins for a libified merge_recursive()

Previously, callers of merge_trees() or merge_recursive() expected that code to die() with an error message. This used to be okay because we called those commands from scripts, and had a chance to print out a message in case the command failed fatally (read: with exit code 128).

As scripting incurs its own set of problems (portability, speed, idiosynchracies of different shells, limited data structures leading to inefficient code), we are converting more and more of these scripts into builtins, using library functions directly.

We already tried to use merge_recursive() directly in the builtin git-am, for example. Unfortunately, we had to roll it back temporarily because some of the code in merge-recursive.c still deemed it okay to call die(), when the builtin am code really wanted to print out a useful advice after the merge failed fatally. In the next commits, we want to fix that.

The code touched by this commit expected merge_trees() to die() with some useful message when there is an error condition, but merge_trees() is going to be improved by converting all die() calls to return error() instead (i.e. return value -1 after printing out the message as before), so that the caller can react more flexibly.

This is a step to prepare for the version of merge_trees() that no longer dies, even if we just imitate the previous behavior by calling exit(128): this is what callers of e.g. `git merge` have come to expect.

Note that the callers of the sequencer (revert and cherry-pick) already fail fast even for the return value -1; The only difference is that they now get a chance to say "<command> failed".

A caller of merge_trees() might want handle error messages themselves (or even suppress them). As this patch is already complex enough, we leave that change for a later patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 4 +++-
 builtin/merge.c    | 2 ++
 sequencer.c        | 4 ++++
 3 files changed, 9 insertions(+), 1 deletion(-)
Show changes to 3 files +9 −1

builtin/checkout.c, builtin/merge.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index c3486bd..14312f7 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			o.ancestor = old->name;
 			o.branch1 = new->name;
 			o.branch2 = "local";
-			merge_trees(&o, new->commit->tree, work,
+			ret = merge_trees(&o, new->commit->tree, work,
 				old->commit->tree, &result);
+			if (ret < 0)
+				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
 			if (ret)
diff --git a/builtin/merge.c b/builtin/merge.c
index b555a1b..7b898db 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -682,6 +682,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
+		if (clean < 0)
+			exit(128);
 		if (active_cache_changed &&
 		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 			die (_("unable to write %s"), get_index_file());
diff --git a/sequencer.c b/sequencer.c
index c6362d6..13b794a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (clean < 0)
+		return clean;
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
@@ -561,6 +563,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
+		if (res < 0)
+			return res;
 		write_message(&msgbuf, git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 09/17] merge-recursive: handle return values indicating errors

We are about to libify the recursive merge machinery, where we only die() in case of a bug or memory contention. To that end, we must heed negative return values as indicating errors.

This requires our functions to be careful to pass through error conditions in call chains, and for quite a few functions this means that they have to return values to begin with.

The next step will be to convert the places where we currently die() to return negative values (read: -1) instead.

Note that we ignore errors reported by make_room_for_path(), consistent with the previous behavior (update_file_flags() used the return value of make_room_for_path() only to indicate an early return, but not a fatal error): if the error is really a fatal error, we will notice later; If not, it was not that serious a problem to begin with. (Witnesses in favor of this reasoning are t4151-am-abort and t7610-mergetool, which would start failing if we stopped on errors reported by make_room_for_path()).

Note: while this patch makes the code slightly less readable in
update_file_flags() (we introduce a new "goto free_buf;" instead of
an explicit "free(buf); return;"), it is a preparatory change for
the next patch where we will convert all of the die() calls in the same
function to go through the free_buf return path instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 196 +++++++++++++++++++++++++++++++++---------------------
 1 file changed, 121 insertions(+), 75 deletions(-)
Show changes to merge-recursive.c +121 −75
diff --git a/merge-recursive.c b/merge-recursive.c
index 99f4202..209427c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -734,7 +734,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	return error(msg, path, _(": perhaps a D/F conflict?"));
 }
 
-static void update_file_flags(struct merge_options *o,
+static int update_file_flags(struct merge_options *o,
 			      const unsigned char *sha,
 			      unsigned mode,
 			      const char *path,
@@ -775,8 +775,7 @@ static void update_file_flags(struct merge_options *o,
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
-			free(buf);
-			goto update_index;
+			goto free_buf;
 		}
 		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
 			int fd;
@@ -799,20 +798,22 @@ static void update_file_flags(struct merge_options *o,
 		} else
 			die(_("do not know what to do with %06o %s '%s'"),
 			    mode, sha1_to_hex(sha), path);
+ free_buf:
 		free(buf);
 	}
  update_index:
 	if (update_cache)
 		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+	return 0;
 }
 
-static void update_file(struct merge_options *o,
+static int update_file(struct merge_options *o,
 			int clean,
 			const unsigned char *sha,
 			unsigned mode,
 			const char *path)
 {
-	update_file_flags(o, sha, mode, path, o->call_depth || clean, !o->call_depth);
+	return update_file_flags(o, sha, mode, path, o->call_depth || clean, !o->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1008,7 +1009,7 @@ static int merge_file_one(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
-static void handle_change_delete(struct merge_options *o,
+static int handle_change_delete(struct merge_options *o,
 				 const char *path,
 				 const unsigned char *o_sha, int o_mode,
 				 const unsigned char *a_sha, int a_mode,
@@ -1016,6 +1017,7 @@ static void handle_change_delete(struct merge_options *o,
 				 const char *change, const char *change_past)
 {
 	char *renamed = NULL;
+	int ret = 0;
 	if (dir_in_way(path, !o->call_depth)) {
 		renamed = unique_path(o, path, a_sha ? o->branch1 : o->branch2);
 	}
@@ -1026,21 +1028,23 @@ static void handle_change_delete(struct merge_options *o,
 		 * correct; since there is no true "middle point" between
 		 * them, simply reuse the base version for virtual merge base.
 		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_sha, o_mode, renamed ? renamed : path);
+		ret = remove_file_from_cache(path);
+		if (!ret)
+			ret = update_file(o, 0, o_sha, o_mode,
+					  renamed ? renamed : path);
 	} else if (!a_sha) {
 		if (!renamed) {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path);
-			update_file(o, 0, b_sha, b_mode, path);
+			ret = update_file(o, 0, b_sha, b_mode, path);
 		} else {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path, renamed);
-			update_file(o, 0, b_sha, b_mode, renamed);
+			ret = update_file(o, 0, b_sha, b_mode, renamed);
 		}
 	} else {
 		if (!renamed) {
@@ -1053,7 +1057,7 @@ static void handle_change_delete(struct merge_options *o,
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch2, change_past,
 			       o->branch1, o->branch1, path, renamed);
-			update_file(o, 0, a_sha, a_mode, renamed);
+			ret = update_file(o, 0, a_sha, a_mode, renamed);
 		}
 		/*
 		 * No need to call update_file() on path when !renamed, since
@@ -1063,9 +1067,11 @@ static void handle_change_delete(struct merge_options *o,
 		 */
 	}
 	free(renamed);
+
+	return ret;
 }
 
-static void conflict_rename_delete(struct merge_options *o,
+static int conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
 				   const char *other_branch)
@@ -1085,21 +1091,19 @@ static void conflict_rename_delete(struct merge_options *o,
 		b_mode = dest->mode;
 	}
 
-	handle_change_delete(o,
+	if (handle_change_delete(o,
 			     o->call_depth ? orig->path : dest->path,
 			     orig->sha1, orig->mode,
 			     a_sha, a_mode,
 			     b_sha, b_mode,
-			     _("rename"), _("renamed"));
-
-	if (o->call_depth) {
-		remove_file_from_cache(dest->path);
-	} else {
-		update_stages(dest->path, NULL,
+			     _("rename"), _("renamed")))
+		return -1;
+	if (o->call_depth)
+		return remove_file_from_cache(dest->path);
+	else
+		return update_stages(dest->path, NULL,
 			      rename_branch == o->branch1 ? dest : NULL,
 			      rename_branch == o->branch1 ? NULL : dest);
-	}
-
 }
 
 static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
@@ -1115,7 +1119,7 @@ static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
-static void handle_file(struct merge_options *o,
+static int handle_file(struct merge_options *o,
 			struct diff_filespec *rename,
 			int stage,
 			struct rename_conflict_info *ci)
@@ -1125,6 +1129,7 @@ static void handle_file(struct merge_options *o,
 	const char *cur_branch, *other_branch;
 	struct diff_filespec other;
 	struct diff_filespec *add;
+	int ret;
 
 	if (stage == 2) {
 		dst_entry = ci->dst_entry1;
@@ -1139,7 +1144,8 @@ static void handle_file(struct merge_options *o,
 	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
 	if (add) {
 		char *add_name = unique_path(o, rename->path, other_branch);
-		update_file(o, 0, add->sha1, add->mode, add_name);
+		if (update_file(o, 0, add->sha1, add->mode, add_name))
+			return -1;
 
 		remove_file(o, 0, rename->path, 0);
 		dst_name = unique_path(o, rename->path, cur_branch);
@@ -1150,17 +1156,20 @@ static void handle_file(struct merge_options *o,
 			       rename->path, other_branch, dst_name);
 		}
 	}
-	update_file(o, 0, rename->sha1, rename->mode, dst_name);
-	if (stage == 2)
-		update_stages(rename->path, NULL, rename, add);
+	if ((ret = update_file(o, 0, rename->sha1, rename->mode, dst_name)))
+		; /* fall through, do allow dst_name to be released */
+	else if (stage == 2)
+		ret = update_stages(rename->path, NULL, rename, add);
 	else
-		update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
+
+	return ret;
 }
 
-static void conflict_rename_rename_1to2(struct merge_options *o,
+static int conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
@@ -1184,7 +1193,7 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 				 a->sha1, a->mode,
 				 b->sha1, b->mode,
 				 ci->branch1, ci->branch2, &mfi))
-			return;
+			return -1;
 
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
@@ -1192,7 +1201,8 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, mfi.sha, mfi.mode, one->path);
+		if (update_file(o, 0, mfi.sha, mfi.mode, one->path))
+			return -1;
 
 		/*
 		 * Above, we put the merged content at the merge-base's
@@ -1203,22 +1213,26 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * resolving the conflict at that path in its favor.
 		 */
 		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
-		if (add)
-			update_file(o, 0, add->sha1, add->mode, a->path);
+		if (add) {
+			if (update_file(o, 0, add->sha1, add->mode, a->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(a->path);
 		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
-		if (add)
-			update_file(o, 0, add->sha1, add->mode, b->path);
+		if (add) {
+			if (update_file(o, 0, add->sha1, add->mode, b->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(b->path);
-	} else {
-		handle_file(o, a, 2, ci);
-		handle_file(o, b, 3, ci);
-	}
+	} else if (handle_file(o, a, 2, ci) || handle_file(o, b, 3, ci))
+		return -1;
+
+	return 0;
 }
 
-static void conflict_rename_rename_2to1(struct merge_options *o,
+static int conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* Two files, a & b, were renamed to the same thing, c. */
@@ -1229,6 +1243,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	char *path = c1->path; /* == c2->path */
 	struct merge_file_info mfi_c1;
 	struct merge_file_info mfi_c2;
+	int ret;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1245,7 +1260,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 					    o->branch1, ci->ren2_other.path,
 					    o->branch2, c2->path, &mfi_c2))
-		return;
+		return -1;
 
 	if (o->call_depth) {
 		/*
@@ -1256,19 +1271,25 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 		 * again later for the non-recursive merge.
 		 */
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, mfi_c1.sha, mfi_c1.mode, a->path);
-		update_file(o, 0, mfi_c2.sha, mfi_c2.mode, b->path);
+		ret = update_file(o, 0, mfi_c1.sha, mfi_c1.mode, a->path);
+		if (!ret)
+			ret = update_file(o, 0, mfi_c2.sha, mfi_c2.mode,
+				b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, _("Renaming %s to %s and %s to %s instead"),
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, mfi_c1.sha, mfi_c1.mode, new_path1);
-		update_file(o, 0, mfi_c2.sha, mfi_c2.mode, new_path2);
+		ret = update_file(o, 0, mfi_c1.sha, mfi_c1.mode, new_path1);
+		if (!ret)
+			ret = update_file(o, 0, mfi_c2.sha, mfi_c2.mode,
+				new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
+
+	return ret;
 }
 
 static int process_renames(struct merge_options *o,
@@ -1451,12 +1472,13 @@ static int process_renames(struct merge_options *o,
 				 * update_file_flags() instead of
 				 * update_file().
 				 */
-				update_file_flags(o,
+				if (update_file_flags(o,
 						  ren1->pair->two->sha1,
 						  ren1->pair->two->mode,
 						  ren1_dst,
 						  1, /* update_cache */
-						  0  /* update_wd    */);
+						  0  /* update_wd    */))
+					clean_merge = -1;
 			} else if (!sha_eq(dst_other.sha1, null_sha1)) {
 				clean_merge = 0;
 				try_merge = 1;
@@ -1469,20 +1491,26 @@ static int process_renames(struct merge_options *o,
 					if (merge_file_one(o, ren1_dst, null_sha1, 0,
 							 ren1->pair->two->sha1, ren1->pair->two->mode,
 							 dst_other.sha1, dst_other.mode,
-							 branch1, branch2, &mfi))
-						return -1;
+							 branch1, branch2, &mfi)) {
+						clean_merge = -1;
+						goto cleanup_and_return;
+					}
 					output(o, 1, _("Adding merged %s"), ren1_dst);
-					update_file(o, 0, mfi.sha, mfi.mode, ren1_dst);
+					if (update_file(o, 0, mfi.sha, mfi.mode, ren1_dst))
+						clean_merge = -1;
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, _("Adding as %s instead"), new_path);
-					update_file(o, 0, dst_other.sha1, dst_other.mode, new_path);
+					if (update_file(o, 0, dst_other.sha1, dst_other.mode, new_path))
+						clean_merge = -1;
 					free(new_path);
 				}
 			} else
 				try_merge = 1;
 
+			if (clean_merge < 0)
+				goto cleanup_and_return;
 			if (try_merge) {
 				struct diff_filespec *one, *a, *b;
 				src_other.path = (char *)ren1_src;
@@ -1509,6 +1537,7 @@ static int process_renames(struct merge_options *o,
 			}
 		}
 	}
+cleanup_and_return:
 	string_list_clear(&a_by_dst, 0);
 	string_list_clear(&b_by_dst, 0);
 
@@ -1571,13 +1600,13 @@ error_return:
 	return ret;
 }
 
-static void handle_modify_delete(struct merge_options *o,
+static int handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 unsigned char *o_sha, int o_mode,
 				 unsigned char *a_sha, int a_mode,
 				 unsigned char *b_sha, int b_mode)
 {
-	handle_change_delete(o,
+	return handle_change_delete(o,
 			     path,
 			     o_sha, o_mode,
 			     a_sha, a_mode,
@@ -1656,7 +1685,8 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			update_stages(path, &one, &a, &b);
+			if (update_stages(path, &one, &a, &b))
+				return -1;
 	}
 
 	if (df_conflict_remains) {
@@ -1664,30 +1694,33 @@ static int merge_content(struct merge_options *o,
 		if (o->call_depth) {
 			remove_file_from_cache(path);
 		} else {
-			if (!mfi.clean)
-				update_stages(path, &one, &a, &b);
-			else {
+			if (!mfi.clean) {
+				if (update_stages(path, &one, &a, &b))
+					return -1;
+			} else {
 				int file_from_stage2 = was_tracked(path);
 				struct diff_filespec merged;
 				hashcpy(merged.sha1, mfi.sha);
 				merged.mode = mfi.mode;
 
-				update_stages(path, NULL,
+				if (update_stages(path, NULL,
 					      file_from_stage2 ? &merged : NULL,
-					      file_from_stage2 ? NULL : &merged);
+					      file_from_stage2 ? NULL : &merged))
+					return -1;
 			}
 
 		}
 		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, _("Adding as %s instead"), new_path);
-		update_file(o, 0, mfi.sha, mfi.mode, new_path);
+		if (update_file(o, 0, mfi.sha, mfi.mode, new_path)) {
+			free(new_path);
+			return -1;
+		}
 		free(new_path);
 		mfi.clean = 0;
-	} else {
-		update_file(o, mfi.clean, mfi.sha, mfi.mode, path);
-	}
+	} else if (update_file(o, mfi.clean, mfi.sha, mfi.mode, path))
+		return -1;
 	return mfi.clean;
-
 }
 
 /* Per entry merge function */
@@ -1715,17 +1748,21 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
-			conflict_rename_delete(o, conflict_info->pair1,
+			if (conflict_rename_delete(o,
+					       conflict_info->pair1,
 					       conflict_info->branch1,
-					       conflict_info->branch2);
+					       conflict_info->branch2))
+				clean_merge = -1;
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info);
+			if (conflict_rename_rename_1to2(o, conflict_info))
+				clean_merge = -1;
 			break;
 		case RENAME_TWO_FILES_TO_ONE:
 			clean_merge = 0;
-			conflict_rename_rename_2to1(o, conflict_info);
+			if (conflict_rename_rename_2to1(o, conflict_info))
+				clean_merge = -1;
 			break;
 		default:
 			entry->processed = 0;
@@ -1745,8 +1782,9 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_modify_delete(o, path, o_sha, o_mode,
-					     a_sha, a_mode, b_sha, b_mode);
+			if (handle_modify_delete(o, path, o_sha, o_mode,
+						 a_sha, a_mode, b_sha, b_mode))
+				clean_merge = -1;
 		}
 	} else if ((!o_sha && a_sha && !b_sha) ||
 		   (!o_sha && !a_sha && b_sha)) {
@@ -1778,14 +1816,16 @@ static int process_entry(struct merge_options *o,
 			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s"),
 			       conf, path, other_branch, path, new_path);
-			update_file(o, 0, sha, mode, new_path);
-			if (o->call_depth)
+			if (update_file(o, 0, sha, mode, new_path))
+				clean_merge = -1;
+			else if (o->call_depth)
 				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, _("Adding %s"), path);
 			/* do not overwrite file if already present */
-			update_file_flags(o, sha, mode, path, 1, !a_sha);
+			if (update_file_flags(o, sha, mode, path, 1, !a_sha))
+				clean_merge = -1;
 		}
 	} else if (a_sha && b_sha) {
 		/* Case C: Added in both (check for same permissions) and */
@@ -1848,12 +1888,18 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
+		if (clean < 0)
+			return clean;
 		for (i = entries->nr-1; 0 <= i; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_entry(o, path, e))
-				clean = 0;
+			if (!e->processed) {
+				int ret = process_entry(o, path, e);
+				if (!ret)
+					clean = 0;
+				else if (ret < 0)
+					return ret;
+			}
 		}
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 08/17] merge-recursive: allow write_tree_from_memory() to error out

It is possible that a tree cannot be written (think: disk full). We will want to give the caller a chance to clean up instead of letting the program die() in such a case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +2 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index 3e3667f..99f4202 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1873,8 +1873,8 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	if (o->call_depth)
-		*result = write_tree_from_memory(o);
+	if (o->call_depth && !(*result = write_tree_from_memory(o)))
+		return -1;
 
 	return clean;
 }
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 10/17] merge-recursive: switch to returning errors instead of dying

The recursive merge machinery is supposed to be a library function, i.e. it should return an error when it fails. Originally the functions were part of the builtin "merge-recursive", though, where it was simpler to call die() and be done with error handling.

The existing callers were already prepared to detect negative return values to indicate errors and to behave as previously: exit with code 128 (which is the same thing that die() does, after printing the message).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 59 +++++++++++++++++++++++++++++++------------------------
 1 file changed, 33 insertions(+), 26 deletions(-)
Show changes to merge-recursive.c +33 −26
diff --git a/merge-recursive.c b/merge-recursive.c
index 209427c..10010a4 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -266,8 +266,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		active_cache_tree = cache_tree();
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
-	    cache_tree_update(&the_index, 0) < 0)
-		die(_("error building trees"));
+	    cache_tree_update(&the_index, 0) < 0) {
+		error(_("error building trees"));
+		return NULL;
+	}
 
 	result = lookup_tree(active_cache_tree->sha1);
 
@@ -708,12 +710,10 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
-		if (status == SCLD_EXISTS) {
+		if (status == SCLD_EXISTS)
 			/* something else exists */
-			error(msg, path, _(": perhaps a D/F conflict?"));
-			return -1;
-		}
-		die(msg, path, "");
+			return error(msg, path, _(": perhaps a D/F conflict?"));
+		return error(msg, path, "");
 	}
 
 	/*
@@ -741,6 +741,8 @@ static int update_file_flags(struct merge_options *o,
 			      int update_cache,
 			      int update_wd)
 {
+	int ret = 0;
+
 	if (o->call_depth)
 		update_wd = 0;
 
@@ -761,9 +763,11 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(sha, &type, &size);
 		if (!buf)
-			die(_("cannot read object %s '%s'"), sha1_to_hex(sha), path);
-		if (type != OBJ_BLOB)
-			die(_("blob expected for %s '%s'"), sha1_to_hex(sha), path);
+			return error(_("cannot read object %s '%s'"), sha1_to_hex(sha), path);
+		if (type != OBJ_BLOB) {
+			ret = error(_("blob expected for %s '%s'"), sha1_to_hex(sha), path);
+			goto free_buf;
+		}
 		if (S_ISREG(mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
 			if (convert_to_working_tree(path, buf, size, &strbuf)) {
@@ -784,8 +788,10 @@ static int update_file_flags(struct merge_options *o,
 			else
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-			if (fd < 0)
-				die_errno(_("failed to open '%s'"), path);
+			if (fd < 0) {
+				ret = error_errno(_("failed to open '%s'"), path);
+				goto free_buf;
+			}
 			write_in_full(fd, buf, size);
 			close(fd);
 		} else if (S_ISLNK(mode)) {
@@ -793,18 +799,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				die_errno(_("failed to symlink '%s'"), path);
+				ret = error_errno(_("failed to symlink '%s'"), path);
 			free(lnk);
 		} else
-			die(_("do not know what to do with %06o %s '%s'"),
-			    mode, sha1_to_hex(sha), path);
+			ret = error(_("do not know what to do with %06o %s '%s'"),
+				mode, sha1_to_hex(sha), path);
  free_buf:
 		free(buf);
 	}
  update_index:
-	if (update_cache)
+	if (!ret && update_cache)
 		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
-	return 0;
+	return ret;
 }
 
 static int update_file(struct merge_options *o,
@@ -930,20 +936,22 @@ static int merge_file_1(struct merge_options *o,
 			hashcpy(result->sha, a->sha1);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
-			int merge_status;
+			int ret = 0, merge_status;
 
 			merge_status = merge_3way(o, &result_buf, one, a, b,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				die(_("Failed to execute internal merge"));
+				ret = error(_("Failed to execute internal merge"));
 
-			if (write_sha1_file(result_buf.ptr, result_buf.size,
+			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 					    blob_type, result->sha))
-				die(_("Unable to add %s to database"),
-				    a->path);
+				ret = error(_("Unable to add %s to database"),
+					a->path);
 
 			free(result_buf.ptr);
+			if (ret)
+				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
 			result->clean = merge_submodule(result->sha,
@@ -1868,11 +1876,10 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			die(_("merging of trees %s and %s failed"),
+			error(_("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
-		else
-			exit(128);
+		return -1;
 	}
 
 	if (unmerged_cache()) {
@@ -2003,7 +2010,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			die(_("merge returned no commit"));
+			return error(_("merge returned no commit"));
 	}
 
 	discard_cache();
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:24 UTC · re: Johannes Schindelin · lore

[PATCH v2 12/17] am -3: use merge_recursive() directly again

Last October, we had to change this code to run `git merge-recursive` in a child process: git-am wants to print some helpful advice when the merge failed, but the code in question was not prepared to return, it die()d instead.

We are finally at a point when the code *is* prepared to return errors, and can avoid the child process again.

This reverts commit c63d4b2 (am -3: do not let failed merge from completing the error codepath, 2015-10-09).

Note: the code now calls merge_recursive_generic() again. Unlike
merge_trees() and merge_recursive(), this function returns 0 upon success,
as most of Git's functions. Therefore, the error value -1 naturally is
handled correctly, and we do not have to take care of it specifically.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c | 49 ++++++++++++++++---------------------------------
 1 file changed, 16 insertions(+), 33 deletions(-)
Show changes to builtin/am.c +16 −33
diff --git a/builtin/am.c b/builtin/am.c
index f07f89a..be652f9 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1578,44 +1578,16 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
 }
 
 /**
- * Do the three-way merge using fake ancestor, her tree constructed
- * from the fake ancestor and the postimage of the patch, and our
- * state.
- */
-static int run_fallback_merge_recursive(const struct am_state *state,
-					unsigned char *orig_tree,
-					unsigned char *our_tree,
-					unsigned char *her_tree)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-
-	cp.git_cmd = 1;
-
-	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(her_tree), linelen(state->msg), state->msg);
-	if (state->quiet)
-		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
-
-	argv_array_push(&cp.args, "merge-recursive");
-	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
-	argv_array_push(&cp.args, "--");
-	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(her_tree));
-
-	status = run_command(&cp) ? (-1) : 0;
-	discard_cache();
-	read_cache();
-	return status;
-}
-
-/**
  * Attempt a threeway merge, using index_path as the temporary index.
  */
 static int fall_back_threeway(const struct am_state *state, const char *index_path)
 {
 	unsigned char orig_tree[GIT_SHA1_RAWSZ], her_tree[GIT_SHA1_RAWSZ],
 		      our_tree[GIT_SHA1_RAWSZ];
+	const unsigned char *bases[1] = {orig_tree};
+	struct merge_options o;
+	struct commit *result;
+	char *her_tree_name;
 
 	if (get_sha1("HEAD", our_tree) < 0)
 		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
@@ -1667,11 +1639,22 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	 * changes.
 	 */
 
-	if (run_fallback_merge_recursive(state, orig_tree, our_tree, her_tree)) {
+	init_merge_options(&o);
+
+	o.branch1 = "HEAD";
+	her_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
+	o.branch2 = her_tree_name;
+
+	if (state->quiet)
+		o.verbosity = 0;
+
+	if (merge_recursive_generic(&o, our_tree, her_tree, 1, bases, &result)) {
 		rerere(state->allow_rerere_autoupdate);
+		free(her_tree_name);
 		return error(_("Failed to merge in the changes."));
 	}
 
+	free(her_tree_name);
 	return 0;
 }
 
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:23 UTC · re: Johannes Schindelin · lore

[PATCH v2 11/17] am: counteract gender bias

Since d1c5f2a (Add git-am, applymbox replacement., 2005-10-07), i.e. for almost 11 years already, we demonstrated our disrespect to the pioneers of software development like Ada Lovelace, Grace Hopper and Margaret Hamilton, by pretending that each and every software developer is male ("his_tree"). It appears almost as if we weren't fully aware that the first professional software developers were all female.

We know our field to have this unfortunate gender bias that has nothing to do with qualification or biological reasons, and we are very sad about the current gender imbalance of the Git developer community.

Let's start changing that by using the variable name "her_tree" for an equal number of years out of fairness, and change to the gender neutral "their_tree" after that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
Show changes to builtin/am.c +8 −8
diff --git a/builtin/am.c b/builtin/am.c
index 3dfe70b..f07f89a 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1578,14 +1578,14 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
 }
 
 /**
- * Do the three-way merge using fake ancestor, his tree constructed
+ * Do the three-way merge using fake ancestor, her tree constructed
  * from the fake ancestor and the postimage of the patch, and our
  * state.
  */
 static int run_fallback_merge_recursive(const struct am_state *state,
 					unsigned char *orig_tree,
 					unsigned char *our_tree,
-					unsigned char *his_tree)
+					unsigned char *her_tree)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
 	int status;
@@ -1593,7 +1593,7 @@ static int run_fallback_merge_recursive(const struct am_state *state,
 	cp.git_cmd = 1;
 
 	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(his_tree), linelen(state->msg), state->msg);
+			 sha1_to_hex(her_tree), linelen(state->msg), state->msg);
 	if (state->quiet)
 		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
 
@@ -1601,7 +1601,7 @@ static int run_fallback_merge_recursive(const struct am_state *state,
 	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
 	argv_array_push(&cp.args, "--");
 	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(his_tree));
+	argv_array_push(&cp.args, sha1_to_hex(her_tree));
 
 	status = run_command(&cp) ? (-1) : 0;
 	discard_cache();
@@ -1614,7 +1614,7 @@ static int run_fallback_merge_recursive(const struct am_state *state,
  */
 static int fall_back_threeway(const struct am_state *state, const char *index_path)
 {
-	unsigned char orig_tree[GIT_SHA1_RAWSZ], his_tree[GIT_SHA1_RAWSZ],
+	unsigned char orig_tree[GIT_SHA1_RAWSZ], her_tree[GIT_SHA1_RAWSZ],
 		      our_tree[GIT_SHA1_RAWSZ];
 
 	if (get_sha1("HEAD", our_tree) < 0)
@@ -1651,7 +1651,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		return error(_("Did you hand edit your patch?\n"
 				"It does not apply to blobs recorded in its index."));
 
-	if (write_index_as_tree(his_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(her_tree, &the_index, index_path, 0, NULL))
 		return error("could not write tree");
 
 	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
@@ -1661,13 +1661,13 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 
 	/*
 	 * This is not so wrong. Depending on which base we picked, orig_tree
-	 * may be wildly different from ours, but his_tree has the same set of
+	 * may be wildly different from ours, but her_tree has the same set of
 	 * wildly different changes in parts the patch did not touch, so
 	 * recursive ends up canceling them, saying that we reverted all those
 	 * changes.
 	 */
 
-	if (run_fallback_merge_recursive(state, orig_tree, our_tree, his_tree)) {
+	if (run_fallback_merge_recursive(state, orig_tree, our_tree, her_tree)) {
 		rerere(state->allow_rerere_autoupdate);
 		return error(_("Failed to merge in the changes."));
 	}
-- 
2.9.0.280.g32e2a70
Junio C Hamano· Jul 6, 2016, 21:22 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 11/17] am: counteract gender bias

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 5 quoted lines
> Since d1c5f2a (Add git-am, applymbox replacement., 2005-10-07), i.e. for
> almost 11 years already,...
> ...Let's start changing that by using the variable name "her_tree" for an
> equal number of years out of fairness, and change to the gender neutral
> "their_tree" after that.

I doubt this kind fo distraction is desirable in the middle of a seriously heavy series like this one. As a standalone clean-up to turn these directly to "their" that everybody would agree on and can be merged down quickly to 'master' that does not have to keep the body of the main topic waiting for the dust to settle might be a better approach.

Unless you are trying to discourage the reviewers, that is ;-).
Johannes Schindelin· Jul 7, 2016, 11:30 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 11/17] am: counteract gender bias

Hi Junio,
On Wed, 6 Jul 2016, Junio C Hamano wrote:
Show 16 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Since d1c5f2a (Add git-am, applymbox replacement., 2005-10-07), i.e. for
> > almost 11 years already,...
> > ...Let's start changing that by using the variable name "her_tree" for an
> > equal number of years out of fairness, and change to the gender neutral
> > "their_tree" after that.
> 
> I doubt this kind fo distraction is desirable in the middle of a
> seriously heavy series like this one.  As a standalone clean-up to
> turn these directly to "their" that everybody would agree on and can
> be merged down quickly to 'master' that does not have to keep the
> body of the main topic waiting for the dust to settle might be a
> better approach.
> 
> Unless you are trying to discourage the reviewers, that is ;-).

Funny. In other comments, I am asked to patch things that are truly unrelated to the patch series' intent, and here I am asked to refrain from cleaning up the code before I touch it.

I am really curious, though. Has it not been our practice to encourage preparatory patches like white-space or const fixes as part of patch series that touch a certain part of the code that needed fixing? I deem this here patch to be much, much more important than a mere white-space or const fix.

Since you asked so nicely, I will break out this patch from the patch series, of course, but please note that it will now look as if I willfully snuck in an unrelated change in the next patch, just because I was not allowed to prepare the code properly.

Ciao, Dscho

Junio C Hamano· Jul 7, 2016, 15:26 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 11/17] am: counteract gender bias

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 10 quoted lines
>> I doubt this kind fo distraction is desirable in the middle of a
>> seriously heavy series like this one.  As a standalone clean-up to
>> turn these directly to "their" that everybody would agree on and can
>> be merged down quickly to 'master' that does not have to keep the
>> body of the main topic waiting for the dust to settle might be a
>> better approach.
>>  ...
> I am really curious, though. Has it not been our practice to encourage
> preparatory patches like white-space or const fixes as part of patch
> series that touch a certain part of the code that needed fixing?

Yes, isn't that "preparatory clean-up" what I said "might be a better approach"?

Johannes Schindelin· Jul 7, 2016, 15:54 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 11/17] am: counteract gender bias

Hi Junio,
On Thu, 7 Jul 2016, Junio C Hamano wrote:
Show 15 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> I doubt this kind fo distraction is desirable in the middle of a
> >> seriously heavy series like this one.  As a standalone clean-up to
> >> turn these directly to "their" that everybody would agree on and can
> >> be merged down quickly to 'master' that does not have to keep the
> >> body of the main topic waiting for the dust to settle might be a
> >> better approach.
> >>  ...
> > I am really curious, though. Has it not been our practice to encourage
> > preparatory patches like white-space or const fixes as part of patch
> > series that touch a certain part of the code that needed fixing?
> 
> Yes, isn't that "preparatory clean-up" what I said "might be a
> better approach"?

What I meant by "preparatory clean-up" is a patch in the patch series, just as I had it.

Now it is a separate patch "series".

Ciao, Dscho

Junio C Hamano· Jul 7, 2016, 16:03 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 11/17] am: counteract gender bias

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 24 quoted lines
> Hi Junio,
>
> On Thu, 7 Jul 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> >> I doubt this kind fo distraction is desirable in the middle of a
>> >> seriously heavy series like this one.  As a standalone clean-up to
>> >> turn these directly to "their" that everybody would agree on and can
>> >> be merged down quickly to 'master' that does not have to keep the
>> >> body of the main topic waiting for the dust to settle might be a
>> >> better approach.
>> >>  ...
>> > I am really curious, though. Has it not been our practice to encourage
>> > preparatory patches like white-space or const fixes as part of patch
>> > series that touch a certain part of the code that needed fixing?
>> 
>> Yes, isn't that "preparatory clean-up" what I said "might be a
>> better approach"?
>
> What I meant by "preparatory clean-up" is a patch in the patch series,
> just as I had it.
>
> Now it is a separate patch "series".

The main thing I had trouble with was to have it in the middle at 11/17. Since I do not have time to reorder other people's patches, every time you reroll the main series, this step will need to be looked at together with others. Having it near the beginning as preparatory clean-up, or even better a separate series that is an uncontroversial preparatory clean-up that the main topic depends on, is what I meant.

You build the main "series" on top of what 'master' would have when the separate one that "does not have to keep the body of the main topic waiting" gets merged. Because the separate one would be something "everybody would agree on and can be merged down quickly", that will not slow the main topic down.

Johannes Schindelin· Jul 5, 2016, 11:24 UTC · re: Johannes Schindelin · lore

[PATCH v2 13/17] merge-recursive: flush output buffer before printing error messages

The data structure passed to the recursive merge machinery has a feature where the caller can ask for the output to be buffered into a strbuf, by setting the field 'buffer_output'.

Previously, we simply swallowed the buffered output when showing error messages. With this patch, we show the output first, and only then print the error message.

Currently, the only user of that buffering is merge_recursive() itself, to avoid the progress output to interfere.

In the next patches, we will introduce a new buffer_output mode that forces merge_recursive() to retain the output buffer for further processing by the caller. If the caller asked for that, we will then also write the error messages into the output buffer. This is necessary to give the caller more control not only how to react in case of errors but also control how/if to display the error messages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 110 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 65 insertions(+), 45 deletions(-)
Show changes to merge-recursive.c +65 −45
diff --git a/merge-recursive.c b/merge-recursive.c
index 10010a4..0eb23a6 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,28 @@
 #include "dir.h"
 #include "submodule.h"
 
+static void flush_output(struct merge_options *o)
+{
+	if (o->obuf.len) {
+		fputs(o->obuf.buf, stdout);
+		strbuf_reset(&o->obuf);
+	}
+}
+
+static int err(struct merge_options *o, const char *err, ...)
+{
+	va_list params;
+
+	va_start(params, err);
+	flush_output(o);
+	strbuf_vaddf(&o->obuf, err, params);
+	error("%s", o->obuf.buf);
+	strbuf_reset(&o->obuf);
+	va_end(params);
+
+	return -1;
+}
+
 static struct tree *shift_tree_object(struct tree *one, struct tree *two,
 				      const char *subtree_shift)
 {
@@ -148,14 +170,6 @@ static int show(struct merge_options *o, int v)
 	return (!o->call_depth && o->verbosity >= v) || o->verbosity >= 5;
 }
 
-static void flush_output(struct merge_options *o)
-{
-	if (o->obuf.len) {
-		fputs(o->obuf.buf, stdout);
-		strbuf_reset(&o->obuf);
-	}
-}
-
 __attribute__((format (printf, 3, 4)))
 static void output(struct merge_options *o, int v, const char *fmt, ...)
 {
@@ -198,7 +212,8 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)
 	}
 }
 
-static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,
+static int add_cacheinfo(struct merge_options *o,
+		unsigned int mode, const unsigned char *sha1,
 		const char *path, int stage, int refresh, int options)
 {
 	struct cache_entry *ce;
@@ -206,7 +221,7 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,
 			      (refresh ? (CE_MATCH_REFRESH |
 					  CE_MATCH_IGNORE_MISSING) : 0 ));
 	if (!ce)
-		return error(_("addinfo_cache failed for path '%s'"), path);
+		return err(o, _("addinfo_cache failed for path '%s'"), path);
 	return add_cache_entry(ce, options);
 }
 
@@ -267,7 +282,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
 	    cache_tree_update(&the_index, 0) < 0) {
-		error(_("error building trees"));
+		err(o, _("error building trees"));
 		return NULL;
 	}
 
@@ -535,7 +550,8 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages(const char *path, const struct diff_filespec *o,
+static int update_stages(struct merge_options *opt, const char *path,
+			 const struct diff_filespec *o,
 			 const struct diff_filespec *a,
 			 const struct diff_filespec *b)
 {
@@ -554,13 +570,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 		if (remove_file_from_cache(path))
 			return -1;
 	if (o)
-		if (add_cacheinfo(o->mode, o->sha1, path, 1, 0, options))
+		if (add_cacheinfo(opt, o->mode, o->sha1, path, 1, 0, options))
 			return -1;
 	if (a)
-		if (add_cacheinfo(a->mode, a->sha1, path, 2, 0, options))
+		if (add_cacheinfo(opt, a->mode, a->sha1, path, 2, 0, options))
 			return -1;
 	if (b)
-		if (add_cacheinfo(b->mode, b->sha1, path, 3, 0, options))
+		if (add_cacheinfo(opt, b->mode, b->sha1, path, 3, 0, options))
 			return -1;
 	return 0;
 }
@@ -712,8 +728,8 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (status) {
 		if (status == SCLD_EXISTS)
 			/* something else exists */
-			return error(msg, path, _(": perhaps a D/F conflict?"));
-		return error(msg, path, "");
+			return err(o, msg, path, _(": perhaps a D/F conflict?"));
+		return err(o, msg, path, "");
 	}
 
 	/*
@@ -721,7 +737,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	 * tracking it.
 	 */
 	if (would_lose_untracked(path))
-		return error(_("refusing to lose untracked file at '%s'"),
+		return err(o, _("refusing to lose untracked file at '%s'"),
 			     path);
 
 	/* Successful unlink is good.. */
@@ -731,7 +747,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (errno == ENOENT)
 		return 0;
 	/* .. but not some other error (who really cares what?) */
-	return error(msg, path, _(": perhaps a D/F conflict?"));
+	return err(o, msg, path, _(": perhaps a D/F conflict?"));
 }
 
 static int update_file_flags(struct merge_options *o,
@@ -763,9 +779,9 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(sha, &type, &size);
 		if (!buf)
-			return error(_("cannot read object %s '%s'"), sha1_to_hex(sha), path);
+			return err(o, _("cannot read object %s '%s'"), sha1_to_hex(sha), path);
 		if (type != OBJ_BLOB) {
-			ret = error(_("blob expected for %s '%s'"), sha1_to_hex(sha), path);
+			ret = err(o, _("blob expected for %s '%s'"), sha1_to_hex(sha), path);
 			goto free_buf;
 		}
 		if (S_ISREG(mode)) {
@@ -789,7 +805,8 @@ static int update_file_flags(struct merge_options *o,
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
 			if (fd < 0) {
-				ret = error_errno(_("failed to open '%s'"), path);
+				ret = err(o, _("failed to open '%s': %s"),
+					path, strerror(errno));
 				goto free_buf;
 			}
 			write_in_full(fd, buf, size);
@@ -799,17 +816,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				ret = error_errno(_("failed to symlink '%s'"), path);
+				ret = err(o, _("failed to symlink '%s': %s"),
+					path, strerror(errno));
 			free(lnk);
 		} else
-			ret = error(_("do not know what to do with %06o %s '%s'"),
+			ret = err(o, _("do not know what to do with %06o %s '%s'"),
 				mode, sha1_to_hex(sha), path);
  free_buf:
 		free(buf);
 	}
  update_index:
 	if (!ret && update_cache)
-		add_cacheinfo(mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+		add_cacheinfo(o, mode, sha, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
 	return ret;
 }
 
@@ -942,11 +960,11 @@ static int merge_file_1(struct merge_options *o,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				ret = error(_("Failed to execute internal merge"));
+				ret = err(o, _("Failed to execute internal merge"));
 
 			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 					    blob_type, result->sha))
-				ret = error(_("Unable to add %s to database"),
+				ret = err(o, _("Unable to add %s to database"),
 					a->path);
 
 			free(result_buf.ptr);
@@ -1109,7 +1127,7 @@ static int conflict_rename_delete(struct merge_options *o,
 	if (o->call_depth)
 		return remove_file_from_cache(dest->path);
 	else
-		return update_stages(dest->path, NULL,
+		return update_stages(o, dest->path, NULL,
 			      rename_branch == o->branch1 ? dest : NULL,
 			      rename_branch == o->branch1 ? NULL : dest);
 }
@@ -1167,9 +1185,9 @@ static int handle_file(struct merge_options *o,
 	if ((ret = update_file(o, 0, rename->sha1, rename->mode, dst_name)))
 		; /* fall through, do allow dst_name to be released */
 	else if (stage == 2)
-		ret = update_stages(rename->path, NULL, rename, add);
+		ret = update_stages(o, rename->path, NULL, rename, add);
 	else
-		ret = update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(o, rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
@@ -1557,23 +1575,25 @@ static unsigned char *stage_sha(const unsigned char *sha, unsigned mode)
 	return (is_null_sha1(sha) || mode == 0) ? NULL: (unsigned char *)sha;
 }
 
-static int read_sha1_strbuf(const unsigned char *sha1, struct strbuf *dst)
+static int read_sha1_strbuf(struct merge_options *o,
+	const unsigned char *sha1, struct strbuf *dst)
 {
 	void *buf;
 	enum object_type type;
 	unsigned long size;
 	buf = read_sha1_file(sha1, &type, &size);
 	if (!buf)
-		return error(_("cannot read object %s"), sha1_to_hex(sha1));
+		return err(o, _("cannot read object %s"), sha1_to_hex(sha1));
 	if (type != OBJ_BLOB) {
 		free(buf);
-		return error(_("object %s is not a blob"), sha1_to_hex(sha1));
+		return err(o, _("object %s is not a blob"), sha1_to_hex(sha1));
 	}
 	strbuf_attach(dst, buf, size, size + 1);
 	return 0;
 }
 
-static int blob_unchanged(const unsigned char *o_sha,
+static int blob_unchanged(struct merge_options *opt,
+			  const unsigned char *o_sha,
 			  unsigned o_mode,
 			  const unsigned char *a_sha,
 			  unsigned a_mode,
@@ -1591,7 +1611,7 @@ static int blob_unchanged(const unsigned char *o_sha,
 		return 0;
 
 	assert(o_sha && a_sha);
-	if (read_sha1_strbuf(o_sha, &o) || read_sha1_strbuf(a_sha, &a))
+	if (read_sha1_strbuf(opt, o_sha, &o) || read_sha1_strbuf(opt, a_sha, &a))
 		goto error_return;
 	/*
 	 * Note: binary | is used so that both renormalizations are
@@ -1680,7 +1700,7 @@ static int merge_content(struct merge_options *o,
 		 */
 		path_renamed_outside_HEAD = !path2 || !strcmp(path, path2);
 		if (!path_renamed_outside_HEAD) {
-			add_cacheinfo(mfi.mode, mfi.sha, path,
+			add_cacheinfo(o, mfi.mode, mfi.sha, path,
 				      0, (!o->call_depth), 0);
 			return mfi.clean;
 		}
@@ -1693,7 +1713,7 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			if (update_stages(path, &one, &a, &b))
+			if (update_stages(o, path, &one, &a, &b))
 				return -1;
 	}
 
@@ -1703,7 +1723,7 @@ static int merge_content(struct merge_options *o,
 			remove_file_from_cache(path);
 		} else {
 			if (!mfi.clean) {
-				if (update_stages(path, &one, &a, &b))
+				if (update_stages(o, path, &one, &a, &b))
 					return -1;
 			} else {
 				int file_from_stage2 = was_tracked(path);
@@ -1711,7 +1731,7 @@ static int merge_content(struct merge_options *o,
 				hashcpy(merged.sha1, mfi.sha);
 				merged.mode = mfi.mode;
 
-				if (update_stages(path, NULL,
+				if (update_stages(o, path, NULL,
 					      file_from_stage2 ? &merged : NULL,
 					      file_from_stage2 ? NULL : &merged))
 					return -1;
@@ -1779,8 +1799,8 @@ static int process_entry(struct merge_options *o,
 	} else if (o_sha && (!a_sha || !b_sha)) {
 		/* Case A: Deleted in one */
 		if ((!a_sha && !b_sha) ||
-		    (!b_sha && blob_unchanged(o_sha, o_mode, a_sha, a_mode, normalize, path)) ||
-		    (!a_sha && blob_unchanged(o_sha, o_mode, b_sha, b_mode, normalize, path))) {
+		    (!b_sha && blob_unchanged(o, o_sha, o_mode, a_sha, a_mode, normalize, path)) ||
+		    (!a_sha && blob_unchanged(o, o_sha, o_mode, b_sha, b_mode, normalize, path))) {
 			/* Deleted in both or deleted in one and
 			 * unchanged in the other */
 			if (a_sha)
@@ -1876,7 +1896,7 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			error(_("merging of trees %s and %s failed"),
+			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
 		return -1;
@@ -2010,7 +2030,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			return error(_("merge returned no commit"));
+			return err(o, _("merge returned no commit"));
 	}
 
 	discard_cache();
@@ -2069,7 +2089,7 @@ int merge_recursive_generic(struct merge_options *o,
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
 			if (!(base = get_ref(base_list[i], sha1_to_hex(base_list[i]))))
-				return error(_("Could not parse object '%s'"),
+				return err(o, _("Could not parse object '%s'"),
 					sha1_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
 		}
@@ -2083,7 +2103,7 @@ int merge_recursive_generic(struct merge_options *o,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
-		return error(_("Unable to write index."));
+		return err(o, _("Unable to write index."));
 
 	return clean ? 0 : 1;
 }
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:24 UTC · re: Johannes Schindelin · lore

[PATCH v2 14/17] merge-recursive: write the commit title in one go

In 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we changed the code such that it prints the output in one go, to avoid interfering with the progress output.

Let's make sure that the same holds true when outputting the commit title: previously, we used several printf() statements to stdout and speculated that stdout's buffer is large enough to hold the entire commit title.

Apart from making that speculation unnecessary, we change the code to add the message to the output buffer before flushing for another reason: the next commit will introduce a new level of output buffering, where the caller can request the output not to be flushed, but to be retained for further processing.

This latter feature will be needed when teaching the sequencer to do rebase -i's brunt work: it wants to control the output of the cherry-picks (i.e. recursive merges).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
Show changes to merge-recursive.c +9 −8
diff --git a/merge-recursive.c b/merge-recursive.c
index 0eb23a6..81836f2 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -191,25 +191,26 @@ static void output(struct merge_options *o, int v, const char *fmt, ...)
 
 static void output_commit_title(struct merge_options *o, struct commit *commit)
 {
-	int i;
-	flush_output(o);
-	for (i = o->call_depth; i--;)
-		fputs("  ", stdout);
+	strbuf_addchars(&o->obuf, ' ', o->call_depth * 2);
 	if (commit->util)
-		printf("virtual %s\n", merge_remote_util(commit)->name);
+		strbuf_addf(&o->obuf, "virtual %s\n",
+			merge_remote_util(commit)->name);
 	else {
-		printf("%s ", find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV));
+		strbuf_addf(&o->obuf, "%s ",
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV));
 		if (parse_commit(commit) != 0)
-			printf(_("(bad commit)\n"));
+			strbuf_addf(&o->obuf, _("(bad commit)\n"));
 		else {
 			const char *title;
 			const char *msg = get_commit_buffer(commit, NULL);
 			int len = find_commit_subject(msg, &title);
 			if (len)
-				printf("%.*s\n", len, title);
+				strbuf_addf(&o->obuf, "%.*s\n", len, title);
 			unuse_commit_buffer(commit, msg);
 		}
 	}
+	flush_output(o);
 }
 
 static int add_cacheinfo(struct merge_options *o,
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:24 UTC · re: Johannes Schindelin · lore

[PATCH v2 16/17] Ensure that the output buffer is released after calling merge_trees()

The recursive merge machinery accumulates its output in an output buffer, to be flushed at the end of merge_recursive(). At this point, we forgot to release the output buffer.

When calling merge_trees() (i.e. the non-recursive part of the recursive merge) directly, the output buffer is never flushed because the caller may be merge_recursive() which wants to flush the output itself.

For the same reason, merge_trees() cannot release the output buffer: it may still be needed.

Forgetting to release the output buffer did not matter much when running git-checkout, or git-merge-recursive, because we exited after the operation anyway. Ever since cherry-pick learned to pick a commit range, however, this memory leak had the potential of becoming a problem.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 1 +
 merge-recursive.c  | 2 ++
 sequencer.c        | 1 +
 3 files changed, 4 insertions(+)
Show changes to 3 files +4 −0

builtin/checkout.c, merge-recursive.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 14312f7..ced4ac4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -573,6 +573,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
+			strbuf_release(&o.obuf);
 			if (ret)
 				return ret;
 		}
diff --git a/merge-recursive.c b/merge-recursive.c
index 29cbdac..fdc624a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2059,6 +2059,8 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
 	flush_output(o);
+	if (o->buffer_output < 2)
+		strbuf_release(&o->obuf);
 	if (show(o, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       o->needed_rename_limit, 0);
diff --git a/sequencer.c b/sequencer.c
index 13b794a..8ceeb1b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
 
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:24 UTC · re: Johannes Schindelin · lore

[PATCH v2 15/17] merge-recursive: offer an option to retain the output in 'obuf'

Since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we already accumulate the output in a buffer. The idea was to avoid interfering with the progress output that goes to stderr, which is unbuffered, when we write to stdout, which is buffered.

We extend that buffering to allow the caller to handle the output (possibly suppressing it). This will help us when extending the sequencer to do rebase -i's brunt work: it does not want the picks to print anything by default but instead determine itself whether to print the output or not.

Note that we also redirect the error messages into the output buffer
when the caller asked not to flush the output buffer, for two reasons:
1) to retain the correct output order, and 2) to allow the caller to
suppress *all* output.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++++++----
 merge-recursive.h |  2 +-
 2 files changed, 14 insertions(+), 5 deletions(-)
Show changes to 2 files +14 −5

merge-recursive.c, merge-recursive.h

diff --git a/merge-recursive.c b/merge-recursive.c
index 81836f2..29cbdac 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -25,7 +25,7 @@
 
 static void flush_output(struct merge_options *o)
 {
-	if (o->obuf.len) {
+	if (o->buffer_output < 2 && o->obuf.len) {
 		fputs(o->obuf.buf, stdout);
 		strbuf_reset(&o->obuf);
 	}
@@ -36,10 +36,19 @@ static int err(struct merge_options *o, const char *err, ...)
 	va_list params;
 
 	va_start(params, err);
-	flush_output(o);
+	if (o->buffer_output < 2)
+		flush_output(o);
+	else {
+		strbuf_complete(&o->obuf, '\n');
+		strbuf_addstr(&o->obuf, "error: ");
+	}
 	strbuf_vaddf(&o->obuf, err, params);
-	error("%s", o->obuf.buf);
-	strbuf_reset(&o->obuf);
+	if (o->buffer_output > 1)
+		strbuf_addch(&o->obuf, '\n');
+	else {
+		error("%s", o->obuf.buf);
+		strbuf_reset(&o->obuf);
+	}
 	va_end(params);
 
 	return -1;
diff --git a/merge-recursive.h b/merge-recursive.h
index 52f0201..407d4fc 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -13,7 +13,7 @@ struct merge_options {
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
 	const char *subtree_shift;
-	unsigned buffer_output : 1;
+	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-- 
2.9.0.280.g32e2a70
Johannes Schindelin· Jul 5, 2016, 11:24 UTC · re: Johannes Schindelin · lore

[PATCH v2 17/17] merge-recursive: flush output buffer even when erroring out

Ever since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we had a problem: When the merge failed in a fatal way, all regular output was swallowed because we called die() and did not get a chance to drain the output buffers.

To fix this, several modifications were necessary:
- we needed to stop die()ing, to give callers a chance to do something
  when an error occurred (in this case, flush the output buffers),
- we needed to delay printing the error message so that the caller can
  print the buffered output before that, and
- we needed to make sure that the output buffers are flushed even when
  the return value indicates an error.

The first two changes were introduced through earlier commits in this patch series, and this commit addresses the third one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to merge-recursive.c +1 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index fdc624a..d94f853 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2050,6 +2050,7 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	flush_output(o);
 	if (clean < 0)
 		return clean;
 
@@ -2058,7 +2059,6 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(o);
 	if (o->buffer_output < 2)
 		strbuf_release(&o->obuf);
 	if (show(o, 2))
-- 
2.9.0.280.g32e2a70
Junio C Hamano· Jul 6, 2016, 21:26 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 00/17] Use merge_recursive() directly in the builtin am

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> This is the second iteration of the long-awaited re-roll of the attempt to
> avoid spawning merge-recursive from the builtin am and use merge_recursive()
> directly instead.

I wanted to queue this in 'pu', but an unfortunate series that changes the convert_to_git() infrastructure has serious conflicts with the changes in this series. I am still unsure if these changes cannot be done without butchering the calling convention of what leads to convert_to_git(), but in the meantime, this cannot yet be merged to 'pu'.

Johannes Schindelin· Jul 7, 2016, 11:16 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 00/17] Use merge_recursive() directly in the builtin am

Hi Junio,
On Wed, 6 Jul 2016, Junio C Hamano wrote:
Show 12 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This is the second iteration of the long-awaited re-roll of the attempt to
> > avoid spawning merge-recursive from the builtin am and use merge_recursive()
> > directly instead.
> 
> I wanted to queue this in 'pu', but an unfortunate series that
> changes the convert_to_git() infrastructure has serious conflicts
> with the changes in this series.  I am still unsure if these
> changes cannot be done without butchering the calling convention
> of what leads to convert_to_git(), but in the meantime, this cannot
> yet be merged to 'pu'.

There is nothing butchered there. The secret to the merge conflicts is the sha1 -> oid conversion paired with Vasco's untranslating of BUG: reports without upcasing the "bug:" prefix.

I will rebase to `pu` and re-send.

Ciao, Dscho

Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 00/16] Use merge_recursive() directly in the builtin am

This is the second iteration of the long-awaited re-roll of the attempt to avoid spawning merge-recursive from the builtin am and use merge_recursive() directly instead.

The *real* reason for the reroll is that I need a libified recursive merge to accelerate the interactive rebase by teaching the sequencer to do rebase -i's grunt work.

In this endeavor, we need to be extra careful to retain backwards compatibility. The test script t6022-merge-rename.sh, for example, verifies that `git pull` exits with status 128 in case of a fatal error. To that end, we need to make sure that fatal errors are handled by existing (builtin) users via exit(128) (or die(), which calls exit(128) at the end). New users (such as a builtin helper doing rebase -i's grunt work) may want to print some helpful advice what happened and how to get out of this mess before erroring out.

The changes relative to the second iteration of this patch series:
- the was_tracked() function was adjusted as per Junio's suggestions
- the "counter gender bias" patch was submitted separately, in
  $gmane/299009 (please note that the "am -3: use merge_recursive()
  directly again" patch is now slightly awkward as a consequence)
- this patch series is on top of 'pu', to address the conflicts with
  the 'jh/clean-smudge-annex' and the 'bc/cocci' branches
- please note that the interdiff does not show the full picture: I
  generated it relative to v2 rebased on top of pu (resolving many
  merge conflicts in the process that are hidden from the interdiff)

This patch series touches rather important code. Now that I addressed concerns such as fixing translated bug reports, I would appreciate thorough reviews with a focus on the critical parts of the code, those that could result in regressions.

Johannes Schindelin (16):
  Verify that `git pull --rebase` shows the helpful advice when failing
  Report bugs consistently
  Avoid translating bug messages
  merge-recursive: clarify code in was_tracked()
  Prepare the builtins for a libified merge_recursive()
  merge_recursive: abort properly upon errors
  merge-recursive: avoid returning a wholesale struct
  merge-recursive: allow write_tree_from_memory() to error out
  merge-recursive: handle return values indicating errors
  merge-recursive: switch to returning errors instead of dying
  am -3: use merge_recursive() directly again
  merge-recursive: flush output buffer before printing error messages
  merge-recursive: write the commit title in one go
  merge-recursive: offer an option to retain the output in 'obuf'
  Ensure that the output buffer is released after calling merge_trees()
  merge-recursive: flush output buffer even when erroring out
 builtin/am.c           |  63 +++---
 builtin/checkout.c     |   5 +-
 builtin/ls-files.c     |   3 +-
 builtin/merge.c        |   2 +
 builtin/update-index.c |   2 +-
 grep.c                 |   8 +-
 imap-send.c            |   4 +-
 merge-recursive.c      | 508 +++++++++++++++++++++++++++++--------------------
 merge-recursive.h      |   2 +-
 sequencer.c            |   5 +
 sha1_file.c            |   4 +-
 t/t5520-pull.sh        |  30 +++
 trailer.c              |   2 +-
 transport.c            |   2 +-
 wt-status.c            |   4 +-
 15 files changed, 382 insertions(+), 262 deletions(-)
Published-As: https://github.com/dscho/git/releases/tag/am-3-merge-recursive-direct-v3
Interdiff vs v2:
 diff --git a/builtin/am.c b/builtin/am.c
 index d626532..8dc4239 100644
 --- a/builtin/am.c
 +++ b/builtin/am.c
 @@ -29,6 +29,7 @@
  #include "prompt.h"
  #include "mailinfo.h"
  #include "apply.h"
 +#include "object.h"
  
  /**
   * Returns the length of the first line of msg.
 @@ -1627,15 +1628,14 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
   */
  static int fall_back_threeway(const struct am_state *state, const char *index_path)
  {
 -	unsigned char orig_tree[GIT_SHA1_RAWSZ], her_tree[GIT_SHA1_RAWSZ],
 -		      our_tree[GIT_SHA1_RAWSZ];
 -	const unsigned char *bases[1] = {orig_tree};
 +	struct object_id orig_tree, her_tree, our_tree;
 +	const struct object_id *bases[1] = { &orig_tree };
  	struct merge_options o;
  	struct commit *result;
  	char *her_tree_name;
  
 -	if (get_sha1("HEAD", our_tree) < 0)
 -		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
 +	if (get_oid("HEAD", &our_tree) < 0)
 +		hashcpy(our_tree.hash, EMPTY_TREE_SHA1_BIN);
  
  	if (build_fake_ancestor(state, index_path))
  		return error("could not build fake ancestor");
 @@ -1643,7 +1643,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  	discard_cache();
  	read_cache_from(index_path);
  
 -	if (write_index_as_tree(orig_tree, &the_index, index_path, 0, NULL))
 +	if (write_index_as_tree(orig_tree.hash, &the_index, index_path, 0, NULL))
  		return error(_("Repository lacks necessary blobs to fall back on 3-way merge."));
  
  	say(state, stdout, _("Using index info to reconstruct a base tree..."));
 @@ -1659,7 +1659,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  		init_revisions(&rev_info, NULL);
  		rev_info.diffopt.output_format = DIFF_FORMAT_NAME_STATUS;
  		diff_opt_parse(&rev_info.diffopt, &diff_filter_str, 1, rev_info.prefix);
 -		add_pending_sha1(&rev_info, "HEAD", our_tree, 0);
 +		add_pending_sha1(&rev_info, "HEAD", our_tree.hash, 0);
  		diff_setup_done(&rev_info.diffopt);
  		run_diff_index(&rev_info, 1);
  	}
 @@ -1668,7 +1668,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  		return error(_("Did you hand edit your patch?\n"
  				"It does not apply to blobs recorded in its index."));
  
 -	if (write_index_as_tree(her_tree, &the_index, index_path, 0, NULL))
 +	if (write_index_as_tree(her_tree.hash, &the_index, index_path, 0, NULL))
  		return error("could not write tree");
  
  	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
 @@ -1678,7 +1678,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  
  	/*
  	 * This is not so wrong. Depending on which base we picked, orig_tree
 -	 * may be wildly different from ours, but her_tree has the same set of
 +	 * may be wildly different from ours, but his_tree has the same set of
  	 * wildly different changes in parts the patch did not touch, so
  	 * recursive ends up canceling them, saying that we reverted all those
  	 * changes.
 @@ -1693,7 +1693,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  	if (state->quiet)
  		o.verbosity = 0;
  
 -	if (merge_recursive_generic(&o, our_tree, her_tree, 1, bases, &result)) {
 +	if (merge_recursive_generic(&o, &our_tree, &her_tree, 1, bases, &result)) {
  		rerere(state->allow_rerere_autoupdate);
  		free(her_tree_name);
  		return error(_("Failed to merge in the changes."));
 diff --git a/merge-recursive.c b/merge-recursive.c
 index f80ad35..cb701ee 100644
 --- a/merge-recursive.c
 +++ b/merge-recursive.c
 @@ -686,20 +686,19 @@ static int was_tracked(const char *path)
  {
  	int pos = cache_name_pos(path, strlen(path));
  
 -	if (pos >= 0)
 -		return pos < active_nr;
 +	if (0 <= pos)
 +		/* we have been tracking this path */
 +		return 1;
 +
  	/*
 -	 * cache_name_pos() looks for stage == 0, even if we did not ask for
 -	 * it. Let's look for stage == 2 now.
 +	 * Look for an unmerged entry for the path,
 +	 * specifically stage #2, which would indicate
 +	 * that "our" side before the merge started
 +	 * had the path tracked (and resulted in a conflict).
  	 */
 -	for (pos = -1 - pos; pos < active_nr &&
 -	     !strcmp(path, active_cache[pos]->name); pos++)
 -		/*
 -		 * If stage #0, it is definitely tracked.
 -		 * If it has stage #2 then it was tracked
 -		 * before this merge started.  All other
 -		 * cases the path was not tracked.
 -		 */
 +	for (pos = -1 - pos;
 +	     pos < active_nr && !strcmp(path, active_cache[pos]->name);
 +	     pos++)
  		if (ce_stage(active_cache[pos]) == 2)
  			return 1;
  	return 0;
-- 
2.9.0.278.g1caae67

base-commit: 6addd022ce5331ee7dc41781ded714e5d5f01206
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 02/16] Report bugs consistently

The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:".

As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/ls-files.c     |  3 ++-
 builtin/update-index.c |  2 +-
 grep.c                 |  8 ++++----
 imap-send.c            |  4 ++--
 merge-recursive.c      | 15 +++++++--------
 sha1_file.c            |  4 ++--
 trailer.c              |  2 +-
 transport.c            |  2 +-
 wt-status.c            |  4 ++--
 9 files changed, 22 insertions(+), 22 deletions(-)
Show changes to 9 files +22 −22

builtin/ls-files.c, builtin/update-index.c, grep.c, imap-send.c, merge-recursive.c, sha1_file.c, trailer.c, transport.c, wt-status.c

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f02e3d2..00ea91a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
 				 */
 				pos = cache_name_pos(ent->name, ent->len);
 				if (0 <= pos)
-					die("bug in show-killed-files");
+					die("BUG: killed-file %.*s not found",
+						ent->len, ent->name);
 				pos = -pos - 1;
 				while (pos < active_nr &&
 				       ce_stage(active_cache[pos]))
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 25af040..881a4b0 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1149,7 +1149,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		report(_("Untracked cache enabled for '%s'"), get_git_work_tree());
 		break;
 	default:
-		die("Bug: bad untracked_cache value: %d", untracked_cache);
+		die("BUG: bad untracked_cache value: %d", untracked_cache);
 	}
 
 	if (use_watchman > 0) {
diff --git a/grep.c b/grep.c
index 394c856..22cbb73 100644
--- a/grep.c
+++ b/grep.c
@@ -693,10 +693,10 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 	for (p = opt->header_list; p; p = p->next) {
 		if (p->token != GREP_PATTERN_HEAD)
-			die("bug: a non-header pattern in grep header list.");
+			die("BUG: a non-header pattern in grep header list.");
 		if (p->field < GREP_HEADER_FIELD_MIN ||
 		    GREP_HEADER_FIELD_MAX <= p->field)
-			die("bug: unknown header field %d", p->field);
+			die("BUG: unknown header field %d", p->field);
 		compile_regexp(p, opt);
 	}
 
@@ -709,7 +709,7 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 		h = compile_pattern_atom(&pp);
 		if (!h || pp != p->next)
-			die("bug: malformed header expr");
+			die("BUG: malformed header expr");
 		if (!header_group[p->field]) {
 			header_group[p->field] = h;
 			continue;
@@ -1514,7 +1514,7 @@ static int grep_source_1(struct grep_opt *opt, struct grep_source *gs, int colle
 		case GREP_BINARY_TEXT:
 			break;
 		default:
-			die("bug: unknown binary handling mode");
+			die("BUG: unknown binary handling mode");
 		}
 	}
 
diff --git a/imap-send.c b/imap-send.c
index db0fafe..67d67f8 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -506,12 +506,12 @@ static char *next_arg(char **s)
 
 static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 {
-	int ret;
+	int ret = -1;
 	va_list va;
 
 	va_start(va, fmt);
 	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
-		die("Fatal: buffer too small. Please report a bug.");
+		die("BUG: buffer too small (%d < %d)", ret, blen);
 	va_end(va);
 	return ret;
 }
diff --git a/merge-recursive.c b/merge-recursive.c
index 067f656..05b9789 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -259,7 +259,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
-		die("Bug in merge-recursive.c");
+		die("BUG: unmerged index entries in merge-recursive.c");
 	}
 
 	if (!active_cache_tree)
@@ -982,9 +982,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
-		} else {
-			die(_("unsupported object type in the tree"));
-		}
+		} else
+			die(_("BUG: unsupported object type in the tree"));
 	}
 
 	return result;
@@ -1370,7 +1369,7 @@ static int process_renames(struct merge_options *o,
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1_src != ren2_src");
+				die("BUG: ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
@@ -1404,7 +1403,7 @@ static int process_renames(struct merge_options *o,
 			ren2 = lookup->util;
 			ren2_dst = ren2->pair->two->path;
 			if (strcmp(ren1_dst, ren2_dst) != 0)
-				die("ren1_dst != ren2_dst");
+				die("BUG: ren1_dst != ren2_dst");
 
 			clean_merge = 0;
 			ren2->processed = 1;
@@ -1828,7 +1827,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("Fatal merge failure, shouldn't happen."));
+		die(_("BUG: fatal merge failure, shouldn't happen."));
 
 	return clean_merge;
 }
@@ -1886,7 +1885,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("Unprocessed path??? %s"),
+				die(_("BUG: unprocessed path??? %s"),
 				    entries->items[i].string);
 		}
 
diff --git a/sha1_file.c b/sha1_file.c
index df62eaf..27dadff 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -795,7 +795,7 @@ void close_all_packs(void)
 
 	for (p = packed_git; p; p = p->next)
 		if (p->do_not_close)
-			die("BUG! Want to close pack marked 'do-not-close'");
+			die("BUG: want to close pack marked 'do-not-close'");
 		else
 			close_pack(p);
 }
@@ -2336,7 +2336,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 	case OBJ_OFS_DELTA:
 	case OBJ_REF_DELTA:
 		if (data)
-			die("BUG in unpack_entry: left loop at a valid delta");
+			die("BUG: unpack_entry: left loop at a valid delta");
 		break;
 	case OBJ_COMMIT:
 	case OBJ_TREE:
diff --git a/trailer.c b/trailer.c
index 8e48a5c..c6ea9ac 100644
--- a/trailer.c
+++ b/trailer.c
@@ -562,7 +562,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
-		die("internal bug in trailer.c");
+		die("BUG: trailer.c: unhandled type %d", type);
 	}
 	return 0;
 }
diff --git a/transport.c b/transport.c
index ad9ac15..d1dd29a 100644
--- a/transport.c
+++ b/transport.c
@@ -577,7 +577,7 @@ void transport_take_over(struct transport *transport,
 	struct git_transport_data *data;
 
 	if (!transport->smart_options)
-		die("Bug detected: Taking over transport requires non-NULL "
+		die("BUG: taking over transport requires non-NULL "
 		    "smart_options field.");
 
 	data = xcalloc(1, sizeof(*data));
diff --git a/wt-status.c b/wt-status.c
index de62ab2..3fb86a4 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die("bug: unhandled unmerged status %x", stagemask);
+		die("BUG: unhandled unmerged status %x", stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die("bug: unhandled diff status %c", status);
+		die("BUG: unhandled diff status %c", status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 03/16] Avoid translating bug messages

While working on the patch series that avoids die()ing in recursive merges, the issue came up that bug reports (i.e. die("BUG: ...") constructs) should never be translated, as the target audience is the Git developer community, not necessarily the current user, and hence a translated message would make it *harder* to address the problem.

So let's stop translating the obvious ones. As it is really, really outside the purview of this patch series to see whether there are more die() statements that report bugs and are currently translated, that task is left for another day and patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +3 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 05b9789..ea0df22 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -983,7 +983,7 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
 		} else
-			die(_("BUG: unsupported object type in the tree"));
+			die("BUG: unsupported object type in the tree");
 	}
 
 	return result;
@@ -1827,7 +1827,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("BUG: fatal merge failure, shouldn't happen."));
+		die("BUG: fatal merge failure, shouldn't happen.");
 
 	return clean_merge;
 }
@@ -1885,7 +1885,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("BUG: unprocessed path??? %s"),
+				die("BUG: unprocessed path??? %s",
 				    entries->items[i].string);
 		}
 
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 05/16] Prepare the builtins for a libified merge_recursive()

Previously, callers of merge_trees() or merge_recursive() expected that code to die() with an error message. This used to be okay because we called those commands from scripts, and had a chance to print out a message in case the command failed fatally (read: with exit code 128).

As scripting incurs its own set of problems (portability, speed, idiosynchracies of different shells, limited data structures leading to inefficient code), we are converting more and more of these scripts into builtins, using library functions directly.

We already tried to use merge_recursive() directly in the builtin git-am, for example. Unfortunately, we had to roll it back temporarily because some of the code in merge-recursive.c still deemed it okay to call die(), when the builtin am code really wanted to print out a useful advice after the merge failed fatally. In the next commits, we want to fix that.

The code touched by this commit expected merge_trees() to die() with some useful message when there is an error condition, but merge_trees() is going to be improved by converting all die() calls to return error() instead (i.e. return value -1 after printing out the message as before), so that the caller can react more flexibly.

This is a step to prepare for the version of merge_trees() that no longer dies, even if we just imitate the previous behavior by calling exit(128): this is what callers of e.g. `git merge` have come to expect.

Note that the callers of the sequencer (revert and cherry-pick) already fail fast even for the return value -1; The only difference is that they now get a chance to say "<command> failed".

A caller of merge_trees() might want handle error messages themselves (or even suppress them). As this patch is already complex enough, we leave that change for a later patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 4 +++-
 builtin/merge.c    | 2 ++
 sequencer.c        | 4 ++++
 3 files changed, 9 insertions(+), 1 deletion(-)
Show changes to 3 files +9 −1

builtin/checkout.c, builtin/merge.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 27c1a05..07dea3b 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			o.ancestor = old->name;
 			o.branch1 = new->name;
 			o.branch2 = "local";
-			merge_trees(&o, new->commit->tree, work,
+			ret = merge_trees(&o, new->commit->tree, work,
 				old->commit->tree, &result);
+			if (ret < 0)
+				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
 			if (ret)
diff --git a/builtin/merge.c b/builtin/merge.c
index 46b88ad..6837e15 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -682,6 +682,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
+		if (clean < 0)
+			exit(128);
 		if (active_cache_changed &&
 		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 			die (_("unable to write %s"), get_index_file());
diff --git a/sequencer.c b/sequencer.c
index cdfac82..286a435 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (clean < 0)
+		return clean;
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
@@ -559,6 +561,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
+		if (res < 0)
+			return res;
 		write_message(&msgbuf, git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 06/16] merge_recursive: abort properly upon errors

There are a couple of places where return values indicating errors are ignored. Let's teach them manners.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +8 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index 469741d..37c181a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1974,8 +1974,9 @@ int merge_recursive(struct merge_options *o,
 		saved_b2 = o->branch2;
 		o->branch1 = "Temporary merge branch 1";
 		o->branch2 = "Temporary merge branch 2";
-		merge_recursive(o, merged_common_ancestors, iter->item,
-				NULL, &merged_common_ancestors);
+		if (merge_recursive(o, merged_common_ancestors, iter->item,
+				NULL, &merged_common_ancestors) < 0)
+			return -1;
 		o->branch1 = saved_b1;
 		o->branch2 = saved_b2;
 		o->call_depth--;
@@ -1991,6 +1992,8 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	if (clean < 0)
+		return clean;
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
@@ -2047,6 +2050,9 @@ int merge_recursive_generic(struct merge_options *o,
 	hold_locked_index(lock, 1);
 	clean = merge_recursive(o, head_commit, next_commit, ca,
 			result);
+	if (clean < 0)
+		return clean;
+
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 		return error(_("Unable to write index."));
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 04/16] merge-recursive: clarify code in was_tracked()

It can be puzzling to see that was_tracked() asks to get an index entry by name, but does not take a negative return value for an answer.

The reason we have to do this is that cache_name_pos() only looks for entries in stage 0, even if nobody asked for any stage in particular.

Let's rewrite the logic a little bit, to handle the easy case early: if cache_name_pos() returned a non-negative position, we know it is a match, and we do not even have to compare the name again (cache_name_pos() did that for us already). We can say right away: yes, this file was tracked.

Only if there was no exact match do we need to look harder for any matching entry in stage 2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)
Show changes to merge-recursive.c +14 −16
diff --git a/merge-recursive.c b/merge-recursive.c
index ea0df22..469741d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -658,23 +658,21 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
-	if (pos < 0)
-		pos = -1 - pos;
-	while (pos < active_nr &&
-	       !strcmp(path, active_cache[pos]->name)) {
-		/*
-		 * If stage #0, it is definitely tracked.
-		 * If it has stage #2 then it was tracked
-		 * before this merge started.  All other
-		 * cases the path was not tracked.
-		 */
-		switch (ce_stage(active_cache[pos])) {
-		case 0:
-		case 2:
+	if (0 <= pos)
+		/* we have been tracking this path */
+		return 1;
+
+	/*
+	 * Look for an unmerged entry for the path,
+	 * specifically stage #2, which would indicate
+	 * that "our" side before the merge started
+	 * had the path tracked (and resulted in a conflict).
+	 */
+	for (pos = -1 - pos;
+	     pos < active_nr && !strcmp(path, active_cache[pos]->name);
+	     pos++)
+		if (ce_stage(active_cache[pos]) == 2)
 			return 1;
-		}
-		pos++;
-	}
 	return 0;
 }
 
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 01/16] Verify that `git pull --rebase` shows the helpful advice when failing

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t5520-pull.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
Show changes to t/t5520-pull.sh +30 −0
diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
index 5d4880e..217b416 100755
--- a/t/t5520-pull.sh
+++ b/t/t5520-pull.sh
@@ -255,6 +255,36 @@ test_expect_success '--rebase' '
 	test new = "$(git show HEAD:file2)"
 '
 
+test_expect_success '--rebase with conflicts shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b seq &&
+	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
+	git add seq.txt &&
+	test_tick &&
+	git commit -m "Add seq.txt" &&
+	printf "6\\n" >>seq.txt &&
+	test_tick &&
+	git commit -m "Append to seq.txt" seq.txt &&
+	git checkout -b with-conflicts HEAD^ &&
+	printf "conflicting\\n" >>seq.txt &&
+	test_tick &&
+	git commit -m "Create conflict" seq.txt &&
+	test_must_fail git pull --rebase . seq 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+test_expect_success 'failed --rebase shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b diverging &&
+	test_commit attributes .gitattributes "* text=auto" attrs &&
+	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
+	git update-index --cacheinfo 0644 $sha1 file &&
+	git commit -m v1-with-cr &&
+	git checkout -f -b fails-to-rebase HEAD^ &&
+	test_commit v2-without-cr file "2" file2-lf &&
+	test_must_fail git pull --rebase . diverging 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
 test_expect_success '--rebase fails with multiple branches' '
 	git reset --hard before-rebase &&
 	test_must_fail git pull --rebase . copy master 2>err &&
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 07/16] merge-recursive: avoid returning a wholesale struct

It is technically allowed, as per C89, for functions' return type to be complete structs (i.e. *not* just pointers to structs).

However, it was just an oversight of this developer when converting Python code to C code in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which introduced such a return type.

Besides, by converting this construct to pass in the struct, we can now start returning a value that can indicate errors in future patches. This will help the current effort to libify merge-recursive.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 91 +++++++++++++++++++++++++++++--------------------------
 1 file changed, 48 insertions(+), 43 deletions(-)
Show changes to merge-recursive.c +48 −43
diff --git a/merge-recursive.c b/merge-recursive.c
index 37c181a..d9221ce 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -910,47 +910,47 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file_1(struct merge_options *o,
+static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
 					   const struct diff_filespec *b,
 					   const char *branch1,
-					   const char *branch2)
+					   const char *branch2,
+					   struct merge_file_info *result)
 {
-	struct merge_file_info result;
-	result.merge = 0;
-	result.clean = 1;
+	result->merge = 0;
+	result->clean = 1;
 
 	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
-		result.clean = 0;
+		result->clean = 0;
 		if (S_ISREG(a->mode)) {
-			result.mode = a->mode;
-			oidcpy(&result.oid, &a->oid);
+			result->mode = a->mode;
+			oidcpy(&result->oid, &a->oid);
 		} else {
-			result.mode = b->mode;
-			oidcpy(&result.oid, &b->oid);
+			result->mode = b->mode;
+			oidcpy(&result->oid, &b->oid);
 		}
 	} else {
 		if (!oid_eq(&a->oid, &one->oid) && !oid_eq(&b->oid, &one->oid))
-			result.merge = 1;
+			result->merge = 1;
 
 		/*
 		 * Merge modes
 		 */
 		if (a->mode == b->mode || a->mode == one->mode)
-			result.mode = b->mode;
+			result->mode = b->mode;
 		else {
-			result.mode = a->mode;
+			result->mode = a->mode;
 			if (b->mode != one->mode) {
-				result.clean = 0;
-				result.merge = 1;
+				result->clean = 0;
+				result->merge = 1;
 			}
 		}
 
 		if (oid_eq(&a->oid, &b->oid) || oid_eq(&a->oid, &one->oid))
-			oidcpy(&result.oid, &b->oid);
+			oidcpy(&result->oid, &b->oid);
 		else if (oid_eq(&b->oid, &one->oid))
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
 			int merge_status;
@@ -962,64 +962,65 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 				die(_("Failed to execute internal merge"));
 
 			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result.oid.hash))
+					    blob_type, result->oid.hash))
 				die(_("Unable to add %s to database"),
 				    a->path);
 
 			free(result_buf.ptr);
-			result.clean = (merge_status == 0);
+			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result.clean = merge_submodule(result.oid.hash,
+			result->clean = merge_submodule(result->oid.hash,
 						       one->path,
 						       one->oid.hash,
 						       a->oid.hash,
 						       b->oid.hash,
 						       !o->call_depth);
 		} else if (S_ISLNK(a->mode)) {
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 
 			if (!oid_eq(&a->oid, &b->oid))
-				result.clean = 0;
+				result->clean = 0;
 		} else
 			die("BUG: unsupported object type in the tree");
 	}
 
-	return result;
+	return 0;
 }
 
-static struct merge_file_info
-merge_file_special_markers(struct merge_options *o,
+static int merge_file_special_markers(struct merge_options *o,
 			   const struct diff_filespec *one,
 			   const struct diff_filespec *a,
 			   const struct diff_filespec *b,
 			   const char *branch1,
 			   const char *filename1,
 			   const char *branch2,
-			   const char *filename2)
+			   const char *filename2,
+			   struct merge_file_info *mfi)
 {
 	char *side1 = NULL;
 	char *side2 = NULL;
-	struct merge_file_info mfi;
+	int ret;
 
 	if (filename1)
 		side1 = xstrfmt("%s:%s", branch1, filename1);
 	if (filename2)
 		side2 = xstrfmt("%s:%s", branch2, filename2);
 
-	mfi = merge_file_1(o, one, a, b,
-			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	ret = merge_file_1(o, one, a, b,
+		side1 ? side1 : branch1, side2 ? side2 : branch2, mfi);
 	free(side1);
 	free(side2);
-	return mfi;
+	return ret;
 }
 
-static struct merge_file_info merge_file_one(struct merge_options *o,
+static int merge_file_one(struct merge_options *o,
 					 const char *path,
 					 const struct object_id *o_oid, int o_mode,
 					 const struct object_id *a_oid, int a_mode,
 					 const struct object_id *b_oid, int b_mode,
 					 const char *branch1,
-					 const char *branch2)
+					 const char *branch2,
+					 struct merge_file_info *mfi)
 {
 	struct diff_filespec one, a, b;
 
@@ -1030,7 +1031,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
 	a.mode = a_mode;
 	oidcpy(&b.oid, b_oid);
 	b.mode = b_mode;
-	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
 static void handle_change_delete(struct merge_options *o,
@@ -1203,11 +1204,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct merge_file_info mfi;
 		struct diff_filespec other;
 		struct diff_filespec *add;
-		mfi = merge_file_one(o, one->path,
+		if (merge_file_one(o, one->path,
 				 &one->oid, one->mode,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
-				 ci->branch1, ci->branch2);
+				 ci->branch1, ci->branch2, &mfi))
+			return;
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1261,12 +1263,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
+	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
 					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path);
-	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
+					    o->branch2, ci->ren1_other.path, &mfi_c1) ||
+	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path);
+					    o->branch2, c2->path, &mfi_c2))
+		return;
 
 	if (o->call_depth) {
 		/*
@@ -1489,12 +1492,13 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					mfi = merge_file_one(o, ren1_dst, &null_oid, 0,
+					if (merge_file_one(o, ren1_dst, &null_oid, 0,
 							 &ren1->pair->two->oid,
 							 ren1->pair->two->mode,
 							 &dst_other.oid,
 							 dst_other.mode,
-							 branch1, branch2);
+							 branch1, branch2, &mfi))
+						return -1;
 					output(o, 1, _("Adding merged %s"), ren1_dst);
 					update_file(o, 0, &mfi.oid,
 						    mfi.mode, ren1_dst);
@@ -1652,9 +1656,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_special_markers(o, &one, &a, &b,
+	if (merge_file_special_markers(o, &one, &a, &b,
 					 o->branch1, path1,
-					 o->branch2, path2);
+					 o->branch2, path2, &mfi))
+		return -1;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    oid_eq(&mfi.oid, a_oid) && mfi.mode == a_mode) {
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 08/16] merge-recursive: allow write_tree_from_memory() to error out

It is possible that a tree cannot be written (think: disk full). We will want to give the caller a chance to clean up instead of letting the program die() in such a case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +2 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index d9221ce..09f4e27 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1903,8 +1903,8 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	if (o->call_depth)
-		*result = write_tree_from_memory(o);
+	if (o->call_depth && !(*result = write_tree_from_memory(o)))
+		return -1;
 
 	return clean;
 }
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 09/16] merge-recursive: handle return values indicating errors

We are about to libify the recursive merge machinery, where we only die() in case of a bug or memory contention. To that end, we must heed negative return values as indicating errors.

This requires our functions to be careful to pass through error conditions in call chains, and for quite a few functions this means that they have to return values to begin with.

The next step will be to convert the places where we currently die() to return negative values (read: -1) instead.

Note that we ignore errors reported by make_room_for_path(), consistent with the previous behavior (update_file_flags() used the return value of make_room_for_path() only to indicate an early return, but not a fatal error): if the error is really a fatal error, we will notice later; If not, it was not that serious a problem to begin with. (Witnesses in favor of this reasoning are t4151-am-abort and t7610-mergetool, which would start failing if we stopped on errors reported by make_room_for_path()).

Note: while this patch makes the code slightly less readable in
update_file_flags() (we introduce a new "goto free_buf;" instead of
an explicit "free(buf); return;"), it is a preparatory change for
the next patch where we will convert all of the die() calls in the same
function to go through the free_buf return path instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 199 +++++++++++++++++++++++++++++++++---------------------
 1 file changed, 123 insertions(+), 76 deletions(-)
Show changes to merge-recursive.c +123 −76
diff --git a/merge-recursive.c b/merge-recursive.c
index 09f4e27..89eb937 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -733,7 +733,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	return error(msg, path, _(": perhaps a D/F conflict?"));
 }
 
-static void update_file_flags(struct merge_options *o,
+static int update_file_flags(struct merge_options *o,
 			      const struct object_id *oid,
 			      unsigned mode,
 			      const char *path,
@@ -766,8 +766,7 @@ static void update_file_flags(struct merge_options *o,
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
-			free(buf);
-			goto update_index;
+			goto free_buf;
 		}
 		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
 			int fd;
@@ -823,20 +822,22 @@ static void update_file_flags(struct merge_options *o,
 		} else
 			die(_("do not know what to do with %06o %s '%s'"),
 			    mode, oid_to_hex(oid), path);
+ free_buf:
 		free(buf);
 	}
  update_index:
 	if (update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+	return 0;
 }
 
-static void update_file(struct merge_options *o,
+static int update_file(struct merge_options *o,
 			int clean,
 			const struct object_id *oid,
 			unsigned mode,
 			const char *path)
 {
-	update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
+	return update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1034,7 +1035,7 @@ static int merge_file_one(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
-static void handle_change_delete(struct merge_options *o,
+static int handle_change_delete(struct merge_options *o,
 				 const char *path,
 				 const struct object_id *o_oid, int o_mode,
 				 const struct object_id *a_oid, int a_mode,
@@ -1042,6 +1043,7 @@ static void handle_change_delete(struct merge_options *o,
 				 const char *change, const char *change_past)
 {
 	char *renamed = NULL;
+	int ret = 0;
 	if (dir_in_way(path, !o->call_depth)) {
 		renamed = unique_path(o, path, a_oid ? o->branch1 : o->branch2);
 	}
@@ -1052,21 +1054,22 @@ static void handle_change_delete(struct merge_options *o,
 		 * correct; since there is no true "middle point" between
 		 * them, simply reuse the base version for virtual merge base.
 		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_oid, o_mode, renamed ? renamed : path);
+		ret = remove_file_from_cache(path);
+		if (!ret)
+			ret = update_file(o, 0, o_oid, o_mode, renamed ? renamed : path);
 	} else if (!a_oid) {
 		if (!renamed) {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path);
-			update_file(o, 0, b_oid, b_mode, path);
+			ret = update_file(o, 0, b_oid, b_mode, path);
 		} else {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path, renamed);
-			update_file(o, 0, b_oid, b_mode, renamed);
+			ret = update_file(o, 0, b_oid, b_mode, renamed);
 		}
 	} else {
 		if (!renamed) {
@@ -1079,7 +1082,7 @@ static void handle_change_delete(struct merge_options *o,
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch2, change_past,
 			       o->branch1, o->branch1, path, renamed);
-			update_file(o, 0, a_oid, a_mode, renamed);
+			ret = update_file(o, 0, a_oid, a_mode, renamed);
 		}
 		/*
 		 * No need to call update_file() on path when !renamed, since
@@ -1089,9 +1092,11 @@ static void handle_change_delete(struct merge_options *o,
 		 */
 	}
 	free(renamed);
+
+	return ret;
 }
 
-static void conflict_rename_delete(struct merge_options *o,
+static int conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
 				   const char *other_branch)
@@ -1111,21 +1116,20 @@ static void conflict_rename_delete(struct merge_options *o,
 		b_mode = dest->mode;
 	}
 
-	handle_change_delete(o,
+	if (handle_change_delete(o,
 			     o->call_depth ? orig->path : dest->path,
 			     &orig->oid, orig->mode,
 			     a_oid, a_mode,
 			     b_oid, b_mode,
-			     _("rename"), _("renamed"));
+			     _("rename"), _("renamed")))
+		return -1;
 
-	if (o->call_depth) {
-		remove_file_from_cache(dest->path);
-	} else {
-		update_stages(dest->path, NULL,
+	if (o->call_depth)
+		return remove_file_from_cache(dest->path);
+	else
+		return update_stages(dest->path, NULL,
 			      rename_branch == o->branch1 ? dest : NULL,
 			      rename_branch == o->branch1 ? NULL : dest);
-	}
-
 }
 
 static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
@@ -1141,7 +1145,7 @@ static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
-static void handle_file(struct merge_options *o,
+static int handle_file(struct merge_options *o,
 			struct diff_filespec *rename,
 			int stage,
 			struct rename_conflict_info *ci)
@@ -1151,6 +1155,7 @@ static void handle_file(struct merge_options *o,
 	const char *cur_branch, *other_branch;
 	struct diff_filespec other;
 	struct diff_filespec *add;
+	int ret;
 
 	if (stage == 2) {
 		dst_entry = ci->dst_entry1;
@@ -1165,7 +1170,8 @@ static void handle_file(struct merge_options *o,
 	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
 	if (add) {
 		char *add_name = unique_path(o, rename->path, other_branch);
-		update_file(o, 0, &add->oid, add->mode, add_name);
+		if (update_file(o, 0, &add->oid, add->mode, add_name))
+			return -1;
 
 		remove_file(o, 0, rename->path, 0);
 		dst_name = unique_path(o, rename->path, cur_branch);
@@ -1176,17 +1182,20 @@ static void handle_file(struct merge_options *o,
 			       rename->path, other_branch, dst_name);
 		}
 	}
-	update_file(o, 0, &rename->oid, rename->mode, dst_name);
-	if (stage == 2)
-		update_stages(rename->path, NULL, rename, add);
+	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
+		; /* fall through, do allow dst_name to be released */
+	else if (stage == 2)
+		ret = update_stages(rename->path, NULL, rename, add);
 	else
-		update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
+
+	return ret;
 }
 
-static void conflict_rename_rename_1to2(struct merge_options *o,
+static int conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
@@ -1209,14 +1218,16 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
 				 ci->branch1, ci->branch2, &mfi))
-			return;
+			return -1;
+
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, &mfi.oid, mfi.mode, one->path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, one->path))
+			return -1;
 
 		/*
 		 * Above, we put the merged content at the merge-base's
@@ -1227,22 +1238,26 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * resolving the conflict at that path in its favor.
 		 */
 		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, a->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, a->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(a->path);
 		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, b->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, b->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(b->path);
-	} else {
-		handle_file(o, a, 2, ci);
-		handle_file(o, b, 3, ci);
-	}
+	} else if (handle_file(o, a, 2, ci) || handle_file(o, b, 3, ci))
+		return -1;
+
+	return 0;
 }
 
-static void conflict_rename_rename_2to1(struct merge_options *o,
+static int conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* Two files, a & b, were renamed to the same thing, c. */
@@ -1253,6 +1268,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	char *path = c1->path; /* == c2->path */
 	struct merge_file_info mfi_c1;
 	struct merge_file_info mfi_c2;
+	int ret;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1269,7 +1285,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 					    o->branch1, ci->ren2_other.path,
 					    o->branch2, c2->path, &mfi_c2))
-		return;
+		return -1;
 
 	if (o->call_depth) {
 		/*
@@ -1280,19 +1296,25 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 		 * again later for the non-recursive merge.
 		 */
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, b->path);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+				b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, _("Renaming %s to %s and %s to %s instead"),
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, new_path2);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+				new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
+
+	return ret;
 }
 
 static int process_renames(struct merge_options *o,
@@ -1477,12 +1499,13 @@ static int process_renames(struct merge_options *o,
 				 * update_file_flags() instead of
 				 * update_file().
 				 */
-				update_file_flags(o,
+				if (update_file_flags(o,
 						  &ren1->pair->two->oid,
 						  ren1->pair->two->mode,
 						  ren1_dst,
 						  1, /* update_cache */
-						  0  /* update_wd    */);
+						  0  /* update_wd    */))
+					clean_merge = -1;
 			} else if (!oid_eq(&dst_other.oid, &null_oid)) {
 				clean_merge = 0;
 				try_merge = 1;
@@ -1497,22 +1520,28 @@ static int process_renames(struct merge_options *o,
 							 ren1->pair->two->mode,
 							 &dst_other.oid,
 							 dst_other.mode,
-							 branch1, branch2, &mfi))
-						return -1;
+							 branch1, branch2, &mfi)) {
+						clean_merge = -1;
+						goto cleanup_and_return;
+					}
 					output(o, 1, _("Adding merged %s"), ren1_dst);
-					update_file(o, 0, &mfi.oid,
-						    mfi.mode, ren1_dst);
+					if (update_file(o, 0, &mfi.oid,
+							mfi.mode, ren1_dst))
+						clean_merge = -1;
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, _("Adding as %s instead"), new_path);
-					update_file(o, 0, &dst_other.oid,
-						    dst_other.mode, new_path);
+					if (update_file(o, 0, &dst_other.oid,
+						    dst_other.mode, new_path))
+						clean_merge = -1;
 					free(new_path);
 				}
 			} else
 				try_merge = 1;
 
+			if (clean_merge < 0)
+				goto cleanup_and_return;
 			if (try_merge) {
 				struct diff_filespec *one, *a, *b;
 				src_other.path = (char *)ren1_src;
@@ -1539,6 +1568,7 @@ static int process_renames(struct merge_options *o,
 			}
 		}
 	}
+cleanup_and_return:
 	string_list_clear(&a_by_dst, 0);
 	string_list_clear(&b_by_dst, 0);
 
@@ -1601,13 +1631,13 @@ error_return:
 	return ret;
 }
 
-static void handle_modify_delete(struct merge_options *o,
+static int handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 struct object_id *o_oid, int o_mode,
 				 struct object_id *a_oid, int a_mode,
 				 struct object_id *b_oid, int b_mode)
 {
-	handle_change_delete(o,
+	return handle_change_delete(o,
 			     path,
 			     o_oid, o_mode,
 			     a_oid, a_mode,
@@ -1686,7 +1716,8 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			update_stages(path, &one, &a, &b);
+			if (update_stages(path, &one, &a, &b))
+				return -1;
 	}
 
 	if (df_conflict_remains) {
@@ -1694,30 +1725,33 @@ static int merge_content(struct merge_options *o,
 		if (o->call_depth) {
 			remove_file_from_cache(path);
 		} else {
-			if (!mfi.clean)
-				update_stages(path, &one, &a, &b);
-			else {
+			if (!mfi.clean) {
+				if (update_stages(path, &one, &a, &b))
+					return -1;
+			} else {
 				int file_from_stage2 = was_tracked(path);
 				struct diff_filespec merged;
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				update_stages(path, NULL,
+				if (update_stages(path, NULL,
 					      file_from_stage2 ? &merged : NULL,
-					      file_from_stage2 ? NULL : &merged);
+					      file_from_stage2 ? NULL : &merged))
+					return -1;
 			}
 
 		}
 		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, _("Adding as %s instead"), new_path);
-		update_file(o, 0, &mfi.oid, mfi.mode, new_path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, new_path)) {
+			free(new_path);
+			return -1;
+		}
 		free(new_path);
 		mfi.clean = 0;
-	} else {
-		update_file(o, mfi.clean, &mfi.oid, mfi.mode, path);
-	}
+	} else if (update_file(o, mfi.clean, &mfi.oid, mfi.mode, path))
+		return -1;
 	return mfi.clean;
-
 }
 
 /* Per entry merge function */
@@ -1745,17 +1779,21 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
-			conflict_rename_delete(o, conflict_info->pair1,
+			if (conflict_rename_delete(o,
+					       conflict_info->pair1,
 					       conflict_info->branch1,
-					       conflict_info->branch2);
+					       conflict_info->branch2))
+				clean_merge = -1;
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info);
+			if (conflict_rename_rename_1to2(o, conflict_info))
+				clean_merge = -1;
 			break;
 		case RENAME_TWO_FILES_TO_ONE:
 			clean_merge = 0;
-			conflict_rename_rename_2to1(o, conflict_info);
+			if (conflict_rename_rename_2to1(o, conflict_info))
+				clean_merge = -1;
 			break;
 		default:
 			entry->processed = 0;
@@ -1775,8 +1813,9 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_modify_delete(o, path, o_oid, o_mode,
-					     a_oid, a_mode, b_oid, b_mode);
+			if (handle_modify_delete(o, path, o_oid, o_mode,
+					     a_oid, a_mode, b_oid, b_mode))
+				clean_merge = -1;
 		}
 	} else if ((!o_oid && a_oid && !b_oid) ||
 		   (!o_oid && !a_oid && b_oid)) {
@@ -1808,14 +1847,16 @@ static int process_entry(struct merge_options *o,
 			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s"),
 			       conf, path, other_branch, path, new_path);
-			update_file(o, 0, oid, mode, new_path);
-			if (o->call_depth)
+			if (update_file(o, 0, oid, mode, new_path))
+				clean_merge = -1;
+			else if (o->call_depth)
 				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, _("Adding %s"), path);
 			/* do not overwrite file if already present */
-			update_file_flags(o, oid, mode, path, 1, !a_oid);
+			if (update_file_flags(o, oid, mode, path, 1, !a_oid))
+				clean_merge = -1;
 		}
 	} else if (a_oid && b_oid) {
 		/* Case C: Added in both (check for same permissions) and */
@@ -1878,12 +1919,18 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
+		if (clean < 0)
+			return clean;
 		for (i = entries->nr-1; 0 <= i; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_entry(o, path, e))
-				clean = 0;
+			if (!e->processed) {
+				int ret = process_entry(o, path, e);
+				if (!ret)
+					clean = 0;
+				else if (ret < 0)
+					return ret;
+			}
 		}
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 11/16] am -3: use merge_recursive() directly again

Last October, we had to change this code to run `git merge-recursive` in a child process: git-am wants to print some helpful advice when the merge failed, but the code in question was not prepared to return, it die()d instead.

We are finally at a point when the code *is* prepared to return errors, and can avoid the child process again.

This reverts commit c63d4b2 (am -3: do not let failed merge from completing the error codepath, 2015-10-09), with the necessary changes to adjust for the fact that Git's source code changed in the meantime (such as: using OIDs instead of hashes in the recursive merge). While at it, the same undesired gender bias is addressed in the same spirit as in the separately submitted patch in $gmane/299009.

Note: the code now calls merge_recursive_generic() again. Unlike
merge_trees() and merge_recursive(), this function returns 0 upon success,
as most of Git's functions. Therefore, the error value -1 naturally is
handled correctly, and we do not have to take care of it specifically.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c | 63 ++++++++++++++++++++++--------------------------------------
 1 file changed, 23 insertions(+), 40 deletions(-)
Show changes to builtin/am.c +23 −40
diff --git a/builtin/am.c b/builtin/am.c
index f9a724e..8dc4239 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -29,6 +29,7 @@
 #include "prompt.h"
 #include "mailinfo.h"
 #include "apply.h"
+#include "object.h"
 
 /**
  * Returns the length of the first line of msg.
@@ -1623,47 +1624,18 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
 }
 
 /**
- * Do the three-way merge using fake ancestor, his tree constructed
- * from the fake ancestor and the postimage of the patch, and our
- * state.
- */
-static int run_fallback_merge_recursive(const struct am_state *state,
-					unsigned char *orig_tree,
-					unsigned char *our_tree,
-					unsigned char *his_tree)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-
-	cp.git_cmd = 1;
-
-	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(his_tree), linelen(state->msg), state->msg);
-	if (state->quiet)
-		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
-
-	argv_array_push(&cp.args, "merge-recursive");
-	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
-	argv_array_push(&cp.args, "--");
-	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(his_tree));
-
-	status = run_command(&cp) ? (-1) : 0;
-	discard_cache();
-	read_cache();
-	return status;
-}
-
-/**
  * Attempt a threeway merge, using index_path as the temporary index.
  */
 static int fall_back_threeway(const struct am_state *state, const char *index_path)
 {
-	unsigned char orig_tree[GIT_SHA1_RAWSZ], his_tree[GIT_SHA1_RAWSZ],
-		      our_tree[GIT_SHA1_RAWSZ];
+	struct object_id orig_tree, her_tree, our_tree;
+	const struct object_id *bases[1] = { &orig_tree };
+	struct merge_options o;
+	struct commit *result;
+	char *her_tree_name;
 
-	if (get_sha1("HEAD", our_tree) < 0)
-		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
+	if (get_oid("HEAD", &our_tree) < 0)
+		hashcpy(our_tree.hash, EMPTY_TREE_SHA1_BIN);
 
 	if (build_fake_ancestor(state, index_path))
 		return error("could not build fake ancestor");
@@ -1671,7 +1643,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	discard_cache();
 	read_cache_from(index_path);
 
-	if (write_index_as_tree(orig_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(orig_tree.hash, &the_index, index_path, 0, NULL))
 		return error(_("Repository lacks necessary blobs to fall back on 3-way merge."));
 
 	say(state, stdout, _("Using index info to reconstruct a base tree..."));
@@ -1687,7 +1659,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		init_revisions(&rev_info, NULL);
 		rev_info.diffopt.output_format = DIFF_FORMAT_NAME_STATUS;
 		diff_opt_parse(&rev_info.diffopt, &diff_filter_str, 1, rev_info.prefix);
-		add_pending_sha1(&rev_info, "HEAD", our_tree, 0);
+		add_pending_sha1(&rev_info, "HEAD", our_tree.hash, 0);
 		diff_setup_done(&rev_info.diffopt);
 		run_diff_index(&rev_info, 1);
 	}
@@ -1696,7 +1668,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		return error(_("Did you hand edit your patch?\n"
 				"It does not apply to blobs recorded in its index."));
 
-	if (write_index_as_tree(his_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(her_tree.hash, &the_index, index_path, 0, NULL))
 		return error("could not write tree");
 
 	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
@@ -1712,11 +1684,22 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	 * changes.
 	 */
 
-	if (run_fallback_merge_recursive(state, orig_tree, our_tree, his_tree)) {
+	init_merge_options(&o);
+
+	o.branch1 = "HEAD";
+	her_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
+	o.branch2 = her_tree_name;
+
+	if (state->quiet)
+		o.verbosity = 0;
+
+	if (merge_recursive_generic(&o, &our_tree, &her_tree, 1, bases, &result)) {
 		rerere(state->allow_rerere_autoupdate);
+		free(her_tree_name);
 		return error(_("Failed to merge in the changes."));
 	}
 
+	free(her_tree_name);
 	return 0;
 }
 
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 10/16] merge-recursive: switch to returning errors instead of dying

The recursive merge machinery is supposed to be a library function, i.e. it should return an error when it fails. Originally the functions were part of the builtin "merge-recursive", though, where it was simpler to call die() and be done with error handling.

The existing callers were already prepared to detect negative return values to indicate errors and to behave as previously: exit with code 128 (which is the same thing that die() does, after printing the message).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 63 +++++++++++++++++++++++++++++++------------------------
 1 file changed, 36 insertions(+), 27 deletions(-)
Show changes to merge-recursive.c +36 −27
diff --git a/merge-recursive.c b/merge-recursive.c
index 89eb937..7c9f22c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -266,8 +266,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		active_cache_tree = cache_tree();
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
-	    cache_tree_update(&the_index, 0) < 0)
-		die(_("error building trees"));
+	    cache_tree_update(&the_index, 0) < 0) {
+		error(_("error building trees"));
+		return NULL;
+	}
 
 	result = lookup_tree(active_cache_tree->sha1);
 
@@ -707,12 +709,10 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
-		if (status == SCLD_EXISTS) {
+		if (status == SCLD_EXISTS)
 			/* something else exists */
-			error(msg, path, _(": perhaps a D/F conflict?"));
-			return -1;
-		}
-		die(msg, path, "");
+			return error(msg, path, _(": perhaps a D/F conflict?"));
+		return error(msg, path, "");
 	}
 
 	/*
@@ -740,6 +740,8 @@ static int update_file_flags(struct merge_options *o,
 			      int update_cache,
 			      int update_wd)
 {
+	int ret = 0;
+
 	if (o->call_depth)
 		update_wd = 0;
 
@@ -760,9 +762,11 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			die(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
-		if (type != OBJ_BLOB)
-			die(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+		if (type != OBJ_BLOB) {
+			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			goto free_buf;
+		}
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
@@ -778,8 +782,10 @@ static int update_file_flags(struct merge_options *o,
 			else
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-			if (fd < 0)
-				die_errno(_("failed to open '%s'"), path);
+			if (fd < 0) {
+				ret = error_errno(_("failed to open '%s'"), path);
+				goto free_buf;
+			}
 
 			smudge_to_file = can_smudge_to_file(path);
 
@@ -792,8 +798,10 @@ static int update_file_flags(struct merge_options *o,
 					 * creation. */
 					smudge_to_file = 0;
 					fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-					if (fd < 0)
-						die_errno(_("failed to open '%s'"), path);
+					if (fd < 0) {
+						ret = error_errno(_("failed to open '%s'"), path);
+						goto free_buf;
+					}
 				}
 				else {
 					close(fd);
@@ -817,18 +825,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				die_errno(_("failed to symlink '%s'"), path);
+				ret = error_errno(_("failed to symlink '%s'"), path);
 			free(lnk);
 		} else
-			die(_("do not know what to do with %06o %s '%s'"),
+			ret = error(_("do not know what to do with %06o %s '%s'"),
 			    mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
-	if (update_cache)
+	if (!ret && update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
-	return 0;
+	return ret;
 }
 
 static int update_file(struct merge_options *o,
@@ -954,20 +962,22 @@ static int merge_file_1(struct merge_options *o,
 			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
-			int merge_status;
+			int ret = 0, merge_status;
 
 			merge_status = merge_3way(o, &result_buf, one, a, b,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				die(_("Failed to execute internal merge"));
+				ret = error(_("Failed to execute internal merge"));
 
-			if (write_sha1_file(result_buf.ptr, result_buf.size,
+			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 					    blob_type, result->oid.hash))
-				die(_("Unable to add %s to database"),
-				    a->path);
+				ret = error(_("Unable to add %s to database"),
+					a->path);
 
 			free(result_buf.ptr);
+			if (ret)
+				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
 			result->clean = merge_submodule(result->oid.hash,
@@ -1899,11 +1909,10 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			die(_("merging of trees %s and %s failed"),
+			error(_("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
-		else
-			exit(128);
+		return -1;
 	}
 
 	if (unmerged_cache()) {
@@ -2034,7 +2043,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			die(_("merge returned no commit"));
+			return error(_("merge returned no commit"));
 	}
 
 	discard_cache();
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 12/16] merge-recursive: flush output buffer before printing error messages

The data structure passed to the recursive merge machinery has a feature where the caller can ask for the output to be buffered into a strbuf, by setting the field 'buffer_output'.

Previously, we simply swallowed the buffered output when showing error messages. With this patch, we show the output first, and only then print the error message.

Currently, the only user of that buffering is merge_recursive() itself, to avoid the progress output to interfere.

In the next patches, we will introduce a new buffer_output mode that forces merge_recursive() to retain the output buffer for further processing by the caller. If the caller asked for that, we will then also write the error messages into the output buffer. This is necessary to give the caller more control not only how to react in case of errors but also control how/if to display the error messages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 112 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 66 insertions(+), 46 deletions(-)
Show changes to merge-recursive.c +66 −46
diff --git a/merge-recursive.c b/merge-recursive.c
index 7c9f22c..205ea04 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,28 @@
 #include "dir.h"
 #include "submodule.h"
 
+static void flush_output(struct merge_options *o)
+{
+	if (o->obuf.len) {
+		fputs(o->obuf.buf, stdout);
+		strbuf_reset(&o->obuf);
+	}
+}
+
+static int err(struct merge_options *o, const char *err, ...)
+{
+	va_list params;
+
+	va_start(params, err);
+	flush_output(o);
+	strbuf_vaddf(&o->obuf, err, params);
+	error("%s", o->obuf.buf);
+	strbuf_reset(&o->obuf);
+	va_end(params);
+
+	return -1;
+}
+
 static struct tree *shift_tree_object(struct tree *one, struct tree *two,
 				      const char *subtree_shift)
 {
@@ -148,14 +170,6 @@ static int show(struct merge_options *o, int v)
 	return (!o->call_depth && o->verbosity >= v) || o->verbosity >= 5;
 }
 
-static void flush_output(struct merge_options *o)
-{
-	if (o->obuf.len) {
-		fputs(o->obuf.buf, stdout);
-		strbuf_reset(&o->obuf);
-	}
-}
-
 __attribute__((format (printf, 3, 4)))
 static void output(struct merge_options *o, int v, const char *fmt, ...)
 {
@@ -198,7 +212,8 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)
 	}
 }
 
-static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
+static int add_cacheinfo(struct merge_options *o,
+		unsigned int mode, const struct object_id *oid,
 		const char *path, int stage, int refresh, int options)
 {
 	struct cache_entry *ce;
@@ -206,7 +221,7 @@ static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
 			      (refresh ? (CE_MATCH_REFRESH |
 					  CE_MATCH_IGNORE_MISSING) : 0 ));
 	if (!ce)
-		return error(_("addinfo_cache failed for path '%s'"), path);
+		return err(o, _("addinfo_cache failed for path '%s'"), path);
 	return add_cache_entry(ce, options);
 }
 
@@ -267,7 +282,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
 	    cache_tree_update(&the_index, 0) < 0) {
-		error(_("error building trees"));
+		err(o, _("error building trees"));
 		return NULL;
 	}
 
@@ -535,7 +550,8 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages(const char *path, const struct diff_filespec *o,
+static int update_stages(struct merge_options *opt, const char *path,
+			 const struct diff_filespec *o,
 			 const struct diff_filespec *a,
 			 const struct diff_filespec *b)
 {
@@ -554,13 +570,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 		if (remove_file_from_cache(path))
 			return -1;
 	if (o)
-		if (add_cacheinfo(o->mode, &o->oid, path, 1, 0, options))
+		if (add_cacheinfo(opt, o->mode, &o->oid, path, 1, 0, options))
 			return -1;
 	if (a)
-		if (add_cacheinfo(a->mode, &a->oid, path, 2, 0, options))
+		if (add_cacheinfo(opt, a->mode, &a->oid, path, 2, 0, options))
 			return -1;
 	if (b)
-		if (add_cacheinfo(b->mode, &b->oid, path, 3, 0, options))
+		if (add_cacheinfo(opt, b->mode, &b->oid, path, 3, 0, options))
 			return -1;
 	return 0;
 }
@@ -711,8 +727,8 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (status) {
 		if (status == SCLD_EXISTS)
 			/* something else exists */
-			return error(msg, path, _(": perhaps a D/F conflict?"));
-		return error(msg, path, "");
+			return err(o, msg, path, _(": perhaps a D/F conflict?"));
+		return err(o, msg, path, "");
 	}
 
 	/*
@@ -720,7 +736,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	 * tracking it.
 	 */
 	if (would_lose_untracked(path))
-		return error(_("refusing to lose untracked file at '%s'"),
+		return err(o, _("refusing to lose untracked file at '%s'"),
 			     path);
 
 	/* Successful unlink is good.. */
@@ -730,7 +746,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (errno == ENOENT)
 		return 0;
 	/* .. but not some other error (who really cares what?) */
-	return error(msg, path, _(": perhaps a D/F conflict?"));
+	return err(o, msg, path, _(": perhaps a D/F conflict?"));
 }
 
 static int update_file_flags(struct merge_options *o,
@@ -762,9 +778,9 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+			return err(o, _("cannot read object %s '%s'"), oid_to_hex(oid), path);
 		if (type != OBJ_BLOB) {
-			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			ret = err(o, _("blob expected for %s '%s'"), oid_to_hex(oid), path);
 			goto free_buf;
 		}
 
@@ -783,7 +799,8 @@ static int update_file_flags(struct merge_options *o,
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
 			if (fd < 0) {
-				ret = error_errno(_("failed to open '%s'"), path);
+				ret = err(o, _("failed to open '%s': %s"),
+					path, strerror(errno));
 				goto free_buf;
 			}
 
@@ -825,17 +842,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				ret = error_errno(_("failed to symlink '%s'"), path);
+				ret = err(o, _("failed to symlink '%s': %s"),
+					path, strerror(errno));
 			free(lnk);
 		} else
-			ret = error(_("do not know what to do with %06o %s '%s'"),
-			    mode, oid_to_hex(oid), path);
+			ret = err(o, _("do not know what to do with %06o %s '%s'"),
+				mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
 	if (!ret && update_cache)
-		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+		add_cacheinfo(o, mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
 	return ret;
 }
 
@@ -968,11 +986,11 @@ static int merge_file_1(struct merge_options *o,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				ret = error(_("Failed to execute internal merge"));
+				ret = err(o, _("Failed to execute internal merge"));
 
 			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 					    blob_type, result->oid.hash))
-				ret = error(_("Unable to add %s to database"),
+				ret = err(o, _("Unable to add %s to database"),
 					a->path);
 
 			free(result_buf.ptr);
@@ -1137,7 +1155,7 @@ static int conflict_rename_delete(struct merge_options *o,
 	if (o->call_depth)
 		return remove_file_from_cache(dest->path);
 	else
-		return update_stages(dest->path, NULL,
+		return update_stages(o, dest->path, NULL,
 			      rename_branch == o->branch1 ? dest : NULL,
 			      rename_branch == o->branch1 ? NULL : dest);
 }
@@ -1195,9 +1213,9 @@ static int handle_file(struct merge_options *o,
 	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
 		; /* fall through, do allow dst_name to be released */
 	else if (stage == 2)
-		ret = update_stages(rename->path, NULL, rename, add);
+		ret = update_stages(o, rename->path, NULL, rename, add);
 	else
-		ret = update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(o, rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
@@ -1590,23 +1608,25 @@ static struct object_id *stage_oid(const struct object_id *oid, unsigned mode)
 	return (is_null_oid(oid) || mode == 0) ? NULL: (struct object_id *)oid;
 }
 
-static int read_oid_strbuf(const struct object_id *oid, struct strbuf *dst)
+static int read_oid_strbuf(struct merge_options *o,
+	const struct object_id *oid, struct strbuf *dst)
 {
 	void *buf;
 	enum object_type type;
 	unsigned long size;
 	buf = read_sha1_file(oid->hash, &type, &size);
 	if (!buf)
-		return error(_("cannot read object %s"), oid_to_hex(oid));
+		return err(o, _("cannot read object %s"), oid_to_hex(oid));
 	if (type != OBJ_BLOB) {
 		free(buf);
-		return error(_("object %s is not a blob"), oid_to_hex(oid));
+		return err(o, _("object %s is not a blob"), oid_to_hex(oid));
 	}
 	strbuf_attach(dst, buf, size, size + 1);
 	return 0;
 }
 
-static int blob_unchanged(const struct object_id *o_oid,
+static int blob_unchanged(struct merge_options *opt,
+			  const struct object_id *o_oid,
 			  unsigned o_mode,
 			  const struct object_id *a_oid,
 			  unsigned a_mode,
@@ -1624,7 +1644,7 @@ static int blob_unchanged(const struct object_id *o_oid,
 		return 0;
 
 	assert(o_oid && a_oid);
-	if (read_oid_strbuf(o_oid, &o) || read_oid_strbuf(a_oid, &a))
+	if (read_oid_strbuf(opt, o_oid, &o) || read_oid_strbuf(opt, a_oid, &a))
 		goto error_return;
 	/*
 	 * Note: binary | is used so that both renormalizations are
@@ -1713,7 +1733,7 @@ static int merge_content(struct merge_options *o,
 		 */
 		path_renamed_outside_HEAD = !path2 || !strcmp(path, path2);
 		if (!path_renamed_outside_HEAD) {
-			add_cacheinfo(mfi.mode, &mfi.oid, path,
+			add_cacheinfo(o, mfi.mode, &mfi.oid, path,
 				      0, (!o->call_depth), 0);
 			return mfi.clean;
 		}
@@ -1726,7 +1746,7 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			if (update_stages(path, &one, &a, &b))
+			if (update_stages(o, path, &one, &a, &b))
 				return -1;
 	}
 
@@ -1736,7 +1756,7 @@ static int merge_content(struct merge_options *o,
 			remove_file_from_cache(path);
 		} else {
 			if (!mfi.clean) {
-				if (update_stages(path, &one, &a, &b))
+				if (update_stages(o, path, &one, &a, &b))
 					return -1;
 			} else {
 				int file_from_stage2 = was_tracked(path);
@@ -1744,7 +1764,7 @@ static int merge_content(struct merge_options *o,
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				if (update_stages(path, NULL,
+				if (update_stages(o, path, NULL,
 					      file_from_stage2 ? &merged : NULL,
 					      file_from_stage2 ? NULL : &merged))
 					return -1;
@@ -1812,8 +1832,8 @@ static int process_entry(struct merge_options *o,
 	} else if (o_oid && (!a_oid || !b_oid)) {
 		/* Case A: Deleted in one */
 		if ((!a_oid && !b_oid) ||
-		    (!b_oid && blob_unchanged(o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
-		    (!a_oid && blob_unchanged(o_oid, o_mode, b_oid, b_mode, normalize, path))) {
+		    (!b_oid && blob_unchanged(o, o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
+		    (!a_oid && blob_unchanged(o, o_oid, o_mode, b_oid, b_mode, normalize, path))) {
 			/* Deleted in both or deleted in one and
 			 * unchanged in the other */
 			if (a_oid)
@@ -1909,7 +1929,7 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			error(_("merging of trees %s and %s failed"),
+			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
 		return -1;
@@ -2043,7 +2063,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			return error(_("merge returned no commit"));
+			return err(o, _("merge returned no commit"));
 	}
 
 	discard_cache();
@@ -2102,7 +2122,7 @@ int merge_recursive_generic(struct merge_options *o,
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
 			if (!(base = get_ref(base_list[i], oid_to_hex(base_list[i]))))
-				return error(_("Could not parse object '%s'"),
+				return err(o, _("Could not parse object '%s'"),
 					oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
 		}
@@ -2116,7 +2136,7 @@ int merge_recursive_generic(struct merge_options *o,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
-		return error(_("Unable to write index."));
+		return err(o, _("Unable to write index."));
 
 	return clean ? 0 : 1;
 }
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we already accumulate the output in a buffer. The idea was to avoid interfering with the progress output that goes to stderr, which is unbuffered, when we write to stdout, which is buffered.

We extend that buffering to allow the caller to handle the output (possibly suppressing it). This will help us when extending the sequencer to do rebase -i's brunt work: it does not want the picks to print anything by default but instead determine itself whether to print the output or not.

Note that we also redirect the error messages into the output buffer
when the caller asked not to flush the output buffer, for two reasons:
1) to retain the correct output order, and 2) to allow the caller to
suppress *all* output.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++++++----
 merge-recursive.h |  2 +-
 2 files changed, 14 insertions(+), 5 deletions(-)
Show changes to 2 files +14 −5

merge-recursive.c, merge-recursive.h

diff --git a/merge-recursive.c b/merge-recursive.c
index ad5b961..10d3355 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -25,7 +25,7 @@
 
 static void flush_output(struct merge_options *o)
 {
-	if (o->obuf.len) {
+	if (o->buffer_output < 2 && o->obuf.len) {
 		fputs(o->obuf.buf, stdout);
 		strbuf_reset(&o->obuf);
 	}
@@ -36,10 +36,19 @@ static int err(struct merge_options *o, const char *err, ...)
 	va_list params;
 
 	va_start(params, err);
-	flush_output(o);
+	if (o->buffer_output < 2)
+		flush_output(o);
+	else {
+		strbuf_complete(&o->obuf, '\n');
+		strbuf_addstr(&o->obuf, "error: ");
+	}
 	strbuf_vaddf(&o->obuf, err, params);
-	error("%s", o->obuf.buf);
-	strbuf_reset(&o->obuf);
+	if (o->buffer_output > 1)
+		strbuf_addch(&o->obuf, '\n');
+	else {
+		error("%s", o->obuf.buf);
+		strbuf_reset(&o->obuf);
+	}
 	va_end(params);
 
 	return -1;
diff --git a/merge-recursive.h b/merge-recursive.h
index d415724..340704c 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -13,7 +13,7 @@ struct merge_options {
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
 	const char *subtree_shift;
-	unsigned buffer_output : 1;
+	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 13/16] merge-recursive: write the commit title in one go

In 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we changed the code such that it prints the output in one go, to avoid interfering with the progress output.

Let's make sure that the same holds true when outputting the commit title: previously, we used several printf() statements to stdout and speculated that stdout's buffer is large enough to hold the entire commit title.

Apart from making that speculation unnecessary, we change the code to add the message to the output buffer before flushing for another reason: the next commit will introduce a new level of output buffering, where the caller can request the output not to be flushed, but to be retained for further processing.

This latter feature will be needed when teaching the sequencer to do rebase -i's brunt work: it wants to control the output of the cherry-picks (i.e. recursive merges).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
Show changes to merge-recursive.c +9 −8
diff --git a/merge-recursive.c b/merge-recursive.c
index 205ea04..ad5b961 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -191,25 +191,26 @@ static void output(struct merge_options *o, int v, const char *fmt, ...)
 
 static void output_commit_title(struct merge_options *o, struct commit *commit)
 {
-	int i;
-	flush_output(o);
-	for (i = o->call_depth; i--;)
-		fputs("  ", stdout);
+	strbuf_addchars(&o->obuf, ' ', o->call_depth * 2);
 	if (commit->util)
-		printf("virtual %s\n", merge_remote_util(commit)->name);
+		strbuf_addf(&o->obuf, "virtual %s\n",
+			merge_remote_util(commit)->name);
 	else {
-		printf("%s ", find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV));
+		strbuf_addf(&o->obuf, "%s ",
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV));
 		if (parse_commit(commit) != 0)
-			printf(_("(bad commit)\n"));
+			strbuf_addf(&o->obuf, _("(bad commit)\n"));
 		else {
 			const char *title;
 			const char *msg = get_commit_buffer(commit, NULL);
 			int len = find_commit_subject(msg, &title);
 			if (len)
-				printf("%.*s\n", len, title);
+				strbuf_addf(&o->obuf, "%.*s\n", len, title);
 			unuse_commit_buffer(commit, msg);
 		}
 	}
+	flush_output(o);
 }
 
 static int add_cacheinfo(struct merge_options *o,
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 15/16] Ensure that the output buffer is released after calling merge_trees()

The recursive merge machinery accumulates its output in an output buffer, to be flushed at the end of merge_recursive(). At this point, we forgot to release the output buffer.

When calling merge_trees() (i.e. the non-recursive part of the recursive merge) directly, the output buffer is never flushed because the caller may be merge_recursive() which wants to flush the output itself.

For the same reason, merge_trees() cannot release the output buffer: it may still be needed.

Forgetting to release the output buffer did not matter much when running git-checkout, or git-merge-recursive, because we exited after the operation anyway. Ever since cherry-pick learned to pick a commit range, however, this memory leak had the potential of becoming a problem.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 1 +
 merge-recursive.c  | 2 ++
 sequencer.c        | 1 +
 3 files changed, 4 insertions(+)
Show changes to 3 files +4 −0

builtin/checkout.c, merge-recursive.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 07dea3b..8d852d4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -573,6 +573,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
+			strbuf_release(&o.obuf);
 			if (ret)
 				return ret;
 		}
diff --git a/merge-recursive.c b/merge-recursive.c
index 10d3355..c0be0c5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2092,6 +2092,8 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
 	flush_output(o);
+	if (o->buffer_output < 2)
+		strbuf_release(&o->obuf);
 	if (show(o, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       o->needed_rename_limit, 0);
diff --git a/sequencer.c b/sequencer.c
index 286a435..ec50519 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
 
-- 
2.9.0.278.g1caae67
Johannes Schindelin· Jul 7, 2016, 14:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 16/16] merge-recursive: flush output buffer even when erroring out

Ever since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we had a problem: When the merge failed in a fatal way, all regular output was swallowed because we called die() and did not get a chance to drain the output buffers.

To fix this, several modifications were necessary:
- we needed to stop die()ing, to give callers a chance to do something
  when an error occurred (in this case, flush the output buffers),
- we needed to delay printing the error message so that the caller can
  print the buffered output before that, and
- we needed to make sure that the output buffers are flushed even when
  the return value indicates an error.

The first two changes were introduced through earlier commits in this patch series, and this commit addresses the third one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to merge-recursive.c +1 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index c0be0c5..cb701ee 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2083,6 +2083,7 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	flush_output(o);
 	if (clean < 0)
 		return clean;
 
@@ -2091,7 +2092,6 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(o);
 	if (o->buffer_output < 2)
 		strbuf_release(&o->obuf);
 	if (show(o, 2))
-- 
2.9.0.278.g1caae67
Junio C Hamano· Jul 12, 2016, 21:27 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> This is the second iteration of the long-awaited re-roll of the attempt to
> avoid spawning merge-recursive from the builtin am and use merge_recursive()
> directly instead.
This is actually the third iteration.

I am trying to tease dependencies apart and apply this on a more reasonable base than a commit that happened to be at 'pu' on one day, but this would probably take some time, and I may give up merging it anywhere for today's integration cycle. We'll see.

Johannes Schindelin· Jul 14, 2016, 14:03 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Hi Junio,
On Tue, 12 Jul 2016, Junio C Hamano wrote:
Show 7 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This is the second iteration of the long-awaited re-roll of the attempt to
> > avoid spawning merge-recursive from the builtin am and use merge_recursive()
> > directly instead.
> 
> This is actually the third iteration.
It is.
> I am trying to tease dependencies apart and apply this on a more
> reasonable base than a commit that happened to be at 'pu' on one
> day, but this would probably take some time, and I may give up
> merging it anywhere for today's integration cycle.  We'll see.

The two topics that are in 'pu' and conflict with this series are 'jh/clean-smudge-annex' and 'bc/cocci'.

It also conflicted with 'va/i18n-even-more', but that one was merged to 'master'.

Now, I think it would be okay to wait for 'bc/cocci' to go to 'master' before integrating the 'am-3-merge-recursive-direct' branch, but I would want to avoid waiting for 'jh/clean-smudge-annex'.

Do you concur? If so, I will rebase onto 'master' as soon as 'bc/cocci' lands in there.

Ciao, Dscho

Junio C Hamano· Jul 14, 2016, 19:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 12 quoted lines
> The two topics that are in 'pu' and conflict with this series are
> 'jh/clean-smudge-annex' and 'bc/cocci'.
>
> It also conflicted with 'va/i18n-even-more', but that one was merged to
> 'master'.
>
> Now, I think it would be okay to wait for 'bc/cocci' to go to 'master'
> before integrating the 'am-3-merge-recursive-direct' branch, but I would
> want to avoid waiting for 'jh/clean-smudge-annex'.
>
> Do you concur? If so, I will rebase onto 'master' as soon as 'bc/cocci'
> lands in there.

I do not have a strong preference myself. As you are not proposing to eject jh/clean-smudge-annex from my tree, I'd have to resolve the conflicts when the second topic is merged after one topic, whichever happens to be more ready. IOW, such a rebase adds to my workload.

Like it or not, it is normal for different topics to want to touch the overlapping area or one topic to invalidate an assumption the other topic is based on, and working well together does not have to mean leaving the conflict resolution to the sole responsibility of the integrator. A clean way forward may be for everybody involved (and I see you forgot to add Joey to this thread) to agree that this topic is more ready than jh/clean-smudge-annex and you and Joey to work together to rebase jh/clean-smudge-annex on top of this topic (or possibly the other way around), making him wait for you.

Junio C Hamano· Jul 19, 2016, 00:17 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Junio C Hamano <gitster@pobox.com> writes:
Show 11 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
>> The two topics that are in 'pu' and conflict with this series are
>> 'jh/clean-smudge-annex' and 'bc/cocci'.
>>
>> It also conflicted with 'va/i18n-even-more', but that one was merged to
>> 'master'.
>>
>> Now, I think it would be okay to wait for 'bc/cocci' to go to 'master'
>> before integrating the 'am-3-merge-recursive-direct' branch, but I would
>> want to avoid waiting for 'jh/clean-smudge-annex'.

Nothing seems to be happening on jh/clean-smudge-annex front, so unless we hear otherwise from Joey in the coming couple of days, let's get js/am-3-merge-recursive-direct untangled from its dependencies and get it into a shape to hit 'next'. You can assume that I'll merge bc/cocci and js/am-call-theirs-theirs-in-fallback-3way to 'master' during that time, so an appropriate base to use would be the result of

    git checkout master^0
    git merge bc/cocci
    git merge js/am-call-theirs-theirs-in-fallback-3way
    git merge cc/apply-am

if you want a semi-solid base to rebuild am-3-merge-recursive-direct on. I am not 100% sure about the doneness of cc/apply-am yet, though but it could be that am-3-merge-recursive-direct does not have to depend on it at all. You'd know better than I do ;-)

After that, I'll see if the conflict resolution is manageable when remerging jh/clean-smudge-annex to 'pu', and if it is not, I'll worry about it at that point.

Thanks.
Johannes Schindelin· Jul 19, 2016, 12:31 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Hi Junio,
On Mon, 18 Jul 2016, Junio C Hamano wrote:
Show 18 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> >> The two topics that are in 'pu' and conflict with this series are
> >> 'jh/clean-smudge-annex' and 'bc/cocci'.
> >>
> >> It also conflicted with 'va/i18n-even-more', but that one was merged to
> >> 'master'.
> >>
> >> Now, I think it would be okay to wait for 'bc/cocci' to go to 'master'
> >> before integrating the 'am-3-merge-recursive-direct' branch, but I would
> >> want to avoid waiting for 'jh/clean-smudge-annex'.
> 
> Nothing seems to be happening on jh/clean-smudge-annex front, so
> unless we hear otherwise from Joey in the coming couple of days,
> let's get js/am-3-merge-recursive-direct untangled from its
> dependencies and get it into a shape to hit 'next'.
Okay, I will rebase and re-submit.
Show 11 quoted lines
> You can assume that I'll merge bc/cocci and
> js/am-call-theirs-theirs-in-fallback-3way to 'master' during that time,
> so an appropriate base to use would be the result of
> 
>     git checkout master^0
>     git merge bc/cocci
>     git merge js/am-call-theirs-theirs-in-fallback-3way
>     git merge cc/apply-am
> 
> if you want a semi-solid base to rebuild am-3-merge-recursive-direct
> on.

Okay. The way my `mail-patch-series.sh` script works, however, is that it infers the base commit by testing which tip among pu, next and master is the most appropriate.

So I guess I will have to hack it up a bit to allow basing my patch series on something custom.

> I am not 100% sure about the doneness of cc/apply-am yet, though but it
> could be that am-3-merge-recursive-direct does not have to depend on it
> at all.  You'd know better than I do ;-)

I agree on the doneness of cc/apply-am (as you know, I tried to help it along but my comments had an adverse effect).

And no: nothing in my entire rebase--helper patches relies on cc/apply-am (I do not even believe that anything conflicts with it, either).

> After that, I'll see if the conflict resolution is manageable when
> remerging jh/clean-smudge-annex to 'pu', and if it is not, I'll worry
> about it at that point.
I can help with that, too.

Ciao, Dscho

Johannes Schindelin· Jul 19, 2016, 14:28 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Hi Junio,
On Tue, 19 Jul 2016, Johannes Schindelin wrote:
Show 15 quoted lines
> On Mon, 18 Jul 2016, Junio C Hamano wrote:
> 
> > Junio C Hamano <gitster@pobox.com> writes:
> > 
> > You can assume that I'll merge bc/cocci and
> > js/am-call-theirs-theirs-in-fallback-3way to 'master' during that time,
> > so an appropriate base to use would be the result of
> > 
> >     git checkout master^0
> >     git merge bc/cocci
> >     git merge js/am-call-theirs-theirs-in-fallback-3way
> >     git merge cc/apply-am
> > 
> > if you want a semi-solid base to rebuild am-3-merge-recursive-direct
> > on.

I do not need cc/apply-am at all, it turns out, but my patch series has a minor conflict with 'jc/renormalize-merge-kill-safer-crlf'.

Since you indicated that you want to cook that branch a bit in 'next' first, I will rebase to master+bc/cocci+js/am-call-theirs-theirs and re-submit.

Ciao, Dscho

Junio C Hamano· Jul 19, 2016, 19:33 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/16] Use merge_recursive() directly in the builtin am

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 6 quoted lines
> I do not need cc/apply-am at all, it turns out, but my patch series has a
> minor conflict with 'jc/renormalize-merge-kill-safer-crlf'.
>
> Since you indicated that you want to cook that branch a bit in 'next'
> first, I will rebase to master+bc/cocci+js/am-call-theirs-theirs and
> re-submit.

Thanks. I suspect the renomalization would graduate earlier than the topic in question, but leaving dependency to the minimum and have rerere take care of minor conflicts has proved to be a better approach in general over time; the base you chose above sounds appropriate.

Thanks.
Johannes Schindelin· Jul 22, 2016, 12:23 UTC · re: Johannes Schindelin · lore

[PATCH v4 00/16] Use merge_recursive() directly in the builtin am

This is the fourth iteration of the long-awaited re-roll of the attempt to avoid spawning merge-recursive from the builtin am and use merge_recursive() directly instead.

The *real* reason for the reroll is that I need a libified recursive merge to accelerate the interactive rebase by teaching the sequencer to do rebase -i's grunt work. Coming with a very nice 3x-5x speedup of `rebase -i`.

In this endeavor, we need to be extra careful to retain backwards compatibility. The test script t6022-merge-rename.sh, for example, verifies that `git pull` exits with status 128 in case of a fatal error. To that end, we need to make sure that fatal errors are handled by existing (builtin) users via exit(128) (or die(), which calls exit(128) at the end). New users (such as a builtin helper doing rebase -i's grunt work) may want to print some helpful advice what happened and how to get out of this mess before erroring out.

The changes relative to the second iteration of this patch series (I have a feeling that nobody reviewed the 3rd iteration because it was based on `pu`):

- the was_tracked() function was adjusted as per Junio's suggestions
- the "counter gender bias" patch was submitted, and accepted,
  separately, even if the version we settled on sends a much weaker
  message than I would have preferred
- this patch series is on top of 'master' again
- a test was introduced to verify that we do not reintroduce the bug
  which required our hot fix to spawn the recursive merge again
- while at it, I fixed a ton of incorrect indentations that I missed in
  the first three iterations
- as the bc/cocci branch was merged, a couple of fixups were necessary
  to the patch that avoids spawning the recursive merge
- the interdiff is relative to v2 rebased onto master. That means that
  I had to simulate the "her_tree" change for the sake of the interdiff.

This patch series touches rather important code. Now that I addressed concerns such as fixing translated bug reports, I would appreciate thorough reviews with a focus on the critical parts of the code, those that could result in regressions.

Johannes Schindelin (16):
  Verify that `git pull --rebase` shows the helpful advice when failing
  Report bugs consistently
  Avoid translating bug messages
  merge-recursive: clarify code in was_tracked()
  Prepare the builtins for a libified merge_recursive()
  merge_recursive: abort properly upon errors
  merge-recursive: avoid returning a wholesale struct
  merge-recursive: allow write_tree_from_memory() to error out
  merge-recursive: handle return values indicating errors
  merge-recursive: switch to returning errors instead of dying
  am -3: use merge_recursive() directly again
  merge-recursive: flush output buffer before printing error messages
  merge-recursive: write the commit title in one go
  merge-recursive: offer an option to retain the output in 'obuf'
  Ensure that the output buffer is released after calling merge_trees()
  merge-recursive: flush output buffer even when erroring out
 builtin/am.c           |  62 ++----
 builtin/checkout.c     |   5 +-
 builtin/ls-files.c     |   3 +-
 builtin/merge.c        |   2 +
 builtin/update-index.c |   2 +-
 grep.c                 |   8 +-
 imap-send.c            |   4 +-
 merge-recursive.c      | 571 +++++++++++++++++++++++++++++--------------------
 merge-recursive.h      |   2 +-
 sequencer.c            |   5 +
 sha1_file.c            |   4 +-
 t/t5520-pull.sh        |  30 +++
 trailer.c              |   2 +-
 transport.c            |   2 +-
 wt-status.c            |   4 +-
 15 files changed, 413 insertions(+), 293 deletions(-)
Published-As: https://github.com/dscho/git/releases/tag/am-3-merge-recursive-direct-v4
Interdiff vs v3:
 diff --git a/builtin/am.c b/builtin/am.c
 index bf4f79b..cfb79ea 100644
 --- a/builtin/am.c
 +++ b/builtin/am.c
 @@ -1583,15 +1583,14 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
   */
  static int fall_back_threeway(const struct am_state *state, const char *index_path)
  {
 -	unsigned char orig_tree[GIT_SHA1_RAWSZ], her_tree[GIT_SHA1_RAWSZ],
 -		      our_tree[GIT_SHA1_RAWSZ];
 -	const unsigned char *bases[1] = {orig_tree};
 +	struct object_id orig_tree, their_tree, our_tree;
 +	const struct object_id *bases[1] = { &orig_tree };
  	struct merge_options o;
  	struct commit *result;
 -	char *her_tree_name;
 +	char *their_tree_name;
  
 -	if (get_sha1("HEAD", our_tree) < 0)
 -		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
 +	if (get_oid("HEAD", &our_tree) < 0)
 +		hashcpy(our_tree.hash, EMPTY_TREE_SHA1_BIN);
  
  	if (build_fake_ancestor(state, index_path))
  		return error("could not build fake ancestor");
 @@ -1599,7 +1598,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  	discard_cache();
  	read_cache_from(index_path);
  
 -	if (write_index_as_tree(orig_tree, &the_index, index_path, 0, NULL))
 +	if (write_index_as_tree(orig_tree.hash, &the_index, index_path, 0, NULL))
  		return error(_("Repository lacks necessary blobs to fall back on 3-way merge."));
  
  	say(state, stdout, _("Using index info to reconstruct a base tree..."));
 @@ -1615,7 +1614,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  		init_revisions(&rev_info, NULL);
  		rev_info.diffopt.output_format = DIFF_FORMAT_NAME_STATUS;
  		diff_opt_parse(&rev_info.diffopt, &diff_filter_str, 1, rev_info.prefix);
 -		add_pending_sha1(&rev_info, "HEAD", our_tree, 0);
 +		add_pending_sha1(&rev_info, "HEAD", our_tree.hash, 0);
  		diff_setup_done(&rev_info.diffopt);
  		run_diff_index(&rev_info, 1);
  	}
 @@ -1624,7 +1623,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  		return error(_("Did you hand edit your patch?\n"
  				"It does not apply to blobs recorded in its index."));
  
 -	if (write_index_as_tree(her_tree, &the_index, index_path, 0, NULL))
 +	if (write_index_as_tree(their_tree.hash, &the_index, index_path, 0, NULL))
  		return error("could not write tree");
  
  	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
 @@ -1634,7 +1633,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  
  	/*
  	 * This is not so wrong. Depending on which base we picked, orig_tree
 -	 * may be wildly different from ours, but her_tree has the same set of
 +	 * may be wildly different from ours, but their_tree has the same set of
  	 * wildly different changes in parts the patch did not touch, so
  	 * recursive ends up canceling them, saying that we reverted all those
  	 * changes.
 @@ -1643,19 +1642,19 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
  	init_merge_options(&o);
  
  	o.branch1 = "HEAD";
 -	her_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
 -	o.branch2 = her_tree_name;
 +	their_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
 +	o.branch2 = their_tree_name;
  
  	if (state->quiet)
  		o.verbosity = 0;
  
 -	if (merge_recursive_generic(&o, our_tree, her_tree, 1, bases, &result)) {
 +	if (merge_recursive_generic(&o, &our_tree, &their_tree, 1, bases, &result)) {
  		rerere(state->allow_rerere_autoupdate);
 -		free(her_tree_name);
 +		free(their_tree_name);
  		return error(_("Failed to merge in the changes."));
  	}
  
 -	free(her_tree_name);
 +	free(their_tree_name);
  	return 0;
  }
  
 diff --git a/merge-recursive.c b/merge-recursive.c
 index cd2a533..501cfb5 100644
 --- a/merge-recursive.c
 +++ b/merge-recursive.c
 @@ -686,20 +686,19 @@ static int was_tracked(const char *path)
  {
  	int pos = cache_name_pos(path, strlen(path));
  
 -	if (pos >= 0)
 -		return pos < active_nr;
 +	if (0 <= pos)
 +		/* we have been tracking this path */
 +		return 1;
 +
  	/*
 -	 * cache_name_pos() looks for stage == 0, even if we did not ask for
 -	 * it. Let's look for stage == 2 now.
 +	 * Look for an unmerged entry for the path,
 +	 * specifically stage #2, which would indicate
 +	 * that "our" side before the merge started
 +	 * had the path tracked (and resulted in a conflict).
  	 */
 -	for (pos = -1 - pos; pos < active_nr &&
 -	     !strcmp(path, active_cache[pos]->name); pos++)
 -		/*
 -		 * If stage #0, it is definitely tracked.
 -		 * If it has stage #2 then it was tracked
 -		 * before this merge started.  All other
 -		 * cases the path was not tracked.
 -		 */
 +	for (pos = -1 - pos;
 +	     pos < active_nr && !strcmp(path, active_cache[pos]->name);
 +	     pos++)
  		if (ce_stage(active_cache[pos]) == 2)
  			return 1;
  	return 0;
 @@ -761,11 +760,11 @@ static int make_room_for_path(struct merge_options *o, const char *path)
  }
  
  static int update_file_flags(struct merge_options *o,
 -			      const struct object_id *oid,
 -			      unsigned mode,
 -			      const char *path,
 -			      int update_cache,
 -			      int update_wd)
 +			     const struct object_id *oid,
 +			     unsigned mode,
 +			     const char *path,
 +			     int update_cache,
 +			     int update_wd)
  {
  	int ret = 0;
  
 @@ -816,7 +815,7 @@ static int update_file_flags(struct merge_options *o,
  			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
  			if (fd < 0) {
  				ret = err(o, _("failed to open '%s': %s"),
 -					path, strerror(errno));
 +					  path, strerror(errno));
  				goto free_buf;
  			}
  			write_in_full(fd, buf, size);
 @@ -830,8 +829,9 @@ static int update_file_flags(struct merge_options *o,
  					path, strerror(errno));
  			free(lnk);
  		} else
 -			ret = err(o, _("do not know what to do with %06o %s '%s'"),
 -				mode, oid_to_hex(oid), path);
 +			ret = err(o,
 +				  _("do not know what to do with %06o %s '%s'"),
 +				  mode, oid_to_hex(oid), path);
   free_buf:
  		free(buf);
  	}
 @@ -842,10 +842,10 @@ static int update_file_flags(struct merge_options *o,
  }
  
  static int update_file(struct merge_options *o,
 -			int clean,
 -			const struct object_id *oid,
 -			unsigned mode,
 -			const char *path)
 +		       int clean,
 +		       const struct object_id *oid,
 +		       unsigned mode,
 +		       const char *path)
  {
  	return update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
  }
 @@ -973,9 +973,9 @@ static int merge_file_1(struct merge_options *o,
  				ret = err(o, _("Failed to execute internal merge"));
  
  			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 -					    blob_type, result->oid.hash))
 +						    blob_type, result->oid.hash))
  				ret = err(o, _("Unable to add %s to database"),
 -					a->path);
 +					  a->path);
  
  			free(result_buf.ptr);
  			if (ret)
 @@ -1020,7 +1020,8 @@ static int merge_file_special_markers(struct merge_options *o,
  		side2 = xstrfmt("%s:%s", branch2, filename2);
  
  	ret = merge_file_1(o, one, a, b,
 -		side1 ? side1 : branch1, side2 ? side2 : branch2, mfi);
 +			   side1 ? side1 : branch1,
 +			   side2 ? side2 : branch2, mfi);
  	free(side1);
  	free(side2);
  	return ret;
 @@ -1068,7 +1069,8 @@ static int handle_change_delete(struct merge_options *o,
  		 */
  		ret = remove_file_from_cache(path);
  		if (!ret)
 -			ret = update_file(o, 0, o_oid, o_mode, renamed ? renamed : path);
 +			ret = update_file(o, 0, o_oid, o_mode,
 +					  renamed ? renamed : path);
  	} else if (!a_oid) {
  		if (!renamed) {
  			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 @@ -1129,19 +1131,19 @@ static int conflict_rename_delete(struct merge_options *o,
  	}
  
  	if (handle_change_delete(o,
 -			     o->call_depth ? orig->path : dest->path,
 -			     &orig->oid, orig->mode,
 -			     a_oid, a_mode,
 -			     b_oid, b_mode,
 -			     _("rename"), _("renamed")))
 +				 o->call_depth ? orig->path : dest->path,
 +				 &orig->oid, orig->mode,
 +				 a_oid, a_mode,
 +				 b_oid, b_mode,
 +				 _("rename"), _("renamed")))
  		return -1;
  
  	if (o->call_depth)
  		return remove_file_from_cache(dest->path);
  	else
  		return update_stages(o, dest->path, NULL,
 -			      rename_branch == o->branch1 ? dest : NULL,
 -			      rename_branch == o->branch1 ? NULL : dest);
 +				     rename_branch == o->branch1 ? dest : NULL,
 +				     rename_branch == o->branch1 ? NULL : dest);
  }
  
  static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 @@ -1292,11 +1294,11 @@ static int conflict_rename_rename_2to1(struct merge_options *o,
  	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
  
  	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
 -					    o->branch1, c1->path,
 -					    o->branch2, ci->ren1_other.path, &mfi_c1) ||
 +				       o->branch1, c1->path,
 +				       o->branch2, ci->ren1_other.path, &mfi_c1) ||
  	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 -					    o->branch1, ci->ren2_other.path,
 -					    o->branch2, c2->path, &mfi_c2))
 +				       o->branch1, ci->ren2_other.path,
 +				       o->branch2, c2->path, &mfi_c2))
  		return -1;
  
  	if (o->call_depth) {
 @@ -1311,7 +1313,7 @@ static int conflict_rename_rename_2to1(struct merge_options *o,
  		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
  		if (!ret)
  			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
 -				b->path);
 +					  b->path);
  	} else {
  		char *new_path1 = unique_path(o, path, ci->branch1);
  		char *new_path2 = unique_path(o, path, ci->branch2);
 @@ -1321,7 +1323,7 @@ static int conflict_rename_rename_2to1(struct merge_options *o,
  		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
  		if (!ret)
  			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
 -				new_path2);
 +					  new_path2);
  		free(new_path2);
  		free(new_path1);
  	}
 @@ -1512,11 +1514,11 @@ static int process_renames(struct merge_options *o,
  				 * update_file().
  				 */
  				if (update_file_flags(o,
 -						  &ren1->pair->two->oid,
 -						  ren1->pair->two->mode,
 -						  ren1_dst,
 -						  1, /* update_cache */
 -						  0  /* update_wd    */))
 +						      &ren1->pair->two->oid,
 +						      ren1->pair->two->mode,
 +						      ren1_dst,
 +						      1, /* update_cache */
 +						      0  /* update_wd    */))
  					clean_merge = -1;
  			} else if (!oid_eq(&dst_other.oid, &null_oid)) {
  				clean_merge = 0;
 @@ -1528,11 +1530,11 @@ static int process_renames(struct merge_options *o,
  				if (o->call_depth) {
  					struct merge_file_info mfi;
  					if (merge_file_one(o, ren1_dst, &null_oid, 0,
 -							 &ren1->pair->two->oid,
 -							 ren1->pair->two->mode,
 -							 &dst_other.oid,
 -							 dst_other.mode,
 -							 branch1, branch2, &mfi)) {
 +							   &ren1->pair->two->oid,
 +							   ren1->pair->two->mode,
 +							   &dst_other.oid,
 +							   dst_other.mode,
 +							   branch1, branch2, &mfi)) {
  						clean_merge = -1;
  						goto cleanup_and_return;
  					}
 @@ -1545,7 +1547,7 @@ static int process_renames(struct merge_options *o,
  					char *new_path = unique_path(o, ren1_dst, branch2);
  					output(o, 1, _("Adding as %s instead"), new_path);
  					if (update_file(o, 0, &dst_other.oid,
 -						    dst_other.mode, new_path))
 +							dst_other.mode, new_path))
  						clean_merge = -1;
  					free(new_path);
  				}
 @@ -1652,11 +1654,11 @@ static int handle_modify_delete(struct merge_options *o,
  				 struct object_id *b_oid, int b_mode)
  {
  	return handle_change_delete(o,
 -			     path,
 -			     o_oid, o_mode,
 -			     a_oid, a_mode,
 -			     b_oid, b_mode,
 -			     _("modify"), _("modified"));
 +				    path,
 +				    o_oid, o_mode,
 +				    a_oid, a_mode,
 +				    b_oid, b_mode,
 +				    _("modify"), _("modified"));
  }
  
  static int merge_content(struct merge_options *o,
 @@ -1701,8 +1703,8 @@ static int merge_content(struct merge_options *o,
  			df_conflict_remains = 1;
  	}
  	if (merge_file_special_markers(o, &one, &a, &b,
 -					 o->branch1, path1,
 -					 o->branch2, path2, &mfi))
 +				       o->branch1, path1,
 +				       o->branch2, path2, &mfi))
  		return -1;
  
  	if (mfi.clean && !df_conflict_remains &&
 @@ -1749,8 +1751,8 @@ static int merge_content(struct merge_options *o,
  				merged.mode = mfi.mode;
  
  				if (update_stages(o, path, NULL,
 -					      file_from_stage2 ? &merged : NULL,
 -					      file_from_stage2 ? NULL : &merged))
 +						  file_from_stage2 ? &merged : NULL,
 +						  file_from_stage2 ? NULL : &merged))
  					return -1;
  			}
  
 @@ -1794,9 +1796,9 @@ static int process_entry(struct merge_options *o,
  		case RENAME_DELETE:
  			clean_merge = 0;
  			if (conflict_rename_delete(o,
 -					       conflict_info->pair1,
 -					       conflict_info->branch1,
 -					       conflict_info->branch2))
 +						   conflict_info->pair1,
 +						   conflict_info->branch1,
 +						   conflict_info->branch2))
  				clean_merge = -1;
  			break;
  		case RENAME_ONE_FILE_TO_TWO:
 @@ -1828,7 +1830,7 @@ static int process_entry(struct merge_options *o,
  			/* Modify/delete; deleted side may have put a directory in the way */
  			clean_merge = 0;
  			if (handle_modify_delete(o, path, o_oid, o_mode,
 -					     a_oid, a_mode, b_oid, b_mode))
 +						 a_oid, a_mode, b_oid, b_mode))
  				clean_merge = -1;
  		}
  	} else if ((!o_oid && a_oid && !b_oid) ||
 @@ -2040,7 +2042,7 @@ int merge_recursive(struct merge_options *o,
  		o->branch1 = "Temporary merge branch 1";
  		o->branch2 = "Temporary merge branch 2";
  		if (merge_recursive(o, merged_common_ancestors, iter->item,
 -				NULL, &merged_common_ancestors) < 0)
 +				    NULL, &merged_common_ancestors) < 0)
  			return -1;
  		o->branch1 = saved_b1;
  		o->branch2 = saved_b2;
-- 
2.9.0.281.g286a8d9

base-commit: 08bb3500a2a718c3c78b0547c68601cafa7a8fd9
Johannes Schindelin· Jul 22, 2016, 12:24 UTC · re: Johannes Schindelin · lore

[PATCH v4 01/16] Verify that `git pull --rebase` shows the helpful advice when failing

It was noticed by Brendan Forster last October that the builtin `git am` regressed on that. Our hot fix reverted to spawning the recursive merge instead of using it as a library function.

As we are about to revert that hot fix, after making the recursive merge a true library function (i.e. a function that does not die() in case of "normal" errors), let's add a test that verifies that we do not regress on the same problem which made the hot fix necessary in the first place.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t5520-pull.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
Show changes to t/t5520-pull.sh +30 −0
diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
index 37ebbcf..d289056 100755
--- a/t/t5520-pull.sh
+++ b/t/t5520-pull.sh
@@ -255,6 +255,36 @@ test_expect_success '--rebase' '
 	test new = "$(git show HEAD:file2)"
 '
 
+test_expect_success '--rebase with conflicts shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b seq &&
+	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
+	git add seq.txt &&
+	test_tick &&
+	git commit -m "Add seq.txt" &&
+	printf "6\\n" >>seq.txt &&
+	test_tick &&
+	git commit -m "Append to seq.txt" seq.txt &&
+	git checkout -b with-conflicts HEAD^ &&
+	printf "conflicting\\n" >>seq.txt &&
+	test_tick &&
+	git commit -m "Create conflict" seq.txt &&
+	test_must_fail git pull --rebase . seq 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+test_expect_success 'failed --rebase shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b diverging &&
+	test_commit attributes .gitattributes "* text=auto" attrs &&
+	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
+	git update-index --cacheinfo 0644 $sha1 file &&
+	git commit -m v1-with-cr &&
+	git checkout -f -b fails-to-rebase HEAD^ &&
+	test_commit v2-without-cr file "2" file2-lf &&
+	test_must_fail git pull --rebase . diverging 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
 test_expect_success '--rebase fails with multiple branches' '
 	git reset --hard before-rebase &&
 	test_must_fail git pull --rebase . copy master 2>err &&
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 25, 2016, 21:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 01/16] Verify that `git pull --rebase` shows the helpful advice when failing

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> +test_expect_success '--rebase with conflicts shows advice' '
> +	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
> +	git checkout -b seq &&
> +	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
Make this more readble by using test-write-lines, perhaps?
Show 14 quoted lines
> +	git add seq.txt &&
> +	test_tick &&
> +	git commit -m "Add seq.txt" &&
> +	printf "6\\n" >>seq.txt &&
> +	test_tick &&
> +	git commit -m "Append to seq.txt" seq.txt &&
> +	git checkout -b with-conflicts HEAD^ &&
> +	printf "conflicting\\n" >>seq.txt &&
> +	test_tick &&
> +	git commit -m "Create conflict" seq.txt &&
> +	test_must_fail git pull --rebase . seq 2>err >out &&
> +	grep "When you have resolved this problem" out
> +'
> +test_expect_success 'failed --rebase shows advice' '
Need a blank line before this one.
Show 7 quoted lines
> +	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
> +	git checkout -b diverging &&
> +	test_commit attributes .gitattributes "* text=auto" attrs &&
> +	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
> +	git update-index --cacheinfo 0644 $sha1 file &&
> +	git commit -m v1-with-cr &&
> +	git checkout -f -b fails-to-rebase HEAD^ &&

It is unclear what the "-f" is for; is it attempting to clean up a potential mess previous steps might have left? We didn't have it in the previous test above.

Show 8 quoted lines
> +	test_commit v2-without-cr file "2" file2-lf &&
> +	test_must_fail git pull --rebase . diverging 2>err >out &&
> +	grep "When you have resolved this problem" out
> +'
> +
>  test_expect_success '--rebase fails with multiple branches' '
>  	git reset --hard before-rebase &&
>  	test_must_fail git pull --rebase . copy master 2>err &&

Not worth a reroll but after this series settles we would probably want to address some of the above up with a follow-up clean-up patch.

Johannes Schindelin· Jul 26, 2016, 12:21 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 01/16] Verify that `git pull --rebase` shows the helpful advice when failing

Hi Junio,
On Mon, 25 Jul 2016, Junio C Hamano wrote:
Show 8 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +test_expect_success '--rebase with conflicts shows advice' '
> > +	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
> > +	git checkout -b seq &&
> > +	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
> 
> Make this more readble by using test-write-lines, perhaps?
Or even test_seq. Thanks for pointing my nose to this.
Show 16 quoted lines
> > +	git add seq.txt &&
> > +	test_tick &&
> > +	git commit -m "Add seq.txt" &&
> > +	printf "6\\n" >>seq.txt &&
> > +	test_tick &&
> > +	git commit -m "Append to seq.txt" seq.txt &&
> > +	git checkout -b with-conflicts HEAD^ &&
> > +	printf "conflicting\\n" >>seq.txt &&
> > +	test_tick &&
> > +	git commit -m "Create conflict" seq.txt &&
> > +	test_must_fail git pull --rebase . seq 2>err >out &&
> > +	grep "When you have resolved this problem" out
> > +'
> > +test_expect_success 'failed --rebase shows advice' '
> 
> Need a blank line before this one.
Yep, sorry.
Show 11 quoted lines
> > +	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
> > +	git checkout -b diverging &&
> > +	test_commit attributes .gitattributes "* text=auto" attrs &&
> > +	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
> > +	git update-index --cacheinfo 0644 $sha1 file &&
> > +	git commit -m v1-with-cr &&
> > +	git checkout -f -b fails-to-rebase HEAD^ &&
> 
> It is unclear what the "-f" is for; is it attempting to clean up a
> potential mess previous steps might have left?  We didn't have it in
> the previous test above.

It is there to clean up a very non-potential mess: forcing a CR/LF into a file marked with `text=auto` makes a royal mess. Neither `git reset --hard` nor `git stash` will make that file clean. As a consequence, the `git checkout` without an `-f` would *always* fail "because of uncommitted changes".

I clarified that in a comment.
Show 11 quoted lines
> > +	test_commit v2-without-cr file "2" file2-lf &&
> > +	test_must_fail git pull --rebase . diverging 2>err >out &&
> > +	grep "When you have resolved this problem" out
> > +'
> > +
> >  test_expect_success '--rebase fails with multiple branches' '
> >  	git reset --hard before-rebase &&
> >  	test_must_fail git pull --rebase . copy master 2>err &&
> 
> Not worth a reroll but after this series settles we would probably
> want to address some of the above up with a follow-up clean-up patch.
As I re-roll anyway, no big deal.

Ciao, Dscho

Johannes Schindelin· Jul 22, 2016, 12:24 UTC · re: Johannes Schindelin · lore

[PATCH v4 02/16] Report bugs consistently

The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:".

As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/ls-files.c     |  3 ++-
 builtin/update-index.c |  2 +-
 grep.c                 |  8 ++++----
 imap-send.c            |  4 ++--
 merge-recursive.c      | 15 +++++++--------
 sha1_file.c            |  4 ++--
 trailer.c              |  2 +-
 transport.c            |  2 +-
 wt-status.c            |  4 ++--
 9 files changed, 22 insertions(+), 22 deletions(-)
Show changes to 9 files +22 −22

builtin/ls-files.c, builtin/update-index.c, grep.c, imap-send.c, merge-recursive.c, sha1_file.c, trailer.c, transport.c, wt-status.c

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f02e3d2..00ea91a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
 				 */
 				pos = cache_name_pos(ent->name, ent->len);
 				if (0 <= pos)
-					die("bug in show-killed-files");
+					die("BUG: killed-file %.*s not found",
+						ent->len, ent->name);
 				pos = -pos - 1;
 				while (pos < active_nr &&
 				       ce_stage(active_cache[pos]))
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 6cdfd5f..ba04b19 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1146,7 +1146,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		report(_("Untracked cache enabled for '%s'"), get_git_work_tree());
 		break;
 	default:
-		die("Bug: bad untracked_cache value: %d", untracked_cache);
+		die("BUG: bad untracked_cache value: %d", untracked_cache);
 	}
 
 	if (active_cache_changed) {
diff --git a/grep.c b/grep.c
index 394c856..22cbb73 100644
--- a/grep.c
+++ b/grep.c
@@ -693,10 +693,10 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 	for (p = opt->header_list; p; p = p->next) {
 		if (p->token != GREP_PATTERN_HEAD)
-			die("bug: a non-header pattern in grep header list.");
+			die("BUG: a non-header pattern in grep header list.");
 		if (p->field < GREP_HEADER_FIELD_MIN ||
 		    GREP_HEADER_FIELD_MAX <= p->field)
-			die("bug: unknown header field %d", p->field);
+			die("BUG: unknown header field %d", p->field);
 		compile_regexp(p, opt);
 	}
 
@@ -709,7 +709,7 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 		h = compile_pattern_atom(&pp);
 		if (!h || pp != p->next)
-			die("bug: malformed header expr");
+			die("BUG: malformed header expr");
 		if (!header_group[p->field]) {
 			header_group[p->field] = h;
 			continue;
@@ -1514,7 +1514,7 @@ static int grep_source_1(struct grep_opt *opt, struct grep_source *gs, int colle
 		case GREP_BINARY_TEXT:
 			break;
 		default:
-			die("bug: unknown binary handling mode");
+			die("BUG: unknown binary handling mode");
 		}
 	}
 
diff --git a/imap-send.c b/imap-send.c
index db0fafe..67d67f8 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -506,12 +506,12 @@ static char *next_arg(char **s)
 
 static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 {
-	int ret;
+	int ret = -1;
 	va_list va;
 
 	va_start(va, fmt);
 	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
-		die("Fatal: buffer too small. Please report a bug.");
+		die("BUG: buffer too small (%d < %d)", ret, blen);
 	va_end(va);
 	return ret;
 }
diff --git a/merge-recursive.c b/merge-recursive.c
index 48fe7e7..7400034 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -259,7 +259,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
-		die("Bug in merge-recursive.c");
+		die("BUG: unmerged index entries in merge-recursive.c");
 	}
 
 	if (!active_cache_tree)
@@ -957,9 +957,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
-		} else {
-			die(_("unsupported object type in the tree"));
-		}
+		} else
+			die(_("BUG: unsupported object type in the tree"));
 	}
 
 	return result;
@@ -1345,7 +1344,7 @@ static int process_renames(struct merge_options *o,
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1_src != ren2_src");
+				die("BUG: ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
@@ -1379,7 +1378,7 @@ static int process_renames(struct merge_options *o,
 			ren2 = lookup->util;
 			ren2_dst = ren2->pair->two->path;
 			if (strcmp(ren1_dst, ren2_dst) != 0)
-				die("ren1_dst != ren2_dst");
+				die("BUG: ren1_dst != ren2_dst");
 
 			clean_merge = 0;
 			ren2->processed = 1;
@@ -1803,7 +1802,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("Fatal merge failure, shouldn't happen."));
+		die(_("BUG: fatal merge failure, shouldn't happen."));
 
 	return clean_merge;
 }
@@ -1861,7 +1860,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("Unprocessed path??? %s"),
+				die(_("BUG: unprocessed path??? %s"),
 				    entries->items[i].string);
 		}
 
diff --git a/sha1_file.c b/sha1_file.c
index d5e1121..5085fe0 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -795,7 +795,7 @@ void close_all_packs(void)
 
 	for (p = packed_git; p; p = p->next)
 		if (p->do_not_close)
-			die("BUG! Want to close pack marked 'do-not-close'");
+			die("BUG: want to close pack marked 'do-not-close'");
 		else
 			close_pack(p);
 }
@@ -2330,7 +2330,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 	case OBJ_OFS_DELTA:
 	case OBJ_REF_DELTA:
 		if (data)
-			die("BUG in unpack_entry: left loop at a valid delta");
+			die("BUG: unpack_entry: left loop at a valid delta");
 		break;
 	case OBJ_COMMIT:
 	case OBJ_TREE:
diff --git a/trailer.c b/trailer.c
index 8e48a5c..c6ea9ac 100644
--- a/trailer.c
+++ b/trailer.c
@@ -562,7 +562,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
-		die("internal bug in trailer.c");
+		die("BUG: trailer.c: unhandled type %d", type);
 	}
 	return 0;
 }
diff --git a/transport.c b/transport.c
index 59b911e..5d446da 100644
--- a/transport.c
+++ b/transport.c
@@ -563,7 +563,7 @@ void transport_take_over(struct transport *transport,
 	struct git_transport_data *data;
 
 	if (!transport->smart_options)
-		die("Bug detected: Taking over transport requires non-NULL "
+		die("BUG: taking over transport requires non-NULL "
 		    "smart_options field.");
 
 	data = xcalloc(1, sizeof(*data));
diff --git a/wt-status.c b/wt-status.c
index de62ab2..3fb86a4 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die("bug: unhandled unmerged status %x", stagemask);
+		die("BUG: unhandled unmerged status %x", stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die("bug: unhandled diff status %c", status);
+		die("BUG: unhandled diff status %c", status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 25, 2016, 21:44 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 02/16] Report bugs consistently

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 19 quoted lines
> diff --git a/imap-send.c b/imap-send.c
> index db0fafe..67d67f8 100644
> --- a/imap-send.c
> +++ b/imap-send.c
> @@ -506,12 +506,12 @@ static char *next_arg(char **s)
>  
>  static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
>  {
> -	int ret;
> +	int ret = -1;
>  	va_list va;
>  
>  	va_start(va, fmt);
>  	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
> -		die("Fatal: buffer too small. Please report a bug.");
> +		die("BUG: buffer too small (%d < %d)", ret, blen);
>  	va_end(va);
>  	return ret;
>  }

If "you gave me this size but you need at least this much" is truly worth reporting, then this is misleading (ret is shown as -1 but you do not even know how much is necessary). In any case, this should be done as a separate step anyway.

All the other hunks looked sensible.
Thanks.
Jeff King· Jul 25, 2016, 22:17 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 02/16] Report bugs consistently

On Mon, Jul 25, 2016 at 02:44:25PM -0700, Junio C Hamano wrote:
Show 24 quoted lines
> > diff --git a/imap-send.c b/imap-send.c
> > index db0fafe..67d67f8 100644
> > --- a/imap-send.c
> > +++ b/imap-send.c
> > @@ -506,12 +506,12 @@ static char *next_arg(char **s)
> >  
> >  static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
> >  {
> > -	int ret;
> > +	int ret = -1;
> >  	va_list va;
> >  
> >  	va_start(va, fmt);
> >  	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
> > -		die("Fatal: buffer too small. Please report a bug.");
> > +		die("BUG: buffer too small (%d < %d)", ret, blen);
> >  	va_end(va);
> >  	return ret;
> >  }
> 
> If "you gave me this size but you need at least this much" is truly
> worth reporting, then this is misleading (ret is shown as -1 but you
> do not even know how much is necessary).  In any case, this should
> be done as a separate step anyway.

Hrm, isn't "ret" going to be the necessary size? According to the standard, it should tell us how many bytes were needed, not "-1" (this is the "your vsnprintf is broken" case handled by the strbuf code).

I do think the numbers are reversed, though. It should be "blen < ret".
-Peff
Junio C Hamano· Jul 25, 2016, 22:36 UTC · re: Jeff King · lore

Re: [PATCH v4 02/16] Report bugs consistently

Jeff King <peff@peff.net> writes:
Show 30 quoted lines
> On Mon, Jul 25, 2016 at 02:44:25PM -0700, Junio C Hamano wrote:
>
>> > diff --git a/imap-send.c b/imap-send.c
>> > index db0fafe..67d67f8 100644
>> > --- a/imap-send.c
>> > +++ b/imap-send.c
>> > @@ -506,12 +506,12 @@ static char *next_arg(char **s)
>> >  
>> >  static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
>> >  {
>> > -	int ret;
>> > +	int ret = -1;
>> >  	va_list va;
>> >  
>> >  	va_start(va, fmt);
>> >  	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
>> > -		die("Fatal: buffer too small. Please report a bug.");
>> > +		die("BUG: buffer too small (%d < %d)", ret, blen);
>> >  	va_end(va);
>> >  	return ret;
>> >  }
>> 
>> If "you gave me this size but you need at least this much" is truly
>> worth reporting, then this is misleading (ret is shown as -1 but you
>> do not even know how much is necessary).  In any case, this should
>> be done as a separate step anyway.
>
> Hrm, isn't "ret" going to be the necessary size? According to the
> standard, it should tell us how many bytes were needed, not "-1" (this
> is the "your vsnprintf is broken" case handled by the strbuf code).

Yes. If blen <= 0, we do not even do vsnprintf() and that is why Dscho added "int ret = -1" initialization; otherwise his new die() would end up referencing uninitialized ret.

> I do think the numbers are reversed, though. It should be "blen < ret".
That, too ;-)
Johannes Schindelin· Jul 26, 2016, 12:24 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 02/16] Report bugs consistently

Hi Junio & Peff,
On Mon, 25 Jul 2016, Junio C Hamano wrote:
Show 36 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > On Mon, Jul 25, 2016 at 02:44:25PM -0700, Junio C Hamano wrote:
> >
> >> > diff --git a/imap-send.c b/imap-send.c
> >> > index db0fafe..67d67f8 100644
> >> > --- a/imap-send.c
> >> > +++ b/imap-send.c
> >> > @@ -506,12 +506,12 @@ static char *next_arg(char **s)
> >> >  
> >> >  static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
> >> >  {
> >> > -	int ret;
> >> > +	int ret = -1;
> >> >  	va_list va;
> >> >  
> >> >  	va_start(va, fmt);
> >> >  	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
> >> > -		die("Fatal: buffer too small. Please report a bug.");
> >> > +		die("BUG: buffer too small (%d < %d)", ret, blen);
> >> >  	va_end(va);
> >> >  	return ret;
> >> >  }
> >> 
> >> If "you gave me this size but you need at least this much" is truly
> >> worth reporting, then this is misleading (ret is shown as -1 but you
> >> do not even know how much is necessary).  In any case, this should
> >> be done as a separate step anyway.
> >
> > Hrm, isn't "ret" going to be the necessary size? According to the
> > standard, it should tell us how many bytes were needed, not "-1" (this
> > is the "your vsnprintf is broken" case handled by the strbuf code).
> 
> Yes.  If blen <= 0, we do not even do vsnprintf() and that is why
> Dscho added "int ret = -1" initialization; otherwise his new die()
> would end up referencing uninitialized ret.

Exactly. While I was fixing this bug message, it occurred to me that it makes little sense to ask a user to report a bug when it is unknown how small the buffer was and what would have been the desired buffer size. So I did a fly-by fix.

However, it is true that this is completely outside the purpose of this patch series (in fact, most of this patch is completely outside the purpose, and I am regretting that dearly, as the patch series' submission already takes almost a month, and I only now get people to review the critical parts of the changes).

So I simply backed out the more verbose message and *only* do the obvious thing: replace the "Fatal:" prefix by a "BUG:" one.

> > I do think the numbers are reversed, though. It should be "blen < ret".
> 
> That, too ;-)
True. All the better that I reverted that part of the patch ;-)

Ciao, Dscho

Johannes Schindelin· Jul 22, 2016, 12:24 UTC · re: Johannes Schindelin · lore

[PATCH v4 03/16] Avoid translating bug messages

While working on the patch series that avoids die()ing in recursive merges, the issue came up that bug reports (i.e. die("BUG: ...") constructs) should never be translated, as the target audience is the Git developer community, not necessarily the current user, and hence a translated message would make it *harder* to address the problem.

So let's stop translating the obvious ones. As it is really, really outside the purview of this patch series to see whether there are more die() statements that report bugs and are currently translated, that task is left for another day and patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +3 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 7400034..e0d5a57 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -958,7 +958,7 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
 		} else
-			die(_("BUG: unsupported object type in the tree"));
+			die("BUG: unsupported object type in the tree");
 	}
 
 	return result;
@@ -1802,7 +1802,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("BUG: fatal merge failure, shouldn't happen."));
+		die("BUG: fatal merge failure, shouldn't happen.");
 
 	return clean_merge;
 }
@@ -1860,7 +1860,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("BUG: unprocessed path??? %s"),
+				die("BUG: unprocessed path??? %s",
 				    entries->items[i].string);
 		}
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 04/16] merge-recursive: clarify code in was_tracked()

It can be puzzling to see that was_tracked() asks to get an index entry by name, but does not take a negative return value for an answer.

The reason we have to do this is that cache_name_pos() only looks for entries in stage 0, even if nobody asked for any stage in particular.

Let's rewrite the logic a little bit, to handle the easy case early: if cache_name_pos() returned a non-negative position, we know it is a match, and we do not even have to compare the name again (cache_name_pos() did that for us already). We can say right away: yes, this file was tracked.

Only if there was no exact match do we need to look harder for any matching entry in stage 2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)
Show changes to merge-recursive.c +14 −16
diff --git a/merge-recursive.c b/merge-recursive.c
index e0d5a57..dc3182b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -658,23 +658,21 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
-	if (pos < 0)
-		pos = -1 - pos;
-	while (pos < active_nr &&
-	       !strcmp(path, active_cache[pos]->name)) {
-		/*
-		 * If stage #0, it is definitely tracked.
-		 * If it has stage #2 then it was tracked
-		 * before this merge started.  All other
-		 * cases the path was not tracked.
-		 */
-		switch (ce_stage(active_cache[pos])) {
-		case 0:
-		case 2:
+	if (0 <= pos)
+		/* we have been tracking this path */
+		return 1;
+
+	/*
+	 * Look for an unmerged entry for the path,
+	 * specifically stage #2, which would indicate
+	 * that "our" side before the merge started
+	 * had the path tracked (and resulted in a conflict).
+	 */
+	for (pos = -1 - pos;
+	     pos < active_nr && !strcmp(path, active_cache[pos]->name);
+	     pos++)
+		if (ce_stage(active_cache[pos]) == 2)
 			return 1;
-		}
-		pos++;
-	}
 	return 0;
 }
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 05/16] Prepare the builtins for a libified merge_recursive()

Previously, callers of merge_trees() or merge_recursive() expected that code to die() with an error message. This used to be okay because we called those commands from scripts, and had a chance to print out a message in case the command failed fatally (read: with exit code 128).

As scripting incurs its own set of problems (portability, speed, idiosynchracies of different shells, limited data structures leading to inefficient code), we are converting more and more of these scripts into builtins, using library functions directly.

We already tried to use merge_recursive() directly in the builtin git-am, for example. Unfortunately, we had to roll it back temporarily because some of the code in merge-recursive.c still deemed it okay to call die(), when the builtin am code really wanted to print out a useful advice after the merge failed fatally. In the next commits, we want to fix that.

The code touched by this commit expected merge_trees() to die() with some useful message when there is an error condition, but merge_trees() is going to be improved by converting all die() calls to return error() instead (i.e. return value -1 after printing out the message as before), so that the caller can react more flexibly.

This is a step to prepare for the version of merge_trees() that no longer dies, even if we just imitate the previous behavior by calling exit(128): this is what callers of e.g. `git merge` have come to expect.

Note that the callers of the sequencer (revert and cherry-pick) already fail fast even for the return value -1; The only difference is that they now get a chance to say "<command> failed".

A caller of merge_trees() might want handle error messages themselves (or even suppress them). As this patch is already complex enough, we leave that change for a later patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 4 +++-
 builtin/merge.c    | 2 ++
 sequencer.c        | 4 ++++
 3 files changed, 9 insertions(+), 1 deletion(-)
Show changes to 3 files +9 −1

builtin/checkout.c, builtin/merge.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 27c1a05..07dea3b 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			o.ancestor = old->name;
 			o.branch1 = new->name;
 			o.branch2 = "local";
-			merge_trees(&o, new->commit->tree, work,
+			ret = merge_trees(&o, new->commit->tree, work,
 				old->commit->tree, &result);
+			if (ret < 0)
+				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
 			if (ret)
diff --git a/builtin/merge.c b/builtin/merge.c
index 19b3bc2..148a9a5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -673,6 +673,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
+		if (clean < 0)
+			exit(128);
 		if (active_cache_changed &&
 		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 			die (_("unable to write %s"), get_index_file());
diff --git a/sequencer.c b/sequencer.c
index cdfac82..286a435 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (clean < 0)
+		return clean;
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
@@ -559,6 +561,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
+		if (res < 0)
+			return res;
 		write_message(&msgbuf, git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 06/16] merge_recursive: abort properly upon errors

There are a couple of places where return values indicating errors are ignored. Let's teach them manners.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +8 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index dc3182b..2d4cb80 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1949,8 +1949,9 @@ int merge_recursive(struct merge_options *o,
 		saved_b2 = o->branch2;
 		o->branch1 = "Temporary merge branch 1";
 		o->branch2 = "Temporary merge branch 2";
-		merge_recursive(o, merged_common_ancestors, iter->item,
-				NULL, &merged_common_ancestors);
+		if (merge_recursive(o, merged_common_ancestors, iter->item,
+				    NULL, &merged_common_ancestors) < 0)
+			return -1;
 		o->branch1 = saved_b1;
 		o->branch2 = saved_b2;
 		o->call_depth--;
@@ -1966,6 +1967,8 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	if (clean < 0)
+		return clean;
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
@@ -2022,6 +2025,9 @@ int merge_recursive_generic(struct merge_options *o,
 	hold_locked_index(lock, 1);
 	clean = merge_recursive(o, head_commit, next_commit, ca,
 			result);
+	if (clean < 0)
+		return clean;
+
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 		return error(_("Unable to write index."));
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 25, 2016, 22:09 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> There are a couple of places where return values indicating errors
> are ignored. Let's teach them manners.

That is because the return value never indicated errors before this series, isn't it? A true error used to be expressed by dying, and the return value indicating "cleanliness" of the merge were deliberately ignored.

The world order changed by previous patches in this series and the callers need to be updated to take the new kind of return values into account. That is not teaching them manners ;-)

Show 21 quoted lines
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  merge-recursive.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index dc3182b..2d4cb80 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1949,8 +1949,9 @@ int merge_recursive(struct merge_options *o,
>  		saved_b2 = o->branch2;
>  		o->branch1 = "Temporary merge branch 1";
>  		o->branch2 = "Temporary merge branch 2";
> -		merge_recursive(o, merged_common_ancestors, iter->item,
> -				NULL, &merged_common_ancestors);
> +		if (merge_recursive(o, merged_common_ancestors, iter->item,
> +				    NULL, &merged_common_ancestors) < 0)
> +			return -1;
>  		o->branch1 = saved_b1;
>  		o->branch2 = saved_b2;
>  		o->call_depth--;
This hunk feels somewhat wrong as-is.

There is a comment before the pre-context explaining why cleanness flag is ignored. It needs to be updated. We still do not care about cleanliness, i.e. 0=clean, 1=merged with conflict, but we now can get negative values so we need to reject and return early if this call indicates an error.

Thee other two hunks make sense.
Thanks.
Show 19 quoted lines
> @@ -1966,6 +1967,8 @@ int merge_recursive(struct merge_options *o,
>  	o->ancestor = "merged common ancestors";
>  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
>  			    &mrtree);
> +	if (clean < 0)
> +		return clean;
>  
>  	if (o->call_depth) {
>  		*result = make_virtual_commit(mrtree, "merged tree");
> @@ -2022,6 +2025,9 @@ int merge_recursive_generic(struct merge_options *o,
>  	hold_locked_index(lock, 1);
>  	clean = merge_recursive(o, head_commit, next_commit, ca,
>  			result);
> +	if (clean < 0)
> +		return clean;
> +
>  	if (active_cache_changed &&
>  	    write_locked_index(&the_index, lock, COMMIT_LOCK))
>  		return error(_("Unable to write index."));
Johannes Schindelin· Jul 26, 2016, 12:26 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Mon, 25 Jul 2016, Junio C Hamano wrote:
Show 13 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > There are a couple of places where return values indicating errors
> > are ignored. Let's teach them manners.
> 
> That is because the return value never indicated errors before this
> series, isn't it?  A true error used to be expressed by dying, and
> the return value indicating "cleanliness" of the merge were
> deliberately ignored.
> 
> The world order changed by previous patches in this series and the
> callers need to be updated to take the new kind of return values
> into account.  That is not teaching them manners ;-)
I reworded that commit message.
Show 29 quoted lines
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  merge-recursive.c | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index dc3182b..2d4cb80 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -1949,8 +1949,9 @@ int merge_recursive(struct merge_options *o,
> >  		saved_b2 = o->branch2;
> >  		o->branch1 = "Temporary merge branch 1";
> >  		o->branch2 = "Temporary merge branch 2";
> > -		merge_recursive(o, merged_common_ancestors, iter->item,
> > -				NULL, &merged_common_ancestors);
> > +		if (merge_recursive(o, merged_common_ancestors, iter->item,
> > +				    NULL, &merged_common_ancestors) < 0)
> > +			return -1;
> >  		o->branch1 = saved_b1;
> >  		o->branch2 = saved_b2;
> >  		o->call_depth--;
> 
> This hunk feels somewhat wrong as-is.
> 
> There is a comment before the pre-context explaining why cleanness
> flag is ignored.  It needs to be updated.  We still do not care
> about cleanliness, i.e. 0=clean, 1=merged with conflict, but we now
> can get negative values so we need to reject and return early if
> this call indicates an error.
I updated that comment. Hopefully now the hunk makes sense ;-)

Ciao, Dscho

Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 07/16] merge-recursive: avoid returning a wholesale struct

It is technically allowed, as per C89, for functions' return type to be complete structs (i.e. *not* just pointers to structs).

However, it was just an oversight of this developer when converting Python code to C code in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which introduced such a return type.

Besides, by converting this construct to pass in the struct, we can now start returning a value that can indicate errors in future patches. This will help the current effort to libify merge-recursive.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 106 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 56 insertions(+), 50 deletions(-)
Show changes to merge-recursive.c +56 −50
diff --git a/merge-recursive.c b/merge-recursive.c
index 2d4cb80..e1cea96 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -885,47 +885,47 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file_1(struct merge_options *o,
+static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
 					   const struct diff_filespec *b,
 					   const char *branch1,
-					   const char *branch2)
+					   const char *branch2,
+					   struct merge_file_info *result)
 {
-	struct merge_file_info result;
-	result.merge = 0;
-	result.clean = 1;
+	result->merge = 0;
+	result->clean = 1;
 
 	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
-		result.clean = 0;
+		result->clean = 0;
 		if (S_ISREG(a->mode)) {
-			result.mode = a->mode;
-			oidcpy(&result.oid, &a->oid);
+			result->mode = a->mode;
+			oidcpy(&result->oid, &a->oid);
 		} else {
-			result.mode = b->mode;
-			oidcpy(&result.oid, &b->oid);
+			result->mode = b->mode;
+			oidcpy(&result->oid, &b->oid);
 		}
 	} else {
 		if (!oid_eq(&a->oid, &one->oid) && !oid_eq(&b->oid, &one->oid))
-			result.merge = 1;
+			result->merge = 1;
 
 		/*
 		 * Merge modes
 		 */
 		if (a->mode == b->mode || a->mode == one->mode)
-			result.mode = b->mode;
+			result->mode = b->mode;
 		else {
-			result.mode = a->mode;
+			result->mode = a->mode;
 			if (b->mode != one->mode) {
-				result.clean = 0;
-				result.merge = 1;
+				result->clean = 0;
+				result->merge = 1;
 			}
 		}
 
 		if (oid_eq(&a->oid, &b->oid) || oid_eq(&a->oid, &one->oid))
-			oidcpy(&result.oid, &b->oid);
+			oidcpy(&result->oid, &b->oid);
 		else if (oid_eq(&b->oid, &one->oid))
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
 			int merge_status;
@@ -937,64 +937,66 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 				die(_("Failed to execute internal merge"));
 
 			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result.oid.hash))
+					    blob_type, result->oid.hash))
 				die(_("Unable to add %s to database"),
 				    a->path);
 
 			free(result_buf.ptr);
-			result.clean = (merge_status == 0);
+			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result.clean = merge_submodule(result.oid.hash,
+			result->clean = merge_submodule(result->oid.hash,
 						       one->path,
 						       one->oid.hash,
 						       a->oid.hash,
 						       b->oid.hash,
 						       !o->call_depth);
 		} else if (S_ISLNK(a->mode)) {
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 
 			if (!oid_eq(&a->oid, &b->oid))
-				result.clean = 0;
+				result->clean = 0;
 		} else
 			die("BUG: unsupported object type in the tree");
 	}
 
-	return result;
+	return 0;
 }
 
-static struct merge_file_info
-merge_file_special_markers(struct merge_options *o,
+static int merge_file_special_markers(struct merge_options *o,
 			   const struct diff_filespec *one,
 			   const struct diff_filespec *a,
 			   const struct diff_filespec *b,
 			   const char *branch1,
 			   const char *filename1,
 			   const char *branch2,
-			   const char *filename2)
+			   const char *filename2,
+			   struct merge_file_info *mfi)
 {
 	char *side1 = NULL;
 	char *side2 = NULL;
-	struct merge_file_info mfi;
+	int ret;
 
 	if (filename1)
 		side1 = xstrfmt("%s:%s", branch1, filename1);
 	if (filename2)
 		side2 = xstrfmt("%s:%s", branch2, filename2);
 
-	mfi = merge_file_1(o, one, a, b,
-			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	ret = merge_file_1(o, one, a, b,
+			   side1 ? side1 : branch1,
+			   side2 ? side2 : branch2, mfi);
 	free(side1);
 	free(side2);
-	return mfi;
+	return ret;
 }
 
-static struct merge_file_info merge_file_one(struct merge_options *o,
+static int merge_file_one(struct merge_options *o,
 					 const char *path,
 					 const struct object_id *o_oid, int o_mode,
 					 const struct object_id *a_oid, int a_mode,
 					 const struct object_id *b_oid, int b_mode,
 					 const char *branch1,
-					 const char *branch2)
+					 const char *branch2,
+					 struct merge_file_info *mfi)
 {
 	struct diff_filespec one, a, b;
 
@@ -1005,7 +1007,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
 	a.mode = a_mode;
 	oidcpy(&b.oid, b_oid);
 	b.mode = b_mode;
-	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
 static void handle_change_delete(struct merge_options *o,
@@ -1178,11 +1180,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct merge_file_info mfi;
 		struct diff_filespec other;
 		struct diff_filespec *add;
-		mfi = merge_file_one(o, one->path,
+		if (merge_file_one(o, one->path,
 				 &one->oid, one->mode,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
-				 ci->branch1, ci->branch2);
+				 ci->branch1, ci->branch2, &mfi))
+			return;
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1236,12 +1239,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
-					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path);
-	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
-					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path);
+	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
+				       o->branch1, c1->path,
+				       o->branch2, ci->ren1_other.path, &mfi_c1) ||
+	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
+				       o->branch1, ci->ren2_other.path,
+				       o->branch2, c2->path, &mfi_c2))
+		return;
 
 	if (o->call_depth) {
 		/*
@@ -1464,12 +1468,13 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					mfi = merge_file_one(o, ren1_dst, &null_oid, 0,
-							 &ren1->pair->two->oid,
-							 ren1->pair->two->mode,
-							 &dst_other.oid,
-							 dst_other.mode,
-							 branch1, branch2);
+					if (merge_file_one(o, ren1_dst, &null_oid, 0,
+							   &ren1->pair->two->oid,
+							   ren1->pair->two->mode,
+							   &dst_other.oid,
+							   dst_other.mode,
+							   branch1, branch2, &mfi))
+						return -1;
 					output(o, 1, _("Adding merged %s"), ren1_dst);
 					update_file(o, 0, &mfi.oid,
 						    mfi.mode, ren1_dst);
@@ -1627,9 +1632,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_special_markers(o, &one, &a, &b,
-					 o->branch1, path1,
-					 o->branch2, path2);
+	if (merge_file_special_markers(o, &one, &a, &b,
+				       o->branch1, path1,
+				       o->branch2, path2, &mfi))
+		return -1;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    oid_eq(&mfi.oid, a_oid) && mfi.mode == a_mode) {
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 08/16] merge-recursive: allow write_tree_from_memory() to error out

It is possible that a tree cannot be written (think: disk full). We will want to give the caller a chance to clean up instead of letting the program die() in such a case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +2 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index e1cea96..d5b9b1c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1879,8 +1879,8 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	if (o->call_depth)
-		*result = write_tree_from_memory(o);
+	if (o->call_depth && !(*result = write_tree_from_memory(o)))
+		return -1;
 
 	return clean;
 }
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 09/16] merge-recursive: handle return values indicating errors

We are about to libify the recursive merge machinery, where we only die() in case of a bug or memory contention. To that end, we must heed negative return values as indicating errors.

This requires our functions to be careful to pass through error conditions in call chains, and for quite a few functions this means that they have to return values to begin with.

The next step will be to convert the places where we currently die() to return negative values (read: -1) instead.

Note that we ignore errors reported by make_room_for_path(), consistent with the previous behavior (update_file_flags() used the return value of make_room_for_path() only to indicate an early return, but not a fatal error): if the error is really a fatal error, we will notice later; If not, it was not that serious a problem to begin with. (Witnesses in favor of this reasoning are t4151-am-abort and t7610-mergetool, which would start failing if we stopped on errors reported by make_room_for_path()).

Also note: while this patch makes the code slightly less readable in update_file_flags() (we introduce a new "goto free_buf;" instead of an explicit "free(buf); return;"), it is a preparatory change for the next patch where we will convert all of the die() calls in the same function to go through the free_buf return path instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 252 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 150 insertions(+), 102 deletions(-)
Show changes to merge-recursive.c +150 −102
diff --git a/merge-recursive.c b/merge-recursive.c
index d5b9b1c..697ba03 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -733,12 +733,12 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	return error(msg, path, _(": perhaps a D/F conflict?"));
 }
 
-static void update_file_flags(struct merge_options *o,
-			      const struct object_id *oid,
-			      unsigned mode,
-			      const char *path,
-			      int update_cache,
-			      int update_wd)
+static int update_file_flags(struct merge_options *o,
+			     const struct object_id *oid,
+			     unsigned mode,
+			     const char *path,
+			     int update_cache,
+			     int update_wd)
 {
 	if (o->call_depth)
 		update_wd = 0;
@@ -774,8 +774,7 @@ static void update_file_flags(struct merge_options *o,
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
-			free(buf);
-			goto update_index;
+			goto free_buf;
 		}
 		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
 			int fd;
@@ -798,20 +797,22 @@ static void update_file_flags(struct merge_options *o,
 		} else
 			die(_("do not know what to do with %06o %s '%s'"),
 			    mode, oid_to_hex(oid), path);
+ free_buf:
 		free(buf);
 	}
  update_index:
 	if (update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+	return 0;
 }
 
-static void update_file(struct merge_options *o,
-			int clean,
-			const struct object_id *oid,
-			unsigned mode,
-			const char *path)
+static int update_file(struct merge_options *o,
+		       int clean,
+		       const struct object_id *oid,
+		       unsigned mode,
+		       const char *path)
 {
-	update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
+	return update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1010,7 +1011,7 @@ static int merge_file_one(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
-static void handle_change_delete(struct merge_options *o,
+static int handle_change_delete(struct merge_options *o,
 				 const char *path,
 				 const struct object_id *o_oid, int o_mode,
 				 const struct object_id *a_oid, int a_mode,
@@ -1018,6 +1019,7 @@ static void handle_change_delete(struct merge_options *o,
 				 const char *change, const char *change_past)
 {
 	char *renamed = NULL;
+	int ret = 0;
 	if (dir_in_way(path, !o->call_depth)) {
 		renamed = unique_path(o, path, a_oid ? o->branch1 : o->branch2);
 	}
@@ -1028,21 +1030,23 @@ static void handle_change_delete(struct merge_options *o,
 		 * correct; since there is no true "middle point" between
 		 * them, simply reuse the base version for virtual merge base.
 		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_oid, o_mode, renamed ? renamed : path);
+		ret = remove_file_from_cache(path);
+		if (!ret)
+			ret = update_file(o, 0, o_oid, o_mode,
+					  renamed ? renamed : path);
 	} else if (!a_oid) {
 		if (!renamed) {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path);
-			update_file(o, 0, b_oid, b_mode, path);
+			ret = update_file(o, 0, b_oid, b_mode, path);
 		} else {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path, renamed);
-			update_file(o, 0, b_oid, b_mode, renamed);
+			ret = update_file(o, 0, b_oid, b_mode, renamed);
 		}
 	} else {
 		if (!renamed) {
@@ -1055,7 +1059,7 @@ static void handle_change_delete(struct merge_options *o,
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch2, change_past,
 			       o->branch1, o->branch1, path, renamed);
-			update_file(o, 0, a_oid, a_mode, renamed);
+			ret = update_file(o, 0, a_oid, a_mode, renamed);
 		}
 		/*
 		 * No need to call update_file() on path when !renamed, since
@@ -1065,9 +1069,11 @@ static void handle_change_delete(struct merge_options *o,
 		 */
 	}
 	free(renamed);
+
+	return ret;
 }
 
-static void conflict_rename_delete(struct merge_options *o,
+static int conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
 				   const char *other_branch)
@@ -1087,21 +1093,20 @@ static void conflict_rename_delete(struct merge_options *o,
 		b_mode = dest->mode;
 	}
 
-	handle_change_delete(o,
-			     o->call_depth ? orig->path : dest->path,
-			     &orig->oid, orig->mode,
-			     a_oid, a_mode,
-			     b_oid, b_mode,
-			     _("rename"), _("renamed"));
-
-	if (o->call_depth) {
-		remove_file_from_cache(dest->path);
-	} else {
-		update_stages(dest->path, NULL,
-			      rename_branch == o->branch1 ? dest : NULL,
-			      rename_branch == o->branch1 ? NULL : dest);
-	}
+	if (handle_change_delete(o,
+				 o->call_depth ? orig->path : dest->path,
+				 &orig->oid, orig->mode,
+				 a_oid, a_mode,
+				 b_oid, b_mode,
+				 _("rename"), _("renamed")))
+		return -1;
 
+	if (o->call_depth)
+		return remove_file_from_cache(dest->path);
+	else
+		return update_stages(dest->path, NULL,
+				     rename_branch == o->branch1 ? dest : NULL,
+				     rename_branch == o->branch1 ? NULL : dest);
 }
 
 static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
@@ -1117,7 +1122,7 @@ static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
-static void handle_file(struct merge_options *o,
+static int handle_file(struct merge_options *o,
 			struct diff_filespec *rename,
 			int stage,
 			struct rename_conflict_info *ci)
@@ -1127,6 +1132,7 @@ static void handle_file(struct merge_options *o,
 	const char *cur_branch, *other_branch;
 	struct diff_filespec other;
 	struct diff_filespec *add;
+	int ret;
 
 	if (stage == 2) {
 		dst_entry = ci->dst_entry1;
@@ -1141,7 +1147,8 @@ static void handle_file(struct merge_options *o,
 	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
 	if (add) {
 		char *add_name = unique_path(o, rename->path, other_branch);
-		update_file(o, 0, &add->oid, add->mode, add_name);
+		if (update_file(o, 0, &add->oid, add->mode, add_name))
+			return -1;
 
 		remove_file(o, 0, rename->path, 0);
 		dst_name = unique_path(o, rename->path, cur_branch);
@@ -1152,17 +1159,20 @@ static void handle_file(struct merge_options *o,
 			       rename->path, other_branch, dst_name);
 		}
 	}
-	update_file(o, 0, &rename->oid, rename->mode, dst_name);
-	if (stage == 2)
-		update_stages(rename->path, NULL, rename, add);
+	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
+		; /* fall through, do allow dst_name to be released */
+	else if (stage == 2)
+		ret = update_stages(rename->path, NULL, rename, add);
 	else
-		update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
+
+	return ret;
 }
 
-static void conflict_rename_rename_1to2(struct merge_options *o,
+static int conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
@@ -1185,14 +1195,16 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
 				 ci->branch1, ci->branch2, &mfi))
-			return;
+			return -1;
+
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, &mfi.oid, mfi.mode, one->path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, one->path))
+			return -1;
 
 		/*
 		 * Above, we put the merged content at the merge-base's
@@ -1203,22 +1215,26 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * resolving the conflict at that path in its favor.
 		 */
 		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, a->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, a->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(a->path);
 		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, b->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, b->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(b->path);
-	} else {
-		handle_file(o, a, 2, ci);
-		handle_file(o, b, 3, ci);
-	}
+	} else if (handle_file(o, a, 2, ci) || handle_file(o, b, 3, ci))
+		return -1;
+
+	return 0;
 }
 
-static void conflict_rename_rename_2to1(struct merge_options *o,
+static int conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* Two files, a & b, were renamed to the same thing, c. */
@@ -1229,6 +1245,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	char *path = c1->path; /* == c2->path */
 	struct merge_file_info mfi_c1;
 	struct merge_file_info mfi_c2;
+	int ret;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1245,7 +1262,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 				       o->branch1, ci->ren2_other.path,
 				       o->branch2, c2->path, &mfi_c2))
-		return;
+		return -1;
 
 	if (o->call_depth) {
 		/*
@@ -1256,19 +1273,25 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 		 * again later for the non-recursive merge.
 		 */
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, b->path);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+					  b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, _("Renaming %s to %s and %s to %s instead"),
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, new_path2);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+					  new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
+
+	return ret;
 }
 
 static int process_renames(struct merge_options *o,
@@ -1453,12 +1476,13 @@ static int process_renames(struct merge_options *o,
 				 * update_file_flags() instead of
 				 * update_file().
 				 */
-				update_file_flags(o,
-						  &ren1->pair->two->oid,
-						  ren1->pair->two->mode,
-						  ren1_dst,
-						  1, /* update_cache */
-						  0  /* update_wd    */);
+				if (update_file_flags(o,
+						      &ren1->pair->two->oid,
+						      ren1->pair->two->mode,
+						      ren1_dst,
+						      1, /* update_cache */
+						      0  /* update_wd    */))
+					clean_merge = -1;
 			} else if (!oid_eq(&dst_other.oid, &null_oid)) {
 				clean_merge = 0;
 				try_merge = 1;
@@ -1473,22 +1497,28 @@ static int process_renames(struct merge_options *o,
 							   ren1->pair->two->mode,
 							   &dst_other.oid,
 							   dst_other.mode,
-							   branch1, branch2, &mfi))
-						return -1;
+							   branch1, branch2, &mfi)) {
+						clean_merge = -1;
+						goto cleanup_and_return;
+					}
 					output(o, 1, _("Adding merged %s"), ren1_dst);
-					update_file(o, 0, &mfi.oid,
-						    mfi.mode, ren1_dst);
+					if (update_file(o, 0, &mfi.oid,
+							mfi.mode, ren1_dst))
+						clean_merge = -1;
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, _("Adding as %s instead"), new_path);
-					update_file(o, 0, &dst_other.oid,
-						    dst_other.mode, new_path);
+					if (update_file(o, 0, &dst_other.oid,
+							dst_other.mode, new_path))
+						clean_merge = -1;
 					free(new_path);
 				}
 			} else
 				try_merge = 1;
 
+			if (clean_merge < 0)
+				goto cleanup_and_return;
 			if (try_merge) {
 				struct diff_filespec *one, *a, *b;
 				src_other.path = (char *)ren1_src;
@@ -1515,6 +1545,7 @@ static int process_renames(struct merge_options *o,
 			}
 		}
 	}
+cleanup_and_return:
 	string_list_clear(&a_by_dst, 0);
 	string_list_clear(&b_by_dst, 0);
 
@@ -1577,18 +1608,18 @@ static int blob_unchanged(const struct object_id *o_oid,
 	return ret;
 }
 
-static void handle_modify_delete(struct merge_options *o,
+static int handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 struct object_id *o_oid, int o_mode,
 				 struct object_id *a_oid, int a_mode,
 				 struct object_id *b_oid, int b_mode)
 {
-	handle_change_delete(o,
-			     path,
-			     o_oid, o_mode,
-			     a_oid, a_mode,
-			     b_oid, b_mode,
-			     _("modify"), _("modified"));
+	return handle_change_delete(o,
+				    path,
+				    o_oid, o_mode,
+				    a_oid, a_mode,
+				    b_oid, b_mode,
+				    _("modify"), _("modified"));
 }
 
 static int merge_content(struct merge_options *o,
@@ -1662,7 +1693,8 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			update_stages(path, &one, &a, &b);
+			if (update_stages(path, &one, &a, &b))
+				return -1;
 	}
 
 	if (df_conflict_remains) {
@@ -1670,30 +1702,33 @@ static int merge_content(struct merge_options *o,
 		if (o->call_depth) {
 			remove_file_from_cache(path);
 		} else {
-			if (!mfi.clean)
-				update_stages(path, &one, &a, &b);
-			else {
+			if (!mfi.clean) {
+				if (update_stages(path, &one, &a, &b))
+					return -1;
+			} else {
 				int file_from_stage2 = was_tracked(path);
 				struct diff_filespec merged;
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				update_stages(path, NULL,
-					      file_from_stage2 ? &merged : NULL,
-					      file_from_stage2 ? NULL : &merged);
+				if (update_stages(path, NULL,
+						  file_from_stage2 ? &merged : NULL,
+						  file_from_stage2 ? NULL : &merged))
+					return -1;
 			}
 
 		}
 		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, _("Adding as %s instead"), new_path);
-		update_file(o, 0, &mfi.oid, mfi.mode, new_path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, new_path)) {
+			free(new_path);
+			return -1;
+		}
 		free(new_path);
 		mfi.clean = 0;
-	} else {
-		update_file(o, mfi.clean, &mfi.oid, mfi.mode, path);
-	}
+	} else if (update_file(o, mfi.clean, &mfi.oid, mfi.mode, path))
+		return -1;
 	return mfi.clean;
-
 }
 
 /* Per entry merge function */
@@ -1721,17 +1756,21 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
-			conflict_rename_delete(o, conflict_info->pair1,
-					       conflict_info->branch1,
-					       conflict_info->branch2);
+			if (conflict_rename_delete(o,
+						   conflict_info->pair1,
+						   conflict_info->branch1,
+						   conflict_info->branch2))
+				clean_merge = -1;
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info);
+			if (conflict_rename_rename_1to2(o, conflict_info))
+				clean_merge = -1;
 			break;
 		case RENAME_TWO_FILES_TO_ONE:
 			clean_merge = 0;
-			conflict_rename_rename_2to1(o, conflict_info);
+			if (conflict_rename_rename_2to1(o, conflict_info))
+				clean_merge = -1;
 			break;
 		default:
 			entry->processed = 0;
@@ -1751,8 +1790,9 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_modify_delete(o, path, o_oid, o_mode,
-					     a_oid, a_mode, b_oid, b_mode);
+			if (handle_modify_delete(o, path, o_oid, o_mode,
+						 a_oid, a_mode, b_oid, b_mode))
+				clean_merge = -1;
 		}
 	} else if ((!o_oid && a_oid && !b_oid) ||
 		   (!o_oid && !a_oid && b_oid)) {
@@ -1784,14 +1824,16 @@ static int process_entry(struct merge_options *o,
 			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s"),
 			       conf, path, other_branch, path, new_path);
-			update_file(o, 0, oid, mode, new_path);
-			if (o->call_depth)
+			if (update_file(o, 0, oid, mode, new_path))
+				clean_merge = -1;
+			else if (o->call_depth)
 				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, _("Adding %s"), path);
 			/* do not overwrite file if already present */
-			update_file_flags(o, oid, mode, path, 1, !a_oid);
+			if (update_file_flags(o, oid, mode, path, 1, !a_oid))
+				clean_merge = -1;
 		}
 	} else if (a_oid && b_oid) {
 		/* Case C: Added in both (check for same permissions) and */
@@ -1854,12 +1896,18 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
+		if (clean < 0)
+			return clean;
 		for (i = entries->nr-1; 0 <= i; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_entry(o, path, e))
-				clean = 0;
+			if (!e->processed) {
+				int ret = process_entry(o, path, e);
+				if (!ret)
+					clean = 0;
+				else if (ret < 0)
+					return ret;
+			}
 		}
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 10/16] merge-recursive: switch to returning errors instead of dying

The recursive merge machinery is supposed to be a library function, i.e. it should return an error when it fails. Originally the functions were part of the builtin "merge-recursive", though, where it was simpler to call die() and be done with error handling.

The existing callers were already prepared to detect negative return values to indicate errors and to behave as previously: exit with code 128 (which is the same thing that die() does, after printing the message).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 62 +++++++++++++++++++++++++++++++------------------------
 1 file changed, 35 insertions(+), 27 deletions(-)
Show changes to merge-recursive.c +35 −27
diff --git a/merge-recursive.c b/merge-recursive.c
index 697ba03..24b42d6 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -266,8 +266,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		active_cache_tree = cache_tree();
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
-	    cache_tree_update(&the_index, 0) < 0)
-		die(_("error building trees"));
+	    cache_tree_update(&the_index, 0) < 0) {
+		error(_("error building trees"));
+		return NULL;
+	}
 
 	result = lookup_tree(active_cache_tree->sha1);
 
@@ -707,12 +709,10 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
-		if (status == SCLD_EXISTS) {
+		if (status == SCLD_EXISTS)
 			/* something else exists */
-			error(msg, path, _(": perhaps a D/F conflict?"));
-			return -1;
-		}
-		die(msg, path, "");
+			return error(msg, path, _(": perhaps a D/F conflict?"));
+		return error(msg, path, "");
 	}
 
 	/*
@@ -740,6 +740,8 @@ static int update_file_flags(struct merge_options *o,
 			     int update_cache,
 			     int update_wd)
 {
+	int ret = 0;
+
 	if (o->call_depth)
 		update_wd = 0;
 
@@ -760,9 +762,11 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			die(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
-		if (type != OBJ_BLOB)
-			die(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+		if (type != OBJ_BLOB) {
+			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			goto free_buf;
+		}
 		if (S_ISREG(mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
 			if (convert_to_working_tree(path, buf, size, &strbuf)) {
@@ -783,8 +787,11 @@ static int update_file_flags(struct merge_options *o,
 			else
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-			if (fd < 0)
-				die_errno(_("failed to open '%s'"), path);
+			if (fd < 0) {
+				ret = error_errno(_("failed to open '%s'"),
+						  path);
+				goto free_buf;
+			}
 			write_in_full(fd, buf, size);
 			close(fd);
 		} else if (S_ISLNK(mode)) {
@@ -792,18 +799,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				die_errno(_("failed to symlink '%s'"), path);
+				ret = error_errno(_("failed to symlink '%s'"), path);
 			free(lnk);
 		} else
-			die(_("do not know what to do with %06o %s '%s'"),
-			    mode, oid_to_hex(oid), path);
+			ret = error(_("do not know what to do with %06o %s '%s'"),
+				    mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
-	if (update_cache)
+	if (!ret && update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
-	return 0;
+	return ret;
 }
 
 static int update_file(struct merge_options *o,
@@ -929,20 +936,22 @@ static int merge_file_1(struct merge_options *o,
 			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
-			int merge_status;
+			int ret = 0, merge_status;
 
 			merge_status = merge_3way(o, &result_buf, one, a, b,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				die(_("Failed to execute internal merge"));
+				ret = error(_("Failed to execute internal merge"));
 
-			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result->oid.hash))
-				die(_("Unable to add %s to database"),
-				    a->path);
+			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
+						    blob_type, result->oid.hash))
+				ret = error(_("Unable to add %s to database"),
+					    a->path);
 
 			free(result_buf.ptr);
+			if (ret)
+				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
 			result->clean = merge_submodule(result->oid.hash,
@@ -1876,11 +1885,10 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			die(_("merging of trees %s and %s failed"),
+			error(_("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
-		else
-			exit(128);
+		return -1;
 	}
 
 	if (unmerged_cache()) {
@@ -2011,7 +2019,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			die(_("merge returned no commit"));
+			return error(_("merge returned no commit"));
 	}
 
 	discard_cache();
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 11/16] am -3: use merge_recursive() directly again

Last October, we had to change this code to run `git merge-recursive` in a child process: git-am wants to print some helpful advice when the merge failed, but the code in question was not prepared to return, it die()d instead.

We are finally at a point when the code *is* prepared to return errors, and can avoid the child process again.

This reverts commit c63d4b2 (am -3: do not let failed merge from completing the error codepath, 2015-10-09), with the necessary changes to adjust for the fact that Git's source code changed in the meantime (such as: using OIDs instead of hashes in the recursive merge, and a removed gender bias).

Note: the code now calls merge_recursive_generic() again. Unlike
merge_trees() and merge_recursive(), this function returns 0 upon success,
as most of Git's functions. Therefore, the error value -1 naturally is
handled correctly, and we do not have to take care of it specifically.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c | 62 +++++++++++++++++++++---------------------------------------
 1 file changed, 22 insertions(+), 40 deletions(-)
Show changes to builtin/am.c +22 −40
diff --git a/builtin/am.c b/builtin/am.c
index b77bf11..cfb79ea 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1579,47 +1579,18 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
 }
 
 /**
- * Do the three-way merge using fake ancestor, their tree constructed
- * from the fake ancestor and the postimage of the patch, and our
- * state.
- */
-static int run_fallback_merge_recursive(const struct am_state *state,
-					unsigned char *orig_tree,
-					unsigned char *our_tree,
-					unsigned char *their_tree)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-
-	cp.git_cmd = 1;
-
-	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(their_tree), linelen(state->msg), state->msg);
-	if (state->quiet)
-		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
-
-	argv_array_push(&cp.args, "merge-recursive");
-	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
-	argv_array_push(&cp.args, "--");
-	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(their_tree));
-
-	status = run_command(&cp) ? (-1) : 0;
-	discard_cache();
-	read_cache();
-	return status;
-}
-
-/**
  * Attempt a threeway merge, using index_path as the temporary index.
  */
 static int fall_back_threeway(const struct am_state *state, const char *index_path)
 {
-	unsigned char orig_tree[GIT_SHA1_RAWSZ], their_tree[GIT_SHA1_RAWSZ],
-		      our_tree[GIT_SHA1_RAWSZ];
+	struct object_id orig_tree, their_tree, our_tree;
+	const struct object_id *bases[1] = { &orig_tree };
+	struct merge_options o;
+	struct commit *result;
+	char *their_tree_name;
 
-	if (get_sha1("HEAD", our_tree) < 0)
-		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
+	if (get_oid("HEAD", &our_tree) < 0)
+		hashcpy(our_tree.hash, EMPTY_TREE_SHA1_BIN);
 
 	if (build_fake_ancestor(state, index_path))
 		return error("could not build fake ancestor");
@@ -1627,7 +1598,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	discard_cache();
 	read_cache_from(index_path);
 
-	if (write_index_as_tree(orig_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(orig_tree.hash, &the_index, index_path, 0, NULL))
 		return error(_("Repository lacks necessary blobs to fall back on 3-way merge."));
 
 	say(state, stdout, _("Using index info to reconstruct a base tree..."));
@@ -1643,7 +1614,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		init_revisions(&rev_info, NULL);
 		rev_info.diffopt.output_format = DIFF_FORMAT_NAME_STATUS;
 		diff_opt_parse(&rev_info.diffopt, &diff_filter_str, 1, rev_info.prefix);
-		add_pending_sha1(&rev_info, "HEAD", our_tree, 0);
+		add_pending_sha1(&rev_info, "HEAD", our_tree.hash, 0);
 		diff_setup_done(&rev_info.diffopt);
 		run_diff_index(&rev_info, 1);
 	}
@@ -1652,7 +1623,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		return error(_("Did you hand edit your patch?\n"
 				"It does not apply to blobs recorded in its index."));
 
-	if (write_index_as_tree(their_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(their_tree.hash, &the_index, index_path, 0, NULL))
 		return error("could not write tree");
 
 	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
@@ -1668,11 +1639,22 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	 * changes.
 	 */
 
-	if (run_fallback_merge_recursive(state, orig_tree, our_tree, their_tree)) {
+	init_merge_options(&o);
+
+	o.branch1 = "HEAD";
+	their_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
+	o.branch2 = their_tree_name;
+
+	if (state->quiet)
+		o.verbosity = 0;
+
+	if (merge_recursive_generic(&o, &our_tree, &their_tree, 1, bases, &result)) {
 		rerere(state->allow_rerere_autoupdate);
+		free(their_tree_name);
 		return error(_("Failed to merge in the changes."));
 	}
 
+	free(their_tree_name);
 	return 0;
 }
 
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 25, 2016, 22:19 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 11/16] am -3: use merge_recursive() directly again

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> Note: the code now calls merge_recursive_generic() again. Unlike
> merge_trees() and merge_recursive(), this function returns 0 upon success,
> as most of Git's functions. Therefore, the error value -1 naturally is
> handled correctly, and we do not have to take care of it specifically.

I've finished reading through up to this point and I'd stop for now.

Some of the patches I didn't look beyond the context presented in the patches, so it is very possible that I missed leaks caused by early returns and things like that, but I didn't see anything glaringly wrong. Looks very promising.

Thanks.
Johannes Schindelin· Jul 26, 2016, 12:30 UTC · re: Junio C Hamano · lore

Re: [PATCH v4 11/16] am -3: use merge_recursive() directly again

Hi Junio,
On Mon, 25 Jul 2016, Junio C Hamano wrote:
Show 9 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Note: the code now calls merge_recursive_generic() again. Unlike
> > merge_trees() and merge_recursive(), this function returns 0 upon success,
> > as most of Git's functions. Therefore, the error value -1 naturally is
> > handled correctly, and we do not have to take care of it specifically.
> 
> I've finished reading through up to this point and I'd stop for
> now.

If you want, I can break out the subsequent patches into a separate series. I just thought that you might want to have them here, as I implemented them in response to the concern you raised in a previous iteration of the same patch series: you pointed out that returning a negative error value still does not let the caller handle the error message, and with the subsequent patches that is now possible, too.

> Some of the patches I didn't look beyond the context presented in
> the patches, so it is very possible that I missed leaks caused by
> early returns and things like that, but I didn't see anything
> glaringly wrong.  Looks very promising.
Thanks.

I did try my best to catch all resource leaks, but I did stare at those patches so often and so long that it is easy for some obvious bug to have slipped by. Therefore, I would appreciate it if you (or somebody as diligent as you) could have a careful look in particular at the "merge-recursive: switch to returning errors instead of dying" patch.

I may have missed something as stupid as an unclosed file handle, after all.

Thank you, Dscho

Junio C Hamano· Jul 26, 2016, 17:12 UTC · re: Johannes Schindelin · lore

Re: [PATCH v4 11/16] am -3: use merge_recursive() directly again

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> If you want, I can break out the subsequent patches into a separate
> series.

I do not think that would help anybody, as we'll have to review them anyway. If some of the the later ones were "oops, this earlier step did an incomplete job and here is a fix-up", then squashing them into the step where such a mistake happens may reduce the review load, but I suspect that is not the case (iow, they are enhancements and the earlier ones can stand on their own).

> I may have missed something as stupid as an unclosed file handle, after
> all.

Yes, reviewing the same change over and over, especially if the change is your own, tends to have diminishing rate of bug yields, which is why they need to be reviewed by fresh set of eyes.

Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 12/16] merge-recursive: flush output buffer before printing error messages

The data structure passed to the recursive merge machinery has a feature where the caller can ask for the output to be buffered into a strbuf, by setting the field 'buffer_output'.

Previously, we simply swallowed the buffered output when showing error messages. With this patch, we show the output first, and only then print the error message.

Currently, the only user of that buffering is merge_recursive() itself, to avoid the progress output to interfere.

In the next patches, we will introduce a new buffer_output mode that forces merge_recursive() to retain the output buffer for further processing by the caller. If the caller asked for that, we will then also write the error messages into the output buffer. This is necessary to give the caller more control not only how to react in case of errors but also control how/if to display the error messages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 116 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 68 insertions(+), 48 deletions(-)
Show changes to merge-recursive.c +68 −48
diff --git a/merge-recursive.c b/merge-recursive.c
index 24b42d6..3ef1e2f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,28 @@
 #include "dir.h"
 #include "submodule.h"
 
+static void flush_output(struct merge_options *o)
+{
+	if (o->obuf.len) {
+		fputs(o->obuf.buf, stdout);
+		strbuf_reset(&o->obuf);
+	}
+}
+
+static int err(struct merge_options *o, const char *err, ...)
+{
+	va_list params;
+
+	va_start(params, err);
+	flush_output(o);
+	strbuf_vaddf(&o->obuf, err, params);
+	error("%s", o->obuf.buf);
+	strbuf_reset(&o->obuf);
+	va_end(params);
+
+	return -1;
+}
+
 static struct tree *shift_tree_object(struct tree *one, struct tree *two,
 				      const char *subtree_shift)
 {
@@ -148,14 +170,6 @@ static int show(struct merge_options *o, int v)
 	return (!o->call_depth && o->verbosity >= v) || o->verbosity >= 5;
 }
 
-static void flush_output(struct merge_options *o)
-{
-	if (o->obuf.len) {
-		fputs(o->obuf.buf, stdout);
-		strbuf_reset(&o->obuf);
-	}
-}
-
 __attribute__((format (printf, 3, 4)))
 static void output(struct merge_options *o, int v, const char *fmt, ...)
 {
@@ -198,7 +212,8 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)
 	}
 }
 
-static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
+static int add_cacheinfo(struct merge_options *o,
+		unsigned int mode, const struct object_id *oid,
 		const char *path, int stage, int refresh, int options)
 {
 	struct cache_entry *ce;
@@ -206,7 +221,7 @@ static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
 			      (refresh ? (CE_MATCH_REFRESH |
 					  CE_MATCH_IGNORE_MISSING) : 0 ));
 	if (!ce)
-		return error(_("addinfo_cache failed for path '%s'"), path);
+		return err(o, _("addinfo_cache failed for path '%s'"), path);
 	return add_cache_entry(ce, options);
 }
 
@@ -267,7 +282,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
 	    cache_tree_update(&the_index, 0) < 0) {
-		error(_("error building trees"));
+		err(o, _("error building trees"));
 		return NULL;
 	}
 
@@ -535,7 +550,8 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages(const char *path, const struct diff_filespec *o,
+static int update_stages(struct merge_options *opt, const char *path,
+			 const struct diff_filespec *o,
 			 const struct diff_filespec *a,
 			 const struct diff_filespec *b)
 {
@@ -554,13 +570,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 		if (remove_file_from_cache(path))
 			return -1;
 	if (o)
-		if (add_cacheinfo(o->mode, &o->oid, path, 1, 0, options))
+		if (add_cacheinfo(opt, o->mode, &o->oid, path, 1, 0, options))
 			return -1;
 	if (a)
-		if (add_cacheinfo(a->mode, &a->oid, path, 2, 0, options))
+		if (add_cacheinfo(opt, a->mode, &a->oid, path, 2, 0, options))
 			return -1;
 	if (b)
-		if (add_cacheinfo(b->mode, &b->oid, path, 3, 0, options))
+		if (add_cacheinfo(opt, b->mode, &b->oid, path, 3, 0, options))
 			return -1;
 	return 0;
 }
@@ -711,8 +727,8 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (status) {
 		if (status == SCLD_EXISTS)
 			/* something else exists */
-			return error(msg, path, _(": perhaps a D/F conflict?"));
-		return error(msg, path, "");
+			return err(o, msg, path, _(": perhaps a D/F conflict?"));
+		return err(o, msg, path, "");
 	}
 
 	/*
@@ -720,7 +736,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	 * tracking it.
 	 */
 	if (would_lose_untracked(path))
-		return error(_("refusing to lose untracked file at '%s'"),
+		return err(o, _("refusing to lose untracked file at '%s'"),
 			     path);
 
 	/* Successful unlink is good.. */
@@ -730,7 +746,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (errno == ENOENT)
 		return 0;
 	/* .. but not some other error (who really cares what?) */
-	return error(msg, path, _(": perhaps a D/F conflict?"));
+	return err(o, msg, path, _(": perhaps a D/F conflict?"));
 }
 
 static int update_file_flags(struct merge_options *o,
@@ -762,9 +778,9 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+			return err(o, _("cannot read object %s '%s'"), oid_to_hex(oid), path);
 		if (type != OBJ_BLOB) {
-			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			ret = err(o, _("blob expected for %s '%s'"), oid_to_hex(oid), path);
 			goto free_buf;
 		}
 		if (S_ISREG(mode)) {
@@ -788,8 +804,8 @@ static int update_file_flags(struct merge_options *o,
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
 			if (fd < 0) {
-				ret = error_errno(_("failed to open '%s'"),
-						  path);
+				ret = err(o, _("failed to open '%s': %s"),
+					  path, strerror(errno));
 				goto free_buf;
 			}
 			write_in_full(fd, buf, size);
@@ -799,17 +815,19 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				ret = error_errno(_("failed to symlink '%s'"), path);
+				ret = err(o, _("failed to symlink '%s': %s"),
+					path, strerror(errno));
 			free(lnk);
 		} else
-			ret = error(_("do not know what to do with %06o %s '%s'"),
-				    mode, oid_to_hex(oid), path);
+			ret = err(o,
+				  _("do not know what to do with %06o %s '%s'"),
+				  mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
 	if (!ret && update_cache)
-		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+		add_cacheinfo(o, mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
 	return ret;
 }
 
@@ -942,12 +960,12 @@ static int merge_file_1(struct merge_options *o,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				ret = error(_("Failed to execute internal merge"));
+				ret = err(o, _("Failed to execute internal merge"));
 
 			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 						    blob_type, result->oid.hash))
-				ret = error(_("Unable to add %s to database"),
-					    a->path);
+				ret = err(o, _("Unable to add %s to database"),
+					  a->path);
 
 			free(result_buf.ptr);
 			if (ret)
@@ -1113,7 +1131,7 @@ static int conflict_rename_delete(struct merge_options *o,
 	if (o->call_depth)
 		return remove_file_from_cache(dest->path);
 	else
-		return update_stages(dest->path, NULL,
+		return update_stages(o, dest->path, NULL,
 				     rename_branch == o->branch1 ? dest : NULL,
 				     rename_branch == o->branch1 ? NULL : dest);
 }
@@ -1171,9 +1189,9 @@ static int handle_file(struct merge_options *o,
 	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
 		; /* fall through, do allow dst_name to be released */
 	else if (stage == 2)
-		ret = update_stages(rename->path, NULL, rename, add);
+		ret = update_stages(o, rename->path, NULL, rename, add);
 	else
-		ret = update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(o, rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
@@ -1566,23 +1584,25 @@ static struct object_id *stage_oid(const struct object_id *oid, unsigned mode)
 	return (is_null_oid(oid) || mode == 0) ? NULL: (struct object_id *)oid;
 }
 
-static int read_oid_strbuf(const struct object_id *oid, struct strbuf *dst)
+static int read_oid_strbuf(struct merge_options *o,
+	const struct object_id *oid, struct strbuf *dst)
 {
 	void *buf;
 	enum object_type type;
 	unsigned long size;
 	buf = read_sha1_file(oid->hash, &type, &size);
 	if (!buf)
-		return error(_("cannot read object %s"), oid_to_hex(oid));
+		return err(o, _("cannot read object %s"), oid_to_hex(oid));
 	if (type != OBJ_BLOB) {
 		free(buf);
-		return error(_("object %s is not a blob"), oid_to_hex(oid));
+		return err(o, _("object %s is not a blob"), oid_to_hex(oid));
 	}
 	strbuf_attach(dst, buf, size, size + 1);
 	return 0;
 }
 
-static int blob_unchanged(const struct object_id *o_oid,
+static int blob_unchanged(struct merge_options *opt,
+			  const struct object_id *o_oid,
 			  unsigned o_mode,
 			  const struct object_id *a_oid,
 			  unsigned a_mode,
@@ -1600,7 +1620,7 @@ static int blob_unchanged(const struct object_id *o_oid,
 		return 0;
 
 	assert(o_oid && a_oid);
-	if (read_oid_strbuf(o_oid, &o) || read_oid_strbuf(a_oid, &a))
+	if (read_oid_strbuf(opt, o_oid, &o) || read_oid_strbuf(opt, a_oid, &a))
 		goto error_return;
 	/*
 	 * Note: binary | is used so that both renormalizations are
@@ -1689,7 +1709,7 @@ static int merge_content(struct merge_options *o,
 		 */
 		path_renamed_outside_HEAD = !path2 || !strcmp(path, path2);
 		if (!path_renamed_outside_HEAD) {
-			add_cacheinfo(mfi.mode, &mfi.oid, path,
+			add_cacheinfo(o, mfi.mode, &mfi.oid, path,
 				      0, (!o->call_depth), 0);
 			return mfi.clean;
 		}
@@ -1702,7 +1722,7 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			if (update_stages(path, &one, &a, &b))
+			if (update_stages(o, path, &one, &a, &b))
 				return -1;
 	}
 
@@ -1712,7 +1732,7 @@ static int merge_content(struct merge_options *o,
 			remove_file_from_cache(path);
 		} else {
 			if (!mfi.clean) {
-				if (update_stages(path, &one, &a, &b))
+				if (update_stages(o, path, &one, &a, &b))
 					return -1;
 			} else {
 				int file_from_stage2 = was_tracked(path);
@@ -1720,7 +1740,7 @@ static int merge_content(struct merge_options *o,
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				if (update_stages(path, NULL,
+				if (update_stages(o, path, NULL,
 						  file_from_stage2 ? &merged : NULL,
 						  file_from_stage2 ? NULL : &merged))
 					return -1;
@@ -1788,8 +1808,8 @@ static int process_entry(struct merge_options *o,
 	} else if (o_oid && (!a_oid || !b_oid)) {
 		/* Case A: Deleted in one */
 		if ((!a_oid && !b_oid) ||
-		    (!b_oid && blob_unchanged(o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
-		    (!a_oid && blob_unchanged(o_oid, o_mode, b_oid, b_mode, normalize, path))) {
+		    (!b_oid && blob_unchanged(o, o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
+		    (!a_oid && blob_unchanged(o, o_oid, o_mode, b_oid, b_mode, normalize, path))) {
 			/* Deleted in both or deleted in one and
 			 * unchanged in the other */
 			if (a_oid)
@@ -1885,7 +1905,7 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			error(_("merging of trees %s and %s failed"),
+			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
 		return -1;
@@ -2019,7 +2039,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			return error(_("merge returned no commit"));
+			return err(o, _("merge returned no commit"));
 	}
 
 	discard_cache();
@@ -2078,7 +2098,7 @@ int merge_recursive_generic(struct merge_options *o,
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
 			if (!(base = get_ref(base_list[i], oid_to_hex(base_list[i]))))
-				return error(_("Could not parse object '%s'"),
+				return err(o, _("Could not parse object '%s'"),
 					oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
 		}
@@ -2092,7 +2112,7 @@ int merge_recursive_generic(struct merge_options *o,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
-		return error(_("Unable to write index."));
+		return err(o, _("Unable to write index."));
 
 	return clean ? 0 : 1;
 }
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we already accumulate the output in a buffer. The idea was to avoid interfering with the progress output that goes to stderr, which is unbuffered, when we write to stdout, which is buffered.

We extend that buffering to allow the caller to handle the output (possibly suppressing it). This will help us when extending the sequencer to do rebase -i's brunt work: it does not want the picks to print anything by default but instead determine itself whether to print the output or not.

Note that we also redirect the error messages into the output buffer
when the caller asked not to flush the output buffer, for two reasons:
1) to retain the correct output order, and 2) to allow the caller to
suppress *all* output.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++++++----
 merge-recursive.h |  2 +-
 2 files changed, 14 insertions(+), 5 deletions(-)
Show changes to 2 files +14 −5

merge-recursive.c, merge-recursive.h

diff --git a/merge-recursive.c b/merge-recursive.c
index 2bc41fe..1746c38 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -25,7 +25,7 @@
 
 static void flush_output(struct merge_options *o)
 {
-	if (o->obuf.len) {
+	if (o->buffer_output < 2 && o->obuf.len) {
 		fputs(o->obuf.buf, stdout);
 		strbuf_reset(&o->obuf);
 	}
@@ -36,10 +36,19 @@ static int err(struct merge_options *o, const char *err, ...)
 	va_list params;
 
 	va_start(params, err);
-	flush_output(o);
+	if (o->buffer_output < 2)
+		flush_output(o);
+	else {
+		strbuf_complete(&o->obuf, '\n');
+		strbuf_addstr(&o->obuf, "error: ");
+	}
 	strbuf_vaddf(&o->obuf, err, params);
-	error("%s", o->obuf.buf);
-	strbuf_reset(&o->obuf);
+	if (o->buffer_output > 1)
+		strbuf_addch(&o->obuf, '\n');
+	else {
+		error("%s", o->obuf.buf);
+		strbuf_reset(&o->obuf);
+	}
 	va_end(params);
 
 	return -1;
diff --git a/merge-recursive.h b/merge-recursive.h
index d415724..340704c 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -13,7 +13,7 @@ struct merge_options {
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
 	const char *subtree_shift;
-	unsigned buffer_output : 1;
+	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 15/16] Ensure that the output buffer is released after calling merge_trees()

The recursive merge machinery accumulates its output in an output buffer, to be flushed at the end of merge_recursive(). At this point, we forgot to release the output buffer.

When calling merge_trees() (i.e. the non-recursive part of the recursive merge) directly, the output buffer is never flushed because the caller may be merge_recursive() which wants to flush the output itself.

For the same reason, merge_trees() cannot release the output buffer: it may still be needed.

Forgetting to release the output buffer did not matter much when running git-checkout, or git-merge-recursive, because we exited after the operation anyway. Ever since cherry-pick learned to pick a commit range, however, this memory leak had the potential of becoming a problem.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 1 +
 merge-recursive.c  | 2 ++
 sequencer.c        | 1 +
 3 files changed, 4 insertions(+)
Show changes to 3 files +4 −0

builtin/checkout.c, merge-recursive.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 07dea3b..8d852d4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -573,6 +573,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
+			strbuf_release(&o.obuf);
 			if (ret)
 				return ret;
 		}
diff --git a/merge-recursive.c b/merge-recursive.c
index 1746c38..dcf1535 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2068,6 +2068,8 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
 	flush_output(o);
+	if (o->buffer_output < 2)
+		strbuf_release(&o->obuf);
 	if (show(o, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       o->needed_rename_limit, 0);
diff --git a/sequencer.c b/sequencer.c
index 286a435..ec50519 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:25 UTC · re: Johannes Schindelin · lore

[PATCH v4 13/16] merge-recursive: write the commit title in one go

In 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we changed the code such that it prints the output in one go, to avoid interfering with the progress output.

Let's make sure that the same holds true when outputting the commit title: previously, we used several printf() statements to stdout and speculated that stdout's buffer is large enough to hold the entire commit title.

Apart from making that speculation unnecessary, we change the code to add the message to the output buffer before flushing for another reason: the next commit will introduce a new level of output buffering, where the caller can request the output not to be flushed, but to be retained for further processing.

This latter feature will be needed when teaching the sequencer to do rebase -i's brunt work: it wants to control the output of the cherry-picks (i.e. recursive merges).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
Show changes to merge-recursive.c +9 −8
diff --git a/merge-recursive.c b/merge-recursive.c
index 3ef1e2f..2bc41fe 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -191,25 +191,26 @@ static void output(struct merge_options *o, int v, const char *fmt, ...)
 
 static void output_commit_title(struct merge_options *o, struct commit *commit)
 {
-	int i;
-	flush_output(o);
-	for (i = o->call_depth; i--;)
-		fputs("  ", stdout);
+	strbuf_addchars(&o->obuf, ' ', o->call_depth * 2);
 	if (commit->util)
-		printf("virtual %s\n", merge_remote_util(commit)->name);
+		strbuf_addf(&o->obuf, "virtual %s\n",
+			merge_remote_util(commit)->name);
 	else {
-		printf("%s ", find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV));
+		strbuf_addf(&o->obuf, "%s ",
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV));
 		if (parse_commit(commit) != 0)
-			printf(_("(bad commit)\n"));
+			strbuf_addf(&o->obuf, _("(bad commit)\n"));
 		else {
 			const char *title;
 			const char *msg = get_commit_buffer(commit, NULL);
 			int len = find_commit_subject(msg, &title);
 			if (len)
-				printf("%.*s\n", len, title);
+				strbuf_addf(&o->obuf, "%.*s\n", len, title);
 			unuse_commit_buffer(commit, msg);
 		}
 	}
+	flush_output(o);
 }
 
 static int add_cacheinfo(struct merge_options *o,
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 22, 2016, 12:26 UTC · re: Johannes Schindelin · lore

[PATCH v4 16/16] merge-recursive: flush output buffer even when erroring out

Ever since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we had a problem: When the merge failed in a fatal way, all regular output was swallowed because we called die() and did not get a chance to drain the output buffers.

To fix this, several modifications were necessary:
- we needed to stop die()ing, to give callers a chance to do something
  when an error occurred (in this case, flush the output buffers),
- we needed to delay printing the error message so that the caller can
  print the buffered output before that, and
- we needed to make sure that the output buffers are flushed even when
  the return value indicates an error.

The first two changes were introduced through earlier commits in this patch series, and this commit addresses the third one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to merge-recursive.c +1 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index dcf1535..501cfb5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2059,6 +2059,7 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	flush_output(o);
 	if (clean < 0)
 		return clean;
 
@@ -2067,7 +2068,6 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(o);
 	if (o->buffer_output < 2)
 		strbuf_release(&o->obuf);
 	if (show(o, 2))
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:05 UTC · re: Johannes Schindelin · lore

[PATCH v5 00/16] Use merge_recursive() directly in the builtin am

This is the fifth iteration of the long-awaited re-roll of the attempt to avoid spawning merge-recursive from the builtin am and use merge_recursive() directly instead.

The *real* reason for the reroll is that I need a libified recursive merge to accelerate the interactive rebase by teaching the sequencer to do rebase -i's grunt work. Coming with a very nice 3x-5x speedup of `rebase -i`.

In this endeavor, we need to be extra careful to retain backwards compatibility. The test script t6022-merge-rename.sh, for example, verifies that `git pull` exits with status 128 in case of a fatal error. To that end, we need to make sure that fatal errors are handled by existing (builtin) users via exit(128) (or die(), which calls exit(128) at the end). New users (such as a builtin helper doing rebase -i's grunt work) may want to print some helpful advice what happened and how to get out of this mess before erroring out.

The changes relative to the fourth iteration of this patch series:
- the first patch which introduces a tests case to t5520 was prettified:
  - it uses test_seq now,
  - it avoids `printf` when `echo` does the job, too,
  - it adds a missing empty line between test cases, and
  - it clarifies why we need to check out with force.
- the change that would have made the bug report about a too-small
  buffer in imap-send was reverted, because it did more than was claimed
  by the commit message.
- the "Let's teach them manners" part of one commit message was replaced
  with a less flippant description.
- the comment before merge_recursive() that claims that the return value
  is ignored now clarifies that it is ignored unless it indicates an
  error.
- during the rebase to `master`, a trivial merge conflict with the
  `jc/renormalize-merge-kill-safer-crlf` branch was resolved.

This patch series touches rather important code. Now that I addressed concerns such as prettifying some test code, I would appreciate thorough reviews with a focus on the critical parts of the code, those that could result in regressions.

Johannes Schindelin (16):
  t5520: verify that `pull --rebase` shows the helpful advice when
    failing
  Report bugs consistently
  Avoid translating bug messages
  merge-recursive: clarify code in was_tracked()
  Prepare the builtins for a libified merge_recursive()
  merge_recursive: abort properly upon errors
  merge-recursive: avoid returning a wholesale struct
  merge-recursive: allow write_tree_from_memory() to error out
  merge-recursive: handle return values indicating errors
  merge-recursive: switch to returning errors instead of dying
  am -3: use merge_recursive() directly again
  merge-recursive: flush output buffer before printing error messages
  merge-recursive: write the commit title in one go
  merge-recursive: offer an option to retain the output in 'obuf'
  Ensure that the output buffer is released after calling merge_trees()
  merge-recursive: flush output buffer even when erroring out
 builtin/am.c           |  62 ++----
 builtin/checkout.c     |   5 +-
 builtin/ls-files.c     |   3 +-
 builtin/merge.c        |   2 +
 builtin/update-index.c |   2 +-
 grep.c                 |   8 +-
 imap-send.c            |   2 +-
 merge-recursive.c      | 578 +++++++++++++++++++++++++++++--------------------
 merge-recursive.h      |   2 +-
 sequencer.c            |   5 +
 sha1_file.c            |   4 +-
 t/t5520-pull.sh        |  32 +++
 trailer.c              |   2 +-
 transport.c            |   2 +-
 wt-status.c            |   4 +-
 15 files changed, 418 insertions(+), 295 deletions(-)
Published-As: https://github.com/dscho/git/releases/tag/am-3-merge-recursive-direct-v5
Interdiff vs v4:
 diff --git a/imap-send.c b/imap-send.c
 index 67d67f8..0f5f476 100644
 --- a/imap-send.c
 +++ b/imap-send.c
 @@ -506,12 +506,12 @@ static char *next_arg(char **s)
  
  static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
  {
 -	int ret = -1;
 +	int ret;
  	va_list va;
  
  	va_start(va, fmt);
  	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
 -		die("BUG: buffer too small (%d < %d)", ret, blen);
 +		die("BUG: buffer too small. Please report a bug.");
  	va_end(va);
  	return ret;
  }
 diff --git a/merge-recursive.c b/merge-recursive.c
 index a3d12e6..66e93e0 100644
 --- a/merge-recursive.c
 +++ b/merge-recursive.c
 @@ -2041,9 +2041,10 @@ int merge_recursive(struct merge_options *o,
  		/*
  		 * When the merge fails, the result contains files
  		 * with conflict markers. The cleanness flag is
 -		 * ignored, it was never actually used, as result of
 -		 * merge_trees has always overwritten it: the committed
 -		 * "conflicts" were already resolved.
 +		 * ignored (unless indicating an error), it was never
 +		 * actually used, as result of merge_trees has always
 +		 * overwritten it: the committed "conflicts" were
 +		 * already resolved.
  		 */
  		discard_cache();
  		saved_b1 = o->branch1;
 diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
 index d289056..6ad37b5 100755
 --- a/t/t5520-pull.sh
 +++ b/t/t5520-pull.sh
 @@ -258,20 +258,21 @@ test_expect_success '--rebase' '
  test_expect_success '--rebase with conflicts shows advice' '
  	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
  	git checkout -b seq &&
 -	printf "1\\n2\\n3\\n4\\n5\\n" >seq.txt &&
 +	test_seq 5 >seq.txt &&
  	git add seq.txt &&
  	test_tick &&
  	git commit -m "Add seq.txt" &&
 -	printf "6\\n" >>seq.txt &&
 +	echo 6 >>seq.txt &&
  	test_tick &&
  	git commit -m "Append to seq.txt" seq.txt &&
  	git checkout -b with-conflicts HEAD^ &&
 -	printf "conflicting\\n" >>seq.txt &&
 +	echo conflicting >>seq.txt &&
  	test_tick &&
  	git commit -m "Create conflict" seq.txt &&
  	test_must_fail git pull --rebase . seq 2>err >out &&
  	grep "When you have resolved this problem" out
  '
 +
  test_expect_success 'failed --rebase shows advice' '
  	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
  	git checkout -b diverging &&
 @@ -279,6 +280,7 @@ test_expect_success 'failed --rebase shows advice' '
  	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
  	git update-index --cacheinfo 0644 $sha1 file &&
  	git commit -m v1-with-cr &&
 +	# force checkout because `git reset --hard` will not leave clean `file`
  	git checkout -f -b fails-to-rebase HEAD^ &&
  	test_commit v2-without-cr file "2" file2-lf &&
  	test_must_fail git pull --rebase . diverging 2>err >out &&
-- 
2.9.0.281.g286a8d9

base-commit: 8c6d1f9807c67532e7fb545a944b064faff0f70b
Johannes Schindelin· Jul 26, 2016, 16:05 UTC · re: Johannes Schindelin · lore

[PATCH v5 01/16] t5520: verify that `pull --rebase` shows the helpful advice when failing

It was noticed by Brendan Forster last October that the builtin `git am` regressed on that. Our hot fix reverted to spawning the recursive merge instead of using it as a library function.

As we are about to revert that hot fix, after making the recursive merge a true library function (i.e. a function that does not die() in case of "normal" errors), let's add a test that verifies that we do not regress on the same problem which made the hot fix necessary in the first place.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t5520-pull.sh | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
Show changes to t/t5520-pull.sh +32 −0
diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
index 37ebbcf..6ad37b5 100755
--- a/t/t5520-pull.sh
+++ b/t/t5520-pull.sh
@@ -255,6 +255,38 @@ test_expect_success '--rebase' '
 	test new = "$(git show HEAD:file2)"
 '
 
+test_expect_success '--rebase with conflicts shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b seq &&
+	test_seq 5 >seq.txt &&
+	git add seq.txt &&
+	test_tick &&
+	git commit -m "Add seq.txt" &&
+	echo 6 >>seq.txt &&
+	test_tick &&
+	git commit -m "Append to seq.txt" seq.txt &&
+	git checkout -b with-conflicts HEAD^ &&
+	echo conflicting >>seq.txt &&
+	test_tick &&
+	git commit -m "Create conflict" seq.txt &&
+	test_must_fail git pull --rebase . seq 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
+test_expect_success 'failed --rebase shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b diverging &&
+	test_commit attributes .gitattributes "* text=auto" attrs &&
+	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
+	git update-index --cacheinfo 0644 $sha1 file &&
+	git commit -m v1-with-cr &&
+	# force checkout because `git reset --hard` will not leave clean `file`
+	git checkout -f -b fails-to-rebase HEAD^ &&
+	test_commit v2-without-cr file "2" file2-lf &&
+	test_must_fail git pull --rebase . diverging 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
 test_expect_success '--rebase fails with multiple branches' '
 	git reset --hard before-rebase &&
 	test_must_fail git pull --rebase . copy master 2>err &&
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:05 UTC · re: Johannes Schindelin · lore

[PATCH v5 02/16] Report bugs consistently

The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:".

As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/ls-files.c     |  3 ++-
 builtin/update-index.c |  2 +-
 grep.c                 |  8 ++++----
 imap-send.c            |  2 +-
 merge-recursive.c      | 15 +++++++--------
 sha1_file.c            |  4 ++--
 trailer.c              |  2 +-
 transport.c            |  2 +-
 wt-status.c            |  4 ++--
 9 files changed, 21 insertions(+), 21 deletions(-)
Show changes to 9 files +21 −21

builtin/ls-files.c, builtin/update-index.c, grep.c, imap-send.c, merge-recursive.c, sha1_file.c, trailer.c, transport.c, wt-status.c

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f02e3d2..00ea91a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
 				 */
 				pos = cache_name_pos(ent->name, ent->len);
 				if (0 <= pos)
-					die("bug in show-killed-files");
+					die("BUG: killed-file %.*s not found",
+						ent->len, ent->name);
 				pos = -pos - 1;
 				while (pos < active_nr &&
 				       ce_stage(active_cache[pos]))
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 6cdfd5f..ba04b19 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1146,7 +1146,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		report(_("Untracked cache enabled for '%s'"), get_git_work_tree());
 		break;
 	default:
-		die("Bug: bad untracked_cache value: %d", untracked_cache);
+		die("BUG: bad untracked_cache value: %d", untracked_cache);
 	}
 
 	if (active_cache_changed) {
diff --git a/grep.c b/grep.c
index 394c856..22cbb73 100644
--- a/grep.c
+++ b/grep.c
@@ -693,10 +693,10 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 	for (p = opt->header_list; p; p = p->next) {
 		if (p->token != GREP_PATTERN_HEAD)
-			die("bug: a non-header pattern in grep header list.");
+			die("BUG: a non-header pattern in grep header list.");
 		if (p->field < GREP_HEADER_FIELD_MIN ||
 		    GREP_HEADER_FIELD_MAX <= p->field)
-			die("bug: unknown header field %d", p->field);
+			die("BUG: unknown header field %d", p->field);
 		compile_regexp(p, opt);
 	}
 
@@ -709,7 +709,7 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 		h = compile_pattern_atom(&pp);
 		if (!h || pp != p->next)
-			die("bug: malformed header expr");
+			die("BUG: malformed header expr");
 		if (!header_group[p->field]) {
 			header_group[p->field] = h;
 			continue;
@@ -1514,7 +1514,7 @@ static int grep_source_1(struct grep_opt *opt, struct grep_source *gs, int colle
 		case GREP_BINARY_TEXT:
 			break;
 		default:
-			die("bug: unknown binary handling mode");
+			die("BUG: unknown binary handling mode");
 		}
 	}
 
diff --git a/imap-send.c b/imap-send.c
index db0fafe..0f5f476 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -511,7 +511,7 @@ static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 
 	va_start(va, fmt);
 	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
-		die("Fatal: buffer too small. Please report a bug.");
+		die("BUG: buffer too small. Please report a bug.");
 	va_end(va);
 	return ret;
 }
diff --git a/merge-recursive.c b/merge-recursive.c
index a4a1195..4338b73 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -268,7 +268,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
-		die("Bug in merge-recursive.c");
+		die("BUG: unmerged index entries in merge-recursive.c");
 	}
 
 	if (!active_cache_tree)
@@ -966,9 +966,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
-		} else {
-			die(_("unsupported object type in the tree"));
-		}
+		} else
+			die(_("BUG: unsupported object type in the tree"));
 	}
 
 	return result;
@@ -1354,7 +1353,7 @@ static int process_renames(struct merge_options *o,
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1_src != ren2_src");
+				die("BUG: ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
@@ -1388,7 +1387,7 @@ static int process_renames(struct merge_options *o,
 			ren2 = lookup->util;
 			ren2_dst = ren2->pair->two->path;
 			if (strcmp(ren1_dst, ren2_dst) != 0)
-				die("ren1_dst != ren2_dst");
+				die("BUG: ren1_dst != ren2_dst");
 
 			clean_merge = 0;
 			ren2->processed = 1;
@@ -1812,7 +1811,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("Fatal merge failure, shouldn't happen."));
+		die(_("BUG: fatal merge failure, shouldn't happen."));
 
 	return clean_merge;
 }
@@ -1870,7 +1869,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("Unprocessed path??? %s"),
+				die(_("BUG: unprocessed path??? %s"),
 				    entries->items[i].string);
 		}
 
diff --git a/sha1_file.c b/sha1_file.c
index d5e1121..5085fe0 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -795,7 +795,7 @@ void close_all_packs(void)
 
 	for (p = packed_git; p; p = p->next)
 		if (p->do_not_close)
-			die("BUG! Want to close pack marked 'do-not-close'");
+			die("BUG: want to close pack marked 'do-not-close'");
 		else
 			close_pack(p);
 }
@@ -2330,7 +2330,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 	case OBJ_OFS_DELTA:
 	case OBJ_REF_DELTA:
 		if (data)
-			die("BUG in unpack_entry: left loop at a valid delta");
+			die("BUG: unpack_entry: left loop at a valid delta");
 		break;
 	case OBJ_COMMIT:
 	case OBJ_TREE:
diff --git a/trailer.c b/trailer.c
index 8e48a5c..c6ea9ac 100644
--- a/trailer.c
+++ b/trailer.c
@@ -562,7 +562,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
-		die("internal bug in trailer.c");
+		die("BUG: trailer.c: unhandled type %d", type);
 	}
 	return 0;
 }
diff --git a/transport.c b/transport.c
index b233e3e..04d9454 100644
--- a/transport.c
+++ b/transport.c
@@ -566,7 +566,7 @@ void transport_take_over(struct transport *transport,
 	struct git_transport_data *data;
 
 	if (!transport->smart_options)
-		die("Bug detected: Taking over transport requires non-NULL "
+		die("BUG: taking over transport requires non-NULL "
 		    "smart_options field.");
 
 	data = xcalloc(1, sizeof(*data));
diff --git a/wt-status.c b/wt-status.c
index 19cbc39..f8ae0c2 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die("bug: unhandled unmerged status %x", stagemask);
+		die("BUG: unhandled unmerged status %x", stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die("bug: unhandled diff status %c", status);
+		die("BUG: unhandled diff status %c", status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:05 UTC · re: Johannes Schindelin · lore

[PATCH v5 03/16] Avoid translating bug messages

While working on the patch series that avoids die()ing in recursive merges, the issue came up that bug reports (i.e. die("BUG: ...") constructs) should never be translated, as the target audience is the Git developer community, not necessarily the current user, and hence a translated message would make it *harder* to address the problem.

So let's stop translating the obvious ones. As it is really, really outside the purview of this patch series to see whether there are more die() statements that report bugs and are currently translated, that task is left for another day and patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +3 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 4338b73..1b6db87 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -967,7 +967,7 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
 		} else
-			die(_("BUG: unsupported object type in the tree"));
+			die("BUG: unsupported object type in the tree");
 	}
 
 	return result;
@@ -1811,7 +1811,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("BUG: fatal merge failure, shouldn't happen."));
+		die("BUG: fatal merge failure, shouldn't happen.");
 
 	return clean_merge;
 }
@@ -1869,7 +1869,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("BUG: unprocessed path??? %s"),
+				die("BUG: unprocessed path??? %s",
 				    entries->items[i].string);
 		}
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:05 UTC · re: Johannes Schindelin · lore

[PATCH v5 04/16] merge-recursive: clarify code in was_tracked()

It can be puzzling to see that was_tracked() asks to get an index entry by name, but does not take a negative return value for an answer.

The reason we have to do this is that cache_name_pos() only looks for entries in stage 0, even if nobody asked for any stage in particular.

Let's rewrite the logic a little bit, to handle the easy case early: if cache_name_pos() returned a non-negative position, we know it is a match, and we do not even have to compare the name again (cache_name_pos() did that for us already). We can say right away: yes, this file was tracked.

Only if there was no exact match do we need to look harder for any matching entry in stage 2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)
Show changes to merge-recursive.c +14 −16
diff --git a/merge-recursive.c b/merge-recursive.c
index 1b6db87..3a652b7 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -667,23 +667,21 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
-	if (pos < 0)
-		pos = -1 - pos;
-	while (pos < active_nr &&
-	       !strcmp(path, active_cache[pos]->name)) {
-		/*
-		 * If stage #0, it is definitely tracked.
-		 * If it has stage #2 then it was tracked
-		 * before this merge started.  All other
-		 * cases the path was not tracked.
-		 */
-		switch (ce_stage(active_cache[pos])) {
-		case 0:
-		case 2:
+	if (0 <= pos)
+		/* we have been tracking this path */
+		return 1;
+
+	/*
+	 * Look for an unmerged entry for the path,
+	 * specifically stage #2, which would indicate
+	 * that "our" side before the merge started
+	 * had the path tracked (and resulted in a conflict).
+	 */
+	for (pos = -1 - pos;
+	     pos < active_nr && !strcmp(path, active_cache[pos]->name);
+	     pos++)
+		if (ce_stage(active_cache[pos]) == 2)
 			return 1;
-		}
-		pos++;
-	}
 	return 0;
 }
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 05/16] Prepare the builtins for a libified merge_recursive()

Previously, callers of merge_trees() or merge_recursive() expected that code to die() with an error message. This used to be okay because we called those commands from scripts, and had a chance to print out a message in case the command failed fatally (read: with exit code 128).

As scripting incurs its own set of problems (portability, speed, idiosynchracies of different shells, limited data structures leading to inefficient code), we are converting more and more of these scripts into builtins, using library functions directly.

We already tried to use merge_recursive() directly in the builtin git-am, for example. Unfortunately, we had to roll it back temporarily because some of the code in merge-recursive.c still deemed it okay to call die(), when the builtin am code really wanted to print out a useful advice after the merge failed fatally. In the next commits, we want to fix that.

The code touched by this commit expected merge_trees() to die() with some useful message when there is an error condition, but merge_trees() is going to be improved by converting all die() calls to return error() instead (i.e. return value -1 after printing out the message as before), so that the caller can react more flexibly.

This is a step to prepare for the version of merge_trees() that no longer dies, even if we just imitate the previous behavior by calling exit(128): this is what callers of e.g. `git merge` have come to expect.

Note that the callers of the sequencer (revert and cherry-pick) already fail fast even for the return value -1; The only difference is that they now get a chance to say "<command> failed".

A caller of merge_trees() might want handle error messages themselves (or even suppress them). As this patch is already complex enough, we leave that change for a later patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 4 +++-
 builtin/merge.c    | 2 ++
 sequencer.c        | 4 ++++
 3 files changed, 9 insertions(+), 1 deletion(-)
Show changes to 3 files +9 −1

builtin/checkout.c, builtin/merge.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 27c1a05..07dea3b 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			o.ancestor = old->name;
 			o.branch1 = new->name;
 			o.branch2 = "local";
-			merge_trees(&o, new->commit->tree, work,
+			ret = merge_trees(&o, new->commit->tree, work,
 				old->commit->tree, &result);
+			if (ret < 0)
+				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
 			if (ret)
diff --git a/builtin/merge.c b/builtin/merge.c
index 19b3bc2..148a9a5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -673,6 +673,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
+		if (clean < 0)
+			exit(128);
 		if (active_cache_changed &&
 		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 			die (_("unable to write %s"), get_index_file());
diff --git a/sequencer.c b/sequencer.c
index cdfac82..286a435 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (clean < 0)
+		return clean;
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
@@ -559,6 +561,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
+		if (res < 0)
+			return res;
 		write_message(&msgbuf, git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 06/16] merge_recursive: abort properly upon errors

There are a couple of places where return values never indicated errors before, as wie simply died instead of returning.

But now negative return values mean that there was an error and we have to abort the operation. Let's do exactly that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
Show changes to merge-recursive.c +12 −5
diff --git a/merge-recursive.c b/merge-recursive.c
index 3a652b7..58ced25 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1949,17 +1949,19 @@ int merge_recursive(struct merge_options *o,
 		/*
 		 * When the merge fails, the result contains files
 		 * with conflict markers. The cleanness flag is
-		 * ignored, it was never actually used, as result of
-		 * merge_trees has always overwritten it: the committed
-		 * "conflicts" were already resolved.
+		 * ignored (unless indicating an error), it was never
+		 * actually used, as result of merge_trees has always
+		 * overwritten it: the committed "conflicts" were
+		 * already resolved.
 		 */
 		discard_cache();
 		saved_b1 = o->branch1;
 		saved_b2 = o->branch2;
 		o->branch1 = "Temporary merge branch 1";
 		o->branch2 = "Temporary merge branch 2";
-		merge_recursive(o, merged_common_ancestors, iter->item,
-				NULL, &merged_common_ancestors);
+		if (merge_recursive(o, merged_common_ancestors, iter->item,
+				    NULL, &merged_common_ancestors) < 0)
+			return -1;
 		o->branch1 = saved_b1;
 		o->branch2 = saved_b2;
 		o->call_depth--;
@@ -1975,6 +1977,8 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	if (clean < 0)
+		return clean;
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
@@ -2031,6 +2035,9 @@ int merge_recursive_generic(struct merge_options *o,
 	hold_locked_index(lock, 1);
 	clean = merge_recursive(o, head_commit, next_commit, ca,
 			result);
+	if (clean < 0)
+		return clean;
+
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 		return error(_("Unable to write index."));
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 07/16] merge-recursive: avoid returning a wholesale struct

It is technically allowed, as per C89, for functions' return type to be complete structs (i.e. *not* just pointers to structs).

However, it was just an oversight of this developer when converting Python code to C code in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which introduced such a return type.

Besides, by converting this construct to pass in the struct, we can now start returning a value that can indicate errors in future patches. This will help the current effort to libify merge-recursive.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 106 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 56 insertions(+), 50 deletions(-)
Show changes to merge-recursive.c +56 −50
diff --git a/merge-recursive.c b/merge-recursive.c
index 58ced25..2be1e17 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -894,47 +894,47 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file_1(struct merge_options *o,
+static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
 					   const struct diff_filespec *b,
 					   const char *branch1,
-					   const char *branch2)
+					   const char *branch2,
+					   struct merge_file_info *result)
 {
-	struct merge_file_info result;
-	result.merge = 0;
-	result.clean = 1;
+	result->merge = 0;
+	result->clean = 1;
 
 	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
-		result.clean = 0;
+		result->clean = 0;
 		if (S_ISREG(a->mode)) {
-			result.mode = a->mode;
-			oidcpy(&result.oid, &a->oid);
+			result->mode = a->mode;
+			oidcpy(&result->oid, &a->oid);
 		} else {
-			result.mode = b->mode;
-			oidcpy(&result.oid, &b->oid);
+			result->mode = b->mode;
+			oidcpy(&result->oid, &b->oid);
 		}
 	} else {
 		if (!oid_eq(&a->oid, &one->oid) && !oid_eq(&b->oid, &one->oid))
-			result.merge = 1;
+			result->merge = 1;
 
 		/*
 		 * Merge modes
 		 */
 		if (a->mode == b->mode || a->mode == one->mode)
-			result.mode = b->mode;
+			result->mode = b->mode;
 		else {
-			result.mode = a->mode;
+			result->mode = a->mode;
 			if (b->mode != one->mode) {
-				result.clean = 0;
-				result.merge = 1;
+				result->clean = 0;
+				result->merge = 1;
 			}
 		}
 
 		if (oid_eq(&a->oid, &b->oid) || oid_eq(&a->oid, &one->oid))
-			oidcpy(&result.oid, &b->oid);
+			oidcpy(&result->oid, &b->oid);
 		else if (oid_eq(&b->oid, &one->oid))
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
 			int merge_status;
@@ -946,64 +946,66 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 				die(_("Failed to execute internal merge"));
 
 			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result.oid.hash))
+					    blob_type, result->oid.hash))
 				die(_("Unable to add %s to database"),
 				    a->path);
 
 			free(result_buf.ptr);
-			result.clean = (merge_status == 0);
+			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result.clean = merge_submodule(result.oid.hash,
+			result->clean = merge_submodule(result->oid.hash,
 						       one->path,
 						       one->oid.hash,
 						       a->oid.hash,
 						       b->oid.hash,
 						       !o->call_depth);
 		} else if (S_ISLNK(a->mode)) {
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 
 			if (!oid_eq(&a->oid, &b->oid))
-				result.clean = 0;
+				result->clean = 0;
 		} else
 			die("BUG: unsupported object type in the tree");
 	}
 
-	return result;
+	return 0;
 }
 
-static struct merge_file_info
-merge_file_special_markers(struct merge_options *o,
+static int merge_file_special_markers(struct merge_options *o,
 			   const struct diff_filespec *one,
 			   const struct diff_filespec *a,
 			   const struct diff_filespec *b,
 			   const char *branch1,
 			   const char *filename1,
 			   const char *branch2,
-			   const char *filename2)
+			   const char *filename2,
+			   struct merge_file_info *mfi)
 {
 	char *side1 = NULL;
 	char *side2 = NULL;
-	struct merge_file_info mfi;
+	int ret;
 
 	if (filename1)
 		side1 = xstrfmt("%s:%s", branch1, filename1);
 	if (filename2)
 		side2 = xstrfmt("%s:%s", branch2, filename2);
 
-	mfi = merge_file_1(o, one, a, b,
-			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	ret = merge_file_1(o, one, a, b,
+			   side1 ? side1 : branch1,
+			   side2 ? side2 : branch2, mfi);
 	free(side1);
 	free(side2);
-	return mfi;
+	return ret;
 }
 
-static struct merge_file_info merge_file_one(struct merge_options *o,
+static int merge_file_one(struct merge_options *o,
 					 const char *path,
 					 const struct object_id *o_oid, int o_mode,
 					 const struct object_id *a_oid, int a_mode,
 					 const struct object_id *b_oid, int b_mode,
 					 const char *branch1,
-					 const char *branch2)
+					 const char *branch2,
+					 struct merge_file_info *mfi)
 {
 	struct diff_filespec one, a, b;
 
@@ -1014,7 +1016,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
 	a.mode = a_mode;
 	oidcpy(&b.oid, b_oid);
 	b.mode = b_mode;
-	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
 static void handle_change_delete(struct merge_options *o,
@@ -1187,11 +1189,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct merge_file_info mfi;
 		struct diff_filespec other;
 		struct diff_filespec *add;
-		mfi = merge_file_one(o, one->path,
+		if (merge_file_one(o, one->path,
 				 &one->oid, one->mode,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
-				 ci->branch1, ci->branch2);
+				 ci->branch1, ci->branch2, &mfi))
+			return;
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1245,12 +1248,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
-					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path);
-	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
-					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path);
+	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
+				       o->branch1, c1->path,
+				       o->branch2, ci->ren1_other.path, &mfi_c1) ||
+	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
+				       o->branch1, ci->ren2_other.path,
+				       o->branch2, c2->path, &mfi_c2))
+		return;
 
 	if (o->call_depth) {
 		/*
@@ -1473,12 +1477,13 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					mfi = merge_file_one(o, ren1_dst, &null_oid, 0,
-							 &ren1->pair->two->oid,
-							 ren1->pair->two->mode,
-							 &dst_other.oid,
-							 dst_other.mode,
-							 branch1, branch2);
+					if (merge_file_one(o, ren1_dst, &null_oid, 0,
+							   &ren1->pair->two->oid,
+							   ren1->pair->two->mode,
+							   &dst_other.oid,
+							   dst_other.mode,
+							   branch1, branch2, &mfi))
+						return -1;
 					output(o, 1, _("Adding merged %s"), ren1_dst);
 					update_file(o, 0, &mfi.oid,
 						    mfi.mode, ren1_dst);
@@ -1636,9 +1641,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_special_markers(o, &one, &a, &b,
-					 o->branch1, path1,
-					 o->branch2, path2);
+	if (merge_file_special_markers(o, &one, &a, &b,
+				       o->branch1, path1,
+				       o->branch2, path2, &mfi))
+		return -1;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    oid_eq(&mfi.oid, a_oid) && mfi.mode == a_mode) {
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 08/16] merge-recursive: allow write_tree_from_memory() to error out

It is possible that a tree cannot be written (think: disk full). We will want to give the caller a chance to clean up instead of letting the program die() in such a case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +2 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index 2be1e17..1f86338 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1888,8 +1888,8 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	if (o->call_depth)
-		*result = write_tree_from_memory(o);
+	if (o->call_depth && !(*result = write_tree_from_memory(o)))
+		return -1;
 
 	return clean;
 }
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 09/16] merge-recursive: handle return values indicating errors

We are about to libify the recursive merge machinery, where we only die() in case of a bug or memory contention. To that end, we must heed negative return values as indicating errors.

This requires our functions to be careful to pass through error conditions in call chains, and for quite a few functions this means that they have to return values to begin with.

The next step will be to convert the places where we currently die() to return negative values (read: -1) instead.

Note that we ignore errors reported by make_room_for_path(), consistent with the previous behavior (update_file_flags() used the return value of make_room_for_path() only to indicate an early return, but not a fatal error): if the error is really a fatal error, we will notice later; If not, it was not that serious a problem to begin with. (Witnesses in favor of this reasoning are t4151-am-abort and t7610-mergetool, which would start failing if we stopped on errors reported by make_room_for_path()).

Also note: while this patch makes the code slightly less readable in update_file_flags() (we introduce a new "goto free_buf;" instead of an explicit "free(buf); return;"), it is a preparatory change for the next patch where we will convert all of the die() calls in the same function to go through the free_buf return path instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 252 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 150 insertions(+), 102 deletions(-)
Show changes to merge-recursive.c +150 −102
diff --git a/merge-recursive.c b/merge-recursive.c
index 1f86338..6beb1e4 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -742,12 +742,12 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	return error(msg, path, _(": perhaps a D/F conflict?"));
 }
 
-static void update_file_flags(struct merge_options *o,
-			      const struct object_id *oid,
-			      unsigned mode,
-			      const char *path,
-			      int update_cache,
-			      int update_wd)
+static int update_file_flags(struct merge_options *o,
+			     const struct object_id *oid,
+			     unsigned mode,
+			     const char *path,
+			     int update_cache,
+			     int update_wd)
 {
 	if (o->call_depth)
 		update_wd = 0;
@@ -783,8 +783,7 @@ static void update_file_flags(struct merge_options *o,
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
-			free(buf);
-			goto update_index;
+			goto free_buf;
 		}
 		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
 			int fd;
@@ -807,20 +806,22 @@ static void update_file_flags(struct merge_options *o,
 		} else
 			die(_("do not know what to do with %06o %s '%s'"),
 			    mode, oid_to_hex(oid), path);
+ free_buf:
 		free(buf);
 	}
  update_index:
 	if (update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+	return 0;
 }
 
-static void update_file(struct merge_options *o,
-			int clean,
-			const struct object_id *oid,
-			unsigned mode,
-			const char *path)
+static int update_file(struct merge_options *o,
+		       int clean,
+		       const struct object_id *oid,
+		       unsigned mode,
+		       const char *path)
 {
-	update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
+	return update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1019,7 +1020,7 @@ static int merge_file_one(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
-static void handle_change_delete(struct merge_options *o,
+static int handle_change_delete(struct merge_options *o,
 				 const char *path,
 				 const struct object_id *o_oid, int o_mode,
 				 const struct object_id *a_oid, int a_mode,
@@ -1027,6 +1028,7 @@ static void handle_change_delete(struct merge_options *o,
 				 const char *change, const char *change_past)
 {
 	char *renamed = NULL;
+	int ret = 0;
 	if (dir_in_way(path, !o->call_depth)) {
 		renamed = unique_path(o, path, a_oid ? o->branch1 : o->branch2);
 	}
@@ -1037,21 +1039,23 @@ static void handle_change_delete(struct merge_options *o,
 		 * correct; since there is no true "middle point" between
 		 * them, simply reuse the base version for virtual merge base.
 		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_oid, o_mode, renamed ? renamed : path);
+		ret = remove_file_from_cache(path);
+		if (!ret)
+			ret = update_file(o, 0, o_oid, o_mode,
+					  renamed ? renamed : path);
 	} else if (!a_oid) {
 		if (!renamed) {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path);
-			update_file(o, 0, b_oid, b_mode, path);
+			ret = update_file(o, 0, b_oid, b_mode, path);
 		} else {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path, renamed);
-			update_file(o, 0, b_oid, b_mode, renamed);
+			ret = update_file(o, 0, b_oid, b_mode, renamed);
 		}
 	} else {
 		if (!renamed) {
@@ -1064,7 +1068,7 @@ static void handle_change_delete(struct merge_options *o,
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch2, change_past,
 			       o->branch1, o->branch1, path, renamed);
-			update_file(o, 0, a_oid, a_mode, renamed);
+			ret = update_file(o, 0, a_oid, a_mode, renamed);
 		}
 		/*
 		 * No need to call update_file() on path when !renamed, since
@@ -1074,9 +1078,11 @@ static void handle_change_delete(struct merge_options *o,
 		 */
 	}
 	free(renamed);
+
+	return ret;
 }
 
-static void conflict_rename_delete(struct merge_options *o,
+static int conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
 				   const char *other_branch)
@@ -1096,21 +1102,20 @@ static void conflict_rename_delete(struct merge_options *o,
 		b_mode = dest->mode;
 	}
 
-	handle_change_delete(o,
-			     o->call_depth ? orig->path : dest->path,
-			     &orig->oid, orig->mode,
-			     a_oid, a_mode,
-			     b_oid, b_mode,
-			     _("rename"), _("renamed"));
-
-	if (o->call_depth) {
-		remove_file_from_cache(dest->path);
-	} else {
-		update_stages(dest->path, NULL,
-			      rename_branch == o->branch1 ? dest : NULL,
-			      rename_branch == o->branch1 ? NULL : dest);
-	}
+	if (handle_change_delete(o,
+				 o->call_depth ? orig->path : dest->path,
+				 &orig->oid, orig->mode,
+				 a_oid, a_mode,
+				 b_oid, b_mode,
+				 _("rename"), _("renamed")))
+		return -1;
 
+	if (o->call_depth)
+		return remove_file_from_cache(dest->path);
+	else
+		return update_stages(dest->path, NULL,
+				     rename_branch == o->branch1 ? dest : NULL,
+				     rename_branch == o->branch1 ? NULL : dest);
 }
 
 static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
@@ -1126,7 +1131,7 @@ static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
-static void handle_file(struct merge_options *o,
+static int handle_file(struct merge_options *o,
 			struct diff_filespec *rename,
 			int stage,
 			struct rename_conflict_info *ci)
@@ -1136,6 +1141,7 @@ static void handle_file(struct merge_options *o,
 	const char *cur_branch, *other_branch;
 	struct diff_filespec other;
 	struct diff_filespec *add;
+	int ret;
 
 	if (stage == 2) {
 		dst_entry = ci->dst_entry1;
@@ -1150,7 +1156,8 @@ static void handle_file(struct merge_options *o,
 	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
 	if (add) {
 		char *add_name = unique_path(o, rename->path, other_branch);
-		update_file(o, 0, &add->oid, add->mode, add_name);
+		if (update_file(o, 0, &add->oid, add->mode, add_name))
+			return -1;
 
 		remove_file(o, 0, rename->path, 0);
 		dst_name = unique_path(o, rename->path, cur_branch);
@@ -1161,17 +1168,20 @@ static void handle_file(struct merge_options *o,
 			       rename->path, other_branch, dst_name);
 		}
 	}
-	update_file(o, 0, &rename->oid, rename->mode, dst_name);
-	if (stage == 2)
-		update_stages(rename->path, NULL, rename, add);
+	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
+		; /* fall through, do allow dst_name to be released */
+	else if (stage == 2)
+		ret = update_stages(rename->path, NULL, rename, add);
 	else
-		update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
+
+	return ret;
 }
 
-static void conflict_rename_rename_1to2(struct merge_options *o,
+static int conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
@@ -1194,14 +1204,16 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
 				 ci->branch1, ci->branch2, &mfi))
-			return;
+			return -1;
+
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, &mfi.oid, mfi.mode, one->path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, one->path))
+			return -1;
 
 		/*
 		 * Above, we put the merged content at the merge-base's
@@ -1212,22 +1224,26 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * resolving the conflict at that path in its favor.
 		 */
 		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, a->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, a->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(a->path);
 		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, b->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, b->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(b->path);
-	} else {
-		handle_file(o, a, 2, ci);
-		handle_file(o, b, 3, ci);
-	}
+	} else if (handle_file(o, a, 2, ci) || handle_file(o, b, 3, ci))
+		return -1;
+
+	return 0;
 }
 
-static void conflict_rename_rename_2to1(struct merge_options *o,
+static int conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* Two files, a & b, were renamed to the same thing, c. */
@@ -1238,6 +1254,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	char *path = c1->path; /* == c2->path */
 	struct merge_file_info mfi_c1;
 	struct merge_file_info mfi_c2;
+	int ret;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1254,7 +1271,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 				       o->branch1, ci->ren2_other.path,
 				       o->branch2, c2->path, &mfi_c2))
-		return;
+		return -1;
 
 	if (o->call_depth) {
 		/*
@@ -1265,19 +1282,25 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 		 * again later for the non-recursive merge.
 		 */
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, b->path);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+					  b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, _("Renaming %s to %s and %s to %s instead"),
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, new_path2);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+					  new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
+
+	return ret;
 }
 
 static int process_renames(struct merge_options *o,
@@ -1462,12 +1485,13 @@ static int process_renames(struct merge_options *o,
 				 * update_file_flags() instead of
 				 * update_file().
 				 */
-				update_file_flags(o,
-						  &ren1->pair->two->oid,
-						  ren1->pair->two->mode,
-						  ren1_dst,
-						  1, /* update_cache */
-						  0  /* update_wd    */);
+				if (update_file_flags(o,
+						      &ren1->pair->two->oid,
+						      ren1->pair->two->mode,
+						      ren1_dst,
+						      1, /* update_cache */
+						      0  /* update_wd    */))
+					clean_merge = -1;
 			} else if (!oid_eq(&dst_other.oid, &null_oid)) {
 				clean_merge = 0;
 				try_merge = 1;
@@ -1482,22 +1506,28 @@ static int process_renames(struct merge_options *o,
 							   ren1->pair->two->mode,
 							   &dst_other.oid,
 							   dst_other.mode,
-							   branch1, branch2, &mfi))
-						return -1;
+							   branch1, branch2, &mfi)) {
+						clean_merge = -1;
+						goto cleanup_and_return;
+					}
 					output(o, 1, _("Adding merged %s"), ren1_dst);
-					update_file(o, 0, &mfi.oid,
-						    mfi.mode, ren1_dst);
+					if (update_file(o, 0, &mfi.oid,
+							mfi.mode, ren1_dst))
+						clean_merge = -1;
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, _("Adding as %s instead"), new_path);
-					update_file(o, 0, &dst_other.oid,
-						    dst_other.mode, new_path);
+					if (update_file(o, 0, &dst_other.oid,
+							dst_other.mode, new_path))
+						clean_merge = -1;
 					free(new_path);
 				}
 			} else
 				try_merge = 1;
 
+			if (clean_merge < 0)
+				goto cleanup_and_return;
 			if (try_merge) {
 				struct diff_filespec *one, *a, *b;
 				src_other.path = (char *)ren1_src;
@@ -1524,6 +1554,7 @@ static int process_renames(struct merge_options *o,
 			}
 		}
 	}
+cleanup_and_return:
 	string_list_clear(&a_by_dst, 0);
 	string_list_clear(&b_by_dst, 0);
 
@@ -1586,18 +1617,18 @@ static int blob_unchanged(const struct object_id *o_oid,
 	return ret;
 }
 
-static void handle_modify_delete(struct merge_options *o,
+static int handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 struct object_id *o_oid, int o_mode,
 				 struct object_id *a_oid, int a_mode,
 				 struct object_id *b_oid, int b_mode)
 {
-	handle_change_delete(o,
-			     path,
-			     o_oid, o_mode,
-			     a_oid, a_mode,
-			     b_oid, b_mode,
-			     _("modify"), _("modified"));
+	return handle_change_delete(o,
+				    path,
+				    o_oid, o_mode,
+				    a_oid, a_mode,
+				    b_oid, b_mode,
+				    _("modify"), _("modified"));
 }
 
 static int merge_content(struct merge_options *o,
@@ -1671,7 +1702,8 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			update_stages(path, &one, &a, &b);
+			if (update_stages(path, &one, &a, &b))
+				return -1;
 	}
 
 	if (df_conflict_remains) {
@@ -1679,30 +1711,33 @@ static int merge_content(struct merge_options *o,
 		if (o->call_depth) {
 			remove_file_from_cache(path);
 		} else {
-			if (!mfi.clean)
-				update_stages(path, &one, &a, &b);
-			else {
+			if (!mfi.clean) {
+				if (update_stages(path, &one, &a, &b))
+					return -1;
+			} else {
 				int file_from_stage2 = was_tracked(path);
 				struct diff_filespec merged;
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				update_stages(path, NULL,
-					      file_from_stage2 ? &merged : NULL,
-					      file_from_stage2 ? NULL : &merged);
+				if (update_stages(path, NULL,
+						  file_from_stage2 ? &merged : NULL,
+						  file_from_stage2 ? NULL : &merged))
+					return -1;
 			}
 
 		}
 		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, _("Adding as %s instead"), new_path);
-		update_file(o, 0, &mfi.oid, mfi.mode, new_path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, new_path)) {
+			free(new_path);
+			return -1;
+		}
 		free(new_path);
 		mfi.clean = 0;
-	} else {
-		update_file(o, mfi.clean, &mfi.oid, mfi.mode, path);
-	}
+	} else if (update_file(o, mfi.clean, &mfi.oid, mfi.mode, path))
+		return -1;
 	return mfi.clean;
-
 }
 
 /* Per entry merge function */
@@ -1730,17 +1765,21 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
-			conflict_rename_delete(o, conflict_info->pair1,
-					       conflict_info->branch1,
-					       conflict_info->branch2);
+			if (conflict_rename_delete(o,
+						   conflict_info->pair1,
+						   conflict_info->branch1,
+						   conflict_info->branch2))
+				clean_merge = -1;
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info);
+			if (conflict_rename_rename_1to2(o, conflict_info))
+				clean_merge = -1;
 			break;
 		case RENAME_TWO_FILES_TO_ONE:
 			clean_merge = 0;
-			conflict_rename_rename_2to1(o, conflict_info);
+			if (conflict_rename_rename_2to1(o, conflict_info))
+				clean_merge = -1;
 			break;
 		default:
 			entry->processed = 0;
@@ -1760,8 +1799,9 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_modify_delete(o, path, o_oid, o_mode,
-					     a_oid, a_mode, b_oid, b_mode);
+			if (handle_modify_delete(o, path, o_oid, o_mode,
+						 a_oid, a_mode, b_oid, b_mode))
+				clean_merge = -1;
 		}
 	} else if ((!o_oid && a_oid && !b_oid) ||
 		   (!o_oid && !a_oid && b_oid)) {
@@ -1793,14 +1833,16 @@ static int process_entry(struct merge_options *o,
 			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s"),
 			       conf, path, other_branch, path, new_path);
-			update_file(o, 0, oid, mode, new_path);
-			if (o->call_depth)
+			if (update_file(o, 0, oid, mode, new_path))
+				clean_merge = -1;
+			else if (o->call_depth)
 				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, _("Adding %s"), path);
 			/* do not overwrite file if already present */
-			update_file_flags(o, oid, mode, path, 1, !a_oid);
+			if (update_file_flags(o, oid, mode, path, 1, !a_oid))
+				clean_merge = -1;
 		}
 	} else if (a_oid && b_oid) {
 		/* Case C: Added in both (check for same permissions) and */
@@ -1863,12 +1905,18 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
+		if (clean < 0)
+			return clean;
 		for (i = entries->nr-1; 0 <= i; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_entry(o, path, e))
-				clean = 0;
+			if (!e->processed) {
+				int ret = process_entry(o, path, e);
+				if (!ret)
+					clean = 0;
+				else if (ret < 0)
+					return ret;
+			}
 		}
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 10/16] merge-recursive: switch to returning errors instead of dying

The recursive merge machinery is supposed to be a library function, i.e. it should return an error when it fails. Originally the functions were part of the builtin "merge-recursive", though, where it was simpler to call die() and be done with error handling.

The existing callers were already prepared to detect negative return values to indicate errors and to behave as previously: exit with code 128 (which is the same thing that die() does, after printing the message).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 62 +++++++++++++++++++++++++++++++------------------------
 1 file changed, 35 insertions(+), 27 deletions(-)
Show changes to merge-recursive.c +35 −27
diff --git a/merge-recursive.c b/merge-recursive.c
index 6beb1e4..bc59815 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -275,8 +275,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		active_cache_tree = cache_tree();
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
-	    cache_tree_update(&the_index, 0) < 0)
-		die(_("error building trees"));
+	    cache_tree_update(&the_index, 0) < 0) {
+		error(_("error building trees"));
+		return NULL;
+	}
 
 	result = lookup_tree(active_cache_tree->sha1);
 
@@ -716,12 +718,10 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
-		if (status == SCLD_EXISTS) {
+		if (status == SCLD_EXISTS)
 			/* something else exists */
-			error(msg, path, _(": perhaps a D/F conflict?"));
-			return -1;
-		}
-		die(msg, path, "");
+			return error(msg, path, _(": perhaps a D/F conflict?"));
+		return error(msg, path, "");
 	}
 
 	/*
@@ -749,6 +749,8 @@ static int update_file_flags(struct merge_options *o,
 			     int update_cache,
 			     int update_wd)
 {
+	int ret = 0;
+
 	if (o->call_depth)
 		update_wd = 0;
 
@@ -769,9 +771,11 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			die(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
-		if (type != OBJ_BLOB)
-			die(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+		if (type != OBJ_BLOB) {
+			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			goto free_buf;
+		}
 		if (S_ISREG(mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
 			if (convert_to_working_tree(path, buf, size, &strbuf)) {
@@ -792,8 +796,11 @@ static int update_file_flags(struct merge_options *o,
 			else
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-			if (fd < 0)
-				die_errno(_("failed to open '%s'"), path);
+			if (fd < 0) {
+				ret = error_errno(_("failed to open '%s'"),
+						  path);
+				goto free_buf;
+			}
 			write_in_full(fd, buf, size);
 			close(fd);
 		} else if (S_ISLNK(mode)) {
@@ -801,18 +808,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				die_errno(_("failed to symlink '%s'"), path);
+				ret = error_errno(_("failed to symlink '%s'"), path);
 			free(lnk);
 		} else
-			die(_("do not know what to do with %06o %s '%s'"),
-			    mode, oid_to_hex(oid), path);
+			ret = error(_("do not know what to do with %06o %s '%s'"),
+				    mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
-	if (update_cache)
+	if (!ret && update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
-	return 0;
+	return ret;
 }
 
 static int update_file(struct merge_options *o,
@@ -938,20 +945,22 @@ static int merge_file_1(struct merge_options *o,
 			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
-			int merge_status;
+			int ret = 0, merge_status;
 
 			merge_status = merge_3way(o, &result_buf, one, a, b,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				die(_("Failed to execute internal merge"));
+				ret = error(_("Failed to execute internal merge"));
 
-			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result->oid.hash))
-				die(_("Unable to add %s to database"),
-				    a->path);
+			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
+						    blob_type, result->oid.hash))
+				ret = error(_("Unable to add %s to database"),
+					    a->path);
 
 			free(result_buf.ptr);
+			if (ret)
+				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
 			result->clean = merge_submodule(result->oid.hash,
@@ -1885,11 +1894,10 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			die(_("merging of trees %s and %s failed"),
+			error(_("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
-		else
-			exit(128);
+		return -1;
 	}
 
 	if (unmerged_cache()) {
@@ -2021,7 +2029,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			die(_("merge returned no commit"));
+			return error(_("merge returned no commit"));
 	}
 
 	discard_cache();
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 13/16] merge-recursive: write the commit title in one go

In 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we changed the code such that it prints the output in one go, to avoid interfering with the progress output.

Let's make sure that the same holds true when outputting the commit title: previously, we used several printf() statements to stdout and speculated that stdout's buffer is large enough to hold the entire commit title.

Apart from making that speculation unnecessary, we change the code to add the message to the output buffer before flushing for another reason: the next commit will introduce a new level of output buffering, where the caller can request the output not to be flushed, but to be retained for further processing.

This latter feature will be needed when teaching the sequencer to do rebase -i's brunt work: it wants to control the output of the cherry-picks (i.e. recursive merges).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
Show changes to merge-recursive.c +9 −8
diff --git a/merge-recursive.c b/merge-recursive.c
index 71a0aa0..723b8d0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -191,25 +191,26 @@ static void output(struct merge_options *o, int v, const char *fmt, ...)
 
 static void output_commit_title(struct merge_options *o, struct commit *commit)
 {
-	int i;
-	flush_output(o);
-	for (i = o->call_depth; i--;)
-		fputs("  ", stdout);
+	strbuf_addchars(&o->obuf, ' ', o->call_depth * 2);
 	if (commit->util)
-		printf("virtual %s\n", merge_remote_util(commit)->name);
+		strbuf_addf(&o->obuf, "virtual %s\n",
+			merge_remote_util(commit)->name);
 	else {
-		printf("%s ", find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV));
+		strbuf_addf(&o->obuf, "%s ",
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV));
 		if (parse_commit(commit) != 0)
-			printf(_("(bad commit)\n"));
+			strbuf_addf(&o->obuf, _("(bad commit)\n"));
 		else {
 			const char *title;
 			const char *msg = get_commit_buffer(commit, NULL);
 			int len = find_commit_subject(msg, &title);
 			if (len)
-				printf("%.*s\n", len, title);
+				strbuf_addf(&o->obuf, "%.*s\n", len, title);
 			unuse_commit_buffer(commit, msg);
 		}
 	}
+	flush_output(o);
 }
 
 static int add_cacheinfo(struct merge_options *o,
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 27, 2016, 22:36 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 13/16] merge-recursive: write the commit title in one go

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> Let's make sure that the same holds true when outputting the commit
> title: previously, we used several printf() statements to stdout and
> speculated that stdout's buffer is large enough to hold the entire
> commit title.
s/speculate/assume/; other than that looks very sensible.
Johannes Schindelin· Aug 1, 2016, 09:53 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 13/16] merge-recursive: write the commit title in one go

Hi Junio,
On Wed, 27 Jul 2016, Junio C Hamano wrote:
Show 8 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Let's make sure that the same holds true when outputting the commit
> > title: previously, we used several printf() statements to stdout and
> > speculated that stdout's buffer is large enough to hold the entire
> > commit title.
> 
> s/speculate/assume/; other than that looks very sensible.
Fixed.

Thanks, Dscho

Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 11/16] am -3: use merge_recursive() directly again

Last October, we had to change this code to run `git merge-recursive` in a child process: git-am wants to print some helpful advice when the merge failed, but the code in question was not prepared to return, it die()d instead.

We are finally at a point when the code *is* prepared to return errors, and can avoid the child process again.

This reverts commit c63d4b2 (am -3: do not let failed merge from completing the error codepath, 2015-10-09), with the necessary changes to adjust for the fact that Git's source code changed in the meantime (such as: using OIDs instead of hashes in the recursive merge, and a removed gender bias).

Note: the code now calls merge_recursive_generic() again. Unlike
merge_trees() and merge_recursive(), this function returns 0 upon success,
as most of Git's functions. Therefore, the error value -1 naturally is
handled correctly, and we do not have to take care of it specifically.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c | 62 +++++++++++++++++++++---------------------------------------
 1 file changed, 22 insertions(+), 40 deletions(-)
Show changes to builtin/am.c +22 −40
diff --git a/builtin/am.c b/builtin/am.c
index b77bf11..cfb79ea 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1579,47 +1579,18 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
 }
 
 /**
- * Do the three-way merge using fake ancestor, their tree constructed
- * from the fake ancestor and the postimage of the patch, and our
- * state.
- */
-static int run_fallback_merge_recursive(const struct am_state *state,
-					unsigned char *orig_tree,
-					unsigned char *our_tree,
-					unsigned char *their_tree)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-
-	cp.git_cmd = 1;
-
-	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(their_tree), linelen(state->msg), state->msg);
-	if (state->quiet)
-		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
-
-	argv_array_push(&cp.args, "merge-recursive");
-	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
-	argv_array_push(&cp.args, "--");
-	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(their_tree));
-
-	status = run_command(&cp) ? (-1) : 0;
-	discard_cache();
-	read_cache();
-	return status;
-}
-
-/**
  * Attempt a threeway merge, using index_path as the temporary index.
  */
 static int fall_back_threeway(const struct am_state *state, const char *index_path)
 {
-	unsigned char orig_tree[GIT_SHA1_RAWSZ], their_tree[GIT_SHA1_RAWSZ],
-		      our_tree[GIT_SHA1_RAWSZ];
+	struct object_id orig_tree, their_tree, our_tree;
+	const struct object_id *bases[1] = { &orig_tree };
+	struct merge_options o;
+	struct commit *result;
+	char *their_tree_name;
 
-	if (get_sha1("HEAD", our_tree) < 0)
-		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
+	if (get_oid("HEAD", &our_tree) < 0)
+		hashcpy(our_tree.hash, EMPTY_TREE_SHA1_BIN);
 
 	if (build_fake_ancestor(state, index_path))
 		return error("could not build fake ancestor");
@@ -1627,7 +1598,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	discard_cache();
 	read_cache_from(index_path);
 
-	if (write_index_as_tree(orig_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(orig_tree.hash, &the_index, index_path, 0, NULL))
 		return error(_("Repository lacks necessary blobs to fall back on 3-way merge."));
 
 	say(state, stdout, _("Using index info to reconstruct a base tree..."));
@@ -1643,7 +1614,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		init_revisions(&rev_info, NULL);
 		rev_info.diffopt.output_format = DIFF_FORMAT_NAME_STATUS;
 		diff_opt_parse(&rev_info.diffopt, &diff_filter_str, 1, rev_info.prefix);
-		add_pending_sha1(&rev_info, "HEAD", our_tree, 0);
+		add_pending_sha1(&rev_info, "HEAD", our_tree.hash, 0);
 		diff_setup_done(&rev_info.diffopt);
 		run_diff_index(&rev_info, 1);
 	}
@@ -1652,7 +1623,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		return error(_("Did you hand edit your patch?\n"
 				"It does not apply to blobs recorded in its index."));
 
-	if (write_index_as_tree(their_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(their_tree.hash, &the_index, index_path, 0, NULL))
 		return error("could not write tree");
 
 	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
@@ -1668,11 +1639,22 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	 * changes.
 	 */
 
-	if (run_fallback_merge_recursive(state, orig_tree, our_tree, their_tree)) {
+	init_merge_options(&o);
+
+	o.branch1 = "HEAD";
+	their_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
+	o.branch2 = their_tree_name;
+
+	if (state->quiet)
+		o.verbosity = 0;
+
+	if (merge_recursive_generic(&o, &our_tree, &their_tree, 1, bases, &result)) {
 		rerere(state->allow_rerere_autoupdate);
+		free(their_tree_name);
 		return error(_("Failed to merge in the changes."));
 	}
 
+	free(their_tree_name);
 	return 0;
 }
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 12/16] merge-recursive: flush output buffer before printing error messages

The data structure passed to the recursive merge machinery has a feature where the caller can ask for the output to be buffered into a strbuf, by setting the field 'buffer_output'.

Previously, we simply swallowed the buffered output when showing error messages. With this patch, we show the output first, and only then print the error message.

Currently, the only user of that buffering is merge_recursive() itself, to avoid the progress output to interfere.

In the next patches, we will introduce a new buffer_output mode that forces merge_recursive() to retain the output buffer for further processing by the caller. If the caller asked for that, we will then also write the error messages into the output buffer. This is necessary to give the caller more control not only how to react in case of errors but also control how/if to display the error messages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 116 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 68 insertions(+), 48 deletions(-)
Show changes to merge-recursive.c +68 −48
diff --git a/merge-recursive.c b/merge-recursive.c
index bc59815..71a0aa0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,28 @@
 #include "dir.h"
 #include "submodule.h"
 
+static void flush_output(struct merge_options *o)
+{
+	if (o->obuf.len) {
+		fputs(o->obuf.buf, stdout);
+		strbuf_reset(&o->obuf);
+	}
+}
+
+static int err(struct merge_options *o, const char *err, ...)
+{
+	va_list params;
+
+	va_start(params, err);
+	flush_output(o);
+	strbuf_vaddf(&o->obuf, err, params);
+	error("%s", o->obuf.buf);
+	strbuf_reset(&o->obuf);
+	va_end(params);
+
+	return -1;
+}
+
 static struct tree *shift_tree_object(struct tree *one, struct tree *two,
 				      const char *subtree_shift)
 {
@@ -148,14 +170,6 @@ static int show(struct merge_options *o, int v)
 	return (!o->call_depth && o->verbosity >= v) || o->verbosity >= 5;
 }
 
-static void flush_output(struct merge_options *o)
-{
-	if (o->obuf.len) {
-		fputs(o->obuf.buf, stdout);
-		strbuf_reset(&o->obuf);
-	}
-}
-
 __attribute__((format (printf, 3, 4)))
 static void output(struct merge_options *o, int v, const char *fmt, ...)
 {
@@ -198,7 +212,8 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)
 	}
 }
 
-static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
+static int add_cacheinfo(struct merge_options *o,
+		unsigned int mode, const struct object_id *oid,
 		const char *path, int stage, int refresh, int options)
 {
 	struct cache_entry *ce;
@@ -206,7 +221,7 @@ static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
 
 	ce = make_cache_entry(mode, oid ? oid->hash : null_sha1, path, stage, 0);
 	if (!ce)
-		return error(_("addinfo_cache failed for path '%s'"), path);
+		return err(o, _("addinfo_cache failed for path '%s'"), path);
 
 	ret = add_cache_entry(ce, options);
 	if (refresh) {
@@ -276,7 +291,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
 	    cache_tree_update(&the_index, 0) < 0) {
-		error(_("error building trees"));
+		err(o, _("error building trees"));
 		return NULL;
 	}
 
@@ -544,7 +559,8 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages(const char *path, const struct diff_filespec *o,
+static int update_stages(struct merge_options *opt, const char *path,
+			 const struct diff_filespec *o,
 			 const struct diff_filespec *a,
 			 const struct diff_filespec *b)
 {
@@ -563,13 +579,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 		if (remove_file_from_cache(path))
 			return -1;
 	if (o)
-		if (add_cacheinfo(o->mode, &o->oid, path, 1, 0, options))
+		if (add_cacheinfo(opt, o->mode, &o->oid, path, 1, 0, options))
 			return -1;
 	if (a)
-		if (add_cacheinfo(a->mode, &a->oid, path, 2, 0, options))
+		if (add_cacheinfo(opt, a->mode, &a->oid, path, 2, 0, options))
 			return -1;
 	if (b)
-		if (add_cacheinfo(b->mode, &b->oid, path, 3, 0, options))
+		if (add_cacheinfo(opt, b->mode, &b->oid, path, 3, 0, options))
 			return -1;
 	return 0;
 }
@@ -720,8 +736,8 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (status) {
 		if (status == SCLD_EXISTS)
 			/* something else exists */
-			return error(msg, path, _(": perhaps a D/F conflict?"));
-		return error(msg, path, "");
+			return err(o, msg, path, _(": perhaps a D/F conflict?"));
+		return err(o, msg, path, "");
 	}
 
 	/*
@@ -729,7 +745,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	 * tracking it.
 	 */
 	if (would_lose_untracked(path))
-		return error(_("refusing to lose untracked file at '%s'"),
+		return err(o, _("refusing to lose untracked file at '%s'"),
 			     path);
 
 	/* Successful unlink is good.. */
@@ -739,7 +755,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (errno == ENOENT)
 		return 0;
 	/* .. but not some other error (who really cares what?) */
-	return error(msg, path, _(": perhaps a D/F conflict?"));
+	return err(o, msg, path, _(": perhaps a D/F conflict?"));
 }
 
 static int update_file_flags(struct merge_options *o,
@@ -771,9 +787,9 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+			return err(o, _("cannot read object %s '%s'"), oid_to_hex(oid), path);
 		if (type != OBJ_BLOB) {
-			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			ret = err(o, _("blob expected for %s '%s'"), oid_to_hex(oid), path);
 			goto free_buf;
 		}
 		if (S_ISREG(mode)) {
@@ -797,8 +813,8 @@ static int update_file_flags(struct merge_options *o,
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
 			if (fd < 0) {
-				ret = error_errno(_("failed to open '%s'"),
-						  path);
+				ret = err(o, _("failed to open '%s': %s"),
+					  path, strerror(errno));
 				goto free_buf;
 			}
 			write_in_full(fd, buf, size);
@@ -808,17 +824,19 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				ret = error_errno(_("failed to symlink '%s'"), path);
+				ret = err(o, _("failed to symlink '%s': %s"),
+					path, strerror(errno));
 			free(lnk);
 		} else
-			ret = error(_("do not know what to do with %06o %s '%s'"),
-				    mode, oid_to_hex(oid), path);
+			ret = err(o,
+				  _("do not know what to do with %06o %s '%s'"),
+				  mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
 	if (!ret && update_cache)
-		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+		add_cacheinfo(o, mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
 	return ret;
 }
 
@@ -951,12 +969,12 @@ static int merge_file_1(struct merge_options *o,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				ret = error(_("Failed to execute internal merge"));
+				ret = err(o, _("Failed to execute internal merge"));
 
 			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 						    blob_type, result->oid.hash))
-				ret = error(_("Unable to add %s to database"),
-					    a->path);
+				ret = err(o, _("Unable to add %s to database"),
+					  a->path);
 
 			free(result_buf.ptr);
 			if (ret)
@@ -1122,7 +1140,7 @@ static int conflict_rename_delete(struct merge_options *o,
 	if (o->call_depth)
 		return remove_file_from_cache(dest->path);
 	else
-		return update_stages(dest->path, NULL,
+		return update_stages(o, dest->path, NULL,
 				     rename_branch == o->branch1 ? dest : NULL,
 				     rename_branch == o->branch1 ? NULL : dest);
 }
@@ -1180,9 +1198,9 @@ static int handle_file(struct merge_options *o,
 	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
 		; /* fall through, do allow dst_name to be released */
 	else if (stage == 2)
-		ret = update_stages(rename->path, NULL, rename, add);
+		ret = update_stages(o, rename->path, NULL, rename, add);
 	else
-		ret = update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(o, rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
@@ -1575,23 +1593,25 @@ static struct object_id *stage_oid(const struct object_id *oid, unsigned mode)
 	return (is_null_oid(oid) || mode == 0) ? NULL: (struct object_id *)oid;
 }
 
-static int read_oid_strbuf(const struct object_id *oid, struct strbuf *dst)
+static int read_oid_strbuf(struct merge_options *o,
+	const struct object_id *oid, struct strbuf *dst)
 {
 	void *buf;
 	enum object_type type;
 	unsigned long size;
 	buf = read_sha1_file(oid->hash, &type, &size);
 	if (!buf)
-		return error(_("cannot read object %s"), oid_to_hex(oid));
+		return err(o, _("cannot read object %s"), oid_to_hex(oid));
 	if (type != OBJ_BLOB) {
 		free(buf);
-		return error(_("object %s is not a blob"), oid_to_hex(oid));
+		return err(o, _("object %s is not a blob"), oid_to_hex(oid));
 	}
 	strbuf_attach(dst, buf, size, size + 1);
 	return 0;
 }
 
-static int blob_unchanged(const struct object_id *o_oid,
+static int blob_unchanged(struct merge_options *opt,
+			  const struct object_id *o_oid,
 			  unsigned o_mode,
 			  const struct object_id *a_oid,
 			  unsigned a_mode,
@@ -1609,7 +1629,7 @@ static int blob_unchanged(const struct object_id *o_oid,
 		return 0;
 
 	assert(o_oid && a_oid);
-	if (read_oid_strbuf(o_oid, &o) || read_oid_strbuf(a_oid, &a))
+	if (read_oid_strbuf(opt, o_oid, &o) || read_oid_strbuf(opt, a_oid, &a))
 		goto error_return;
 	/*
 	 * Note: binary | is used so that both renormalizations are
@@ -1698,7 +1718,7 @@ static int merge_content(struct merge_options *o,
 		 */
 		path_renamed_outside_HEAD = !path2 || !strcmp(path, path2);
 		if (!path_renamed_outside_HEAD) {
-			add_cacheinfo(mfi.mode, &mfi.oid, path,
+			add_cacheinfo(o, mfi.mode, &mfi.oid, path,
 				      0, (!o->call_depth), 0);
 			return mfi.clean;
 		}
@@ -1711,7 +1731,7 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			if (update_stages(path, &one, &a, &b))
+			if (update_stages(o, path, &one, &a, &b))
 				return -1;
 	}
 
@@ -1721,7 +1741,7 @@ static int merge_content(struct merge_options *o,
 			remove_file_from_cache(path);
 		} else {
 			if (!mfi.clean) {
-				if (update_stages(path, &one, &a, &b))
+				if (update_stages(o, path, &one, &a, &b))
 					return -1;
 			} else {
 				int file_from_stage2 = was_tracked(path);
@@ -1729,7 +1749,7 @@ static int merge_content(struct merge_options *o,
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				if (update_stages(path, NULL,
+				if (update_stages(o, path, NULL,
 						  file_from_stage2 ? &merged : NULL,
 						  file_from_stage2 ? NULL : &merged))
 					return -1;
@@ -1797,8 +1817,8 @@ static int process_entry(struct merge_options *o,
 	} else if (o_oid && (!a_oid || !b_oid)) {
 		/* Case A: Deleted in one */
 		if ((!a_oid && !b_oid) ||
-		    (!b_oid && blob_unchanged(o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
-		    (!a_oid && blob_unchanged(o_oid, o_mode, b_oid, b_mode, normalize, path))) {
+		    (!b_oid && blob_unchanged(o, o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
+		    (!a_oid && blob_unchanged(o, o_oid, o_mode, b_oid, b_mode, normalize, path))) {
 			/* Deleted in both or deleted in one and
 			 * unchanged in the other */
 			if (a_oid)
@@ -1894,7 +1914,7 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			error(_("merging of trees %s and %s failed"),
+			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
 		return -1;
@@ -2029,7 +2049,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			return error(_("merge returned no commit"));
+			return err(o, _("merge returned no commit"));
 	}
 
 	discard_cache();
@@ -2088,7 +2108,7 @@ int merge_recursive_generic(struct merge_options *o,
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
 			if (!(base = get_ref(base_list[i], oid_to_hex(base_list[i]))))
-				return error(_("Could not parse object '%s'"),
+				return err(o, _("Could not parse object '%s'"),
 					oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
 		}
@@ -2102,7 +2122,7 @@ int merge_recursive_generic(struct merge_options *o,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
-		return error(_("Unable to write index."));
+		return err(o, _("Unable to write index."));
 
 	return clean ? 0 : 1;
 }
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 27, 2016, 21:37 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 12/16] merge-recursive: flush output buffer before printing error messages

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 7 quoted lines
> The data structure passed to the recursive merge machinery has a feature
> where the caller can ask for the output to be buffered into a strbuf, by
> setting the field 'buffer_output'.
>
> Previously, we simply swallowed the buffered output when showing error
> messages. With this patch, we show the output first, and only then print
> the error message.

I didn't quite understand this paragraph until I realized that you meant "when showing die message". We died without flushing, losing accumulated output.

Show 6 quoted lines
> +static int err(struct merge_options *o, const char *err, ...)
> +{
> +	va_list params;
> +
> +	va_start(params, err);
> +	flush_output(o);

I would have written the above two swapped; va_start() logically is about what happens in the next four lines.

> +	strbuf_vaddf(&o->obuf, err, params);
> +	error("%s", o->obuf.buf);
> +	strbuf_reset(&o->obuf);
Sneaky ;-)

The remainder replaces error(...) with err(o, ...) and updates the callchain to pass the merge_options around, which looked good.

Thanks.
Junio C Hamano· Jul 27, 2016, 21:53 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 12/16] merge-recursive: flush output buffer before printing error messages

On Wed, Jul 27, 2016 at 2:37 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 6 quoted lines
>
>> +     strbuf_vaddf(&o->obuf, err, params);
>> +     error("%s", o->obuf.buf);
>> +     strbuf_reset(&o->obuf);
>
> Sneaky ;-)

Just to avoid confusion, I am _fine_ with this "we happen to have a strbuf that we know to be empty at this point, so let's reuse it and clean after ourselves before returning".

I just found it somewhere between clever and ugly, and "sneaky" was the first word that came to my mind.

Johannes Schindelin· Aug 1, 2016, 09:18 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 12/16] merge-recursive: flush output buffer before printing error messages

Hi Junio,
On Wed, 27 Jul 2016, Junio C Hamano wrote:
Show 13 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The data structure passed to the recursive merge machinery has a feature
> > where the caller can ask for the output to be buffered into a strbuf, by
> > setting the field 'buffer_output'.
> >
> > Previously, we simply swallowed the buffered output when showing error
> > messages. With this patch, we show the output first, and only then print
> > the error message.
> 
> I didn't quite understand this paragraph until I realized that you
> meant "when showing die message".  We died without flushing, losing
> accumulated output.
I rephrased it, using your explanation.
Show 9 quoted lines
> > +static int err(struct merge_options *o, const char *err, ...)
> > +{
> > +	va_list params;
> > +
> > +	va_start(params, err);
> > +	flush_output(o);
> 
> I would have written the above two swapped; va_start() logically
> is about what happens in the next four lines.

For some reason, I thought that `va_start()` must be the first statement of the function. Fixed.

Show 5 quoted lines
> > +	strbuf_vaddf(&o->obuf, err, params);
> > +	error("%s", o->obuf.buf);
> > +	strbuf_reset(&o->obuf);
> 
> Sneaky ;-)

Thanks ;-) Dscho

Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we already accumulate the output in a buffer. The idea was to avoid interfering with the progress output that goes to stderr, which is unbuffered, when we write to stdout, which is buffered.

We extend that buffering to allow the caller to handle the output (possibly suppressing it). This will help us when extending the sequencer to do rebase -i's brunt work: it does not want the picks to print anything by default but instead determine itself whether to print the output or not.

Note that we also redirect the error messages into the output buffer
when the caller asked not to flush the output buffer, for two reasons:
1) to retain the correct output order, and 2) to allow the caller to
suppress *all* output.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++++++----
 merge-recursive.h |  2 +-
 2 files changed, 14 insertions(+), 5 deletions(-)
Show changes to 2 files +14 −5

merge-recursive.c, merge-recursive.h

diff --git a/merge-recursive.c b/merge-recursive.c
index 723b8d0..311cfa4 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -25,7 +25,7 @@
 
 static void flush_output(struct merge_options *o)
 {
-	if (o->obuf.len) {
+	if (o->buffer_output < 2 && o->obuf.len) {
 		fputs(o->obuf.buf, stdout);
 		strbuf_reset(&o->obuf);
 	}
@@ -36,10 +36,19 @@ static int err(struct merge_options *o, const char *err, ...)
 	va_list params;
 
 	va_start(params, err);
-	flush_output(o);
+	if (o->buffer_output < 2)
+		flush_output(o);
+	else {
+		strbuf_complete(&o->obuf, '\n');
+		strbuf_addstr(&o->obuf, "error: ");
+	}
 	strbuf_vaddf(&o->obuf, err, params);
-	error("%s", o->obuf.buf);
-	strbuf_reset(&o->obuf);
+	if (o->buffer_output > 1)
+		strbuf_addch(&o->obuf, '\n');
+	else {
+		error("%s", o->obuf.buf);
+		strbuf_reset(&o->obuf);
+	}
 	va_end(params);
 
 	return -1;
diff --git a/merge-recursive.h b/merge-recursive.h
index d415724..340704c 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -13,7 +13,7 @@ struct merge_options {
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
 	const char *subtree_shift;
-	unsigned buffer_output : 1;
+	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 27, 2016, 22:09 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 15 quoted lines
> Since 66a155b (Enable output buffering in merge-recursive., 2007-01-14),
> we already accumulate the output in a buffer. The idea was to avoid
> interfering with the progress output that goes to stderr, which is
> unbuffered, when we write to stdout, which is buffered.
>
> We extend that buffering to allow the caller to handle the output
> (possibly suppressing it). This will help us when extending the
> sequencer to do rebase -i's brunt work: it does not want the picks to
> print anything by default but instead determine itself whether to print
> the output or not.
>
> Note that we also redirect the error messages into the output buffer
> when the caller asked not to flush the output buffer, for two reasons:
> 1) to retain the correct output order, and 2) to allow the caller to
> suppress *all* output.

I am not yet sure if it makes sense to mix both the regular output and an error into the same buffer for the callers to process (your "reason 1)" above), and this looks like a wrong way to allow a caller that wants no output (your "reason 2)" above). A caller that wants to massage the output would want to know which ones are errors and which ones are not, I would imagine, and setting a knob to make both output() and err() a no-op would be a more suitable way to give a caller a total silence.

At least I cannot yet judge if this is a good change, only from the material presented here. That does not mean I have a concrete reason to say this is a bad change, either. Only from the material presented here, I cannot judge that, either.

Show 53 quoted lines
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  merge-recursive.c | 17 +++++++++++++----
>  merge-recursive.h |  2 +-
>  2 files changed, 14 insertions(+), 5 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 723b8d0..311cfa4 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -25,7 +25,7 @@
>  
>  static void flush_output(struct merge_options *o)
>  {
> -	if (o->obuf.len) {
> +	if (o->buffer_output < 2 && o->obuf.len) {
>  		fputs(o->obuf.buf, stdout);
>  		strbuf_reset(&o->obuf);
>  	}
> @@ -36,10 +36,19 @@ static int err(struct merge_options *o, const char *err, ...)
>  	va_list params;
>  
>  	va_start(params, err);
> -	flush_output(o);
> +	if (o->buffer_output < 2)
> +		flush_output(o);
> +	else {
> +		strbuf_complete(&o->obuf, '\n');
> +		strbuf_addstr(&o->obuf, "error: ");
> +	}
>  	strbuf_vaddf(&o->obuf, err, params);
> -	error("%s", o->obuf.buf);
> -	strbuf_reset(&o->obuf);
> +	if (o->buffer_output > 1)
> +		strbuf_addch(&o->obuf, '\n');
> +	else {
> +		error("%s", o->obuf.buf);
> +		strbuf_reset(&o->obuf);
> +	}
>  	va_end(params);
>  
>  	return -1;
> diff --git a/merge-recursive.h b/merge-recursive.h
> index d415724..340704c 100644
> --- a/merge-recursive.h
> +++ b/merge-recursive.h
> @@ -13,7 +13,7 @@ struct merge_options {
>  		MERGE_RECURSIVE_THEIRS
>  	} recursive_variant;
>  	const char *subtree_shift;
> -	unsigned buffer_output : 1;
> +	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
>  	unsigned renormalize : 1;

Once a field ceases to be a boolean, it is OK not to squish it into a bitfield like this for a struct that we will have only a very small number of instances of. Treating it just like "verbosity", which occupies a whole int even though it can only get up to 5 or so, would be more appropriate.

>  	long xdl_opts;
>  	int verbosity;
Junio C Hamano· Jul 28, 2016, 00:17 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Junio C Hamano <gitster@pobox.com> writes:
Show 8 quoted lines
> I am not yet sure if it makes sense to mix both the regular output
> and an error into the same buffer for the callers to process (your
> "reason 1)" above), and this looks like a wrong way to allow a
> caller that wants no output (your "reason 2)" above).  A caller that
> wants to massage the output would want to know which ones are errors
> and which ones are not, I would imagine, and setting a knob to make
> both output() and err() a no-op would be a more suitable way to give
> a caller a total silence.

I actually now see how this would work well for "reason 2)". If a caller wants to run the function and wants to pretend as if it did not run anything when it failed, for example, using this to spool all output and error to a strbuf and discard it when the function returns an error, and emit the spooled output to standard output and standard error in the order the lines were collected when the function returns a success, would be a good way to do so.

That however brings me back to the "reason 1" thing. Shouldn't the spooling be done not just with a single strbuf, but with an array of <bool, const char *> tuples, where the bool says if it is for the standard output, and the string holds the message? output() would mark its element in the array with true, while err() would do so with false.

Johannes Schindelin· Aug 1, 2016, 09:34 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Hi Junio,
On Wed, 27 Jul 2016, Junio C Hamano wrote:
Show 18 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > I am not yet sure if it makes sense to mix both the regular output
> > and an error into the same buffer for the callers to process (your
> > "reason 1)" above), and this looks like a wrong way to allow a
> > caller that wants no output (your "reason 2)" above).  A caller that
> > wants to massage the output would want to know which ones are errors
> > and which ones are not, I would imagine, and setting a knob to make
> > both output() and err() a no-op would be a more suitable way to give
> > a caller a total silence.
> 
> I actually now see how this would work well for "reason 2)".  If a
> caller wants to run the function and wants to pretend as if it did
> not run anything when it failed, for example, using this to spool
> all output and error to a strbuf and discard it when the function
> returns an error, and emit the spooled output to standard output and
> standard error in the order the lines were collected when the
> function returns a success, would be a good way to do so.

That is actually the exact opposite of the intended usage: when any `pick` in an interactive rebase succeeds, its output is discarded so as not to bother the user. We show the complete output only when it fails.

Show 6 quoted lines
> That however brings me back to the "reason 1" thing.  Shouldn't the
> spooling be done not just with a single strbuf, but with an array of
> <bool, const char *> tuples, where the bool says if it is for the
> standard output, and the string holds the message?  output() would
> mark its element in the array with true, while err() would do so
> with false.

I would like to *not* deviate further from my original focus. My willingness to address comments that have little to nothing to do with accelerating the interactive rebase has cost me already over a month, with this patch series alone.

Do not get me wrong: I think you are right in your assessment that a future caller of the recursive merge might need to be able to tell apart which lines of the output are error messages from the rest, and still retain the order.

But then, maybe no future caller will require this.

What is certain: right now, no caller requires it, and neither do my queued-up patches to speed up the interactive rebase.

And another thing is also certain: *iff* a future caller really will require that fine-grained information of the output, then it will be dramatically easier to implement this on top of the patches we are discussing right now.

In short: I hope you agree with me that it is safe to defer the <bool, const char*> tuple implementation to the time when we will *actually* need it.

Ciao, Dscho

Junio C Hamano· Aug 1, 2016, 19:09 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 11 quoted lines
>> I actually now see how this would work well for "reason 2)".  If a
>> caller wants to run the function and wants to pretend as if it did
>> not run anything when it failed, for example, using this to spool
>> all output and error to a strbuf and discard it when the function
>> returns an error, and emit the spooled output to standard output and
>> standard error in the order the lines were collected when the
>> function returns a success, would be a good way to do so.
>
> That is actually the exact opposite of the intended usage: when any `pick`
> in an interactive rebase succeeds, its output is discarded so as not to
> bother the user. We show the complete output only when it fails.

Oh, it makes sense, too, to show the output only when there is an error.

But in that case, there would be both messages meant for the standard output and also meant for the standard error, and we need some way to make sure they go to the right channel.

I however do not think an array of <bool, const char *> is the only way to achieve that. We can get away by a single strbuf that accumulates all output() that we have seen so far, i.e. "we only accumulate output() and ignore flush() as long as what we see are only from output()" mode.

Then the err() routine operating under this new mode can show what has been accumulated to the standard output (because with this tweak I am outlining here, by definition, the strbuf will only keep the output() material and not err() things), show the err() message, and switch back to the normal "we accumulate output() and honor flush()" mode. Of course, when we are doing multiple rounds, the mode must be reset to "accumulate output and ignore flush" mode at the beginning of each rouhd.

That would give us "silence if there is no error, but if we are showing error, show them to the standard error, while giving non-error message to the standard output".

Johannes Schindelin· Aug 2, 2016, 08:01 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Hi Junio,
On Mon, 1 Aug 2016, Junio C Hamano wrote:
Show 16 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> I actually now see how this would work well for "reason 2)".  If a
> >> caller wants to run the function and wants to pretend as if it did
> >> not run anything when it failed, for example, using this to spool all
> >> output and error to a strbuf and discard it when the function returns
> >> an error, and emit the spooled output to standard output and standard
> >> error in the order the lines were collected when the function returns
> >> a success, would be a good way to do so.
> >
> > That is actually the exact opposite of the intended usage: when any
> > `pick` in an interactive rebase succeeds, its output is discarded so
> > as not to bother the user. We show the complete output only when it
> > fails.
> 
> Oh, it makes sense, too, to show the output only when there is an error.
Thanks ;-)
> But in that case, there would be both messages meant for the
> standard output and also meant for the standard error, and we need
> some way to make sure they go to the right channel.

Not necessarily. Let's have a look at our existing code in git-rebase.sh:

	output () {
		case "$verbose" in
		'')
			output=$("$@" 2>&1 )
			status=$?
			test $status != 0 && printf "%s\n" "$output"
			return $status
			;;
		*)
			"$@"
			;;
		esac
	}

This incredibly well-named function (</sarcasm>, my fault: dfa49f3 (Shut "git rebase -i" up when no --verbose was given, 2007-07-23)) accumulates all output, both stdout and stderr, and shows it only in case of an error.

Crucially, *all* output goes to stdout. No distinction is being made between stdout and stderr.

This is the existing behavior of rebase -i.
Show 18 quoted lines
> I however do not think an array of <bool, const char *> is the only
> way to achieve that.  We can get away by a single strbuf that
> accumulates all output() that we have seen so far, i.e. "we only
> accumulate output() and ignore flush() as long as what we see are
> only from output()" mode.
> 
> Then the err() routine operating under this new mode can show what
> has been accumulated to the standard output (because with this tweak
> I am outlining here, by definition, the strbuf will only keep the
> output() material and not err() things), show the err() message, and
> switch back to the normal "we accumulate output() and honor flush()"
> mode.  Of course, when we are doing multiple rounds, the mode must
> be reset to "accumulate output and ignore flush" mode at the
> beginning of each rouhd.
> 
> That would give us "silence if there is no error, but if we are
> showing error, show them to the standard error, while giving
> non-error message to the standard output".

It all makes sense what you say. In case you want to preserve the channel in some future modification.

However, I am right now most concerned about keeping existing behavior as faithfully as possible (with the exception of execution speed, which I want to improve dramatically).

As such, it would be a serious mistake to implement that mode and use it in the rebase--helper: it would very likely cause regressions in existing scripts, probably even my own.

So I do understand your concern, and I agree that it would make for a fine design, in a different context than this patch series.

Ciao, Dscho

Junio C Hamano· Aug 2, 2016, 21:19 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 27 quoted lines
>> But in that case, there would be both messages meant for the
>> standard output and also meant for the standard error, and we need
>> some way to make sure they go to the right channel.
>
> Not necessarily. Let's have a look at our existing code in
> git-rebase.sh:
>
> 	output () {
> 		case "$verbose" in
> 		'')
> 			output=$("$@" 2>&1 )
> 			status=$?
> 			test $status != 0 && printf "%s\n" "$output"
> 			return $status
> 			;;
> 		*)
> 			"$@"
> 			;;
> 		esac
> 	}
>
> This incredibly well-named function (</sarcasm>, my fault: dfa49f3 (Shut "git
> rebase -i" up when no --verbose was given, 2007-07-23)) accumulates all
> output, both stdout and stderr, and shows it only in case of an error.
>
> Crucially, *all* output goes to stdout. No distinction is being made
> between stdout and stderr.
Show 6 quoted lines
> ...
> This is the existing behavior of rebase -i.
> ...
> As such, it would be a serious mistake to implement that mode and use it
> in the rebase--helper: it would very likely cause regressions in existing
> scripts, probably even my own.

Sounds like we are desperately trying to find an excuse to do a wrong thing by finding an existing piece of code that did a wrong thing already.

That leaves a bad taste in my mouth, but as "rebase -i" is meant to be an "interactive" command, I would imagine that nobody would have expected to run it as "git rebase -i >/dev/null" in order to view only the error messages (or vice versa with "2>errs").

So OK then, at least for now.
Johannes Schindelin· Aug 1, 2016, 09:35 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Hi Junio,
On Wed, 27 Jul 2016, Junio C Hamano wrote:
Show 19 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/merge-recursive.h b/merge-recursive.h
> > index d415724..340704c 100644
> > --- a/merge-recursive.h
> > +++ b/merge-recursive.h
> > @@ -13,7 +13,7 @@ struct merge_options {
> >  		MERGE_RECURSIVE_THEIRS
> >  	} recursive_variant;
> >  	const char *subtree_shift;
> > -	unsigned buffer_output : 1;
> > +	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
> >  	unsigned renormalize : 1;
> 
> Once a field ceases to be a boolean, it is OK not to squish it into
> a bitfield like this for a struct that we will have only a very
> small number of instances of.  Treating it just like "verbosity",
> which occupies a whole int even though it can only get up to 5 or
> so, would be more appropriate.
I changed it to an int.

Thanks, Dscho

Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 15/16] Ensure that the output buffer is released after calling merge_trees()

The recursive merge machinery accumulates its output in an output buffer, to be flushed at the end of merge_recursive(). At this point, we forgot to release the output buffer.

When calling merge_trees() (i.e. the non-recursive part of the recursive merge) directly, the output buffer is never flushed because the caller may be merge_recursive() which wants to flush the output itself.

For the same reason, merge_trees() cannot release the output buffer: it may still be needed.

Forgetting to release the output buffer did not matter much when running git-checkout, or git-merge-recursive, because we exited after the operation anyway. Ever since cherry-pick learned to pick a commit range, however, this memory leak had the potential of becoming a problem.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 1 +
 merge-recursive.c  | 2 ++
 sequencer.c        | 1 +
 3 files changed, 4 insertions(+)
Show changes to 3 files +4 −0

builtin/checkout.c, merge-recursive.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 07dea3b..8d852d4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -573,6 +573,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
+			strbuf_release(&o.obuf);
 			if (ret)
 				return ret;
 		}
diff --git a/merge-recursive.c b/merge-recursive.c
index 311cfa4..a16b150 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2078,6 +2078,8 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
 	flush_output(o);
+	if (o->buffer_output < 2)
+		strbuf_release(&o->obuf);
 	if (show(o, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       o->needed_rename_limit, 0);
diff --git a/sequencer.c b/sequencer.c
index 286a435..ec50519 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
 
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 27, 2016, 22:15 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 15/16] Ensure that the output buffer is released after calling merge_trees()

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 13 quoted lines
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 311cfa4..a16b150 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2078,6 +2078,8 @@ int merge_recursive(struct merge_options *o,
>  		commit_list_insert(h2, &(*result)->parents->next);
>  	}
>  	flush_output(o);
> +	if (o->buffer_output < 2)
> +		strbuf_release(&o->obuf);
>  	if (show(o, 2))
>  		diff_warn_rename_limit("merge.renamelimit",
>  				       o->needed_rename_limit, 0);

Other two hunks looked good, but this one I am not sure what is going on. It this were "if call-depth says we are called by another merge_recursive, do not discard the buffer", I would understand, but why does this have to be tied to o->buffer_output being "we buffer the output but not errors"?

Johannes Schindelin· Aug 1, 2016, 09:40 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 15/16] Ensure that the output buffer is released after calling merge_trees()

Hi Junio,
On Wed, 27 Jul 2016, Junio C Hamano wrote:
Show 21 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index 311cfa4..a16b150 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -2078,6 +2078,8 @@ int merge_recursive(struct merge_options *o,
> >  		commit_list_insert(h2, &(*result)->parents->next);
> >  	}
> >  	flush_output(o);
> > +	if (o->buffer_output < 2)
> > +		strbuf_release(&o->obuf);
> >  	if (show(o, 2))
> >  		diff_warn_rename_limit("merge.renamelimit",
> >  				       o->needed_rename_limit, 0);
> 
> Other two hunks looked good, but this one I am not sure what is
> going on.  It this were "if call-depth says we are called by another
> merge_recursive, do not discard the buffer", I would understand, but
> why does this have to be tied to o->buffer_output being "we buffer
> the output but not errors"?
Good point. I changed it to test for !o->call_depth in addition.

We must not release the output buffer here if buffer_output >= 2: the level "2" of the buffer_output field says that the caller of the recursive merge expects the output to remain in the buffer until after `merge_recursive()` returns.

Ciao, Dscho

Johannes Schindelin· Jul 26, 2016, 16:06 UTC · re: Johannes Schindelin · lore

[PATCH v5 16/16] merge-recursive: flush output buffer even when erroring out

Ever since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we had a problem: When the merge failed in a fatal way, all regular output was swallowed because we called die() and did not get a chance to drain the output buffers.

To fix this, several modifications were necessary:
- we needed to stop die()ing, to give callers a chance to do something
  when an error occurred (in this case, flush the output buffers),
- we needed to delay printing the error message so that the caller can
  print the buffered output before that, and
- we needed to make sure that the output buffers are flushed even when
  the return value indicates an error.

The first two changes were introduced through earlier commits in this patch series, and this commit addresses the third one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to merge-recursive.c +1 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index a16b150..66e93e0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2069,6 +2069,7 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	flush_output(o);
 	if (clean < 0)
 		return clean;
 
@@ -2077,7 +2078,6 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(o);
 	if (o->buffer_output < 2)
 		strbuf_release(&o->obuf);
 	if (show(o, 2))
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Jul 27, 2016, 22:20 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 16/16] merge-recursive: flush output buffer even when erroring out

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 11 quoted lines
> diff --git a/merge-recursive.c b/merge-recursive.c
> index a16b150..66e93e0 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2069,6 +2069,7 @@ int merge_recursive(struct merge_options *o,
>  	o->ancestor = "merged common ancestors";
>  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
>  			    &mrtree);
> +	flush_output(o);
>  	if (clean < 0)
>  		return clean;

This is of course a good change, but we need to assume that no further output is made from the remainder of the function for the change in the next hunk to remove the existing flush to be correct.

And once we assume that, then the "we no longer need this buffer, so release it" added in 15/16 can also move here, right?

I am wondering if there is a low-impact way to make sure that assumption will not be broken.

Show 8 quoted lines
> @@ -2077,7 +2078,6 @@ int merge_recursive(struct merge_options *o,
>  		commit_list_insert(h1, &(*result)->parents);
>  		commit_list_insert(h2, &(*result)->parents->next);
>  	}
> -	flush_output(o);
>  	if (o->buffer_output < 2)
>  		strbuf_release(&o->obuf);
>  	if (show(o, 2))
Johannes Schindelin· Aug 1, 2016, 09:49 UTC · re: Junio C Hamano · lore

Re: [PATCH v5 16/16] merge-recursive: flush output buffer even when erroring out

Hi Junio,
On Wed, 27 Jul 2016, Junio C Hamano wrote:
Show 17 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index a16b150..66e93e0 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -2069,6 +2069,7 @@ int merge_recursive(struct merge_options *o,
> >  	o->ancestor = "merged common ancestors";
> >  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
> >  			    &mrtree);
> > +	flush_output(o);
> >  	if (clean < 0)
> >  		return clean;
> 
> This is of course a good change, but we need to assume that no
> further output is made from the remainder of the function for the
> change in the next hunk to remove the existing flush to be correct.

Please note that nothing prevents the code further down from adding more output. All we do here is flushing the output *so far*, in case we return an error. And of course nothing gets flushed if buffer_output == 2, because that value states that the caller wants to take care of displaying the output herself.

But you made me realize that I cannot simply *move* the flush_output() call here, in case that code in between will eventually add output.

Thanks, Dscho

Junio C Hamano· Aug 1, 2016, 18:32 UTC · re: Johannes Schindelin · lore

Re: [PATCH v5 16/16] merge-recursive: flush output buffer even when erroring out

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 6 quoted lines
>> This is of course a good change, but we need to assume that no
>> further output is made from the remainder of the function for the
>> change in the next hunk to remove the existing flush to be correct.
> ...
> But you made me realize that I cannot simply *move* the flush_output()
> call here, in case that code in between will eventually add output.

Yup, that removal of the original one was the only thing I was pointing out.

Johannes Schindelin· Aug 1, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH v6 00/16] Use merge_recursive() directly in the builtin am

This is the sixth iteration of the long-awaited re-roll of the attempt to avoid spawning merge-recursive from the builtin am and use merge_recursive() directly instead.

The *real* reason for the reroll is that I need a libified recursive merge to accelerate the interactive rebase by teaching the sequencer to do rebase -i's grunt work. Coming with a very nice 3x-5x speedup of `rebase -i`.

In this endeavor, we need to be extra careful to retain backwards compatibility. The test script t6022-merge-rename.sh, for example, verifies that `git pull` exits with status 128 in case of a fatal error. To that end, we need to make sure that fatal errors are handled by existing (builtin) users via exit(128) (or die(), which calls exit(128) at the end). New users (such as a builtin helper doing rebase -i's grunt work) may want to print some helpful advice what happened and how to get out of this mess before erroring out.

The changes relative to the fifth iteration of this patch series:
- the commit message that talked about swallowing messages was improved
- the va_start()/va_end() statements were moved closer to the vararg usage
- the `buffer_output` field is no longer a bit field because it is now
  logically more like the verbosity level, which we traditionally keep
  as a full `int`
- the flush_output() call is no longer moved, but instead added to the
  code path when we return early
- the commit message saying that we speculated about stdout's buffer
  size now instead states that we made an assumption

This patch series touches rather important code. I appreciate thorough reviews with a focus on the critical parts of the code, those that could result in regressions.

Johannes Schindelin (16):
  t5520: verify that `pull --rebase` shows the helpful advice when
    failing
  Report bugs consistently
  Avoid translating bug messages
  merge-recursive: clarify code in was_tracked()
  Prepare the builtins for a libified merge_recursive()
  merge_recursive: abort properly upon errors
  merge-recursive: avoid returning a wholesale struct
  merge-recursive: allow write_tree_from_memory() to error out
  merge-recursive: handle return values indicating errors
  merge-recursive: switch to returning errors instead of dying
  am -3: use merge_recursive() directly again
  merge-recursive: flush output buffer before printing error messages
  merge-recursive: write the commit title in one go
  merge-recursive: offer an option to retain the output in 'obuf'
  Ensure that the output buffer is released after calling merge_trees()
  merge-recursive: flush output buffer even when erroring out
 builtin/am.c           |  62 ++----
 builtin/checkout.c     |   5 +-
 builtin/ls-files.c     |   3 +-
 builtin/merge.c        |   2 +
 builtin/update-index.c |   2 +-
 grep.c                 |   8 +-
 imap-send.c            |   2 +-
 merge-recursive.c      | 578 +++++++++++++++++++++++++++++--------------------
 merge-recursive.h      |   2 +-
 sequencer.c            |   5 +
 sha1_file.c            |   4 +-
 t/t5520-pull.sh        |  32 +++
 trailer.c              |   2 +-
 transport.c            |   2 +-
 wt-status.c            |   4 +-
 15 files changed, 419 insertions(+), 294 deletions(-)
Published-As: https://github.com/dscho/git/releases/tag/am-3-merge-recursive-direct-v6
Interdiff vs v5:
 diff --git a/merge-recursive.c b/merge-recursive.c
 index 66e93e0..c9e4dbc 100644
 --- a/merge-recursive.c
 +++ b/merge-recursive.c
 @@ -35,21 +35,21 @@ static int err(struct merge_options *o, const char *err, ...)
  {
  	va_list params;
  
 -	va_start(params, err);
  	if (o->buffer_output < 2)
  		flush_output(o);
  	else {
  		strbuf_complete(&o->obuf, '\n');
  		strbuf_addstr(&o->obuf, "error: ");
  	}
 +	va_start(params, err);
  	strbuf_vaddf(&o->obuf, err, params);
 +	va_end(params);
  	if (o->buffer_output > 1)
  		strbuf_addch(&o->obuf, '\n');
  	else {
  		error("%s", o->obuf.buf);
  		strbuf_reset(&o->obuf);
  	}
 -	va_end(params);
  
  	return -1;
  }
 @@ -2069,16 +2069,18 @@ int merge_recursive(struct merge_options *o,
  	o->ancestor = "merged common ancestors";
  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
  			    &mrtree);
 -	flush_output(o);
 -	if (clean < 0)
 +	if (clean < 0) {
 +		flush_output(o);
  		return clean;
 +	}
  
  	if (o->call_depth) {
  		*result = make_virtual_commit(mrtree, "merged tree");
  		commit_list_insert(h1, &(*result)->parents);
  		commit_list_insert(h2, &(*result)->parents->next);
  	}
 -	if (o->buffer_output < 2)
 +	flush_output(o);
 +	if (!o->call_depth && o->buffer_output < 2)
  		strbuf_release(&o->obuf);
  	if (show(o, 2))
  		diff_warn_rename_limit("merge.renamelimit",
 diff --git a/merge-recursive.h b/merge-recursive.h
 index 340704c..735343b 100644
 --- a/merge-recursive.h
 +++ b/merge-recursive.h
 @@ -13,7 +13,7 @@ struct merge_options {
  		MERGE_RECURSIVE_THEIRS
  	} recursive_variant;
  	const char *subtree_shift;
 -	unsigned buffer_output : 2; /* 1: output at end, 2: keep buffered */
 +	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
  	unsigned renormalize : 1;
  	long xdl_opts;
  	int verbosity;
-- 
2.9.0.281.g286a8d9

base-commit: f8f7adce9fc50a11a764d57815602dcb818d1816
Johannes Schindelin· Aug 1, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH v6 01/16] t5520: verify that `pull --rebase` shows the helpful advice when failing

It was noticed by Brendan Forster last October that the builtin `git am` regressed on that. Our hot fix reverted to spawning the recursive merge instead of using it as a library function.

As we are about to revert that hot fix, after making the recursive merge a true library function (i.e. a function that does not die() in case of "normal" errors), let's add a test that verifies that we do not regress on the same problem which made the hot fix necessary in the first place.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t5520-pull.sh | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
Show changes to t/t5520-pull.sh +32 −0
diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
index 37ebbcf..6ad37b5 100755
--- a/t/t5520-pull.sh
+++ b/t/t5520-pull.sh
@@ -255,6 +255,38 @@ test_expect_success '--rebase' '
 	test new = "$(git show HEAD:file2)"
 '
 
+test_expect_success '--rebase with conflicts shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b seq &&
+	test_seq 5 >seq.txt &&
+	git add seq.txt &&
+	test_tick &&
+	git commit -m "Add seq.txt" &&
+	echo 6 >>seq.txt &&
+	test_tick &&
+	git commit -m "Append to seq.txt" seq.txt &&
+	git checkout -b with-conflicts HEAD^ &&
+	echo conflicting >>seq.txt &&
+	test_tick &&
+	git commit -m "Create conflict" seq.txt &&
+	test_must_fail git pull --rebase . seq 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
+test_expect_success 'failed --rebase shows advice' '
+	test_when_finished "git rebase --abort; git checkout -f to-rebase" &&
+	git checkout -b diverging &&
+	test_commit attributes .gitattributes "* text=auto" attrs &&
+	sha1="$(printf "1\\r\\n" | git hash-object -w --stdin)" &&
+	git update-index --cacheinfo 0644 $sha1 file &&
+	git commit -m v1-with-cr &&
+	# force checkout because `git reset --hard` will not leave clean `file`
+	git checkout -f -b fails-to-rebase HEAD^ &&
+	test_commit v2-without-cr file "2" file2-lf &&
+	test_must_fail git pull --rebase . diverging 2>err >out &&
+	grep "When you have resolved this problem" out
+'
+
 test_expect_success '--rebase fails with multiple branches' '
 	git reset --hard before-rebase &&
 	test_must_fail git pull --rebase . copy master 2>err &&
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:36 UTC · re: Johannes Schindelin · lore

[PATCH v6 02/16] Report bugs consistently

The vast majority of error messages in Git's source code which report a bug use the convention to prefix the message with "BUG:".

As part of cleaning up merge-recursive to stop die()ing except in case of detected bugs, let's just make the remainder of the bug reports consistent with the de facto rule.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/ls-files.c     |  3 ++-
 builtin/update-index.c |  2 +-
 grep.c                 |  8 ++++----
 imap-send.c            |  2 +-
 merge-recursive.c      | 15 +++++++--------
 sha1_file.c            |  4 ++--
 trailer.c              |  2 +-
 transport.c            |  2 +-
 wt-status.c            |  4 ++--
 9 files changed, 21 insertions(+), 21 deletions(-)
Show changes to 9 files +21 −21

builtin/ls-files.c, builtin/update-index.c, grep.c, imap-send.c, merge-recursive.c, sha1_file.c, trailer.c, transport.c, wt-status.c

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f02e3d2..00ea91a 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -118,7 +118,8 @@ static void show_killed_files(struct dir_struct *dir)
 				 */
 				pos = cache_name_pos(ent->name, ent->len);
 				if (0 <= pos)
-					die("bug in show-killed-files");
+					die("BUG: killed-file %.*s not found",
+						ent->len, ent->name);
 				pos = -pos - 1;
 				while (pos < active_nr &&
 				       ce_stage(active_cache[pos]))
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 6cdfd5f..ba04b19 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1146,7 +1146,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		report(_("Untracked cache enabled for '%s'"), get_git_work_tree());
 		break;
 	default:
-		die("Bug: bad untracked_cache value: %d", untracked_cache);
+		die("BUG: bad untracked_cache value: %d", untracked_cache);
 	}
 
 	if (active_cache_changed) {
diff --git a/grep.c b/grep.c
index 394c856..22cbb73 100644
--- a/grep.c
+++ b/grep.c
@@ -693,10 +693,10 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 	for (p = opt->header_list; p; p = p->next) {
 		if (p->token != GREP_PATTERN_HEAD)
-			die("bug: a non-header pattern in grep header list.");
+			die("BUG: a non-header pattern in grep header list.");
 		if (p->field < GREP_HEADER_FIELD_MIN ||
 		    GREP_HEADER_FIELD_MAX <= p->field)
-			die("bug: unknown header field %d", p->field);
+			die("BUG: unknown header field %d", p->field);
 		compile_regexp(p, opt);
 	}
 
@@ -709,7 +709,7 @@ static struct grep_expr *prep_header_patterns(struct grep_opt *opt)
 
 		h = compile_pattern_atom(&pp);
 		if (!h || pp != p->next)
-			die("bug: malformed header expr");
+			die("BUG: malformed header expr");
 		if (!header_group[p->field]) {
 			header_group[p->field] = h;
 			continue;
@@ -1514,7 +1514,7 @@ static int grep_source_1(struct grep_opt *opt, struct grep_source *gs, int colle
 		case GREP_BINARY_TEXT:
 			break;
 		default:
-			die("bug: unknown binary handling mode");
+			die("BUG: unknown binary handling mode");
 		}
 	}
 
diff --git a/imap-send.c b/imap-send.c
index db0fafe..0f5f476 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -511,7 +511,7 @@ static int nfsnprintf(char *buf, int blen, const char *fmt, ...)
 
 	va_start(va, fmt);
 	if (blen <= 0 || (unsigned)(ret = vsnprintf(buf, blen, fmt, va)) >= (unsigned)blen)
-		die("Fatal: buffer too small. Please report a bug.");
+		die("BUG: buffer too small. Please report a bug.");
 	va_end(va);
 	return ret;
 }
diff --git a/merge-recursive.c b/merge-recursive.c
index a4a1195..4338b73 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -268,7 +268,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
 					(int)ce_namelen(ce), ce->name);
 		}
-		die("Bug in merge-recursive.c");
+		die("BUG: unmerged index entries in merge-recursive.c");
 	}
 
 	if (!active_cache_tree)
@@ -966,9 +966,8 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
-		} else {
-			die(_("unsupported object type in the tree"));
-		}
+		} else
+			die(_("BUG: unsupported object type in the tree"));
 	}
 
 	return result;
@@ -1354,7 +1353,7 @@ static int process_renames(struct merge_options *o,
 			const char *ren2_dst = ren2->pair->two->path;
 			enum rename_type rename_type;
 			if (strcmp(ren1_src, ren2_src) != 0)
-				die("ren1_src != ren2_src");
+				die("BUG: ren1_src != ren2_src");
 			ren2->dst_entry->processed = 1;
 			ren2->processed = 1;
 			if (strcmp(ren1_dst, ren2_dst) != 0) {
@@ -1388,7 +1387,7 @@ static int process_renames(struct merge_options *o,
 			ren2 = lookup->util;
 			ren2_dst = ren2->pair->two->path;
 			if (strcmp(ren1_dst, ren2_dst) != 0)
-				die("ren1_dst != ren2_dst");
+				die("BUG: ren1_dst != ren2_dst");
 
 			clean_merge = 0;
 			ren2->processed = 1;
@@ -1812,7 +1811,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("Fatal merge failure, shouldn't happen."));
+		die(_("BUG: fatal merge failure, shouldn't happen."));
 
 	return clean_merge;
 }
@@ -1870,7 +1869,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("Unprocessed path??? %s"),
+				die(_("BUG: unprocessed path??? %s"),
 				    entries->items[i].string);
 		}
 
diff --git a/sha1_file.c b/sha1_file.c
index cb571ac..ebc640e 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -795,7 +795,7 @@ void close_all_packs(void)
 
 	for (p = packed_git; p; p = p->next)
 		if (p->do_not_close)
-			die("BUG! Want to close pack marked 'do-not-close'");
+			die("BUG: want to close pack marked 'do-not-close'");
 		else
 			close_pack(p);
 }
@@ -2330,7 +2330,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 	case OBJ_OFS_DELTA:
 	case OBJ_REF_DELTA:
 		if (data)
-			die("BUG in unpack_entry: left loop at a valid delta");
+			die("BUG: unpack_entry: left loop at a valid delta");
 		break;
 	case OBJ_COMMIT:
 	case OBJ_TREE:
diff --git a/trailer.c b/trailer.c
index 8e48a5c..c6ea9ac 100644
--- a/trailer.c
+++ b/trailer.c
@@ -562,7 +562,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
-		die("internal bug in trailer.c");
+		die("BUG: trailer.c: unhandled type %d", type);
 	}
 	return 0;
 }
diff --git a/transport.c b/transport.c
index b233e3e..04d9454 100644
--- a/transport.c
+++ b/transport.c
@@ -566,7 +566,7 @@ void transport_take_over(struct transport *transport,
 	struct git_transport_data *data;
 
 	if (!transport->smart_options)
-		die("Bug detected: Taking over transport requires non-NULL "
+		die("BUG: taking over transport requires non-NULL "
 		    "smart_options field.");
 
 	data = xcalloc(1, sizeof(*data));
diff --git a/wt-status.c b/wt-status.c
index 19cbc39..f8ae0c2 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -263,7 +263,7 @@ static const char *wt_status_unmerged_status_string(int stagemask)
 	case 7:
 		return _("both modified:");
 	default:
-		die("bug: unhandled unmerged status %x", stagemask);
+		die("BUG: unhandled unmerged status %x", stagemask);
 	}
 }
 
@@ -388,7 +388,7 @@ static void wt_status_print_change_data(struct wt_status *s,
 	status_printf(s, color(WT_STATUS_HEADER, s), "\t");
 	what = wt_status_diff_status_string(status);
 	if (!what)
-		die("bug: unhandled diff status %c", status);
+		die("BUG: unhandled diff status %c", status);
 	len = label_width - utf8_strwidth(what);
 	assert(len >= 0);
 	if (status == DIFF_STATUS_COPIED || status == DIFF_STATUS_RENAMED)
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 05/16] Prepare the builtins for a libified merge_recursive()

Previously, callers of merge_trees() or merge_recursive() expected that code to die() with an error message. This used to be okay because we called those commands from scripts, and had a chance to print out a message in case the command failed fatally (read: with exit code 128).

As scripting incurs its own set of problems (portability, speed, idiosynchracies of different shells, limited data structures leading to inefficient code), we are converting more and more of these scripts into builtins, using library functions directly.

We already tried to use merge_recursive() directly in the builtin git-am, for example. Unfortunately, we had to roll it back temporarily because some of the code in merge-recursive.c still deemed it okay to call die(), when the builtin am code really wanted to print out a useful advice after the merge failed fatally. In the next commits, we want to fix that.

The code touched by this commit expected merge_trees() to die() with some useful message when there is an error condition, but merge_trees() is going to be improved by converting all die() calls to return error() instead (i.e. return value -1 after printing out the message as before), so that the caller can react more flexibly.

This is a step to prepare for the version of merge_trees() that no longer dies, even if we just imitate the previous behavior by calling exit(128): this is what callers of e.g. `git merge` have come to expect.

Note that the callers of the sequencer (revert and cherry-pick) already fail fast even for the return value -1; The only difference is that they now get a chance to say "<command> failed".

A caller of merge_trees() might want handle error messages themselves (or even suppress them). As this patch is already complex enough, we leave that change for a later patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 4 +++-
 builtin/merge.c    | 2 ++
 sequencer.c        | 4 ++++
 3 files changed, 9 insertions(+), 1 deletion(-)
Show changes to 3 files +9 −1

builtin/checkout.c, builtin/merge.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 27c1a05..07dea3b 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -567,8 +567,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			o.ancestor = old->name;
 			o.branch1 = new->name;
 			o.branch2 = "local";
-			merge_trees(&o, new->commit->tree, work,
+			ret = merge_trees(&o, new->commit->tree, work,
 				old->commit->tree, &result);
+			if (ret < 0)
+				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
 			if (ret)
diff --git a/builtin/merge.c b/builtin/merge.c
index 19b3bc2..148a9a5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -673,6 +673,8 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
+		if (clean < 0)
+			exit(128);
 		if (active_cache_changed &&
 		    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 			die (_("unable to write %s"), get_index_file());
diff --git a/sequencer.c b/sequencer.c
index cdfac82..286a435 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (clean < 0)
+		return clean;
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
@@ -559,6 +561,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
+		if (res < 0)
+			return res;
 		write_message(&msgbuf, git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Aug 1, 2016, 18:40 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 05/16] Prepare the builtins for a libified merge_recursive()

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 7 quoted lines
> Previously, callers of merge_trees() or merge_recursive() expected that
> code to die() with an error message. This used to be okay because we
> called those commands from scripts, and had a chance to print out a
> message in case the command failed fatally (read: with exit code 128).
>
> As scripting incurs its own set of problems (portability, speed,
> idiosynchracies of different shells, limited data structures leading to

I think I typofixed this when I queued the previous one on 'pu' already, but s/synch/sync/;

Johannes Schindelin· Aug 2, 2016, 08:02 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 05/16] Prepare the builtins for a libified merge_recursive()

Hi Junio,
On Mon, 1 Aug 2016, Junio C Hamano wrote:
Show 12 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Previously, callers of merge_trees() or merge_recursive() expected that
> > code to die() with an error message. This used to be okay because we
> > called those commands from scripts, and had a chance to print out a
> > message in case the command failed fatally (read: with exit code 128).
> >
> > As scripting incurs its own set of problems (portability, speed,
> > idiosynchracies of different shells, limited data structures leading to
> 
> I think I typofixed this when I queued the previous one on 'pu'
> already, but s/synch/sync/; 
Whoops. Fixed locally.

Ciao, Dscho

Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 04/16] merge-recursive: clarify code in was_tracked()

It can be puzzling to see that was_tracked() asks to get an index entry by name, but does not take a negative return value for an answer.

The reason we have to do this is that cache_name_pos() only looks for entries in stage 0, even if nobody asked for any stage in particular.

Let's rewrite the logic a little bit, to handle the easy case early: if cache_name_pos() returned a non-negative position, we know it is a match, and we do not even have to compare the name again (cache_name_pos() did that for us already). We can say right away: yes, this file was tracked.

Only if there was no exact match do we need to look harder for any matching entry in stage 2.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)
Show changes to merge-recursive.c +14 −16
diff --git a/merge-recursive.c b/merge-recursive.c
index 1b6db87..3a652b7 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -667,23 +667,21 @@ static int was_tracked(const char *path)
 {
 	int pos = cache_name_pos(path, strlen(path));
 
-	if (pos < 0)
-		pos = -1 - pos;
-	while (pos < active_nr &&
-	       !strcmp(path, active_cache[pos]->name)) {
-		/*
-		 * If stage #0, it is definitely tracked.
-		 * If it has stage #2 then it was tracked
-		 * before this merge started.  All other
-		 * cases the path was not tracked.
-		 */
-		switch (ce_stage(active_cache[pos])) {
-		case 0:
-		case 2:
+	if (0 <= pos)
+		/* we have been tracking this path */
+		return 1;
+
+	/*
+	 * Look for an unmerged entry for the path,
+	 * specifically stage #2, which would indicate
+	 * that "our" side before the merge started
+	 * had the path tracked (and resulted in a conflict).
+	 */
+	for (pos = -1 - pos;
+	     pos < active_nr && !strcmp(path, active_cache[pos]->name);
+	     pos++)
+		if (ce_stage(active_cache[pos]) == 2)
 			return 1;
-		}
-		pos++;
-	}
 	return 0;
 }
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:43 UTC · re: Johannes Schindelin · lore

[PATCH v6 03/16] Avoid translating bug messages

While working on the patch series that avoids die()ing in recursive merges, the issue came up that bug reports (i.e. die("BUG: ...") constructs) should never be translated, as the target audience is the Git developer community, not necessarily the current user, and hence a translated message would make it *harder* to address the problem.

So let's stop translating the obvious ones. As it is really, really outside the purview of this patch series to see whether there are more die() statements that report bugs and are currently translated, that task is left for another day and patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Show changes to merge-recursive.c +3 −3
diff --git a/merge-recursive.c b/merge-recursive.c
index 4338b73..1b6db87 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -967,7 +967,7 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 			if (!oid_eq(&a->oid, &b->oid))
 				result.clean = 0;
 		} else
-			die(_("BUG: unsupported object type in the tree"));
+			die("BUG: unsupported object type in the tree");
 	}
 
 	return result;
@@ -1811,7 +1811,7 @@ static int process_entry(struct merge_options *o,
 		 */
 		remove_file(o, 1, path, !a_mode);
 	} else
-		die(_("BUG: fatal merge failure, shouldn't happen."));
+		die("BUG: fatal merge failure, shouldn't happen.");
 
 	return clean_merge;
 }
@@ -1869,7 +1869,7 @@ int merge_trees(struct merge_options *o,
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
 			if (!e->processed)
-				die(_("BUG: unprocessed path??? %s"),
+				die("BUG: unprocessed path??? %s",
 				    entries->items[i].string);
 		}
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 11/16] am -3: use merge_recursive() directly again

Last October, we had to change this code to run `git merge-recursive` in a child process: git-am wants to print some helpful advice when the merge failed, but the code in question was not prepared to return, it die()d instead.

We are finally at a point when the code *is* prepared to return errors, and can avoid the child process again.

This reverts commit c63d4b2 (am -3: do not let failed merge from completing the error codepath, 2015-10-09), with the necessary changes to adjust for the fact that Git's source code changed in the meantime (such as: using OIDs instead of hashes in the recursive merge, and a removed gender bias).

Note: the code now calls merge_recursive_generic() again. Unlike
merge_trees() and merge_recursive(), this function returns 0 upon success,
as most of Git's functions. Therefore, the error value -1 naturally is
handled correctly, and we do not have to take care of it specifically.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/am.c | 62 +++++++++++++++++++++---------------------------------------
 1 file changed, 22 insertions(+), 40 deletions(-)
Show changes to builtin/am.c +22 −40
diff --git a/builtin/am.c b/builtin/am.c
index b77bf11..cfb79ea 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1579,47 +1579,18 @@ static int build_fake_ancestor(const struct am_state *state, const char *index_f
 }
 
 /**
- * Do the three-way merge using fake ancestor, their tree constructed
- * from the fake ancestor and the postimage of the patch, and our
- * state.
- */
-static int run_fallback_merge_recursive(const struct am_state *state,
-					unsigned char *orig_tree,
-					unsigned char *our_tree,
-					unsigned char *their_tree)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	int status;
-
-	cp.git_cmd = 1;
-
-	argv_array_pushf(&cp.env_array, "GITHEAD_%s=%.*s",
-			 sha1_to_hex(their_tree), linelen(state->msg), state->msg);
-	if (state->quiet)
-		argv_array_push(&cp.env_array, "GIT_MERGE_VERBOSITY=0");
-
-	argv_array_push(&cp.args, "merge-recursive");
-	argv_array_push(&cp.args, sha1_to_hex(orig_tree));
-	argv_array_push(&cp.args, "--");
-	argv_array_push(&cp.args, sha1_to_hex(our_tree));
-	argv_array_push(&cp.args, sha1_to_hex(their_tree));
-
-	status = run_command(&cp) ? (-1) : 0;
-	discard_cache();
-	read_cache();
-	return status;
-}
-
-/**
  * Attempt a threeway merge, using index_path as the temporary index.
  */
 static int fall_back_threeway(const struct am_state *state, const char *index_path)
 {
-	unsigned char orig_tree[GIT_SHA1_RAWSZ], their_tree[GIT_SHA1_RAWSZ],
-		      our_tree[GIT_SHA1_RAWSZ];
+	struct object_id orig_tree, their_tree, our_tree;
+	const struct object_id *bases[1] = { &orig_tree };
+	struct merge_options o;
+	struct commit *result;
+	char *their_tree_name;
 
-	if (get_sha1("HEAD", our_tree) < 0)
-		hashcpy(our_tree, EMPTY_TREE_SHA1_BIN);
+	if (get_oid("HEAD", &our_tree) < 0)
+		hashcpy(our_tree.hash, EMPTY_TREE_SHA1_BIN);
 
 	if (build_fake_ancestor(state, index_path))
 		return error("could not build fake ancestor");
@@ -1627,7 +1598,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	discard_cache();
 	read_cache_from(index_path);
 
-	if (write_index_as_tree(orig_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(orig_tree.hash, &the_index, index_path, 0, NULL))
 		return error(_("Repository lacks necessary blobs to fall back on 3-way merge."));
 
 	say(state, stdout, _("Using index info to reconstruct a base tree..."));
@@ -1643,7 +1614,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		init_revisions(&rev_info, NULL);
 		rev_info.diffopt.output_format = DIFF_FORMAT_NAME_STATUS;
 		diff_opt_parse(&rev_info.diffopt, &diff_filter_str, 1, rev_info.prefix);
-		add_pending_sha1(&rev_info, "HEAD", our_tree, 0);
+		add_pending_sha1(&rev_info, "HEAD", our_tree.hash, 0);
 		diff_setup_done(&rev_info.diffopt);
 		run_diff_index(&rev_info, 1);
 	}
@@ -1652,7 +1623,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 		return error(_("Did you hand edit your patch?\n"
 				"It does not apply to blobs recorded in its index."));
 
-	if (write_index_as_tree(their_tree, &the_index, index_path, 0, NULL))
+	if (write_index_as_tree(their_tree.hash, &the_index, index_path, 0, NULL))
 		return error("could not write tree");
 
 	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
@@ -1668,11 +1639,22 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	 * changes.
 	 */
 
-	if (run_fallback_merge_recursive(state, orig_tree, our_tree, their_tree)) {
+	init_merge_options(&o);
+
+	o.branch1 = "HEAD";
+	their_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
+	o.branch2 = their_tree_name;
+
+	if (state->quiet)
+		o.verbosity = 0;
+
+	if (merge_recursive_generic(&o, &our_tree, &their_tree, 1, bases, &result)) {
 		rerere(state->allow_rerere_autoupdate);
+		free(their_tree_name);
 		return error(_("Failed to merge in the changes."));
 	}
 
+	free(their_tree_name);
 	return 0;
 }
 
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 09/16] merge-recursive: handle return values indicating errors

We are about to libify the recursive merge machinery, where we only die() in case of a bug or memory contention. To that end, we must heed negative return values as indicating errors.

This requires our functions to be careful to pass through error conditions in call chains, and for quite a few functions this means that they have to return values to begin with.

The next step will be to convert the places where we currently die() to return negative values (read: -1) instead.

Note that we ignore errors reported by make_room_for_path(), consistent with the previous behavior (update_file_flags() used the return value of make_room_for_path() only to indicate an early return, but not a fatal error): if the error is really a fatal error, we will notice later; If not, it was not that serious a problem to begin with. (Witnesses in favor of this reasoning are t4151-am-abort and t7610-mergetool, which would start failing if we stopped on errors reported by make_room_for_path()).

Also note: while this patch makes the code slightly less readable in update_file_flags() (we introduce a new "goto free_buf;" instead of an explicit "free(buf); return;"), it is a preparatory change for the next patch where we will convert all of the die() calls in the same function to go through the free_buf return path instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 252 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 150 insertions(+), 102 deletions(-)
Show changes to merge-recursive.c +150 −102
diff --git a/merge-recursive.c b/merge-recursive.c
index 1f86338..6beb1e4 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -742,12 +742,12 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	return error(msg, path, _(": perhaps a D/F conflict?"));
 }
 
-static void update_file_flags(struct merge_options *o,
-			      const struct object_id *oid,
-			      unsigned mode,
-			      const char *path,
-			      int update_cache,
-			      int update_wd)
+static int update_file_flags(struct merge_options *o,
+			     const struct object_id *oid,
+			     unsigned mode,
+			     const char *path,
+			     int update_cache,
+			     int update_wd)
 {
 	if (o->call_depth)
 		update_wd = 0;
@@ -783,8 +783,7 @@ static void update_file_flags(struct merge_options *o,
 
 		if (make_room_for_path(o, path) < 0) {
 			update_wd = 0;
-			free(buf);
-			goto update_index;
+			goto free_buf;
 		}
 		if (S_ISREG(mode) || (!has_symlinks && S_ISLNK(mode))) {
 			int fd;
@@ -807,20 +806,22 @@ static void update_file_flags(struct merge_options *o,
 		} else
 			die(_("do not know what to do with %06o %s '%s'"),
 			    mode, oid_to_hex(oid), path);
+ free_buf:
 		free(buf);
 	}
  update_index:
 	if (update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+	return 0;
 }
 
-static void update_file(struct merge_options *o,
-			int clean,
-			const struct object_id *oid,
-			unsigned mode,
-			const char *path)
+static int update_file(struct merge_options *o,
+		       int clean,
+		       const struct object_id *oid,
+		       unsigned mode,
+		       const char *path)
 {
-	update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
+	return update_file_flags(o, oid, mode, path, o->call_depth || clean, !o->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1019,7 +1020,7 @@ static int merge_file_one(struct merge_options *o,
 	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
-static void handle_change_delete(struct merge_options *o,
+static int handle_change_delete(struct merge_options *o,
 				 const char *path,
 				 const struct object_id *o_oid, int o_mode,
 				 const struct object_id *a_oid, int a_mode,
@@ -1027,6 +1028,7 @@ static void handle_change_delete(struct merge_options *o,
 				 const char *change, const char *change_past)
 {
 	char *renamed = NULL;
+	int ret = 0;
 	if (dir_in_way(path, !o->call_depth)) {
 		renamed = unique_path(o, path, a_oid ? o->branch1 : o->branch2);
 	}
@@ -1037,21 +1039,23 @@ static void handle_change_delete(struct merge_options *o,
 		 * correct; since there is no true "middle point" between
 		 * them, simply reuse the base version for virtual merge base.
 		 */
-		remove_file_from_cache(path);
-		update_file(o, 0, o_oid, o_mode, renamed ? renamed : path);
+		ret = remove_file_from_cache(path);
+		if (!ret)
+			ret = update_file(o, 0, o_oid, o_mode,
+					  renamed ? renamed : path);
 	} else if (!a_oid) {
 		if (!renamed) {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path);
-			update_file(o, 0, b_oid, b_mode, path);
+			ret = update_file(o, 0, b_oid, b_mode, path);
 		} else {
 			output(o, 1, _("CONFLICT (%s/delete): %s deleted in %s "
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch1, change_past,
 			       o->branch2, o->branch2, path, renamed);
-			update_file(o, 0, b_oid, b_mode, renamed);
+			ret = update_file(o, 0, b_oid, b_mode, renamed);
 		}
 	} else {
 		if (!renamed) {
@@ -1064,7 +1068,7 @@ static void handle_change_delete(struct merge_options *o,
 			       "and %s in %s. Version %s of %s left in tree at %s."),
 			       change, path, o->branch2, change_past,
 			       o->branch1, o->branch1, path, renamed);
-			update_file(o, 0, a_oid, a_mode, renamed);
+			ret = update_file(o, 0, a_oid, a_mode, renamed);
 		}
 		/*
 		 * No need to call update_file() on path when !renamed, since
@@ -1074,9 +1078,11 @@ static void handle_change_delete(struct merge_options *o,
 		 */
 	}
 	free(renamed);
+
+	return ret;
 }
 
-static void conflict_rename_delete(struct merge_options *o,
+static int conflict_rename_delete(struct merge_options *o,
 				   struct diff_filepair *pair,
 				   const char *rename_branch,
 				   const char *other_branch)
@@ -1096,21 +1102,20 @@ static void conflict_rename_delete(struct merge_options *o,
 		b_mode = dest->mode;
 	}
 
-	handle_change_delete(o,
-			     o->call_depth ? orig->path : dest->path,
-			     &orig->oid, orig->mode,
-			     a_oid, a_mode,
-			     b_oid, b_mode,
-			     _("rename"), _("renamed"));
-
-	if (o->call_depth) {
-		remove_file_from_cache(dest->path);
-	} else {
-		update_stages(dest->path, NULL,
-			      rename_branch == o->branch1 ? dest : NULL,
-			      rename_branch == o->branch1 ? NULL : dest);
-	}
+	if (handle_change_delete(o,
+				 o->call_depth ? orig->path : dest->path,
+				 &orig->oid, orig->mode,
+				 a_oid, a_mode,
+				 b_oid, b_mode,
+				 _("rename"), _("renamed")))
+		return -1;
 
+	if (o->call_depth)
+		return remove_file_from_cache(dest->path);
+	else
+		return update_stages(dest->path, NULL,
+				     rename_branch == o->branch1 ? dest : NULL,
+				     rename_branch == o->branch1 ? NULL : dest);
 }
 
 static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
@@ -1126,7 +1131,7 @@ static struct diff_filespec *filespec_from_entry(struct diff_filespec *target,
 	return target;
 }
 
-static void handle_file(struct merge_options *o,
+static int handle_file(struct merge_options *o,
 			struct diff_filespec *rename,
 			int stage,
 			struct rename_conflict_info *ci)
@@ -1136,6 +1141,7 @@ static void handle_file(struct merge_options *o,
 	const char *cur_branch, *other_branch;
 	struct diff_filespec other;
 	struct diff_filespec *add;
+	int ret;
 
 	if (stage == 2) {
 		dst_entry = ci->dst_entry1;
@@ -1150,7 +1156,8 @@ static void handle_file(struct merge_options *o,
 	add = filespec_from_entry(&other, dst_entry, stage ^ 1);
 	if (add) {
 		char *add_name = unique_path(o, rename->path, other_branch);
-		update_file(o, 0, &add->oid, add->mode, add_name);
+		if (update_file(o, 0, &add->oid, add->mode, add_name))
+			return -1;
 
 		remove_file(o, 0, rename->path, 0);
 		dst_name = unique_path(o, rename->path, cur_branch);
@@ -1161,17 +1168,20 @@ static void handle_file(struct merge_options *o,
 			       rename->path, other_branch, dst_name);
 		}
 	}
-	update_file(o, 0, &rename->oid, rename->mode, dst_name);
-	if (stage == 2)
-		update_stages(rename->path, NULL, rename, add);
+	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
+		; /* fall through, do allow dst_name to be released */
+	else if (stage == 2)
+		ret = update_stages(rename->path, NULL, rename, add);
 	else
-		update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
+
+	return ret;
 }
 
-static void conflict_rename_rename_1to2(struct merge_options *o,
+static int conflict_rename_rename_1to2(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* One file was renamed in both branches, but to different names. */
@@ -1194,14 +1204,16 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
 				 ci->branch1, ci->branch2, &mfi))
-			return;
+			return -1;
+
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
 		 * pathname and then either rename the add-source file to that
 		 * unique path, or use that unique path instead of src here.
 		 */
-		update_file(o, 0, &mfi.oid, mfi.mode, one->path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, one->path))
+			return -1;
 
 		/*
 		 * Above, we put the merged content at the merge-base's
@@ -1212,22 +1224,26 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		 * resolving the conflict at that path in its favor.
 		 */
 		add = filespec_from_entry(&other, ci->dst_entry1, 2 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, a->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, a->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(a->path);
 		add = filespec_from_entry(&other, ci->dst_entry2, 3 ^ 1);
-		if (add)
-			update_file(o, 0, &add->oid, add->mode, b->path);
+		if (add) {
+			if (update_file(o, 0, &add->oid, add->mode, b->path))
+				return -1;
+		}
 		else
 			remove_file_from_cache(b->path);
-	} else {
-		handle_file(o, a, 2, ci);
-		handle_file(o, b, 3, ci);
-	}
+	} else if (handle_file(o, a, 2, ci) || handle_file(o, b, 3, ci))
+		return -1;
+
+	return 0;
 }
 
-static void conflict_rename_rename_2to1(struct merge_options *o,
+static int conflict_rename_rename_2to1(struct merge_options *o,
 					struct rename_conflict_info *ci)
 {
 	/* Two files, a & b, were renamed to the same thing, c. */
@@ -1238,6 +1254,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	char *path = c1->path; /* == c2->path */
 	struct merge_file_info mfi_c1;
 	struct merge_file_info mfi_c2;
+	int ret;
 
 	output(o, 1, _("CONFLICT (rename/rename): "
 	       "Rename %s->%s in %s. "
@@ -1254,7 +1271,7 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
 				       o->branch1, ci->ren2_other.path,
 				       o->branch2, c2->path, &mfi_c2))
-		return;
+		return -1;
 
 	if (o->call_depth) {
 		/*
@@ -1265,19 +1282,25 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 		 * again later for the non-recursive merge.
 		 */
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, b->path);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, a->path);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+					  b->path);
 	} else {
 		char *new_path1 = unique_path(o, path, ci->branch1);
 		char *new_path2 = unique_path(o, path, ci->branch2);
 		output(o, 1, _("Renaming %s to %s and %s to %s instead"),
 		       a->path, new_path1, b->path, new_path2);
 		remove_file(o, 0, path, 0);
-		update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
-		update_file(o, 0, &mfi_c2.oid, mfi_c2.mode, new_path2);
+		ret = update_file(o, 0, &mfi_c1.oid, mfi_c1.mode, new_path1);
+		if (!ret)
+			ret = update_file(o, 0, &mfi_c2.oid, mfi_c2.mode,
+					  new_path2);
 		free(new_path2);
 		free(new_path1);
 	}
+
+	return ret;
 }
 
 static int process_renames(struct merge_options *o,
@@ -1462,12 +1485,13 @@ static int process_renames(struct merge_options *o,
 				 * update_file_flags() instead of
 				 * update_file().
 				 */
-				update_file_flags(o,
-						  &ren1->pair->two->oid,
-						  ren1->pair->two->mode,
-						  ren1_dst,
-						  1, /* update_cache */
-						  0  /* update_wd    */);
+				if (update_file_flags(o,
+						      &ren1->pair->two->oid,
+						      ren1->pair->two->mode,
+						      ren1_dst,
+						      1, /* update_cache */
+						      0  /* update_wd    */))
+					clean_merge = -1;
 			} else if (!oid_eq(&dst_other.oid, &null_oid)) {
 				clean_merge = 0;
 				try_merge = 1;
@@ -1482,22 +1506,28 @@ static int process_renames(struct merge_options *o,
 							   ren1->pair->two->mode,
 							   &dst_other.oid,
 							   dst_other.mode,
-							   branch1, branch2, &mfi))
-						return -1;
+							   branch1, branch2, &mfi)) {
+						clean_merge = -1;
+						goto cleanup_and_return;
+					}
 					output(o, 1, _("Adding merged %s"), ren1_dst);
-					update_file(o, 0, &mfi.oid,
-						    mfi.mode, ren1_dst);
+					if (update_file(o, 0, &mfi.oid,
+							mfi.mode, ren1_dst))
+						clean_merge = -1;
 					try_merge = 0;
 				} else {
 					char *new_path = unique_path(o, ren1_dst, branch2);
 					output(o, 1, _("Adding as %s instead"), new_path);
-					update_file(o, 0, &dst_other.oid,
-						    dst_other.mode, new_path);
+					if (update_file(o, 0, &dst_other.oid,
+							dst_other.mode, new_path))
+						clean_merge = -1;
 					free(new_path);
 				}
 			} else
 				try_merge = 1;
 
+			if (clean_merge < 0)
+				goto cleanup_and_return;
 			if (try_merge) {
 				struct diff_filespec *one, *a, *b;
 				src_other.path = (char *)ren1_src;
@@ -1524,6 +1554,7 @@ static int process_renames(struct merge_options *o,
 			}
 		}
 	}
+cleanup_and_return:
 	string_list_clear(&a_by_dst, 0);
 	string_list_clear(&b_by_dst, 0);
 
@@ -1586,18 +1617,18 @@ static int blob_unchanged(const struct object_id *o_oid,
 	return ret;
 }
 
-static void handle_modify_delete(struct merge_options *o,
+static int handle_modify_delete(struct merge_options *o,
 				 const char *path,
 				 struct object_id *o_oid, int o_mode,
 				 struct object_id *a_oid, int a_mode,
 				 struct object_id *b_oid, int b_mode)
 {
-	handle_change_delete(o,
-			     path,
-			     o_oid, o_mode,
-			     a_oid, a_mode,
-			     b_oid, b_mode,
-			     _("modify"), _("modified"));
+	return handle_change_delete(o,
+				    path,
+				    o_oid, o_mode,
+				    a_oid, a_mode,
+				    b_oid, b_mode,
+				    _("modify"), _("modified"));
 }
 
 static int merge_content(struct merge_options *o,
@@ -1671,7 +1702,8 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			update_stages(path, &one, &a, &b);
+			if (update_stages(path, &one, &a, &b))
+				return -1;
 	}
 
 	if (df_conflict_remains) {
@@ -1679,30 +1711,33 @@ static int merge_content(struct merge_options *o,
 		if (o->call_depth) {
 			remove_file_from_cache(path);
 		} else {
-			if (!mfi.clean)
-				update_stages(path, &one, &a, &b);
-			else {
+			if (!mfi.clean) {
+				if (update_stages(path, &one, &a, &b))
+					return -1;
+			} else {
 				int file_from_stage2 = was_tracked(path);
 				struct diff_filespec merged;
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				update_stages(path, NULL,
-					      file_from_stage2 ? &merged : NULL,
-					      file_from_stage2 ? NULL : &merged);
+				if (update_stages(path, NULL,
+						  file_from_stage2 ? &merged : NULL,
+						  file_from_stage2 ? NULL : &merged))
+					return -1;
 			}
 
 		}
 		new_path = unique_path(o, path, rename_conflict_info->branch1);
 		output(o, 1, _("Adding as %s instead"), new_path);
-		update_file(o, 0, &mfi.oid, mfi.mode, new_path);
+		if (update_file(o, 0, &mfi.oid, mfi.mode, new_path)) {
+			free(new_path);
+			return -1;
+		}
 		free(new_path);
 		mfi.clean = 0;
-	} else {
-		update_file(o, mfi.clean, &mfi.oid, mfi.mode, path);
-	}
+	} else if (update_file(o, mfi.clean, &mfi.oid, mfi.mode, path))
+		return -1;
 	return mfi.clean;
-
 }
 
 /* Per entry merge function */
@@ -1730,17 +1765,21 @@ static int process_entry(struct merge_options *o,
 			break;
 		case RENAME_DELETE:
 			clean_merge = 0;
-			conflict_rename_delete(o, conflict_info->pair1,
-					       conflict_info->branch1,
-					       conflict_info->branch2);
+			if (conflict_rename_delete(o,
+						   conflict_info->pair1,
+						   conflict_info->branch1,
+						   conflict_info->branch2))
+				clean_merge = -1;
 			break;
 		case RENAME_ONE_FILE_TO_TWO:
 			clean_merge = 0;
-			conflict_rename_rename_1to2(o, conflict_info);
+			if (conflict_rename_rename_1to2(o, conflict_info))
+				clean_merge = -1;
 			break;
 		case RENAME_TWO_FILES_TO_ONE:
 			clean_merge = 0;
-			conflict_rename_rename_2to1(o, conflict_info);
+			if (conflict_rename_rename_2to1(o, conflict_info))
+				clean_merge = -1;
 			break;
 		default:
 			entry->processed = 0;
@@ -1760,8 +1799,9 @@ static int process_entry(struct merge_options *o,
 		} else {
 			/* Modify/delete; deleted side may have put a directory in the way */
 			clean_merge = 0;
-			handle_modify_delete(o, path, o_oid, o_mode,
-					     a_oid, a_mode, b_oid, b_mode);
+			if (handle_modify_delete(o, path, o_oid, o_mode,
+						 a_oid, a_mode, b_oid, b_mode))
+				clean_merge = -1;
 		}
 	} else if ((!o_oid && a_oid && !b_oid) ||
 		   (!o_oid && !a_oid && b_oid)) {
@@ -1793,14 +1833,16 @@ static int process_entry(struct merge_options *o,
 			output(o, 1, _("CONFLICT (%s): There is a directory with name %s in %s. "
 			       "Adding %s as %s"),
 			       conf, path, other_branch, path, new_path);
-			update_file(o, 0, oid, mode, new_path);
-			if (o->call_depth)
+			if (update_file(o, 0, oid, mode, new_path))
+				clean_merge = -1;
+			else if (o->call_depth)
 				remove_file_from_cache(path);
 			free(new_path);
 		} else {
 			output(o, 2, _("Adding %s"), path);
 			/* do not overwrite file if already present */
-			update_file_flags(o, oid, mode, path, 1, !a_oid);
+			if (update_file_flags(o, oid, mode, path, 1, !a_oid))
+				clean_merge = -1;
 		}
 	} else if (a_oid && b_oid) {
 		/* Case C: Added in both (check for same permissions) and */
@@ -1863,12 +1905,18 @@ int merge_trees(struct merge_options *o,
 		re_head  = get_renames(o, head, common, head, merge, entries);
 		re_merge = get_renames(o, merge, common, head, merge, entries);
 		clean = process_renames(o, re_head, re_merge);
+		if (clean < 0)
+			return clean;
 		for (i = entries->nr-1; 0 <= i; i--) {
 			const char *path = entries->items[i].string;
 			struct stage_data *e = entries->items[i].util;
-			if (!e->processed
-				&& !process_entry(o, path, e))
-				clean = 0;
+			if (!e->processed) {
+				int ret = process_entry(o, path, e);
+				if (!ret)
+					clean = 0;
+				else if (ret < 0)
+					return ret;
+			}
 		}
 		for (i = 0; i < entries->nr; i++) {
 			struct stage_data *e = entries->items[i].util;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 14/16] merge-recursive: offer an option to retain the output in 'obuf'

Since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we already accumulate the output in a buffer. The idea was to avoid interfering with the progress output that goes to stderr, which is unbuffered, when we write to stdout, which is buffered.

We extend that buffering to allow the caller to handle the output (possibly suppressing it). This will help us when extending the sequencer to do rebase -i's brunt work: it does not want the picks to print anything by default but instead determine itself whether to print the output or not.

Note that we also redirect the error messages into the output buffer
when the caller asked not to flush the output buffer, for two reasons:
1) to retain the correct output order, and 2) to allow the caller to
suppress *all* output.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++++++----
 merge-recursive.h |  2 +-
 2 files changed, 14 insertions(+), 5 deletions(-)
Show changes to 2 files +14 −5

merge-recursive.c, merge-recursive.h

diff --git a/merge-recursive.c b/merge-recursive.c
index 99c9635..ec50932 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -25,7 +25,7 @@
 
 static void flush_output(struct merge_options *o)
 {
-	if (o->obuf.len) {
+	if (o->buffer_output < 2 && o->obuf.len) {
 		fputs(o->obuf.buf, stdout);
 		strbuf_reset(&o->obuf);
 	}
@@ -35,12 +35,21 @@ static int err(struct merge_options *o, const char *err, ...)
 {
 	va_list params;
 
-	flush_output(o);
+	if (o->buffer_output < 2)
+		flush_output(o);
+	else {
+		strbuf_complete(&o->obuf, '\n');
+		strbuf_addstr(&o->obuf, "error: ");
+	}
 	va_start(params, err);
 	strbuf_vaddf(&o->obuf, err, params);
 	va_end(params);
-	error("%s", o->obuf.buf);
-	strbuf_reset(&o->obuf);
+	if (o->buffer_output > 1)
+		strbuf_addch(&o->obuf, '\n');
+	else {
+		error("%s", o->obuf.buf);
+		strbuf_reset(&o->obuf);
+	}
 
 	return -1;
 }
diff --git a/merge-recursive.h b/merge-recursive.h
index d415724..735343b 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -13,7 +13,7 @@ struct merge_options {
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
 	const char *subtree_shift;
-	unsigned buffer_output : 1;
+	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 10/16] merge-recursive: switch to returning errors instead of dying

The recursive merge machinery is supposed to be a library function, i.e. it should return an error when it fails. Originally the functions were part of the builtin "merge-recursive", though, where it was simpler to call die() and be done with error handling.

The existing callers were already prepared to detect negative return values to indicate errors and to behave as previously: exit with code 128 (which is the same thing that die() does, after printing the message).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 62 +++++++++++++++++++++++++++++++------------------------
 1 file changed, 35 insertions(+), 27 deletions(-)
Show changes to merge-recursive.c +35 −27
diff --git a/merge-recursive.c b/merge-recursive.c
index 6beb1e4..bc59815 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -275,8 +275,10 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 		active_cache_tree = cache_tree();
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
-	    cache_tree_update(&the_index, 0) < 0)
-		die(_("error building trees"));
+	    cache_tree_update(&the_index, 0) < 0) {
+		error(_("error building trees"));
+		return NULL;
+	}
 
 	result = lookup_tree(active_cache_tree->sha1);
 
@@ -716,12 +718,10 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	/* Make sure leading directories are created */
 	status = safe_create_leading_directories_const(path);
 	if (status) {
-		if (status == SCLD_EXISTS) {
+		if (status == SCLD_EXISTS)
 			/* something else exists */
-			error(msg, path, _(": perhaps a D/F conflict?"));
-			return -1;
-		}
-		die(msg, path, "");
+			return error(msg, path, _(": perhaps a D/F conflict?"));
+		return error(msg, path, "");
 	}
 
 	/*
@@ -749,6 +749,8 @@ static int update_file_flags(struct merge_options *o,
 			     int update_cache,
 			     int update_wd)
 {
+	int ret = 0;
+
 	if (o->call_depth)
 		update_wd = 0;
 
@@ -769,9 +771,11 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			die(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
-		if (type != OBJ_BLOB)
-			die(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+		if (type != OBJ_BLOB) {
+			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			goto free_buf;
+		}
 		if (S_ISREG(mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
 			if (convert_to_working_tree(path, buf, size, &strbuf)) {
@@ -792,8 +796,11 @@ static int update_file_flags(struct merge_options *o,
 			else
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
-			if (fd < 0)
-				die_errno(_("failed to open '%s'"), path);
+			if (fd < 0) {
+				ret = error_errno(_("failed to open '%s'"),
+						  path);
+				goto free_buf;
+			}
 			write_in_full(fd, buf, size);
 			close(fd);
 		} else if (S_ISLNK(mode)) {
@@ -801,18 +808,18 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				die_errno(_("failed to symlink '%s'"), path);
+				ret = error_errno(_("failed to symlink '%s'"), path);
 			free(lnk);
 		} else
-			die(_("do not know what to do with %06o %s '%s'"),
-			    mode, oid_to_hex(oid), path);
+			ret = error(_("do not know what to do with %06o %s '%s'"),
+				    mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
-	if (update_cache)
+	if (!ret && update_cache)
 		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
-	return 0;
+	return ret;
 }
 
 static int update_file(struct merge_options *o,
@@ -938,20 +945,22 @@ static int merge_file_1(struct merge_options *o,
 			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
-			int merge_status;
+			int ret = 0, merge_status;
 
 			merge_status = merge_3way(o, &result_buf, one, a, b,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				die(_("Failed to execute internal merge"));
+				ret = error(_("Failed to execute internal merge"));
 
-			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result->oid.hash))
-				die(_("Unable to add %s to database"),
-				    a->path);
+			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
+						    blob_type, result->oid.hash))
+				ret = error(_("Unable to add %s to database"),
+					    a->path);
 
 			free(result_buf.ptr);
+			if (ret)
+				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
 			result->clean = merge_submodule(result->oid.hash,
@@ -1885,11 +1894,10 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			die(_("merging of trees %s and %s failed"),
+			error(_("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
-		else
-			exit(128);
+		return -1;
 	}
 
 	if (unmerged_cache()) {
@@ -2021,7 +2029,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			die(_("merge returned no commit"));
+			return error(_("merge returned no commit"));
 	}
 
 	discard_cache();
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 12/16] merge-recursive: flush output buffer before printing error messages

The data structure passed to the recursive merge machinery has a feature where the caller can ask for the output to be buffered into a strbuf, by setting the field 'buffer_output'.

Previously, we died without flushing, losing accumulated output. With this patch, we show the output first, and only then print the error message.

Currently, the only user of that buffering is merge_recursive() itself, to avoid the progress output to interfere.

In the next patches, we will introduce a new buffer_output mode that forces merge_recursive() to retain the output buffer for further processing by the caller. If the caller asked for that, we will then also write the error messages into the output buffer. This is necessary to give the caller more control not only how to react in case of errors but also control how/if to display the error messages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 116 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 68 insertions(+), 48 deletions(-)
Show changes to merge-recursive.c +68 −48
diff --git a/merge-recursive.c b/merge-recursive.c
index bc59815..b972a83 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,28 @@
 #include "dir.h"
 #include "submodule.h"
 
+static void flush_output(struct merge_options *o)
+{
+	if (o->obuf.len) {
+		fputs(o->obuf.buf, stdout);
+		strbuf_reset(&o->obuf);
+	}
+}
+
+static int err(struct merge_options *o, const char *err, ...)
+{
+	va_list params;
+
+	flush_output(o);
+	va_start(params, err);
+	strbuf_vaddf(&o->obuf, err, params);
+	va_end(params);
+	error("%s", o->obuf.buf);
+	strbuf_reset(&o->obuf);
+
+	return -1;
+}
+
 static struct tree *shift_tree_object(struct tree *one, struct tree *two,
 				      const char *subtree_shift)
 {
@@ -148,14 +170,6 @@ static int show(struct merge_options *o, int v)
 	return (!o->call_depth && o->verbosity >= v) || o->verbosity >= 5;
 }
 
-static void flush_output(struct merge_options *o)
-{
-	if (o->obuf.len) {
-		fputs(o->obuf.buf, stdout);
-		strbuf_reset(&o->obuf);
-	}
-}
-
 __attribute__((format (printf, 3, 4)))
 static void output(struct merge_options *o, int v, const char *fmt, ...)
 {
@@ -198,7 +212,8 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)
 	}
 }
 
-static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
+static int add_cacheinfo(struct merge_options *o,
+		unsigned int mode, const struct object_id *oid,
 		const char *path, int stage, int refresh, int options)
 {
 	struct cache_entry *ce;
@@ -206,7 +221,7 @@ static int add_cacheinfo(unsigned int mode, const struct object_id *oid,
 
 	ce = make_cache_entry(mode, oid ? oid->hash : null_sha1, path, stage, 0);
 	if (!ce)
-		return error(_("addinfo_cache failed for path '%s'"), path);
+		return err(o, _("addinfo_cache failed for path '%s'"), path);
 
 	ret = add_cache_entry(ce, options);
 	if (refresh) {
@@ -276,7 +291,7 @@ struct tree *write_tree_from_memory(struct merge_options *o)
 
 	if (!cache_tree_fully_valid(active_cache_tree) &&
 	    cache_tree_update(&the_index, 0) < 0) {
-		error(_("error building trees"));
+		err(o, _("error building trees"));
 		return NULL;
 	}
 
@@ -544,7 +559,8 @@ static struct string_list *get_renames(struct merge_options *o,
 	return renames;
 }
 
-static int update_stages(const char *path, const struct diff_filespec *o,
+static int update_stages(struct merge_options *opt, const char *path,
+			 const struct diff_filespec *o,
 			 const struct diff_filespec *a,
 			 const struct diff_filespec *b)
 {
@@ -563,13 +579,13 @@ static int update_stages(const char *path, const struct diff_filespec *o,
 		if (remove_file_from_cache(path))
 			return -1;
 	if (o)
-		if (add_cacheinfo(o->mode, &o->oid, path, 1, 0, options))
+		if (add_cacheinfo(opt, o->mode, &o->oid, path, 1, 0, options))
 			return -1;
 	if (a)
-		if (add_cacheinfo(a->mode, &a->oid, path, 2, 0, options))
+		if (add_cacheinfo(opt, a->mode, &a->oid, path, 2, 0, options))
 			return -1;
 	if (b)
-		if (add_cacheinfo(b->mode, &b->oid, path, 3, 0, options))
+		if (add_cacheinfo(opt, b->mode, &b->oid, path, 3, 0, options))
 			return -1;
 	return 0;
 }
@@ -720,8 +736,8 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (status) {
 		if (status == SCLD_EXISTS)
 			/* something else exists */
-			return error(msg, path, _(": perhaps a D/F conflict?"));
-		return error(msg, path, "");
+			return err(o, msg, path, _(": perhaps a D/F conflict?"));
+		return err(o, msg, path, "");
 	}
 
 	/*
@@ -729,7 +745,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	 * tracking it.
 	 */
 	if (would_lose_untracked(path))
-		return error(_("refusing to lose untracked file at '%s'"),
+		return err(o, _("refusing to lose untracked file at '%s'"),
 			     path);
 
 	/* Successful unlink is good.. */
@@ -739,7 +755,7 @@ static int make_room_for_path(struct merge_options *o, const char *path)
 	if (errno == ENOENT)
 		return 0;
 	/* .. but not some other error (who really cares what?) */
-	return error(msg, path, _(": perhaps a D/F conflict?"));
+	return err(o, msg, path, _(": perhaps a D/F conflict?"));
 }
 
 static int update_file_flags(struct merge_options *o,
@@ -771,9 +787,9 @@ static int update_file_flags(struct merge_options *o,
 
 		buf = read_sha1_file(oid->hash, &type, &size);
 		if (!buf)
-			return error(_("cannot read object %s '%s'"), oid_to_hex(oid), path);
+			return err(o, _("cannot read object %s '%s'"), oid_to_hex(oid), path);
 		if (type != OBJ_BLOB) {
-			ret = error(_("blob expected for %s '%s'"), oid_to_hex(oid), path);
+			ret = err(o, _("blob expected for %s '%s'"), oid_to_hex(oid), path);
 			goto free_buf;
 		}
 		if (S_ISREG(mode)) {
@@ -797,8 +813,8 @@ static int update_file_flags(struct merge_options *o,
 				mode = 0666;
 			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
 			if (fd < 0) {
-				ret = error_errno(_("failed to open '%s'"),
-						  path);
+				ret = err(o, _("failed to open '%s': %s"),
+					  path, strerror(errno));
 				goto free_buf;
 			}
 			write_in_full(fd, buf, size);
@@ -808,17 +824,19 @@ static int update_file_flags(struct merge_options *o,
 			safe_create_leading_directories_const(path);
 			unlink(path);
 			if (symlink(lnk, path))
-				ret = error_errno(_("failed to symlink '%s'"), path);
+				ret = err(o, _("failed to symlink '%s': %s"),
+					path, strerror(errno));
 			free(lnk);
 		} else
-			ret = error(_("do not know what to do with %06o %s '%s'"),
-				    mode, oid_to_hex(oid), path);
+			ret = err(o,
+				  _("do not know what to do with %06o %s '%s'"),
+				  mode, oid_to_hex(oid), path);
  free_buf:
 		free(buf);
 	}
  update_index:
 	if (!ret && update_cache)
-		add_cacheinfo(mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
+		add_cacheinfo(o, mode, oid, path, 0, update_wd, ADD_CACHE_OK_TO_ADD);
 	return ret;
 }
 
@@ -951,12 +969,12 @@ static int merge_file_1(struct merge_options *o,
 						  branch1, branch2);
 
 			if ((merge_status < 0) || !result_buf.ptr)
-				ret = error(_("Failed to execute internal merge"));
+				ret = err(o, _("Failed to execute internal merge"));
 
 			if (!ret && write_sha1_file(result_buf.ptr, result_buf.size,
 						    blob_type, result->oid.hash))
-				ret = error(_("Unable to add %s to database"),
-					    a->path);
+				ret = err(o, _("Unable to add %s to database"),
+					  a->path);
 
 			free(result_buf.ptr);
 			if (ret)
@@ -1122,7 +1140,7 @@ static int conflict_rename_delete(struct merge_options *o,
 	if (o->call_depth)
 		return remove_file_from_cache(dest->path);
 	else
-		return update_stages(dest->path, NULL,
+		return update_stages(o, dest->path, NULL,
 				     rename_branch == o->branch1 ? dest : NULL,
 				     rename_branch == o->branch1 ? NULL : dest);
 }
@@ -1180,9 +1198,9 @@ static int handle_file(struct merge_options *o,
 	if ((ret = update_file(o, 0, &rename->oid, rename->mode, dst_name)))
 		; /* fall through, do allow dst_name to be released */
 	else if (stage == 2)
-		ret = update_stages(rename->path, NULL, rename, add);
+		ret = update_stages(o, rename->path, NULL, rename, add);
 	else
-		ret = update_stages(rename->path, NULL, add, rename);
+		ret = update_stages(o, rename->path, NULL, add, rename);
 
 	if (dst_name != rename->path)
 		free(dst_name);
@@ -1575,23 +1593,25 @@ static struct object_id *stage_oid(const struct object_id *oid, unsigned mode)
 	return (is_null_oid(oid) || mode == 0) ? NULL: (struct object_id *)oid;
 }
 
-static int read_oid_strbuf(const struct object_id *oid, struct strbuf *dst)
+static int read_oid_strbuf(struct merge_options *o,
+	const struct object_id *oid, struct strbuf *dst)
 {
 	void *buf;
 	enum object_type type;
 	unsigned long size;
 	buf = read_sha1_file(oid->hash, &type, &size);
 	if (!buf)
-		return error(_("cannot read object %s"), oid_to_hex(oid));
+		return err(o, _("cannot read object %s"), oid_to_hex(oid));
 	if (type != OBJ_BLOB) {
 		free(buf);
-		return error(_("object %s is not a blob"), oid_to_hex(oid));
+		return err(o, _("object %s is not a blob"), oid_to_hex(oid));
 	}
 	strbuf_attach(dst, buf, size, size + 1);
 	return 0;
 }
 
-static int blob_unchanged(const struct object_id *o_oid,
+static int blob_unchanged(struct merge_options *opt,
+			  const struct object_id *o_oid,
 			  unsigned o_mode,
 			  const struct object_id *a_oid,
 			  unsigned a_mode,
@@ -1609,7 +1629,7 @@ static int blob_unchanged(const struct object_id *o_oid,
 		return 0;
 
 	assert(o_oid && a_oid);
-	if (read_oid_strbuf(o_oid, &o) || read_oid_strbuf(a_oid, &a))
+	if (read_oid_strbuf(opt, o_oid, &o) || read_oid_strbuf(opt, a_oid, &a))
 		goto error_return;
 	/*
 	 * Note: binary | is used so that both renormalizations are
@@ -1698,7 +1718,7 @@ static int merge_content(struct merge_options *o,
 		 */
 		path_renamed_outside_HEAD = !path2 || !strcmp(path, path2);
 		if (!path_renamed_outside_HEAD) {
-			add_cacheinfo(mfi.mode, &mfi.oid, path,
+			add_cacheinfo(o, mfi.mode, &mfi.oid, path,
 				      0, (!o->call_depth), 0);
 			return mfi.clean;
 		}
@@ -1711,7 +1731,7 @@ static int merge_content(struct merge_options *o,
 		output(o, 1, _("CONFLICT (%s): Merge conflict in %s"),
 				reason, path);
 		if (rename_conflict_info && !df_conflict_remains)
-			if (update_stages(path, &one, &a, &b))
+			if (update_stages(o, path, &one, &a, &b))
 				return -1;
 	}
 
@@ -1721,7 +1741,7 @@ static int merge_content(struct merge_options *o,
 			remove_file_from_cache(path);
 		} else {
 			if (!mfi.clean) {
-				if (update_stages(path, &one, &a, &b))
+				if (update_stages(o, path, &one, &a, &b))
 					return -1;
 			} else {
 				int file_from_stage2 = was_tracked(path);
@@ -1729,7 +1749,7 @@ static int merge_content(struct merge_options *o,
 				oidcpy(&merged.oid, &mfi.oid);
 				merged.mode = mfi.mode;
 
-				if (update_stages(path, NULL,
+				if (update_stages(o, path, NULL,
 						  file_from_stage2 ? &merged : NULL,
 						  file_from_stage2 ? NULL : &merged))
 					return -1;
@@ -1797,8 +1817,8 @@ static int process_entry(struct merge_options *o,
 	} else if (o_oid && (!a_oid || !b_oid)) {
 		/* Case A: Deleted in one */
 		if ((!a_oid && !b_oid) ||
-		    (!b_oid && blob_unchanged(o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
-		    (!a_oid && blob_unchanged(o_oid, o_mode, b_oid, b_mode, normalize, path))) {
+		    (!b_oid && blob_unchanged(o, o_oid, o_mode, a_oid, a_mode, normalize, path)) ||
+		    (!a_oid && blob_unchanged(o, o_oid, o_mode, b_oid, b_mode, normalize, path))) {
 			/* Deleted in both or deleted in one and
 			 * unchanged in the other */
 			if (a_oid)
@@ -1894,7 +1914,7 @@ int merge_trees(struct merge_options *o,
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
-			error(_("merging of trees %s and %s failed"),
+			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
 		return -1;
@@ -2029,7 +2049,7 @@ int merge_recursive(struct merge_options *o,
 		o->call_depth--;
 
 		if (!merged_common_ancestors)
-			return error(_("merge returned no commit"));
+			return err(o, _("merge returned no commit"));
 	}
 
 	discard_cache();
@@ -2088,7 +2108,7 @@ int merge_recursive_generic(struct merge_options *o,
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
 			if (!(base = get_ref(base_list[i], oid_to_hex(base_list[i]))))
-				return error(_("Could not parse object '%s'"),
+				return err(o, _("Could not parse object '%s'"),
 					oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
 		}
@@ -2102,7 +2122,7 @@ int merge_recursive_generic(struct merge_options *o,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
-		return error(_("Unable to write index."));
+		return err(o, _("Unable to write index."));
 
 	return clean ? 0 : 1;
 }
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 07/16] merge-recursive: avoid returning a wholesale struct

It is technically allowed, as per C89, for functions' return type to be complete structs (i.e. *not* just pointers to structs).

However, it was just an oversight of this developer when converting Python code to C code in 6d297f8 (Status update on merge-recursive in C, 2006-07-08) which introduced such a return type.

Besides, by converting this construct to pass in the struct, we can now start returning a value that can indicate errors in future patches. This will help the current effort to libify merge-recursive.c.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 106 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 56 insertions(+), 50 deletions(-)
Show changes to merge-recursive.c +56 −50
diff --git a/merge-recursive.c b/merge-recursive.c
index 58ced25..2be1e17 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -894,47 +894,47 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
-static struct merge_file_info merge_file_1(struct merge_options *o,
+static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
 					   const struct diff_filespec *b,
 					   const char *branch1,
-					   const char *branch2)
+					   const char *branch2,
+					   struct merge_file_info *result)
 {
-	struct merge_file_info result;
-	result.merge = 0;
-	result.clean = 1;
+	result->merge = 0;
+	result->clean = 1;
 
 	if ((S_IFMT & a->mode) != (S_IFMT & b->mode)) {
-		result.clean = 0;
+		result->clean = 0;
 		if (S_ISREG(a->mode)) {
-			result.mode = a->mode;
-			oidcpy(&result.oid, &a->oid);
+			result->mode = a->mode;
+			oidcpy(&result->oid, &a->oid);
 		} else {
-			result.mode = b->mode;
-			oidcpy(&result.oid, &b->oid);
+			result->mode = b->mode;
+			oidcpy(&result->oid, &b->oid);
 		}
 	} else {
 		if (!oid_eq(&a->oid, &one->oid) && !oid_eq(&b->oid, &one->oid))
-			result.merge = 1;
+			result->merge = 1;
 
 		/*
 		 * Merge modes
 		 */
 		if (a->mode == b->mode || a->mode == one->mode)
-			result.mode = b->mode;
+			result->mode = b->mode;
 		else {
-			result.mode = a->mode;
+			result->mode = a->mode;
 			if (b->mode != one->mode) {
-				result.clean = 0;
-				result.merge = 1;
+				result->clean = 0;
+				result->merge = 1;
 			}
 		}
 
 		if (oid_eq(&a->oid, &b->oid) || oid_eq(&a->oid, &one->oid))
-			oidcpy(&result.oid, &b->oid);
+			oidcpy(&result->oid, &b->oid);
 		else if (oid_eq(&b->oid, &one->oid))
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 		else if (S_ISREG(a->mode)) {
 			mmbuffer_t result_buf;
 			int merge_status;
@@ -946,64 +946,66 @@ static struct merge_file_info merge_file_1(struct merge_options *o,
 				die(_("Failed to execute internal merge"));
 
 			if (write_sha1_file(result_buf.ptr, result_buf.size,
-					    blob_type, result.oid.hash))
+					    blob_type, result->oid.hash))
 				die(_("Unable to add %s to database"),
 				    a->path);
 
 			free(result_buf.ptr);
-			result.clean = (merge_status == 0);
+			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result.clean = merge_submodule(result.oid.hash,
+			result->clean = merge_submodule(result->oid.hash,
 						       one->path,
 						       one->oid.hash,
 						       a->oid.hash,
 						       b->oid.hash,
 						       !o->call_depth);
 		} else if (S_ISLNK(a->mode)) {
-			oidcpy(&result.oid, &a->oid);
+			oidcpy(&result->oid, &a->oid);
 
 			if (!oid_eq(&a->oid, &b->oid))
-				result.clean = 0;
+				result->clean = 0;
 		} else
 			die("BUG: unsupported object type in the tree");
 	}
 
-	return result;
+	return 0;
 }
 
-static struct merge_file_info
-merge_file_special_markers(struct merge_options *o,
+static int merge_file_special_markers(struct merge_options *o,
 			   const struct diff_filespec *one,
 			   const struct diff_filespec *a,
 			   const struct diff_filespec *b,
 			   const char *branch1,
 			   const char *filename1,
 			   const char *branch2,
-			   const char *filename2)
+			   const char *filename2,
+			   struct merge_file_info *mfi)
 {
 	char *side1 = NULL;
 	char *side2 = NULL;
-	struct merge_file_info mfi;
+	int ret;
 
 	if (filename1)
 		side1 = xstrfmt("%s:%s", branch1, filename1);
 	if (filename2)
 		side2 = xstrfmt("%s:%s", branch2, filename2);
 
-	mfi = merge_file_1(o, one, a, b,
-			   side1 ? side1 : branch1, side2 ? side2 : branch2);
+	ret = merge_file_1(o, one, a, b,
+			   side1 ? side1 : branch1,
+			   side2 ? side2 : branch2, mfi);
 	free(side1);
 	free(side2);
-	return mfi;
+	return ret;
 }
 
-static struct merge_file_info merge_file_one(struct merge_options *o,
+static int merge_file_one(struct merge_options *o,
 					 const char *path,
 					 const struct object_id *o_oid, int o_mode,
 					 const struct object_id *a_oid, int a_mode,
 					 const struct object_id *b_oid, int b_mode,
 					 const char *branch1,
-					 const char *branch2)
+					 const char *branch2,
+					 struct merge_file_info *mfi)
 {
 	struct diff_filespec one, a, b;
 
@@ -1014,7 +1016,7 @@ static struct merge_file_info merge_file_one(struct merge_options *o,
 	a.mode = a_mode;
 	oidcpy(&b.oid, b_oid);
 	b.mode = b_mode;
-	return merge_file_1(o, &one, &a, &b, branch1, branch2);
+	return merge_file_1(o, &one, &a, &b, branch1, branch2, mfi);
 }
 
 static void handle_change_delete(struct merge_options *o,
@@ -1187,11 +1189,12 @@ static void conflict_rename_rename_1to2(struct merge_options *o,
 		struct merge_file_info mfi;
 		struct diff_filespec other;
 		struct diff_filespec *add;
-		mfi = merge_file_one(o, one->path,
+		if (merge_file_one(o, one->path,
 				 &one->oid, one->mode,
 				 &a->oid, a->mode,
 				 &b->oid, b->mode,
-				 ci->branch1, ci->branch2);
+				 ci->branch1, ci->branch2, &mfi))
+			return;
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1245,12 +1248,13 @@ static void conflict_rename_rename_2to1(struct merge_options *o,
 	remove_file(o, 1, a->path, o->call_depth || would_lose_untracked(a->path));
 	remove_file(o, 1, b->path, o->call_depth || would_lose_untracked(b->path));
 
-	mfi_c1 = merge_file_special_markers(o, a, c1, &ci->ren1_other,
-					    o->branch1, c1->path,
-					    o->branch2, ci->ren1_other.path);
-	mfi_c2 = merge_file_special_markers(o, b, &ci->ren2_other, c2,
-					    o->branch1, ci->ren2_other.path,
-					    o->branch2, c2->path);
+	if (merge_file_special_markers(o, a, c1, &ci->ren1_other,
+				       o->branch1, c1->path,
+				       o->branch2, ci->ren1_other.path, &mfi_c1) ||
+	    merge_file_special_markers(o, b, &ci->ren2_other, c2,
+				       o->branch1, ci->ren2_other.path,
+				       o->branch2, c2->path, &mfi_c2))
+		return;
 
 	if (o->call_depth) {
 		/*
@@ -1473,12 +1477,13 @@ static int process_renames(struct merge_options *o,
 				       ren1_dst, branch2);
 				if (o->call_depth) {
 					struct merge_file_info mfi;
-					mfi = merge_file_one(o, ren1_dst, &null_oid, 0,
-							 &ren1->pair->two->oid,
-							 ren1->pair->two->mode,
-							 &dst_other.oid,
-							 dst_other.mode,
-							 branch1, branch2);
+					if (merge_file_one(o, ren1_dst, &null_oid, 0,
+							   &ren1->pair->two->oid,
+							   ren1->pair->two->mode,
+							   &dst_other.oid,
+							   dst_other.mode,
+							   branch1, branch2, &mfi))
+						return -1;
 					output(o, 1, _("Adding merged %s"), ren1_dst);
 					update_file(o, 0, &mfi.oid,
 						    mfi.mode, ren1_dst);
@@ -1636,9 +1641,10 @@ static int merge_content(struct merge_options *o,
 		if (dir_in_way(path, !o->call_depth))
 			df_conflict_remains = 1;
 	}
-	mfi = merge_file_special_markers(o, &one, &a, &b,
-					 o->branch1, path1,
-					 o->branch2, path2);
+	if (merge_file_special_markers(o, &one, &a, &b,
+				       o->branch1, path1,
+				       o->branch2, path2, &mfi))
+		return -1;
 
 	if (mfi.clean && !df_conflict_remains &&
 	    oid_eq(&mfi.oid, a_oid) && mfi.mode == a_mode) {
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Aug 4, 2016, 18:09 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 07/16] merge-recursive: avoid returning a wholesale struct

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 10 quoted lines
> It is technically allowed, as per C89, for functions' return type to
> be complete structs (i.e. *not* just pointers to structs).
>
> However, it was just an oversight of this developer when converting
> Python code to C code in 6d297f8 (Status update on merge-recursive in
> C, 2006-07-08) which introduced such a return type.
>
> Besides, by converting this construct to pass in the struct, we can now
> start returning a value that can indicate errors in future patches. This
> will help the current effort to libify merge-recursive.c.

I do not think returning a small struct by value is unconditionally a bad thing, but I do agree with you that this change makes the resulting code much easier to read, especially once this starts returning errors.

Good.
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 15/16] Ensure that the output buffer is released after calling merge_trees()

The recursive merge machinery accumulates its output in an output buffer, to be flushed at the end of merge_recursive(). At this point, we forgot to release the output buffer.

When calling merge_trees() (i.e. the non-recursive part of the recursive merge) directly, the output buffer is never flushed because the caller may be merge_recursive() which wants to flush the output itself.

For the same reason, merge_trees() cannot release the output buffer: it may still be needed.

Forgetting to release the output buffer did not matter much when running git-checkout, or git-merge-recursive, because we exited after the operation anyway. Ever since cherry-pick learned to pick a commit range, however, this memory leak had the potential of becoming a problem.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/checkout.c | 1 +
 merge-recursive.c  | 2 ++
 sequencer.c        | 1 +
 3 files changed, 4 insertions(+)
Show changes to 3 files +4 −0

builtin/checkout.c, merge-recursive.c, sequencer.c

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 07dea3b..8d852d4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -573,6 +573,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 				exit(128);
 			ret = reset_tree(new->commit->tree, opts, 0,
 					 writeout_error);
+			strbuf_release(&o.obuf);
 			if (ret)
 				return ret;
 		}
diff --git a/merge-recursive.c b/merge-recursive.c
index ec50932..9e527de 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2078,6 +2078,8 @@ int merge_recursive(struct merge_options *o,
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
 	flush_output(o);
+	if (!o->call_depth && o->buffer_output < 2)
+		strbuf_release(&o->obuf);
 	if (show(o, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       o->needed_rename_limit, 0);
diff --git a/sequencer.c b/sequencer.c
index 286a435..ec50519 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -293,6 +293,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
 
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Aug 4, 2016, 18:18 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 15/16] Ensure that the output buffer is released after calling merge_trees()

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 14 quoted lines
> The recursive merge machinery accumulates its output in an output
> buffer, to be flushed at the end of merge_recursive(). At this point,
> we forgot to release the output buffer.
> ...
> diff --git a/merge-recursive.c b/merge-recursive.c
> index ec50932..9e527de 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2078,6 +2078,8 @@ int merge_recursive(struct merge_options *o,
>  		commit_list_insert(h2, &(*result)->parents->next);
>  	}
>  	flush_output(o);
> +	if (!o->call_depth && o->buffer_output < 2)
> +		strbuf_release(&o->obuf);
OK, with !o->call_depth the change makes sense to me.
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 08/16] merge-recursive: allow write_tree_from_memory() to error out

It is possible that a tree cannot be written (think: disk full). We will want to give the caller a chance to clean up instead of letting the program die() in such a case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to merge-recursive.c +2 −2
diff --git a/merge-recursive.c b/merge-recursive.c
index 2be1e17..1f86338 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1888,8 +1888,8 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	if (o->call_depth)
-		*result = write_tree_from_memory(o);
+	if (o->call_depth && !(*result = write_tree_from_memory(o)))
+		return -1;
 
 	return clean;
 }
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Aug 4, 2016, 18:14 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 08/16] merge-recursive: allow write_tree_from_memory() to error out

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 21 quoted lines
> It is possible that a tree cannot be written (think: disk full). We
> will want to give the caller a chance to clean up instead of letting
> the program die() in such a case.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  merge-recursive.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 2be1e17..1f86338 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1888,8 +1888,8 @@ int merge_trees(struct merge_options *o,
>  	else
>  		clean = 1;
>  
> -	if (o->call_depth)
> -		*result = write_tree_from_memory(o);
> +	if (o->call_depth && !(*result = write_tree_from_memory(o)))
> +		return -1;

I'll let it pass, but we avoid assignment in a conditional part for a good reason: it can become unreadable pretty quickly. Writing it in a long-hand, e.g.

	if (o->call_depth) {
        	*result = write_tree_from_memory(o);
                if (!*result)
                	return -1;
	}

future-proofs against the "o->call_depth" condition part and "write_tree_from_memory(o)" operation part becoming longer, possibly needing multiple statements.

The change itself is correct.
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 06/16] merge_recursive: abort properly upon errors

There are a couple of places where return values never indicated errors before, as wie simply died instead of returning.

But now negative return values mean that there was an error and we have to abort the operation. Let's do exactly that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
Show changes to merge-recursive.c +12 −5
diff --git a/merge-recursive.c b/merge-recursive.c
index 3a652b7..58ced25 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1949,17 +1949,19 @@ int merge_recursive(struct merge_options *o,
 		/*
 		 * When the merge fails, the result contains files
 		 * with conflict markers. The cleanness flag is
-		 * ignored, it was never actually used, as result of
-		 * merge_trees has always overwritten it: the committed
-		 * "conflicts" were already resolved.
+		 * ignored (unless indicating an error), it was never
+		 * actually used, as result of merge_trees has always
+		 * overwritten it: the committed "conflicts" were
+		 * already resolved.
 		 */
 		discard_cache();
 		saved_b1 = o->branch1;
 		saved_b2 = o->branch2;
 		o->branch1 = "Temporary merge branch 1";
 		o->branch2 = "Temporary merge branch 2";
-		merge_recursive(o, merged_common_ancestors, iter->item,
-				NULL, &merged_common_ancestors);
+		if (merge_recursive(o, merged_common_ancestors, iter->item,
+				    NULL, &merged_common_ancestors) < 0)
+			return -1;
 		o->branch1 = saved_b1;
 		o->branch2 = saved_b2;
 		o->call_depth--;
@@ -1975,6 +1977,8 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
+	if (clean < 0)
+		return clean;
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
@@ -2031,6 +2035,9 @@ int merge_recursive_generic(struct merge_options *o,
 	hold_locked_index(lock, 1);
 	clean = merge_recursive(o, head_commit, next_commit, ca,
 			result);
+	if (clean < 0)
+		return clean;
+
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, lock, COMMIT_LOCK))
 		return error(_("Unable to write index."));
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Aug 1, 2016, 18:41 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> Subject: Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors
s/merge_/merge-/; for this one alone.
> There are a couple of places where return values never indicated errors
> before, as wie simply died instead of returning.
s/wie/we/;
Johannes Schindelin· Aug 2, 2016, 08:12 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Mon, 1 Aug 2016, Junio C Hamano wrote:
Show 5 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Subject: Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors
> 
> s/merge_/merge-/; for this one alone.

I see that de8946de1694a8cf311daab7b2c416d76cb04d23 still shows it with an underscore instead of a dash, while you fixed my other tyops in this patch series.

I tend to think that the underscore is correct: this change is not so much about the builtin (which is written with a dash) but about the function (written with an underscore, used by more than just merge-recursive, e.g. cherry-pick).

> > There are a couple of places where return values never indicated errors
> > before, as wie simply died instead of returning.
> 
> s/wie/we/;
Right. What can I say, I am surrounded by too many Germans.

I fixed this locally, in case another re-roll should be required. What you have in `pu` looks correct to me, though. Let me know if you want me to re-submit nevertheless.

BTW I should have said this earlier: I run all my rebases, all my merges, all my cherry-picks using a Git version with these patches for months (of course, the patches have changed, but not in the most critical parts I was concerned about, the parts where die() calls were replaced). If I would have found any regression, I would have notified you immediately, of course.

Ciao, Dscho

Junio C Hamano· Aug 2, 2016, 21:26 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 10 quoted lines
>> > There are a couple of places where return values never indicated errors
>> > before, as wie simply died instead of returning.
>> 
>> s/wie/we/;
>
> Right. What can I say, I am surrounded by too many Germans.
>
> I fixed this locally, in case another re-roll should be required. What you
> have in `pu` looks correct to me, though. Let me know if you want me to
> re-submit nevertheless.

I usually do this kind of obvious typofix and consistency fix without even mentioning them in my review comments to reduce the noise levels. But that works better ONLY if the patch authors then fetch from 'pu' and replace their copies with what I fixed up already and base their reroll on top by amending and/or building on top (of course, that also requires my local fix must all be limited to uncontroversial ones).

So either I should change my workflow and mention any and all typofixes in my review comments (which consumes the review bandwidth), or I should force patch authors to do the "fetch from 'pu' and replace" somehow to avoid this kind of back-and-forth.

I am not sure which should be the way to go.
Johannes Schindelin· Aug 3, 2016, 11:59 UTC · re: Junio C Hamano · lore

patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Tue, 2 Aug 2016, Junio C Hamano wrote:
> So either I should change my workflow and mention any and all
> typofixes in my review comments (which consumes the review
> bandwidth), or I should force patch authors to do the "fetch from
> 'pu' and replace" somehow to avoid this kind of back-and-forth.

It never occurred to me that I should replace any of my local branches with versions you have in `pu`. For several reasons:

- just as you do not pull from me, lest patches enter your repository that
  you have not reviewed, I do not simply pull from you,
- I thought the point of avoiding GitHub in the review process at all
  costs and require everybody to go via the mailing list instead was to
  keep the review process open? These silent changes fly in the face of
  that,
- I agree that the mail-based process is cumbersome and error-prone, but
  we won't fix it by asking contributors to dig through the `pu` of the
  day, somehow stay up-to-date with possibly more silent typofixes the
  next day, somehow manage to figure out what changes exactly were
  introduced to their patches, and replace their local work.
- even if we asked for all that trouble, the commits in `pu` are all
  signed off by you. These sign-offs would have to be stripped out
  tediously when making local changes.

In short, I agree that our patch submission process is a saber tooth tiger that still reflects pre-Git times. While we use Git's tools, the workflow really tries to cut out Git as much as possible, in favor of pure mails with non-corrupted, non-HTML patches in them, a charmingly anachronistic requirement until you try to use state-of-the-art mail clients to send them.

I disagree, however, with the suggestion to sift through your `pu` branch and to somehow replace local branches with the commits found there.

I guess it is time to revisit our patch submission process if it does not even work between the two of us.

Ideally, we would come up with a process that
- makes everything easier for maintainers and contributors alike,
- tracks the history of the patch iterations (answering the question "what
  changed between iterations?"),
- *actually* integrates with Git (to see what I mean, try to find the
  commit corresponding to a given mail containing a patch, and then try to
  find the previous iteration's version of the same commit, and weep),
- provides machine-readable metadata about the context, e.g. to jump back
  and forth between the full file contents and the patch, or to indicate
  the dependency on another branch,
- facilitates "back contributions", i.e. letting contributors accept
  changes suggested by reviewers *with minimal effort*.
- uses Git itself as much as possible, i.e. no additional tools written in
  "you must learn this new language, it's awesome, believe me, it's huge"

The biggest obstacles I see are 1) the integration with the mailing list (which is ironic because contributing via the list used to be a boon, not a burden) and 2) maintaining the integrity between what has been reviewed and what is actually in the branch.

This is nothing we will solve overnight, of course. But I think we will have to fix this.

Food for thought. Dscho

Junio C Hamano· Aug 3, 2016, 15:33 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Wed, Aug 3, 2016 at 4:59 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

>
> I disagree, however, with the suggestion to sift through your `pu` branch
> and to somehow replace local branches with the commits found there.

To be more in line with the "e-mailed patch" workflow, I think what I should do is to send the version I queued with fixups back to the list as follow-up. Just like reviewers review, the maintainer reviews and queues, the original author should be able to work in the same workflow, i.e. reading and applying an improved version of the patch from her mailbox.

Jeff King· Aug 3, 2016, 16:34 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Wed, Aug 03, 2016 at 08:33:12AM -0700, Junio C Hamano wrote:
Show 11 quoted lines
> On Wed, Aug 3, 2016 at 4:59 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > I disagree, however, with the suggestion to sift through your `pu` branch
> > and to somehow replace local branches with the commits found there.
> 
> To be more in line with the "e-mailed patch" workflow, I think what I should
> do is to send the version I queued with fixups back to the list as follow-up.
> Just like reviewers review, the maintainer reviews and queues, the original
> author should be able to work in the same workflow, i.e. reading and applying
> an improved version of the patch from her mailbox.

Leaving aside Dscho's questions of whether pulling patches from email is convenient for most submitters (it certainly is for me, but I recognize that it is not for many), I would much rather see incremental fixup patches from you than whole "here's what I queued" responses.

The reason is that your fixups may not be the only ones needed. There may be others on the list that come before or after, and I may even have already made fixes locally for "v2" that haven't been on the list. If I haven't made any changes yet, I can throw out my topic, start with what you queued, and then apply other changes incrementally. But if I have, then I need to convert yours to a diff, which requires checking out the same base, applying yours, and running diff. Much easier to get the diff in the first place. :)

That only covers changes to the code, though. It does not help with fixups to commit messages. It would be neat to have a microformat for specifying and applying patches to commit messages.

-Peff
Junio C Hamano· Aug 3, 2016, 16:53 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Jeff King <peff@peff.net> writes:
Show 18 quoted lines
> On Wed, Aug 03, 2016 at 08:33:12AM -0700, Junio C Hamano wrote:
>
>> On Wed, Aug 3, 2016 at 4:59 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> >
>> > I disagree, however, with the suggestion to sift through your `pu` branch
>> > and to somehow replace local branches with the commits found there.
>> 
>> To be more in line with the "e-mailed patch" workflow, I think what I should
>> do is to send the version I queued with fixups back to the list as follow-up.
>> Just like reviewers review, the maintainer reviews and queues, the original
>> author should be able to work in the same workflow, i.e. reading and applying
>> an improved version of the patch from her mailbox.
>
> Leaving aside Dscho's questions of whether pulling patches from email is
> convenient for most submitters (it certainly is for me, but I recognize
> that it is not for many), I would much rather see incremental fixup
> patches from you than whole "here's what I queued" responses.

Ah, yes, I misspoke. It should be either an incremental diff or in-line comment to spell out what got changed as a response to the patch.

I find myself fixing the title the most often, which is part of the "log message" you pointed out that would not convey well with the "incremental diff" approach.

Jeff King· Aug 3, 2016, 16:56 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Wed, Aug 03, 2016 at 09:53:18AM -0700, Junio C Hamano wrote:
Show 12 quoted lines
> > Leaving aside Dscho's questions of whether pulling patches from email is
> > convenient for most submitters (it certainly is for me, but I recognize
> > that it is not for many), I would much rather see incremental fixup
> > patches from you than whole "here's what I queued" responses.
> 
> Ah, yes, I misspoke.  It should be either an incremental diff or
> in-line comment to spell out what got changed as a response to the
> patch.
> 
> I find myself fixing the title the most often, which is part of the
> "log message" you pointed out that would not convey well with the
> "incremental diff" approach.

I mentioned a micro-format elsewhere in my message. And it certainly is nice to have something that can be applied in an automatic way. But in practice, most review comments, for the commit message _or_ the text, are given in human-readable terms. And as a human, I read and apply them in sequence.

That pushes work onto the submitter, but saves work from the reviewers, who can quickly say "something like this..." without having to worry about making a full change, formatting it as a diff, etc.

I do think that's the right time-tradeoff to be making, as we have more submitters than reviewers.

-Peff
Johannes Schindelin· Aug 4, 2016, 15:29 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Peff,
On Wed, 3 Aug 2016, Jeff King wrote:
Show 17 quoted lines
> On Wed, Aug 03, 2016 at 09:53:18AM -0700, Junio C Hamano wrote:
> 
> > > Leaving aside Dscho's questions of whether pulling patches from email is
> > > convenient for most submitters (it certainly is for me, but I recognize
> > > that it is not for many), I would much rather see incremental fixup
> > > patches from you than whole "here's what I queued" responses.
> > 
> > Ah, yes, I misspoke.  It should be either an incremental diff or
> > in-line comment to spell out what got changed as a response to the
> > patch.
> > 
> > I find myself fixing the title the most often, which is part of the
> > "log message" you pointed out that would not convey well with the
> > "incremental diff" approach.
> 
> I mentioned a micro-format elsewhere in my message. And it certainly is
> nice to have something that can be applied in an automatic way.

Indeed. This is what I meant by my (succinct to the point of being intelligible, admittedly) reword! suggestion.

Let's clarify this idea.

I find myself using fixup! and squash! commits a lot. Actually, let me pull out the Linux key for that. I use those commits A LOT.

I know, I opposed the introduction of this feature initially (and I think that my concerns were nicely addressed by Junio's suggestion to guard this feature behind the --autosquash option). Guess what: I was wrong.

And I am really missing the same functionality for the commit message munging. These days, I find myself using `git commit --allow-empty --squash=$COMMIT -c $COMMIT` very often, duplicating the first line, adding an empty line between them, deleting the "squash! " prefix from the now-third line, and then editing the commit message as I want to. When it comes to cleaning up the branch via rebase -ki, I simply jump to the empty line after the squash! line and delete everything before it.

This is as repetitive, tedious and un-fun to me as having to transmogrify patches from the nice and cozy Git universe into the not-at-all compatible universe of mails (I congratulate you personally, Peff, for finding a mail client that works for you. I am still looking for one that does not suck, Alpine being the least sucky I settled for).

So my idea was to introduce a new --reword=<commit> option to `git commit` that would commit an empty patch and let the user edit the commit message, later replacing the original one with the new one. This is not *quite* as nice as I want it, because it makes the changes unobvious. On the other hand, I would not find a series of sed commands more obvious, in particular because that limits you in the ways of sed. And, you know, regexp. I like them, but I know many people cannot really work with them.

> But in practice, most review comments, for the commit message _or_ the
> text, are given in human-readable terms. And as a human, I read and
> apply them in sequence.
So true. I do the very same.
Show 6 quoted lines
> That pushes work onto the submitter, but saves work from the reviewers,
> who can quickly say "something like this..." without having to worry
> about making a full change, formatting it as a diff, etc.
> 
> I do think that's the right time-tradeoff to be making, as we have more
> submitters than reviewers.

I agree that it is the right trade-off. TBH I was shocked when I learned how much effort Junio puts into applying my patches. I do not want that. I want my branch to reflect pretty precisely (modulo sign-off, of course) what is going to be integrated into Git's source code.

I'd much prefer to resubmit a cleaned-up version, even if it was just the commit subjects, and be certain that `pu` and my branch are on the same page.

Instead, Junio puts in a tremendous amount of work, and it does not help anybody, because the local branches *still* do not have his fixups, and as a consequence subsequent iterations of the patch series will have to be fixed up *again*.

Just compare https://github.com/git/git/compare/1fd7e78...6999bc7 to https://github.com/dscho/git/compare/f8f7adc...3b4494c (the onelines are enough to show you just how different things are).

I'd much prefer the contributor (me, in this case) to put in a little more work, and have things consistent. And avoid unnecessary work on both sides.

Ciao, Dscho

Jeff King· Aug 4, 2016, 18:07 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Thu, Aug 04, 2016 at 05:29:52PM +0200, Johannes Schindelin wrote:
Show 7 quoted lines
> So my idea was to introduce a new --reword=<commit> option to `git commit`
> that would commit an empty patch and let the user edit the commit message,
> later replacing the original one with the new one. This is not *quite* as
> nice as I want it, because it makes the changes unobvious. On the other
> hand, I would not find a series of sed commands more obvious, in
> particular because that limits you in the ways of sed. And, you know,
> regexp. I like them, but I know many people cannot really work with them.

I don't have a real opinion on this. I probably wouldn't use it, but I have no problem with it existing. I think it's somewhat orthogonal to the idea of _transmitting_ those reword operations to somebody else.

Show 11 quoted lines
> > That pushes work onto the submitter, but saves work from the reviewers,
> > who can quickly say "something like this..." without having to worry
> > about making a full change, formatting it as a diff, etc.
> > 
> > I do think that's the right time-tradeoff to be making, as we have more
> > submitters than reviewers.
> 
> I agree that it is the right trade-off. TBH I was shocked when I learned
> how much effort Junio puts into applying my patches. I do not want that. I
> want my branch to reflect pretty precisely (modulo sign-off, of course)
> what is going to be integrated into Git's source code.

Like you, I have occasionally been bitten by Junio doing a fixup, and then I end up re-rolling, and lose that fixup (or have to deal with porting it forward with awkward tools).

But I think such fixups are a calculated risk. Sometimes they save a lot of time, both for the maintainer and the contributor, when they manage to prevent another round-trip of the patch series to the list.

IOW, if the flow is something like:
  1. Contributor sends patches. People review.
  2. Minor fixups noticed by maintainer, fixed while applying.
  3. Only one small fixup needed from review. Contributor sends
     squashable patch. Maintainer squashes.

then I think that is a net win over sending the whole series again, for the contributor (who does not bother sending), reviewers (who really only need to look at the interdiff, which is what that squash is in the first place), and the maintainer (who can squash just as easily as re-applying the whole series).

It does mean the "final" version of the series is never on the list. It has to be pieced together from the squash (and sometimes step 2 is not even mentioned on-list).

So I think it is really a judgement call for step (3) on what is a "small" fixup, and whether it is easier for everybody to look at the squash interdiff and say "yep, that's right", versus re-reviewing the whole series.

Show 8 quoted lines
> I'd much prefer to resubmit a cleaned-up version, even if it was just the
> commit subjects, and be certain that `pu` and my branch are on the same
> page.
> 
> Instead, Junio puts in a tremendous amount of work, and it does not help
> anybody, because the local branches *still* do not have his fixups, and as
> a consequence subsequent iterations of the patch series will have to be
> fixed up *again*.

And that is the flip side. If the flow above does not happen, then step 2 just becomes a pain.

I don't have a silver bullet or anything. I'm mostly just musing.
-Peff
Junio C Hamano· Aug 4, 2016, 21:12 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Jeff King <peff@peff.net> writes:
> Like you, I have occasionally been bitten by Junio doing a fixup, and
> then I end up re-rolling, and lose that fixup.

... which usually is caught when I receive the reroll, as I try to apply to the same base and compare the result with the previous round.

> But I think such fixups are a calculated risk. Sometimes they save a lot
> of time, both for the maintainer and the contributor, when they manage
> to prevent another round-trip of the patch series to the list.
Yes.
Show 5 quoted lines
> IOW, if the flow is something like:
>
>   1. Contributor sends patches. People review.
>
>   2. Minor fixups noticed by maintainer, fixed while applying.
This includes different kinds of things:
    a) Trivially correct fixes given in other people's review.
    b) Minor fixups by the maintainer, to code.
    c) Minor fixups by the maintainer, to proposed log message.
    d) "apply --whitespace=fix" whose result I do not even actively
       keep track of.
Show 8 quoted lines
>   3. Only one small fixup needed from review. Contributor sends
>      squashable patch. Maintainer squashes.
>
> then I think that is a net win over sending the whole series again, for
> the contributor (who does not bother sending), reviewers (who really
> only need to look at the interdiff, which is what that squash is in the
> first place), and the maintainer (who can squash just as easily as
> re-applying the whole series).
> And that is the flip side. If the flow above does not happen, then step
> 2 just becomes a pain.
I think I can
 * stop taking 2-a).  This is less work for me, but some
   contributors are leaky and can lose obviously good suggestions,
   so I am not sure if that is an overall win for the quality of the
   end product;
 * do a separate "SQUASH???" commit and send them out for 2-b).
   This is a lot more work for a large series, but about the same
   amount of work (except for "remembering to send them out" part)
   for a small-ish topic.  A contributor needs to realize that I
   deal with _all_ the incoming topics, not just from topics from
   one contributor, and small additional work tends to add up.

to reduce #2. Essentially, doing these two are about moving more responsibility of keeping track of good suggestions in the review discussion to the contributor, but a bad thing is that it does not mean that the maintainer can stop keeping track of them. We need a way to make sure leaky contributors do not repeat the same issue in their reroll that has already been pointed out and whose solutions presented in the previous review. Fetching from 'pu' and compare has been one way to do so (that is why I publish 'pu' in the first place, not to "build on", but to "view"), but apparently not many contributors are comfortable with that idea.

There is no good way to reduce 2-c) and 2-d), but on the other hand, there is no strong need for a special channel to propagate these back. 2-c) can be a regular review comment (but again that risks "the leaky contributor" problem) and 2-d) will happen automatically when applying the rerolled version.

Jeff King· Aug 5, 2016, 08:17 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Thu, Aug 04, 2016 at 02:12:22PM -0700, Junio C Hamano wrote:
Show 21 quoted lines
> >   2. Minor fixups noticed by maintainer, fixed while applying.
> 
> This includes different kinds of things:
> 
>     a) Trivially correct fixes given in other people's review.
> 
>     b) Minor fixups by the maintainer, to code.
> 
>     c) Minor fixups by the maintainer, to proposed log message.
> 
>     d) "apply --whitespace=fix" whose result I do not even actively
>        keep track of.
>
> [...]
>
> I think I can
> 
>  * stop taking 2-a).  This is less work for me, but some
>    contributors are leaky and can lose obviously good suggestions,
>    so I am not sure if that is an overall win for the quality of the
>    end product;

Actually, I think the 2-a class is what often saves a re-roll. Somebody points out a typo in a commit message or a comment, and it quite often gets picked up by you without having another round-trip to the list.

If you want to save work by not doing so, that's fine with me. But this is the gamble I was talking about. I think it's actually often less work to do the fixup than to look at another re-roll (especially with the "leaky contributor" thing where you have to make sure all fixes were applied). So it's a win if it saves the re-roll, but sometimes you end up having to look at the re-roll anyway.

Show 6 quoted lines
>  * do a separate "SQUASH???" commit and send them out for 2-b).
>    This is a lot more work for a large series, but about the same
>    amount of work (except for "remembering to send them out" part)
>    for a small-ish topic.  A contributor needs to realize that I
>    deal with _all_ the incoming topics, not just from topics from
>    one contributor, and small additional work tends to add up.

I think these are largely the same as 2-a. You are just wearing two hats, reviewer and maintainer. Which I guess lets you take a shortcut sometimes (and just fix without mentioning it), but fundamentally the "gamble" aspect is the same, I think.

-Peff
Johannes Schindelin· Aug 5, 2016, 15:51 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Thu, 4 Aug 2016, Junio C Hamano wrote:
Show 7 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > Like you, I have occasionally been bitten by Junio doing a fixup, and
> > then I end up re-rolling, and lose that fixup.
> 
> ... which usually is caught when I receive the reroll, as I try to apply
> to the same base and compare the result with the previous round.

I find it incredibly nice of you to do those fix-ups, sorry to state that as clearly only now. It's just that I hate to see your time wasted, in particular when *I* waste it because I missed the fix-ups in `pu`.

Show 5 quoted lines
> > But I think such fixups are a calculated risk. Sometimes they save a
> > lot of time, both for the maintainer and the contributor, when they
> > manage to prevent another round-trip of the patch series to the list.
> 
> Yes.

FWIW I am more than just willing to spend a little more time on applying fix-ups and re-rolling patch series (and dual-publishing them via my public repository), if it helps lighten your burden.

Show 35 quoted lines
> > IOW, if the flow is something like:
> >
> >   1. Contributor sends patches. People review.
> >
> >   2. Minor fixups noticed by maintainer, fixed while applying.
> 
> This includes different kinds of things:
> 
>     a) Trivially correct fixes given in other people's review.
> 
>     b) Minor fixups by the maintainer, to code.
> 
>     c) Minor fixups by the maintainer, to proposed log message.
> 
>     d) "apply --whitespace=fix" whose result I do not even actively
>        keep track of.
> 
> >   3. Only one small fixup needed from review. Contributor sends
> >      squashable patch. Maintainer squashes.
> >
> > then I think that is a net win over sending the whole series again, for
> > the contributor (who does not bother sending), reviewers (who really
> > only need to look at the interdiff, which is what that squash is in the
> > first place), and the maintainer (who can squash just as easily as
> > re-applying the whole series).
> 
> > And that is the flip side. If the flow above does not happen, then step
> > 2 just becomes a pain.
> 
> I think I can
> 
>  * stop taking 2-a).  This is less work for me, but some
>    contributors are leaky and can lose obviously good suggestions,
>    so I am not sure if that is an overall win for the quality of the
>    end product;

If you had a `git commit --reword` command to touch up commit messages, would that help you, together with the `git commit --fixup` command for code changes? The branches in `pu` would have your fix-ups as strictly separate commits on top of the contributed patches, and the branches would need to be sent through rebase -i before merging to `next`, of course.

The idea would be to not forget your fixups in subsequent iterations, but simply rebase them on top of the new iteration.

It would still not solve my problem that there is no convenient way to jump from your commits in `pu` to the corresponding ones in my local branch. But that is my problem, not yours.

Ciao, Dscho

Johannes Schindelin· Aug 3, 2016, 16:07 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Wed, 3 Aug 2016, Junio C Hamano wrote:
Show 11 quoted lines
> On Wed, Aug 3, 2016 at 4:59 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > I disagree, however, with the suggestion to sift through your `pu` branch
> > and to somehow replace local branches with the commits found there.
> 
> To be more in line with the "e-mailed patch" workflow, I think what I should
> do is to send the version I queued with fixups back to the list as follow-up.
> Just like reviewers review, the maintainer reviews and queues, the original
> author should be able to work in the same workflow, i.e. reading and applying
> an improved version of the patch from her mailbox.

You seem to assume that it isn't cumbersome for people like me to extract patches out of mails and to replace existing commits using those patches.

So it probably comes as a huge surprise to you to learn that this *is* cumbersome for me.

I got too used to the ease of git push, git pull with or without --rebase, and many other Git commands. Having to transmogrify code changes from commits in Git into a completely different universe: plain text patches in my mailbox, and back, losing all kinds of data in the process, is just not at all that easy. And it costs a lot of time.

In short: if you start "submitting patches" back to me via mail, it does not help me. It makes things harder for me. In particular when you add your sign-off to every patch and I have to strip it.

If you change your workflow, I would humbly request that you do it in a way that makes things easier on both sides, not harder.

It would be a totally different matter, of course, if you used the branches I publish via my GitHub repository, added fixup! and squash! commits, published the result to a public repository and then told me to pull from there, that would make things easier. We could even introduce a reword! construct, to make the review of the suggested edits of the commit message easier. I could easily verify that my branch head agrees with the base commit of your branch, I could build proper tooling around this workflow, and it would lighten my load.

I guess what I am saying is that we might just as well start using this awesome tool to work with code, that tool named "Git".

Ciao, Dscho

Stefan Beller· Aug 3, 2016, 17:47 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Wed, Aug 3, 2016 at 9:07 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 21 quoted lines
> Hi Junio,
>
> On Wed, 3 Aug 2016, Junio C Hamano wrote:
>
>> On Wed, Aug 3, 2016 at 4:59 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> >
>> > I disagree, however, with the suggestion to sift through your `pu` branch
>> > and to somehow replace local branches with the commits found there.
>>
>> To be more in line with the "e-mailed patch" workflow, I think what I should
>> do is to send the version I queued with fixups back to the list as follow-up.
>> Just like reviewers review, the maintainer reviews and queues, the original
>> author should be able to work in the same workflow, i.e. reading and applying
>> an improved version of the patch from her mailbox.
>
> You seem to assume that it isn't cumbersome for people like me to extract
> patches out of mails and to replace existing commits using those patches.
>
> So it probably comes as a huge surprise to you to learn that this *is*
> cumbersome for me.

It is also cumbersome for me, because I never had the need to setup a proper mail client that has the strength to apply patches. The need was not there as I tend to apply only rarely patches by email, so I can go the painful way each time.

But if we as a community decide that we bounce emails back and forth, I (and you) may have to find a proper email client that is easy to work with, e.g. one key shortcut to apply a patch series to the HEAD of your local repository.

Show 10 quoted lines
>
> I got too used to the ease of git push, git pull with or without --rebase,
> and many other Git commands. Having to transmogrify code changes from
> commits in Git into a completely different universe: plain text patches in
> my mailbox, and back, losing all kinds of data in the process, is just
> not at all that easy. And it costs a lot of time.
>
> In short: if you start "submitting patches" back to me via mail, it does
> not help me. It makes things harder for me. In particular when you add
> your sign-off to every patch and I have to strip it.

You don't have to strip the sign off, as it shows the flow of the patch, e.g.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

may indicate you proposed a patch, Junio picked it up (and fixed a typo optionally), I obtained the patch (via mail, via Git?) improved it, you improved it further and then Junio took it and merged it upstream.

>
> If you change your workflow, I would humbly request that you do it in a
> way that makes things easier on both sides, not harder.

When attending the Git Merge conference in May, gregkh said roughtly: "We deliberately waste developers time, because it is not scarce. Maintainers time is scarce however " and it stuck with me. (and I am a developer, not a maintainer ;( so at least the kernel community deems it ok to waste my time).

While that is true for the kernel community, I guess it is also true for the Git community, unless Junio (and the community) want to appoint a bunch of maintainer lieutenants, such that they outnumber the number of developers, e.g. divided by areas of the code: a refs backend maintainer, a submodule maintainer, ... or rather by area of usage: a porcelain UI maintainer, a git-on-server maintainer.

Though Git is not as diverse and large as the kernel, so the horde of maintainers would step onto each feet quite frequently IMHO.

Show 12 quoted lines
>
> It would be a totally different matter, of course, if you used the
> branches I publish via my GitHub repository, added fixup! and squash!
> commits, published the result to a public repository and then told me to
> pull from there, that would make things easier. We could even introduce a
> reword! construct, to make the review of the suggested edits of the commit
> message easier. I could easily verify that my branch head agrees with the
> base commit of your branch, I could build proper tooling around this
> workflow, and it would lighten my load.
>
> I guess what I am saying is that we might just as well start using this
> awesome tool to work with code, that tool named "Git".

I think Git itself is for the tracking the code and managing it, e.g. merging, moving, keeping it. That doesn't quite include modifying and creating code (e.g. there is no "git edit" command)

If we were to change our workflows drastically, I'd propose to go a way[1] similar to notedb in Gerrit, or git-series, which defines a common review format, such that we have a "protocol" how to store the review data and how to store the progress of potential collaboration and then we can develop tools against that protocol. Some people want to have a web UI, whereas others want to have a text only thing as they are faster keyboard only.

[1] http://git.661346.n2.nabble.com/Working-towards-a-common-review-format-for-git-td7645242.html

Thanks for starting this discussion, Stefan

Johannes Schindelin· Aug 4, 2016, 15:58 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Stefan,
On Wed, 3 Aug 2016, Stefan Beller wrote:
Show 30 quoted lines
> On Wed, Aug 3, 2016 at 9:07 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 3 Aug 2016, Junio C Hamano wrote:
> >
> >> On Wed, Aug 3, 2016 at 4:59 AM, Johannes Schindelin
> >> <Johannes.Schindelin@gmx.de> wrote:
> >> >
> >> > I disagree, however, with the suggestion to sift through your `pu`
> >> > branch and to somehow replace local branches with the commits found
> >> > there.
> >>
> >> To be more in line with the "e-mailed patch" workflow, I think what I
> >> should do is to send the version I queued with fixups back to the
> >> list as follow-up.  Just like reviewers review, the maintainer
> >> reviews and queues, the original author should be able to work in the
> >> same workflow, i.e. reading and applying an improved version of the
> >> patch from her mailbox.
> >
> > You seem to assume that it isn't cumbersome for people like me to
> > extract patches out of mails and to replace existing commits using
> > those patches.
> >
> > So it probably comes as a huge surprise to you to learn that this *is*
> > cumbersome for me.
> 
> It is also cumbersome for me, because I never had the need to setup a
> proper mail client that has the strength to apply patches. The need was
> not there as I tend to apply only rarely patches by email, so I can go
> the painful way each time.

The reason is clear, too. Mail clients serve humans. That is their purpose. Humans do not care all that much whether the text was preserved exactly as the sender wrote it, except rich text (read: HTML), of course.

Show 23 quoted lines
> > I got too used to the ease of git push, git pull with or without
> > --rebase, and many other Git commands. Having to transmogrify code
> > changes from commits in Git into a completely different universe:
> > plain text patches in my mailbox, and back, losing all kinds of data
> > in the process, is just not at all that easy. And it costs a lot of
> > time.
> >
> > In short: if you start "submitting patches" back to me via mail, it
> > does not help me. It makes things harder for me. In particular when
> > you add your sign-off to every patch and I have to strip it.
> 
> You don't have to strip the sign off, as it shows the flow of the patch,
> e.g.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> 
> may indicate you proposed a patch, Junio picked it up (and fixed a typo
> optionally), I obtained the patch (via mail, via Git?) improved it, you
> improved it further and then Junio took it and merged it upstream.

Recently, I got yelled at because I took one of Junio's patches, made a couple of changes, and *added* my sign-off.

Before that incident, I agreed with you that it may make for a nice record of the back-and-forth that eventually resulted in the patch in question. Now, I am not so sure anymore.

Show 8 quoted lines
> > If you change your workflow, I would humbly request that you do it in
> > a way that makes things easier on both sides, not harder.
> 
> When attending the Git Merge conference in May, gregkh said roughtly:
> "We deliberately waste developers time, because it is not scarce.
> Maintainers time is scarce however " and it stuck with me. (and I am a
> developer, not a maintainer ;( so at least the kernel community deems it
> ok to waste my time).

Yeah. It was not the only thing I disagreed with in his talk. To be a little bit blunt (by my standards, not by LKML's standards, that is): the Linux kernel mailing list is not necessarily anything I would want to use as a role model.

I agree that maintainers' time is scarce.
I am one.

So of course I agree with that statement. What I disagree with is that it is okay to *waste* contributors' time. That's just inconsiderate. And I say that also because I am a contributor *in addition* to being a maintainer.

As a consequence, I commend Greg for recognizing that the patch submission process must be light on the maintainer. And I would have commended him even further if he had realized that proper tooling should waste nothing, and no one's time.

Show 6 quoted lines
> While that is true for the kernel community, I guess it is also true for
> the Git community, unless Junio (and the community) want to appoint a
> bunch of maintainer lieutenants, such that they outnumber the number of
> developers, e.g. divided by areas of the code: a refs backend
> maintainer, a submodule maintainer, ...  or rather by area of usage: a
> porcelain UI maintainer, a git-on-server maintainer.
As I mentioned earlier, I do not care much about following LKML's example.

What I see on this here list is that many a potential contributor is scared away, that we waste precious time (also the maintainer's) pointing out in what way certain contributions do not follow the guide lines, and that even old-timers sometimes submit patches that are white-space corrupted.

That is a *huge* waste of time. In my opinion, the culprit is that we do not use appropriate tools. To a mail client, everything looks like a nail. Wrong metaphor, but you get the point.

Show 15 quoted lines
> > It would be a totally different matter, of course, if you used the
> > branches I publish via my GitHub repository, added fixup! and squash!
> > commits, published the result to a public repository and then told me
> > to pull from there, that would make things easier. We could even
> > introduce a reword! construct, to make the review of the suggested
> > edits of the commit message easier. I could easily verify that my
> > branch head agrees with the base commit of your branch, I could build
> > proper tooling around this workflow, and it would lighten my load.
> >
> > I guess what I am saying is that we might just as well start using this
> > awesome tool to work with code, that tool named "Git".
> 
> I think Git itself is for the tracking the code and managing it, e.g.
> merging, moving, keeping it. That doesn't quite include modifying and
> creating code (e.g. there is no "git edit" command)

Git is not only for tracking the code. It knows about editors (core.editor), it can export .zip files (git-archive), it can show human-readable (not machine-readable) word diffs, etc.

Whenever we needed a certain functionality, we added it.
> If we were to change our workflows drastically, I'd propose to
> go a way[1] similar to notedb in Gerrit, or git-series,

Gerrit is a huge, non-distributed system. Complex, too. If we change the patch submission process, we should make things *easier*, not *harder*. So I think Gerrit is pretty much out of the question.

Even requiring every contributor to register with GitHub would be too much of a limitation, I would wager.

And when I said I have zero interest in tools that use the "latest and greatest language", I was hinting at git-series. Rust may be a fine and wonderful language. Implementing git-series in Rust, however, immediately limited the potential engagement with developers dramatically.

Additionally, I would like to point out that defining a way to store reviews in Git is not necessarily improving the way our code contribution process works. If you want to record the discussions revolving around the code, I think public-inbox already does a pretty good job at that.

I guess I have no really good idea yet, either, how to retain the ease of access of sending mails to the list, yet somehow keep a strong tie with the original data stored in Git.

Ciao, Dscho

Stefan Beller· Aug 4, 2016, 16:42 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 7 quoted lines
>
>> If we were to change our workflows drastically, I'd propose to
>> go a way[1] similar to notedb in Gerrit, or git-series,
>
> Gerrit is a huge, non-distributed system. Complex, too. If we change the
> patch submission process, we should make things *easier*, not *harder*. So
> I think Gerrit is pretty much out of the question.
I did not propose to use Gerrit or git-series or git appraise.

However whenever I see these projects talking to each other, the talk focuses on a "unified review storage format" pretty often, which would make them compatible with each other. So then I could choose git-series, while you could go with git appraise (although that is written in go, so maybe too fancy already ;) Or there could be another new program written in C that follows the "review format".

Show 13 quoted lines
>
> Even requiring every contributor to register with GitHub would be too much
> of a limitation, I would wager.
>
> And when I said I have zero interest in tools that use the "latest and
> greatest language", I was hinting at git-series. Rust may be a fine and
> wonderful language. Implementing git-series in Rust, however, immediately
> limited the potential engagement with developers dramatically.
>
> Additionally, I would like to point out that defining a way to store
> reviews in Git is not necessarily improving the way our code contribution
> process works. If you want to record the discussions revolving around the
> code, I think public-inbox already does a pretty good job at that.

Yeah recording is great, but we want to talk about replying and modifying a series? So if I see a patch flying by on the mailing list, ideally I could attach a "!fixup, signed off by Stefan" thing to that patch. (I said "thing" as I do not necessarily mean email here.

>
> I guess I have no really good idea yet, either, how to retain the ease of
> access of sending mails to the list, yet somehow keep a strong tie with
> the original data stored in Git.
Does it have to be email? Transmitting text could be solved differently as well.

With git push/fetch we can interact with a git remote and pertain the state (commits, ancestor graph) at a full level even including notes that comment on commits.

git send-email/format-patch recently learned to include a base commit (xy/format-patch-base), maybe we need a counter part to git send-email that downloads a series from your mailbox, such that a local branch can be transmitted to via

    "git send-email --base=origin/master --include-notes --name=sb/new-series"

and completely reconstructed (i.e. the commit sha1s even match) including notes via:

    git fetch-email --name=sb/new-series

That way would ensure we have a "simple" way to transmit patches back and forth and adding potential fixups.

You wrote:
Show 6 quoted lines
> In short, I agree that our patch submission process is a saber tooth tiger
> that still reflects pre-Git times. While we use Git's tools, the workflow
> really tries to cut out Git as much as possible, in favor of pure mails
> with non-corrupted, non-HTML patches in them, a charmingly anachronistic
> requirement until you try to use state-of-the-art mail clients to send
> them.
And there are two ways out:
* either we teach git how to deal with emails (completely, i.e.
sending+receiving)
* or we change the development model (e.g. no emails any more)
There is no golden third way IMHO.

Thanks, Stefan

Eric Wong· Aug 4, 2016, 20:17 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Stefan Beller <sbeller@google.com> wrote:
Show 8 quoted lines
> On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > I guess I have no really good idea yet, either, how to retain the ease of
> > access of sending mails to the list, yet somehow keep a strong tie with
> > the original data stored in Git.
> 
> Does it have to be email? Transmitting text could be solved
> differently as well.

I've thought a lot about this over the years still think email is the least bad.

Anti-spam tools for other messaging systems are far behind, proprietary, or non-existent. bugzilla.kernel.org has been hit hard lately and I see plenty of bug-tracker-to-mail spam as a result from projects that use web-based bug trackers.

And email spam filtering isn't even that great
(and I think it needs to be better for IPv6 and .onion adoption
 since much of it is still IPv4-oriented blacklisting).

I guess a blockchain (*coin) implementation might work (like hashcash is used for email anti-spam), but the ones I've glanced at all look like a bigger waste of electricity than email filters.

Of course, centralized systems are unacceptable to me; and with that I'll never claim any network service I run will be reliable :)

Johannes Schindelin· Aug 5, 2016, 08:24 UTC · re: Eric Wong · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Eric,
On Thu, 4 Aug 2016, Eric Wong wrote:
Show 12 quoted lines
> Stefan Beller <sbeller@google.com> wrote:
> > On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > > I guess I have no really good idea yet, either, how to retain the ease of
> > > access of sending mails to the list, yet somehow keep a strong tie with
> > > the original data stored in Git.
> > 
> > Does it have to be email? Transmitting text could be solved
> > differently as well.
> 
> I've thought a lot about this over the years still think email
> is the least bad.

Not only that: people are *familiar* with it. And they have *access* to it.

> Anti-spam tools for other messaging systems are far behind,
> proprietary, or non-existent.  bugzilla.kernel.org has been hit
> hard lately and I see plenty of bug-tracker-to-mail spam as a
> result from projects that use web-based bug trackers.

Plus, they are all centralized. Do you want to *require* contibutors to register with a new service?

> I guess a blockchain (*coin) implementation might work (like
> hashcash is used for email anti-spam), but the ones I've glanced
> at all look like a bigger waste of electricity than email
> filters.

I am not even so much concerned with ecological considerations here. Just the price of entry would be prohibitive.

> Of course, centralized systems are unacceptable to me;
> and with that I'll never claim any network service I run
> will be reliable :)

Hehehe. I guess that's why the public-inbox is backed by a Git repository... BTW is it auto-mirrored anywhere?

Ciao, Dscho

Eric Wong· Aug 5, 2016, 08:50 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Agreed on all above points :>
Show 7 quoted lines
> On Thu, 4 Aug 2016, Eric Wong wrote:
> > Of course, centralized systems are unacceptable to me;
> > and with that I'll never claim any network service I run
> > will be reliable :)
> 
> Hehehe. I guess that's why the public-inbox is backed by a Git
> repository... BTW is it auto-mirrored anywhere?
Yep, and the code is AGPL so others can always replicate it.

I just have the onions mentioned at the bottom of the HTML pages running something like:

	torsocks git fetch && public-inbox-index && sleep 30
in a loop[1].
	http://hjrcffqmbrq6wope.onion/git
	http://czquwvybam4bgbro.onion/git

I do encourage anybody who is able to, to run their own mirrors (off their existing email subscription) so my MX doesn't become an SPOF, either. I sorta documented it in my original announcement:

https://public-inbox.org/git/20160710004813.GA20210@dcvr.yhbt.net/ https://public-inbox.org/INSTALL

Feel free to ask me (+ meta@public-inbox.org) for install/running questions related to public-inbox (haven't gotten to making it work outside of Debian stable, though).

[1] one of my far-off goals for git be: "git fetch --wait-for-update"
    to avoid needless wakeups
Johannes Schindelin· Aug 5, 2016, 08:20 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Stefan,
On Thu, 4 Aug 2016, Stefan Beller wrote:
Show 15 quoted lines
> On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> >> If we were to change our workflows drastically, I'd propose to go a
> >> way[1] similar to notedb in Gerrit, or git-series,
> >
> > Gerrit is a huge, non-distributed system. Complex, too. If we change
> > the patch submission process, we should make things *easier*, not
> > *harder*. So I think Gerrit is pretty much out of the question.
> 
> I did not propose to use Gerrit or git-series or git appraise.
> 
> However whenever I see these projects talking to each other, the talk
> focuses on a "unified review storage format" pretty often, which would
> make them compatible with each other.

Unless you have a splendid idea how to integrate that unified format with our review process on the mailing list, I think this makes for a fine discussion elsewhere. I'd really like to focus on the Git project and its patch contribution process in this here thread.

> So then I could choose git-series, while you could go with git appraise
> (although that is written in go, so maybe too fancy already ;)
I think you misunderstood me in a big way.

New languages are awesome. I play with new toys whenever I find the time (and streamlining the contribution process would give me more time for that). You are talking to a person who implemented the Householder transformation in Postscript, a 6502 assembler in Forth, and a music composing system in Emacs Lisp. No language is to fancy for me. "me", as in "me, personally".

Now let's think about Git for a moment, and its language choices, and the rationale behind them. The majority of the critically important code is written in C. Is C a good language? Decent, yes, but of course also limiting, Resource leaks are very easy to overlook, and we have a share of them. No object orientation, so when we need to "subclass", we have no compile time safety. The pre-processor constructs make static analysis nigh impossible. Plenty of downsides. So why was it chosen? Developers are *familiar* with it, that's why. Similar considerations apply to the use of shell scripts, and to Perl, to a certain extent.

I am not talking about contrib/ of course. That's fair game, it contains only non-critical/fringe stuff.

Note that the same rationale goes for choosing to accept patch submissions via mail to a list that is not subscribers-only.

When it comes to inviting developers to contribute to your project, personal preferences become irrelevant, the deciding factor becomes how easy it is to join. Is the language popular, many developers already familiar with it? Is the build system readily available? Are the maintainers responsive?

I vividly remember my reaction to Darcs, for example. It's written in Haskell. I am a mathematician originally, so Haskell appeals to me. Did the choice of language appear to be designed to keep contributors out? To me, it looked that way.

Other example: submitGit. I really like what its intention. For a while, I even hoped to move my *own* patch submissions to submitGit. I planned to help get the kinks out of the code by contributing to it. It is written in Scala, using a web application and testing framework I have not encountered elsewhere. I struggled with installing it locally and wrapping my head around the coding paradigms, for 1.5 days (which was all I could afford at that time). Then I had to give up. Which made me very sad. I would not have written my mail-path-series.sh tool if submitGit had been written in node.js, for example, with which I am familiar enough to jump right in.

So I hope you understand better now why I find Rust a poor choice for something like git-series, because it should not waste contributors' time by insisting that their price of entry is learning a new language they are unfamiliar with, using a new packaging system, installing a new build setup. I would find Clojure, Crystal or Swift just as poor a choice. Even node.js. It is just too much of a "Keep Out" sign for busy developers. And all the developers worth their salt are busy.

Show 10 quoted lines
> > Additionally, I would like to point out that defining a way to store
> > reviews in Git is not necessarily improving the way our code
> > contribution process works. If you want to record the discussions
> > revolving around the code, I think public-inbox already does a pretty
> > good job at that.
> 
> Yeah recording is great, but we want to talk about replying and
> modifying a series? So if I see a patch flying by on the mailing list,
> ideally I could attach a "!fixup, signed off by Stefan" thing to that
> patch. (I said "thing" as I do not necessarily mean email here.

Right. I briefly considered suggesting a new tool that would operate on attachments, integrating tightly with the local git.git checkout. Briefly. I had to reject this idea because I do not think that requiring new tools just to contribute to Git would fly well.

Show 6 quoted lines
> > I guess I have no really good idea yet, either, how to retain the ease
> > of access of sending mails to the list, yet somehow keep a strong tie
> > with the original data stored in Git.
> 
> Does it have to be email? Transmitting text could be solved differently
> as well.

Well, you can only convince old-timers like Junio and Peff incrementally, by showing them something that makes their life easier, and that they do not *have* to use.

Additionally, keep in mind that the single thing *all* potential contributors have in common is access to email.

So yes, I think that any improvement would have to happen incrementally, opt-in. Meaning: on top of the current process.

> With git push/fetch we can interact with a git remote and pertain the
> state (commits, ancestor graph) at a full level even including notes
> that comment on commits.
Including much more, in fact: *any* kind of data.

But how to build on top of the current process, where some reviewers jump in via NNTP, for crying out loud? How to ensure the integrity between what is flying around as mails and what is present in the Git repository?

> git send-email/format-patch recently learned to include a base commit

You may have noticed that my mail-patch-series.sh-generated code submissions contain that base commit. But they still do not contain the SHA-1s of my local commits corresponding to the patches, and even if they did, the replies with suggested edits would most likely have lost said information.

I also hate to break it to you that git-send-email is not going to be part of any solution.

Show 14 quoted lines
> You wrote:
> > In short, I agree that our patch submission process is a saber tooth
> > tiger that still reflects pre-Git times. While we use Git's tools, the
> > workflow really tries to cut out Git as much as possible, in favor of
> > pure mails with non-corrupted, non-HTML patches in them, a charmingly
> > anachronistic requirement until you try to use state-of-the-art mail
> > clients to send them.
> 
> And there are two ways out:
> * either we teach git how to deal with emails (completely, i.e.
> sending+receiving)
> * or we change the development model (e.g. no emails any more)
> 
> There is no golden third way IMHO.
There are plenty more options.

In Git for Windows, I would accept patches via mail (curiously, nobody tried that in the past 12 months, not that I recall). I accept Pull Requests. I try to use patches mentioned in issue comments (!) and apply them.

The point is: you do not *have* to limit yourself to accepting patches *only* in one way.

Another option would be to come up with a non-opinionated tool that helps with submitting *and accepting* patches via mail. Non-opinionated, as in: it does not expect to write an entire raw mail and have that entire raw mail transmitted intact. It could, for example, generate human-consumable plain text and *also* an attachment that the tool understands. This tool could then even show a GUI to help with inspecting the relevant code/patches.

Yet another option would be to have a tool that integrates with the Git repository of the Git mailing list represented by public-inbox.

Plenty more options.

Ciao, Dscho

Stefan Beller· Aug 5, 2016, 17:59 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Johannes,

On Fri, Aug 5, 2016 at 1:20 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 46 quoted lines
> Hi Stefan,
>
> On Thu, 4 Aug 2016, Stefan Beller wrote:
>
>> On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> >
>> >> If we were to change our workflows drastically, I'd propose to go a
>> >> way[1] similar to notedb in Gerrit, or git-series,
>> >
>> > Gerrit is a huge, non-distributed system. Complex, too. If we change
>> > the patch submission process, we should make things *easier*, not
>> > *harder*. So I think Gerrit is pretty much out of the question.
>>
>> I did not propose to use Gerrit or git-series or git appraise.
>>
>> However whenever I see these projects talking to each other, the talk
>> focuses on a "unified review storage format" pretty often, which would
>> make them compatible with each other.
>
> Unless you have a splendid idea how to integrate that unified format with
> our review process on the mailing list, I think this makes for a fine
> discussion elsewhere. I'd really like to focus on the Git project and its
> patch contribution process in this here thread.
>
>> So then I could choose git-series, while you could go with git appraise
>> (although that is written in go, so maybe too fancy already ;)
>
> I think you misunderstood me in a big way.
>
> New languages are awesome. I play with new toys whenever I find the time
> (and streamlining the contribution process would give me more time for
> that). You are talking to a person who implemented the Householder
> transformation in Postscript, a 6502 assembler in Forth, and a music
> composing system in Emacs Lisp. No language is to fancy for me. "me", as
> in "me, personally".
>
> Now let's think about Git for a moment, and its language choices, and the
> rationale behind them. The majority of the critically important code is
> written in C. Is C a good language? Decent, yes, but of course also
> limiting, Resource leaks are very easy to overlook, and we have a share of
> them. No object orientation, so when we need to "subclass", we have no
> compile time safety. The pre-processor constructs make static analysis
> nigh impossible. Plenty of downsides. So why was it chosen? Developers are
> *familiar* with it, that's why. Similar considerations apply to the use of
> shell scripts, and to Perl, to a certain extent.

Which is changing slowly over time IMHO. The "new generation" of developers may not have in-depth knowledge of shelll or perl any more, but rather python or java maybe.

Show 12 quoted lines
>
> I am not talking about contrib/ of course. That's fair game, it contains
> only non-critical/fringe stuff.
>
> Note that the same rationale goes for choosing to accept patch submissions
> via mail to a list that is not subscribers-only.
>
> When it comes to inviting developers to contribute to your project,
> personal preferences become irrelevant, the deciding factor becomes how
> easy it is to join. Is the language popular, many developers already
> familiar with it? Is the build system readily available? Are the
> maintainers responsive?
I agree. I really do.

I had some discussion at lunch yesterday about different attitudes of open source projects towards new contributions. An example was the Eclipse projects that scare off potential new contributors (specially these fly by single patch submissions) as you need to interact through their instance of Gerrit after signing a CLA.

Git on the other hand doesn't do a bad job, e.g. I started with a patch that ought to be a single shot drive by patch. About 780 others wrote one patch and were never seen again:

    $ git shortlog -sne |grep 1 |wc -l
    788

So our process is not too bad for the time. However email is as a communication tool is dying [1] eventually?

Let's examine if we get less one time contributions over time:
    for i in $(seq 11 -1 1) ; do
        x_1=$(git shortlog -sne --since $i.years --until
$(($i-1)).years |grep 1 |wc -l)
        printf "$i\t$x_1\n"
    done

11 81 10 94 9 175 8 148 7 139 6 118 5 108 4 149 3 109 2 106 1 99

Looking at these numbers the number of one time patch contributions spiked 9 years ago and declined since then (4 years ago seems to be an outlier)

And I do not think the decline of one off contributions is because Git as a project is in decline, but has other reasons. Maybe the project is in better shape now that there are less one-off worthy contributions? Or the contribution process is not appealing to a lot of new comers?

[1] I could not find a scientific paper that evaluates communcation habits of people, but these may give a feel for it: https://www.quora.com/Why-are-young-people-abandoning-email https://news.ycombinator.com/item?id=3554466 http://www.twistimage.com/blog/archives/nobody-uses-email-anymore/ http://www.emailisnotdead.com/

Show 16 quoted lines
>
> I vividly remember my reaction to Darcs, for example. It's written in
> Haskell. I am a mathematician originally, so Haskell appeals to me. Did
> the choice of language appear to be designed to keep contributors out? To
> me, it looked that way.
>
> Other example: submitGit. I really like what its intention. For a while, I
> even hoped to move my *own* patch submissions to submitGit. I planned to
> help get the kinks out of the code by contributing to it. It is written in
> Scala, using a web application and testing framework I have not
> encountered elsewhere. I struggled with installing it locally and wrapping
> my head around the coding paradigms, for 1.5 days (which was all I could
> afford at that time). Then I had to give up. Which made me very sad. I
> would not have written my mail-path-series.sh tool if submitGit had been
> written in node.js, for example, with which I am familiar enough to jump
> right in.

Why did you desire to set it up yourself and not just use it? (I'd guess one of the underlying reasons could be sticking to FOSS principles, i.e. you want to be in control of what you use; But there I would argue to just approach it pragmatically. When it ceases to exist, there is time to look for an alternative.)

Show 8 quoted lines
>
> So I hope you understand better now why I find Rust a poor choice for
> something like git-series, because it should not waste contributors' time
> by insisting that their price of entry is learning a new language they are
> unfamiliar with, using a new packaging system, installing a new build
> setup. I would find Clojure, Crystal or Swift just as poor a choice. Even
> node.js. It is just too much of a "Keep Out" sign for busy developers. And
> all the developers worth their salt are busy.

Ideally you would not need to touch the internals of said tool, hence no need to learn Rust? Of course no tool is perfect, so eventually you'd want to look at the internals.

You can take the same argument for our current contribution process. Is it easier to learn a new language or setup a proper mail client that doesn't screw up your emails? Depending on the language, it is really easy to get started and have some results out in a day or two.

Show 16 quoted lines
>
>> > Additionally, I would like to point out that defining a way to store
>> > reviews in Git is not necessarily improving the way our code
>> > contribution process works. If you want to record the discussions
>> > revolving around the code, I think public-inbox already does a pretty
>> > good job at that.
>>
>> Yeah recording is great, but we want to talk about replying and
>> modifying a series? So if I see a patch flying by on the mailing list,
>> ideally I could attach a "!fixup, signed off by Stefan" thing to that
>> patch. (I said "thing" as I do not necessarily mean email here.
>
> Right. I briefly considered suggesting a new tool that would operate on
> attachments, integrating tightly with the local git.git checkout. Briefly.
> I had to reject this idea because I do not think that requiring new tools
> just to contribute to Git would fly well.

Well it would be part of Git? I consider Git a toolbox that has all kinds of useful tools to deal with code, i.e. transporting, diffing, storing. And transporting is a broad category: we have push/pull with various protocols, bundles for sneaker net and send-email for "pushing over email".

So we can have another transport tool that helps our own workflow?
Show 14 quoted lines
>
>> > I guess I have no really good idea yet, either, how to retain the ease
>> > of access of sending mails to the list, yet somehow keep a strong tie
>> > with the original data stored in Git.
>>
>> Does it have to be email? Transmitting text could be solved differently
>> as well.
>
> Well, you can only convince old-timers like Junio and Peff incrementally,
> by showing them something that makes their life easier, and that they do
> not *have* to use.
>
> Additionally, keep in mind that the single thing *all* potential
> contributors have in common is access to email.

As I said above, this may not be true any more in the far future. However I guess email is still a good base layer.

>
> So yes, I think that any improvement would have to happen incrementally,
> opt-in. Meaning: on top of the current process.

The current process is free text on a mailing list for replies, so developing a tool on top of this "data format" is really hard.

Show 10 quoted lines
>
>> With git push/fetch we can interact with a git remote and pertain the
>> state (commits, ancestor graph) at a full level even including notes
>> that comment on commits.
>
> Including much more, in fact: *any* kind of data.
>
> But how to build on top of the current process, where some reviewers jump
> in via NNTP, for crying out loud? How to ensure the integrity between what
> is flying around as mails and what is present in the Git repository?

Currently we don't. It breaks all the time. (C.f. Junio: "This is whitespace broken,; no need to resend I'll fix it up locally")

Show 11 quoted lines
>
>> git send-email/format-patch recently learned to include a base commit
>
> You may have noticed that my mail-patch-series.sh-generated code
> submissions contain that base commit. But they still do not contain the
> SHA-1s of my local commits corresponding to the patches, and even if they
> did, the replies with suggested edits would most likely have lost said
> information.
>
> I also hate to break it to you that git-send-email is not going to be part
> of any solution.

It's written in perl, so it's not one of the core parts of Git that you mentioned above. I do use it though for my submission process.

But both send-email as well as mail-patch-series as well as git-series are all about the *sending* part. Not about the back and forth part, i.e. these don't deal with: "here is a fixup on top". And by that I mean receiving mails and applying them. git-am is there as a front-end once you obtained the mail, but from what I get, your original problem is to get up to date with the latest state, that is either in pu or a proposed fixup mail on top of your series?

Show 22 quoted lines
>
>> You wrote:
>> > In short, I agree that our patch submission process is a saber tooth
>> > tiger that still reflects pre-Git times. While we use Git's tools, the
>> > workflow really tries to cut out Git as much as possible, in favor of
>> > pure mails with non-corrupted, non-HTML patches in them, a charmingly
>> > anachronistic requirement until you try to use state-of-the-art mail
>> > clients to send them.
>>
>> And there are two ways out:
>> * either we teach git how to deal with emails (completely, i.e.
>> sending+receiving)
>> * or we change the development model (e.g. no emails any more)
>>
>> There is no golden third way IMHO.
>
> There are plenty more options.
>
> In Git for Windows, I would accept patches via mail (curiously, nobody
> tried that in the past 12 months, not that I recall). I accept Pull
> Requests. I try to use patches mentioned in issue comments (!) and apply
> them.
That is great!
>
> The point is: you do not *have* to limit yourself to accepting patches
> *only* in one way.
I am not sure who the "you" is here.
You said:
> Well, you can only convince old-timers like Junio and Peff incrementally,
> by showing them something that makes their life easier, and that they do
> not *have* to use.

They like email and it works for them. Are you suggesting Junio and Jeff should accept pull request like you do?

For me as a contributor, I'll just use email for now for exactly this reason. I have it setup and know the quirks (which I occasionally forget, so it breaks again), but it is working "good enough".

Show 8 quoted lines
>
> Another option would be to come up with a non-opinionated tool that helps
> with submitting *and accepting* patches via mail. Non-opinionated, as in:
> it does not expect to write an entire raw mail and have that entire raw
> mail transmitted intact. It could, for example, generate human-consumable
> plain text and *also* an attachment that the tool understands. This tool
> could then even show a GUI to help with inspecting the relevant
> code/patches.

So a new email client that is specially adapted to the needs of the Git workflow?

>
> Yet another option would be to have a tool that integrates with the Git
> repository of the Git mailing list represented by public-inbox.

So my first reaction to that would be: you could push you patches to that public inbox and it is translated to emails sent on your behalf.

Which is reinventing submitGit just in a more accessible language?
Show 5 quoted lines
>
> Plenty more options.
>
> Ciao,
> Dscho

Thanks for writing such a long email, I missed some of the more subtle points before indeed.

Thanks, Stefan

Josh Triplett· Aug 5, 2016, 19:21 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Mon, Sep 17, 2001 at 10:00:00AM +0000, Stefan Beller wrote:
Show 7 quoted lines
> But both send-email as well as mail-patch-series as well as git-series
> are all about the *sending* part. Not about the back and forth part, i.e.
> these don't deal with: "here is a fixup on top". And by that I mean
> receiving mails and applying them. git-am is there as a front-end
> once you obtained the mail, but from what I get, your original problem
> is to get up to date with the latest state, that is either in pu or a proposed
> fixup mail on top of your series?

git-series, at least, is intended to handle the back-and-forth: if you actually publish the series and not just the final result, someone could pull the series, make a (non-fast-forwarding) change to that, make a new series commit, and publish their modified version of your series. You could then incorporate that change. One of the use cases I developed it for was collaborative development of a patch series.

(That workflow still needs a lot more tool assistance to become fully usable, not least of which to assist with the process of merging changes to the series. Working on that.)

Eric Wong· Aug 5, 2016, 21:01 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Stefan Beller <sbeller@google.com> wrote:
Show 9 quoted lines
> On Fri, Aug 5, 2016 at 1:20 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > Yet another option would be to have a tool that integrates with the Git
> > repository of the Git mailing list represented by public-inbox.
> 
> So my first reaction to that would be: you could push you patches to
> that public inbox and it is translated to emails sent on your behalf.
> 
> Which is reinventing submitGit just in a more accessible language?

Maybe vger.kernel.org could open its submission (587) port like Debian does. Unfortunately, this hurts the ability to Cc: folks directly and makes vger even more of an SPOF (I guess submitGit also has this problem)

Fwiw, I have a public-inbox for my miscellaneous patch barf at spew@80x24.org - https://80x24.org/spew/ http://ou63pmih66umazou.onion/spew/

I will throw up my untested/half-tested patches for various projects so I can still access it across various NATs and firewalls.

Using Tor for send-email often works (depending on which blacklists the exit node is on), but is totally optional (you'd expose your IP).

==> ~/bin/spew <==
#!/bin/sh
exec torsocks git send-email \
	--smtp-domain=80x24.org \
	--smtp-debug=1 \
	--smtp-server-port=submission \
	--smtp-server=80x24.org \
	--to ${DEST-spew@80x24.org} \
	--suppress-cc=all \
	"$@"
All: Feel free to use it for any Free Software projects, too;
or better yet, host your own :)
Johannes Schindelin· Aug 6, 2016, 08:58 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Stefan,

just quickly (i.e. addressing only one point, will try to address more at a later date) because I want to be mostly offline this weekend:

On Fri, 5 Aug 2016, Stefan Beller wrote:
Show 8 quoted lines
> On Fri, Aug 5, 2016 at 1:20 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > I also hate to break it to you that git-send-email is not going to be
> > part of any solution.
> 
> It's written in perl, so it's not one of the core parts of Git that you
> mentioned above. I do use it though for my submission process.

The problem is not Perl, but how fiddly it is to set up. And that you lose all the niceties of an email client (e.g. when you want to add a comment before the diff stat that is not intended to become a part of the commit message).

But I had an apostrophe last night. I might have been a bit overzealous to claim that git-send-email won't be a part of the solution. It cannot be a *user-visible* part of any solution, that still holds true.

So here is the apostrophe: why not implement a bot that listens to the PRs on GitHub, and accepts commands such as "@<whatever>bot please send this upstream" via comments on the PR. It would then send the patches to the list, from its own email address, on behalf of the contributor.

Lots of things to kink out, such as: does it need to be moderated? Record what was submitted in its own git.git fork? Accept replies and attach them to the correct PR? Maybe annotate those replies with the commits whose patches were discussed? Maybe send out replies on the PR as emails? Maybe try to auto-apply suggested patches? Cc: people who commented on earlier iterations of the patch series? Maybe make interaction smarter using an AI bot framework?

If only I had lots of time on my hand, I'd start by prototyping a node.js server and hooking it up via webhooks, then show it off so others can tinker with it.

This is not completely unlike submitGit, which was a good first attempt, but I never used it because I needed it to do so much more than it already did, *and* it complicated everything by requiring users to register with an extra step to allow submitGit to send email on their behalf. It also made contributing to it harder by choosing some non-standard web app framework. Also, I really do not like having to go to a different website just to send a GitHub PR to the list.

Anyway, that was my brain fart for the day.

Ciao, Dscho

Junio C Hamano· Aug 6, 2016, 18:33 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> The problem is not Perl, but how fiddly it is to set up. And that you lose
> all the niceties of an email client (e.g. when you want to add a comment
> before the diff stat that is not intended to become a part of the commit
> message).

Just this part. I do not think that is fair to send-email. You are blaming its "feature" that allows it to drive format-patch, which I do not consider is the proper part of the command and to which I kept saying from the early days of its introduction that I'd never use it and I think we should discourage its use exactly because it encourages a bad workflow (i.e. you skip the final proof-reading before sending out, and you cannot add footnote comments).

Treat it like an MSA just like Thunderbird, just designed to be more suited to send out patches without corruption, and you will be OK. You work, commit and write your message with your favourite editor, do format-patch, reword or add footnote with your favourite editor, and then send it out. You can avoid letting other MSAs that may corrupt whitespaces touch what you will send out if you used send-email, but that is not mandatory. As long as your favourite MSA does not corrupt your message, you can use it.

Somebody mentioned "configuring it is hard--why does the user have to know SMTP subtleties", and that may be a valid complaint against the primary part of send-email. The solution for that is not to discard it with bathwater, but make it just as easy as other MSAs, say, Thunderbird, to configure for an average user who can configure these other MUAs.

Eric Wong· Aug 6, 2016, 21:43 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Junio C Hamano <gitster@pobox.com> wrote:
Show 6 quoted lines
> Somebody mentioned "configuring it is hard--why does the user have
> to know SMTP subtleties", and that may be a valid complaint against
> the primary part of send-email.  The solution for that is not to
> discard it with bathwater, but make it just as easy as other MSAs,
> say, Thunderbird, to configure for an average user who can configure
> these other MUAs.

Sadly, the average user does not use an MUA, SMTP or IMAP, anymore. It's all webmail or apps using proprietary protocols. Embrace, extend, extinguish :<

Johannes Schindelin· Aug 7, 2016, 08:46 UTC · re: Eric Wong · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi,
On Sat, 6 Aug 2016, Eric Wong wrote:
Show 11 quoted lines
> Junio C Hamano <gitster@pobox.com> wrote:
> > Somebody mentioned "configuring it is hard--why does the user have
> > to know SMTP subtleties", and that may be a valid complaint against
> > the primary part of send-email.  The solution for that is not to
> > discard it with bathwater, but make it just as easy as other MSAs,
> > say, Thunderbird, to configure for an average user who can configure
> > these other MUAs.
> 
> Sadly, the average user does not use an MUA, SMTP or IMAP, anymore.
> It's all webmail or apps using proprietary protocols.
> Embrace, extend, extinguish :<

I think you both got it wrong. The original citizens were the mail clients that allowed you to communicate with other human beings. Webmail is just a new generation of the same commodity. It is our usage to transport machine-readable content (and not as an attachment!) that is the intruder.

It's not making things better if we require users to use a second mail client for sending out patches, and, oh, it does nothing to help with reintegrating patches back into Git, were they had been before taking that perilous and lossy journey through that medium called email.

Ciao, Dscho

Junio C Hamano· Aug 8, 2016, 17:22 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> I think you both got it wrong. The original citizens were the mail
> clients that allowed you to communicate with other human beings.
> ... It is our usage to transport machine-readable content (and not
> as an attachment!) that is the intruder.
It is not "its is our usage".

You are too young to remember or too old to remember the history, or you are knowingly distorting it. The original users of "patch" and "diff" expected that e-mail to be a medium to safely exchange changes to programs among themselves.

Johannes Schindelin· Aug 9, 2016, 11:48 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Mon, 8 Aug 2016, Junio C Hamano wrote:
Show 13 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > I think you both got it wrong. The original citizens were the mail
> > clients that allowed you to communicate with other human beings.
> > ... It is our usage to transport machine-readable content (and not
> > as an attachment!) that is the intruder.
> 
> It is not "its is our usage".
> 
> You are too young to remember or too old to remember the history, or
> you are knowingly distorting it.  The original users of "patch" and
> "diff" expected that e-mail to be a medium to safely exchange
> changes to programs among themselves.

If you are saying that transporting patches via email was the original purpose of email, then it is not exactly I who is misremembering history.

But that is not what you meant, I believe. You probably wanted to point out that the Git developers are not the first ones to abuse the medium known as email that way. And you are correct, of course. And I never claimed anything else. I just said that the problem is our usage of emails as a means to transport byte-exact content intended primarily to be consumed by a program instead of a human. It does not matter whether others did that before us. It is the problem we face right now, that is the important part of my message.

And even if it seems as if you are eagerly defending this system, I do not believe even a microsecond that you think it is a good system. I believe that you, too, would welcome a better review/contribution system that is easier to use, more welcoming to new users, less error-prone and less time-wasting than the current, email-based one, just like you jumped on Git as a better SCM when it came around, from whatever inadequate system you came from.

Ciao, Dscho

Lars Schneider· Aug 7, 2016, 11:09 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Show 53 quoted lines
> On 06 Aug 2016, at 10:58, Johannes Schindelin <johannes.schindelin@gmx.de> wrote:
> 
> Hi Stefan,
> 
> just quickly (i.e. addressing only one point, will try to address more at
> a later date) because I want to be mostly offline this weekend:
> 
> On Fri, 5 Aug 2016, Stefan Beller wrote:
> 
>> On Fri, Aug 5, 2016 at 1:20 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>> 
>>> I also hate to break it to you that git-send-email is not going to be
>>> part of any solution.
>> 
>> It's written in perl, so it's not one of the core parts of Git that you
>> mentioned above. I do use it though for my submission process.
> 
> The problem is not Perl, but how fiddly it is to set up. And that you lose
> all the niceties of an email client (e.g. when you want to add a comment
> before the diff stat that is not intended to become a part of the commit
> message).
> 
> But I had an apostrophe last night. I might have been a bit overzealous to
> claim that git-send-email won't be a part of the solution. It cannot be
> a *user-visible* part of any solution, that still holds true.
> 
> So here is the apostrophe: why not implement a bot that listens to the PRs
> on GitHub, and accepts commands such as "@<whatever>bot please send this
> upstream" via comments on the PR. It would then send the patches to the
> list, from its own email address, on behalf of the contributor.
> 
> Lots of things to kink out, such as: does it need to be moderated? Record
> what was submitted in its own git.git fork? Accept replies and attach them
> to the correct PR? Maybe annotate those replies with the commits whose
> patches were discussed? Maybe send out replies on the PR as emails? Maybe
> try to auto-apply suggested patches? Cc: people who commented on earlier
> iterations of the patch series? Maybe make interaction smarter using an AI
> bot framework?
> 
> If only I had lots of time on my hand, I'd start by prototyping a node.js
> server and hooking it up via webhooks, then show it off so others can
> tinker with it.
> 
> This is not completely unlike submitGit, which was a good first attempt,
> but I never used it because I needed it to do so much more than it already
> did, *and* it complicated everything by requiring users to register with
> an extra step to allow submitGit to send email on their behalf. It also
> made contributing to it harder by choosing some non-standard web app
> framework. Also, I really do not like having to go to a different website
> just to send a GitHub PR to the list.
> 
> Anyway, that was my brain fart for the day.

Great discussion! I would like to share my perspective a someone who is a (relatively speaking) new Git contributor and who has never interacted on mailing lists before Git:

1.) "git format-patch" and "git send-email" work great for me. It took some
    time to learn how they work but now I have my own "submit.sh" based
    on those tools and posting a new series is a piece of cake.
2.) Initially it was hard for me to ensure that my patches don't break build or 
    tests on Linux and OS X. Travis CI helps me a lot. I just wished we could
    get Windows support, too.
3.) I noticed that I get two types of reviews. The first kind points out things
    in my patch that are obviously wrong. The second kind are things that require
    a discussion. When I get feedback of the first kind, then I am always super
    eager to send out a new roll just because I don't want any other reviewer
    to waste time on obviously wrong patches. However, I have the impression
    that frequent re-rolls are frowned upon. If we would use Git for the patches
    instead of email, then I could add "squash" patches to indicate changes in
    the current roll that will be squashed in the next roll (I know I could
    send squash patches as email, too... but for me that gets confusing quickly).
4.) Reviewing patches is super hard for me because my email client does not
    support patch color highlighting and I can't easily expand context or look at
    the history of code touched by the patch (e.g via git blame). I tried to setup
    Alpine but I wasn't happy with the interface either. I like patches with a GitHub
    URL for review but then I need to find the right line in the original email to
    write a comment.

Again, this is just my point of view as a "newbie" and I definitively don't expect the Git community to change their established workflows.

Cheers, Lars

Junio C Hamano· Aug 8, 2016, 17:29 UTC · re: Lars Schneider · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Lars Schneider <larsxschneider@gmail.com> writes:
>     ... then I am always super
>     eager to send out a new roll just because I don't want any other reviewer
>     to waste time on obviously wrong patches. However, I have the impression
>     that frequent re-rolls are frowned upon.

Correct. A good middle-ground is to just reply with "Yes, thanks for your suggestion, will fix in the next round", while receiving review comments. Good reviewers who value their time will not to waste their time by responding on a point that has already been pointed out and acknowledged.

Show 6 quoted lines
> 4.) Reviewing patches is super hard for me because my email client does not
>     support patch color highlighting and I can't easily expand context or look at
>     the history of code touched by the patch (e.g via git blame). I tried to setup
>     Alpine but I wasn't happy with the interface either. I like patches with a GitHub
>     URL for review but then I need to find the right line in the original email to
>     write a comment.

Unless a patch is about an area you are super familiar with so that you know what is beyond the context of the patch to be able to judge if the change is good in the context of the file being touched, it is always hard to review from inside a mail reader.

Running "git am" is a good first step to review such a patch, as that lets you view the resulting code with the full power of Git. As you gain experience on the codebase, you'll be able to spot more problems while in your mail reader.

Johannes Schindelin· Aug 9, 2016, 11:41 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Junio,
On Mon, 8 Aug 2016, Junio C Hamano wrote:
Show 19 quoted lines
> Lars Schneider <larsxschneider@gmail.com> writes:
> 
> > 4.) Reviewing patches is super hard for me because my email client
> > does not support patch color highlighting and I can't easily expand
> > context or look at the history of code touched by the patch (e.g via
> > git blame). I tried to setup Alpine but I wasn't happy with the
> > interface either. I like patches with a GitHub URL for review but then
> > I need to find the right line in the original email to write a
> > comment.
> 
> Unless a patch is about an area you are super familiar with so that you
> know what is beyond the context of the patch to be able to judge if the
> change is good in the context of the file being touched, it is always
> hard to review from inside a mail reader.
> 
> Running "git am" is a good first step to review such a patch, as that
> lets you view the resulting code with the full power of Git.  As you
> gain experience on the codebase, you'll be able to spot more problems
> while in your mail reader.

I am glad that you agree that the requirement to manually transform the patches back into Git (where they had been originally to begin with) is cumbersome. This is the first time that I see you admit it ;-)

Ciao, Dscho

Junio C Hamano· Aug 9, 2016, 17:25 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 15 quoted lines
> On Mon, 8 Aug 2016, Junio C Hamano wrote:
>
>> Unless a patch is about an area you are super familiar with so that you
>> know what is beyond the context of the patch to be able to judge if the
>> change is good in the context of the file being touched, it is always
>> hard to review from inside a mail reader.
>> 
>> Running "git am" is a good first step to review such a patch, as that
>> lets you view the resulting code with the full power of Git.  As you
>> gain experience on the codebase, you'll be able to spot more problems
>> while in your mail reader.
>
> I am glad that you agree that the requirement to manually transform the
> patches back into Git (where they had been originally to begin with) is
> cumbersome. This is the first time that I see you admit it ;-)

I was about to apologize for writing a statement that can be misread, but I do not think what I wrote can be misinterpreted, even if a reader deliberately tries to twist the words s/he reads, to lead to such a conclusion, so I won't.

I merely said that reviewing a change in an unfamiliar area is harder (not "cumbersome", but "needs understanding first") with a patch, and it is easier to see changes in context by applying (which is an easy, not "cumbersome", process).

Eric Wong· Aug 5, 2016, 18:46 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 11 quoted lines
> On Thu, 4 Aug 2016, Stefan Beller wrote:
> > git send-email/format-patch recently learned to include a base commit
> 
> You may have noticed that my mail-patch-series.sh-generated code
> submissions contain that base commit. But they still do not contain the
> SHA-1s of my local commits corresponding to the patches, and even if they
> did, the replies with suggested edits would most likely have lost said
> information.
> 
> I also hate to break it to you that git-send-email is not going to be part
> of any solution.
I think it ought to be.  Some reasons I like emailing patches are:
* there's no taking it back once it's sent
* it's backed up within seconds by thousands of subscribers :)
* doesn't require the reader to have an active connection
  to fetch out-of-band
* doesn't require the reader to be on the same machine capable
  of cloning/building the project

There are times when I've been on a slow machine, or offline when I wanted to read some patches.

However, I do like including a pull request in cover letters of a patch series (not necessary for one-offs).

But on a side note, I also find it depressing that SMTP is uncompressed and TLS compression is (still?) unsafe. At least I use ssh tunnels w/ compression for IMAP/SMTP to my own server.

Johannes Schindelin· Aug 6, 2016, 08:44 UTC · re: Eric Wong · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Eric,
On Fri, 5 Aug 2016, Eric Wong wrote:
Show 16 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > On Thu, 4 Aug 2016, Stefan Beller wrote:
> > > git send-email/format-patch recently learned to include a base commit
> > 
> > You may have noticed that my mail-patch-series.sh-generated code
> > submissions contain that base commit. But they still do not contain the
> > SHA-1s of my local commits corresponding to the patches, and even if they
> > did, the replies with suggested edits would most likely have lost said
> > information.
> > 
> > I also hate to break it to you that git-send-email is not going to be part
> > of any solution.
> 
> I think it ought to be.  Some reasons I like emailing patches are:
> 
> [...]

What I said is that *git-send-email* is not going to be part of any solution.

Note that I said *git-send-email*, not "emailing patches".

What many people on this list forget is that few email users *ever* touch their email configuration. Asking them to figure out their SMTP settings and then to make git-send-email work is, uhm, quite a bit unrealistic.

Ciao, Dscho

Richard Ipsum· Aug 5, 2016, 11:59 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Thu, Aug 04, 2016 at 09:42:18AM -0700, Stefan Beller wrote:
Show 18 quoted lines
> On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> >> If we were to change our workflows drastically, I'd propose to
> >> go a way[1] similar to notedb in Gerrit, or git-series,
> >
> > Gerrit is a huge, non-distributed system. Complex, too. If we change the
> > patch submission process, we should make things *easier*, not *harder*. So
> > I think Gerrit is pretty much out of the question.
> 
> I did not propose to use Gerrit or git-series or git appraise.
> 
> However whenever I see these projects talking to each other, the talk focuses on
> a "unified review storage format" pretty often, which would make them compatible
> with each other. So then I could choose git-series, while you could go with
> git appraise (although that is written in go, so maybe too fancy already ;)
> Or there could be another new program written in C that follows the "review
> format".

This "unified review storage format" really does seem to be the missing piece. The tool I've been working on for the past year (git-candidate) was initially aimed at contrib[1], and was written in perl solely to satisfy contrib rules. It would have been python otherwise.

The feedback from that thread[1], was that while git-candidate itself seemed interesting it would be unreasonable to bless a particular tool's format. So it seems to me that even if git-series had been written in perl rather than rust it could have expected a similar response to that of git-candidate, possibly.

As Stefan says, if we're able to establish a standard for storing review data in git then it doesn't really matter what the tools are written in.

For what it's worth my possibly quite shoddy attempt at a library implementing a possible review format for git[2] is written in perl, mostly to satisfy contrib requirements.

Show 8 quoted lines
> >
> > Even requiring every contributor to register with GitHub would be too much
> > of a limitation, I would wager.
> >
> > And when I said I have zero interest in tools that use the "latest and
> > greatest language", I was hinting at git-series. Rust may be a fine and
> > wonderful language. Implementing git-series in Rust, however, immediately
> > limited the potential engagement with developers dramatically.

Ironically contrib's language requirements actually raised the bar for me because it meant that I had to learn perl.

Show 5 quoted lines
> >
> > Additionally, I would like to point out that defining a way to store
> > reviews in Git is not necessarily improving the way our code contribution
> > process works. If you want to record the discussions revolving around the
> > code, I think public-inbox already does a pretty good job at that.

I agree, and must apologise if this response has been too off topic, in any case I hope at least some of it was useful to someone.

Hope this helps, Richard Ipsum

[1]: http://www.mail-archive.com/git%40vger.kernel.org/msg80972.html [2]: https://bitbucket.org/richardipsum/perl-notedb

Johannes Schindelin· Aug 5, 2016, 15:24 UTC · re: Richard Ipsum · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Richard,
On Fri, 5 Aug 2016, Richard Ipsum wrote:
Show 22 quoted lines
> On Thu, Aug 04, 2016 at 09:42:18AM -0700, Stefan Beller wrote:
> > On Thu, Aug 4, 2016 at 8:58 AM, Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > >> If we were to change our workflows drastically, I'd propose to go a
> > >> way[1] similar to notedb in Gerrit, or git-series,
> > >
> > > Gerrit is a huge, non-distributed system. Complex, too. If we change
> > > the patch submission process, we should make things *easier*, not
> > > *harder*. So I think Gerrit is pretty much out of the question.
> > 
> > I did not propose to use Gerrit or git-series or git appraise.
> > 
> > However whenever I see these projects talking to each other, the talk
> > focuses on a "unified review storage format" pretty often, which would
> > make them compatible with each other. So then I could choose
> > git-series, while you could go with git appraise (although that is
> > written in go, so maybe too fancy already ;) Or there could be another
> > new program written in C that follows the "review format".
> 
> This "unified review storage format" really does seem to be the missing
> piece.

FWIW I do not think so. The real trick will be to come up with an improvement to the process that lets Junio and Peff continue to work as before, because It Works For Them, while at the same time letting other people (such as myself) use easy-to-configure tools that add substantial convenience.

Which, to me, means that the missing piece is a clever idea how to integrate with the mail-based process, without requiring everybody and her dog to switch to a specific mail client.

> The tool I've been working on for the past year (git-candidate) was
> initially aimed at contrib[1], and was written in perl solely to satisfy
> contrib rules. It would have been python otherwise.
Oh...?

$ git ls-files contrib/\*.py | wc -l 4

And for that matter:

$ git ls-files contrib/\*.go | wc -l 4

In fact, there are even PHP scripts:
$ git ls-files contrib | sed -n 's/.*\.//p' | sort | grep -v '.....' |
	uniq | tr '\n' ' '
bash c el Git go perl php pl pm py rst sh tcsh txt zsh

But again, I do not think that it makes sense to focus too much on a language, or on a file format, before we came up with a strategy how to *not* require everybody to change their current ways.

Ciao, Dscho

Richard Ipsum· Aug 6, 2016, 16:45 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Fri, Aug 05, 2016 at 05:24:14PM +0200, Johannes Schindelin wrote: [snip]

Show 13 quoted lines
> > 
> > This "unified review storage format" really does seem to be the missing
> > piece.
> 
> FWIW I do not think so. The real trick will be to come up with an
> improvement to the process that lets Junio and Peff continue to work as
> before, because It Works For Them, while at the same time letting other
> people (such as myself) use easy-to-configure tools that add substantial
> convenience.
> 
> Which, to me, means that the missing piece is a clever idea how to
> integrate with the mail-based process, without requiring everybody and her
> dog to switch to a specific mail client.

Fair enough, yes it seems to me that git's own review process is probably a separate discussion.

As far as review tools such as git-appraise, git-series and git-candidate are concerned, the review storage format really is the missing piece though, in my opinion, at least if we want to live in a world with compatible review tooling.

Show 14 quoted lines
> 
> > The tool I've been working on for the past year (git-candidate) was
> > initially aimed at contrib[1], and was written in perl solely to satisfy
> > contrib rules. It would have been python otherwise.
> 
> Oh...?
> 
> $ git ls-files contrib/\*.py | wc -l
> 4
> 
> And for that matter:
> 
> $ git ls-files contrib/\*.go | wc -l
> 4

I read this guide[1] before I started, and wanted to be on the safe side. Maybe that was a mistake... :/

Show 10 quoted lines
> 
> In fact, there are even PHP scripts:
> 
> $ git ls-files contrib | sed -n 's/.*\.//p' | sort | grep -v '.....' |
> 	uniq | tr '\n' ' '
> bash c el Git go perl php pl pm py rst sh tcsh txt zsh
> 
> But again, I do not think that it makes sense to focus too much on a
> language, or on a file format, before we came up with a strategy how to
> *not* require everybody to change their current ways.
Fair enough. :)

Thanks, Richard Ipsum

[1]: https://www.kernel.org/pub/software/scm/git/docs/howto/new-command.html
Michael Haggerty· Aug 8, 2016, 22:20 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On 08/04/2016 05:58 PM, Johannes Schindelin wrote:
> [...]
> Even requiring every contributor to register with GitHub would be too much
> of a limitation, I would wager.
> [...]

Is it *really* so insane to consider moving collaboration on the Git project to GitHub or some other similar platform?

* Many, MANY of the most prominent open-source projects are already
using GitHub. Many potential contributors already know how to use it and
already have accounts. Casual observers (e.g., people who only want to
clone the repo and/or read issues and PRs) don't even need an account.
* Even if you don't already have a GitHub account, it is vastly easier
to create one than to set up our current contribution workflow: figure
out the correct SMTP settings for your email provider, configure
git-send-email, test it and work out the kinks, figure out how to use
git-am (and even then, actually using git-am is a tedious chore for
people who don't use an email client that can run it automatically) [1].
We've seen how difficult our current workflow is by observing GSoC
candidates attempting to send their first patch. What we haven't seen is
the invisible GSoC candidates and other potential contributors who never
even get as far as attempting to send a patch.
* Interactions that involve code are done using Git commands directly,
via exchanging bona fide Git commits. Which means that...
* Commits have unambiguous SHA-1s, which we can use when discussing
them, linking to them, merging them, etc. It will no longer be a matter
of detective work to find out whether a discussion is about v1 or v3 of
a patch series, let alone v3 with some cleanups that were silently added
by Junio.
* Discussion of pull requests can be done either
  * via the website (super easy for beginners but powerful for
experienced users),
  * by setting up email notifications for your account and replying to
those emails, or
  * via an API.
  Such discussion is all in markdown, which supports light formatting,
hyperlinks, and @-mentions.
* GitHub creates permalink URLs for all of the important artifacts:
commits, pull requests, pull request comments, etc. These all can be
referenced easily from any discussion. This web of cross-links
accumulates over time and adds a lot of context to discussions.
* GitHub keeps spam under control.

I admit that if we move to GitHub we would be vulnerable if the company turns evil or goes bankrupt. But is that really a bigger risk than we accepted by relying on Gmane (a one-person hobbyist operation) for many of our historical permalinks, which are now broken? In any case, each of us has a mirror of the code, and there are utilities for backing up other GitHub metadata. *If* GitHub becomes evil, there will be a lot of other open-source projects in the same boat, so I am confident that the tooling for salvaging such information will quickly become excellent.

Currently we force potential Git contributors to learn an email-based workflow that is almost unique to this project, rather than steering them towards a workflow (Git plus, potentially, GitHub) that they probably already know, or if not is worth learning because the knowledge will carry across to most other open-source projects, not to mention their professional careers.

We would want to set down guidelines for how we use GitHub. For example, we might insist that each version of a patch series (v1, v2, etc.) be submitted as a fresh pull request with references to the previous version(s) (which also automatically creates forwards links from the previous versions to the new version). We might want to set up some robots to help with repetitive activities, like style review, pinging the right people, etc.

Junio, I'm very sensitive to the need not to decrease your efficiency. But isn't there a chance that this could *increase* your efficiency? Is it worth an experiment?

Is the Git project really such a unique snowflake that we need to use a workflow (and force it on our contributors) that is different than the workflows used by most other open-source projects?

Disclaimer: I work for GitHub, but in this email I'm speaking for myself.
Michael

[1] I concede that people who refuse on ideological grounds to use proprietary software will find this step insurmountable. Perhaps we could come up with a workaround for such people.

Junio C Hamano· Aug 8, 2016, 22:36 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Michael Haggerty <mhagger@alum.mit.edu> writes:
> Is it *really* so insane to consider moving collaboration on the Git
> project to GitHub or some other similar platform?

I only know how "pull requests" and "issues" discussion in GitHub Web interface _currently_ looks like, so if you have even more wonderful thing in the works, I _might_ be swayed otherwise, but I do not think it is sane to expect that the same quality and quantity of reviews as we do here can be maintained with that thing.

Michael Haggerty· Aug 8, 2016, 23:20 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On 08/09/2016 12:36 AM, Junio C Hamano wrote:
Show 8 quoted lines
> Michael Haggerty <mhagger@alum.mit.edu> writes:
> 
>> Is it *really* so insane to consider moving collaboration on the Git
>> project to GitHub or some other similar platform?
> 
> I only know how "pull requests" and "issues" discussion in GitHub
> Web interface _currently_ looks like, so if you have even more
> wonderful thing in the works, I _might_ be swayed otherwise,

If I did I couldn't say anyway, so let's assume that current GitHub is what's on the table [1].

There are a couple of recent code-review improvements that you might have missed:

* You can now get email updates about your own activity [2]. (Previously
you would only get emails about the activity of other people, which
would leave holes in the email record of the conversation.)
* There is also now better visibility of code review comments regarding
lines that are no longer part of a PR [3].
> but I
> do not think it is sane to expect that the same quality and quantity
> of reviews as we do here can be maintained with that thing.

Could you elaborate why you would expect quality and/or quantity of reviews to suffer? I'm really curious, and I'd be happy to pass your feedback along to my colleagues.

Here are some factors that I think will *improve* reviews:
* While you are reviewing patches, you can "zoom out" to see code beyond
the usual diff context. Currently a reviewer who wants more context has
to transition from reading the diff in email to applying the patch and
viewing it in another tool. Then the reviewer has to go back to email to
leave the comment.
* If you want to compile/run/edit/profile the code, you just need to
"git fetch" rather than messing around with "git am". For more involved
suggestions, it is possible to propose a PR against the original PR.
* It is easy to summon somebody else into the review conversation by
@-mentioning them. That person immediately can see the whole history of
the PR. (This is an improvement on the status quo, where a new entrant
to a conversation might have to dig through the email trash or an email
archive to see emails that they overlooked before they were added to the
CC list.)
* It is easy to subscribe/unsubscribe from particular discussions [4].
This makes it easier to follow the discussions you are interested in
without getting swamped with emails about other discussions. You can
unsubscribe from a discussion permanently, or in such a way that a new
@-mention brings you back in.
* It is easy to mention other PRs/commits/issues in a discussion, and
those mentions become clickable links (no jumping back and forth between
email client and web browser). Of course you can also link to arbitrary
URLs (e.g., mailing list archives).
* It is possible to search old issues and PRs for additional context.
(Granted, the history of the project from its ML era would have to be
searched separately.)

Given that I work for GitHub, I'm uncomfortable doing any more advocacy here. If people have concrete questions, I'd be happy to answer them, on the list or in private.

Michael

[1] In general, GitHub *does* get better over time, and we would benefit from any future improvements. [2] https://github.com/blog/2203-email-updates-about-your-own-activity [3] https://github.com/blog/2123-more-code-review-tools [4] https://github.com/blog/2183-improvements-to-notification-emails

Michael J Gruber· Aug 9, 2016, 08:11 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Michael Haggerty venit, vidit, dixit 09.08.2016 01:20:
> Given that I work for GitHub, I'm uncomfortable doing any more advocacy
> here. If people have concrete questions, I'd be happy to answer them, on
> the list or in private.

You're doing a great job differentiating between your roles as a member of the Git devel community and as a GitHub employee, so please keep the discussion here.

Maybe two more points of input for the discussion:

off-line capabilities =====================

The current workflow here seems to work best when you are subscribed to the git-ml and have your own (local, maybe selective) copy of git-ml in your (text-based) MUA (mutt, (al)pine, emacs, ...) that can jump right into git-am and such directly. I'm not sure how important the "off-line" aspect of that is for some of us, and how that could be replicated on GitHub - while PRs and such may be Git based behind the scenes there seems to be no way to clone that info and work from a local clone. (Don't know if GitLab is more open.)

My own setup ============

My usual MUA is Thunderbird because of its integration with calendars and address books. I usually read and post to mailing lists via nntp/gmane, that works best for me for several reasons (e.g. archive available when I need it).

For git-ml, I had to learn early on to answer by e-mail to git-ml rather than by nntp-reply because proper nntp-replies somehow didn't meet the expectations of the e-mail users (double copies because of the cc-policy or such, I don't remember).

I use git sendemail even for small throw-in patches because the git-ml does not allow attachments but wants patches (files) as in-line text, and Thunderbird possibly reformats text (because it's text, you know).

When I want to try out a proposed patch I have to "save message" and run git-am because patches don't come as file attachments on the git-ml (can't use "save/open attachment"+ git-apply) nor a PR (can't use git fetch nor view in browser). If it's a series, I have to do that for each invididual patch, which usually means I simply don't (or rely on Junio doing it and fetch his xy/topic branch).

And more often than not, patches from series do not appear in sequence, not threaded on top of the cover letter (in the gmane nntp version of git-ml), and it usually happens for the same people again and again, which tells me it's a git sendemail config issue and not gmane.

So really, everytime I interact with the git-ml I think about switching to mutt or such just for git-ml, even though over time I have gotten used to the number of hoops that I have to jump through if I want to interact with git-ml.

Suggestion ==========

Maybe the current gmane woes are a good reason to try out something different for a month or two, with copies to the git-ml, and with the default being to revert back to git-ml after that and discuss what we've learned. As a result we may improve our workflow here, get GitHub to improve, and maybe switch or not. Either way we could profit from that.

Michael
Jeff King· Aug 9, 2016, 10:57 UTC · re: Michael J Gruber · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 10:11:30AM +0200, Michael J Gruber wrote:
Show 13 quoted lines
> Maybe two more points of input for the discussion:
> 
> off-line capabilities
> =====================
> 
> The current workflow here seems to work best when you are subscribed to
> the git-ml and have your own (local, maybe selective) copy of git-ml in
> your (text-based) MUA (mutt, (al)pine, emacs, ...) that can jump right
> into git-am and such directly. I'm not sure how important the "off-line"
> aspect of that is for some of us, and how that could be replicated on
> GitHub - while PRs and such may be Git based behind the scenes there
> seems to be no way to clone that info and work from a local clone.
> (Don't know if GitLab is more open.)

You can pull it all out via GitHub's HTTP API, but the question is what format you would use to store it locally (and which tools you would then use to play with it).

I haven't really tried this lately, though, so I don't know if there is information that the API would be missing.

I do have a dream of writing a tool that sucks in GitHub PRs to a fake email thread, lets me make my responses inline in an editor, and then pushes it back up as PR comments (finding the right positions based on the quoted context).

> For git-ml, I had to learn early on to answer by e-mail to git-ml rather
> than by nntp-reply because proper nntp-replies somehow didn't meet the
> expectations of the e-mail users (double copies because of the cc-policy
> or such, I don't remember).

At least some people's workflows seem to send two copies to the list. For instance, Jakub's <2bfd9cf5-a9fa-7650-21e9-9ceb9cc34d8b@gmail.com> got delivered to me via the list twice. Once directly from gmail with:

  To: Oleg Taranenko <olegtaranenko@gmail.com>,
      Junio C Hamano <gitster@pobox.com>
  Cc: git@vger.kernel.org
and once via gmane with:
  To: git@vger.kernel.org
  Cc: git@vger.kernel.org

It's like this with all of his messages (sorry I can't point to the duplicates in an archive; they have the same message-id, so public-inbox treats them as a single unit).

Replying to the second one breaks the usual "cc-everybody" rule. Sending duplicates means everybody sees it twice (3 times if they're on the cc list!), and the second copy still has the bogus headers (so people replying need to pick the right one).

> I use git sendemail even for small throw-in patches because the git-ml
> does not allow attachments but wants patches (files) as in-line text,
> and Thunderbird possibly reformats text (because it's text, you know).

I wonder if this is something we could change. I do not personally have any problem with attached patches. "git am" knows how to apply them, and mutt is smart enough to show text/* by default, and to include it in quoted text on reply. So the output of "git format-patch --attach" works fine for me. But it may not be as nice in other MUAs, and we have to care about all of the other reviewers.

Show 6 quoted lines
> When I want to try out a proposed patch I have to "save message" and run
> git-am because patches don't come as file attachments on the git-ml
> (can't use "save/open attachment"+ git-apply) nor a PR (can't use git
> fetch nor view in browser). If it's a series, I have to do that for each
> invididual patch, which usually means I simply don't (or rely on Junio
> doing it and fetch his xy/topic branch).

So you would like the opposite of my dream tool, I think: something that takes mailing list conversations and turns them into PRs.

(My real dream is actually to have a bidirectional version of the tool, so that everybody can use whatever interface they like, and nobody has to care about somebody else's preferences).

> And more often than not, patches from series do not appear in sequence,
> not threaded on top of the cover letter (in the gmane nntp version of
> git-ml), and it usually happens for the same people again and again,
> which tells me it's a git sendemail config issue and not gmane.

Just a guess, but I suspect this is caused by people who use "rebase -i" to rearrange patches. When format-patch writes out the patches, it uses the author date as the "Date" field, which means it may be out of order. I think send-email will always write out a new, monotonically increasing date. But I suspect other workflows (e.g., imap-send and then mailing from a MUA) blindly re-use that date.

Show 8 quoted lines
> Suggestion
> ==========
> 
> Maybe the current gmane woes are a good reason to try out something
> different for a month or two, with copies to the git-ml, and with the
> default being to revert back to git-ml after that and discuss what we've
> learned. As a result we may improve our workflow here, get GitHub to
> improve, and maybe switch or not. Either way we could profit from that.

I think public-inbox is a nice step forward on the reading side (it's a lot easier to get raw patches out of it, for example). But it doesn't help much with sending (and sending is a tricky subject; anytime you promise to send mail on behalf of somebody, you're going to attract spammers).

-Peff
Josh Triplett· Aug 10, 2016, 00:46 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 06:57:05AM -0400, Jeff King wrote:
Show 27 quoted lines
> On Tue, Aug 09, 2016 at 10:11:30AM +0200, Michael J Gruber wrote:
> 
> > Maybe two more points of input for the discussion:
> > 
> > off-line capabilities
> > =====================
> > 
> > The current workflow here seems to work best when you are subscribed to
> > the git-ml and have your own (local, maybe selective) copy of git-ml in
> > your (text-based) MUA (mutt, (al)pine, emacs, ...) that can jump right
> > into git-am and such directly. I'm not sure how important the "off-line"
> > aspect of that is for some of us, and how that could be replicated on
> > GitHub - while PRs and such may be Git based behind the scenes there
> > seems to be no way to clone that info and work from a local clone.
> > (Don't know if GitLab is more open.)
> 
> You can pull it all out via GitHub's HTTP API, but the question is what
> format you would use to store it locally (and which tools you would then
> use to play with it).
> 
> I haven't really tried this lately, though, so I don't know if there is
> information that the API would be missing.
> 
> I do have a dream of writing a tool that sucks in GitHub PRs to a fake
> email thread, lets me make my responses inline in an editor, and then
> pushes it back up as PR comments (finding the right positions based on
> the quoted context).
You might try https://github.com/joeyh/github-backup
Jeff King· Aug 9, 2016, 11:37 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 01:20:05AM +0200, Michael Haggerty wrote:
Show 7 quoted lines
> > but I
> > do not think it is sane to expect that the same quality and quantity
> > of reviews as we do here can be maintained with that thing.
> 
> Could you elaborate why you would expect quality and/or quantity of
> reviews to suffer? I'm really curious, and I'd be happy to pass your
> feedback along to my colleagues.

Having done a lot of review here on the mailing list, as well as in GitHub PRs, I vastly prefer the mailing list.

Here's a random list of things that I think I would miss:
 - I really like the flow of having the commit message and diff dumped
   in my editor. I'm very efficient at slicing and dicing text, omitting
   uninteresting quoted bits, etc.
   Web text boxes feel like a straitjacket. I do have a browser plugin
   that opens them in vim. That helps, but it breaks the flow (I make a
   comment, save the file, click "comment", then read to the next place,
   click "+", then start a new vim instance for that comment).  Besides
   the tedium of clicking around, it loses the "unit" size of a single
   email, where I may make many comments, go back and revise earlier
   comments after reading more of the patch, etc.
 - I really like the flow of having conversations next to patches. I can
   look at the index of the mailing list folder and see what people are
   talking about, how big the threads are, etc, at a glance. Moving
   between messages and threads involve single keystrokes.
   Similarly, having local storage is _fast_. I think GitHub is fine for
   a web app. But when I'm reading a high-volume mailing list, I really
   want to flip around quickly. If there's even 500ms to get to the next
   message or thread, it feels clunky and slow to me. Obviously I spend
   more than 500ms _reading_ most messages (though for some I see the
   first paragraph and immediately jump away). It's just the latency
   when I've decided I'm done with one thing and want to move to the
   next.
 - For that matter, GitHub doesn't really have a good tool for random
   conversations. There are issues, which you can vaguely use like a
   thread, but it doesn't quite feel the same.
   I think part of it is that I can view the mailing list both as a
   series of threads _and_ as a stream of messages. So sometimes I mark
   a thread as "read", and then see the next day that there are a ton of
   new messages on it. Maybe those are uninteresting (and it's a single
   keystroke to mark the thread again), but maybe that's a hint that
   there's interesting discussion going on.
   The threading in GitHub comments and pull requests is also not great.
   Each PR or issue is its own thread, but it's totally flat inside.
   There are no sub-threads to organize discussion, and it's sometimes
   hard to see what people are replying to.
 - When I move between a discussion and patches on the list and my local
   git checkout, it's important to do so with minimal fuss. Which means
   I want to use _context_ in my workflow. If I'm reading a thread, I
   want there to be a keystroke for "jump to this thread in my
   checkout". That's (relatively) easy for me to script via mutt (grab
   these patches, apply them). It's a bit harder in the browser (the
   best I've got is to copy-paste the URL to a script that pulls out the
   PR number, then fetches and checks it out).
 - A sort-of feature: the mailing list is actually fairly decentralized,
   because of the "reply-to-all" convention. I don't know if anybody
   else noticed, but vger seemed to be down Friday evening and Saturday
   morning (at least my messages to the list got 400 SMTP codes, and no
   new messages were delivered to me). But I still had some
   conversations going with people, because our messages were mailed
   directly (and the list eventually caught up).
   Now that probably doesn't matter for GitHub, which seems to have
   fairly reasonable uptime. It would matter if we picked a centralized
   tool that didn't.
There are probably more, but I've run out of ranting steam for now. :)
> Here are some factors that I think will *improve* reviews:

I was going to respond point-by-point to a few of these, but I think I covered most of it above. In short, I agree with many of the benefits you list. In most cases, I've already reaped those benefits for my own workflow (e.g., my "git am" workflow is pretty efficient now). But not everybody has done so, and it's a lot to ask of casual contributors.

> Given that I work for GitHub, I'm uncomfortable doing any more advocacy
> here. If people have concrete questions, I'd be happy to answer them, on
> the list or in private.
Hopefully I provided some counterpoint. ;)
-Peff
Junio C Hamano· Aug 9, 2016, 17:34 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Jeff King <peff@peff.net> writes all what I wanted to say, and a lot more, so I don't have to say much ;-)

Show 13 quoted lines
>  - I really like the flow of having conversations next to patches. I can
>    look at the index of the mailing list folder and see what people are
>    talking about, how big the threads are, etc, at a glance. Moving
>    between messages and threads involve single keystrokes.
>
>    Similarly, having local storage is _fast_. I think GitHub is fine for
>    a web app. But when I'm reading a high-volume mailing list, I really
>    want to flip around quickly. If there's even 500ms to get to the next
>    message or thread, it feels clunky and slow to me. Obviously I spend
>    more than 500ms _reading_ most messages (though for some I see the
>    first paragraph and immediately jump away). It's just the latency
>    when I've decided I'm done with one thing and want to move to the
>    next.

Viewing threads in a threaded mail client to help prioritizing various topics being discussed is what I value the most and I am not sure how I can be as efficient with the pull-request page.

>    The threading in GitHub comments and pull requests is also not great.
>    Each PR or issue is its own thread, but it's totally flat inside.
>    There are no sub-threads to organize discussion, and it's sometimes
>    hard to see what people are replying to.

It may be a good UI that is optimized for drive-by contributors. It is just that it is not very well suited (compared to mailing list discussions) to conduct high-volume exchange of ideas and changes efficiently.

Jeff King· Aug 9, 2016, 17:50 UTC · re: Junio C Hamano · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 10:34:11AM -0700, Junio C Hamano wrote:
Show 9 quoted lines
> >    The threading in GitHub comments and pull requests is also not great.
> >    Each PR or issue is its own thread, but it's totally flat inside.
> >    There are no sub-threads to organize discussion, and it's sometimes
> >    hard to see what people are replying to.
> 
> It may be a good UI that is optimized for drive-by contributors.  It
> is just that it is not very well suited (compared to mailing list
> discussions) to conduct high-volume exchange of ideas and changes
> efficiently.

I think that's something to ponder; can we have a workflow where drive-by contributors can use something that has a lower learning/setup curve, but long-term contributors might opt for something more powerful?

I think SubmitGit is a step in that direction. It does still require switching to the mailing list for subsequent conversation, though. It would be interesting to see something like SubmitGit that puts its own email in the "From", and that processes email replies into PR comments, and then subsequent PR comments into emails (i.e., part of my "dream tool" from earlier). It's not clear to me whether the result would just end up being irritating for both sides to use (because it doesn't _quite_ conform to the norms of each format). But it would be fun to find out.

-Peff
Junio C Hamano· Aug 9, 2016, 19:19 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Jeff King <peff@peff.net> writes:
Show 12 quoted lines
> On Tue, Aug 09, 2016 at 10:34:11AM -0700, Junio C Hamano wrote:
>
>> It may be a good UI that is optimized for drive-by contributors.  It
>> is just that it is not very well suited (compared to mailing list
>> discussions) to conduct high-volume exchange of ideas and changes
>> efficiently.
>
> I think that's something to ponder; can we have a workflow where
> drive-by contributors can use something that has a lower learning/setup
> curve, but long-term contributors might opt for something more powerful?
>
> I think SubmitGit is a step in that direction.

Yes, agreed 100% with that. The author of the tool must be praised by getting added to the Cc: line in this discussion ;-)

Show 8 quoted lines
> It does still require
> switching to the mailing list for subsequent conversation, though. It
> would be interesting to see something like SubmitGit that puts its own
> email in the "From", and that processes email replies into PR comments,
> and then subsequent PR comments into emails (i.e., part of my "dream tool"
> from earlier). It's not clear to me whether the result would just end up
> being irritating for both sides to use (because it doesn't _quite_
> conform to the norms of each format). But it would be fun to find out.

Perhaps. I do not know if I like that second and subsequent steps for SubmitGit, but its first step as currently deployed I am very happy with.

Duy Nguyen· Aug 9, 2016, 18:43 UTC · re: Jeff King · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 9, 2016 at 1:37 PM, Jeff King <peff@peff.net> wrote:
>    That's (relatively) easy for me to script via mutt (grab
>    these patches, apply them).

Could you share your mutt set up pleaaase? I've been wanting this for a long time, but never used mutt long enough to bother with a proper setup like this (I blame gmail).

-- 
Duy
Stefan Beller· Aug 9, 2016, 18:50 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 9, 2016 at 11:43 AM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 7 quoted lines
> On Tue, Aug 9, 2016 at 1:37 PM, Jeff King <peff@peff.net> wrote:
>>    That's (relatively) easy for me to script via mutt (grab
>>    these patches, apply them).
>
> Could you share your mutt set up pleaaase? I've been wanting this for
> a long time, but never used mutt long enough to bother with a proper
> setup like this (I blame gmail).

That is my complaint^H^H^H^H position, too. I always wanted to switch to a more powerful setup than git-send-email for sending /gmail for reading, but I could not convince myself the steep learning/setup curve is worth it eventually as it is "not broken enough" to do the change right now.

My experiments with mutts, have left these lines in my ~/.muttrc

Show 5 quoted lines
> # use shift + A to apply a patch in the working dir
> # macro index A ":unset pipe_decode\n|git am -3\n:set pipe_decode\n"
> # macro pager A ":unset pipe_decode\n|git am -3\n:set pipe_decode\n"
>
> macro index A ":set folder='.'\n:copy-message\n"

(IIRC they were broken for many patches, but I got applying one patch to work. Which sucks for long email series.)

Jeff King· Aug 9, 2016, 18:58 UTC · re: Stefan Beller · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 11:50:51AM -0700, Stefan Beller wrote:
Show 11 quoted lines
> > Could you share your mutt set up pleaaase? I've been wanting this for
> > a long time, but never used mutt long enough to bother with a proper
> > setup like this (I blame gmail).
> 
> 
> That is my complaint^H^H^H^H position, too.
> I always wanted to switch to a more powerful
> setup than git-send-email for sending /gmail for reading,
> but I could not convince myself the steep learning/setup curve
> is worth it eventually as it is "not broken enough" to do the change
> right now.

I think I may have shared it before, but here is the script I use to send emails. It dumps you in mutt, and then I have:

  macro index,pager b ":set edit_headers=yes<enter><resend-message>:set edit_headers=no<enter>"

to send the message ("b" is for "bounce", which I think may be another Pine-ism).

-- >8 -- #!/bin/sh

upstream_branch() {
  current=`git symbolic-ref HEAD`
  upstream=`git for-each-ref --format='%(upstream)' "$current"`
  if test -n "$upstream"; then
    echo $upstream
  else
    echo origin
  fi
}
get_reply_headers() {
  perl -ne '
    if (defined $opt) {
      if (/^\s+(.*)/) {
        $val .= " $1";
        next;
      }
      print "--$opt=", quotemeta($val), " ";
      $opt = $val = undef;
    }
    if (/^(cc|to):\s*(.*)/i) {
      $opt = lc($1);
      $val = $2;
    }
    elsif (/^message-id:\s*(.*)/i) {
      $opt = "in-reply-to";
      $val = $1;
    }
    elsif (/^subject:\s*\[PATCH v(\d+)/i) {
      print "-v$1 ";
    }
    elsif (/^$/) {
      last;
    }
  '
}
has_nonoption=
for i in "$@"; do
  case "$i" in
    -[0-9]) has_nonoption=yes ;;
    -*) ;;
     *) has_nonoption=yes
  esac
done
git rev-parse || exit 1

: ${REPLY:=$HOME/patch} test -e "$REPLY" && eval "set -- `get_reply_headers <\"$REPLY\"` \"\$@\"" test "$has_nonoption" = "yes" || set -- "$@" `upstream_branch`

git format-patch -s --stdout --from "$@" >.mbox
if test -t 1; then
  mutt -e 'set sort=mailbox-order' -f .mbox
else
  perl -lne '
    if (/^Subject: (.*)/) {
      $subject = $1;
    }
    elsif ($subject && /^\s+(.*)/) {
      $subject .= " $1";
    }
    elsif ($subject) {
      print $subject;
      $subject = undef;
    }
  ' .mbox |
  sed -e 's/\[PATCH /[/' \
      -e 's/]/]:/' \
      -e 's/^/  /'
fi
rm -f .mbox
Jeff King· Aug 9, 2016, 18:55 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 08:43:59PM +0200, Duy Nguyen wrote:
Show 7 quoted lines
> On Tue, Aug 9, 2016 at 1:37 PM, Jeff King <peff@peff.net> wrote:
> >    That's (relatively) easy for me to script via mutt (grab
> >    these patches, apply them).
> 
> Could you share your mutt set up pleaaase? I've been wanting this for
> a long time, but never used mutt long enough to bother with a proper
> setup like this (I blame gmail).
It's actually pretty simple. The relevant config from my .muttrc is:
   macro pager,index D '<shell-escape>rm -f $HOME/patch<enter>'
   macro pager,index A '<copy-message>~/patch<enter><enter>'

I use "~/patch" as a rendezvous point, and then "git am ~/patch" from my other terminal. That avoids mutt having to know which repo to apply to, and keeps the "am" process in its own terminal (which is handy if it runs into conflicts, for example).

So generally I would "D" to clear out the contents of ~/patch, and then "A" whichever patches I want to apply. I often use mutt's aggregate selection for that. My bindings are:

  bind index \; tag-pattern
  bind index a tag-prefix

which I think come from pine (which I used for many years before switching to mutt probably 15 years ago). I don't recall the default keybindings.

Anyway, you can either tag using a pattern (with ";"), or tag mails individually (using "t", the default), and then "a-A" to apply the "A" to all of them (if you are in the habit of tagging all of them and then doing "A" in one swoop, you could also get rid of the separate "D" command and just make "A" imply it).

-Peff
Duy Nguyen· Aug 9, 2016, 18:36 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 9, 2016 at 1:20 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> Could you elaborate why you would expect quality and/or quantity of
> reviews to suffer? I'm really curious, and I'd be happy to pass your
> feedback along to my colleagues.

Since I have been using github at work for a couple months, I do have a few complaints if it will ever become the way of reviewing things in git. Some of these may be covered by other people already (I haven't read all new mails in this thread yet)

 - Github PRs seem to encourage seeing changes as a whole, not a
separate commits. Or at least it's not so convenient to view separate
commits (e.g. not easy to go to the next commit)
 - The ability to show all outdated comments, in case I want to search
through them.
 - I have a feeling that commits in PRs are sorted by authordate, not
in topological order. The order of commits being committed is
important sometimes.
 - Not showing leading spaces mixing with TABs, or trailing spaces
 - I would love to have all patches numbered so can refer to them as
1/7, 2/5... instead of just short sha1 (and I think you have the
ability to refer to "1/7 of iteration 2", see next bullet point)
 -I guess you can manage multiple iterations of a topic with one
iteration per PR, then linking them together. It would be nicer to
somehow bake the iteration concept directly to a PR so we can switch
between them, or do interdiff. I know, this is more of a improvement
request than complaint because ML is not really better.
 - Offline support would be very nice. I'm only most of the time, but
sometimes I do work on git stuff offline.
 - We lose the integration with ML, I think. Sometimes the user
reports a bug here, then we reply back with a patch. With github, I
guess we reply back with a PR number, then further discussion may go
there, some discussion may still be on ML.
Show 6 quoted lines
> * It is easy to summon somebody else into the review conversation by
> @-mentioning them. That person immediately can see the whole history of
> the PR. (This is an improvement on the status quo, where a new entrant
> to a conversation might have to dig through the email trash or an email
> archive to see emails that they overlooked before they were added to the
> CC list.)

On the other hand, we can't just CC anybody anymore because we don't know if they have a github account (or the account name for that matter). Or does github allow @-ing email addresses too? We record people's email address, not github account names.

> * It is possible to search old issues and PRs for additional context.
> (Granted, the history of the project from its ML era would have to be
> searched separately.)

To me searching in email is still better. Maybe I haven't fully explored github search capabilities

-- 
Duy
Duy Nguyen· Aug 9, 2016, 18:38 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 9, 2016 at 8:36 PM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 9 quoted lines
> On Tue, Aug 9, 2016 at 1:20 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> Could you elaborate why you would expect quality and/or quantity of
>> reviews to suffer? I'm really curious, and I'd be happy to pass your
>> feedback along to my colleagues.
>
> Since I have been using github at work for a couple months, I do have
> a few complaints if it will ever become the way of reviewing things in
> git. Some of these may be covered by other people already (I haven't
> read all new mails in this thread yet)

Another super nit thing: use monospace font for commit messages, or at least have an option for that.

-- 
Duy
Duy Nguyen· Aug 9, 2016, 04:22 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 9, 2016 at 12:20 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
Show 8 quoted lines
> On 08/04/2016 05:58 PM, Johannes Schindelin wrote:
>> [...]
>> Even requiring every contributor to register with GitHub would be too much
>> of a limitation, I would wager.
>> [...]
>
> Is it *really* so insane to consider moving collaboration on the Git
> project to GitHub or some other similar platform?

In the very unlikely event that github is shut down, how do we get all review comments out of it, assuming that we will use pull requests for review?

-- 
Duy
Richard Ipsum· Aug 9, 2016, 09:17 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 06:22:21AM +0200, Duy Nguyen wrote:
Show 13 quoted lines
> On Tue, Aug 9, 2016 at 12:20 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> > On 08/04/2016 05:58 PM, Johannes Schindelin wrote:
> >> [...]
> >> Even requiring every contributor to register with GitHub would be too much
> >> of a limitation, I would wager.
> >> [...]
> >
> > Is it *really* so insane to consider moving collaboration on the Git
> > project to GitHub or some other similar platform?
> 
> In the very unlikely event that github is shut down, how do we get all
> review comments out of it, assuming that we will use pull requests for
> review?

For what it's worth this is exactly why I think it would be worthwhile for git to establish a common review format, services like Github/Gitlab could then start storing reviews and comments in the git repo rather than in a separate sql database.

Gerrit is already doing this with notedb, which literally gives you a git log of a review. Admittedly with Gerrit the change metadata sits in a separate git repo, still, this is much better than the current situation with Github and Gitlab in my opinion.

I apologise once again if my comments here are somewhat unrelated, but I feel there is at least some overlap, since the existence of a common review format for git could potentially make Github/Gitlab a more attractive option, if Github/Gitlab chose to adopt such a format.

Really I think that reviews shouldn't be stored on mailing lists, and they shouldn't be stored in sql databases, they should be stored in git.

Jeff King· Aug 9, 2016, 10:34 UTC · re: Richard Ipsum · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 10:17:22AM +0100, Richard Ipsum wrote:
Show 8 quoted lines
> > In the very unlikely event that github is shut down, how do we get all
> > review comments out of it, assuming that we will use pull requests for
> > review?
> 
> For what it's worth this is exactly why I think it would be worthwhile for git
> to establish a common review format, services like Github/Gitlab could then
> start storing reviews and comments in the git repo rather than in a separate
> sql database.

I doubt that the "rather than" part will ever happen. Git does not make a very good database, and certainly not when you want to do things that cut across repositories (like, say, efficiently get all review comments made by one user).

It would be nice to have a common interchange format, though. In theory that could feed into (and out of) a more efficient representation on the backend of the site. It doesn't _have_ to be git-based, but it would be nice if it was.

Somebody asked elsewhere "what happens if GitHub goes away?". And the answer is that you can already get all of that data out in a programmatic way, via the API. But since there's no common interchange format, you'd be stuck writing a conversion to whatever format your new destination uses.

-Peff
Michael Haggerty· Aug 9, 2016, 10:19 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On 08/09/2016 06:22 AM, Duy Nguyen wrote:
Show 13 quoted lines
> On Tue, Aug 9, 2016 at 12:20 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> On 08/04/2016 05:58 PM, Johannes Schindelin wrote:
>>> [...]
>>> Even requiring every contributor to register with GitHub would be too much
>>> of a limitation, I would wager.
>>> [...]
>>
>> Is it *really* so insane to consider moving collaboration on the Git
>> project to GitHub or some other similar platform?
> 
> In the very unlikely event that github is shut down, how do we get all
> review comments out of it, assuming that we will use pull requests for
> review?

I don't have any experience with these tools, but a quick search turns up the following possibilities (among others):

* github-backup (by Joey Hess): https://github.com/joeyh/github-backup
* python-github-backup: https://github.com/josegonzalez/python-github-backup
* BackHub (commercial service): https://backhub.co/
* Import GitHub project into GitLab:
http://docs.gitlab.com/ce/workflow/importing/import_projects_from_github.html
Michael
Johannes Schindelin· Aug 9, 2016, 12:07 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Michael,
On Tue, 9 Aug 2016, Michael Haggerty wrote:
Show 8 quoted lines
> On 08/04/2016 05:58 PM, Johannes Schindelin wrote:
> > [...]
> > Even requiring every contributor to register with GitHub would be too
> > much of a limitation, I would wager.
> > [...]
> 
> Is it *really* so insane to consider moving collaboration on the Git
> project to GitHub or some other similar platform?

Speaking for myself, I do prefer GitHub's UI to mail, by a lot. Not only because it is more focused on the code, but because it integrates so nicely with Git, which email distinctly does not.

So I personally would not have the least bit of a problem to switch to GitHub (that's indeed what Git for Windows did, getting substantially more contributions than we would otherwise have).

And of course I use the email notifications quite a bit. They are really convenient: I get my updates via my mail program, still, and the discussion I want to participate in is just one click away.

The reason why I stated that GitHub is out of the question is that I expected resistance against it. But you are right: I should not have ruled it out so categorically, it is not at all my call to make.

Ciao, Dscho

Eric Wong· Aug 9, 2016, 18:28 UTC · re: Michael Haggerty · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Michael Haggerty <mhagger@alum.mit.edu> wrote:
Show 5 quoted lines
> On 08/04/2016 05:58 PM, Johannes Schindelin wrote:
> > [...]
> > Even requiring every contributor to register with GitHub would be too much
> > of a limitation, I would wager.
> > [...]
Show 8 quoted lines
> * Discussion of pull requests can be done either
>   * via the website (super easy for beginners but powerful for
> experienced users),
>   * by setting up email notifications for your account and replying to
> those emails, or
>   * via an API.
>   Such discussion is all in markdown, which supports light formatting,
> hyperlinks, and @-mentions.
<snip>
Show 7 quoted lines
> Disclaimer: I work for GitHub, but in this email I'm speaking for myself.
> 
> Michael
> 
> [1] I concede that people who refuse on ideological grounds to use
> proprietary software will find this step insurmountable. Perhaps we
> could come up with a workaround for such people.

I'm one of those ideological people and I don't see an acceptable workaround. GitHub already has misfeatures designed to lock people in into centralized messaging:

* pull request feature doesn't work for self-hosted repos
  (this disincentivizes people from running and improving
   git-daemon/git-http-backend/etc...)
* "noreply" email addresses
* @-mentions you wrote about
* custom email notifications

This is a problem with Gitlab, Redmine, etc, too: they cannot interoperate with each other.

At least for now, large proprietary mail providers like Gmail still interoperate with whatever Free Software SMTP software I run. I dread the day when that is no longer true.

Some of these problems I hope public-inbox (or something like it) can fix and turn the tide towards email, again. In contrast, public-inbox is designed to push decentralization:

* "reply" links are instructions for "git send-email" which
  encourage reply-to-all (this applies to what Jeff said
  about vger going down, I noticed it, too)
* anybody can clone the code + repo, replicate the
  instances, and tweak it to their needs.
* public-inbox.org/git/$MESSAGE_ID/t.atom allows subscriptions
  to Atom feeds without any registration or user-tracking
* Message-IDs are exposed for proper threading and interop
* low-bandwidth, Tor-friendly design to encourage deployments
  even behind NATs and firewalls.

Anyways, my optimistic side might interpret your advocacy as GitHub already feeling threatened by public-inbox. I certainly wouldn't expect it at this stage, but I certainly hope it will be the case one day :)

Disclaimer: I've always been willing to risk a lifetime of
unemployment for ideology.
Josh Triplett· Aug 10, 2016, 00:55 UTC · re: Eric Wong · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Tue, Aug 09, 2016 at 06:28:00PM +0000, Eric Wong wrote:
> Some of these problems I hope public-inbox (or something like
> it) can fix and turn the tide towards email, again.

This really seems like the dichotomy that drives people towards central services like GitHub or GitLab. We need an alternative that doesn't involve email, or at the very least, doesn't require people to use email directly. Half of the pain in the process comes from coaxing email clients that don't treat mail text as sacrosanct to leave it alone and not mangle it. (Some of that would go away if we accepted attachments with inline disposition, but not all of it. All of it would go away if the submission process just involved "git push" to an appropriate location.)

- Josh Triplett
Eric Wong· Aug 10, 2016, 01:57 UTC · re: Josh Triplett · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Josh Triplett <josh@joshtriplett.org> wrote:
Show 13 quoted lines
> On Tue, Aug 09, 2016 at 06:28:00PM +0000, Eric Wong wrote:
> > Some of these problems I hope public-inbox (or something like
> > it) can fix and turn the tide towards email, again.
> 
> This really seems like the dichotomy that drives people towards central
> services like GitHub or GitLab.  We need an alternative that doesn't
> involve email, or at the very least, doesn't require people to use email
> directly.  Half of the pain in the process comes from coaxing email
> clients that don't treat mail text as sacrosanct to leave it alone and
> not mangle it.  (Some of that would go away if we accepted attachments
> with inline disposition, but not all of it.  All of it would go away if
> the submission process just involved "git push" to an appropriate
> location.)

I don't mind patches as attachments and did some work a few months ago to ensure they're individually downloadable in the public-inbox WWW interface (along with full mboxrd messages)[1].

Fwiw, attachments are preferred in perl5-porters, and it might be acceptable on LKML, even. Not my call, though.

Having a push/pull-only workflow would still require some sort of messaging system to notify others. Ideally that message would have the output of "git request-pull" to ensure people are on the same page; but I'd prefer patches (either attachments or inline) continue to be sent anyways in case the server is down or the reader is offline or on a machine without git.

[1] see Brian's (who is new, here) initial email for diff-highlight:
    https://public-inbox.org/git/20160728162712.GA29220@tci.corp.yp.com/
Jakub Narębski· Aug 10, 2016, 07:30 UTC · re: Josh Triplett · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On 10 August 2016 at 02:55, Josh Triplett <josh@joshtriplett.org> wrote:
Show 13 quoted lines
> On Tue, Aug 09, 2016 at 06:28:00PM +0000, Eric Wong wrote:
>> Some of these problems I hope public-inbox (or something like
>> it) can fix and turn the tide towards email, again.
>
> This really seems like the dichotomy that drives people towards central
> services like GitHub or GitLab.  We need an alternative that doesn't
> involve email, or at the very least, doesn't require people to use email
> directly.  Half of the pain in the process comes from coaxing email
> clients that don't treat mail text as sacrosanct to leave it alone and
> not mangle it.  (Some of that would go away if we accepted attachments
> with inline disposition, but not all of it.  All of it would go away if
> the submission process just involved "git push" to an appropriate
> location.)

But submission is less important than review. And for review it is usually better (except gigantic series) to have patch text for review with the review. And threading. And (meta)-versioning of series. And place for proof-of-concept / weather-balon patches...

-- 
Jakub Narebski
Josh Triplett· Aug 10, 2016, 19:30 UTC · re: Jakub Narębski · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Wed, Aug 10, 2016 at 09:30:01AM +0200, Jakub Narębski wrote:
Show 18 quoted lines
> On 10 August 2016 at 02:55, Josh Triplett <josh@joshtriplett.org> wrote:
> > On Tue, Aug 09, 2016 at 06:28:00PM +0000, Eric Wong wrote:
> >> Some of these problems I hope public-inbox (or something like
> >> it) can fix and turn the tide towards email, again.
> >
> > This really seems like the dichotomy that drives people towards central
> > services like GitHub or GitLab.  We need an alternative that doesn't
> > involve email, or at the very least, doesn't require people to use email
> > directly.  Half of the pain in the process comes from coaxing email
> > clients that don't treat mail text as sacrosanct to leave it alone and
> > not mangle it.  (Some of that would go away if we accepted attachments
> > with inline disposition, but not all of it.  All of it would go away if
> > the submission process just involved "git push" to an appropriate
> > location.)
> 
> But submission is less important than review. And for review it is
> usually better (except gigantic series) to have patch text for review
> with the review.

Agreed. However, submission typically requires more work than review, because the patch text must remain applicable. For review, as long as the email client you use to respond doesn't do something horrible like *re-wrap* the quoted patch text, the result will work as a review.

Ideally, I'd love to see 1) a review UI that stores line-by-line reviews into a common format and can translate those to email, and 2) a mechanism to translate reviews written by email and quoting into the review format and store them with the repository.

> And (meta)-versioning of series.
I've got a documented format for that. :)
> And place for proof-of-concept / weather-balon patches...
Same place as all other patches, just with an "RFC" tag on them.
Junio C Hamano· Aug 10, 2016, 21:14 UTC · re: Josh Triplett · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Josh Triplett <josh@joshtriplett.org> writes:
Show 8 quoted lines
>> But submission is less important than review. And for review it is
>> usually better (except gigantic series) to have patch text for review
>> with the review.
>
> Agreed.  However, submission typically requires more work than review,
> because the patch text must remain applicable.  For review, as long as
> the email client you use to respond doesn't do something horrible like
> *re-wrap* the quoted patch text, the result will work as a review.

Yup. That is why we say "please send patch inline; when asked to send it as an attachment, please do so".

Duy Nguyen· Aug 5, 2016, 14:55 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

On Wed, Aug 3, 2016 at 6:07 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 6 quoted lines
> It would be a totally different matter, of course, if you used the
> branches I publish via my GitHub repository, added fixup! and squash!
> commits, published the result to a public repository and then told me to
> pull from there, that would make things easier. We could even introduce a
> reword! construct, to make the review of the suggested edits of the commit
> message easier.

On the topic of fixup and squash and everything. Is anyone else annoyed that the commit title is taken for fixup!, squash! instructions? After you have added a few of them, "git log --oneline" becomes useless. All you see is "fixup! A", "fixup! A", "fixup! B", "fixup! A".

Would it be better to let the user control the title? We still need the cue "fixup!", "squash!"... at the beginning of the title, but the original commit reference is appended at the end, like s-o-b lines.

-- 
Duy
Johannes Schindelin· Aug 5, 2016, 15:13 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Duy,
On Fri, 5 Aug 2016, Duy Nguyen wrote:
> On the topic of fixup and squash and everything. Is anyone else
> annoyed that the commit title is taken for fixup!, squash!
> instructions?

I do not know about others, but I am not annoyed by those commit titles. I think they make tons of sense.

Ciao, Dscho

Philip Oakley· Aug 5, 2016, 18:42 UTC · re: Duy Nguyen · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

From: "Duy Nguyen" <pclouds@gmail.com>
Show 19 quoted lines
> On Wed, Aug 3, 2016 at 6:07 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> It would be a totally different matter, of course, if you used the
>> branches I publish via my GitHub repository, added fixup! and squash!
>> commits, published the result to a public repository and then told me to
>> pull from there, that would make things easier. We could even introduce a
>> reword! construct, to make the review of the suggested edits of the 
>> commit
>> message easier.
>
> On the topic of fixup and squash and everything. Is anyone else
> annoyed that the commit title is taken for fixup!, squash!
> instructions? After you have added a few of them, "git log --oneline"
> becomes useless. All you see is "fixup! A", "fixup! A", "fixup! B",
> "fixup! A".
>
> Would it be better to let the user control the title? We still need
> the cue "fixup!", "squash!"... at the beginning of the title, but the
> original commit reference is appended at the end, like s-o-b lines.

In the same vein, I always want,with my workflow, to use "fixup! <short_sha1>".

This would be to save trying to retype the title correctly, and simply use the abbreviated sha1, which would nicely allow an extra short summaty of what the fixup is about.

Philip 
Johannes Schindelin· Aug 6, 2016, 08:38 UTC · re: Philip Oakley · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Hi Philip,
On Fri, 5 Aug 2016, Philip Oakley wrote:
> In the same vein, I always want,with my workflow, to use "fixup!
> <short_sha1>".
Good news for you: this is already supported. See here:
https://github.com/git/git/blob/v2.9.2/git-rebase--interactive.sh#L786-L796

Ciao, Dscho

Philip Oakley· Aug 6, 2016, 17:45 UTC · re: Johannes Schindelin · lore

Re: patch submission process, was Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

From: "Johannes Schindelin" <Johannes.Schindelin@gmx.de>
Show 11 quoted lines
> Hi Philip,
>
> On Fri, 5 Aug 2016, Philip Oakley wrote:
>
>> In the same vein, I always want,with my workflow, to use "fixup!
>> <short_sha1>".
>
> Good news for you: this is already supported. See here:
>
> https://github.com/git/git/blob/v2.9.2/git-rebase--interactive.sh#L786-L796
>
That's odd <knowing look>, I never saw any of that in the documentation...

Blame says it was 68d5d03 (rebase: teach --autosquash to match on sha1 in addition to message, 2010-11-04) which was before I discovered Git. Maybe another documentation fixup needed ;-)

Mind you I'm not sure about 22c5b13 (rebase -i: handle fixup! fixup! in --autosquash, 2013-06-27) which looks to only allow one fixup, but maybe I'm misreading. [e.g. recieve multiple fixups from the list, or need extra fixups as code of documentation is tested]

The capability is still good to know.
Philip 
Junio C Hamano· Aug 2, 2016, 22:28 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 06/16] merge_recursive: abort properly upon errors

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> I tend to think that the underscore is correct: this change is not so much
> about the builtin (which is written with a dash) but about the function
> (written with an underscore, used by more than just merge-recursive, e.g.
> cherry-pick).

Yes, I agree. "merge-recursive:" prefix is about either the built-in command, or the machinery as a whole to support that built-in command. It is preferrable to use "merge_recursive():" if we are talking about a single function.

Thanks.
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 13/16] merge-recursive: write the commit title in one go

In 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we changed the code such that it prints the output in one go, to avoid interfering with the progress output.

Let's make sure that the same holds true when outputting the commit title: previously, we used several printf() statements to stdout and assumed that stdout's buffer is large enough to hold the entire commit title.

Apart from making that speculation unnecessary, we change the code to add the message to the output buffer before flushing for another reason: the next commit will introduce a new level of output buffering, where the caller can request the output not to be flushed, but to be retained for further processing.

This latter feature will be needed when teaching the sequencer to do rebase -i's brunt work: it wants to control the output of the cherry-picks (i.e. recursive merges).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
Show changes to merge-recursive.c +9 −8
diff --git a/merge-recursive.c b/merge-recursive.c
index b972a83..99c9635 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -191,25 +191,26 @@ static void output(struct merge_options *o, int v, const char *fmt, ...)
 
 static void output_commit_title(struct merge_options *o, struct commit *commit)
 {
-	int i;
-	flush_output(o);
-	for (i = o->call_depth; i--;)
-		fputs("  ", stdout);
+	strbuf_addchars(&o->obuf, ' ', o->call_depth * 2);
 	if (commit->util)
-		printf("virtual %s\n", merge_remote_util(commit)->name);
+		strbuf_addf(&o->obuf, "virtual %s\n",
+			merge_remote_util(commit)->name);
 	else {
-		printf("%s ", find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV));
+		strbuf_addf(&o->obuf, "%s ",
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV));
 		if (parse_commit(commit) != 0)
-			printf(_("(bad commit)\n"));
+			strbuf_addf(&o->obuf, _("(bad commit)\n"));
 		else {
 			const char *title;
 			const char *msg = get_commit_buffer(commit, NULL);
 			int len = find_commit_subject(msg, &title);
 			if (len)
-				printf("%.*s\n", len, title);
+				strbuf_addf(&o->obuf, "%.*s\n", len, title);
 			unuse_commit_buffer(commit, msg);
 		}
 	}
+	flush_output(o);
 }
 
 static int add_cacheinfo(struct merge_options *o,
-- 
2.9.0.281.g286a8d9
Johannes Schindelin· Aug 1, 2016, 11:44 UTC · re: Johannes Schindelin · lore

[PATCH v6 16/16] merge-recursive: flush output buffer even when erroring out

Ever since 66a155b (Enable output buffering in merge-recursive., 2007-01-14), we had a problem: When the merge failed in a fatal way, all regular output was swallowed because we called die() and did not get a chance to drain the output buffers.

To fix this, several modifications were necessary:
- we needed to stop die()ing, to give callers a chance to do something
  when an error occurred (in this case, flush the output buffers),
- we needed to delay printing the error message so that the caller can
  print the buffered output before that, and
- we needed to make sure that the output buffers are flushed even when
  the return value indicates an error.

The first two changes were introduced through earlier commits in this patch series, and this commit addresses the third one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-recursive.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to merge-recursive.c +3 −1
diff --git a/merge-recursive.c b/merge-recursive.c
index 9e527de..c9e4dbc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2069,8 +2069,10 @@ int merge_recursive(struct merge_options *o,
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
 			    &mrtree);
-	if (clean < 0)
+	if (clean < 0) {
+		flush_output(o);
 		return clean;
+	}
 
 	if (o->call_depth) {
 		*result = make_virtual_commit(mrtree, "merged tree");
-- 
2.9.0.281.g286a8d9
Junio C Hamano· Aug 4, 2016, 18:20 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 16/16] merge-recursive: flush output buffer even when erroring out

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> Ever since 66a155b (Enable output buffering in merge-recursive.,
> 2007-01-14), we had a problem: When the merge failed in a fatal way, all
> regular output was swallowed because we called die() and did not get a
> chance to drain the output buffers.

OK. Even though I really wanted to see somebody else review this series as well, I finished reading it through one more time before that happened, which is unfortunate because I think this is ready to start cooking in 'next' even though I no longer have much faith in my eyes alone after staring at this series so many times---you start missing details.

Thanks.
Johannes Schindelin· Aug 5, 2016, 15:41 UTC · re: Junio C Hamano · lore

Re: [PATCH v6 16/16] merge-recursive: flush output buffer even when erroring out

Hi Junio,
On Thu, 4 Aug 2016, Junio C Hamano wrote:
Show 13 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Ever since 66a155b (Enable output buffering in merge-recursive.,
> > 2007-01-14), we had a problem: When the merge failed in a fatal way, all
> > regular output was swallowed because we called die() and did not get a
> > chance to drain the output buffers.
> 
> OK.  Even though I really wanted to see somebody else review this
> series as well, I finished reading it through one more time before
> that happened, which is unfortunate because I think this is ready to
> start cooking in 'next' even though I no longer have much faith in
> my eyes alone after staring at this series so many times---you start
> missing details.
Yeah, well, it is a rather crucial piece of the code.

But then, I really tried my best to re-review the series a couple of times (with my primary focus on robustness, not elegance) after working on different tasks for a couple of days.

Combined with my long-term dogfooding and my readiness to jump on any breakage I may have introduced, I am relatively confident nevertheless.

Thank you so much for your patient reviews!

Ciao, Dscho

Junio C Hamano· Aug 6, 2016, 16:37 UTC · re: Johannes Schindelin · lore

Re: [PATCH v6 16/16] merge-recursive: flush output buffer even when erroring out

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 12 quoted lines
> On Thu, 4 Aug 2016, Junio C Hamano wrote:
>
>> OK.  Even though I really wanted to see somebody else review this
>> series as well, I finished reading it through one more time before
>> that happened, which is unfortunate because I think this is ready to
>> start cooking in 'next' even though I no longer have much faith in
>> my eyes alone after staring at this series so many times---you start
>> missing details.
>
> Yeah, well, it is a rather crucial piece of the code.
> ...
> Thank you so much for your patient reviews!
Thanks for working on this to you, too ;-)

← back to recent threads