threads / patch / 52339

patch, 2 partsAdvice upon clone --recurse-submodules --reference

Subject: [PATCH 0/2] Advice upon clone --recurse-submodules --reference

## tl;dr

11 messages between Nov 26, 2019 and Dec 3, 2019. Diffs are folded; open one to read it.

replies: 10people: 3as markdown or json

Jonathan Tan· Nov 26, 2019, 01:30 UTC · lore

This patch set came about because of a situation in $DAY_JOB described in the commit message of patch 2. I found the existing documentation and error messages unclear, so I have improved them.

Jonathan Tan (2):
  Doc: explain submodule.alternateErrorStrategy
  submodule--helper: advise on fatal alternate error
 Documentation/config/advice.txt    |  3 +++
 Documentation/config/submodule.txt |  4 +++-
 advice.c                           |  2 ++
 advice.h                           |  1 +
 builtin/submodule--helper.c        | 10 ++++++++++
 5 files changed, 19 insertions(+), 1 deletion(-)
-- 
2.24.0.432.g9d3f5f5b63-goog
Jonathan Tan· Nov 26, 2019, 01:30 UTC · re: Jonathan Tan · lore

[PATCH 1/2] Doc: explain submodule.alternateErrorStrategy

Commit 31224cbdc7 ("clone: recursive and reference option triggers submodule alternates", 2016-08-17) taught Git to support the configuration options "submodule.alternateLocation" and "submodule.alternateErrorStrategy" on a superproject.

If "submodule.alternateLocation" is configured to "superproject" on a superproject, whenever a submodule of that superproject is cloned, it instead computes the analogous alternate path for that submodule from $GIT_DIR/objects/info/alternates of the superproject, and references it.

The "submodule.alternateErrorStrategy" option determines what happens if that alternate cannot be referenced. However, it is not clear that the clone proceeds as if no alternate was specified when that option is not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore, document it accordingly.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
When I said "not clear" above, I mean that it is not clear *to me*, and
I assume that others will feel the same way. Feel free to drop this from
the patch set if the existing documentation is clear to most people.
---
 Documentation/config/submodule.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to Documentation/config/submodule.txt +3 −1
diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt
index 0a1293b051..b33177151c 100644
--- a/Documentation/config/submodule.txt
+++ b/Documentation/config/submodule.txt
@@ -79,4 +79,6 @@ submodule.alternateLocation::
 submodule.alternateErrorStrategy::
 	Specifies how to treat errors with the alternates for a submodule
 	as computed via `submodule.alternateLocation`. Possible values are
-	`ignore`, `info`, `die`. Default is `die`.
+	`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`
+	or `info`, and if there is an error with the computed alternate, the
+	clone proceeds as if no alternate was specified.
-- 
2.24.0.432.g9d3f5f5b63-goog
Jeff King· Nov 27, 2019, 11:32 UTC · re: Jonathan Tan · lore

Re: [PATCH 1/2] Doc: explain submodule.alternateErrorStrategy

On Mon, Nov 25, 2019 at 05:30:59PM -0800, Jonathan Tan wrote:
Show 21 quoted lines
> Commit 31224cbdc7 ("clone: recursive and reference option triggers
> submodule alternates", 2016-08-17) taught Git to support the
> configuration options "submodule.alternateLocation" and
> "submodule.alternateErrorStrategy" on a superproject.
> 
> If "submodule.alternateLocation" is configured to "superproject" on a
> superproject, whenever a submodule of that superproject is cloned, it
> instead computes the analogous alternate path for that submodule from
> $GIT_DIR/objects/info/alternates of the superproject, and references it.
> 
> The "submodule.alternateErrorStrategy" option determines what happens
> if that alternate cannot be referenced. However, it is not clear that
> the clone proceeds as if no alternate was specified when that option is
> not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore,
> document it accordingly.
> 
> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
> When I said "not clear" above, I mean that it is not clear *to me*, and
> I assume that others will feel the same way. Feel free to drop this from
> the patch set if the existing documentation is clear to most people.

FWIW, I learned about these options for the first time from your email, and I think the suggested change makes the behavior much more clear.

-Peff
Junio C Hamano· Nov 27, 2019, 12:30 UTC · re: Jonathan Tan · lore

Re: [PATCH 1/2] Doc: explain submodule.alternateErrorStrategy

Jonathan Tan <jonathantanmy@google.com> writes:
Show 10 quoted lines
> Commit 31224cbdc7 ("clone: recursive and reference option triggers
> submodule alternates", 2016-08-17) taught Git to support the
> configuration options "submodule.alternateLocation" and
> "submodule.alternateErrorStrategy" on a superproject.
> ...
> The "submodule.alternateErrorStrategy" option determines what happens
> if that alternate cannot be referenced. However, it is not clear that
> the clone proceeds as if no alternate was specified when that option is
> not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore,
> document it accordingly.

Given that for everyday use (cf. sha1-file.c::link_alt_odb_entry()) of an alternate is best-effort basis, I have a feeling that it was a design mistake to have the "error strategy" configuration option in the first place, and "clone --reference-if-able" was the result of the same design mistake. We would have been better off if we made these the best-effort features as well.

But the ship has sailed long ago---so I think these two are the best we can do at this point. Perhaps flipping the default to warn may be a longer term improvement, too.

Jonathan Tan· Nov 26, 2019, 01:31 UTC · re: Jonathan Tan · lore

[PATCH 2/2] submodule--helper: advise on fatal alternate error

When recursively cloning a superproject with some shallow modules defined in its .gitmodules, then recloning with "--reference=<path>", an error occurs. For example:

  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    master
  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    --reference master master2
fails with:
  fatal: submodule '<snip>' cannot add alternate: reference repository
  '<snip>' is shallow

When a alternate computed from the superproject's alternate cannot be added, whether in this case or another, advise about configuring the "submodule.alternateErrorStrategy" configuration option and using "--reference-if-able" instead of "--reference" when cloning.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 Documentation/config/advice.txt |  3 +++
 advice.c                        |  2 ++
 advice.h                        |  1 +
 builtin/submodule--helper.c     | 10 ++++++++++
 4 files changed, 16 insertions(+)
Show changes to 4 files +16 −0

Documentation/config/advice.txt, advice.c, advice.h, builtin/submodule--helper.c

diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
index 6aaa360202..d4e698cd3f 100644
--- a/Documentation/config/advice.txt
+++ b/Documentation/config/advice.txt
@@ -107,4 +107,7 @@ advice.*::
 		editor input from the user.
 	nestedTag::
 		Advice shown if a user attempts to recursively tag a tag object.
+	submoduleAlternateErrorStrategyDie:
+		Advice shown when a submodule.alternateErrorStrategy option
+		configured to "die" causes a fatal error.
 --
diff --git a/advice.c b/advice.c
index 3ee0ee2d8f..249c60dcf3 100644
--- a/advice.c
+++ b/advice.c
@@ -30,6 +30,7 @@ int advice_waiting_for_editor = 1;
 int advice_graft_file_deprecated = 1;
 int advice_checkout_ambiguous_remote_branch_name = 1;
 int advice_nested_tag = 1;
+int advice_submodule_alternate_error_strategy_die = 1;
 
 static int advice_use_color = -1;
 static char advice_colors[][COLOR_MAXLEN] = {
@@ -89,6 +90,7 @@ static struct {
 	{ "graftFileDeprecated", &advice_graft_file_deprecated },
 	{ "checkoutAmbiguousRemoteBranchName", &advice_checkout_ambiguous_remote_branch_name },
 	{ "nestedTag", &advice_nested_tag },
+	{ "submoduleAlternateErrorStrategyDie", &advice_submodule_alternate_error_strategy_die },
 
 	/* make this an alias for backward compatibility */
 	{ "pushNonFastForward", &advice_push_update_rejected }
diff --git a/advice.h b/advice.h
index d015404843..b706780614 100644
--- a/advice.h
+++ b/advice.h
@@ -30,6 +30,7 @@ extern int advice_waiting_for_editor;
 extern int advice_graft_file_deprecated;
 extern int advice_checkout_ambiguous_remote_branch_name;
 extern int advice_nested_tag;
+extern int advice_submodule_alternate_error_strategy_die;
 
 int git_default_advice_config(const char *var, const char *value);
 __attribute__((format (printf, 1, 2)))
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2c2395a620..bdfc46fdea 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -19,6 +19,7 @@
 #include "diffcore.h"
 #include "diff.h"
 #include "object-store.h"
+#include "advice.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
@@ -1268,6 +1269,14 @@ struct submodule_alternate_setup {
 #define SUBMODULE_ALTERNATE_SETUP_INIT { NULL, \
 	SUBMODULE_ALTERNATE_ERROR_IGNORE, NULL }
 
+static void advise_submodule_alternate_error_die(void)
+{
+	if (!advice_submodule_alternate_error_strategy_die)
+		return;
+	advise(_("An alternate computed from a superproject's alternate is invalid."));
+	advise(_("To allow Git to clone without an alternate in such a case, set submodule.alternateErrorStrategy to 'info' or, equivalently, clone with '--reference-if-able' instead of '--reference'."));
+}
+
 static int add_possible_reference_from_superproject(
 		struct object_directory *odb, void *sas_cb)
 {
@@ -1299,6 +1308,7 @@ static int add_possible_reference_from_superproject(
 		} else {
 			switch (sas->error_mode) {
 			case SUBMODULE_ALTERNATE_ERROR_DIE:
+				advise_submodule_alternate_error_die();
 				die(_("submodule '%s' cannot add alternate: %s"),
 				    sas->submodule_name, err.buf);
 			case SUBMODULE_ALTERNATE_ERROR_INFO:
-- 
2.24.0.432.g9d3f5f5b63-goog
Jeff King· Nov 27, 2019, 11:49 UTC · re: Jonathan Tan · lore

Re: [PATCH 2/2] submodule--helper: advise on fatal alternate error

On Mon, Nov 25, 2019 at 05:31:00PM -0800, Jonathan Tan wrote:
Show 20 quoted lines
> When recursively cloning a superproject with some shallow modules
> defined in its .gitmodules, then recloning with "--reference=<path>", an
> error occurs. For example:
> 
>   git clone --recurse-submodules --branch=master -j8 \
>     https://android.googlesource.com/platform/superproject \
>     master
>   git clone --recurse-submodules --branch=master -j8 \
>     https://android.googlesource.com/platform/superproject \
>     --reference master master2
> 
> fails with:
> 
>   fatal: submodule '<snip>' cannot add alternate: reference repository
>   '<snip>' is shallow
> 
> When a alternate computed from the superproject's alternate cannot be
> added, whether in this case or another, advise about configuring the
> "submodule.alternateErrorStrategy" configuration option and using
> "--reference-if-able" instead of "--reference" when cloning.

It sounds like that advice sends people in the right direction, which is a good thing.

I kind of wonder if the default for alternateErrorStrategy should be more lenient, but I don't really know much about the feature in the first place. I'll let people who are more clueful ponder that, but certainly your patch is an improvement in the meantime.

One minor suggestion:
Show 7 quoted lines
> +static void advise_submodule_alternate_error_die(void)
> +{
> +	if (!advice_submodule_alternate_error_strategy_die)
> +		return;
> +	advise(_("An alternate computed from a superproject's alternate is invalid."));
> +	advise(_("To allow Git to clone without an alternate in such a case, set submodule.alternateErrorStrategy to 'info' or, equivalently, clone with '--reference-if-able' instead of '--reference'."));
> +}

The advise() function handles newlines gracefully (putting a "hint:" at the start of each line). So you can put both in a single call, and wrap the second long line, which also lets translators see the whole message as a unit. Many of the existing calls define the message outside of any function (e.g., embedded_advice), which makes it easier to see how long the lines will be when displayed.

So maybe:
  static const alternate_error_advice[] = N_(
  "An alternate computed from a superproject's alternate is invalid.\n"
  "To allow Git to clone without an alternate in such a case, set\n"
  "submodule.alternateErrorStrategy to 'info' or, equivalently, clone with\n"
  "'--reference-if-able' instead of '--reference'."
  );
  ...
	switch (sas->error_mode) {
	case SUBMODULE_ALTERNATE_ERROR_DIE:
		if (advice_submodule_alternate_error_strategy_die)
			advise(_(alternate_error_advice));
		die(...);
?
-Peff
Jonathan Tan· Dec 2, 2019, 19:57 UTC · re: Jonathan Tan · lore

[PATCH v2 0/2] Advice upon clone --recurse-submodules --reference

Thanks everyone! Here's an updated v2 following Peff's suggestions on how to write the advice part.

Making submodule.alternateErrorStrategy default to 'info' is reasonable, but I don't know how much it matters in practice - here, submodule.alternateLocation was set because of "clone --recurse-submodules --reference", which also sets submodule.alternateErrorStrategy. I don't know when submodule.alternateLocation would ever be set alone, but then again, I'm not very familiar with this part. I've left that alone for now.

Jonathan Tan (2):
  Doc: explain submodule.alternateErrorStrategy
  submodule--helper: advise on fatal alternate error
 Documentation/config/advice.txt    |  3 +++
 Documentation/config/submodule.txt |  4 +++-
 advice.c                           |  2 ++
 advice.h                           |  1 +
 builtin/submodule--helper.c        | 10 ++++++++++
 5 files changed, 19 insertions(+), 1 deletion(-)
-- 
2.24.0.393.g34dc348eaf-goog
Jonathan Tan· Dec 2, 2019, 19:57 UTC · re: Jonathan Tan · lore

[PATCH v2 1/2] Doc: explain submodule.alternateErrorStrategy

Commit 31224cbdc7 ("clone: recursive and reference option triggers submodule alternates", 2016-08-17) taught Git to support the configuration options "submodule.alternateLocation" and "submodule.alternateErrorStrategy" on a superproject.

If "submodule.alternateLocation" is configured to "superproject" on a superproject, whenever a submodule of that superproject is cloned, it instead computes the analogous alternate path for that submodule from $GIT_DIR/objects/info/alternates of the superproject, and references it.

The "submodule.alternateErrorStrategy" option determines what happens if that alternate cannot be referenced. However, it is not clear that the clone proceeds as if no alternate was specified when that option is not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore, document it accordingly.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 Documentation/config/submodule.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to Documentation/config/submodule.txt +3 −1
diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt
index 0a1293b051..b33177151c 100644
--- a/Documentation/config/submodule.txt
+++ b/Documentation/config/submodule.txt
@@ -79,4 +79,6 @@ submodule.alternateLocation::
 submodule.alternateErrorStrategy::
 	Specifies how to treat errors with the alternates for a submodule
 	as computed via `submodule.alternateLocation`. Possible values are
-	`ignore`, `info`, `die`. Default is `die`.
+	`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`
+	or `info`, and if there is an error with the computed alternate, the
+	clone proceeds as if no alternate was specified.
-- 
2.24.0.393.g34dc348eaf-goog
Jonathan Tan· Dec 2, 2019, 19:57 UTC · re: Jonathan Tan · lore

[PATCH v2 2/2] submodule--helper: advise on fatal alternate error

When recursively cloning a superproject with some shallow modules defined in its .gitmodules, then recloning with "--reference=<path>", an error occurs. For example:

  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    master
  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    --reference master master2
fails with:
  fatal: submodule '<snip>' cannot add alternate: reference repository
  '<snip>' is shallow

When a alternate computed from the superproject's alternate cannot be added, whether in this case or another, advise about configuring the "submodule.alternateErrorStrategy" configuration option and using "--reference-if-able" instead of "--reference" when cloning.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 Documentation/config/advice.txt |  3 +++
 advice.c                        |  2 ++
 advice.h                        |  1 +
 builtin/submodule--helper.c     | 10 ++++++++++
 4 files changed, 16 insertions(+)
Show changes to 4 files +16 −0

Documentation/config/advice.txt, advice.c, advice.h, builtin/submodule--helper.c

diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
index 6aaa360202..d4e698cd3f 100644
--- a/Documentation/config/advice.txt
+++ b/Documentation/config/advice.txt
@@ -107,4 +107,7 @@ advice.*::
 		editor input from the user.
 	nestedTag::
 		Advice shown if a user attempts to recursively tag a tag object.
+	submoduleAlternateErrorStrategyDie:
+		Advice shown when a submodule.alternateErrorStrategy option
+		configured to "die" causes a fatal error.
 --
diff --git a/advice.c b/advice.c
index 3ee0ee2d8f..249c60dcf3 100644
--- a/advice.c
+++ b/advice.c
@@ -30,6 +30,7 @@ int advice_waiting_for_editor = 1;
 int advice_graft_file_deprecated = 1;
 int advice_checkout_ambiguous_remote_branch_name = 1;
 int advice_nested_tag = 1;
+int advice_submodule_alternate_error_strategy_die = 1;
 
 static int advice_use_color = -1;
 static char advice_colors[][COLOR_MAXLEN] = {
@@ -89,6 +90,7 @@ static struct {
 	{ "graftFileDeprecated", &advice_graft_file_deprecated },
 	{ "checkoutAmbiguousRemoteBranchName", &advice_checkout_ambiguous_remote_branch_name },
 	{ "nestedTag", &advice_nested_tag },
+	{ "submoduleAlternateErrorStrategyDie", &advice_submodule_alternate_error_strategy_die },
 
 	/* make this an alias for backward compatibility */
 	{ "pushNonFastForward", &advice_push_update_rejected }
diff --git a/advice.h b/advice.h
index d015404843..b706780614 100644
--- a/advice.h
+++ b/advice.h
@@ -30,6 +30,7 @@ extern int advice_waiting_for_editor;
 extern int advice_graft_file_deprecated;
 extern int advice_checkout_ambiguous_remote_branch_name;
 extern int advice_nested_tag;
+extern int advice_submodule_alternate_error_strategy_die;
 
 int git_default_advice_config(const char *var, const char *value);
 __attribute__((format (printf, 1, 2)))
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2c2395a620..12d546dfbb 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -19,6 +19,7 @@
 #include "diffcore.h"
 #include "diff.h"
 #include "object-store.h"
+#include "advice.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
@@ -1268,6 +1269,13 @@ struct submodule_alternate_setup {
 #define SUBMODULE_ALTERNATE_SETUP_INIT { NULL, \
 	SUBMODULE_ALTERNATE_ERROR_IGNORE, NULL }
 
+static const char alternate_error_advice[] = N_(
+"An alternate computed from a superproject's alternate is invalid.\n"
+"To allow Git to clone without an alternate in such a case, set\n"
+"submodule.alternateErrorStrategy to 'info' or, equivalently, clone with\n"
+"'--reference-if-able' instead of '--reference'."
+);
+
 static int add_possible_reference_from_superproject(
 		struct object_directory *odb, void *sas_cb)
 {
@@ -1299,6 +1307,8 @@ static int add_possible_reference_from_superproject(
 		} else {
 			switch (sas->error_mode) {
 			case SUBMODULE_ALTERNATE_ERROR_DIE:
+				if (advice_submodule_alternate_error_strategy_die)
+					advise(_(alternate_error_advice));
 				die(_("submodule '%s' cannot add alternate: %s"),
 				    sas->submodule_name, err.buf);
 			case SUBMODULE_ALTERNATE_ERROR_INFO:
-- 
2.24.0.393.g34dc348eaf-goog
Jeff King· Dec 3, 2019, 15:39 UTC · re: Jonathan Tan · lore

Re: [PATCH v2 0/2] Advice upon clone --recurse-submodules --reference

On Mon, Dec 02, 2019 at 11:57:50AM -0800, Jonathan Tan wrote:
> Thanks everyone! Here's an updated v2 following Peff's suggestions on
> how to write the advice part.
Thanks, this looks good to me.

I think Junio and I both wondered if the default should be flipped to "info", but I don't feel strongly about it, and this seems like an improvement in the meantime.

-Peff
Junio C Hamano· Dec 3, 2019, 16:50 UTC · re: Jeff King · lore

Re: [PATCH v2 0/2] Advice upon clone --recurse-submodules --reference

Jeff King <peff@peff.net> writes:
Show 10 quoted lines
> On Mon, Dec 02, 2019 at 11:57:50AM -0800, Jonathan Tan wrote:
>
>> Thanks everyone! Here's an updated v2 following Peff's suggestions on
>> how to write the advice part.
>
> Thanks, this looks good to me.
>
> I think Junio and I both wondered if the default should be flipped to
> "info", but I don't feel strongly about it, and this seems like an
> improvement in the meantime.
We indeed did, and I agree this is a good step regardless.
Thanks, both.

← back to recent threads