threads / patch / 51976

v2, 2 partsformat-patch: create leading components of output directory

Subject: [PATCH v2 1/2] format-patch: create leading components of output directory

## tl;dr

7 messages between Oct 5, 2019 and Oct 8, 2019. Diffs are folded; open one to read it.

replies: 6people: 4as markdown or json

Bert Wesarg· Oct 5, 2019, 08:43 UTC · lore
Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
--- 
ChangeLog:
v2:
 * squashed and base new tests on 'dl/format-patch-doc-test-cleanup'
Cc: Denton Liu <liu.denton@gmail.com>
---
 Documentation/config/format.txt    |  2 +-
 Documentation/git-format-patch.txt |  3 ++-
 builtin/log.c                      |  8 ++++++++
 t/t4014-format-patch.sh            | 20 ++++++++++++++++++++
 4 files changed, 31 insertions(+), 2 deletions(-)
Show changes to 4 files +31 −2

Documentation/config/format.txt, Documentation/git-format-patch.txt, builtin/log.c, t/t4014-format-patch.sh

diff --git a/Documentation/config/format.txt b/Documentation/config/format.txt
index cb629fa769..40cad9278f 100644
--- a/Documentation/config/format.txt
+++ b/Documentation/config/format.txt
@@ -81,7 +81,7 @@ format.coverLetter::
 
 format.outputDirectory::
 	Set a custom directory to store the resulting files instead of the
-	current working directory.
+	current working directory. All directory components will be created.
 
 format.useAutoBase::
 	A boolean value which lets you enable the `--base=auto` option of
diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 0ac56f4b70..2035d4d5d5 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -66,7 +66,8 @@ they are created in the current working directory. The default path
 can be set with the `format.outputDirectory` configuration option.
 The `-o` option takes precedence over `format.outputDirectory`.
 To store patches in the current working directory even when
-`format.outputDirectory` points elsewhere, use `-o .`.
+`format.outputDirectory` points elsewhere, use `-o .`. All directory
+components will be created.
 
 By default, the subject of a single patch is "[PATCH] " followed by
 the concatenation of lines from the commit message up to the first blank
diff --git a/builtin/log.c b/builtin/log.c
index c4b35fdaf9..294534aacb 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1770,6 +1770,14 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 			rev.diffopt.use_color = GIT_COLOR_NEVER;
 		if (use_stdout)
 			die(_("standard output, or directory, which one?"));
+		switch (safe_create_leading_directories_const(output_directory)) {
+		case SCLD_OK:
+		case SCLD_EXISTS:
+			break;
+		default:
+			die(_("could not create leading directories "
+			      "of '%s'"), output_directory);
+		}
 		if (mkdir(output_directory, 0777) < 0 && errno != EEXIST)
 			die_errno(_("could not create directory '%s'"),
 				  output_directory);
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 83f52614d3..2f2cd6fea6 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1606,6 +1606,26 @@ test_expect_success 'From line has expected format' '
 	test_cmp from filtered
 '
 
+test_expect_success 'format-patch -o with no leading directories' '
+	rm -fr patches &&
+	git format-patch -o patches master..side &&
+	git rev-list master..side >list &&
+	test_line_count = $(ls patches | wc -l) list
+'
+
+test_expect_success 'format-patch -o with leading existing directories' '
+	git format-patch -o patches/side master..side &&
+	git rev-list master..side >list &&
+	test_line_count = $(ls patches/side | wc -l) list
+'
+
+test_expect_success 'format-patch -o with leading non-existing directories' '
+	rm -fr patches &&
+	git format-patch -o patches/side master..side &&
+	git rev-list master..side >list &&
+	test_line_count = $(ls patches/side | wc -l) list
+'
+
 test_expect_success 'format-patch format.outputDirectory option' '
 	test_config format.outputDirectory patches &&
 	rm -fr patches &&
-- 
2.23.0.11.g242cf7f110
Bert Wesarg· Oct 5, 2019, 08:43 UTC · re: Bert Wesarg · lore

[PATCH v2 2/2] [RFC] format-patch: configure a command to generate the output directory name

The 'format.outputDirectory' configuration is only able to store constant directory names. Though some may use

   $ git format-patch -o $(createdir) …

to name the directory dynamically. Provide a new configuration to be able to store such a command too.

Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
--- 
ChangeLog:
v2:
 * rephrase motivation
Cc: Alexander Kuleshov <kuleshovmail@gmail.com>
Cc: Eric Sunshine <sunshine@sunshineco.com>
Cc: Denton Liu <liu.denton@gmail.com>
Cc: Junio C Hamano <gitster@pobox.com>
---
 Documentation/config/format.txt    |  5 +++++
 Documentation/git-format-patch.txt |  6 +++++-
 builtin/log.c                      | 24 +++++++++++++++++++++++-
 t/t4014-format-patch.sh            | 25 +++++++++++++++++++++++++
 4 files changed, 58 insertions(+), 2 deletions(-)
Show changes to 4 files +58 −2

Documentation/config/format.txt, Documentation/git-format-patch.txt, builtin/log.c, t/t4014-format-patch.sh

diff --git a/Documentation/config/format.txt b/Documentation/config/format.txt
index 40cad9278f..420188a1c6 100644
--- a/Documentation/config/format.txt
+++ b/Documentation/config/format.txt
@@ -83,6 +83,11 @@ format.outputDirectory::
 	Set a custom directory to store the resulting files instead of the
 	current working directory. All directory components will be created.
 
+format.outputDirectoryCmd::
+	The command which is used to name a custom directory to store the
+	resulting files instead of the current working directory. All directory
+	components will be created.
+
 format.useAutoBase::
 	A boolean value which lets you enable the `--base=auto` option of
 	format-patch by default.
diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 2035d4d5d5..4936b9f91d 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -67,7 +67,11 @@ can be set with the `format.outputDirectory` configuration option.
 The `-o` option takes precedence over `format.outputDirectory`.
 To store patches in the current working directory even when
 `format.outputDirectory` points elsewhere, use `-o .`. All directory
-components will be created.
+components will be created. The 'format.outputDirectoryCmd' configuration can
+be used to name a command to produce the directory name programmatically. The
+command should produce the name to its standard output. The
+`format.outputDirectory` configuration takes precedence over
+`format.outputDirectoryCmd`.
 
 By default, the subject of a single patch is "[PATCH] " followed by
 the concatenation of lines from the commit message up to the first blank
diff --git a/builtin/log.c b/builtin/log.c
index 294534aacb..e841306e61 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -775,6 +775,7 @@ static const char *signature = git_version_string;
 static const char *signature_file;
 static int config_cover_letter;
 static const char *config_output_directory;
+static const char *config_output_directory_cmd;
 
 enum {
 	COVER_UNSET,
@@ -857,6 +858,8 @@ static int git_format_config(const char *var, const char *value, void *cb)
 	}
 	if (!strcmp(var, "format.outputdirectory"))
 		return git_config_string(&config_output_directory, var, value);
+	if (!strcmp(var, "format.outputdirectorycmd"))
+		return git_config_string(&config_output_directory_cmd, var, value);
 	if (!strcmp(var, "format.useautobase")) {
 		base_auto = git_config_bool(var, value);
 		return 0;
@@ -1757,8 +1760,27 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (rev.show_notes)
 		init_display_notes(&rev.notes_opt);
 
-	if (!output_directory && !use_stdout)
+	if (!output_directory && !use_stdout) {
+		// outputDirectoryCmd can be preceeded by outputDirectory
+		if (!config_output_directory && config_output_directory_cmd) {
+			struct child_process cp = CHILD_PROCESS_INIT;
+			const char *argv[1];
+			struct strbuf buf = STRBUF_INIT;
+			int rc;
+
+			argv[0] = config_output_directory_cmd;
+			cp.argv = argv;
+			cp.use_shell = 1;
+			rc = capture_command(&cp, &buf, PATH_MAX);
+			if (rc)
+				die(_("outputDirectoryCmd command failed: "
+				      "'%s'"), config_output_directory_cmd);
+			strbuf_setlen(&buf, strcspn(buf.buf, "\r\n"));
+			config_output_directory = strbuf_detach(&buf, NULL);
+		}
+
 		output_directory = config_output_directory;
+	}
 
 	if (!use_stdout)
 		output_directory = set_outdir(prefix, output_directory);
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 2f2cd6fea6..f351d26543 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1642,6 +1642,31 @@ test_expect_success 'format-patch -o overrides format.outputDirectory' '
 	test_path_is_dir patchset
 '
 
+test_expect_success 'format-patch format.outputDirectoryCmd option' '
+	test_config format.outputDirectoryCmd "echo patches" &&
+	rm -fr patches &&
+	git format-patch master..side &&
+	git rev-list master..side >list &&
+	test_line_count = $(ls patches | wc -l) list
+'
+
+test_expect_success 'format-patch format.outputDirectory overrides format.outputDirectoryCmd' '
+	test_config format.outputDirectoryCmd "echo patches" &&
+	test_config format.outputDirectory patchset &&
+	rm -fr patches patchset &&
+	git format-patch master..side &&
+	test_path_is_missing patches &&
+	test_path_is_dir patchset
+'
+
+test_expect_success 'format-patch -o overrides format.outputDirectoryCmd' '
+	test_config format.outputDirectoryCmd "echo patches" &&
+	rm -fr patches patchset &&
+	git format-patch -o patchset master..side &&
+	test_path_is_missing patches &&
+	test_path_is_dir patchset
+'
+
 test_expect_success 'format-patch --base' '
 	git checkout patchid &&
 
-- 
2.23.0.11.g242cf7f110
Junio C Hamano· Oct 6, 2019, 00:57 UTC · re: Bert Wesarg · lore

Re: [PATCH v2 1/2] format-patch: create leading components of output directory

Bert Wesarg <bert.wesarg@googlemail.com> writes:
Show 11 quoted lines
> +		switch (safe_create_leading_directories_const(output_directory)) {
> +		case SCLD_OK:
> +		case SCLD_EXISTS:
> +			break;
> +		default:
> +			die(_("could not create leading directories "
> +			      "of '%s'"), output_directory);
> +		}
>  		if (mkdir(output_directory, 0777) < 0 && errno != EEXIST)
>  			die_errno(_("could not create directory '%s'"),
>  				  output_directory);

There is a slight discrepancy here in that mkdir(..., 0777) is to honor the umask setting of the user who is running the command and does not care about anybody else being able to (or unable to) access the resulting directory. On the other hand, s-c-l-d is (as you can guess from the location the function is defined, sha1-file.c) meant to be used to create hierarchy _inside_ $GIT_DIR/ in such a way that anybody who needs to access the repository can access it (via core.sharedrepository config).

I do not think it matters too much in practice, but
	$ git format-patch -o $HOME/my/patch/depot

that creates intermediate levels that can be writable by other users, only because the repository you took the patches from was shared with other users, may probably be seen as a security bug.

SZEDER Gábor· Oct 7, 2019, 21:03 UTC · re: Bert Wesarg · lore

Re: [PATCH v2 1/2] format-patch: create leading components of output directory

On Sat, Oct 05, 2019 at 10:43:51AM +0200, Bert Wesarg wrote:
Show 13 quoted lines
> diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
> index 83f52614d3..2f2cd6fea6 100755
> --- a/t/t4014-format-patch.sh
> +++ b/t/t4014-format-patch.sh
> @@ -1606,6 +1606,26 @@ test_expect_success 'From line has expected format' '
>  	test_cmp from filtered
>  '
>  
> +test_expect_success 'format-patch -o with no leading directories' '
> +	rm -fr patches &&
> +	git format-patch -o patches master..side &&
> +	git rev-list master..side >list &&
> +	test_line_count = $(ls patches | wc -l) list
This is sort of a nit...

So, these tests check that 'git rev-list ...' lists as many commits as the number of files created by 'git format-patch'. While it doesn't affect the tests' correctness, this is subtly different from checking that 'git format-patch' created as many files as the number of commits listed by 'git rev-list'.

Consider how the tests' output would look like on failure: 'test_line_count' shows an error message that includes the content of the file to be checked, which in this case would consist of a bunch of commit object ids:

  test_line_count: line count for list != 3
  f7af51d27933a90554b6e9212a7e5d4ad1c74569
  bd89fce9f5096eb5cad67c342b40818b7e3ce9e4

On one hand, these object ids won't mean much to anyone who might have to debug such a test failure in the future, and on the other these tests are about 'git format-patch', not about 'git rev-list'. If the check were written like this:

  count=$(git rev-list --count master..side) &&
  ls patches >list &&
  test_line_count = $count list
then the error message on failure would look something like this:
  test_line_count: line count for list != 3
  0001-first.patch
  0002-second.patch
which, I think, would be more useful.
Show 21 quoted lines
> +'
> +
> +test_expect_success 'format-patch -o with leading existing directories' '
> +	git format-patch -o patches/side master..side &&
> +	git rev-list master..side >list &&
> +	test_line_count = $(ls patches/side | wc -l) list
> +'
> +
> +test_expect_success 'format-patch -o with leading non-existing directories' '
> +	rm -fr patches &&
> +	git format-patch -o patches/side master..side &&
> +	git rev-list master..side >list &&
> +	test_line_count = $(ls patches/side | wc -l) list
> +'
> +
>  test_expect_success 'format-patch format.outputDirectory option' '
>  	test_config format.outputDirectory patches &&
>  	rm -fr patches &&
> -- 
> 2.23.0.11.g242cf7f110
> 
Junio C Hamano· Oct 8, 2019, 03:23 UTC · re: SZEDER Gábor · lore

Re: [PATCH v2 1/2] format-patch: create leading components of output directory

SZEDER Gábor <szeder.dev@gmail.com> writes:
Show 16 quoted lines
> On one hand, these object ids won't mean much to anyone who might have
> to debug such a test failure in the future, and on the other these
> tests are about 'git format-patch', not about 'git rev-list'.  If the
> check were written like this:
>
>   count=$(git rev-list --count master..side) &&
>   ls patches >list &&
>   test_line_count = $count list
>
> then the error message on failure would look something like this:
>
>   test_line_count: line count for list != 3
>   0001-first.patch
>   0002-second.patch
>
> which, I think, would be more useful.
That's nice attention to the detail ;-)
Bert Wesarg· Oct 8, 2019, 09:06 UTC · re: SZEDER Gábor · lore

Re: [PATCH v2 1/2] format-patch: create leading components of output directory

On Mon, Oct 7, 2019 at 11:03 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 49 quoted lines
>
> On Sat, Oct 05, 2019 at 10:43:51AM +0200, Bert Wesarg wrote:
> > diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
> > index 83f52614d3..2f2cd6fea6 100755
> > --- a/t/t4014-format-patch.sh
> > +++ b/t/t4014-format-patch.sh
> > @@ -1606,6 +1606,26 @@ test_expect_success 'From line has expected format' '
> >       test_cmp from filtered
> >  '
> >
> > +test_expect_success 'format-patch -o with no leading directories' '
> > +     rm -fr patches &&
> > +     git format-patch -o patches master..side &&
> > +     git rev-list master..side >list &&
> > +     test_line_count = $(ls patches | wc -l) list
>
> This is sort of a nit...
>
> So, these tests check that 'git rev-list ...' lists as many commits as
> the number of files created by 'git format-patch'.  While it doesn't
> affect the tests' correctness, this is subtly different from checking
> that 'git format-patch' created as many files as the number of commits
> listed by 'git rev-list'.
>
> Consider how the tests' output would look like on failure:
> 'test_line_count' shows an error message that includes the content of
> the file to be checked, which in this case would consist of a bunch of
> commit object ids:
>
>   test_line_count: line count for list != 3
>   f7af51d27933a90554b6e9212a7e5d4ad1c74569
>   bd89fce9f5096eb5cad67c342b40818b7e3ce9e4
>
> On one hand, these object ids won't mean much to anyone who might have
> to debug such a test failure in the future, and on the other these
> tests are about 'git format-patch', not about 'git rev-list'.  If the
> check were written like this:
>
>   count=$(git rev-list --count master..side) &&
>   ls patches >list &&
>   test_line_count = $count list
>
> then the error message on failure would look something like this:
>
>   test_line_count: line count for list != 3
>   0001-first.patch
>   0002-second.patch
>
> which, I think, would be more useful.

thanks for this detail. As I copied an existing test with this pattern, I will add a new pre-patch to this serires which applies your idea to the existing test first, before I add new tests for this patch.

Bert
Show 23 quoted lines
>
>
> > +'
> > +
> > +test_expect_success 'format-patch -o with leading existing directories' '
> > +     git format-patch -o patches/side master..side &&
> > +     git rev-list master..side >list &&
> > +     test_line_count = $(ls patches/side | wc -l) list
> > +'
> > +
> > +test_expect_success 'format-patch -o with leading non-existing directories' '
> > +     rm -fr patches &&
> > +     git format-patch -o patches/side master..side &&
> > +     git rev-list master..side >list &&
> > +     test_line_count = $(ls patches/side | wc -l) list
> > +'
> > +
> >  test_expect_success 'format-patch format.outputDirectory option' '
> >       test_config format.outputDirectory patches &&
> >       rm -fr patches &&
> > --
> > 2.23.0.11.g242cf7f110
> >
Denton Liu· Oct 8, 2019, 09:14 UTC · re: SZEDER Gábor · lore

[PATCH] t4014: treat rev-list output as the expected value

In 6bd26f58ea (t4014: use test_line_count() where possible, 2019-08-27), we converted many test cases to take advantage of the test_line_count() function. In one conversion, we inverted the expected and actual value as tested by test_line_count(). Although functionally correct, if format-patch ever produced incorrect output, the debugging output would be a bunch of hashes which would be difficult to debug.

Invert the expected and actual values provided to test_line_count() so that if format-patch produces incorrect output, the debugging output will be a list of human-readable files instead.

Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
Thanks for point out my mistake, Szeder. This patch can be applied to
the tip of 'dl/format-patch-doc-test-cleanup' and then we should base
Bert's changes on top of this branch.
 t/t4014-format-patch.sh | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Show changes to t/t4014-format-patch.sh +3 −2
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 83f52614d3..72b09896cf 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1610,8 +1610,9 @@ test_expect_success 'format-patch format.outputDirectory option' '
 	test_config format.outputDirectory patches &&
 	rm -fr patches &&
 	git format-patch master..side &&
-	git rev-list master..side >list &&
-	test_line_count = $(ls patches | wc -l) list
+	count=$(git rev-list --count master..side) &&
+	ls patches >list &&
+	test_line_count = $count list
 '
 
 test_expect_success 'format-patch -o overrides format.outputDirectory' '
-- 
2.23.0.248.g3a9dd8fb08

← back to recent threads