threads / patch / 44428

patch, 5 partscheck-ref-format: Abolish leak of collapsed refname

Subject: [PATCH 3/5] check-ref-format: Abolish leak of collapsed refname

## tl;dr

18 messages between Nov 4, 2016 and Dec 20, 2016. Diffs are folded; open one to read it.

replies: 17people: 3as markdown or json

Ian Jackson· Nov 4, 2016, 19:13 UTC · lore

[PATCH 0/5] git check-ref-format --stdin --report-errors

I wanted to be able to syntax check lots of proposed refs quickly (please don't ask why - it's complicated!)

So I added a --stdin option to git-check-ref-format. Also it has --report-errors now too so you can get some kind of useful error message if it complains.

It's still not really a good batch mode but it's good enough for my use case. To improve it would involve a new command line option to offer a suitable stdout output format.

There are three small refactoring patches and the two patches with new options and corresponding docs.

Thanks for your attention.

FYI I am not likely to need this again in the near future: it's a one-off use case. So my effort for rework is probably limited. I thought I'd share what I'd done in what I hope is a useful form, anyway.

Regards, Ian.

Ian Jackson· Nov 4, 2016, 19:13 UTC · re: Ian Jackson · lore
collapse_slashes always returns a value from xmallocz.

Right now this leak is not very interesting, since we only call check_one_ref_format once.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 builtin/check-ref-format.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to builtin/check-ref-format.c +3 −1
diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
index f12c19c..020ebe8 100644
--- a/builtin/check-ref-format.c
+++ b/builtin/check-ref-format.c
@@ -63,8 +63,10 @@ static int check_one_ref_format(const char *refname)
 		: check_refname_format(refname, flags);
 	if (got)
 		return 1;
-	if (normalize)
+	if (normalize) {
 		printf("%s\n", refname);
+		free((void*)refname);
+	}
 }
 
 int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
-- 
2.10.1
Michael Haggerty· Dec 19, 2016, 11:09 UTC · re: Ian Jackson · lore

Re: [PATCH 3/5] check-ref-format: Abolish leak of collapsed refname

On 11/04/2016 08:13 PM, Ian Jackson wrote:
Show 18 quoted lines
> collapse_slashes always returns a value from xmallocz.
> 
> Right now this leak is not very interesting, since we only call
> check_one_ref_format once.
> 
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  builtin/check-ref-format.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
> index f12c19c..020ebe8 100644
> --- a/builtin/check-ref-format.c
> +++ b/builtin/check-ref-format.c
> @@ -63,8 +63,10 @@ static int check_one_ref_format(const char *refname)
>  		: check_refname_format(refname, flags);
>  	if (got)
>  		return 1;

If the function returns via the line above, then the memory will still be leaked.

Show 9 quoted lines
> -	if (normalize)
> +	if (normalize) {
>  		printf("%s\n", refname);
> +		free((void*)refname);
> +	}
>  }
>  
>  int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
> 
Michael
Ian Jackson· Nov 4, 2016, 19:13 UTC · re: Ian Jackson · lore

[PATCH 1/5] check-ref-format: Refactor out check_one_ref_format

We are going to want to reuse this.  No functional change right now.
It currently has a hidden memory leak if --normalize is used.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 builtin/check-ref-format.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)
Show changes to builtin/check-ref-format.c +14 −12
diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
index eac4994..4d56caa 100644
--- a/builtin/check-ref-format.c
+++ b/builtin/check-ref-format.c
@@ -48,12 +48,22 @@ static int check_ref_format_branch(const char *arg)
 	return 0;
 }
 
+static int normalize = 0;
+static int flags = 0;
+
+static int check_one_ref_format(const char *refname)
+{
+	if (normalize)
+		refname = collapse_slashes(refname);
+	if (check_refname_format(refname, flags))
+		return 1;
+	if (normalize)
+		printf("%s\n", refname);
+}
+
 int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	int normalize = 0;
-	int flags = 0;
-	const char *refname;
 
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage(builtin_check_ref_format_usage);
@@ -76,13 +86,5 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 	if (! (i == argc - 1))
 		usage(builtin_check_ref_format_usage);
 
-	refname = argv[i];
-	if (normalize)
-		refname = collapse_slashes(refname);
-	if (check_refname_format(refname, flags))
-		return 1;
-	if (normalize)
-		printf("%s\n", refname);
-
-	return 0;
+	return check_one_ref_format(argv[i]);
 }
-- 
2.10.1
Michael Haggerty· Dec 19, 2016, 08:27 UTC · re: Ian Jackson · lore

Re: [PATCH 1/5] check-ref-format: Refactor out check_one_ref_format

On 11/04/2016 08:13 PM, Ian Jackson wrote:
Show 28 quoted lines
> We are going to want to reuse this.  No functional change right now.
> 
> It currently has a hidden memory leak if --normalize is used.
> 
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  builtin/check-ref-format.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
> index eac4994..4d56caa 100644
> --- a/builtin/check-ref-format.c
> +++ b/builtin/check-ref-format.c
> @@ -48,12 +48,22 @@ static int check_ref_format_branch(const char *arg)
>  	return 0;
>  }
>  
> +static int normalize = 0;
> +static int flags = 0;
> +
> +static int check_one_ref_format(const char *refname)
> +{
> +	if (normalize)
> +		refname = collapse_slashes(refname);
> +	if (check_refname_format(refname, flags))
> +		return 1;
> +	if (normalize)
> +		printf("%s\n", refname);
This function needs to `return 0` if it gets to the end.
Show 27 quoted lines
> +}
> +
>  int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  {
>  	int i;
> -	int normalize = 0;
> -	int flags = 0;
> -	const char *refname;
>  
>  	if (argc == 2 && !strcmp(argv[1], "-h"))
>  		usage(builtin_check_ref_format_usage);
> @@ -76,13 +86,5 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  	if (! (i == argc - 1))
>  		usage(builtin_check_ref_format_usage);
>  
> -	refname = argv[i];
> -	if (normalize)
> -		refname = collapse_slashes(refname);
> -	if (check_refname_format(refname, flags))
> -		return 1;
> -	if (normalize)
> -		printf("%s\n", refname);
> -
> -	return 0;
> +	return check_one_ref_format(argv[i]);
>  }
> 
Michael
Ian Jackson· Dec 19, 2016, 13:19 UTC · re: Michael Haggerty · lore

Re: [PATCH 1/5] check-ref-format: Refactor out check_one_ref_format

Michael Haggerty writes ("Re: [PATCH 1/5] check-ref-format: Refactor out check_one_ref_format"):
> On 11/04/2016 08:13 PM, Ian Jackson wrote:
> > +static int check_one_ref_format(const char *refname)
...
> This function needs to `return 0` if it gets to the end.
Indeed it does.  I'm kind of surprised my compiler didn't spot that.
Thanks for the careful review!

Regards, Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Michael Haggerty· Dec 20, 2016, 06:57 UTC · re: Ian Jackson · lore

Re: [PATCH 1/5] check-ref-format: Refactor out check_one_ref_format

On 12/19/2016 02:19 PM, Ian Jackson wrote:
Show 7 quoted lines
> Michael Haggerty writes ("Re: [PATCH 1/5] check-ref-format: Refactor out check_one_ref_format"):
>> On 11/04/2016 08:13 PM, Ian Jackson wrote:
>>> +static int check_one_ref_format(const char *refname)
> ...
>> This function needs to `return 0` if it gets to the end.
> 
> Indeed it does.  I'm kind of surprised my compiler didn't spot that.

Our build system has a `DEVELOPER` option [1] that turns on lots of errors and warnings, and you should turn it on if you haven't already:

    echo DEVELOPER=1 >>config.mak

What exactly it catches depends on what compiler you are using, but it definitely helps if you are using gcc, and I think also if you are using clang.

Michael

[1] https://github.com/git/git/blob/6610af872f6494a061780ec738c8713a034b848b/Documentation/CodingGuidelines#L174-L177

Ian Jackson· Nov 4, 2016, 19:13 UTC · re: Ian Jackson · lore

[PATCH 2/5] check-ref-format: Refactor to make --branch code more common

We are going to want to permit other options with --branch.

So, replace the special case with just an entry for --branch in the parser for ordinary options, and check for option compatibility at the end.

No overall functional change.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 builtin/check-ref-format.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
Show changes to builtin/check-ref-format.c +13 −4
diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
index 4d56caa..f12c19c 100644
--- a/builtin/check-ref-format.c
+++ b/builtin/check-ref-format.c
@@ -49,13 +49,19 @@ static int check_ref_format_branch(const char *arg)
 }
 
 static int normalize = 0;
+static int check_branch = 0;
 static int flags = 0;
 
 static int check_one_ref_format(const char *refname)
 {
+	int got;
+
 	if (normalize)
 		refname = collapse_slashes(refname);
-	if (check_refname_format(refname, flags))
+	got = check_branch
+		? check_ref_format_branch(refname)
+		: check_refname_format(refname, flags);
+	if (got)
 		return 1;
 	if (normalize)
 		printf("%s\n", refname);
@@ -68,9 +74,6 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage(builtin_check_ref_format_usage);
 
-	if (argc == 3 && !strcmp(argv[1], "--branch"))
-		return check_ref_format_branch(argv[2]);
-
 	for (i = 1; i < argc && argv[i][0] == '-'; i++) {
 		if (!strcmp(argv[i], "--normalize") || !strcmp(argv[i], "--print"))
 			normalize = 1;
@@ -80,9 +83,15 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 			flags &= ~REFNAME_ALLOW_ONELEVEL;
 		else if (!strcmp(argv[i], "--refspec-pattern"))
 			flags |= REFNAME_REFSPEC_PATTERN;
+		else if (!strcmp(argv[i], "--branch"))
+			check_branch = 1;
 		else
 			usage(builtin_check_ref_format_usage);
 	}
+
+	if (check_branch && (flags || normalize))
+		usage(builtin_check_ref_format_usage);
+
 	if (! (i == argc - 1))
 		usage(builtin_check_ref_format_usage);
 
-- 
2.10.1
Michael Haggerty· Dec 19, 2016, 11:07 UTC · re: Ian Jackson · lore

Re: [PATCH 2/5] check-ref-format: Refactor to make --branch code more common

On 11/04/2016 08:13 PM, Ian Jackson wrote:
Show 27 quoted lines
> We are going to want to permit other options with --branch.
> 
> So, replace the special case with just an entry for --branch in the
> parser for ordinary options, and check for option compatibility at the
> end.
> 
> No overall functional change.
> 
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  builtin/check-ref-format.c | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
> index 4d56caa..f12c19c 100644
> --- a/builtin/check-ref-format.c
> +++ b/builtin/check-ref-format.c
> @@ -49,13 +49,19 @@ static int check_ref_format_branch(const char *arg)
>  }
>  
>  static int normalize = 0;
> +static int check_branch = 0;
>  static int flags = 0;
>  
>  static int check_one_ref_format(const char *refname)
>  {
> +	int got;

`got` is an unusual name for this variable, and I don't really understand what the word means in this context. Is there a reason not to use the more usual `err`?

Show 32 quoted lines
> +
>  	if (normalize)
>  		refname = collapse_slashes(refname);
> -	if (check_refname_format(refname, flags))
> +	got = check_branch
> +		? check_ref_format_branch(refname)
> +		: check_refname_format(refname, flags);
> +	if (got)
>  		return 1;
>  	if (normalize)
>  		printf("%s\n", refname);
> @@ -68,9 +74,6 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  	if (argc == 2 && !strcmp(argv[1], "-h"))
>  		usage(builtin_check_ref_format_usage);
>  
> -	if (argc == 3 && !strcmp(argv[1], "--branch"))
> -		return check_ref_format_branch(argv[2]);
> -
>  	for (i = 1; i < argc && argv[i][0] == '-'; i++) {
>  		if (!strcmp(argv[i], "--normalize") || !strcmp(argv[i], "--print"))
>  			normalize = 1;
> @@ -80,9 +83,15 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  			flags &= ~REFNAME_ALLOW_ONELEVEL;
>  		else if (!strcmp(argv[i], "--refspec-pattern"))
>  			flags |= REFNAME_REFSPEC_PATTERN;
> +		else if (!strcmp(argv[i], "--branch"))
> +			check_branch = 1;
>  		else
>  			usage(builtin_check_ref_format_usage);
>  	}
> +
> +	if (check_branch && (flags || normalize))

Is there a reason not to allow `--normalize` with `--branch`? (Currently, `git check-ref-format --branch` *does* allow input like `refs/heads/foo`.)

But note that simply allowing `--branch --normalize` without changing `check_one_ref_format()` would mean generating *two* lines of output per reference, so something else would have to change, too.

Show 6 quoted lines
> +		usage(builtin_check_ref_format_usage);
> +
>  	if (! (i == argc - 1))
>  		usage(builtin_check_ref_format_usage);
>  
> 
Michael
Ian Jackson· Dec 19, 2016, 11:54 UTC · re: Michael Haggerty · lore

Re: [PATCH 2/5] check-ref-format: Refactor to make --branch code more common

Michael Haggerty writes ("Re: [PATCH 2/5] check-ref-format: Refactor to make --branch code more common"):
Show 12 quoted lines
> On 11/04/2016 08:13 PM, Ian Jackson wrote:
> >  static int normalize = 0;
> > +static int check_branch = 0;
> >  static int flags = 0;
> >  
> >  static int check_one_ref_format(const char *refname)
> >  {
> > +	int got;
> 
> `got` is an unusual name for this variable, and I don't really
> understand what the word means in this context. Is there a reason not to
> use the more usual `err`?

I have no real opinion about the name of this variable. `err' is a fine name too.

Show 5 quoted lines
> > +	if (check_branch && (flags || normalize))
> 
> Is there a reason not to allow `--normalize` with `--branch`?
> (Currently, `git check-ref-format --branch` *does* allow input like
> `refs/heads/foo`.)

It was like that when I found it :-). I wasn't sure why this restriction was there so I left it alone.

Looking at it again: AFAICT from the documentation --branch is a completely different mode. The effect of --normalize is not to do additional work, but simply to produce additional output. It really means --print-normalized. --branch already prints output, but AFAICT it does not collapse slashes. This seems like a confusing collection of options. But, sorting that out is beyond the scope of what I was trying to do.

In my series I have at least managed not to make any of this any worse, I think: the --stdin option I introduce applies to both modes equally, and doesn't make future improvements to the conflict between --branch and --normalize any harder.

(In _this_ patch, certainly, allowing --normalize with --branch would be wrong, since _this_ patch is just refactoring.)

Thanks, Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Ian Jackson· Nov 4, 2016, 19:13 UTC · re: Ian Jackson · lore

[PATCH 5/5] check-ref-format: New --stdin option

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 Documentation/git-check-ref-format.txt | 10 ++++++++--
 builtin/check-ref-format.c             | 34 +++++++++++++++++++++++++++++++---
 2 files changed, 39 insertions(+), 5 deletions(-)
Show changes to 2 files +39 −5

Documentation/git-check-ref-format.txt, builtin/check-ref-format.c

diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
index e9a2657..5a213ce 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -10,8 +10,9 @@ SYNOPSIS
 [verse]
 'git check-ref-format' [--report-errors] [--normalize]
        [--[no-]allow-onelevel] [--refspec-pattern]
-       <refname>
-'git check-ref-format' [--report-errors] --branch <branchname-shorthand>
+       <refname> | --stdin
+'git check-ref-format' [--report-errors] --branch
+       <branchname-shorthand> | --stdin
 
 DESCRIPTION
 -----------
@@ -109,6 +110,11 @@ OPTIONS
 	If any ref does not check OK, print a message to stderr.
         (By default, git check-ref-format is silent.)
 
+--stdin::
+	Instead of checking on ref supplied on the command line,
+	read refs, one per line, from stdin.  The exit status is
+	0 if all the refs were OK.
+
 
 EXAMPLES
 --------
diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
index 559d5c2..87f52fa 100644
--- a/builtin/check-ref-format.c
+++ b/builtin/check-ref-format.c
@@ -76,6 +76,7 @@ static int check_one_ref_format(const char *refname)
 int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 {
 	int i;
+	int use_stdin = 0;
 
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage(builtin_check_ref_format_usage);
@@ -93,6 +94,8 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 			check_branch = 1;
 		else if (!strcmp(argv[i], "--report-errors"))
 			report_errors = 1;
+		else if (!strcmp(argv[i], "--stdin"))
+			use_stdin = 1;
 		else
 			usage(builtin_check_ref_format_usage);
 	}
@@ -100,8 +103,33 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 	if (check_branch && (flags || normalize))
 		usage(builtin_check_ref_format_usage);
 
-	if (! (i == argc - 1))
-		usage(builtin_check_ref_format_usage);
+	if (!use_stdin) {
+		if (! (i == argc - 1))
+			usage(builtin_check_ref_format_usage);
+
+		return check_one_ref_format(argv[i]);
+	} else {
+		char buffer[2048];
+		int worst = 0;
 
-	return check_one_ref_format(argv[i]);
+		if (! (i == argc))
+			usage(builtin_check_ref_format_usage);
+
+		while (fgets(buffer, sizeof(buffer), stdin)) {
+			char *newline = strchr(buffer, '\n');
+			if (!newline) {
+				fprintf(stderr, "%s --stdin: missing final newline or line too long\n", *argv);
+				exit(127);
+			}
+			*newline = 0;
+			int got = check_one_ref_format(buffer);
+			if (got > worst)
+				worst = got;
+		}
+		if (!feof(stdin)) {
+			perror("reading from stdin");
+			exit(127);
+		}
+		return worst;
+	}
 }
-- 
2.10.1
Michael Haggerty· Dec 19, 2016, 11:22 UTC · re: Ian Jackson · lore

Re: [PATCH 5/5] check-ref-format: New --stdin option

On 11/04/2016 08:13 PM, Ian Jackson wrote:
Show 64 quoted lines
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  Documentation/git-check-ref-format.txt | 10 ++++++++--
>  builtin/check-ref-format.c             | 34 +++++++++++++++++++++++++++++++---
>  2 files changed, 39 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
> index e9a2657..5a213ce 100644
> --- a/Documentation/git-check-ref-format.txt
> +++ b/Documentation/git-check-ref-format.txt
> @@ -10,8 +10,9 @@ SYNOPSIS
>  [verse]
>  'git check-ref-format' [--report-errors] [--normalize]
>         [--[no-]allow-onelevel] [--refspec-pattern]
> -       <refname>
> -'git check-ref-format' [--report-errors] --branch <branchname-shorthand>
> +       <refname> | --stdin
> +'git check-ref-format' [--report-errors] --branch
> +       <branchname-shorthand> | --stdin
>  
>  DESCRIPTION
>  -----------
> @@ -109,6 +110,11 @@ OPTIONS
>  	If any ref does not check OK, print a message to stderr.
>          (By default, git check-ref-format is silent.)
>  
> +--stdin::
> +	Instead of checking on ref supplied on the command line,
> +	read refs, one per line, from stdin.  The exit status is
> +	0 if all the refs were OK.
> +
>  
>  EXAMPLES
>  --------
> diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
> index 559d5c2..87f52fa 100644
> --- a/builtin/check-ref-format.c
> +++ b/builtin/check-ref-format.c
> @@ -76,6 +76,7 @@ static int check_one_ref_format(const char *refname)
>  int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  {
>  	int i;
> +	int use_stdin = 0;
>  
>  	if (argc == 2 && !strcmp(argv[1], "-h"))
>  		usage(builtin_check_ref_format_usage);
> @@ -93,6 +94,8 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  			check_branch = 1;
>  		else if (!strcmp(argv[i], "--report-errors"))
>  			report_errors = 1;
> +		else if (!strcmp(argv[i], "--stdin"))
> +			use_stdin = 1;
>  		else
>  			usage(builtin_check_ref_format_usage);
>  	}
> @@ -100,8 +103,33 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  	if (check_branch && (flags || normalize))
>  		usage(builtin_check_ref_format_usage);
>  
> -	if (! (i == argc - 1))
> -		usage(builtin_check_ref_format_usage);
> +	if (!use_stdin) {
> +		if (! (i == argc - 1))
> +			usage(builtin_check_ref_format_usage);

Given the changes that you made to support `--stdin`, it would be pretty easy to support multiple command line arguments, now, too. (But this needn't be part of your patch series.)

Show 11 quoted lines
> +
> +		return check_one_ref_format(argv[i]);
> +	} else {
> +		char buffer[2048];
> +		int worst = 0;
>  
> -	return check_one_ref_format(argv[i]);
> +		if (! (i == argc))
> +			usage(builtin_check_ref_format_usage);
> +
> +		while (fgets(buffer, sizeof(buffer), stdin)) {

`strbuf_getline()` would make this a lot easier and also eliminate the need to specify a buffer size.

Show 18 quoted lines
> +			char *newline = strchr(buffer, '\n');
> +			if (!newline) {
> +				fprintf(stderr, "%s --stdin: missing final newline or line too long\n", *argv);
> +				exit(127);
> +			}
> +			*newline = 0;
> +			int got = check_one_ref_format(buffer);
> +			if (got > worst)
> +				worst = got;
> +		}
> +		if (!feof(stdin)) {
> +			perror("reading from stdin");
> +			exit(127);
> +		}
> +		return worst;
> +	}
>  }
> 
Michael
Michael Haggerty· Dec 19, 2016, 13:43 UTC · re: Ian Jackson · lore

Re: [PATCH 5/5] check-ref-format: New --stdin option

On 11/04/2016 08:13 PM, Ian Jackson wrote:
Show 55 quoted lines
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  Documentation/git-check-ref-format.txt | 10 ++++++++--
>  builtin/check-ref-format.c             | 34 +++++++++++++++++++++++++++++++---
>  2 files changed, 39 insertions(+), 5 deletions(-)
> 
> [...]
> diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
> index 559d5c2..87f52fa 100644
> --- a/builtin/check-ref-format.c
> +++ b/builtin/check-ref-format.c
> @@ -76,6 +76,7 @@ static int check_one_ref_format(const char *refname)
>  int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  {
>  	int i;
> +	int use_stdin = 0;
>  
>  	if (argc == 2 && !strcmp(argv[1], "-h"))
>  		usage(builtin_check_ref_format_usage);
> @@ -93,6 +94,8 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  			check_branch = 1;
>  		else if (!strcmp(argv[i], "--report-errors"))
>  			report_errors = 1;
> +		else if (!strcmp(argv[i], "--stdin"))
> +			use_stdin = 1;
>  		else
>  			usage(builtin_check_ref_format_usage);
>  	}
> @@ -100,8 +103,33 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
>  	if (check_branch && (flags || normalize))
>  		usage(builtin_check_ref_format_usage);
>  
> -	if (! (i == argc - 1))
> -		usage(builtin_check_ref_format_usage);
> +	if (!use_stdin) {
> +		if (! (i == argc - 1))
> +			usage(builtin_check_ref_format_usage);
> +
> +		return check_one_ref_format(argv[i]);
> +	} else {
> +		char buffer[2048];
> +		int worst = 0;
>  
> -	return check_one_ref_format(argv[i]);
> +		if (! (i == argc))
> +			usage(builtin_check_ref_format_usage);
> +
> +		while (fgets(buffer, sizeof(buffer), stdin)) {
> +			char *newline = strchr(buffer, '\n');
> +			if (!newline) {
> +				fprintf(stderr, "%s --stdin: missing final newline or line too long\n", *argv);
> +				exit(127);
> +			}
> +			*newline = 0;
> +			int got = check_one_ref_format(buffer);

Another minor point: project policy is not to mix declarations and code. Please declare `got` at the top of the block.

Show 11 quoted lines
> +			if (got > worst)
> +				worst = got;
> +		}
> +		if (!feof(stdin)) {
> +			perror("reading from stdin");
> +			exit(127);
> +		}
> +		return worst;
> +	}
>  }
> 
Michael
Ian Jackson· Nov 4, 2016, 19:13 UTC · re: Ian Jackson · lore

[PATCH 4/5] check-ref-format: New --report-errors option

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 Documentation/git-check-ref-format.txt |  8 ++++++--
 builtin/check-ref-format.c             | 10 ++++++++--
 2 files changed, 14 insertions(+), 4 deletions(-)
Show changes to 2 files +14 −4

Documentation/git-check-ref-format.txt, builtin/check-ref-format.c

diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
index 8611a99..e9a2657 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -8,10 +8,10 @@ git-check-ref-format - Ensures that a reference name is well formed
 SYNOPSIS
 --------
 [verse]
-'git check-ref-format' [--normalize]
+'git check-ref-format' [--report-errors] [--normalize]
        [--[no-]allow-onelevel] [--refspec-pattern]
        <refname>
-'git check-ref-format' --branch <branchname-shorthand>
+'git check-ref-format' [--report-errors] --branch <branchname-shorthand>
 
 DESCRIPTION
 -----------
@@ -105,6 +105,10 @@ OPTIONS
 	with a status of 0.  (`--print` is a deprecated way to spell
 	`--normalize`.)
 
+--report-errors::
+	If any ref does not check OK, print a message to stderr.
+        (By default, git check-ref-format is silent.)
+
 
 EXAMPLES
 --------
diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
index 020ebe8..559d5c2 100644
--- a/builtin/check-ref-format.c
+++ b/builtin/check-ref-format.c
@@ -9,7 +9,7 @@
 
 static const char builtin_check_ref_format_usage[] =
 "git check-ref-format [--normalize] [<options>] <refname>\n"
-"   or: git check-ref-format --branch <branchname-shorthand>";
+"   or: git check-ref-format [<options>] --branch <branchname-shorthand>";
 
 /*
  * Return a copy of refname but with leading slashes removed and runs
@@ -51,6 +51,7 @@ static int check_ref_format_branch(const char *arg)
 static int normalize = 0;
 static int check_branch = 0;
 static int flags = 0;
+static int report_errors = 0;
 
 static int check_one_ref_format(const char *refname)
 {
@@ -61,8 +62,11 @@ static int check_one_ref_format(const char *refname)
 	got = check_branch
 		? check_ref_format_branch(refname)
 		: check_refname_format(refname, flags);
-	if (got)
+	if (got) {
+		if (report_errors)
+			fprintf(stderr, "bad ref format: %s\n", refname);
 		return 1;
+	}
 	if (normalize) {
 		printf("%s\n", refname);
 		free((void*)refname);
@@ -87,6 +91,8 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 			flags |= REFNAME_REFSPEC_PATTERN;
 		else if (!strcmp(argv[i], "--branch"))
 			check_branch = 1;
+		else if (!strcmp(argv[i], "--report-errors"))
+			report_errors = 1;
 		else
 			usage(builtin_check_ref_format_usage);
 	}
-- 
2.10.1
Michael Haggerty· Dec 19, 2016, 11:29 UTC · re: Ian Jackson · lore

Re: [PATCH 0/5] git check-ref-format --stdin --report-errors

On 11/04/2016 08:13 PM, Ian Jackson wrote:
Show 20 quoted lines
> I wanted to be able to syntax check lots of proposed refs quickly
> (please don't ask why - it's complicated!)
> 
> So I added a --stdin option to git-check-ref-format.  Also it has
> --report-errors now too so you can get some kind of useful error
> message if it complains.
> 
> It's still not really a good batch mode but it's good enough for my
> use case.  To improve it would involve a new command line option to
> offer a suitable stdout output format.
> 
> There are three small refactoring patches and the two patches with new
> options and corresponding docs.
> 
> Thanks for your attention.
> 
> FYI I am not likely to need this again in the near future: it's a
> one-off use case.  So my effort for rework is probably limited.  I
> thought I'd share what I'd done in what I hope is a useful form,
> anyway.
Thanks for your patches. I left some comments about the individual patches.

I don't know whether this feature will be popular, but it's not a lot of code to add it, so it would be OK with me.

Especially given that the output is not especially machine-readable, it might be more consistent with other commands to call the new feature `--verbose` rather than `--report-errors`.

If it is thought likely that scripts will want to leave a pipe open to this command and feed it one query at a time, then it would be helpful to flush stdout after each reference's result is written. If the opposite use case is common (mass processing of refnames), we could always add a `--buffer` option like the one that `git cat-file --batch` has.

Michael
Ian Jackson· Dec 19, 2016, 16:22 UTC · re: Michael Haggerty · lore

Re: [PATCH 0/5] git check-ref-format --stdin --report-errors

Michael Haggerty writes ("Re: [PATCH 0/5] git check-ref-format --stdin --report-errors"):
> Thanks for your patches. I left some comments about the individual patches.
Thanks for your review.
> I don't know whether this feature will be popular, but it's not a lot of
> code to add it, so it would be OK with me.
Great.
> Especially given that the output is not especially machine-readable, it
> might be more consistent with other commands to call the new feature
> `--verbose` rather than `--report-errors`.
Sure.
Show 5 quoted lines
> If it is thought likely that scripts will want to leave a pipe open to
> this command and feed it one query at a time, then it would be helpful
> to flush stdout after each reference's result is written. If the
> opposite use case is common (mass processing of refnames), we could
> always add a `--buffer` option like the one that `git cat-file --batch` has.

I think it should be unbuffered by default, so I will make that change, along with the fixes from your other mails, and resubmit.

Regards, Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Junio C Hamano· Dec 19, 2016, 18:23 UTC · re: Michael Haggerty · lore

Re: [PATCH 0/5] git check-ref-format --stdin --report-errors

Michael Haggerty <mhagger@alum.mit.edu> writes:
> Especially given that the output is not especially machine-readable, it
> might be more consistent with other commands to call the new feature
> `--verbose` rather than `--report-errors`.

Don't we instead want to structure the output to be machine-readable instead, given that check-ref-format is a very low level plumbing command that is primarily meant for scriptors?

Michael Haggerty· Dec 20, 2016, 07:29 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/5] git check-ref-format --stdin --report-errors

On 12/19/2016 07:23 PM, Junio C Hamano wrote:
Show 9 quoted lines
> Michael Haggerty <mhagger@alum.mit.edu> writes:
> 
>> Especially given that the output is not especially machine-readable, it
>> might be more consistent with other commands to call the new feature
>> `--verbose` rather than `--report-errors`.
> 
> Don't we instead want to structure the output to be machine-readable
> instead, given that check-ref-format is a very low level plumbing
> command that is primarily meant for scriptors?

Of course that would be the ideal. Let's think about what it would look like. Given that the very purpose of the program is to decide whether its inputs are reasonable reference names or not, it would be important to make it bulletproof:

* It could be fed some ugly garbage
* It could be used for security-relevant checks

One obvious choice would be to use NUL separators, but that would make the output mostly unreadable to humans.

Another would be to use LF to terminate each line of output, like
    ok TAB refs/heads/foo LF
    bad TAB refs/heads/bad SP name@@.lock LF

For the LF-terminated `--stdin` input, this should be unambiguous. However, it wouldn't necessarily work for arguments passed in via the command line, for for slight variations on `--stdin` like if we were to add a `-z` option to allow the input to be NUL-terminated.

The 100% solution would probably be to support language-specific quoting, like the `--shell`/`--perl`/`--python`/`--tcl` options accepted by `for-each-ref`, probably with a fifth option for NUL-terminated output. And it should probably also support a `-z` option to make its input NUL-separated. Pretty much all of the infrastructure is already there in `quote.h` and `quote.c`, and the option-parsing could be cribbed from `builtin/for-each-ref.c`, so it wouldn't even be *that* much work to implement.

Michael

← back to recent threads