git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv3 08/10] cat-file: split --batch input lines on whitespace

From
Jeff King <peff@peff.net>
Date
Jul 11, 2013, 20:45 UTC
Message-ID
<20130711204559.GA11399@sigill.intra.peff.net>
In-Reply-To
<20130711113653.GD6015@sigill.intra.peff.net>
On Thu, Jul 11, 2013 at 07:36:53AM -0400, Jeff King wrote:
Show 11 quoted lines
> On Wed, Jul 10, 2013 at 08:59:51PM +0530, Ramkumar Ramachandra wrote:
> 
> > Jeff King wrote:
> > >   git rev-list --objects HEAD |
> > >   git cat-file --batch-check='%(objectsize) %(text)'
> > 
> > If anything, I would have expected %(rest), not %(text).  This atom is
> > specific to commands that accept input via stdin (i.e. not log, f-e-r,
> > branch, or anything else I can think of).
> 
> I considered %(rest) as well. I don't have a strong preference.
Here is the patch re-rolled with s/text/rest/.

Junio, that makes this and 10/10 the only "v3" updates. Let me know if it would be simpler to just resend the whole series.

-- >8 --
Subject: [PATCH] cat-file: split --batch input lines on whitespace

If we get an input line to --batch or --batch-check that looks like "HEAD foo bar", we will currently feed the whole thing to get_sha1(). This means that to use --batch-check with `rev-list --objects`, one must pre-process the input, like:

  git rev-list --objects HEAD |
  cut -d' ' -f1 |
  git cat-file --batch-check

Besides being more typing and slightly less efficient to invoke `cut`, the result loses information: we no longer know which path each object was found at.

This patch teaches cat-file to split input lines at the first whitespace. Everything to the left of the whitespace is considered an object name, and everything to the right is made available as the %(reset) atom. So you can now do:

  git rev-list --objects HEAD |
  git cat-file --batch-check='%(objectsize) %(rest)'
to collect object sizes at particular paths.

Even if %(rest) is not used, we always do the whitespace split (which means you can simply eliminate the `cut` command from the first example above).

This whitespace split is backwards compatible for any reasonable input. Object names cannot contain spaces, so any input with spaces would have resulted in a "missing" line. The only input hurt is if somebody really expected input of the form "HEAD is a fine-looking ref!" to fail; it will now parse HEAD, and make "is a fine-looking ref!" available as %(rest).

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-cat-file.txt | 10 ++++++++--
 builtin/cat-file.c             | 20 +++++++++++++++++++-
 t/t1006-cat-file.sh            |  7 +++++++
 3 files changed, 34 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt
index 06bdc43..68691d4 100644
--- a/Documentation/git-cat-file.txt
+++ b/Documentation/git-cat-file.txt
@@ -88,8 +88,10 @@ from stdin, one per line, and print information about them.
 If `--batch` or `--batch-check` is given, `cat-file` will read objects
 from stdin, one per line, and print information about them.
 
-Each line is considered as a whole object name, and is parsed as if
-given to linkgit:git-rev-parse[1].
+Each line is split at the first whitespace boundary. All characters
+before that whitespace are considered as a whole object name, and are
+parsed as if given to linkgit:git-rev-parse[1]. Characters after that
+whitespace can be accessed using the `%(rest)` atom (see below).
 
 You can specify the information shown for each object by using a custom
 `<format>`. The `<format>` is copied literally to stdout for each
@@ -110,6 +112,10 @@ newline. The available atoms are:
 	The size, in bytes, that the object takes up on disk. See the
 	note about on-disk sizes in the `CAVEATS` section below.
 
+`rest`::
+	The text (if any) found after the first run of whitespace on the
+	input line (i.e., the "rest" of the line).
+
 If no format is specified, the default format is `%(objectname)
 %(objecttype) %(objectsize)`.
 
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 11fa8c0..0e64b41 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -119,6 +119,7 @@ struct expand_data {
 	enum object_type type;
 	unsigned long size;
 	unsigned long disk_size;
+	const char *rest;
 
 	/*
 	 * If mark_query is true, we do not expand anything, but rather
@@ -161,6 +162,9 @@ static void expand_atom(struct strbuf *sb, const char *atom, int len,
 			data->info.disk_sizep = &data->disk_size;
 		else
 			strbuf_addf(sb, "%lu", data->disk_size);
+	} else if (is_atom("rest", atom, len)) {
+		if (!data->mark_query && data->rest)
+			strbuf_addstr(sb, data->rest);
 	} else
 		die("unknown format element: %.*s", len, atom);
 }
@@ -263,7 +267,21 @@ static int batch_objects(struct batch_options *opt)
 	data.mark_query = 0;
 
 	while (strbuf_getline(&buf, stdin, '\n') != EOF) {
-		int error = batch_one_object(buf.buf, opt, &data);
+		char *p;
+		int error;
+
+		/*
+		 * Split at first whitespace, tying off the beginning of the
+		 * string and saving the remainder (or NULL) in data.rest.
+		 */
+		p = strpbrk(buf.buf, " \t");
+		if (p) {
+			while (*p && strchr(" \t", *p))
+				*p++ = '\0';
+		}
+		data.rest = p;
+
+		error = batch_one_object(buf.buf, opt, &data);
 		if (error)
 			return error;
 	}
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 4e911fb..d499d02 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -78,6 +78,13 @@ $content"
 	echo $sha1 | git cat-file --batch-check="%(objecttype) %(objectname)" >actual &&
 	test_cmp expect actual
     '
+
+    test_expect_success '--batch-check with %(rest)' '
+	echo "$type this is some extra content" >expect &&
+	echo "$sha1    this is some extra content" |
+		git cat-file --batch-check="%(objecttype) %(rest)" >actual &&
+	test_cmp expect actual
+    '
 }
 
 hello_content="Hello World"
-- 
1.8.3.rc3.24.gec82cb9
Previous: Junio C HamanoNext: Jeff King
Message 43 of 52 in “cat-file --batch-disk-sizes”
  1. 0/4 cat-file --batch-disk-sizesJeff King, Jul 7, 2013
  2. 1/4 zero-initialize object_info structsJeff King, Jul 7, 2013
  3. Junio C HamanoJul 7, 2013
  4. 2/4 teach sha1_object_info_extended a "disk_size" queryJeff King, Jul 7, 2013
  5. 3/4 cat-file: add --batch-disk-sizes optionJeff King, Jul 7, 2013
  6. Junio C HamanoJul 7, 2013
  7. Jeff KingJul 7, 2013
  8. Duy NguyenJul 8, 2013
  9. Ramkumar RamachandraJul 8, 2013
  10. Duy NguyenJul 8, 2013
  11. Ramkumar RamachandraJul 8, 2013
  12. Duy NguyenJul 9, 2013
  13. Ramkumar RamachandraJul 9, 2013
  14. Jeff KingJul 10, 2013
  15. Junio C HamanoJul 8, 2013
  16. Jeff KingJul 10, 2013
  17. Junio C HamanoJul 11, 2013
  18. brian m. carlsonJul 7, 2013
  19. Jeff KingJul 10, 2013
  20. 4/4 pack-revindex: radix-sort the revindexJeff King, Jul 7, 2013
  21. Shawn PearceJul 7, 2013
  22. Jeff KingJul 8, 2013
  23. Shawn PearceJul 8, 2013
  24. Brandon CaseyJul 8, 2013
  25. Brandon CaseyJul 8, 2013
  26. Jeff KingJul 10, 2013
  27. Jeff KingJul 10, 2013
  28. 00/10 cat-file formats/on-disk sizesJeff King, Jul 10, 2013
  29. 01/10 zero-initialize object_info structsJeff King, Jul 10, 2013
  30. 02/10 teach sha1_object_info_extended a "disk_size" queryJeff King, Jul 10, 2013
  31. 03/10 t1006: modernize output comparisonsJeff King, Jul 10, 2013
  32. 04/10 cat-file: teach --batch to stream blob objectsJeff King, Jul 10, 2013
  33. 05/10 cat-file: refactor --batch option parsingJeff King, Jul 10, 2013
  34. 06/10 cat-file: add --batch-check=<format>Jeff King, Jul 10, 2013
  35. Eric SunshineJul 10, 2013
  36. Ramkumar RamachandraJul 10, 2013
  37. Jeff KingJul 11, 2013
  38. 07/10 cat-file: add %(objectsize:disk) format atomJeff King, Jul 10, 2013
  39. 08/10 cat-file: split --batch input lines on whitespaceJeff King, Jul 10, 2013
  40. Ramkumar RamachandraJul 10, 2013
  41. Jeff KingJul 11, 2013
  42. Junio C HamanoJul 11, 2013
  43. 08/10 cat-file: split --batch input lines on whitespaceJeff King, Jul 11, 2013
  44. 09/10 pack-revindex: use unsigned to store number of objectsJeff King, Jul 10, 2013
  45. 10/10 pack-revindex: radix-sort the revindexJeff King, Jul 10, 2013
  46. Jeff KingJul 10, 2013
  47. Ramkumar RamachandraJul 10, 2013
  48. Jeff KingJul 11, 2013
  49. Brandon CaseyJul 10, 2013
  50. Jeff KingJul 11, 2013
  51. 10/10 pack-revindex: radix-sort the revindexJeff King, Jul 11, 2013
  52. Brandon CaseyJul 11, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.