git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/2] cat-file: default to --buffer when --batch-all-objects is used

From
Jeff King <peff@peff.net>
Date
May 18, 2016, 16:56 UTC
Message-ID
<20160518165613.GB5607@sigill.intra.peff.net>
In-Reply-To
<20160518165411.GA5563@sigill.intra.peff.net>

Traditionally cat-file's batch-mode does not do any output buffering. The reason is that a caller may have pipes connected to its input and output, and would want to use cat-file interactively, getting output immediately for each input it sends.

This may involve a lot of small write() calls, which can be slow. So we introduced --buffer to improve this, but we can't turn it on by default, as it would break the interactive case above.

However, when --batch-all-objects is used, we do not read stdin at all. We generate the output ourselves as quickly as possible, and then exit. In this case buffering is a strict win, and it is simply a hassle for the user to have to remember to specify --buffer.

This patch makes --buffer the default when --batch-all-objects is used. Specifying "--buffer" manually is still OK, and you can even override it with "--no-buffer" if you're a masochist (or debugging).

For some real numbers, running:
  git cat-file --batch-all-objects --batch-check='%(objectname)'
on torvalds/linux goes from:
  real    0m1.464s
  user    0m1.208s
  sys     0m0.252s
to:
  real    0m1.230s
  user    0m1.172s
  sys     0m0.056s
for a 16% speedup.
Suggested-by: Charles Bailey <charles@hashpling.org>
Signed-off-by: Jeff King <peff@peff.net>
---
 builtin/cat-file.c | 4 ++++
 1 file changed, 4 insertions(+)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 5d3180e..618103f 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -504,6 +504,7 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)
 
 	git_config(git_cat_file_config, NULL);
 
+	batch.buffer_output = -1;
 	argc = parse_options(argc, argv, prefix, options, cat_file_usage, 0);
 
 	if (opt) {
@@ -527,6 +528,9 @@ int cmd_cat_file(int argc, const char **argv, const char *prefix)
 		usage_with_options(cat_file_usage, options);
 	}
 
+	if (batch.buffer_output < 0)
+		batch.buffer_output = batch.all_objects;
+
 	if (batch.enabled)
 		return batch_objects(&batch);
 
-- 
2.8.2.888.gecb1fe3
Previous: Jeff King
Message 4 of 4 in “minor cat-file optimizations”
  1. 0/2 minor cat-file optimizationsJeff King, May 18, 2016
  2. Jeff KingMay 18, 2016
  3. 1/2 cat-file: avoid noop calls to sha1_object_info_extendedJeff King, May 18, 2016
  4. 2/2 cat-file: default to --buffer when --batch-all-objects is usedJeff King, May 18, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.