threads / patch / 3499

patchTeach git-checkout-index to read filenames from stdin.

Subject: [PATCH] Teach git-checkout-index to read filenames from stdin.

## tl;dr

5 messages between Mar 1, 2006 and Mar 2, 2006. Diffs are folded; open one to read it.

replies: 4people: 3as markdown or json

Shawn Pearce· Mar 1, 2006, 02:43 UTC · lore

Since git-checkout-index is often used from scripts which may have a stream of filenames they wish to checkout it is more convenient to use --stdin than xargs. On platforms where fork performance is currently sub-optimal and the length of a command line is limited (*cough* Cygwin *cough*) running a single git-checkout-index process for a large number of files beats spawning it multiple times from xargs.

File names are still accepted on the command line if --stdin is not supplied. Nothing is performed if no files are supplied on the command line or by stdin.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 I wonder if Linus will stop using
   find ... -print0 | xargs -0 git-checkout-index --
 with this option available.
 Eh, probably not as xargs requires less typing than -z --stdin.
 Documentation/git-checkout-index.txt |   20 +++++++++++++++--
 checkout-index.c                     |   41 ++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 2 deletions(-)

base 84434f9549d56e522a2eb4de370100f0a6e5e041 last df23c1119d0af1fbac6b8afd296113e155d9a878

Show changes to 2 files +59 −2

Documentation/git-checkout-index.txt, checkout-index.c

diff --git a/Documentation/git-checkout-index.txt b/Documentation/git-checkout-index.txt
index 2a1e526..b0b6588 100644
--- a/Documentation/git-checkout-index.txt
+++ b/Documentation/git-checkout-index.txt
@@ -10,7 +10,9 @@ SYNOPSIS
 --------
 [verse]
 'git-checkout-index' [-u] [-q] [-a] [-f] [-n] [--prefix=<string>]
-		   [--stage=<number>] [--] <file>...
+		   [--stage=<number>]
+		   [-z] [--stdin]
+		   [--] [<file>]\*
 
 DESCRIPTION
 -----------
@@ -45,6 +47,15 @@ OPTIONS
 	Instead of checking out unmerged entries, copy out the
 	files from named stage.  <number> must be between 1 and 3.
 
+--stdin::
+	Instead of taking list of paths from the command line,
+	read list of paths from the standard input.  Paths are
+	separated by LF (i.e. one path per line) by default.
+
+-z::
+	Only meaningful with `--stdin`; paths are separated with
+	NUL character instead of LF.
+
 --::
 	Do not interpret any more arguments as options.
 
@@ -64,7 +75,12 @@ $ find . -name '*.h' -print0 | xargs -0 
 
 which will force all existing `*.h` files to be replaced with their
 cached copies. If an empty command line implied "all", then this would
-force-refresh everything in the index, which was not the point.
+force-refresh everything in the index, which was not the point.  But
+since git-checkout-index accepts --stdin it would be faster to use:
+
+----------------
+$ find . -name '*.h' -print0 | git-checkout-index -f -z --stdin
+----------------
 
 The `--` is just a good idea when you know the rest will be filenames;
 it will prevent problems with a filename of, for example,  `-a`.
diff --git a/checkout-index.c b/checkout-index.c
index 957b4a8..f54c606 100644
--- a/checkout-index.c
+++ b/checkout-index.c
@@ -22,6 +22,10 @@
  *
  *	find . -name '*.h' -print0 | xargs -0 git-checkout-index -f --
  *
+ * or:
+ *
+ *	find . -name '*.h' -print0 | git-checkout-index -f -z --stdin
+ *
  * which will force all existing *.h files to be replaced with
  * their cached copies. If an empty command line implied "all",
  * then this would force-refresh everything in the cache, which
@@ -33,6 +37,8 @@
  * but get used to it in scripting!).
  */
 #include "cache.h"
+#include "strbuf.h"
+#include "quote.h"
 
 static const char *prefix;
 static int prefix_length;
@@ -114,6 +120,8 @@ int main(int argc, char **argv)
 	int i;
 	int newfd = -1;
 	int all = 0;
+	int read_from_stdin = 0;
+	int line_termination = '\n';
 
 	prefix = setup_git_directory();
 	git_config(git_default_config);
@@ -156,6 +164,17 @@ int main(int argc, char **argv)
 				die("cannot open index.lock file.");
 			continue;
 		}
+		if (!strcmp(arg, "-z")) {
+			line_termination = 0;
+			continue;
+		}
+		if (!strcmp(arg, "--stdin")) {
+			if (i != argc - 1)
+				die("--stdin must be at the end");
+			read_from_stdin = 1;
+			i++; /* do not consider arg as a file name */
+			break;
+		}
 		if (!strncmp(arg, "--prefix=", 9)) {
 			state.base_dir = arg+9;
 			state.base_dir_len = strlen(state.base_dir);
@@ -191,9 +210,31 @@ int main(int argc, char **argv)
 
 		if (all)
 			die("git-checkout-index: don't mix '--all' and explicit filenames");
+		if (read_from_stdin)
+			die("git-checkout-index: don't mix '--stdin' and explicit filenames");
 		checkout_file(prefix_path(prefix, prefix_length, arg));
 	}
 
+	if (read_from_stdin) {
+		struct strbuf buf;
+		if (all)
+			die("git-checkout-index: don't mix '--all' and '--stdin'");
+		strbuf_init(&buf);
+		while (1) {
+			char *path_name;
+			read_line(&buf, stdin, line_termination);
+			if (buf.eof)
+				break;
+			if (line_termination && buf.buf[0] == '"')
+				path_name = unquote_c_style(buf.buf, NULL);
+			else
+				path_name = buf.buf;
+			checkout_file(prefix_path(prefix, prefix_length, path_name));
+			if (path_name != buf.buf)
+				free(path_name);
+		}
+	}
+
 	if (all)
 		checkout_all();
 
-- 
1.2.3.gdf23c
Christopher Faylor· Mar 1, 2006, 15:50 UTC · re: Shawn Pearce · lore

Re: [PATCH] Teach git-checkout-index to read filenames from stdin.

On Tue, Feb 28, 2006 at 09:43:33PM -0500, Shawn Pearce wrote:
Show 5 quoted lines
>Since git-checkout-index is often used from scripts which may have a
>stream of filenames they wish to checkout it is more convenient to use
>--stdin than xargs.  On platforms where fork performance is currently
>sub-optimal and the length of a command line is limited (*cough* Cygwin
>*cough*)

AFAIK, the length of the command line for cygwin apps is very large -- if you're using recent versions of Cygwin. I believe that it is longer than the linux default. We bypass the Windows mechanism for setting the command line when a cygwin program starts a cygwin program.

For native Windows programs, the command line length is ~32K but I don't think that git uses any native Windows programs, does it?

cgf
Shawn Pearce· Mar 1, 2006, 17:56 UTC · re: Christopher Faylor · lore

Re: [PATCH] Teach git-checkout-index to read filenames from stdin.

Christopher Faylor <me@cgf.cx> wrote:
Show 7 quoted lines
> AFAIK, the length of the command line for cygwin apps is very large --
> if you're using recent versions of Cygwin.  I believe that it is longer
> than the linux default.  We bypass the Windows mechanism for setting the
> command line when a cygwin program starts a cygwin program.
> 
> For native Windows programs, the command line length is ~32K but I don't
> think that git uses any native Windows programs, does it?

No. Currently GIT is entirely dependent on Cygwin. So GIT wouldn't bump into the ~32K limit due to the cygwin-cygwin feature you mention. But thanks for the information. I had thought I had read somewhere in the Cygwin documentation that the command line length was rather limited (even under cygwin-cygwin calls). Maybe I was just seeing things. :-)

But even if we can get a long set of args into git-checkout-index its probably still better to stream them as you can get both programs working at the same time (rather than waiting for xargs to build the argument buffer) and you are saving yourself at least one fork as you don't need to start xargs just to feed git-checkout-index. Even on Linux where fork is cheap, that's still soemething saved.

-- 
Shawn.
Alex Riesen· Mar 2, 2006, 13:12 UTC · re: Christopher Faylor · lore

Re: [PATCH] Teach git-checkout-index to read filenames from stdin.

On 3/1/06, Christopher Faylor <me@cgf.cx> wrote:
>
> For native Windows programs, the command line length is ~32K but I don't
> think that git uses any native Windows programs, does it?
>
Yes, ActiveState Perl is as native as it gets.
Alex Riesen· Mar 2, 2006, 13:14 UTC · re: Alex Riesen · lore

Re: [PATCH] Teach git-checkout-index to read filenames from stdin.

On 3/2/06, Alex Riesen <raa.lkml@gmail.com> wrote:
Show 8 quoted lines
> On 3/1/06, Christopher Faylor <me@cgf.cx> wrote:
> >
> > For native Windows programs, the command line length is ~32K but I don't
> > think that git uses any native Windows programs, does it?
> >
>
> Yes, ActiveState Perl is as native as it gets.
>

It does use native windows programs, I mean. At least for me it does.

← back to recent threads