From: Phillip Wood <phillip.wood123@gmail.com>
Date: Mon, 08 Aug 2022 13:12:56 GMT
Subject: Re: --batch or some --paths-file for very long lists of paths
Message-ID: <47bfbdb5-c907-6a35-6662-72a446869227@gmail.com>
In-Reply-To: <YuxPNFtTBmkZkiCW@lena.dartmouth.edu>

Hi Yaroslav

On 04/08/2022 23:59, Yaroslav Halchenko wrote:
> Dear Git Gurus,
> 
> In DataLad (https://datalad.org) we are doing "our own" analysis of what
> specific files (not entire directories) should git and git-annex operate
> on.  Obviously, in large repositories (and we have with >100k files)
> that might require invoking  git add  or  git diff  etc with a long list
> of paths specified in the command line.  For that we often split
> invocation into multiple and even resort to   git commit --amend  to
> combine multiple commits then into a single one.
> 
> But I wondered if may be there is already some trick to make such
> commands as   status, diff, add, commit   to operate on arbitrarily long
> list of paths passed to that git command somehow.

A number of porcelain commands have a --pathspec-from-file option that 
takes a file with a list of pathspecs or reads them from stdin. When 
combined with --pathspec-file-nul this handles paths containing newline 
correctly or you can quote them without this option. You can pass 
--literal-pathspecs if you have a list of paths rather than pathspecs.

At the plumbing level you can use "update-index" to add/delete/update 
paths in the index which will read paths from stdin and "checkout-index" 
will also read paths from stdin.

The diff family do not have any support for --pathspec-from-file at the 
moment but I'd be happy to see someone implement it (I think it would be 
fairly straight forward).

> Note that gitglossary (at least in 2.35.1 git on debian) says that
> 
> 	Pathspecs are used on the command line of "git ls-files", "git
>      ls-tree", "git add", "git grep", "git diff", "git checkout", and many other
>      commands ...
> 
> but
> 
> 	$> git ls-tree -h | head -n1
> 	usage: git ls-tree [<options>] <tree-ish> [<path>...]
> 
> so it is <path> not the <pathspec> like (why in stderr this time?)
> 
> 	$> git commit -h 2>&1 | head -n1
> 	usage: git commit [<options>] [--] <pathspec>...
> 
> So if in both cases it is pathspec, may be pathspec could support some
> other magical keyword like :(filelist)/tmp/mylonglistofpaths ?

I like that path magic idea, but as we already have --pathspec-from-file 
I think we'd be better improving support for that.

Best Wishes

Phillip

> Thanks in advance for your time and thoughts,