Re: --batch or some --paths-file for very long lists of paths
- From
- Phillip Wood <phillip.wood123@gmail.com>
- Date
- Aug 8, 2022, 13:12 UTC
- Message-ID
- <47bfbdb5-c907-6a35-6662-72a446869227@gmail.com>
- In-Reply-To
- <YuxPNFtTBmkZkiCW@lena.dartmouth.edu>
Hi Yaroslav
On 04/08/2022 23:59, Yaroslav Halchenko wrote:
Show 13 quoted lines
> Dear Git Gurus, > > In DataLad (https://datalad.org) we are doing "our own" analysis of what > specific files (not entire directories) should git and git-annex operate > on. Obviously, in large repositories (and we have with >100k files) > that might require invoking git add or git diff etc with a long list > of paths specified in the command line. For that we often split > invocation into multiple and even resort to git commit --amend to > combine multiple commits then into a single one. > > But I wondered if may be there is already some trick to make such > commands as status, diff, add, commit to operate on arbitrarily long > list of paths passed to that git command somehow.
A number of porcelain commands have a --pathspec-from-file option that takes a file with a list of pathspecs or reads them from stdin. When combined with --pathspec-file-nul this handles paths containing newline correctly or you can quote them without this option. You can pass --literal-pathspecs if you have a list of paths rather than pathspecs.
At the plumbing level you can use "update-index" to add/delete/update paths in the index which will read paths from stdin and "checkout-index" will also read paths from stdin.
The diff family do not have any support for --pathspec-from-file at the moment but I'd be happy to see someone implement it (I think it would be fairly straight forward).
Show 18 quoted lines
> Note that gitglossary (at least in 2.35.1 git on debian) says that > > Pathspecs are used on the command line of "git ls-files", "git > ls-tree", "git add", "git grep", "git diff", "git checkout", and many other > commands ... > > but > > $> git ls-tree -h | head -n1 > usage: git ls-tree [<options>] <tree-ish> [<path>...] > > so it is <path> not the <pathspec> like (why in stderr this time?) > > $> git commit -h 2>&1 | head -n1 > usage: git commit [<options>] [--] <pathspec>... > > So if in both cases it is pathspec, may be pathspec could support some > other magical keyword like :(filelist)/tmp/mylonglistofpaths ?
I like that path magic idea, but as we already have --pathspec-from-file I think we'd be better improving support for that.
Best Wishes
Phillip
> Thanks in advance for your time and thoughts,