{"thread":{"id":"58268","subject":"--batch or some --paths-file for very long lists of paths","startedAt":"2022-08-04T23:00:22Z","lastAt":"2022-08-08T13:13:07Z","messageCount":2,"participants":["Yaroslav Halchenko","Phillip Wood"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"460682","messageId":"YuxPNFtTBmkZkiCW@lena.dartmouth.edu","threadId":"58268","inReplyTo":null,"subject":"--batch or some --paths-file for very long lists of paths","fromName":"Yaroslav Halchenko","fromEmail":"yoh@onerussian.com","sentAt":"2022-08-04T22:59:00Z","receivedAt":"2022-08-04T23:00:22Z","isPatch":false,"sender":{"key":"yoh@onerussian.com","avatar":"https://gravatar.com/avatar/8901b82415ae451a83aea49409708912726e53620e3ac92320bf1f86548d97e9?d=mp&s=160"},"body":"Dear Git Gurus,\n\nIn DataLad (https://datalad.org) we are doing \"our own\" analysis of what\nspecific files (not entire directories) should git and git-annex operate\non.  Obviously, in large repositories (and we have with >100k files)\nthat might require invoking  git add  or  git diff  etc with a long list\nof paths specified in the command line.  For that we often split\ninvocation into multiple and even resort to   git commit --amend  to\ncombine multiple commits then into a single one.\n\nBut I wondered if may be there is already some trick to make such\ncommands as   status, diff, add, commit   to operate on arbitrarily long\nlist of paths passed to that git command somehow.\n\nNote that gitglossary (at least in 2.35.1 git on debian) says that \n\n\tPathspecs are used on the command line of \"git ls-files\", \"git\n    ls-tree\", \"git add\", \"git grep\", \"git diff\", \"git checkout\", and many other\n    commands ...\n\nbut \n\n\t$> git ls-tree -h | head -n1\n\tusage: git ls-tree [<options>] <tree-ish> [<path>...]\n\nso it is <path> not the <pathspec> like (why in stderr this time?)\n\n\t$> git commit -h 2>&1 | head -n1\n\tusage: git commit [<options>] [--] <pathspec>...\n\nSo if in both cases it is pathspec, may be pathspec could support some\nother magical keyword like :(filelist)/tmp/mylonglistofpaths ?\n\nThanks in advance for your time and thoughts,\n-- \nYaroslav O. Halchenko\nCenter for Open Neuroscience     http://centerforopenneuroscience.org\nDartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755\nWWW:   http://www.linkedin.com/in/yarik        \n\n"},{"id":"460816","messageId":"47bfbdb5-c907-6a35-6662-72a446869227@gmail.com","threadId":"58268","inReplyTo":"YuxPNFtTBmkZkiCW@lena.dartmouth.edu","subject":"Re: --batch or some --paths-file for very long lists of paths","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2022-08-08T13:12:56Z","receivedAt":"2022-08-08T13:13:07Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Yaroslav\n\nOn 04/08/2022 23:59, Yaroslav Halchenko wrote:\n> Dear Git Gurus,\n> \n> In DataLad (https://datalad.org) we are doing \"our own\" analysis of what\n> specific files (not entire directories) should git and git-annex operate\n> on.  Obviously, in large repositories (and we have with >100k files)\n> that might require invoking  git add  or  git diff  etc with a long list\n> of paths specified in the command line.  For that we often split\n> invocation into multiple and even resort to   git commit --amend  to\n> combine multiple commits then into a single one.\n> \n> But I wondered if may be there is already some trick to make such\n> commands as   status, diff, add, commit   to operate on arbitrarily long\n> list of paths passed to that git command somehow.\n\nA number of porcelain commands have a --pathspec-from-file option that \ntakes a file with a list of pathspecs or reads them from stdin. When \ncombined with --pathspec-file-nul this handles paths containing newline \ncorrectly or you can quote them without this option. You can pass \n--literal-pathspecs if you have a list of paths rather than pathspecs.\n\nAt the plumbing level you can use \"update-index\" to add/delete/update \npaths in the index which will read paths from stdin and \"checkout-index\" \nwill also read paths from stdin.\n\nThe diff family do not have any support for --pathspec-from-file at the \nmoment but I'd be happy to see someone implement it (I think it would be \nfairly straight forward).\n\n> Note that gitglossary (at least in 2.35.1 git on debian) says that\n> \n> \tPathspecs are used on the command line of \"git ls-files\", \"git\n>      ls-tree\", \"git add\", \"git grep\", \"git diff\", \"git checkout\", and many other\n>      commands ...\n> \n> but\n> \n> \t$> git ls-tree -h | head -n1\n> \tusage: git ls-tree [<options>] <tree-ish> [<path>...]\n> \n> so it is <path> not the <pathspec> like (why in stderr this time?)\n> \n> \t$> git commit -h 2>&1 | head -n1\n> \tusage: git commit [<options>] [--] <pathspec>...\n> \n> So if in both cases it is pathspec, may be pathspec could support some\n> other magical keyword like :(filelist)/tmp/mylonglistofpaths ?\n\nI like that path magic idea, but as we already have --pathspec-from-file \nI think we'd be better improving support for that.\n\nBest Wishes\n\nPhillip\n\n> Thanks in advance for your time and thoughts,\n"}]}