patcht*: avoid using pipes

Hi, I'm Prathamesh Chavan. As a part of my micropraoject I have been working on "Avoid pipes for git related commands in test suites". I tried sending the patch, but it got blocked since the mail contained more than 100 000 characters. Hence I'll like to attach the link to my branch 'micro-proj', where I did the required changes.

https://github.com/pratham-pc/git/tree/micro-proj

Thanks.

On Tue, Mar 7, 2017 at 8:10 AM, Prathamesh Chavan <pc44800@gmail.com> wrote:

Thanks for working on that microproject!

Yeah, even the github UI seems to have trouble with that commit. (A bit slow, not showing the full content, but rather I needed to click on "load diff" for tests files 7000+)

This is a lot of change (in terms of lines) for a micro project. :)
I'd have two competing advices:
* keep it micro first, e.g. just convert one file,
  send to list, wait for reviewers feedback and incorporate that
  (optional step after having done the full development cycle:
  convert all the other files; each as its own patch)
* split up this one patch into multiple patches, e.g. one
  file per patch, then send a patch series.

The outcome will be the same, but in the first you get feedback quicker, such that hopefully you only need to touch the rest of files after the first file just once.

While I did look at that, not everyone here in the git community does so. (Also for getting your change in, Junio seems to strongly prefer patches on list instead of e.g. fetching and cherry-picking from your github)

When looking at the content, the conversion seems a bit mechanical
(which is fine for a micro project), such as:
...
- test "$(git show --pretty=format:%s | head -n 1)" = "one"
+ test "$(git show --pretty=format:%s >out && head -n 1 <out)" = "one"
...

specifically for the "head" command I don't think it makes a difference in correctness whether you pipe the file into the tool or give the filename, i.e. "head -n 1 out" would work just as fine.

There is a difference in readability, though. For consistency I'd suggest to drop the "<", as the numbers might support:

$ cd t $ git grep head |wc -l # This also counts other occurrences of the string, # not just the invocation of the head tool 2871 $ git grep head |grep "<" |wc -l # same here 58

Another aspect might be performance at scale as the "<" will let the shell open the file and pipe the content via stdin to the head tool, whereas when giving a filename the head tool needs to open the file. Both times the file doesn't need to be read completely, but "head -n 1" can close the file handle early in the game. I dunno.

Thanks, Stefan

Am 07.03.2017 um 18:21 schrieb Stefan Beller:

Welcome to the Git community!

Actually, being a *micro* project, it should stay so. Not doing all of the changes would leave some tasks for other apprentices to get warm with our review process.

Thank you, Stefan, for digging out one particularly interesting case.

True, but!

The intent of removing git invocations from the upstream of a pipe is that a failure exit code is able to stop a && chain.

The example above does not achieve this even after removal of the pipe. The reason is that exit code of process expansions $(...) is usually ignored. To meet the intent, further changes are necessary, for example to:

	git show --pretty=format:%s >out &&
	test "$(head -n 1 out)" = "one"

Side note: There is one exception where the exit code of $(...) is not ignored: when it occurs in the last variable assignment of a command that consists only of variable assignments.

-- Hannes

On Tue, Mar 7, 2017 at 12:39 PM, Johannes Sixt <j6t@kdbg.org> wrote:

right, so just pick one file.

Doh! I was so fixated over discussing whether to use "<" or not, to miss looking for the actual goal.

Thanks for spotting! Stefan

On Tue, Mar 07, 2017 at 12:52:49PM -0800, Stefan Beller wrote:

I also wonder if we really want all invocations of git to be marked up in this way. If the primary goal of the test is checking that a certain git command runs successfully and generates the expected output, then I think it is a good candidate for conversion.

So in a hunk like this:

   test_expect_success 'git commit-tree records the correct tree in a commit' '
   	commit0=$(echo NO | git commit-tree $P) &&
  -	tree=$(git show --pretty=raw $commit0 |
  -		 sed -n -e "s/^tree //p" -e "/^author /q") &&
  +	tree=$(git show --pretty=raw $commit0 >out &&
  +	sed -n -e "s/^tree //p" -e "/^author /q" <out) &&
   	test "z$tree" = "z$P"

we are interested in testing commit-tree, not "git show". Is it worth avoiding pipes there? I admit the cost to using the intermediate file is not huge there, but it feels more awkward and un-shell-like to me as a reader.

-Peff

On Wed, Mar 8, 2017 at 11:33 AM, Jeff King <peff@peff.net> wrote:

Thank you everyone, for reviewing my changes. And as said in the reviews, I'll send a single patch file as my microproject, leaving the other files as low hanging fruit for the others to look at. Also, I try to include as many suggested improvements as possible and will also remember them for my future patches.

I have created the required changes and submitted a single file patch. Also I tried my best to include each of the suggestions in that patch.

https://public-inbox.org/git/0102015aae7b8536-00c57d0a-1d48-4153-a202-87c4ea9e0e19-000000@eu-west-1.amazonses.com/

On Wed, Mar 8, 2017 at 7:02 PM, Prathamesh Chavan <pc44800@gmail.com> wrote: