git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH jn/fast-import-blob-access] t9300: use perl "head -c" clone in place of "dd bs=1 count=16000" kluge

From
Jonathan Nieder <jrnieder@gmail.com>
Date
Dec 13, 2010, 09:28 UTC
Message-ID
<20101213092805.GB23443@burratino>
In-Reply-To
<7vfwu25e9q.fsf@alter.siamese.dyndns.org>

It is unfortunate to have to issue thousands of one-byte read calls to work around dd's refusal to buffer input that would fill a block after a short read (a3a6f4, 2010-12-13). We could do better by using "head -c", if it were available on all platforms we cared about. Replace it with some simple perl.

While doing so, restructure 9300.114 to use a subshell instead of a script. Subshells can inherit functions (like the new head_c) from the parent shell while external scripts cannot.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Junio C Hamano wrote:
> It is unfortunate and feels
> yucky that we have to issue 8k+ read(2) of one byte
How about this?
 t/t9300-fast-import.sh |   84 ++++++++++++++++++++++++++++--------------------
 1 files changed, 49 insertions(+), 35 deletions(-)
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index ed28d3c..924a833 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -7,6 +7,23 @@ test_description='test git fast-import utility'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/diff-lib.sh ;# test-lib chdir's into trash
 
+# Print $1 bytes from stdin to stdout.
+#
+# This could be written as "head -c $1", but IRIX "head" does not
+# support the -c option.
+head_c () {
+	perl -e '
+		my $len = $ARGV[1];
+		while ($len > 0) {
+			my $s;
+			my $nread = sysread(STDIN, $s, $len);
+			die "cannot read: $!" unless defined($nread);
+			print $s;
+			$len -= $nread;
+		}
+	' - "$1"
+}
+
 file2_data='file2
 second line of EOF'
 
@@ -1780,44 +1797,41 @@ test_expect_success PIPE 'R: copy using cat-file' '
 	rm -f blobs &&
 	cat >frontend <<-\FRONTEND_END &&
 	#!/bin/sh
-	cat <<EOF &&
-	feature cat-blob
-	blob
-	mark :1
-	data <<BLOB
-	EOF
-	cat big
-	cat <<EOF
-	BLOB
-	cat-blob :1
-	EOF
-
-	read blob_id type size <&3 &&
-	echo "$blob_id $type $size" >response &&
-	dd of=blob bs=1 count=$size <&3 &&
-	read newline <&3 &&
-
-	cat <<EOF &&
-	commit refs/heads/copied
-	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
-	data <<COMMIT
-	copy big file as file3
-	COMMIT
-	M 644 inline file3
-	data <<BLOB
-	EOF
-	cat blob &&
-	cat <<EOF
-	BLOB
-	EOF
 	FRONTEND_END
 
 	mkfifo blobs &&
 	(
 		export GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL GIT_COMMITTER_DATE &&
-		sh frontend 3<blobs |
-		git fast-import --cat-blob-fd=3 3>blobs
-	) &&
+		cat <<-\EOF &&
+		feature cat-blob
+		blob
+		mark :1
+		data <<BLOB
+		EOF
+		cat big &&
+		cat <<-\EOF &&
+		BLOB
+		cat-blob :1
+		EOF
+
+		read blob_id type size <&3 &&
+		echo "$blob_id $type $size" >response &&
+		head_c $size >blob <&3 &&
+		read newline <&3 &&
+
+		cat <<-EOF &&
+		commit refs/heads/copied
+		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+		data <<COMMIT
+		copy big file as file3
+		COMMIT
+		M 644 inline file3
+		data <<BLOB
+		EOF
+		cat blob &&
+		echo BLOB
+	) 3<blobs |
+	git fast-import --cat-blob-fd=3 3>blobs &&
 	git show copied:file3 >actual &&
 	test_cmp expect.response response &&
 	test_cmp big actual
@@ -1845,7 +1859,7 @@ test_expect_success PIPE 'R: print blob mid-commit' '
 		EOF
 
 		read blob_id type size <&3 &&
-		dd of=actual bs=1 count=$size <&3 &&
+		head_c $size >actual <&3 &&
 		read newline <&3 &&
 
 		echo
@@ -1880,7 +1894,7 @@ test_expect_success PIPE 'R: print staged blob within commit' '
 		echo "cat-blob $to_get" &&
 
 		read blob_id type size <&3 &&
-		dd of=actual bs=1 count=$size <&3 &&
+		head_c $size >actual <&3 &&
 		read newline <&3 &&
 
 		echo deleteall
-- 
1.7.2.4.568.g3733c.dirty
Previous: Junio C Hamano
Message 6 of 6 in “SIGPIPE in t9300-fast-import”
  1. Brian GernhardtDec 12, 2010
  2. Jonathan NiederDec 12, 2010
  3. Brian GernhardtDec 12, 2010
  4. t9300: avoid short reads from ddJonathan Nieder, Dec 13, 2010
  5. Junio C HamanoDec 13, 2010
  6. t9300: use perl "head -c" clone in place of "dd bs=1 count=16000" klugeJonathan Nieder, Dec 13, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.