threads / patch / 813

patchgit-tar-tree: add a test case (resent)

Subject: [PATCH] git-tar-tree: add a test case (resent)

## tl;dr

10 messages between Jun 2, 2005 and Jun 3, 2005. Diffs are folded; open one to read it.

replies: 9people: 3as markdown or json

Rene Scharfe· Jun 2, 2005, 18:50 UTC · lore
git-tar-tree: add a simple test case.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Show changes to t/t3200-tar-tree.sh +106 −0
diff --git a/t/t3200-tar-tree.sh b/t/t3200-tar-tree.sh
new file mode 100755
--- /dev/null
+++ b/t/t3200-tar-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+#
+# Copyright (C) 2005 Rene Scharfe
+#
+
+test_description='git-tar-tree and git-get-tar-commit-id test
+
+This test covers the topics of long paths, file contents, commit date
+handling and commit id embedding:
+
+  Paths longer than 100 characters require the use of a pax extended
+  header to store them.  The test creates files with pathes both longer
+  and shorter than 100 chars, and also checks symlinks with long and
+  short pathes both as their own name and as target path.
+
+  The contents of the repository is compared to the extracted tar
+  archive.  The repository contains simple text files, symlinks and a
+  binary file (/bin/sh).
+
+  git-tar-tree applies the commit date to every file in the archive it
+  creates.  The test sets the commit date to a specific value and checks
+  if the tar archive contains that value.
+
+  When giving git-tar-tree a commit id (in contrast to a tree id) it
+  embeds this commit id into the tar archive as a comment.  The test
+  checks the ability of git-get-tar-commit-id to figure it out from the
+  tar file.
+
+'
+
+. ./test-lib.sh
+
+test_expect_success \
+    'populate workdir' \
+    'mkdir a b c &&
+     p48=1.......10........20........30........40......48 &&
+     p50=1.......10........20........30........40........50 &&
+     p98=${p48}${p50} &&
+     echo simple textfile >a/a &&
+     echo 100 chars in path >a/${p98} &&
+     echo 101 chars in path >a/${p98}x &&
+     echo 102 chars in path >a/${p98}xx &&
+     echo 103 chars in path >a/${p98}xxx &&
+     mkdir a/bin &&
+     cp /bin/sh a/bin/sh &&
+     ln -s a a/l1 &&
+     ln -s ${p98}xx a/l100 &&
+     ln -s ${p98}xxx a/l101 &&
+     ln -s ${p98}xxx a/l${p98} &&
+     (cd a && find .) | sort >a.lst'
+
+test_expect_success \
+    'add files to repository' \
+    'find a -type f | xargs git-update-cache --add &&
+     find a -type l | xargs git-update-cache --add &&
+     treeid=`git-write-tree` &&
+     echo $treeid >treeid &&
+     TZ= GIT_COMMITTER_DATE="2005-05-27 22:00:00" \
+     git-commit-tree $treeid </dev/null >.git/HEAD'
+
+test_expect_success \
+    'git-tar-tree' \
+    'git-tar-tree HEAD >b.tar'
+
+test_expect_success \
+    'validate file modification time' \
+    'tar tvf b.tar a/a | awk \{print\ \$4,\$5\} >b.mtime &&
+     echo "2005-05-27 22:00:00" >expected.mtime &&
+     diff expected.mtime b.mtime'
+
+test_expect_success \
+    'git-get-tar-commit-id' \
+    'git-get-tar-commit-id <b.tar >b.commitid &&
+     diff .git/HEAD b.commitid'
+
+test_expect_success \
+    'extract tar archive' \
+    '(cd b && tar xf -) <b.tar'
+
+test_expect_success \
+    'validate filenames' \
+    '(cd b/a && find .) | sort >b.lst &&
+     diff a.lst b.lst'
+
+test_expect_success \
+    'validate file contents' \
+    'diff -r a b/a'
+
+test_expect_success \
+    'git-tar-tree with prefix' \
+    'git-tar-tree HEAD prefix >c.tar'
+
+test_expect_success \
+    'extract tar archive with prefix' \
+    '(cd c && tar xf -) <c.tar'
+
+test_expect_success \
+    'validate filenames with prefix' \
+    '(cd c/prefix/a && find .) | sort >c.lst &&
+     diff a.lst c.lst'
+
+test_expect_success \
+    'validate file contents with prefix' \
+    'diff -r a c/prefix/a'
+
+test_done
Rene Scharfe· Jun 2, 2005, 20:50 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

On Thu, Jun 02, 2005 at 01:05:36PM -0700, Junio C Hamano wrote:
> I like the test but suspect it belongs to t5000 series according
> to the numbering scheme Pasky originaly did.
The README says:
  3 - the other basic commands (e.g. ls-files)
  [...]
  5 - the pull and exporting commands

So I guess you're right. I shouldn't have stopped reading after point three..

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Show changes to t/t5000-tar-tree.sh +106 −0
diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
new file mode 100755
--- /dev/null
+++ b/t/t5000-tar-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+#
+# Copyright (C) 2005 Rene Scharfe
+#
+
+test_description='git-tar-tree and git-get-tar-commit-id test
+
+This test covers the topics of long paths, file contents, commit date
+handling and commit id embedding:
+
+  Paths longer than 100 characters require the use of a pax extended
+  header to store them.  The test creates files with pathes both longer
+  and shorter than 100 chars, and also checks symlinks with long and
+  short pathes both as their own name and as target path.
+
+  The contents of the repository is compared to the extracted tar
+  archive.  The repository contains simple text files, symlinks and a
+  binary file (/bin/sh).
+
+  git-tar-tree applies the commit date to every file in the archive it
+  creates.  The test sets the commit date to a specific value and checks
+  if the tar archive contains that value.
+
+  When giving git-tar-tree a commit id (in contrast to a tree id) it
+  embeds this commit id into the tar archive as a comment.  The test
+  checks the ability of git-get-tar-commit-id to figure it out from the
+  tar file.
+
+'
+
+. ./test-lib.sh
+
+test_expect_success \
+    'populate workdir' \
+    'mkdir a b c &&
+     p48=1.......10........20........30........40......48 &&
+     p50=1.......10........20........30........40........50 &&
+     p98=${p48}${p50} &&
+     echo simple textfile >a/a &&
+     echo 100 chars in path >a/${p98} &&
+     echo 101 chars in path >a/${p98}x &&
+     echo 102 chars in path >a/${p98}xx &&
+     echo 103 chars in path >a/${p98}xxx &&
+     mkdir a/bin &&
+     cp /bin/sh a/bin/sh &&
+     ln -s a a/l1 &&
+     ln -s ${p98}xx a/l100 &&
+     ln -s ${p98}xxx a/l101 &&
+     ln -s ${p98}xxx a/l${p98} &&
+     (cd a && find .) | sort >a.lst'
+
+test_expect_success \
+    'add files to repository' \
+    'find a -type f | xargs git-update-cache --add &&
+     find a -type l | xargs git-update-cache --add &&
+     treeid=`git-write-tree` &&
+     echo $treeid >treeid &&
+     TZ= GIT_COMMITTER_DATE="2005-05-27 22:00:00" \
+     git-commit-tree $treeid </dev/null >.git/HEAD'
+
+test_expect_success \
+    'git-tar-tree' \
+    'git-tar-tree HEAD >b.tar'
+
+test_expect_success \
+    'validate file modification time' \
+    'tar tvf b.tar a/a | awk \{print\ \$4,\$5\} >b.mtime &&
+     echo "2005-05-27 22:00:00" >expected.mtime &&
+     diff expected.mtime b.mtime'
+
+test_expect_success \
+    'git-get-tar-commit-id' \
+    'git-get-tar-commit-id <b.tar >b.commitid &&
+     diff .git/HEAD b.commitid'
+
+test_expect_success \
+    'extract tar archive' \
+    '(cd b && tar xf -) <b.tar'
+
+test_expect_success \
+    'validate filenames' \
+    '(cd b/a && find .) | sort >b.lst &&
+     diff a.lst b.lst'
+
+test_expect_success \
+    'validate file contents' \
+    'diff -r a b/a'
+
+test_expect_success \
+    'git-tar-tree with prefix' \
+    'git-tar-tree HEAD prefix >c.tar'
+
+test_expect_success \
+    'extract tar archive with prefix' \
+    '(cd c && tar xf -) <c.tar'
+
+test_expect_success \
+    'validate filenames with prefix' \
+    '(cd c/prefix/a && find .) | sort >c.lst &&
+     diff a.lst c.lst'
+
+test_expect_success \
+    'validate file contents with prefix' \
+    'diff -r a c/prefix/a'
+
+test_done
Linus Torvalds· Jun 3, 2005, 01:34 UTC · re: Rene Scharfe · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

On Thu, 2 Jun 2005, Rene Scharfe wrote:
>
> git-tar-tree: add a simple test case.
I get:
	* FAIL 6: extract tar archive (cd b && tar xf -) <b.tar
	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
	* FAIL 8: validate file contents diff -r a b/a
	* FAIL 11: validate filenames with prefix (cd c/prefix/a && find .) | sort >c.lst &&
	* FAIL 12: validate file contents with prefix diff -r a c/prefix/a
Hmm?
		Linus
Rene Scharfe· Jun 3, 2005, 05:34 UTC · re: Linus Torvalds · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

Linus Torvalds schrieb:
Show 13 quoted lines
> 
> On Thu, 2 Jun 2005, Rene Scharfe wrote:
> 
>>git-tar-tree: add a simple test case.
> 
> 
> I get:
> 
> 	* FAIL 6: extract tar archive (cd b && tar xf -) <b.tar
> 	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
> 	* FAIL 8: validate file contents diff -r a b/a
> 	* FAIL 11: validate filenames with prefix (cd c/prefix/a && find .) | sort >c.lst &&
> 	* FAIL 12: validate file contents with prefix diff -r a c/prefix/a

What version of tar do you use? Also, can you please send me the output the following?

   cd t; sh t5000-tar-tree.sh; cd trash; tar tvf b.tar

Thanks, Rene

Linus Torvalds· Jun 3, 2005, 14:31 UTC · re: Rene Scharfe · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

On Fri, 3 Jun 2005, Rene Scharfe wrote:
> 
> What version of tar do you use? 
	torvalds@ppc970:~> tar --version
	tar (GNU tar) 1.13.25
	Copyright © 2001 Free Software Foundation, Inc.
	This program comes with NO WARRANTY, to the extent permitted by law.
	You may redistribute it under the terms of the GNU General Public License;
	see the file named COPYING for details.
	Written by John Gilmore and Jay Fenlason.
> Also, can you please send me the output
> the following?
> 
>    cd t; sh t5000-tar-tree.sh; cd trash; tar tvf b.tar
I get
	?rw------- git/git          52 1969-12-31 16:00:00 pax_global_header unknown file type `g'
	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 a/1.......10........20........30........40......481.......10........20........30........40........50
	?rw------- git/git         111 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.paxheader unknown file type `x'
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.data
	?rw------- git/git         112 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.paxheader unknown file type `x'
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.data
	?rw------- git/git         113 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.paxheader unknown file type `x'
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.data
	-rw-r--r-- git/git          16 2005-05-27 15:00:00 a/a
	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/bin/
	-rwxr-xr-x git/git      719924 2005-05-27 15:00:00 a/bin/sh
	tar: Unexpected EOF in archive
	tar: Error is not recoverable: exiting now

which doesn't tell me anything, except I assume it means that my tar doesn't understand extended headers.

		Linus
Rene Scharfe· Jun 3, 2005, 14:43 UTC · re: Linus Torvalds · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

Linus Torvalds schrieb:
Show 7 quoted lines
> 	torvalds@ppc970:~> tar --version
> 	tar (GNU tar) 1.13.25
> 	Copyright © 2001 Free Software Foundation, Inc.
> 	This program comes with NO WARRANTY, to the extent permitted by law.
> 	You may redistribute it under the terms of the GNU General Public License;
> 	see the file named COPYING for details.
> 	Written by John Gilmore and Jay Fenlason.
[...]
Show 19 quoted lines
> I get
> 
> 	?rw------- git/git          52 1969-12-31 16:00:00 pax_global_header unknown file type `g'
> 	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 a/1.......10........20........30........40......481.......10........20........30........40........50
> 	?rw------- git/git         111 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.paxheader unknown file type `x'
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.data
> 	?rw------- git/git         112 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.paxheader unknown file type `x'
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.data
> 	?rw------- git/git         113 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.paxheader unknown file type `x'
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.data
> 	-rw-r--r-- git/git          16 2005-05-27 15:00:00 a/a
> 	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/bin/
> 	-rwxr-xr-x git/git      719924 2005-05-27 15:00:00 a/bin/sh
> 	tar: Unexpected EOF in archive
> 	tar: Error is not recoverable: exiting now
> 
> which doesn't tell me anything, except I assume it means that my tar 
> doesn't understand extended headers.

That's true, but you don't need them if all your pathes' lengths are 100 or shorter. Of course the test exercising long paths will always fail with your version of tar. I should split it out better, so you will at least get a positive result for short paths. I just didn't expect such an old version of tar to be in use anymore. :-P

But your output shows a more serious problem: "Unexpected EOF" means the archive is missing a piece. I sent you a patch for that bug in my previous mail.

Thanks, Rene

Rene Scharfe· Jun 3, 2005, 11:25 UTC · re: Linus Torvalds · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

Linus Torvalds schrieb:
Show 13 quoted lines
> 
> On Thu, 2 Jun 2005, Rene Scharfe wrote:
> 
>>git-tar-tree: add a simple test case.
> 
> 
> I get:
> 
> 	* FAIL 6: extract tar archive (cd b && tar xf -) <b.tar
> 	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
> 	* FAIL 8: validate file contents diff -r a b/a
> 	* FAIL 11: validate filenames with prefix (cd c/prefix/a && find .) | sort >c.lst &&
> 	* FAIL 12: validate file contents with prefix diff -r a c/prefix/a

Meine Fresse, what a stoopid bug. Fortunately I introduced it with one of the other two patches sent together with the test case patch, namely "[PATCH] git-tar-tree: cleanup write_trailer() (resent)". Of course I only wanted to demonstrate how useful those tests really are. ;-)

Btw., I got it right the first time and messed up when resending..

write_trailer() writes the last 10k (a full block) of the tar archive. write_if_needed() writes out a block *if* it is full and then sets the offset to 0. In nine out of ten cases the messed up write_trailer() function didn't manage to fill the block thus not writing anything at all, truncating the archive. I was "lucky" to hit the other case and so my testing ran OK.

Here's a patch.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Show changes to tar-tree.c +1 −1
--- a/tar-tree.c
+++ b/tar-tree.c
@@ -77,7 +77,7 @@
 	write_if_needed();
 	get_record();
 	write_if_needed();
-	if (offset) {
+	while (offset) {
 		get_record();
 		write_if_needed();
 	}
Linus Torvalds· Jun 3, 2005, 14:46 UTC · re: Rene Scharfe · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

On Fri, 3 Jun 2005, Rene Scharfe wrote:
> 
> Here's a patch.
This one fixes failure 6 for me, but 7, 8, 11 and 12 still fail.
Doing a
	sh t5000-tar-tree.sh -i -v
ends with
	* expecting success: (cd b/a && find .) | sort >b.lst &&
	     diff a.lst b.lst
	3,5d2
	< ./1.......10........20........30........40......481.......10........20........30........40........50x
	< ./1.......10........20........30........40......481.......10........20........30........40........50xx
	< ./1.......10........20........30........40......481.......10........20........30........40........50xxx
	10d6
	< ./l1.......10........20........30........40......481.......10........20........30........40........50
	12d7
	< ./l101
	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
	     diff a.lst b.lst

and you already saw my output from "tar tvf b.tar", so you already know that this is probably due to not understanding extended headers.

		Linus
Linus Torvalds· Jun 3, 2005, 14:51 UTC · re: Linus Torvalds · lore

Re: [PATCH] git-tar-tree: add a test case (resent)

On Fri, 3 Jun 2005, Linus Torvalds wrote:
> 
> and you already saw my output from "tar tvf b.tar", so you already know 
> that this is probably due to not understanding extended headers.

My other ppc64 box (running Fedora core 3.92) has no problems, so this is apparently something that has been changed between tar 1.13.25 and 1.15.1 (YDL 4.0 vs Fedora Core tar versions).

		Linus

← back to recent threads