threads / bug / 7813

Bug?: import-tars misbehaves on Subversion tarballs

Subject: Bug?: import-tars misbehaves on Subversion tarballs

## tl;dr

8 messages between Apr 24, 2007 and Apr 24, 2007.

replies: 7people: 4as markdown or json

Karl Hasselström· Apr 24, 2007, 07:39 UTC · lore

import-tars behaves very oddly when I try to import the Subversion tarballs. (For example, http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2 triggers this problem.) It creates two toplevel directories, subversion-1.4.3 and subversion-1.4.3subversion. The former seems to contain at least almost all files; the latter has only a handful of files, all with very long names.

When I unpack it with GNU tar, I get all the files under a single "subversion-1.4.3" directory, as expected.

Could it simply be that import-tars can't handle long filenames somehow? (This is pure speculation, since I know absolutely nothing about the tar format.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Michael Loeffler· Apr 24, 2007, 08:25 UTC · re: Karl Hasselström · lore

Re: Bug?: import-tars misbehaves on Subversion tarballs

Hi,
Am 24.04.2007 um 09:39 schrieb Karl Hasselström:
Show 7 quoted lines
> import-tars behaves very oddly when I try to import the Subversion
> tarballs. (For example,
> http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2
> triggers this problem.) It creates two toplevel directories,
> subversion-1.4.3 and subversion-1.4.3subversion. The former seems to
> contain at least almost all files; the latter has only a handful of
> files, all with very long names.

I looked at this tarball with midnight commander (under MacOS X) and i saw 2 directories, subversion/ and subversion-1.4.3/. Then i looked at it with GNU tar (1.14) and saw only subversion-1.4.3/.

> Could it simply be that import-tars can't handle long filenames
> somehow? (This is pure speculation, since I know absolutely nothing
> about the tar format.)

I don't know, but mc does the same thing. I'll look at it later (i have more time at 18:00 +0200).

bye
Uwe Kleine-König· Apr 24, 2007, 08:40 UTC · re: Karl Hasselström · lore

Re: Bug?: import-tars misbehaves on Subversion tarballs

Hello Karl,
Karl Hasselström wrote:
Show 14 quoted lines
> import-tars behaves very oddly when I try to import the Subversion
> tarballs. (For example,
> http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2
> triggers this problem.) It creates two toplevel directories,
> subversion-1.4.3 and subversion-1.4.3subversion. The former seems to
> contain at least almost all files; the latter has only a handful of
> files, all with very long names.
> 
> When I unpack it with GNU tar, I get all the files under a single
> "subversion-1.4.3" directory, as expected.
> 
> Could it simply be that import-tars can't handle long filenames
> somehow? (This is pure speculation, since I know absolutely nothing
> about the tar format.)

I don't know much, but there are two locations that make up the name (i.e. prefix + name). Can you try the following patch:

---
 contrib/fast-import/import-tars.perl |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl
index 5585a8b..5f3f742 100755
--- a/contrib/fast-import/import-tars.perl
+++ b/contrib/fast-import/import-tars.perl
@@ -64,7 +64,7 @@ foreach my $tar_file (@ARGV)
 		}
 		print FI "\n";
 
-		my $path = "$prefix$name";
+		my $path = "$prefix/$name";
 		$files{$path} = [$next_mark++, $mode];
 
 		$commit_time = $mtime if $mtime > $commit_time;
-- 
1.5.1.1.190.g74474

If it works, we still need a commit log ...

Best regards
Uwe

-- 
Uwe Kleine-König

http://www.google.com/search?q=5%2B7
Karl Hasselström· Apr 24, 2007, 10:09 UTC · re: Uwe Kleine-König · lore

Re: Bug?: import-tars misbehaves on Subversion tarballs

On 2007-04-24 10:40:37 +0200, Uwe Kleine-König wrote:
> I don't know much, but there are two locations that make up the name
> (i.e. prefix + name). Can you try the following patch:

No, it still breaks, but in a new and interesting way: Now I _only_ get the files with long pathnames!

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Uwe Kleine-König· Apr 24, 2007, 11:51 UTC · re: Karl Hasselström · lore

[PATCH] [import-tars] fix importing of subversion tars

add a / between the prefix and name fields of the tar archive if prefix is non-empty.

Signed-off-by: Uwe Kleine-König <ukleinek@informatik.uni-freiburg.de>
---
> No, it still breaks, but in a new and interesting way: Now I _only_
> get the files with long pathnames!

I don't know exactly how standard-conformant it is not to include a trailing / after prefix, but the subversion tar does it.

I checked for a description of the tar format, the best thing I could find is[1]:

	On USTAR format archives, the value of the prefix field, if
	non-null, is prefixed to the name field to allow names longer
	then 100 characters
But it's not specifying if an additional / is needed.
With this patch I get all the filenames right.

Now the common prefix subversion-1.4.3 is stripped, but probably that's a feature.

BTW I shortly tested a tar that had an entry with prefix ending in / and fast-import did the right thing. So something more complicated as:

	if ($prefix and $prefix[-1] eq "/") ...
should not be needed.

Best regards Uwe

[1] http://www.mkssoftware.com/docs/man4/tar.4.asp
 contrib/fast-import/import-tars.perl |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl
index 5585a8b..1842146 100755
--- a/contrib/fast-import/import-tars.perl
+++ b/contrib/fast-import/import-tars.perl
@@ -64,7 +64,12 @@ foreach my $tar_file (@ARGV)
 		}
 		print FI "\n";
 
-		my $path = "$prefix$name";
+		my $path;
+		if ($prefix) {
+			$path = "$prefix/$name";
+		} else {
+			$path = "$name";
+		}
 		$files{$path} = [$next_mark++, $mode];
 
 		$commit_time = $mtime if $mtime > $commit_time;
-- 
1.5.2.rc0.16.g0f57d

-- 
Uwe Kleine-König

exit vi, lesson IV:
Z Z

NB: may write current file
Karl Hasselström· Apr 24, 2007, 12:34 UTC · re: Uwe Kleine-König · lore

Re: [PATCH] [import-tars] fix importing of subversion tars

On 2007-04-24 13:51:04 +0200, Uwe Kleine-König wrote:
> add a / between the prefix and name fields of the tar archive if
> prefix is non-empty.
>
> Signed-off-by: Uwe Kleine-König <ukleinek@informatik.uni-freiburg.de>
Acked-by: Karl Hasselström <kha@treskal.com>
This solves my problem with the Subversion tarballs. Thanks!
-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Karl Hasselström· Apr 24, 2007, 12:44 UTC · re: Uwe Kleine-König · lore

Re: [PATCH] [import-tars] fix importing of subversion tars

On 2007-04-24 13:51:04 +0200, Uwe Kleine-König wrote:
> Now the common prefix subversion-1.4.3 is stripped, but probably
> that's a feature.

Yes, there's code in import-tars that strips the topmost directory if and only if all paths in the tar are contained in it.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Shawn O. Pearce· Apr 24, 2007, 16:17 UTC · re: Uwe Kleine-König · lore

Re: [PATCH] [import-tars] fix importing of subversion tars

Uwe Kleine-K??nig <ukleinek@informatik.uni-freiburg.de> wrote:
> add a / between the prefix and name fields of the tar archive if prefix
> is non-empty.
Thanks, this is in my fastimport tree now.
-- 
Shawn.

← back to recent threads