# Bug?: import-tars misbehaves on Subversion tarballs 8 messages from 2007-04-24 to 2007-04-24. Participants: Karl Hasselström, Uwe Kleine-König, Michael Loeffler, Shawn O. Pearce. Thread: https://gitlist.dev/t/7813 ## Karl Hasselström, 2007-04-24 07:39 Subject: Bug?: import-tars misbehaves on Subversion tarballs Message-ID: <20070424073907.GA22121@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20070424073907.GA22121%40diana.vm.bytemark.co.uk ``` import-tars behaves very oddly when I try to import the Subversion tarballs. (For example, http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2 triggers this problem.) It creates two toplevel directories, subversion-1.4.3 and subversion-1.4.3subversion. The former seems to contain at least almost all files; the latter has only a handful of files, all with very long names. When I unpack it with GNU tar, I get all the files under a single "subversion-1.4.3" directory, as expected. Could it simply be that import-tars can't handle long filenames somehow? (This is pure speculation, since I know absolutely nothing about the tar format.) -- Karl Hasselström, kha@treskal.com www.treskal.com/kalle ``` ## Michael Loeffler, 2007-04-24 08:25 Subject: Re: Bug?: import-tars misbehaves on Subversion tarballs Message-ID: <3BF34FC9-BE89-4A5B-9AC6-57F61C464395@zvpunry.de> URL: https://gitlist.dev/e/3BF34FC9-BE89-4A5B-9AC6-57F61C464395%40zvpunry.de In-Reply-To: <20070424073907.GA22121@diana.vm.bytemark.co.uk> ``` Hi, Am 24.04.2007 um 09:39 schrieb Karl Hasselström: > import-tars behaves very oddly when I try to import the Subversion > tarballs. (For example, > http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2 > triggers this problem.) It creates two toplevel directories, > subversion-1.4.3 and subversion-1.4.3subversion. The former seems to > contain at least almost all files; the latter has only a handful of > files, all with very long names. I looked at this tarball with midnight commander (under MacOS X) and i saw 2 directories, subversion/ and subversion-1.4.3/. Then i looked at it with GNU tar (1.14) and saw only subversion-1.4.3/. > Could it simply be that import-tars can't handle long filenames > somehow? (This is pure speculation, since I know absolutely nothing > about the tar format.) I don't know, but mc does the same thing. I'll look at it later (i have more time at 18:00 +0200). bye ``` ## Uwe Kleine-König, 2007-04-24 08:40 Subject: Re: Bug?: import-tars misbehaves on Subversion tarballs Message-ID: <20070424084037.GA8906@informatik.uni-freiburg.de> URL: https://gitlist.dev/e/20070424084037.GA8906%40informatik.uni-freiburg.de In-Reply-To: <20070424073907.GA22121@diana.vm.bytemark.co.uk> ``` Hello Karl, Karl Hasselström wrote: > import-tars behaves very oddly when I try to import the Subversion > tarballs. (For example, > http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2 > triggers this problem.) It creates two toplevel directories, > subversion-1.4.3 and subversion-1.4.3subversion. The former seems to > contain at least almost all files; the latter has only a handful of > files, all with very long names. > > When I unpack it with GNU tar, I get all the files under a single > "subversion-1.4.3" directory, as expected. > > Could it simply be that import-tars can't handle long filenames > somehow? (This is pure speculation, since I know absolutely nothing > about the tar format.) I don't know much, but there are two locations that make up the name (i.e. prefix + name). Can you try the following patch: --- contrib/fast-import/import-tars.perl | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl index 5585a8b..5f3f742 100755 --- a/contrib/fast-import/import-tars.perl +++ b/contrib/fast-import/import-tars.perl @@ -64,7 +64,7 @@ foreach my $tar_file (@ARGV) } print FI "\n"; - my $path = "$prefix$name"; + my $path = "$prefix/$name"; $files{$path} = [$next_mark++, $mode]; $commit_time = $mtime if $mtime > $commit_time; -- 1.5.1.1.190.g74474 If it works, we still need a commit log ... Best regards Uwe -- Uwe Kleine-König http://www.google.com/search?q=5%2B7 ``` ## Karl Hasselström, 2007-04-24 10:09 Subject: Re: Bug?: import-tars misbehaves on Subversion tarballs Message-ID: <20070424100924.GB23811@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20070424100924.GB23811%40diana.vm.bytemark.co.uk In-Reply-To: <20070424084037.GA8906@informatik.uni-freiburg.de> ``` On 2007-04-24 10:40:37 +0200, Uwe Kleine-König wrote: > I don't know much, but there are two locations that make up the name > (i.e. prefix + name). Can you try the following patch: No, it still breaks, but in a new and interesting way: Now I _only_ get the files with long pathnames! -- Karl Hasselström, kha@treskal.com www.treskal.com/kalle ``` ## Uwe Kleine-König, 2007-04-24 11:51 Subject: [PATCH] [import-tars] fix importing of subversion tars Message-ID: <20070424115104.GA14986@informatik.uni-freiburg.de> URL: https://gitlist.dev/e/20070424115104.GA14986%40informatik.uni-freiburg.de In-Reply-To: <20070424100924.GB23811@diana.vm.bytemark.co.uk> ``` add a / between the prefix and name fields of the tar archive if prefix is non-empty. Signed-off-by: Uwe Kleine-König --- > No, it still breaks, but in a new and interesting way: Now I _only_ > get the files with long pathnames! I don't know exactly how standard-conformant it is not to include a trailing / after prefix, but the subversion tar does it. I checked for a description of the tar format, the best thing I could find is[1]: On USTAR format archives, the value of the prefix field, if non-null, is prefixed to the name field to allow names longer then 100 characters But it's not specifying if an additional / is needed. With this patch I get all the filenames right. Now the common prefix subversion-1.4.3 is stripped, but probably that's a feature. BTW I shortly tested a tar that had an entry with prefix ending in / and fast-import did the right thing. So something more complicated as: if ($prefix and $prefix[-1] eq "/") ... should not be needed. Best regards Uwe [1] http://www.mkssoftware.com/docs/man4/tar.4.asp contrib/fast-import/import-tars.perl | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl index 5585a8b..1842146 100755 --- a/contrib/fast-import/import-tars.perl +++ b/contrib/fast-import/import-tars.perl @@ -64,7 +64,12 @@ foreach my $tar_file (@ARGV) } print FI "\n"; - my $path = "$prefix$name"; + my $path; + if ($prefix) { + $path = "$prefix/$name"; + } else { + $path = "$name"; + } $files{$path} = [$next_mark++, $mode]; $commit_time = $mtime if $mtime > $commit_time; -- 1.5.2.rc0.16.g0f57d -- Uwe Kleine-König exit vi, lesson IV: Z Z NB: may write current file ``` ## Karl Hasselström, 2007-04-24 12:34 Subject: Re: [PATCH] [import-tars] fix importing of subversion tars Message-ID: <20070424123457.GA27766@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20070424123457.GA27766%40diana.vm.bytemark.co.uk In-Reply-To: <20070424115104.GA14986@informatik.uni-freiburg.de> ``` On 2007-04-24 13:51:04 +0200, Uwe Kleine-König wrote: > add a / between the prefix and name fields of the tar archive if > prefix is non-empty. > > Signed-off-by: Uwe Kleine-König Acked-by: Karl Hasselström This solves my problem with the Subversion tarballs. Thanks! -- Karl Hasselström, kha@treskal.com www.treskal.com/kalle ``` ## Karl Hasselström, 2007-04-24 12:44 Subject: Re: [PATCH] [import-tars] fix importing of subversion tars Message-ID: <20070424124412.GA28009@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20070424124412.GA28009%40diana.vm.bytemark.co.uk In-Reply-To: <20070424115104.GA14986@informatik.uni-freiburg.de> ``` On 2007-04-24 13:51:04 +0200, Uwe Kleine-König wrote: > Now the common prefix subversion-1.4.3 is stripped, but probably > that's a feature. Yes, there's code in import-tars that strips the topmost directory if and only if all paths in the tar are contained in it. -- Karl Hasselström, kha@treskal.com www.treskal.com/kalle ``` ## Shawn O. Pearce, 2007-04-24 16:17 Subject: Re: [PATCH] [import-tars] fix importing of subversion tars Message-ID: <20070424161749.GR17480@spearce.org> URL: https://gitlist.dev/e/20070424161749.GR17480%40spearce.org In-Reply-To: <20070424115104.GA14986@informatik.uni-freiburg.de> ``` Uwe Kleine-K??nig wrote: > add a / between the prefix and name fields of the tar archive if prefix > is non-empty. Thanks, this is in my fastimport tree now. -- Shawn. ```