# "git archive" seems to be broken wrt zip files 7 messages from 2011-09-11 to 2011-09-14. Participants: Linus Torvalds, Jeff King, Andreas Schwab, René Scharfe. Thread: https://gitlist.dev/t/28354 ## Linus Torvalds, 2011-09-11 04:58 Subject: "git archive" seems to be broken wrt zip files Message-ID: URL: https://gitlist.dev/e/CA%2B55aFx43OxExGNrJs%2BAyKNtdr%2BKCZZoE%3DiaQTz8uHoUSrQv0w%40mail.gmail.com ``` So I wouldn't ever have noticed on my own, but now that I've tried github for the kernel, somebody else reported that the downloaded zip file (seriously? the kernel as a zip file?) is corrupt. And it doesn't really seem to be a github issue. I can re-create it with a simple git archive --format=zip HEAD -o ../kernel.zip on my kernel repository: the end result does not unzip correctly: mkdir temp-directory cd temp-directory unzip kernel.zip ... inflating: virt/kvm/iommu.c inflating: virt/kvm/irq_comm.c inflating: virt/kvm/kvm_main.c finishing deferred symbolic links: arch/microblaze/boot/dts/system.dts -> ../../platform/generic/system.dts drivers/scsi/aic94xx/aic94xx_reg.h -> /*^J * Aic94xx SAS/SATA driver hardware registers definitions.[ rest of the file ] symlink error: File name too long iow, for some reason that "drivers/scsi/aic94xx/aic94xx_reg.h" file seems to have been encoded as a symlink. Anybody seen this? Linus ``` ## Jeff King, 2011-09-11 06:22 Subject: Re: "git archive" seems to be broken wrt zip files Message-ID: <20110911062206.GA29620@sigill.intra.peff.net> URL: https://gitlist.dev/e/20110911062206.GA29620%40sigill.intra.peff.net In-Reply-To: ``` On Sat, Sep 10, 2011 at 09:58:08PM -0700, Linus Torvalds wrote: > So I wouldn't ever have noticed on my own, but now that I've tried > github for the kernel, somebody else reported that the downloaded zip > file (seriously? the kernel as a zip file?) is corrupt. > > And it doesn't really seem to be a github issue. I can re-create it > with a simple > > git archive --format=zip HEAD -o ../kernel.zip > > on my kernel repository: the end result does not unzip correctly: Hmm. I can easily replicate the problem here, but interestingly it does not happen with sub-trees like: git archive --format=zip HEAD:drivers -o ../kernel.zip Going back in history, I can replicate it with René's 62cdce1 (git-archive --format=zip: add symlink support, 2006-10-07). So there's nothing to bisect. Cc'ing René. -Peff ``` ## Jeff King, 2011-09-11 06:27 Subject: Re: "git archive" seems to be broken wrt zip files Message-ID: <20110911062740.GA8018@sigill.intra.peff.net> URL: https://gitlist.dev/e/20110911062740.GA8018%40sigill.intra.peff.net In-Reply-To: <20110911062206.GA29620@sigill.intra.peff.net> ``` On Sun, Sep 11, 2011 at 02:22:06AM -0400, Jeff King wrote: > Hmm. I can easily replicate the problem here, but interestingly it does > not happen with sub-trees like: > > git archive --format=zip HEAD:drivers -o ../kernel.zip > > Going back in history, I can replicate it with René's 62cdce1 > (git-archive --format=zip: add symlink support, 2006-10-07). So there's > nothing to bisect. Weirder still. I get roughly the same output as you: finishing deferred symbolic links: arch/microblaze/boot/dts/system.dts -> ../../platform/generic/system.dts drivers/scsi/aic94xx/aic94xx_reg.h -> /*^J * Aic94xx SAS/SATA driver... But looking at the generated file with zipinfo, I see: $ zipinfo kernel.zip ... lrwxrwxrwx 2.3 unx 33 b- stor 11-Aug-25 14:02 arch/microblaze/boot/dts/system.dts ... -rw---- 0.0 fat 10470 b- defN 11-Aug-25 14:02 drivers/scsi/aic94xx/aic94xx_reg.h IOW, the zip file looks right. I wonder if this is actually a bug in "unzip". -Peff ``` ## Andreas Schwab, 2011-09-11 13:14 Subject: Re: "git archive" seems to be broken wrt zip files Message-ID: URL: https://gitlist.dev/e/m239g3i5kz.fsf%40igel.home In-Reply-To: <20110911062740.GA8018@sigill.intra.peff.net> ``` Jeff King writes: > IOW, the zip file looks right. I wonder if this is actually a bug in > "unzip". It is. This only happens if you have more then 16k entries and when one of the 16k entry infos is reused it happend to be previously used for a symlink entry. Here's a patch for unzip60 for reference: --- process.c +++ process.c @@ -1751,6 +1751,12 @@ int process_cdir_file_hdr(__G) /* ret = (G.crec.general_purpose_bit_flag & (1 << 11)) == (1 << 11); #endif +#ifdef SYMLINKS + /* Initialize the symlink flag, may be set by the platform-specific + mapattr function. */ + G.pInfo->symlink = 0; +#endif + return PK_COOL; } /* end function process_cdir_file_hdr() */ Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ``` ## René Scharfe, 2011-09-11 15:38 Subject: Re: "git archive" seems to be broken wrt zip files Message-ID: <4E6CD613.9020303@lsrfire.ath.cx> URL: https://gitlist.dev/e/4E6CD613.9020303%40lsrfire.ath.cx In-Reply-To: ``` Am 11.09.2011 15:14, schrieb Andreas Schwab: > Jeff King writes: > >> IOW, the zip file looks right. I wonder if this is actually a bug in >> "unzip". > > It is. This only happens if you have more then 16k entries and when one > of the 16k entry infos is reused it happend to be previously used for a > symlink entry. > > Here's a patch for unzip60 for reference: Oh, thanks, now I can stop scratching my head while trying to find my way around unzip's source and its amazing OS compatibility code. On Windows (the main target for ZIP file support in git archive), 7-Zip, the unzip from msysgit and the native ZIP support extract the archive without any issues (except for 13 files whose names only differ upper/lower case of some chars to 13 other files), by the way. Since they lack symlink support, the target path of arch/microblaze/boot/dts/system.dts is simply written to a regular file, though. René ``` ## Linus Torvalds, 2011-09-14 06:55 Subject: Re: "git archive" seems to be broken wrt zip files Message-ID: URL: https://gitlist.dev/e/CA%2B55aFwLk8XcCDXM0w_wf5RnoB4oGxkjJrUGLB5wgSkx385b_g%40mail.gmail.com In-Reply-To: ``` Re-sent as this clearly never seems to have gone anywhere due to me not realizing that the LF problems meant that while incoming email worked fine, going out through smpt.linux-foundation.org didn't work. Linus On Sun, Sep 11, 2011 at 11:07 AM, Linus Torvalds wrote: > On Sun, Sep 11, 2011 at 6:14 AM, Andreas Schwab wrote: >> >> It is.  This only happens if you have more then 16k entries and when one >> of the 16k entry infos is reused it happend to be previously used for a >> symlink entry. > > Thanks for the explanation. > >> Here's a patch for unzip60 for reference: > > Is this problem known to the unzip developers? Is it actively > maintained any more? The 6.0 release seems to have been from April > 2009, and the web page seems to have a "last updated" of September > 2009.. Is there some other unofficial archive for bugfixes like this? > Distros? > >                            Linus > ``` ## Andreas Schwab, 2011-09-14 08:32 Subject: Re: "git archive" seems to be broken wrt zip files Message-ID: URL: https://gitlist.dev/e/m362kv5xse.fsf%40hase.home In-Reply-To: ``` Linus Torvalds writes: >> Is this problem known to the unzip developers? I have already sent them the patch and they have acknowledged it. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ```