{"thread":{"id":"28354","subject":"\"git archive\" seems to be broken wrt zip files","startedAt":"2011-09-11T04:58:08Z","lastAt":"2011-09-14T08:32:49Z","messageCount":7,"participants":["Linus Torvalds","Jeff King","Andreas Schwab","René Scharfe"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"175285","messageId":"CA+55aFx43OxExGNrJs+AyKNtdr+KCZZoE=iaQTz8uHoUSrQv0w@mail.gmail.com","threadId":"28354","inReplyTo":null,"subject":"\"git archive\" seems to be broken wrt zip files","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-09-11T04:58:08Z","receivedAt":"2011-09-11T04:58:08Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"So I wouldn't ever have noticed on my own, but now that I've tried\ngithub for the kernel, somebody else reported that the downloaded zip\nfile (seriously? the kernel as a zip file?) is corrupt.\n\nAnd it doesn't really seem to be a github issue. I can re-create it\nwith a simple\n\n git archive --format=zip HEAD -o ../kernel.zip\n\non my kernel repository: the end result does not unzip correctly:\n\n mkdir temp-directory\n cd temp-directory\n unzip kernel.zip\n ...\n inflating: virt/kvm/iommu.c\n inflating: virt/kvm/irq_comm.c\n inflating: virt/kvm/kvm_main.c\n finishing deferred symbolic links:\n arch/microblaze/boot/dts/system.dts -> ../../platform/generic/system.dts\n drivers/scsi/aic94xx/aic94xx_reg.h -> /*^J * Aic94xx SAS/SATA\ndriver hardware registers definitions.[ rest of the file ]\n symlink error: File name too long\n\niow, for some reason that \"drivers/scsi/aic94xx/aic94xx_reg.h\" file\nseems to have been encoded as a symlink.\n\nAnybody seen this?\n\n Linus\n"},{"id":"175286","messageId":"20110911062206.GA29620@sigill.intra.peff.net","threadId":"28354","inReplyTo":"CA+55aFx43OxExGNrJs+AyKNtdr+KCZZoE=iaQTz8uHoUSrQv0w@mail.gmail.com","subject":"Re: \"git archive\" seems to be broken wrt zip files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-09-11T06:22:06Z","receivedAt":"2011-09-11T06:22:06Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 10, 2011 at 09:58:08PM -0700, Linus Torvalds wrote:\n\n> So I wouldn't ever have noticed on my own, but now that I've tried\n> github for the kernel, somebody else reported that the downloaded zip\n> file (seriously? the kernel as a zip file?) is corrupt.\n> \n> And it doesn't really seem to be a github issue. I can re-create it\n> with a simple\n> \n> git archive --format=zip HEAD -o ../kernel.zip\n> \n> on my kernel repository: the end result does not unzip correctly:\n\nHmm. I can easily replicate the problem here, but interestingly it does\nnot happen with sub-trees like:\n\n git archive --format=zip HEAD:drivers -o ../kernel.zip\n\nGoing back in history, I can replicate it with René's 62cdce1\n(git-archive --format=zip: add symlink support, 2006-10-07). So there's\nnothing to bisect.\n\nCc'ing René.\n\n-Peff\n"},{"id":"175287","messageId":"20110911062740.GA8018@sigill.intra.peff.net","threadId":"28354","inReplyTo":"20110911062206.GA29620@sigill.intra.peff.net","subject":"Re: \"git archive\" seems to be broken wrt zip files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-09-11T06:27:40Z","receivedAt":"2011-09-11T06:27:40Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Sep 11, 2011 at 02:22:06AM -0400, Jeff King wrote:\n\n> Hmm. I can easily replicate the problem here, but interestingly it does\n> not happen with sub-trees like:\n> \n> git archive --format=zip HEAD:drivers -o ../kernel.zip\n> \n> Going back in history, I can replicate it with René's 62cdce1\n> (git-archive --format=zip: add symlink support, 2006-10-07). So there's\n> nothing to bisect.\n\nWeirder still. I get roughly the same output as you:\n\n finishing deferred symbolic links:\n arch/microblaze/boot/dts/system.dts -> ../../platform/generic/system.dts\n drivers/scsi/aic94xx/aic94xx_reg.h -> /*^J * Aic94xx SAS/SATA driver...\n\nBut looking at the generated file with zipinfo, I see:\n\n $ zipinfo kernel.zip\n ...\n lrwxrwxrwx 2.3 unx 33 b- stor 11-Aug-25 14:02 arch/microblaze/boot/dts/system.dts\n ...\n -rw---- 0.0 fat 10470 b- defN 11-Aug-25 14:02 drivers/scsi/aic94xx/aic94xx_reg.h\n\nIOW, the zip file looks right. I wonder if this is actually a bug in\n\"unzip\".\n\n-Peff\n"},{"id":"175293","messageId":"m239g3i5kz.fsf@igel.home","threadId":"28354","inReplyTo":"20110911062740.GA8018@sigill.intra.peff.net","subject":"Re: \"git archive\" seems to be broken wrt zip files","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2011-09-11T13:14:36Z","receivedAt":"2011-09-11T13:14:36Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Jeff King writes:\n\n> IOW, the zip file looks right. I wonder if this is actually a bug in\n> \"unzip\".\n\nIt is. This only happens if you have more then 16k entries and when one\nof the 16k entry infos is reused it happend to be previously used for a\nsymlink entry.\n\nHere's a patch for unzip60 for reference:\n\n--- process.c\n+++ process.c\n@@ -1751,6 +1751,12 @@ int process_cdir_file_hdr(__G) /* ret\n = (G.crec.general_purpose_bit_flag & (1 << 11)) == (1 << 11);\n #endif\n \n+#ifdef SYMLINKS\n+ /* Initialize the symlink flag, may be set by the platform-specific\n+ mapattr function. */\n+ G.pInfo->symlink = 0;\n+#endif\n+\n return PK_COOL;\n \n } /* end function process_cdir_file_hdr() */\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"175294","messageId":"4E6CD613.9020303@lsrfire.ath.cx","threadId":"28354","inReplyTo":"m239g3i5kz.fsf@igel.home","subject":"Re: \"git archive\" seems to be broken wrt zip files","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2011-09-11T15:38:59Z","receivedAt":"2011-09-11T15:38:59Z","isPatch":false,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 11.09.2011 15:14, schrieb Andreas Schwab:\n> Jeff King writes:\n> \n>> IOW, the zip file looks right. I wonder if this is actually a bug in\n>> \"unzip\".\n> \n> It is. This only happens if you have more then 16k entries and when one\n> of the 16k entry infos is reused it happend to be previously used for a\n> symlink entry.\n> \n> Here's a patch for unzip60 for reference:\n\nOh, thanks, now I can stop scratching my head while trying to find my\nway around unzip's source and its amazing OS compatibility code.\n\nOn Windows (the main target for ZIP file support in git archive), 7-Zip,\nthe unzip from msysgit and the native ZIP support extract the archive\nwithout any issues (except for 13 files whose names only differ\nupper/lower case of some chars to 13 other files), by the way. Since\nthey lack symlink support, the target path of\narch/microblaze/boot/dts/system.dts is simply written to a regular file,\nthough.\n\nRené\n"},{"id":"175454","messageId":"CA+55aFwLk8XcCDXM0w_wf5RnoB4oGxkjJrUGLB5wgSkx385b_g@mail.gmail.com","threadId":"28354","inReplyTo":"CA+55aFxsaE5btVJmM_QaUMcDzBg4df-g8X7NknC6t9UM+oQATw@mail.gmail.com","subject":"Re: \"git archive\" seems to be broken wrt zip files","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2011-09-14T06:55:39Z","receivedAt":"2011-09-14T06:55:39Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"Re-sent as this clearly never seems to have gone anywhere due to me\nnot realizing that the LF problems meant that while incoming email\nworked fine, going out through smpt.linux-foundation.org didn't work.\n\n Linus\n\nOn Sun, Sep 11, 2011 at 11:07 AM, Linus Torvalds\n wrote:\n> On Sun, Sep 11, 2011 at 6:14 AM, Andreas Schwab wrote:\n>>\n>> It is.  This only happens if you have more then 16k entries and when one\n>> of the 16k entry infos is reused it happend to be previously used for a\n>> symlink entry.\n>\n> Thanks for the explanation.\n>\n>> Here's a patch for unzip60 for reference:\n>\n> Is this problem known to the unzip developers? Is it actively\n> maintained any more? The 6.0 release seems to have been from April\n> 2009, and the web page seems to have a \"last updated\" of September\n> 2009.. Is there some other unofficial archive for bugfixes like this?\n> Distros?\n>\n>                            Linus\n>\n"},{"id":"175460","messageId":"m362kv5xse.fsf@hase.home","threadId":"28354","inReplyTo":"CA+55aFwLk8XcCDXM0w_wf5RnoB4oGxkjJrUGLB5wgSkx385b_g@mail.gmail.com","subject":"Re: \"git archive\" seems to be broken wrt zip files","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2011-09-14T08:32:49Z","receivedAt":"2011-09-14T08:32:49Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Linus Torvalds writes:\n\n>> Is this problem known to the unzip developers?\n\nI have already sent them the patch and they have acknowledged it.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"}]}