{"thread":{"id":"13876","subject":"[PATCH - stgit] Patch to allow import of compressed files","startedAt":"2008-06-09T18:38:55Z","lastAt":"2008-06-19T14:17:57Z","messageCount":18,"participants":["Clark Williams","Karl Hasselström","Asheesh Laroia","Sverre Rabbelier","David Kågedal"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"79251","messageId":"484D78BF.6030504@gmail.com","threadId":"13876","inReplyTo":null,"subject":"[PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-09T18:38:55Z","receivedAt":"2008-06-09T18:38:55Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nThis patch allows StGit to directly import compressed (.gz and .bz2) files with\nreasonable patch names.\n\nI do a lot of work on modified kernel trees and usually the first two things imported\nare a stable update patch followed immediately by an -rt patch, both of which are\ncompressed. With this patch I can just copy the files down directly from kernel.org\nand import them, rather than having to keep uncompressed copies around.\n\nHey, I'm lazy... :)\n\nClark\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEYEARECAAYFAkhNeL4ACgkQqA4JVb61b9cu0ACdH/Z71xh4gaD5euF3BgYnIhiO\nAkUAnipyN/dsTBQDyhc6uzFhxdxPeYvJ\n=2G71\n-----END PGP SIGNATURE-----\n\n\nFrom: Clark Williams \n\nPatch to allow import from compressed (.gz and .bz2) files\n\nSigned-off-by: Clark Williams \n---\n\n stgit/commands/imprt.py | 21 +++++++++++++++------\n 1 files changed, 15 insertions(+), 6 deletions(-)\n\ndiff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py\nindex 4a4b792..83dae2f 100644\n--- a/stgit/commands/imprt.py\n+++ b/stgit/commands/imprt.py\n@@ -178,8 +178,22 @@ def __create_patch(filename, message, author_name, author_email,\n def __import_file(filename, options, patch = None):\n \"\"\"Import a patch from a file or standard input\n \"\"\"\n+ if patch:\n+ pname = patch\n+ else:\n+ pname = filename\n+\n if filename:\n- f = file(filename)\n+ if filename.endswith(\".gz\"):\n+ import gzip\n+ f = gzip.open(filename)\n+ pname = filename.replace(\".gz\", \"\")\n+ elif filename.endswith(\".bz2\"):\n+ import bz2\n+ f = bz2.BZ2File(filename, 'r')\n+ pname = filename.replace(\".bz2\", \"\")\n+ else:\n+ f = file(filename)\n else:\n f = sys.stdin\n \n@@ -197,11 +211,6 @@ def __import_file(filename, options, patch = None):\n if filename:\n f.close()\n \n- if patch:\n- pname = patch\n- else:\n- pname = filename\n-\n __create_patch(pname, message, author_name, author_email,\n author_date, diff, options)\n \n"},{"id":"79316","messageId":"20080610063328.GB26965@diana.vm.bytemark.co.uk","threadId":"13876","inReplyTo":"484D78BF.6030504@gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-10T06:33:28Z","receivedAt":"2008-06-10T06:33:28Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-09 13:38:55 -0500, Clark Williams wrote:\n\n> This patch allows StGit to directly import compressed (.gz and .bz2)\n> files with reasonable patch names.\n>\n> I do a lot of work on modified kernel trees and usually the first\n> two things imported are a stable update patch followed immediately\n> by an -rt patch, both of which are compressed. With this patch I can\n> just copy the files down directly from kernel.org and import them,\n> rather than having to keep uncompressed copies around.\n>\n> Hey, I'm lazy... :)\n\nLazy is good. Thanks for the patch!\n\n> + if filename.endswith(\".gz\"):\n> + import gzip\n> + f = gzip.open(filename)\n> + pname = filename.replace(\".gz\", \"\")\n> + elif filename.endswith(\".bz2\"):\n> + import bz2\n> + f = bz2.BZ2File(filename, 'r')\n> + pname = filename.replace(\".bz2\", \"\")\n\nSome comments here:\n\n * By my reading of the docs, the second argument to BZ2File defaults\n to 'r' anyway, so you could omit it.\n\n * We try to use single quotes wherever possible (except when triple\n quoting). You're using a mix ...\n\n * .replace() will happily replace anywhere in the string. Please\n consider using stgit.util.strip_suffix() instead.\n\nAnd last but not least, it'd be terrific if you'd let me bully you\ninto adding .gz and .bz2 test cases for t1800-import. :-)\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"79318","messageId":"alpine.DEB.1.10.0806092335420.17180@swallowtail","threadId":"13876","inReplyTo":"20080610063328.GB26965@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Asheesh Laroia","fromEmail":"asheesh@asheesh.org","sentAt":"2008-06-10T06:38:04Z","receivedAt":"2008-06-10T06:38:04Z","isPatch":true,"sender":{"key":"asheesh@asheesh.org","avatar":"https://avatars.githubusercontent.com/u/25457?v=4"},"body":"On Tue, 10 Jun 2008, Karl Hasselström wrote:\n\n> On 2008-06-09 13:38:55 -0500, Clark Williams wrote:\n>\n>> This patch allows StGit to directly import compressed (.gz and .bz2)\n>> files with reasonable patch names.\n>>\n>> I do a lot of work on modified kernel trees and usually the first\n>> two things imported are a stable update patch followed immediately\n>> by an -rt patch, both of which are compressed. With this patch I can\n>> just copy the files down directly from kernel.org and import them,\n>> rather than having to keep uncompressed copies around.\n>>\n>> Hey, I'm lazy... :)\n>\n> Lazy is good. Thanks for the patch!\n>\n>> + if filename.endswith(\".gz\"):\n>> + import gzip\n>> + f = gzip.open(filename)\n>> + pname = filename.replace(\".gz\", \"\")\n>> + elif filename.endswith(\".bz2\"):\n>> + import bz2\n>> + f = bz2.BZ2File(filename, 'r')\n>> + pname = filename.replace(\".bz2\", \"\")\n>\n> Some comments here:\n>\n> * By my reading of the docs, the second argument to BZ2File defaults\n> to 'r' anyway, so you could omit it.\n\nPeanut gallery question: Why not just always try these methods and catch \nsome format exception if they fail, proceeding to the next possible \ndecompressor (proceeding on to no decompressor)?\n\nThat way if a file is called .GZ, it will still be handled properly; in \nfact, all files would still be handled properly. And these formats leave \nnotes in the first few bytes of the file as to if they should be tried, so \nit's not as if it would come at some performance cost.\n\n-- Asheesh.\n\n-- \nSay \"twenty-three-skiddoo\" to logout."},{"id":"79320","messageId":"bd6139dc0806100107y415a292eqe22af7a7b2215e65@mail.gmail.com","threadId":"13876","inReplyTo":"alpine.DEB.1.10.0806092335420.17180@swallowtail","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T08:07:25Z","receivedAt":"2008-06-10T08:07:25Z","isPatch":true,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Tue, Jun 10, 2008 at 8:38 AM, Asheesh Laroia wrote:\n> Peanut gallery question: Why not just always try these methods and catch\n> some format exception if they fail, proceeding to the next possible\n> decompressor (proceeding on to no decompressor)?\n>\n> That way if a file is called .GZ, it will still be handled properly; in\n> fact, all files would still be handled properly. And these formats leave\n> notes in the first few bytes of the file as to if they should be tried, so\n> it's not as if it would come at some performance cost.\n\nHow about adding in '.tar' decompression as well, -after- the '.gz'\ndecompression? That way you can have .tar.gz's and still be fine.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"79329","messageId":"20080610095349.GA30119@diana.vm.bytemark.co.uk","threadId":"13876","inReplyTo":"bd6139dc0806100107y415a292eqe22af7a7b2215e65@mail.gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-10T09:53:49Z","receivedAt":"2008-06-10T09:53:49Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-10 10:07:25 +0200, Sverre Rabbelier wrote:\n\n> How about adding in '.tar' decompression as well, -after- the '.gz'\n> decompression? That way you can have .tar.gz's and still be fine.\n\nA tar file would presumably contain more than one patch -- it'd be\nmore like a directory of patches than a single patch file.\n\nI'm not saying it wouldn't be nice to support it, but it's not what\nthe original poster needed, and building it would be a bunch of extra\nwork.\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"79330","messageId":"bd6139dc0806100257k4700d364occ08457279e8a735@mail.gmail.com","threadId":"13876","inReplyTo":"20080610095349.GA30119@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T09:57:05Z","receivedAt":"2008-06-10T09:57:05Z","isPatch":true,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Tue, Jun 10, 2008 at 11:53 AM, Karl Hasselström wrote:\n> On 2008-06-10 10:07:25 +0200, Sverre Rabbelier wrote:\n>\n>> How about adding in '.tar' decompression as well, -after- the '.gz'\n>> decompression? That way you can have .tar.gz's and still be fine.\n>\n> A tar file would presumably contain more than one patch -- it'd be\n> more like a directory of patches than a single patch file.\n\nWhy?\n$ tar czvf mypatch.patch.tar.gz mypatch.patch\n\n> I'm not saying it wouldn't be nice to support it, but it's not what\n> the original poster needed, and building it would be a bunch of extra\n> work.\n\nA bunch of extra work? http://docs.python.org/lib/module-tarfile.html\nI say it'd be about as much work as the original patch ;).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"79333","messageId":"20080610102800.GB30119@diana.vm.bytemark.co.uk","threadId":"13876","inReplyTo":"bd6139dc0806100257k4700d364occ08457279e8a735@mail.gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-10T10:28:00Z","receivedAt":"2008-06-10T10:28:00Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-10 11:57:05 +0200, Sverre Rabbelier wrote:\n\n> On Tue, Jun 10, 2008 at 11:53 AM, Karl Hasselström wrote:\n>\n> > A tar file would presumably contain more than one patch -- it'd be\n> > more like a directory of patches than a single patch file.\n>\n> Why?\n> $ tar czvf mypatch.patch.tar.gz mypatch.patch\n\nIf there's just one patch in the tar file, why did you use a tar file\nin the first place instead of just gzipping?\n\nI'm pretty sure that anyone who really has use for the tar-file\ncapability would be using tar files with multiple patches in them.\n\n> > I'm not saying it wouldn't be nice to support it, but it's not\n> > what the original poster needed, and building it would be a bunch\n> > of extra work.\n>\n> A bunch of extra work?\n> http://docs.python.org/lib/module-tarfile.html I say it'd be about\n> as much work as the original patch ;).\n\nI was refering to the fact that due to tar-files in the general case\ncontaining more than one patch, you'd have to modify the parts of\nimprt.py that deal with importing multiple patches at once, in\naddition to the parts the current patch touches.\n\nBut you're probably right that the amount of additional work would not\nbe much more than what went into the current patch.\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"79334","messageId":"bd6139dc0806100333s4395cca4g52b58679d7fc5c25@mail.gmail.com","threadId":"13876","inReplyTo":"20080610102800.GB30119@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Sverre Rabbelier","fromEmail":"alturin@gmail.com","sentAt":"2008-06-10T10:33:25Z","receivedAt":"2008-06-10T10:33:25Z","isPatch":true,"sender":{"key":"alturin@gmail.com","avatar":null},"body":"On Tue, Jun 10, 2008 at 12:28 PM, Karl Hasselström wrote:\n> If there's just one patch in the tar file, why did you use a tar file\n> in the first place instead of just gzipping?\n\nI guess mostly habbit :P. Whenever I zip something I create a gzipped\ntarball because that's how I usually do it.\n\n> I'm pretty sure that anyone who really has use for the tar-file\n> capability would be using tar files with multiple patches in them.\n\nYeah, I guess that's true for most people indeed.\n\n> I was refering to the fact that due to tar-files in the general case\n> containing more than one patch, you'd have to modify the parts of\n> imprt.py that deal with importing multiple patches at once, in\n> addition to the parts the current patch touches.\n\nMhhh, yeah, but should be something like\nfor patch in patches\n applyPatch(patch)\n\n> But you're probably right that the amount of additional work would not\n> be much more than what went into the current patch.\n\nTo just support .tar, yeah, but let's see what the author has to say\nabout this ;).\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"79342","messageId":"484E877E.801@gmail.com","threadId":"13876","inReplyTo":"20080610063328.GB26965@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-10T13:54:06Z","receivedAt":"2008-06-10T13:54:06Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nKarl Hasselström wrote:\n>> + if filename.endswith(\".gz\"):\n>> + import gzip\n>> + f = gzip.open(filename)\n>> + pname = filename.replace(\".gz\", \"\")\n>> + elif filename.endswith(\".bz2\"):\n>> + import bz2\n>> + f = bz2.BZ2File(filename, 'r')\n>> + pname = filename.replace(\".bz2\", \"\")\n> \n> Some comments here:\n> \n> * By my reading of the docs, the second argument to BZ2File defaults\n> to 'r' anyway, so you could omit it.\n\nDone.\n\n> \n> * We try to use single quotes wherever possible (except when triple\n> quoting). You're using a mix ...\n\nI normally use single quotes too, but I've been doing a bunch of C programming\nlately, so that's my excuse and I'm sticking with it. Replaced.\n\n> \n> * .replace() will happily replace anywhere in the string. Please\n> consider using stgit.util.strip_suffix() instead.\n\nAh, didn't know about strip_suffix(). Done.\n\n> \n> And last but not least, it'd be terrific if you'd let me bully you\n> into adding .gz and .bz2 test cases for t1800-import. :-)\n> \n\nI'll work on that. Can't do it right now, but I'll look at the test harness and see\nwhat it'll take.\n\nClark\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEYEARECAAYFAkhOh34ACgkQqA4JVb61b9ea9gCgoV1MZbT2F62WEkduOfmkgdP3\nBwIAnApT1o+VttF4VRHJj4DkPmi/HXfm\n=Uwho\n-----END PGP SIGNATURE-----\n"},{"id":"79343","messageId":"484E87B2.2090506@gmail.com","threadId":"13876","inReplyTo":"20080610063328.GB26965@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-10T13:54:58Z","receivedAt":"2008-06-10T13:54:58Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nKarl Hasselström wrote:\n> \n> Some comments here:\n> \n> * By my reading of the docs, the second argument to BZ2File defaults\n> to 'r' anyway, so you could omit it.\n> \n> * We try to use single quotes wherever possible (except when triple\n> quoting). You're using a mix ...\n> \n> * .replace() will happily replace anywhere in the string. Please\n> consider using stgit.util.strip_suffix() instead.\n> \n> And last but not least, it'd be terrific if you'd let me bully you\n> into adding .gz and .bz2 test cases for t1800-import. :-)\n> \n\nSigh. Never reply before your second cup of coffee.\n\nUpdated patch attached.\n\nClark\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEYEARECAAYFAkhOh7EACgkQqA4JVb61b9c4BgCeJH0GUzQJCDdJ8gx5287KE/KO\nuTwAoKD7V4JMqHnYmFCg01ij5aBbLDbq\n=Yy/n\n-----END PGP SIGNATURE-----\n\n\nPatch to allow import from compressed (.gz and .bz2) files\n\nFrom: Clark Williams \n\nSigned-off-by: Clark Williams \n---\n\n stgit/commands/imprt.py | 21 +++++++++++++++------\n 1 files changed, 15 insertions(+), 6 deletions(-)\n\ndiff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py\nindex 4a4b792..050301a 100644\n--- a/stgit/commands/imprt.py\n+++ b/stgit/commands/imprt.py\n@@ -178,8 +178,22 @@ def __create_patch(filename, message, author_name, author_email,\n def __import_file(filename, options, patch = None):\n \"\"\"Import a patch from a file or standard input\n \"\"\"\n+ if patch:\n+ pname = patch\n+ else:\n+ pname = filename\n+\n if filename:\n- f = file(filename)\n+ if filename.endswith('.gz'):\n+ import gzip\n+ f = gzip.open(filename)\n+ pname = strip_suffix('.gz', filename)\n+ elif filename.endswith('.bz2'):\n+ import bz2\n+ f = bz2.BZ2File(filename)\n+ pname = strip_suffic('.bz2', filename)\n+ else:\n+ f = file(filename)\n else:\n f = sys.stdin\n \n@@ -197,11 +211,6 @@ def __import_file(filename, options, patch = None):\n if filename:\n f.close()\n \n- if patch:\n- pname = patch\n- else:\n- pname = filename\n-\n __create_patch(pname, message, author_name, author_email,\n author_date, diff, options)\n \n"},{"id":"79344","messageId":"484E8833.9030302@gmail.com","threadId":"13876","inReplyTo":"alpine.DEB.1.10.0806092335420.17180@swallowtail","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-10T13:57:07Z","receivedAt":"2008-06-10T13:57:07Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nAsheesh Laroia wrote:\n> \n> Peanut gallery question: Why not just always try these methods and catch\n> some format exception if they fail, proceeding to the next possible\n> decompressor (proceeding on to no decompressor)?\n> \n> That way if a file is called .GZ, it will still be handled properly; in\n> fact, all files would still be handled properly. And these formats\n> leave notes in the first few bytes of the file as to if they should be\n> tried, so it's not as if it would come at some performance cost.\n\nInteresting thought. Do all the decompressors throw an error if the input format\nisn't recognized?\n\nClark\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEYEARECAAYFAkhOiDMACgkQqA4JVb61b9fixACfZLtxY20tXyZA5oLHTBSm5JPj\nApQAnRsb6RkA2YKB5UXhXiEezMm1j1ZS\n=Y7ZK\n-----END PGP SIGNATURE-----\n"},{"id":"79345","messageId":"484E8947.5070508@gmail.com","threadId":"13876","inReplyTo":"bd6139dc0806100107y415a292eqe22af7a7b2215e65@mail.gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-10T14:01:43Z","receivedAt":"2008-06-10T14:01:43Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nSverre Rabbelier wrote:\n> On Tue, Jun 10, 2008 at 8:38 AM, Asheesh Laroia wrote:\n>> Peanut gallery question: Why not just always try these methods and catch\n>> some format exception if they fail, proceeding to the next possible\n>> decompressor (proceeding on to no decompressor)?\n>>\n>> That way if a file is called .GZ, it will still be handled properly; in\n>> fact, all files would still be handled properly. And these formats leave\n>> notes in the first few bytes of the file as to if they should be tried, so\n>> it's not as if it would come at some performance cost.\n> \n> How about adding in '.tar' decompression as well, -after- the '.gz'\n> decompression? That way you can have .tar.gz's and still be fine.\n> \n\nI thought about this, as well as the .zip format. The problem is in multiple files\nand how to handle them. Do you look for a series and automagically add '--series'\nsemantics? What about ignore/replace? If no series file, do you just try to apply\nall of them? In what order? Look for '.patch' suffix and only use those? And what\nabout the old '.tgz' suffix? Lots of different ways to go here.\n\nI don't think it would be a bad thing to add .tar and .zip handling, but I'd rather\nbat it around a little on this list before doing it. That way we can find out how\npeople would like to use it.\n\nClark\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEYEARECAAYFAkhOiUcACgkQqA4JVb61b9cWawCeK+n8uo8XGGlURYG2ImuhUUNY\neZEAn31ZfjC7MgiP1VZ02uksrRJQiQVe\n=Tu6s\n-----END PGP SIGNATURE-----\n"},{"id":"79347","messageId":"alpine.DEB.1.00.0806100703460.22017@alchemy.localdomain","threadId":"13876","inReplyTo":"484E8833.9030302@gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Asheesh Laroia","fromEmail":"asheesh@asheesh.org","sentAt":"2008-06-10T14:04:09Z","receivedAt":"2008-06-10T14:04:09Z","isPatch":true,"sender":{"key":"asheesh@asheesh.org","avatar":"https://avatars.githubusercontent.com/u/25457?v=4"},"body":"On Tue, 10 Jun 2008, Clark Williams wrote:\n\n> Asheesh Laroia wrote:\n>>\n>> Peanut gallery question: Why not just always try these methods and catch\n>> some format exception if they fail, proceeding to the next possible\n>> decompressor (proceeding on to no decompressor)?\n>>\n>> That way if a file is called .GZ, it will still be handled properly; in\n>> fact, all files would still be handled properly. And these formats\n>> leave notes in the first few bytes of the file as to if they should be\n>> tried, so it's not as if it would come at some performance cost.\n>\n> Interesting thought. Do all the decompressors throw an error if the input format\n> isn't recognized?\n\nYup! (Try it!)\n\n-- Asheesh.\n\n-- \nYou might have mail.\n"},{"id":"79346","messageId":"484E8A6F.1060602@gmail.com","threadId":"13876","inReplyTo":"bd6139dc0806100333s4395cca4g52b58679d7fc5c25@mail.gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-10T14:06:39Z","receivedAt":"2008-06-10T14:06:39Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nSverre Rabbelier wrote:\n> On Tue, Jun 10, 2008 at 12:28 PM, Karl Hasselström wrote:\n>> If there's just one patch in the tar file, why did you use a tar file\n>> in the first place instead of just gzipping?\n> \n> I guess mostly habbit :P. Whenever I zip something I create a gzipped\n> tarball because that's how I usually do it.\n> \n>> I'm pretty sure that anyone who really has use for the tar-file\n>> capability would be using tar files with multiple patches in them.\n> \n> Yeah, I guess that's true for most people indeed.\n> \n>> I was refering to the fact that due to tar-files in the general case\n>> containing more than one patch, you'd have to modify the parts of\n>> imprt.py that deal with importing multiple patches at once, in\n>> addition to the parts the current patch touches.\n> \n> Mhhh, yeah, but should be something like\n> for patch in patches\n> applyPatch(patch)\n> \n\nBut, since patches in general are ordered beasts, you can get into trouble if you\ndidn't name your patches such that they lexically sort in the proper order *and* the\ntar library gives them to you in the right order.\n\nI think if I were going to add this, I'd probably look for a series file and use that\n to apply in the proper order. Same for if we support .zip files. Seems like the only\nother thing you could do is read in the list of files and sort them, then apply them.\n\n>> But you're probably right that the amount of additional work would not\n>> be much more than what went into the current patch.\n> \n> To just support .tar, yeah, but let's see what the author has to say\n> about this ;).\n> \n\nAuthor. Mmmmmmm, much nicer than \"Mad Python patch hacker\". I like it!\n\nClark\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEUEARECAAYFAkhOim8ACgkQqA4JVb61b9eMBACWIoP2sEcIdH8+2R60NN26UB/b\nPQCgozvd0Kkonjz7xJlMN5cJqpVzhkY=\n=UOWy\n-----END PGP SIGNATURE-----\n"},{"id":"79435","messageId":"20080611062753.GB15034@diana.vm.bytemark.co.uk","threadId":"13876","inReplyTo":"484E87B2.2090506@gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-11T06:27:53Z","receivedAt":"2008-06-11T06:27:53Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-10 08:54:58 -0500, Clark Williams wrote:\n\n> --- a/stgit/commands/imprt.py\n> +++ b/stgit/commands/imprt.py\n> @@ -178,8 +178,22 @@ def __create_patch(filename, message, author_name, author_email,\n> def __import_file(filename, options, patch = None):\n> \"\"\"Import a patch from a file or standard input\n> \"\"\"\n> + if patch:\n> + pname = patch\n> + else:\n> + pname = filename\n> +\n> if filename:\n> - f = file(filename)\n> + if filename.endswith('.gz'):\n> + import gzip\n> + f = gzip.open(filename)\n> + pname = strip_suffix('.gz', filename)\n> + elif filename.endswith('.bz2'):\n> + import bz2\n> + f = bz2.BZ2File(filename)\n> + pname = strip_suffic('.bz2', filename)\n ^\nHere's why I keep blathering about tests! In Python, you don't have a\ncompiler to catch these for you ...\n\n> + else:\n> + f = file(filename)\n> else:\n> f = sys.stdin\n> \n> @@ -197,11 +211,6 @@ def __import_file(filename, options, patch = None):\n> if filename:\n> f.close()\n> \n> - if patch:\n> - pname = patch\n> - else:\n> - pname = filename\n> -\n\nI just realized a problem with this that was already present in your\nfirst version: if patch != None, so that you set pname = patch, you\noverwrite pname since you strip the .gz/.bz2 suffixes _later_.\n\nOther than that, it looks good. But you sounded tempted to go with the\nidea of just trying the decompressors rather than go by the suffix? I\nthink that'd be an improvement.\n\nAs for testing, you'd simply make two copies of one of the subtests in\nt1800, where you test .gz- and .bz2-compressed versions of the same\npatch. Should take about five minutes to write.\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"79467","messageId":"48500B28.1050505@gmail.com","threadId":"13876","inReplyTo":"20080611062753.GB15034@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Clark Williams","fromEmail":"clark.williams@gmail.com","sentAt":"2008-06-11T17:28:08Z","receivedAt":"2008-06-11T17:28:08Z","isPatch":true,"sender":{"key":"clark.williams@gmail.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA1\n\nKarl Hasselström wrote:\n> On 2008-06-10 08:54:58 -0500, Clark Williams wrote:\n> \n>> --- a/stgit/commands/imprt.py\n>> +++ b/stgit/commands/imprt.py\n>> @@ -178,8 +178,22 @@ def __create_patch(filename, message, author_name, author_email,\n>> def __import_file(filename, options, patch = None):\n>> \"\"\"Import a patch from a file or standard input\n>> \"\"\"\n>> + if patch:\n>> + pname = patch\n>> + else:\n>> + pname = filename\n>> +\n>> if filename:\n>> - f = file(filename)\n>> + if filename.endswith('.gz'):\n>> + import gzip\n>> + f = gzip.open(filename)\n>> + pname = strip_suffix('.gz', filename)\n>> + elif filename.endswith('.bz2'):\n>> + import bz2\n>> + f = bz2.BZ2File(filename)\n>> + pname = strip_suffic('.bz2', filename)\n> ^\n> Here's why I keep blathering about tests! In Python, you don't have a\n> compiler to catch these for you ...\n> \n\nYup, caught that right after I sent the updated patch :)\n\n>> + else:\n>> + f = file(filename)\n>> else:\n>> f = sys.stdin\n>> \n>> @@ -197,11 +211,6 @@ def __import_file(filename, options, patch = None):\n>> if filename:\n>> f.close()\n>> \n>> - if patch:\n>> - pname = patch\n>> - else:\n>> - pname = filename\n>> -\n> \n> I just realized a problem with this that was already present in your\n> first version: if patch != None, so that you set pname = patch, you\n> overwrite pname since you strip the .gz/.bz2 suffixes _later_.\n> \n\nAh, I didn't realize that patchname overrides all (should have). I'll fix that next\ngo-round.\n\n> Other than that, it looks good. But you sounded tempted to go with the\n> idea of just trying the decompressors rather than go by the suffix? I\n> think that'd be an improvement.\n> \n\nYeah, it's tempting. I'll play with it a bit and see what it would take. Seems it\nwould just take a try/except block with logic to make the patchname right (I still\nwould want to remove a .gz/.bz2 suffix from the patchname).\n\n> As for testing, you'd simply make two copies of one of the subtests in\n> t1800, where you test .gz- and .bz2-compressed versions of the same\n> patch. Should take about five minutes to write.\n> \n\nI'll make sure I have tests to go with the next patch.\n\nClark\n\n\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.9 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org\n\niEYEARECAAYFAkhQCygACgkQqA4JVb61b9fckgCfbjep1oC3WT3hxVSo/8y6/FVM\nO8AAn1GEGjvbwerEY0U0N1UlJC8Lv37R\n=RHjB\n-----END PGP SIGNATURE-----\n"},{"id":"79484","messageId":"20080611191408.GB23670@diana.vm.bytemark.co.uk","threadId":"13876","inReplyTo":"48500B28.1050505@gmail.com","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"Karl Hasselström","fromEmail":"kha@treskal.com","sentAt":"2008-06-11T19:14:08Z","receivedAt":"2008-06-11T19:14:08Z","isPatch":true,"sender":{"key":"kha@treskal.com","avatar":"https://gravatar.com/avatar/f0120c734b5279b345075a28521e1ac66acb20c9913ffe9bf6ae97e53f7f3f13?d=mp&s=160"},"body":"On 2008-06-11 12:28:08 -0500, Clark Williams wrote:\n\n> Seems it would just take a try/except block\n\nYes. The only trick here is to catch the specific type of exception\nyou want, and nothing else. You can easily try it out at an\ninteractive Python prompt and just see what type of exception is\nthrown.\n\n> with logic to make the patchname right (I still would want to remove\n> a .gz/.bz2 suffix from the patchname).\n\nIndeed. (And now that's a bit more work, since you'll want to ignore\ncapitalization -- and you can't be sure the suffix is actually there.)\n\n> I'll make sure I have tests to go with the next patch.\n\nExcellent!\n\n-- \nKarl Hasselström, kha@treskal.com\n www.treskal.com/kalle\n"},{"id":"80377","messageId":"878wx14iga.fsf@lysator.liu.se","threadId":"13876","inReplyTo":"20080610063328.GB26965@diana.vm.bytemark.co.uk","subject":"Re: [PATCH - stgit] Patch to allow import of compressed files","fromName":"David Kågedal","fromEmail":"david@kagedal.org","sentAt":"2008-06-19T14:17:57Z","receivedAt":"2008-06-19T14:17:57Z","isPatch":true,"sender":{"key":"david@kagedal.org","avatar":null},"body":"Karl Hasselström writes:\n\n> On 2008-06-09 13:38:55 -0500, Clark Williams wrote:\n>\n>> This patch allows StGit to directly import compressed (.gz and .bz2)\n>> files with reasonable patch names.\n>>\n>> I do a lot of work on modified kernel trees and usually the first\n>> two things imported are a stable update patch followed immediately\n>> by an -rt patch, both of which are compressed. With this patch I can\n>> just copy the files down directly from kernel.org and import them,\n>> rather than having to keep uncompressed copies around.\n>>\n>> Hey, I'm lazy... :)\n>\n> Lazy is good. Thanks for the patch!\n>\n>> + if filename.endswith(\".gz\"):\n>> + import gzip\n>> + f = gzip.open(filename)\n>> + pname = filename.replace(\".gz\", \"\")\n>> + elif filename.endswith(\".bz2\"):\n>> + import bz2\n>> + f = bz2.BZ2File(filename, 'r')\n>> + pname = filename.replace(\".bz2\", \"\")\n>\n> Some comments here:\n>\n> * By my reading of the docs, the second argument to BZ2File defaults\n> to 'r' anyway, so you could omit it.\n>\n> * We try to use single quotes wherever possible (except when triple\n> quoting). You're using a mix ...\n>\n> * .replace() will happily replace anywhere in the string. Please\n> consider using stgit.util.strip_suffix() instead.\n\nOr use os.path.splitext(filename) which will save you a couple of\nendswith calls as well.\n\n> And last but not least, it'd be terrific if you'd let me bully you\n> into adding .gz and .bz2 test cases for t1800-import. :-)\n\n-- \nDavid Kågedal \n"}]}