threads / patch / 13918

patchcompressed import v3

Subject: [StGit PATCH] compressed import v3

## tl;dr

5 messages between Jun 12, 2008 and Jun 14, 2008. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

Clark Williams· Jun 12, 2008, 21:32 UTC · lore
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Karl/Catalin, et al,

Attached is my latest stab at StGit importing patches from compressed files. This version doesn't try to differentiate by the file extension; it just tries to open gzip or bz2 files and if those fail it reverts to text. I'm not completely happy with it, but I've spent about as much time on it as I can afford to (for this week anyway).

Yeah, yeah, Karl, there are four new tests in t1800-import. Let me know if you think there should be more (more for compressed input that is; I'm not crazy enough to sign up to write more tests for *everything*). :)

Clark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkhRlgIACgkQqA4JVb61b9d2ygCfTUPc5I9eXM4947VTrTZ+mO0H +vIAoJJACG94TdnyUIac73lB4UYCZVlG =sa3t -----END PGP SIGNATURE-----

Patch to allow import from compressed files (gzip and bzip2)
From: Clark Williams <williams@redhat.com>
Signed-off-by: Clark Williams <williams@redhat.com>
---
 stgit/commands/imprt.py |   42 ++++++++++++++++++++++++++++++++++++------
 t/t1800-import.sh       |   42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 78 insertions(+), 6 deletions(-)
Show changes to 2 files +78 −6

stgit/commands/imprt.py, t/t1800-import.sh

diff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py
index 4a4b792..fc5cdce 100644
--- a/stgit/commands/imprt.py
+++ b/stgit/commands/imprt.py
@@ -175,14 +175,49 @@ def __create_patch(filename, message, author_name, author_email,
                                  backup = False)
         out.done()
 
+def __mkpatchname(name, suffix):
+    if name.lower().endswith(suffix.lower()):
+        return name[:-len(suffix)]
+    return name
+
+def __gethandleandname(filename):
+    """return a file handle and a patch name derived from filename
+    """
+    # see if it's a gzip'ed patch
+    try:
+        import gzip
+        f = gzip.open(filename)
+        f.read(1)
+        f.seek(0)
+        return (f, __mkpatchname(filename, '.gz'))
+    except IOError, e:
+            pass
+    # see if it's a bzip2'ed patch
+    try:
+        import bz2
+        f = bz2.BZ2File(filename)
+        f.read(1)
+        f.seek(0)
+        return (f, __mkpatchname(filename, '.bz2'))
+    except IOError, e:
+            pass
+    # plain old file...
+    return (open(filename), filename)
+
 def __import_file(filename, options, patch = None):
     """Import a patch from a file or standard input
     """
+    pname = None
     if filename:
-        f = file(filename)
+        (f, pname) = __gethandleandname(filename)
     else:
         f = sys.stdin
 
+    if patch:
+        pname = patch
+    elif not pname:
+        pname = filename
+
     if options.mail:
         try:
             msg = email.message_from_file(f)
@@ -197,11 +232,6 @@ def __import_file(filename, options, patch = None):
     if filename:
         f.close()
 
-    if patch:
-        pname = patch
-    else:
-        pname = filename
-
     __create_patch(pname, message, author_name, author_email,
                    author_date, diff, options)
 
diff --git a/t/t1800-import.sh b/t/t1800-import.sh
index 8c8c9a0..1352743 100755
--- a/t/t1800-import.sh
+++ b/t/t1800-import.sh
@@ -80,4 +80,46 @@ test_expect_success \
     stg delete ..
     '
 
+test_expect_success \
+    'Apply a bzip2 patch created with "git diff"' \
+    '
+    bzip2 -c ../t1800-import/git-diff >../t1800-import/bzip2-git-diff &&
+    stg import ../t1800-import/bzip2-git-diff &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") = 1 ] &&
+    rm ../t1800-import/bzip2-git-diff &&
+    stg delete .. 
+    '
+test_expect_success \
+    'Apply a bzip2 patch with a .bz2 suffix' \
+    '
+    bzip2 -c ../t1800-import/git-diff >../t1800-import/git-diff.bz2 &&
+    stg import ../t1800-import/git-diff.bz2 &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") = 1 ] &&
+    rm ../t1800-import/git-diff.bz2 &&
+    stg delete .. 
+    '
+
+test_expect_success \
+    'Apply a gzip patch created with GNU diff' \
+    '
+    gzip -c ../t1800-import/gnu-diff >../t1800-import/gzip-gnu-diff &&
+    stg import ../t1800-import/gzip-gnu-diff &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") = 1 ] &&
+    rm ../t1800-import/gzip-gnu-diff &&
+    stg delete ..
+    '
+test_expect_success \
+    'Apply a gzip patch with a .gz suffix' \
+    '
+    gzip -c ../t1800-import/gnu-diff >../t1800-import/gnu-diff.gz &&
+    stg import ../t1800-import/gnu-diff.gz &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") = 1 ] &&
+    rm ../t1800-import/gnu-diff.gz &&
+    stg delete ..
+    '
+
 test_done
Karl Hasselström· Jun 13, 2008, 05:11 UTC · re: Clark Williams · lore

Re: [StGit PATCH] compressed import v3

On 2008-06-12 16:32:50 -0500, Clark Williams wrote:
> Attached is my latest stab at StGit importing patches from
> compressed files. This version doesn't try to differentiate by the
> file extension; it just tries to open gzip or bz2 files and if those
> fail it reverts to text.
Very good!
> I'm not completely happy with it, but I've spent about as much time
> on it as I can afford to (for this week anyway).

Heh. I see an opportunity for condensing the code a bit in __gethandleandname(), and you intented the "pass"es funny, but I'll take care of it (or not, seeing as they're kind of minor nits -- in any case, I'll just take the patch off your hands and stop tormenting you about it).

> Yeah, yeah, Karl, there are four new tests in t1800-import. Let me
> know if you think there should be more (more for compressed input
> that is; I'm not crazy enough to sign up to write more tests for
> *everything*). :)
No, this is exactly what I was aiming for.
Bleed 'em dry, but _no more_! ;-)
-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Clark Williams· Jun 13, 2008, 14:53 UTC · re: Karl Hasselström · lore

Re: [StGit PATCH] compressed import v3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Karl Hasselström wrote:
Show 18 quoted lines
> On 2008-06-12 16:32:50 -0500, Clark Williams wrote:
> 
>> Attached is my latest stab at StGit importing patches from
>> compressed files. This version doesn't try to differentiate by the
>> file extension; it just tries to open gzip or bz2 files and if those
>> fail it reverts to text.
> 
> Very good!
> 
>> I'm not completely happy with it, but I've spent about as much time
>> on it as I can afford to (for this week anyway).
> 
> Heh. I see an opportunity for condensing the code a bit in
> __gethandleandname(), and you intented the "pass"es funny, but I'll
> take care of it (or not, seeing as they're kind of minor nits -- in
> any case, I'll just take the patch off your hands and stop tormenting
> you about it).
> 

Ugh, I missed that on the pass'es. I had some other logic in there, attempting to make sure that the IOError was in fact because the input was uncompressed, but I punted that and forgot to unindent the pass. I can't believe emacs didn't catch that for me :)

I'm interested to hear how you'd condense the code in __gethandleandname().
Clark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkhSif0ACgkQqA4JVb61b9dA1gCeMVvcY4GobZLp+k7qskLVgozN vsAAmQFoTxf2xxohyzCnCrRQRs1Gdf+O =lBEr -----END PGP SIGNATURE-----

Karl Hasselström· Jun 14, 2008, 10:26 UTC · re: Clark Williams · lore

Re: [StGit PATCH] compressed import v3

On 2008-06-13 09:53:49 -0500, Clark Williams wrote:
> I'm interested to hear how you'd condense the code in
> __gethandleandname().
Like this, for example:
    import bz2, gzip
    for copen, ext in [(gzip.open, '.gz'), (bz2.BZ2File, '.bz2')]:
        try:
            f = copen(filename)
            f.read(1)
            f.seek(0)
            return (f, __mkpatchname(filename, ext))
        except IOError, e:
            pass

If you don't mind, I'll just fix that up directly in your patch. (I also took the liberty to rename the function to __get_handle_and_name, since that's more consistent with other functions in that file.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Clark Williams· Jun 14, 2008, 15:49 UTC · re: Karl Hasselström · lore

Re: [StGit PATCH] compressed import v3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Karl Hasselström wrote:
Show 21 quoted lines
> On 2008-06-13 09:53:49 -0500, Clark Williams wrote:
> 
>> I'm interested to hear how you'd condense the code in
>> __gethandleandname().
> 
> Like this, for example:
> 
>     import bz2, gzip
>     for copen, ext in [(gzip.open, '.gz'), (bz2.BZ2File, '.bz2')]:
>         try:
>             f = copen(filename)
>             f.read(1)
>             f.seek(0)
>             return (f, __mkpatchname(filename, ext))
>         except IOError, e:
>             pass
> 
> If you don't mind, I'll just fix that up directly in your patch. (I
> also took the liberty to rename the function to __get_handle_and_name,
> since that's more consistent with other functions in that file.)
> 
Ah, that's very nice. Your changes work for me.
Clark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkhT6G8ACgkQqA4JVb61b9cCeQCfedD0BjOeRa3sbGvkRNe2BPMq 6roAoJE7klMbgymRCJ3B+pEjzgnTWO2l =uZ+n -----END PGP SIGNATURE-----

← back to recent threads