threads / discuss / 33026

clean/smudge filters on .zip/.tgz files

Subject: clean/smudge filters on .zip/.tgz files

## tl;dr

3 messages between Feb 26, 2013 and Feb 27, 2013.

replies: 2people: 3as markdown or json

Tim Chase· Feb 26, 2013, 22:38 UTC · lore

Various programs that I use ([Open|Libre]Office, Vym, etc) use a zipped/.tgz'ed file format, usually containing multiple (usually) plain-text files within.

I'm trying to figure out a way for git to treat these as virtual directories for purposes of merging/diffing.

Reading up on clean/smudge filters, it looks like they expect one file coming in and one file going out, rather than one file on one side and a directory-tree of files on the other side.

I tried creating my own pair of clean/smudge filters that would uncompress the files, but there's no good way put multiple files on stdout.

Has anybody else played with such a scheme for uncompressing files as they go into git and recompressing them as they come back out?

-tkc
Johannes Sixt· Feb 27, 2013, 06:39 UTC · re: Tim Chase · lore

Re: clean/smudge filters on .zip/.tgz files

Am 2/26/2013 23:38, schrieb Tim Chase:
Show 17 quoted lines
> Various programs that I use ([Open|Libre]Office, Vym, etc) use a
> zipped/.tgz'ed file format, usually containing multiple
> (usually) plain-text files within.
> 
> I'm trying to figure out a way for git to treat these as virtual
> directories for purposes of merging/diffing.  
> 
> Reading up on clean/smudge filters, it looks like they expect one
> file coming in and one file going out, rather than one file
> on one side and a directory-tree of files on the other side.
> 
> I tried creating my own pair of clean/smudge filters that would
> uncompress the files, but there's no good way put multiple files on
> stdout.
> 
> Has anybody else played with such a scheme for uncompressing files as
> they go into git and recompressing them as they come back out?

I attempted to do something like this for OpenDocument files (I didn't get very far) until I discovered that LibreOffice can save "flat open document files". That combined with the option "save files optimized" switched off results in fairly readable XML in a single file that can even be merged under some circumstances.

You would still need a clean filter that normalizes the style numbers, cross reference marks and other stuff that changes each time LibreOffice saves the file.

-- Hannes
Michael J Gruber· Feb 27, 2013, 15:18 UTC · re: Johannes Sixt · lore

Re: clean/smudge filters on .zip/.tgz files

Johannes Sixt venit, vidit, dixit 27.02.2013 07:39:
Show 31 quoted lines
> Am 2/26/2013 23:38, schrieb Tim Chase:
>> Various programs that I use ([Open|Libre]Office, Vym, etc) use a
>> zipped/.tgz'ed file format, usually containing multiple
>> (usually) plain-text files within.
>>
>> I'm trying to figure out a way for git to treat these as virtual
>> directories for purposes of merging/diffing.  
>>
>> Reading up on clean/smudge filters, it looks like they expect one
>> file coming in and one file going out, rather than one file
>> on one side and a directory-tree of files on the other side.
>>
>> I tried creating my own pair of clean/smudge filters that would
>> uncompress the files, but there's no good way put multiple files on
>> stdout.
>>
>> Has anybody else played with such a scheme for uncompressing files as
>> they go into git and recompressing them as they come back out?
> 
> I attempted to do something like this for OpenDocument files (I didn't get
> very far) until I discovered that LibreOffice can save "flat open document
> files". That combined with the option "save files optimized" switched off
> results in fairly readable XML in a single file that can even be merged
> under some circumstances.
> 
> You would still need a clean filter that normalizes the style numbers,
> cross reference marks and other stuff that changes each time LibreOffice
> saves the file.
> 
> -- Hannes
> 

In general, using "zip -0" is a good way of getting something that delta-compresses well and that can give a meaningful diff (and has no information loss).

The (my) problem is that recompressing a zip archive (i.e. multi-file) is a pita, you can't just use a pipe "unzip | zip -0". You'd have to do that in a temp dir.

Michael

← back to recent threads