Git and OpenDocument (OpenOffice.org) files

Hi,

I found a way to use git comfortably with OpenDocument files (that is, what OpenOffice.org and Koffice produce. Text, Presentations and Spreadsheets).

Briefly, you have to install odf2txt ( http://stosberg.net/odt2txt/ ) and the script below, together with GIT_EXTERNAL_DIFF and/or diff drivers in .gitattributes. That give you the text diff you're used to.

Everything is documented here:

  http://www-verimag.imag.fr/~moy/opendocument/

Remarks are welcome (I'll post some remarks about Git's custom diff driver in a separate thread).

Script available from http://www-verimag.imag.fr/~moy/opendocument/git-oodiff and reproduced here :

#! /bin/sh

# Script acceptable as a value for GIT_EXTERNAL_DIFF. # For example, you can see the changes in your working tree with # # $ GIT_EXTERNAL_DIFF=git-oodiff diff

echo $0 "$@"

if odt2txt "$2"  > /tmp/oodiff.$$.1  && \
    odt2txt "$5" > /tmp/oodiff.$$.2 ; then
    if diff -L "a/$1" -L "b/$1" -u /tmp/oodiff.$$.{1,2}; then
        # no text change
        if diff -q "$2" "$5"; then
            : # no change at all
        else
            echo "OpenDocument files a/$1 and b/$1 files differ (same text content)"
        fi
    fi
else
    # conversion failed. Fall back to plain diff.
    diff -L "a/$1" -L "b/$1" -u "$2" "$5"
fi

rm -f /tmp/oodiff.$$.{1,2}

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

Good.

I think creation/deletion will get /dev/null as the temporary file name, so as long as odt2txt knows how to deal with /dev/null you would not have to worry much about them.

You might want to be careful about unmerged paths, though. They will not get anything other than $1 (name).

You would probably not care about the mode changes for oo documents, but they are available as $4 and $6 respectively, if you care.

Junio C Hamano <gitster@pobox.com> writes:

But odt2txt doesn't know how to deal with /dev/null. New version of git-oodiff that manages it correctly online and below.

I don't know how to manage this correctly, so I just display a message "Unmerged path $1" and die.

I don't care, but the new version still manages them ;-).

All this convince me that the ability to provide a plaintext converter (see the other thread I started) would make it much simpler to write such kind of things. The mode change, for example, could be managed automatically by git, I wouldn't need to write my own 'echo "new mode ..."'.

Thanks for the advices.

On Mon, Aug 27, 2007 at 02:35:14PM +0200, Matthieu Moy <Matthieu.Moy@imag.fr> wrote:

BTW, wouldn't it be more efficient to store the odf files unzipped ?

Mike

Hi,

On Mon, 27 Aug 2007, Mike Hommey wrote:

Efficient how?  Speed-wise: no.  Space-wise: yes.

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

Huh? At least the "Space-wise: yes" seems rather nonsensical. "Speed-wise" is not as clear: it depends on the relation between memory/disk bandwidth and decompression speed.

David Kastrup <dak@gnu.org> writes:

I don't know enough about git delta-compression and OpenDocument, but git has better chance to efficiently delta-compress different versions of the document if they're not compressed themselves.

(but that's a necessary and not sufficient condition. line-based delta-compression wouldn't work if the file is a one-line XML file for example).

Probably network operations would be faster, and checkout would be slower. I wouldn't bet ;-).

Hi,

On Mon, 27 Aug 2007, Matthieu Moy wrote:

A standalone zip archive (which is what an .odt file is, with a defined file structure) cannot be as efficient in compressing text, especially if it is versioned text with relatively few differences between versions, as delta compression.

So yes, you guessed the explanation (which I omitted) correctly.

As for the speed wise: I doubt that unpacking and then repacking can be more efficient than not doing it -- even if the files are transmitted via network. (Remember: blobs are stored compressed, be they in a pack, or loose.)

Ciao, Dscho

On Mon, Aug 27, 2007 at 03:58:04PM +0200, David Kastrup <dak@gnu.org> wrote:

A zipped file will be 100% different at each revision. The unzipped counterpart may be similar for 90% or more between revisions.

Mike

Mike Hommey <mh <at> glandium.org> writes:

In my (modest) experience, not really:

in fact, odf files are a zip collection of many individual files (for instance if you have an impress presentation, the zip collection will contain all the images that appear in the presentation...)

Now: zip is different from .tar.gz in that tar.gz first concatenates the
files and then compresses the overall thing, while zip compresses or stores
the individual files and then concatenates and indexes the result.

The difference is that in a tar.gz file, changing a single byte in one of the internal files can lead to a completely different compressed stream, while in a zip file, changing an internal file only affects the relevant part of the zipped file.

This means that:
- if you have an odf document containing lots of internal objects (e.g.
images) that do not change very much from version to version, git can make
very good deltas.
- conversely if you have an odf document whose size is dominated by proper
content, then git will not be able to make good deltas.

As an example, I am finding that impress presentations (dominated by images) can delta very well, while calc spreadsheets (dominated by content) do not.

Probably it could be nice to make a filter that takes an odf file and re-zips it so that the content.xml inner file is only stored, rather than deflated. Then this could be used with the git file filtering machinery.

Sergio

Hi,

On Mon, 27 Aug 2007, Matthieu Moy wrote:

Heh. I had that problem, too. I added an attribute "*.odt diff=odt" and the diff driver unpacks the zip and executes an xmldiff on the files. Since at times, it is more interesting to do a word based diff, depending on the environment variable WORDDIFF, my diff driver executes "git diff --color-words" instead.

Ciao, Dscho

Mike Hommey <mh@glandium.org> writes:

Ah, right.

This applies however to gzipped files or single-file zip files, and not zipped files in general: a zip file compresses each file individually, so unchanged single files inside of the zip will deltify reasonably well, as opposed to unchanged single files in a .tar.gz file.

But that's a minor point not relevant here, and of course you are right. I just somehow did not register that "store the odf files" was supposed to mean "get checked into git in numerous versions".