{"thread":{"id":"9673","subject":"Git and OpenDocument (OpenOffice.org) files","startedAt":"2007-08-27T09:52:22Z","lastAt":"2007-08-27T15:16:28Z","messageCount":12,"participants":["Matthieu Moy","Junio C Hamano","Johannes Schindelin","Mike Hommey","David Kastrup","Sergio Callegari"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"51668","messageId":"vpqk5rh5mp5.fsf@bauges.imag.fr","threadId":"9673","inReplyTo":null,"subject":"Git and OpenDocument (OpenOffice.org) files","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-08-27T09:52:22Z","receivedAt":"2007-08-27T09:52:22Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Hi,\n\nI found a way to use git comfortably with OpenDocument files (that is,\nwhat OpenOffice.org and Koffice produce. Text, Presentations and\nSpreadsheets).\n\nBriefly, you have to install odf2txt ( http://stosberg.net/odt2txt/ )\nand the script below, together with GIT_EXTERNAL_DIFF and/or diff\ndrivers in .gitattributes. That give you the text diff you're used to.\n\nEverything is documented here:\n\n http://www-verimag.imag.fr/~moy/opendocument/\n\nRemarks are welcome (I'll post some remarks about Git's custom diff\ndriver in a separate thread).\n\n\nScript available from\nhttp://www-verimag.imag.fr/~moy/opendocument/git-oodiff and reproduced\nhere :\n\n#! /bin/sh\n\n# Script acceptable as a value for GIT_EXTERNAL_DIFF.\n# For example, you can see the changes in your working tree with\n# \n# $ GIT_EXTERNAL_DIFF=git-oodiff diff\n\necho $0 \"$@\"\n\nif odt2txt \"$2\" > /tmp/oodiff.$$.1 && \\\n odt2txt \"$5\" > /tmp/oodiff.$$.2 ; then\n if diff -L \"a/$1\" -L \"b/$1\" -u /tmp/oodiff.$$.{1,2}; then\n # no text change\n if diff -q \"$2\" \"$5\"; then\n : # no change at all\n else\n echo \"OpenDocument files a/$1 and b/$1 files differ (same text content)\"\n fi\n fi\nelse\n # conversion failed. Fall back to plain diff.\n diff -L \"a/$1\" -L \"b/$1\" -u \"$2\" \"$5\"\nfi\n\nrm -f /tmp/oodiff.$$.{1,2}\n"},{"id":"51676","messageId":"7vps19jnm1.fsf@gitster.siamese.dyndns.org","threadId":"9673","inReplyTo":"vpqk5rh5mp5.fsf@bauges.imag.fr","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-08-27T10:08:54Z","receivedAt":"2007-08-27T10:08:54Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy writes:\n\n> Remarks are welcome (I'll post some remarks about Git's custom diff\n> driver in a separate thread).\n\nGood.\n\nI think creation/deletion will get /dev/null as the temporary\nfile name, so as long as odt2txt knows how to deal with\n/dev/null you would not have to worry much about them.\n\nYou might want to be careful about unmerged paths, though. They\nwill not get anything other than $1 (name).\n\nYou would probably not care about the mode changes for oo\ndocuments, but they are available as $4 and $6 respectively, if\nyou care.\n"},{"id":"51678","messageId":"Pine.LNX.4.64.0708271109560.28586@racer.site","threadId":"9673","inReplyTo":"vpqk5rh5mp5.fsf@bauges.imag.fr","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-08-27T10:17:28Z","receivedAt":"2007-08-27T10:17:28Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 27 Aug 2007, Matthieu Moy wrote:\n\n> I found a way to use git comfortably with OpenDocument files (that is, \n> what OpenOffice.org and Koffice produce. Text, Presentations and \n> Spreadsheets).\n\nHeh. I had that problem, too. I added an attribute \"*.odt diff=odt\" and \nthe diff driver unpacks the zip and executes an xmldiff on the files. \nSince at times, it is more interesting to do a word based diff, depending \non the environment variable WORDDIFF, my diff driver executes \"git diff \n--color-words\" instead.\n\nCiao,\nDscho\n"},{"id":"51696","messageId":"vpqtzql17gd.fsf@bauges.imag.fr","threadId":"9673","inReplyTo":"7vps19jnm1.fsf@gitster.siamese.dyndns.org","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-08-27T12:35:14Z","receivedAt":"2007-08-27T12:35:14Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"Junio C Hamano writes:\n\n> Matthieu Moy writes:\n>\n>> Remarks are welcome (I'll post some remarks about Git's custom diff\n>> driver in a separate thread).\n>\n> Good.\n>\n> I think creation/deletion will get /dev/null as the temporary\n> file name, so as long as odt2txt knows how to deal with\n> /dev/null you would not have to worry much about them.\n\nBut odt2txt doesn't know how to deal with /dev/null. New version of\ngit-oodiff that manages it correctly online and below.\n\n> You might want to be careful about unmerged paths, though. They\n> will not get anything other than $1 (name).\n\nI don't know how to manage this correctly, so I just display a message\n\"Unmerged path $1\" and die.\n\n> You would probably not care about the mode changes for oo\n> documents, but they are available as $4 and $6 respectively, if\n> you care.\n\nI don't care, but the new version still manages them ;-).\n\nAll this convince me that the ability to provide a plaintext converter\n(see the other thread I started) would make it much simpler to write\nsuch kind of things. The mode change, for example, could be managed\nautomatically by git, I wouldn't need to write my own 'echo \"new\nmode ...\"'.\n\nThanks for the advices.\n\n-- \nMatthieu\n\nhttp://www-verimag.imag.fr/~moy/opendocument/git-oodiff\n\n#! /bin/sh\n\n# Script acceptable as a value for GIT_EXTERNAL_DIFF.\n# For example, you can see the changes in your working tree with\n# \n# $ GIT_EXTERNAL_DIFF=git-oodiff diff\n\nconvert_to_txt ()\n{\n if [ x\"$1\" = x\"/dev/null\" ]; then\n printf \"\" > /tmp/oodiff.$$.\"$2\"\n eval \"label$2=/dev/null/\"\n else\n odt2txt \"$1\" > /tmp/oodiff.$$.\"$2\" 2>/dev/null\n fi\n}\n\necho $(basename $0) \"$2\" \"$5\"\n\nif [ \"$#\" = \"1\" ]; then\n echo \"Unmerged path $1\"\n exit 0\nfi\n\nif [ x\"$4\" = x\".\" ]; then\n echo \"new file mode $7\"\nelif [ x\"$7\" = x\".\" ]; then\n echo \"deleted file mode $4\"\nelif [ x\"$4\" != x\"$7\" ]; then\n echo \"old mode $4\"\n echo \"new mode $7\"\nfi\n\nlabel1=\"a/$1\"\nlabel2=\"a/$1\"\n\nif convert_to_txt \"$2\" \"1\" &&\n convert_to_txt \"$5\" \"2\" ; then\n if diff -L \"$label1\" -L \"$label2\" -u /tmp/oodiff.$$.{1,2}; then\n # no text change\n if diff -q \"$2\" \"$5\"; then\n : # no change at all\n else\n echo \"OpenDocument files a/$1 and b/$1 files differ (same text content)\"\n fi\n fi\nelse\n # conversion failed. Fall back to plain diff.\n diff -L \"$label1\" -L \"$label2\" -u \"$2\" \"$5\"\nfi\n\nrm -f /tmp/oodiff.$$.{1,2}\n"},{"id":"51697","messageId":"20070827130346.GA8821@glandium.org","threadId":"9673","inReplyTo":"vpqtzql17gd.fsf@bauges.imag.fr","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2007-08-27T13:03:46Z","receivedAt":"2007-08-27T13:03:46Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Mon, Aug 27, 2007 at 02:35:14PM +0200, Matthieu Moy wrote:\n> Junio C Hamano writes:\n> \n> > Matthieu Moy writes:\n> >\n> >> Remarks are welcome (I'll post some remarks about Git's custom diff\n> >> driver in a separate thread).\n> >\n> > Good.\n> >\n> > I think creation/deletion will get /dev/null as the temporary\n> > file name, so as long as odt2txt knows how to deal with\n> > /dev/null you would not have to worry much about them.\n> \n> But odt2txt doesn't know how to deal with /dev/null. New version of\n> git-oodiff that manages it correctly online and below.\n\nBTW, wouldn't it be more efficient to store the odf files unzipped ?\n\nMike\n"},{"id":"51706","messageId":"Pine.LNX.4.64.0708271440500.28586@racer.site","threadId":"9673","inReplyTo":"20070827130346.GA8821@glandium.org","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-08-27T13:41:13Z","receivedAt":"2007-08-27T13:41:13Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 27 Aug 2007, Mike Hommey wrote:\n\n> On Mon, Aug 27, 2007 at 02:35:14PM +0200, Matthieu Moy wrote:\n> > Junio C Hamano writes:\n> > \n> > > Matthieu Moy writes:\n> > >\n> > >> Remarks are welcome (I'll post some remarks about Git's custom diff\n> > >> driver in a separate thread).\n> > >\n> > > Good.\n> > >\n> > > I think creation/deletion will get /dev/null as the temporary\n> > > file name, so as long as odt2txt knows how to deal with\n> > > /dev/null you would not have to worry much about them.\n> > \n> > But odt2txt doesn't know how to deal with /dev/null. New version of\n> > git-oodiff that manages it correctly online and below.\n> \n> BTW, wouldn't it be more efficient to store the odf files unzipped ?\n\nEfficient how? Speed-wise: no. Space-wise: yes.\n\nCiao,\nDscho\n"},{"id":"51711","messageId":"866431rser.fsf@lola.quinscape.zz","threadId":"9673","inReplyTo":"Pine.LNX.4.64.0708271440500.28586@racer.site","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-08-27T13:58:04Z","receivedAt":"2007-08-27T13:58:04Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Johannes Schindelin writes:\n\n> On Mon, 27 Aug 2007, Mike Hommey wrote:\n>\n>> BTW, wouldn't it be more efficient to store the odf files unzipped ?\n>\n> Efficient how? Speed-wise: no. Space-wise: yes.\n\nHuh? At least the \"Space-wise: yes\" seems rather nonsensical.\n\"Speed-wise\" is not as clear: it depends on the relation between\nmemory/disk bandwidth and decompression speed.\n\n-- \nDavid Kastrup\n"},{"id":"51712","messageId":"vpqr6lpvzqc.fsf@bauges.imag.fr","threadId":"9673","inReplyTo":"866431rser.fsf@lola.quinscape.zz","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-08-27T14:06:19Z","receivedAt":"2007-08-27T14:06:19Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"David Kastrup writes:\n\n> Johannes Schindelin writes:\n>\n>> On Mon, 27 Aug 2007, Mike Hommey wrote:\n>>\n>>> BTW, wouldn't it be more efficient to store the odf files unzipped ?\n>>\n>> Efficient how? Speed-wise: no. Space-wise: yes.\n>\n> Huh? At least the \"Space-wise: yes\" seems rather nonsensical.\n\nI don't know enough about git delta-compression and OpenDocument, but\ngit has better chance to efficiently delta-compress different versions\nof the document if they're not compressed themselves.\n\n(but that's a necessary and not sufficient condition. line-based\ndelta-compression wouldn't work if the file is a one-line XML file for\nexample).\n\n> \"Speed-wise\" is not as clear: it depends on the relation between\n> memory/disk bandwidth and decompression speed.\n\nProbably network operations would be faster, and checkout would be\nslower. I wouldn't bet ;-).\n\n-- \nMatthieu\n"},{"id":"51713","messageId":"Pine.LNX.4.64.0708271510060.28586@racer.site","threadId":"9673","inReplyTo":"vpqr6lpvzqc.fsf@bauges.imag.fr","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-08-27T14:15:21Z","receivedAt":"2007-08-27T14:15:21Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 27 Aug 2007, Matthieu Moy wrote:\n\n> David Kastrup writes:\n> \n> > Johannes Schindelin writes:\n> >\n> >> On Mon, 27 Aug 2007, Mike Hommey wrote:\n> >>\n> >>> BTW, wouldn't it be more efficient to store the odf files unzipped ?\n> >>\n> >> Efficient how? Speed-wise: no. Space-wise: yes.\n> >\n> > Huh? At least the \"Space-wise: yes\" seems rather nonsensical.\n> \n> I don't know enough about git delta-compression and OpenDocument, but \n> git has better chance to efficiently delta-compress different versions \n> of the document if they're not compressed themselves.\n\nA standalone zip archive (which is what an .odt file is, with a defined \nfile structure) cannot be as efficient in compressing text, especially if \nit is versioned text with relatively few differences between versions, as \ndelta compression.\n\nSo yes, you guessed the explanation (which I omitted) correctly.\n\nAs for the speed wise: I doubt that unpacking and then repacking can be \nmore efficient than not doing it -- even if the files are transmitted via \nnetwork. (Remember: blobs are stored compressed, be they in a pack, or \nloose.)\n\nCiao,\nDscho\n"},{"id":"51714","messageId":"20070827141600.GA11000@glandium.org","threadId":"9673","inReplyTo":"866431rser.fsf@lola.quinscape.zz","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2007-08-27T14:16:00Z","receivedAt":"2007-08-27T14:16:00Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Mon, Aug 27, 2007 at 03:58:04PM +0200, David Kastrup wrote:\n> Johannes Schindelin writes:\n> \n> > On Mon, 27 Aug 2007, Mike Hommey wrote:\n> >\n> >> BTW, wouldn't it be more efficient to store the odf files unzipped ?\n> >\n> > Efficient how? Speed-wise: no. Space-wise: yes.\n> \n> Huh? At least the \"Space-wise: yes\" seems rather nonsensical.\n\nA zipped file will be 100% different at each revision.\nThe unzipped counterpart may be similar for 90% or more between revisions.\n\nMike\n"},{"id":"51716","messageId":"86bqctqap6.fsf@lola.quinscape.zz","threadId":"9673","inReplyTo":"200708271416\u000400.GA11000@glandium.org","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2007-08-27T15:05:57Z","receivedAt":"2007-08-27T15:05:57Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Mike Hommey writes:\n\n> On Mon, Aug 27, 2007 at 03:58:04PM +0200, David Kastrup wrote:\n>> Johannes Schindelin writes:\n>> \n>> > On Mon, 27 Aug 2007, Mike Hommey wrote:\n>> >\n>> >> BTW, wouldn't it be more efficient to store the odf files unzipped ?\n>> >\n>> > Efficient how? Speed-wise: no. Space-wise: yes.\n>> \n>> Huh? At least the \"Space-wise: yes\" seems rather nonsensical.\n>\n> A zipped file will be 100% different at each revision.\n> The unzipped counterpart may be similar for 90% or more between revisions.\n\nAh, right.\n\nThis applies however to gzipped files or single-file zip files, and\nnot zipped files in general: a zip file compresses each file\nindividually, so unchanged single files inside of the zip will deltify\nreasonably well, as opposed to unchanged single files in a .tar.gz\nfile.\n\nBut that's a minor point not relevant here, and of course you are\nright. I just somehow did not register that \"store the odf files\" was\nsupposed to mean \"get checked into git in numerous versions\".\n\n-- \nDavid Kastrup\n"},{"id":"51718","messageId":"loom.20070827T170518-603@post.gmane.org","threadId":"9673","inReplyTo":"20070827141600.GA11000@glandium.org","subject":"Re: Git and OpenDocument (OpenOffice.org) files","fromName":"Sergio Callegari","fromEmail":"scallegari@arces.unibo.it","sentAt":"2007-08-27T15:16:28Z","receivedAt":"2007-08-27T15:16:28Z","isPatch":false,"sender":{"key":"scallegari@arces.unibo.it","avatar":null},"body":"Mike Hommey glandium.org> writes:\n\n\n> \n> A zipped file will be 100% different at each revision.\n> The unzipped counterpart may be similar for 90% or more between revisions.\n> \n> Mike\n> \n\nIn my (modest) experience, not really:\n\nin fact, odf files are a zip collection of many individual files (for instance\nif you have an impress presentation, the zip collection will contain all\nthe images that appear in the presentation...)\n\nNow: zip is different from .tar.gz in that tar.gz first concatenates the\nfiles and then compresses the overall thing, while zip compresses or stores\nthe individual files and then concatenates and indexes the result.\n\nThe difference is that in a tar.gz file, changing a single byte in one of\nthe internal files can lead to a completely different compressed stream,\nwhile in a zip file, changing an internal file only affects the relevant\npart of the zipped file.\n\nThis means that:\n- if you have an odf document containing lots of internal objects (e.g.\nimages) that do not change very much from version to version, git can make\nvery good deltas.\n- conversely if you have an odf document whose size is dominated by proper\ncontent, then git will not be able to make good deltas.\n\nAs an example, I am finding that impress presentations (dominated by images)\ncan delta very well, while calc spreadsheets (dominated by content) do not.\n\nProbably it could be nice to make a filter that takes an odf file and \nre-zips it so that the content.xml inner file is only stored, rather\nthan deflated. Then this could be used with the git file filtering\nmachinery.\n\nSergio\n"}]}