{"thread":{"id":"15547","subject":"Re: Management of opendocument (openoffice.org) files in git","startedAt":"2008-09-16T06:24:39Z","lastAt":"2008-10-10T08:12:50Z","messageCount":5,"participants":["Paolo Bonzini","Sergio Callegari","Michael J Gruber","Peter Krefting"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"90809","messageId":"48CF5127.5040507@gnu.org","threadId":"15547","inReplyTo":null,"subject":"Re: Management of opendocument (openoffice.org) files in git","fromName":"Paolo Bonzini","fromEmail":"bonzini@gnu.org","sentAt":"2008-09-16T06:24:39Z","receivedAt":"2008-09-16T06:24:39Z","isPatch":false,"sender":{"key":"bonzini@gnu.org","avatar":"https://avatars.githubusercontent.com/u/42082?v=4"},"body":">                 profile_unzip=PROFILE_UNZIP_${profile}\n>                 profile_zip=PROFILE_ZIP_${profile}\n>                 UNZIP_OPTS=${!profile_unzip}\n>                 ZIP_OPTS=${!profile_zip}\n\nCan be written (in pure bourne shell) as\n\n  eval UNZIP_OPTS=\\$PROFILE_UNZIP_${profile}\n  eval ZIP_OPTS=\\$PROFILE_ZIP_${profile}\n\n>         --unzip_opts)\n>             UNZIP_OPTS=${UNZIP_OPTS} $2\n\nMissing quotes:\n\n  UNZIP_OPTS=\"${UNZIP_OPTS} $2\"\n\nIt could also be a good idea to do\n\n  UNZIP_OPTS=\"${UNZIP_OPTS} `echo $2 | sed 'y/,/ /' `\"\n\n(compare with the -Wa/-Wl/-Wp options of gcc) so you can do\n\n  [filter \"opendocument\"]\n        clean = \"rezip --unzip-opts -b,-qq,-X --zip-opts -q,-r,-D,-0\"\n        smudge = \"rezip --unzip-opts -b,-qq,-X --zip-opts -q,-r,-D,-6\"\n\nAnd maybe -b,-qq,-X and -q,-r respectively could be added by default?\n\nAnyway, nice script, thanks!\n\nPaolo\n"},{"id":"90811","messageId":"48CF5ABC.5060507@gmail.com","threadId":"15547","inReplyTo":"48CF5127.5040507@gnu.org","subject":"Re: Management of opendocument (openoffice.org) files in git","fromName":"Sergio Callegari","fromEmail":"sergio.callegari@gmail.com","sentAt":"2008-09-16T07:05:32Z","receivedAt":"2008-09-16T07:05:32Z","isPatch":false,"sender":{"key":"sergio.callegari@gmail.com","avatar":"https://gravatar.com/avatar/c98f41317e0422c1e630385de0e3970227b8e5ad15f35ba8586066467cc833bc?d=mp&s=160"},"body":"Thanks for the fixes (particularly the missing quotation!) and the \nsuggestions.\n\nWith regards to\n\n> And maybe -b,-qq,-X and -q,-r respectively could be added by default?\n>\n>   \nI would prefer not to do so:  if you do you get something that is \nsomehow \"specialised\", otherwise you have a totally generic \"rezipper\" \nthat might also find other applications (who knows).\nBTW, that is why I added the profiles, so that there was no need to type \nrepetitive stuff.\n\nSergio\n"},{"id":"90820","messageId":"48CF6A7C.4020604@gnu.org","threadId":"15547","inReplyTo":"48CF5ABC.5060507@gmail.com","subject":"Re: Management of opendocument (openoffice.org) files in git","fromName":"Paolo Bonzini","fromEmail":"bonzini@gnu.org","sentAt":"2008-09-16T08:12:44Z","receivedAt":"2008-09-16T08:12:44Z","isPatch":false,"sender":{"key":"bonzini@gnu.org","avatar":"https://avatars.githubusercontent.com/u/42082?v=4"},"body":"\n> With regards to\n> \n>> And maybe -b,-qq,-X and -q,-r respectively could be added by default?\n>>\n>>   \n> I would prefer not to do so:  if you do you get something that is\n> somehow \"specialised\", otherwise you have a totally generic \"rezipper\"\n> that might also find other applications (who knows).\n\nYeah, but regarding -b/-X, their effect can/should be undone with zip\ncommand line options (-l, -ll, -X).  And for zip's -r option, a rezipper\nthat by default only rezips the top directory does not seem very useful. :-)\n\nYou're right about letting the user specify -qq/-q.  Or maybe you can\nhave a -q/--quiet option to rezip that adds both -qq to unzip, and -q to\nzip.  This way you can use the openoffice profile both in quiet mode\n(for git) and in non-quiet mode (for manual use).\n\nPutting all of this together, it would make the filter look like this:\n\n  [filter \"opendocument\"]\n        clean = \"rezip --quiet --zip-opts -D,-0\"\n        smudge = \"rezip --quiet --zip-opts -D,-6\"\n\nor similarly with profiles:\n\n  [filter \"opendocument\"]\n        clean = \"rezip --quiet -p ODF_UNCOMPRESS\"\n        smudge = \"rezip --quiet -p ODF_COMPRESS\"\n\nAfter my signature you can find my attempt at making rezip more useful\nas a general program.  It supports --quiet, multiple input files, and -\nfor stdin.  (And I learnt from you that >&$foo works).\n\n> BTW, that is why I added the profiles, so that there was no need to type\n> repetitive stuff.\n\nUnderstood.\n\nPaolo\n\n-----8<-----------------------\n#! /bin/sh\n#\n# (c) 2008 Sergio Callegari\n#\n# Rewrites a zip archive, possibly changing the compression level\n\nUSAGE='Usage: rezip OPTIONS FILE...\nwith options:\n  -h, --help               Gives help\n  --unzip-opts OPTIONS     Pass options to unzip helper to read zip file\n  --zip-opts OPTIONS       Pass options to zip helper to write zip file\n  -p, --profile PROFILE    Get options for helpers from profile\n  -q, --quiet              Make unzip and zip quiet\n\nRewrites a zip archive, possibily changing the compression level.\nIf the archive name is unspecified or \"-\", then the command operates\nlike a filter, reading from standard input and writing to standard\noutput.  Options (either space- or comma-separated) can be manually\nprovided to the unzip process doing the read and to the zip process\ndoing the write.  Alternatively a profile can be used to set options\nautomatically.'\n\nPROFILES=\"ODF_UNCOMPRESS ODF_COMPRESS\"\n\nPROFILE_UNZIP_ODF_UNCOMPRESS=\nPROFILE_ZIP_ODF_UNCOMPRESS=-D,-0\nPROFILE_UNZIP_ODF_COMPRESS=\nPROFILE_ZIP_ODF_COMPRESS=-D,-6\n\ndie()\n{\n    echo \"$3$USAGE\n\nAvailable profiles: ${PROFILES}\" >&$1\n    exit $2\n}\n\nUNZIP_OPTS=\"\"\nZIP_OPTS=\"\"\n\nwhile true ; do\n    case \"$1\" in\n        -h | --help)\n            die 1 0 ;;\n\n        # TODO: handle -p*, --profile=* and similarly for other options\n\n        -p | --profile)\n            eval UNZIP_OPTS=\\$PROFILE_UNZIP_$2\n            eval ZIP_OPTS=\\$PROFILE_ZIP_$2\n            shift ;;\n        --unzip-opts)\n            UNZIP_OPTS=\"${UNZIP_OPTS} $2\"\n            shift ;;\n        --zip-opts)\n            ZIP_OPTS=\"${ZIP_OPTS} $2\"\n            shift ;;\n        -q | --quiet)\n            UNZIP_QUIET=-qq\n            ZIP_QUIET=-q ;;\n        -*)\n            die 2 1 \"Invalid option: $1\n\" ;;\n        *)\n            break ;;\n    esac\n    shift\ndone\n\nUNZIP_OPTS=\"$UNZIP_QUIET -b -X `echo $UNZIP_OPTS | sed 'y/,/ /'`\"\nZIP_OPTS=\"$ZIP_QUIET -r `echo $ZIP_OPTS | sed 'y/,/ /'`\"\n\nif [ $# = 0 ] ; then\n    set fnord -\n    shift\nfi\n\nredir=1\nfor filename\ndo\n    if [ \"$filename\" = - ]; then\n        redir=2\n        break\n    fi\ndone\n\nfor filename\ndo\n    workdir=`mktemp -d -t rezip.workdir.XXXXXX`\n\n    if [ \"$filename\" = - ]; then\n        tmpcopy=:\n        filename=`mktemp rezip.zip.XXXXXX`\n        cat > $filename\n    else\n        tmpcopy=false\n    fi\n\n    (case $filename in\n        /*) ;;\n        *) filename=`pwd`/$filename ;;\n    esac\n    cd \"$workdir\"\n    unzip $UNZIP_OPTS \"$filename\" >&$redir\n    zip $ZIP_OPTS \"$filename\" . >&$redir)\n    rm -fr \"$workdir\"\n\n    if $tmpcopy ; then\n        cat \"$filename\"\n        rm \"$filename\"\n    fi\ndone\n\n--------8<------------------------\n"},{"id":"92144","messageId":"48E4C401.90409@drmicha.warpmail.net","threadId":"15547","inReplyTo":"48CF6A7C.4020604@gnu.org","subject":"Re: Management of opendocument (openoffice.org) files in git","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-10-02T12:52:17Z","receivedAt":"2008-10-02T12:52:17Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Following up on the discussion about tracking oo files I conducted a\nminimalistic test. I simulated tracking an oo spreadsheat, where from\none version to the next only a few cells would be entered in an existing\nspreadsheet. These are the sizes of the individual files:\n\n48K     0.ods\n48K     1.ods\n60K     2.ods\n60K     3.ods\n56K     4.ods\n64K     5.ods\n68K     6.ods\n64K     7.ods\n64K     8.ods\n68K     9.ods\n600K    total\n\nI then tracked this in three different ways, each in a fresh repo:\n\n\"packed\": copy $i.ods to t.ods as is, git add t.ods and commit.\n\"unpacked\": use the unzipped contents of $i.ods instead.\n\"rezip\": use the rezipped version (compression 0, using Sergio's script).\n\"oofilter\": use clean/smudge filters (calling Sergio's rezip)\n\nHere are the resulting sizes: first \".git/objects\" as is, then after\nrepacking -adf, finally the total size of .git + the work tree (i.e. the\nlast revision).\n\npacked\n708K    .git/objects\n492K    .git/objects\n692K    .git + wt\n\nunpacked\n1,3M    .git/objects\n144K    .git/objects\n1,5M    .git + wt\n\nrezip\n992K    .git/objects\n148K    .git/objects\n1,4M    .git + wt\n\noofilter\n984K    .git/objects\n148K    .git/objects\n352K    .git + wt\n\nUnsurprisingly, the total size is dominated by the work tree size if you\n have few revisions. (Also, templates and such contribute.)\nNote that git log --stat will report the sizes of packed files in the\nfirst case, but the sizes of unpacked files in all other cases. In\nparticular, it reports a different size for the  HEAD revision than you\nhave in a HEAD checkout.\n\nI tried rewriting \"packed\" after configuring the filters: filter-branch\nrefuses to work with a dirty work-tree, even after \"checkout -f HEAD\"\nand \"reset --hard\". It seems that git status is permanently confused\nhere. (Has anyone successfully rewritten existing oo files?)\n\nI'm not sure about the lessons, but I wanted to share the numbers\nanyways. I think this (your script and its usage) is heading in a useful\ndirection and should maybe made more known, if not made easier from the\ngit side. Also I'm still looking for a good (deterministic) pdf\nrecompressor.\n\nMichael\n\ngit version 1.6.0.2.426.g2cfa6\n"},{"id":"92732","messageId":"Pine.LNX.4.64.0810100910340.10709@ds9.cixit.se","threadId":"15547","inReplyTo":"48E4C401.90409@drmicha.warpmail.net","subject":"Re: Management of opendocument (openoffice.org) files in git","fromName":"Peter Krefting","fromEmail":"peter@softwolves.pp.se","sentAt":"2008-10-10T08:12:50Z","receivedAt":"2008-10-10T08:12:50Z","isPatch":false,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Michael J Gruber:\n\n> I'm not sure about the lessons, but I wanted to share the numbers\n> anyways. I think this (your script and its usage) is heading in a useful\n> direction and should maybe made more known, if not made easier from the\n> git side.\n\nI had very positive experiences with the script for my use-case. I did\npost them to the list, but it seems as if they got lost. At least I\ncan't seem to find them, did they show up?\n\nI had some problems with the script when trying to run it under\nWindows, though. Running Windows-Git from a Cygwin prompt provides some\nconfusion about some of the Unix tools' behaviour that I needed to\nwork around (like removing \"/cygdrive\" prefixes).\n\n-- \n\\\\// Peter - http://www.softwolves.pp.se/\n"}]}