{"thread":{"id":"32957","subject":"RFD: concatening textconv filters","startedAt":"2013-02-21T10:28:31Z","lastAt":"2013-02-21T17:39:46Z","messageCount":2,"participants":["Michael J Gruber","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"209963","messageId":"5125F6CF.50105@drmicha.warpmail.net","threadId":"32957","inReplyTo":null,"subject":"RFD: concatening textconv filters","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2013-02-21T10:28:31Z","receivedAt":"2013-02-21T10:28:31Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"During my day-to-day UGFWIINIT I noticed that we don't do textconv\niteratively. E.g.: I have a file\n\nSuperSecretButDumbFormat.pdf.gpg\n\nand textconv filters with attributes set for *.gpg and *.pdf (using\n\"gpg\" resp. \"pdftotext\"). For Git, the file has only the \"gpg\"\nattribute, of course. In this case, I would have wanted to pass the gpg\noutput through pdftotext.\n\nNow, I can set up an extra filter \"gpgtopdftotext\" for *.pdf.gpg (hoping\nI get the ordering in .gitattributes right), of course, but wondering\nwhether we could and should support concatenating filters by either\n\n- making it easy to request it (say by setting\n\"filter.gpgtopdftotext.textconvpipe\" to a list of textconv filter names\nwhich are to be applied in sequence)\n\nor\n\n- doing it automatically (remove the pattern which triggered the filter,\nand apply attributes again to the resulting pathspec)\n\nMaybe it's just not worth the effort. Or a nice GSoC project ;)\n\nMichael\n"},{"id":"209980","messageId":"7vzjyxva59.fsf@alter.siamese.dyndns.org","threadId":"32957","inReplyTo":"5125F6CF.50105@drmicha.warpmail.net","subject":"Re: RFD: concatening textconv filters","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-02-21T17:39:46Z","receivedAt":"2013-02-21T17:39:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Michael J Gruber <git@drmicha.warpmail.net> writes:\n\n> ... but wondering\n> whether we could and should support concatenating filters by either\n>\n> - making it easy to request it (say by setting\n> \"filter.gpgtopdftotext.textconvpipe\" to a list of textconv filter names\n> which are to be applied in sequence)\n>\n> or\n>\n> - doing it automatically (remove the pattern which triggered the filter,\n> and apply attributes again to the resulting pathspec)\n\nI think what you are getting at is to start from something like this:\n\n\t= .gitattributes =\n\t*.gpg\tdiff=gpg\n        *.pdf\tdiff=pdf\n\nand have \"git cat-file --textconv frotz.pdf.gpg\" (and other textconv\nusers) notice that:\n\n (1) The path matches \"*.gpg\" pattern, and calls for\n     diff.gpg.textconv conversion.  This already happens in the\n     current system.\n\n (2) After stripping the \"*.gpg\" pattern (i.e. look at the part of\n     the path that matched the wildcard part * in the attribute\n     selector), notice that the remainder, \"frotz.pdf\", could match\n     the \"*.pdf\" pattern.  The output from the previous filter could\n     be treated as if it were a blob that is stored in that path.\n\nA few issues that need to be addressed while designing this feature\nthat come to my mind at random are:\n\n * This seems to call for a new concept, but what exactly is that\n   concept?  Your RFD sounds as if you desire a \"cascadable\n   textconv\", but it may be of a somewhat larger scope, \"virtual\n   blob at a virtual path\", which the last sentence in (2) above\n   seems to suggest.\n\n * What is this new concept an attribute to?  If we express this as\n   \"the textconv conversion result of any path with attribute\n   diff=gpg can be treated as the contents of a virtual blob\", then\n   we are making it an attribute of the gpg \"type\", i.e.\n\n\t= .git/config =\n\t[diff \"gpg\"]\n\t\ttextconv = gpg -v\n\t\ttextconvProducesVirtualBlob = yes\n\n   To me, that seems sufficient for this particular application at\n   the first glance, but are there other attributes that may want to\n   produce such virtual blob for further processing?  Is limiting\n   this to textconv too restrictive?  I do not know.\n\n * What is the rule to come up with the \"virtual path\" to base the\n   attribute look-up on for the \"virtual blob contents\"?  In the\n   above example, the pattern was a simple \"*.gpg\", and we used a\n   naïve \"what did the asterisk match?\", but imagine a case where\n   you have some documents that you want to do \"gpg -v\" and some you\n   don't.  You express this by having the former class of files\n   named with \"conv-\" prefix, or some convention that is convenient\n   for you.\n\n   Your .gitattributes may say something like:\n\n\t= .gitattributes =\n        conv-*.gpg\tdiff=gpg\n\n   When deciding what attributes to use to further process the\n   result of conversion (i.e. \"virtual blob contents\") for\n   conv-frotz.pdf.gpg, what virtual path should we use?  Should we\n   use \"conv-frotz.pdf\", or just \"frotz.pdf\"?\n\n   \"The difference does not matter--either would work\" is not a\n   satisfactory answer, once you consider that you may want to have\n   two or more classes of pdf files that you may want to treat\n   differently, just like you did for gpg encrypted files in this\n   example setting.  It seems to suggest that we want to use\n   conv-frotz.pdf as the virtual path, but how would we derive that\n   from the pattern \"conv-*.gpg\" and path \"conv-frotz.pdf.gpg\"?  It\n   appears to me that you would need a way to say between the two\n   literal parts in the pattern, \"conv-\" part needs to be kept but\n   \".gpg\" part needs to be stripped when forming the result.\n"}]}