{"thread":{"id":"23967","subject":"Best way to apply textconv to a working tree file","startedAt":"2010-06-01T13:41:05Z","lastAt":"2010-06-02T15:12:38Z","messageCount":8,"participants":["Clément Poulain","Matthieu Moy","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"142683","messageId":"a1ace6b77167a2ad4b4995e8c4d09761@ensimag.fr","threadId":"23967","inReplyTo":null,"subject":"Best way to apply textconv to a working tree file","fromName":"Clément Poulain","fromEmail":"clement.poulain@ensimag.imag.fr","sentAt":"2010-06-01T13:41:05Z","receivedAt":"2010-06-01T13:41:05Z","isPatch":false,"sender":{"key":"clement.poulain@ensimag.imag.fr","avatar":null},"body":"Hello,\n\nIn order to add textconv support to \"git-gui blame\", we have to run\ntextconv on a file which is in the working tree.\nCurrently, \"git-gui blame\" uses the Tcl function 'open' to display the\ncontent of the working-tree file. This doesn't allow us to run textconv on\nit.\n\nWe are wondering what is the best way to do the textconv. Here are some\nsolutions we thought about:\n\n- For revisions, \"git-gui blame\" is based on \"git cat-file blob\n\". Therefore, we thought of adding a \"--textconv\" option on\ncat-file as well as a \"--working-tree\" option (\"git cat-file --working-tree\n\" will display the content of in the working-tree)\n\n- Use the \"-p\" (pretty print) cat-file's option, with a path beginning by\n./ (to avoid ambiguous name like HEAD)\n\n- Create a \"git textconv\" command, to easily run textconv on any object.\n\nWhich way you think is the best?\n\nThanks for your time and comments.\nRegards\n"},{"id":"142703","messageId":"vpqd3walot1.fsf@bauges.imag.fr","threadId":"23967","inReplyTo":"a1ace6b77167a2ad4b4995e8c4d09761@ensimag.fr","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2010-06-01T16:07:06Z","receivedAt":"2010-06-01T16:07:06Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"[ Just had IRL discussion with Clément and his co-workers ]\n\nClément Poulain writes:\n\n> In order to add textconv support to \"git-gui blame\", we have to run\n> textconv on a file which is in the working tree.\n> Currently, \"git-gui blame\" uses the Tcl function 'open' to display the\n> content of the working-tree file. This doesn't allow us to run textconv on\n> it.\n\nThere are actually two distinct issues :\n\n1) The textconv functionality has an API, but isn't available as\n plumbing for scripting purposes (and git gui would like to be able\n to call textconv from the command-line).\n\n2) If one solves issue 1) by adding textconv support to \"git\n cat-file\", then we can call textconv on a blob in the object\n database, but not on a worktree file. Hence, we still need to add\n support to display worktree files in \"git cat-file\".\n\n> - For revisions, \"git-gui blame\" is based on \"git cat-file blob\n> \". Therefore, we thought of adding a \"--textconv\" option on\n> cat-file as well as a \"--working-tree\" option (\"git cat-file --working-tree\n> \" will display the content of in the working-tree)\n>\n> - Use the \"-p\" (pretty print) cat-file's option, with a path beginning by\n> ./ (to avoid ambiguous name like HEAD)\n\nAfter thinking about it, I think a mix of these solutions would be\nOK:\n\ngit cat-file --textconv # Run textconv on blob\n\ngit cat-file --textconv : # Run textconv on file\n # (in object database)\n\ngit cat-file --textconv ./ # Run textconv on file\n # (in worktree)\n\nThis preserves the property \"1st argument is an option, 2nd argument\nis the thing to do the cat-file on\", hence avoid any possible\nambiguities in command-line option parsing.\n\nAnother option would be to resurect the WOKRTREE/STAGE magit tokens [1]\nproposal, and allow\n\ngit cat-file --textconv WORKTREE:\n\nThat sounds the most natural to me, but it's probably the most\ncontroversial too ...\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/133087\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"142708","messageId":"20100601170426.GC9702@sigill.intra.peff.net","threadId":"23967","inReplyTo":"a1ace6b77167a2ad4b4995e8c4d09761@ensimag.fr","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-06-01T17:04:27Z","receivedAt":"2010-06-01T17:04:27Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 01, 2010 at 03:41:05PM +0200, Clément Poulain wrote:\n\n> We are wondering what is the best way to do the textconv. Here are some\n> solutions we thought about:\n\nOne solution you didn't mention would be to do it all yourself:\n\n driver=`git check-attr diff \"$file\" | cut -d: -f3`\n textconv=`git config diff.$driver.textconv`\n $textconv <$file >$file.converted\n\nThis has the advantage of working with existing versions of git. The\ndownside is that it's more code (e.g., my parsing above is quite sloppy\nand loose. Doing it right would be a few more lines).\n\nFurthermore, it doesn't use the textconv cache at all. For working tree\nfiles, this might not matter (if you pull the sha1 out of the index,\nthough, you can still check the cache, and unchanged working tree files\nare likely to be in the cache). But for blobs in general, the cache is\nworth using.\n\n-Peff\n"},{"id":"142709","messageId":"20100601171359.GD9702@sigill.intra.peff.net","threadId":"23967","inReplyTo":"vpqd3walot1.fsf@bauges.imag.fr","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-06-01T17:14:00Z","receivedAt":"2010-06-01T17:14:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 01, 2010 at 06:07:06PM +0200, Matthieu Moy wrote:\n\n> After thinking about it, I think a mix of these solutions would be\n> OK:\n> \n> git cat-file --textconv # Run textconv on blob\n\nThis wouldn't work. The textconv is defined by the diff driver, which is\nassociated with a pathname, not a blob. You don't have a pathname here\n(and in fact the same blob could potentially even be represented by\ndifferent textconvs if it appeared in different contexts).\n\n> git cat-file --textconv : # Run textconv on file\n> # (in object database)\n\nThis could work in theory, but will not be as easy to implement as you\nhope. The code to parse : will give you only a blob\nsha1, and you will have to look up the tree manually (which you will\nhave to split at the \":\" manually, remembering that it may be as complex\nas \"HEAD@{12:00 yesterday}:foo.c\", then check the diff attr manually for\nthat filename.\n\nIt would be nice if there was some way in the get_sha1* functions to\nsave some context, like tree context and filename. This would be helpful\nfor something like \"git show HEAD:foo.txt\", which probably should be\nrespecting autocrlf and smudge/clean filters.\n\n> git cat-file --textconv ./ # Run textconv on file\n> # (in worktree)\n\nFWIW, I think this combination is the best of the possible syntaxes\noutlined in the original mail.\n\n-Peff\n"},{"id":"142714","messageId":"vpqtypmaceb.fsf@bauges.imag.fr","threadId":"23967","inReplyTo":"20100601171359.GD9702@sigill.intra.peff.net","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2010-06-01T17:30:36Z","receivedAt":"2010-06-01T17:30:36Z","isPatch":false,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Jeff King writes:\n\n>> git cat-file --textconv # Run textconv on blob\n>\n> This wouldn't work. The textconv is defined by the diff driver, which is\n> associated with a pathname,\n\nRight, forget it.\n\n>> git cat-file --textconv : # Run textconv on file\n>> # (in object database)\n>\n> This could work in theory, but will not be as easy to implement as you\n> hope.\n\nActually, Clément has already hit the issue.\n\n> complex as \"HEAD@{12:00 yesterday}:foo.c\"\n\n_that_ is the actual example I was looking for ;-) (with a : inside\nthe pathname to make sure you can't search it from the right).\n\n> It would be nice if there was some way in the get_sha1* functions to\n> save some context, like tree context and filename. This would be helpful\n> for something like \"git show HEAD:foo.txt\", which probably should be\n> respecting autocrlf and smudge/clean filters.\n\nYup. The code to do the parsing is already there, it \"just\" needs to\nbe made available through a clean API.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"142723","messageId":"20100601195036.GA18220@sigill.intra.peff.net","threadId":"23967","inReplyTo":"vpqtypmaceb.fsf@bauges.imag.fr","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-06-01T19:50:36Z","receivedAt":"2010-06-01T19:50:36Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 01, 2010 at 07:30:36PM +0200, Matthieu Moy wrote:\n\n> > It would be nice if there was some way in the get_sha1* functions to\n> > save some context, like tree context and filename. This would be helpful\n> > for something like \"git show HEAD:foo.txt\", which probably should be\n> > respecting autocrlf and smudge/clean filters.\n> \n> Yup. The code to do the parsing is already there, it \"just\" needs to\n> be made available through a clean API.\n\nI was thinking of something like the patch below, which applies\nclean/smudge filters to the output of \"cat-file -p\".\n\nIt keeps a global context for the last sha1 looked up. Probably\nget_sha1_with_mode should be folded into get_sha1_with_context, as mode\nis really just another case of this exact sort of mid-lookup context.\nAnd then we don't get a proliferation of \"get_sha1_with_*\" functions,\nwhich doesn't scale. I'll leave that as an exercise for your students.\n:)\n\nYou could get fancier, including things like which ref we ended up\nlooking at to get to the object. I seem to recall running into a\nsituation where I wanted to know \"foo\" when looking up \"foo^\", but I\ndon't remember where now. I think it makes sense to keep it simple for\nnow, and people could add more context elements as needed.\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex a933eaa..4d1e634 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -87,6 +87,7 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name)\n \tvoid *buf;\n \tunsigned long size;\n \n+\tobject_resolve_context_init(&the_resolve_context);\n \tif (get_sha1(obj_name, sha1))\n \t\tdie(\"Not a valid object name %s\", obj_name);\n \n@@ -129,6 +130,18 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name)\n \t\t\tpprint_tag(sha1, buf, size);\n \t\t\treturn 0;\n \t\t}\n+\t\tif (type == OBJ_BLOB) {\n+\t\t\tstruct strbuf out = STRBUF_INIT;\n+\n+\t\t\tif (!the_resolve_context.path[0])\n+\t\t\t\tbreak;\n+\t\t\tif (convert_to_working_tree(the_resolve_context.path,\n+\t\t\t\t\t\tbuf, size, &out) < 0)\n+\t\t\t\tdie(\"unable to prepare blob for printing\");\n+\t\t\tfree(buf);\n+\t\t\tsize = out.len;\n+\t\t\tbuf = strbuf_detach(&out, NULL);\n+\t\t}\n \n \t\t/* otherwise just spit out the data */\n \t\tbreak;\ndiff --git a/cache.h b/cache.h\nindex c966023..c030083 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -730,6 +730,13 @@ static inline unsigned int hexval(unsigned char c)\n #define MINIMUM_ABBREV 4\n #define DEFAULT_ABBREV 7\n \n+struct object_resolve_context {\n+\tunsigned char tree[20];\n+\tchar path[PATH_MAX];\n+};\n+extern struct object_resolve_context the_resolve_context;\n+void object_resolve_context_init(struct object_resolve_context *orc);\n+\n extern int get_sha1(const char *str, unsigned char *sha1);\n extern int get_sha1_with_mode_1(const char *str, unsigned char *sha1, unsigned *mode, int gently, const char *prefix);\n static inline int get_sha1_with_mode(const char *str, unsigned char *sha1, unsigned *mode)\ndiff --git a/sha1_name.c b/sha1_name.c\nindex bf92417..cc049bf 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -7,6 +7,13 @@\n #include \"refs.h\"\n #include \"remote.h\"\n \n+struct object_resolve_context the_resolve_context;\n+\n+void object_resolve_context_init(struct object_resolve_context *orc)\n+{\n+\tmemset(orc, 0, sizeof(*orc));\n+}\n+\n static int find_short_object_filename(int len, const char *name, unsigned char *sha1)\n {\n \tstruct alternate_object_database *alt;\n@@ -1104,6 +1111,10 @@ int get_sha1_with_mode_1(const char *name, unsigned char *sha1, unsigned *mode,\n \t\t\t\t\t\t\t tree_sha1, object_name);\n \t\t\t\tfree(object_name);\n \t\t\t}\n+\t\t\thashcpy(the_resolve_context.tree, tree_sha1);\n+\t\t\tstrncpy(the_resolve_context.path, filename,\n+\t\t\t\tsizeof(the_resolve_context.path));\n+\t\t\tthe_resolve_context.path[sizeof(the_resolve_context.path)] = '\\0';\n \t\t\treturn ret;\n \t\t} else {\n \t\t\tif (!gently)\n"},{"id":"142806","messageId":"1725d013942b49b804102cca573d2999@ensimag.fr","threadId":"23967","inReplyTo":"20100601170426.GC9702@sigill.intra.peff.net","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Clément Poulain","fromEmail":"clement.poulain@ensimag.imag.fr","sentAt":"2010-06-02T09:56:59Z","receivedAt":"2010-06-02T09:56:59Z","isPatch":false,"sender":{"key":"clement.poulain@ensimag.imag.fr","avatar":null},"body":"On Tue, 1 Jun 2010 13:04:27 -0400, Jeff King wrote:\n> On Tue, Jun 01, 2010 at 03:41:05PM +0200, Clément Poulain wrote:\n> \n>> We are wondering what is the best way to do the textconv. Here are some\n>> solutions we thought about:\n> \n> One solution you didn't mention would be to do it all yourself:\n> \n> driver=`git check-attr diff \"$file\" | cut -d: -f3`\n> textconv=`git config diff.$driver.textconv`\n> $textconv <$file >$file.converted\n> \n> This has the advantage of working with existing versions of git. The\n> downside is that it's more code (e.g., my parsing above is quite sloppy\n> and loose. Doing it right would be a few more lines).\n> \n> Furthermore, it doesn't use the textconv cache at all. For working tree\n> files, this might not matter (if you pull the sha1 out of the index,\n> though, you can still check the cache, and unchanged working tree files\n> are likely to be in the cache). But for blobs in general, the cache is\n> worth using.\n> \n> -Peff\n\nIt seems to really fit for git gui. \nAbout the cache : this method will be only used by git gui (when it's\nlaunched) on working tree files, so it seems OK for me.\nThank you\n"},{"id":"142824","messageId":"a4d60325ac1eba4038ca65739105b20d@ensimag.fr","threadId":"23967","inReplyTo":"20100601195036.GA18220@sigill.intra.peff.net","subject":"Re: Best way to apply textconv to a working tree file","fromName":"Clément Poulain","fromEmail":"clement.poulain@ensimag.imag.fr","sentAt":"2010-06-02T15:12:38Z","receivedAt":"2010-06-02T15:12:38Z","isPatch":false,"sender":{"key":"clement.poulain@ensimag.imag.fr","avatar":null},"body":"We have finally chosen to use the \"git cat-file --textconv\n:\" option, it works fine.\nWe are now using your patch to easily get the path, and it seems to fit\nvery well too :)\n\nThanks a lot\n"}]}