{"thread":{"id":"31643","subject":"[RFC/PATCH 0/3] git log --pretty=lua","startedAt":"2012-09-25T00:23:25Z","lastAt":"2012-10-06T17:33:29Z","messageCount":15,"participants":["Jeff King","Nguyen Thai Ngoc Duy","Robin H. Johnson","Stephen Bash","Matthieu Moy","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"199867","messageId":"20120925002325.GA19560@sigill.intra.peff.net","threadId":"31643","inReplyTo":null,"subject":"[RFC/PATCH 0/3] git log --pretty=lua","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-25T00:23:25Z","receivedAt":"2012-09-25T00:23:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"We've talked off and on about extending the --pretty=format specifiers\nto something more flexible. There's also been talk recently of more\nflexible commit-filtering (e.g., grepping individual notes).  Rather\nthan invent a new Turing-complete language, I thought I'd try building\non somebody else's work by embedding an existing language.\n\nWhy Lua? I don't especially like it as a language. But it's designed for\nthis purpose, which makes it very lightweight and relatively simple to\nembed. Here are timing results for a few log commands (best-of-five,\nwarm cache):\n\n  $ git log --oneline >/dev/null\n  real    0m1.042s\n  user    0m0.660s\n  sys     0m0.372s\n\n  $ git log --tformat:\"%h %s\" >/dev/null\n  real    0m1.039s\n  user    0m0.624s\n  sys     0m0.396s\n\n  $ git log --pretty=lua:'return abbrev(hash()) .. \" \" .. subject()'\n  real    0m1.112s\n  user    0m0.716s\n  sys     0m0.388s\n\nSo you can see that we're a little bit slower than the existing format,\nbut not too much. There may well be some optimizations we can do, too.\nThis is the first time I've ever played with embedding Lua, so I would\nnot be surprised if I got something wrong or suboptimal.\n\nThe syntax, on the other hand...yuck. One thing that makes Lua\nhorrible for this use is that it does not have interpolated strings.\nHowever, there are template libraries for Lua, so maybe there's\nsomething there.\n\nThe patches are:\n\n  [1/3]: pretty: make some commit-parsing helpers more public\n  [2/3]: add basic lua infrastructure\n  [3/3]: add a \"lua\" pretty format\n\nAnd a \"4/3\" patch would probably add \"--lua-filter\" as a revision option\nfor limiting commits.\n\nThe patches are very rough and not meant to be applied. For me, this was\na bit of an experiment. I'm not sure if I like it or not. It seems like\na cool direction to go, but to be perfectly honest, I do not generally\nfeel like git's existing filtering or output are inadequate (sure, it's\nslower to pipe --pretty=raw out to a separate perl filter and then do\nfancy formatting, but it's usually fast enough, and it's very flexible).\n\nSo I don't have plans to work on it more any time soon, but I thought\nI'd share in case anybody is interested. And if somebody wants to pick\nup the topic and run with it, I'd be happy to help.\n\n-Peff\n"},{"id":"199868","messageId":"20120925002417.GA19605@sigill.intra.peff.net","threadId":"31643","inReplyTo":"20120925002325.GA19560@sigill.intra.peff.net","subject":"[PATCH 1/3] pretty: make some commit-parsing helpers more public","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-25T00:24:17Z","receivedAt":"2012-09-25T00:24:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"This is a quick hack to make these functions available for\nthe lua code. It would be way cleaner to move all of the\nincremental parsing bits to format-commit.[ch] and clean up\nthe names (e.g., \"struct chunk\" is not nearly descriptive\nenough for a global).\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n commit.h | 36 ++++++++++++++++++++++++++++++++++++\n pretty.c | 38 ++------------------------------------\n 2 files changed, 38 insertions(+), 36 deletions(-)\n\ndiff --git a/commit.h b/commit.h\nindex 9f21313..71cd4af 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -126,6 +126,42 @@ void pp_remainder(const struct pretty_print_context *pp,\n \t\t  struct strbuf *sb,\n \t\t  int indent);\n \n+struct chunk {\n+\tsize_t off;\n+\tsize_t len;\n+};\n+\n+struct format_commit_context {\n+\tconst struct commit *commit;\n+\tconst struct pretty_print_context *pretty_ctx;\n+\tunsigned commit_header_parsed:1;\n+\tunsigned commit_message_parsed:1;\n+\tunsigned commit_signature_parsed:1;\n+\tstruct {\n+\t\tchar *gpg_output;\n+\t\tchar good_bad;\n+\t\tchar *signer;\n+\t} signature;\n+\tchar *message;\n+\tsize_t width, indent1, indent2;\n+\n+\t/* These offsets are relative to the start of the commit message. */\n+\tstruct chunk author;\n+\tstruct chunk committer;\n+\tstruct chunk encoding;\n+\tsize_t message_off;\n+\tsize_t subject_off;\n+\tsize_t body_off;\n+\n+\t/* The following ones are relative to the result struct strbuf. */\n+\tstruct chunk abbrev_commit_hash;\n+\tstruct chunk abbrev_tree_hash;\n+\tstruct chunk abbrev_parent_hashes;\n+\tsize_t wrap_start;\n+};\n+\n+void parse_commit_header(struct format_commit_context *);\n+void parse_commit_message(struct format_commit_context *);\n \n /** Removes the first commit from a list sorted by date, and adds all\n  * of its parents.\ndiff --git a/pretty.c b/pretty.c\nindex 8b1ea9f..0d4eb3d 100644\n--- a/pretty.c\n+++ b/pretty.c\n@@ -612,40 +612,6 @@ skip:\n \treturn 0; /* unknown placeholder */\n }\n \n-struct chunk {\n-\tsize_t off;\n-\tsize_t len;\n-};\n-\n-struct format_commit_context {\n-\tconst struct commit *commit;\n-\tconst struct pretty_print_context *pretty_ctx;\n-\tunsigned commit_header_parsed:1;\n-\tunsigned commit_message_parsed:1;\n-\tunsigned commit_signature_parsed:1;\n-\tstruct {\n-\t\tchar *gpg_output;\n-\t\tchar good_bad;\n-\t\tchar *signer;\n-\t} signature;\n-\tchar *message;\n-\tsize_t width, indent1, indent2;\n-\n-\t/* These offsets are relative to the start of the commit message. */\n-\tstruct chunk author;\n-\tstruct chunk committer;\n-\tstruct chunk encoding;\n-\tsize_t message_off;\n-\tsize_t subject_off;\n-\tsize_t body_off;\n-\n-\t/* The following ones are relative to the result struct strbuf. */\n-\tstruct chunk abbrev_commit_hash;\n-\tstruct chunk abbrev_tree_hash;\n-\tstruct chunk abbrev_parent_hashes;\n-\tsize_t wrap_start;\n-};\n-\n static int add_again(struct strbuf *sb, struct chunk *chunk)\n {\n \tif (chunk->len) {\n@@ -663,7 +629,7 @@ static int add_again(struct strbuf *sb, struct chunk *chunk)\n \treturn 0;\n }\n \n-static void parse_commit_header(struct format_commit_context *context)\n+void parse_commit_header(struct format_commit_context *context)\n {\n \tconst char *msg = context->message;\n \tint i;\n@@ -749,7 +715,7 @@ const char *format_subject(struct strbuf *sb, const char *msg,\n \treturn msg;\n }\n \n-static void parse_commit_message(struct format_commit_context *c)\n+void parse_commit_message(struct format_commit_context *c)\n {\n \tconst char *msg = c->message + c->message_off;\n \tconst char *start = c->message;\n-- \n1.7.12.1.10.g6537447\n"},{"id":"199869","messageId":"20120925002511.GB19605@sigill.intra.peff.net","threadId":"31643","inReplyTo":"20120925002325.GA19560@sigill.intra.peff.net","subject":"[PATCH 2/3] add basic lua infrastructure","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-25T00:25:12Z","receivedAt":"2012-09-25T00:25:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"This adds a small module for examining parts of a commit\nfrom inside a lua interpreter. Eventually you'll be able to\ndo grep-like filtering and --pretty formatting.\n\nThe most naive presentation would be to parse the whole\ncommit and put it in a lua table. However, instead we build\nupon the incremental parsing used by the --format parser,\nand lazily parse bits of the commit as the lua code requests\nthem.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nSet \"USE_LUA\" in your Makefile to turn it on.\n\n Makefile     |   7 +++\n lua-commit.c | 166 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n lua-commit.h |   9 ++++\n 3 files changed, 182 insertions(+)\n create mode 100644 lua-commit.c\n create mode 100644 lua-commit.h\n\ndiff --git a/Makefile b/Makefile\nindex a49d1db..54473e2 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -636,6 +636,7 @@ LIB_H += log-tree.h\n LIB_H += list-objects.h\n LIB_H += ll-merge.h\n LIB_H += log-tree.h\n+LIB_H += lua-commit.h\n LIB_H += mailmap.h\n LIB_H += merge-file.h\n LIB_H += merge-recursive.h\n@@ -749,6 +750,7 @@ LIB_OBJS += log-tree.o\n LIB_OBJS += ll-merge.o\n LIB_OBJS += lockfile.o\n LIB_OBJS += log-tree.o\n+LIB_OBJS += lua-commit.o\n LIB_OBJS += mailmap.o\n LIB_OBJS += match-trees.o\n LIB_OBJS += merge-file.o\n@@ -1818,6 +1820,11 @@ endif\n        COMPAT_OBJS += compat/nedmalloc/nedmalloc.o\n endif\n \n+ifdef USE_LUA\n+\tBASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n+\tEXTLIBS += `pkg-config --libs lua5.2`\n+endif\n+\n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n \texport GIT_TEST_CMP_USE_COPIED_CONTEXT\n endif\ndiff --git a/lua-commit.c b/lua-commit.c\nnew file mode 100644\nindex 0000000..ce1eeeb\n--- /dev/null\n+++ b/lua-commit.c\n@@ -0,0 +1,166 @@\n+#include \"cache.h\"\n+#include \"lua-commit.h\"\n+#include \"commit.h\"\n+\n+#ifndef USE_LUA\n+\n+static const char msg[] = \"git was built without lua support\";\n+\n+void lua_commit_init(const char *)\n+{\n+\tdie(msg);\n+}\n+\n+void lua_commit_format(struct strbuf *,\n+\t\t       struct format_commit_context *)\n+{\n+\tdie(msg);\n+}\n+\n+#else\n+\n+#include <lua.h>\n+#include <lauxlib.h>\n+#include <lualib.h>\n+\n+static lua_State *lua;\n+\n+/* XXX\n+ * We need to access this from functions called from inside lua. Probably it\n+ * would be cleaner use a lua \"register\" to let each function access it, but I\n+ * haven't looked into it.\n+ */\n+static struct format_commit_context *c;\n+\n+static int lua_fun_hash(lua_State *lua)\n+{\n+\tlua_pushstring(lua, sha1_to_hex(c->commit->object.sha1));\n+\treturn 1;\n+}\n+\n+static int lua_fun_abbrev(lua_State *lua)\n+{\n+\tconst char *hex;\n+\tunsigned char sha1[20];\n+\n+\thex = lua_tostring(lua, -1);\n+\tif (!hex || get_sha1_hex(hex, sha1)) {\n+\t\tlua_pushstring(lua, \"abbrev requires a sha1\");\n+\t\tlua_error(lua);\n+\t}\n+\n+\tlua_pushstring(lua, find_unique_abbrev(sha1, c->pretty_ctx->abbrev));\n+\treturn 1;\n+}\n+\n+static int get_ident(lua_State *lua, const char *line, int len)\n+{\n+\tstruct ident_split s;\n+\n+\tif (split_ident_line(&s, line, len) < 0) {\n+\t\tlua_pushstring(lua, \"unable to parse ident line\");\n+\t\tlua_error(lua);\n+\t}\n+\n+\tlua_createtable(lua, 0, 2);\n+\tlua_pushstring(lua, \"name\");\n+\tlua_pushlstring(lua, s.name_begin, s.name_end - s.name_begin);\n+\tlua_settable(lua, -3);\n+\tlua_pushstring(lua, \"email\");\n+\tlua_pushlstring(lua, s.mail_begin, s.mail_end - s.mail_begin);\n+\tlua_settable(lua, -3);\n+\n+\t/* XXX should also put date in the table */\n+\n+\treturn 1;\n+}\n+\n+static int lua_fun_author(lua_State *lua)\n+{\n+\tif (!c->commit_header_parsed)\n+\t\tparse_commit_header(c);\n+\treturn get_ident(lua, c->message + c->author.off, c->author.len);\n+}\n+\n+static int lua_fun_committer(lua_State *lua)\n+{\n+\tif (!c->commit_header_parsed)\n+\t\tparse_commit_header(c);\n+\treturn get_ident(lua, c->message + c->committer.off, c->committer.len);\n+}\n+\n+static int lua_fun_message(lua_State *lua)\n+{\n+\tlua_pushstring(lua, c->message + c->message_off + 1);\n+\treturn 1;\n+}\n+\n+static int lua_fun_subject(lua_State *lua)\n+{\n+\tstruct strbuf tmp = STRBUF_INIT;\n+\n+\tif (!c->commit_header_parsed)\n+\t\tparse_commit_header(c);\n+\tif (!c->commit_message_parsed)\n+\t\tparse_commit_message(c);\n+\n+\tformat_subject(&tmp, c->message + c->subject_off, \" \");\n+\tlua_pushlstring(lua, tmp.buf, tmp.len);\n+\treturn 1;\n+}\n+\n+static int lua_fun_body(lua_State *lua)\n+{\n+\tif (!c->commit_header_parsed)\n+\t\tparse_commit_header(c);\n+\tif (!c->commit_message_parsed)\n+\t\tparse_commit_message(c);\n+\n+\tlua_pushstring(lua, c->message + c->body_off);\n+\treturn 1;\n+}\n+\n+void lua_commit_init(const char *snippet)\n+{\n+\tif (!lua) {\n+\t\tlua = luaL_newstate();\n+\t\tif (!lua)\n+\t\t\tdie(\"unable to open lua interpreter\");\n+\t\tluaL_openlibs(lua);\n+\n+#define REG(name) do { \\\n+\tlua_pushcfunction(lua, lua_fun_##name); \\\n+\tlua_setglobal(lua, #name); \\\n+} while(0)\n+\n+\t\tREG(hash);\n+\t\tREG(abbrev);\n+\t\tREG(author);\n+\t\tREG(committer);\n+\t\tREG(message);\n+\t\tREG(subject);\n+\t\tREG(body);\n+\t}\n+\n+\tif (luaL_loadstring(lua, snippet))\n+\t\tdie(\"unable to load lua snippet: %s\", snippet);\n+}\n+\n+void lua_commit_format(struct strbuf *out,\n+\t\t       struct format_commit_context *context)\n+{\n+\tconst char *ret;\n+\tsize_t len;\n+\n+\tc = context;\n+\n+\tlua_pushvalue(lua, -1);\n+\tif (lua_pcall(lua, 0, 1, 0))\n+\t\tdie(\"lua failed: %s\", lua_tostring(lua, -1));\n+\n+\tret = lua_tolstring(lua, -1, &len);\n+\tstrbuf_add(out, ret, len);\n+\tlua_pop(lua, 1);\n+}\n+\n+#endif /* USE_LUA */\ndiff --git a/lua-commit.h b/lua-commit.h\nnew file mode 100644\nindex 0000000..aaffced\n--- /dev/null\n+++ b/lua-commit.h\n@@ -0,0 +1,9 @@\n+#ifndef LUA_COMMIT_H\n+#define LUA_COMMIT_H\n+\n+struct format_commit_context;\n+\n+void lua_commit_init(const char *snippet);\n+void lua_commit_format(struct strbuf *out, struct format_commit_context *context);\n+\n+#endif\n-- \n1.7.12.1.10.g6537447\n"},{"id":"199870","messageId":"20120925002539.GC19605@sigill.intra.peff.net","threadId":"31643","inReplyTo":"20120925002325.GA19560@sigill.intra.peff.net","subject":"[PATCH 3/3] add a \"lua\" pretty format","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-25T00:25:39Z","receivedAt":"2012-09-25T00:25:39Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"With this patch, you can do:\n\n  git log --pretty=lua:'\n    return abbrev(hash()) .. \" (\" .. author().email .. \") \" .. subject()\n  '\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n commit.h   |  1 +\n log-tree.c |  3 ++-\n pretty.c   | 21 +++++++++++++++++++--\n 3 files changed, 22 insertions(+), 3 deletions(-)\n\ndiff --git a/commit.h b/commit.h\nindex 71cd4af..8865a36 100644\n--- a/commit.h\n+++ b/commit.h\n@@ -73,6 +73,7 @@ enum cmit_fmt {\n \tCMIT_FMT_ONELINE,\n \tCMIT_FMT_EMAIL,\n \tCMIT_FMT_USERFORMAT,\n+\tCMIT_FMT_LUA,\n \n \tCMIT_FMT_UNSPECIFIED\n };\ndiff --git a/log-tree.c b/log-tree.c\nindex c894930..c8274d1 100644\n--- a/log-tree.c\n+++ b/log-tree.c\n@@ -599,7 +599,8 @@ void show_log(struct rev_info *opt)\n \tif (opt->commit_format == CMIT_FMT_EMAIL) {\n \t\tlog_write_email_headers(opt, commit, &ctx.subject, &extra_headers,\n \t\t\t\t\t&ctx.need_8bit_cte);\n-\t} else if (opt->commit_format != CMIT_FMT_USERFORMAT) {\n+\t} else if (opt->commit_format != CMIT_FMT_USERFORMAT &&\n+\t\t   opt->commit_format != CMIT_FMT_LUA) {\n \t\tfputs(diff_get_color_opt(&opt->diffopt, DIFF_COMMIT), stdout);\n \t\tif (opt->commit_format != CMIT_FMT_ONELINE)\n \t\t\tfputs(\"commit \", stdout);\ndiff --git a/pretty.c b/pretty.c\nindex 0d4eb3d..fdd4258 100644\n--- a/pretty.c\n+++ b/pretty.c\n@@ -10,6 +10,7 @@\n #include \"color.h\"\n #include \"reflog-walk.h\"\n #include \"gpg-interface.h\"\n+#include \"lua-commit.h\"\n \n static char *user_format;\n static struct cmt_fmt_map {\n@@ -33,6 +34,13 @@ static void save_user_format(struct rev_info *rev, const char *cp, int is_tforma\n \trev->commit_format = CMIT_FMT_USERFORMAT;\n }\n \n+static void save_lua_format(struct rev_info *rev, const char *cp, int is_tformat)\n+{\n+\tlua_commit_init(cp);\n+\tsave_user_format(rev, cp, is_tformat);\n+\trev->commit_format = CMIT_FMT_LUA;\n+}\n+\n static int git_pretty_formats_config(const char *var, const char *value, void *cb)\n {\n \tstruct cmt_fmt_map *commit_format = NULL;\n@@ -155,6 +163,10 @@ void get_commit_format(const char *arg, struct rev_info *rev)\n \t\tsave_user_format(rev, strchr(arg, ':') + 1, arg[0] == 't');\n \t\treturn;\n \t}\n+\tif (!prefixcmp(arg, \"lua:\")) {\n+\t\tsave_lua_format(rev, arg + 4, 1);\n+\t\treturn;\n+\t}\n \n \tif (strchr(arg, '%')) {\n \t\tsave_user_format(rev, arg, 1);\n@@ -1168,7 +1180,11 @@ void format_commit_message(const struct commit *commit,\n \t\tfree(enc);\n \t}\n \n-\tstrbuf_expand(sb, format, format_commit_item, &context);\n+\tif (pretty_ctx->fmt == CMIT_FMT_USERFORMAT)\n+\t\tstrbuf_expand(sb, format, format_commit_item, &context);\n+\telse if (pretty_ctx->fmt == CMIT_FMT_LUA)\n+\t\tlua_commit_format(sb, &context);\n+\n \trewrap_message_tail(sb, &context, 0, 0, 0);\n \n \tif (context.message != commit->buffer)\n@@ -1328,7 +1344,8 @@ void pretty_print_commit(const struct pretty_print_context *pp,\n \tconst char *encoding;\n \tint need_8bit_cte = pp->need_8bit_cte;\n \n-\tif (pp->fmt == CMIT_FMT_USERFORMAT) {\n+\tif (pp->fmt == CMIT_FMT_USERFORMAT ||\n+\t    pp->fmt == CMIT_FMT_LUA) {\n \t\tformat_commit_message(commit, user_format, sb, pp);\n \t\treturn;\n \t}\n-- \n1.7.12.1.10.g6537447\n"},{"id":"199874","messageId":"CACsJy8ArwPq=YRAddVtC+m8X0+=tndb2-=tmiMS=m2VX3TWGYg@mail.gmail.com","threadId":"31643","inReplyTo":"20120925002511.GB19605@sigill.intra.peff.net","subject":"Re: [PATCH 2/3] add basic lua infrastructure","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-09-25T01:55:23Z","receivedAt":"2012-09-25T01:55:23Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Sep 25, 2012 at 7:25 AM, Jeff King <peff@peff.net> wrote:\n> +ifdef USE_LUA\n> +       BASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n> +       EXTLIBS += `pkg-config --libs lua5.2`\n> +endif\n> +\n\nI remember we paid noticeable penalty when linking with libcurl to\nmain git binary and Linus removed libcurl from main git, moving it to\ngit-http-*. Do we pay similar penalty linking to liblua?\n-- \nDuy\n"},{"id":"199875","messageId":"robbat2-20120925T032006-952720661Z@orbis-terrarum.net","threadId":"31643","inReplyTo":"20120925002511.GB19605@sigill.intra.peff.net","subject":"Re: [PATCH 2/3] add basic lua infrastructure","fromName":"Robin H. Johnson","fromEmail":"robbat2@gentoo.org","sentAt":"2012-09-25T03:21:10Z","receivedAt":"2012-09-25T03:21:10Z","isPatch":true,"sender":{"key":"robbat2@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/373898?v=4"},"body":"On Mon, Sep 24, 2012 at 08:25:12PM -0400,  Jeff King wrote:\n> +ifdef USE_LUA\n> +\tBASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n> +\tEXTLIBS += `pkg-config --libs lua5.2`\n> +endif\nCan you please hoist the packagename out to a variable? It's just plain\n\"lua\" on Gentoo.\n\n-- \nRobin Hugh Johnson\nGentoo Linux: Developer, Trustee & Infrastructure Lead\nE-Mail     : robbat2@gentoo.org\nGnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85\n"},{"id":"199876","messageId":"20120925034211.GC3656@sigill.intra.peff.net","threadId":"31643","inReplyTo":"robbat2-20120925T032006-952720661Z@orbis-terrarum.net","subject":"Re: [PATCH 2/3] add basic lua infrastructure","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-25T03:42:11Z","receivedAt":"2012-09-25T03:42:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 25, 2012 at 03:21:10AM +0000, Robin H. Johnson wrote:\n\n> On Mon, Sep 24, 2012 at 08:25:12PM -0400,  Jeff King wrote:\n> > +ifdef USE_LUA\n> > +\tBASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n> > +\tEXTLIBS += `pkg-config --libs lua5.2`\n> > +endif\n> Can you please hoist the packagename out to a variable? It's just plain\n> \"lua\" on Gentoo.\n\nYeah. I mentioned these patches were very rough, but I didn't go into\ndetail on all the bad points.  That is definitely one of them. I have no\nidea what the \"normal\" name is; my debian system sticks the version\nnumber in to allow multiple concurrent versions.\n\nI was hoping somebody with more Lua experience could tell me what's\nusual. It would be nice if it just worked out of the box as soon as you\nsaid USE_LUA, but that may not be realistic.\n\n-Peff\n"},{"id":"199878","messageId":"20120925045345.GA5708@sigill.intra.peff.net","threadId":"31643","inReplyTo":"CACsJy8ArwPq=YRAddVtC+m8X0+=tndb2-=tmiMS=m2VX3TWGYg@mail.gmail.com","subject":"Re: [PATCH 2/3] add basic lua infrastructure","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-09-25T04:53:45Z","receivedAt":"2012-09-25T04:53:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 25, 2012 at 08:55:23AM +0700, Nguyen Thai Ngoc Duy wrote:\n\n> On Tue, Sep 25, 2012 at 7:25 AM, Jeff King <peff@peff.net> wrote:\n> > +ifdef USE_LUA\n> > +       BASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n> > +       EXTLIBS += `pkg-config --libs lua5.2`\n> > +endif\n> > +\n> \n> I remember we paid noticeable penalty when linking with libcurl to\n> main git binary and Linus removed libcurl from main git, moving it to\n> git-http-*. Do we pay similar penalty linking to liblua?\n\nI don't think so. The real problem with libcurl is that it brings in a\nton of other libraries:\n\n  $ ldd /usr/lib/x86_64-linux-gnu/libcurl.so | awk '{print $1}'\n  linux-vdso.so.1\n  libidn.so.11\n  libssh2.so.1\n  liblber-2.4.so.2\n  libldap_r-2.4.so.2\n  librt.so.1\n  libgssapi_krb5.so.2\n  libssl.so.1.0.0\n  libcrypto.so.1.0.0\n  librtmp.so.0\n  libz.so.1\n  libc.so.6\n  libgcrypt.so.11\n  libresolv.so.2\n  libsasl2.so.2\n  libgnutls.so.26\n  libpthread.so.0\n  /lib64/ld-linux-x86-64.so.2\n  libkrb5.so.3\n  libk5crypto.so.3\n  libcom_err.so.2\n  libkrb5support.so.0\n  libdl.so.2\n  libkeyutils.so.1\n  libgpg-error.so.0\n  libtasn1.so.3\n  libp11-kit.so.0\n\nCompare with lua:\n\n  $ ldd /usr/lib/x86_64-linux-gnu/liblua5.2.so | awk '{print $1}'\n  linux-vdso.so.1\n  libm.so.6\n  libdl.so.2\n  libc.so.6\n  /lib64/ld-linux-x86-64.so.2\n\nThe original timings from Linus are here:\n\n  http://article.gmane.org/gmane.comp.version-control.git/123946\n\nThe main issue is really hitting all those libraries on a cold cache.\nHere are before-and-after timings of:\n\n  echo 3 >/proc/sys/vm/drop_caches && git\n\nwhich should basically just measure startup time. All times are\nbest-of-five.\n\n  [before]\n  real    0m0.065s\n  user    0m0.000s\n  sys     0m0.004s\n\n  [after]\n  real    0m0.063s\n  user    0m0.000s\n  sys     0m0.004s\n\nSo we actually did better, though the difference is well within the\nrun-to-run noise. I don't think it's a big deal.\n\n-Peff\n"},{"id":"199888","messageId":"CACsJy8Bs4P7R7m7uAqHYdmrXJA1RQmuoC7ybJFHWhW=efffQyw@mail.gmail.com","threadId":"31643","inReplyTo":"20120925002325.GA19560@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 0/3] git log --pretty=lua","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-09-25T11:22:49Z","receivedAt":"2012-09-25T11:22:49Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Sep 25, 2012 at 7:23 AM, Jeff King <peff@peff.net> wrote:\n> We've talked off and on about extending the --pretty=format specifiers\n> to something more flexible. There's also been talk recently of more\n> flexible commit-filtering (e.g., grepping individual notes).  Rather\n> than invent a new Turing-complete language, I thought I'd try building\n> on somebody else's work by embedding an existing language.\n>\n> Why Lua? I don't especially like it as a language. But it's designed for\n> this purpose, which makes it very lightweight and relatively simple to\n> embed.\n\nPersonally I'd prefer a Scheme variant.\n\n> The syntax, on the other hand...yuck.\n\nOops. Scheme is out then.\n\n> One thing that makes Lua\n> horrible for this use is that it does not have interpolated strings.\n> However, there are template libraries for Lua, so maybe there's\n> something there.\n\nFor --pretty, the first thing I looked up was utf-8 support and Lua\ndoes not seem to have that built in. Libraries can help but it'll be\nmore verbose than native language support.\n\n> And a \"4/3\" patch would probably add \"--lua-filter\" as a revision option\n> for limiting commits.\n\nI was thinking of nearly the same thing, except that I hide the\nfilters behind sha-1 extended syntax. Users can link <sha-1>@{foo} to\na lua function, for example.\n\nI wonder what areas in git might benefit from such a scripting\nlanguage, and whether someday we would convert some of git builtin\ncommands to $NEWLANG, if $NEWLANG proves easier to maintain for\ncomplex logic commands. grep and rev-list (searching in general) are\nprobably where $NEWLANG shines. But for really complex searches, one\nmay want to go with libgit2 or other bindings to their favourite\nlanguage than one git may provide. gitignore and gitattributes can\nmake use of $NEWLANG but it has to be really fast. There was talk\nabout conditionals in config file, which $NEWLANG might also help.\n\nIn general I'm quite happy with what git provides. There are many\nadvanced features that I have never used/or even aware of.\n-- \nDuy\n"},{"id":"199889","messageId":"1236918118.55838.1348580955644.JavaMail.root@genarts.com","threadId":"31643","inReplyTo":"CACsJy8Bs4P7R7m7uAqHYdmrXJA1RQmuoC7ybJFHWhW=efffQyw@mail.gmail.com","subject":"Re: [RFC/PATCH 0/3] git log --pretty=lua","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2012-09-25T13:49:15Z","receivedAt":"2012-09-25T13:49:15Z","isPatch":true,"sender":{"key":"bash@genarts.com","avatar":null},"body":"----- Original Message -----\n> From: \"Nguyen Thai Ngoc Duy\" <pclouds@gmail.com>\n> Sent: Tuesday, September 25, 2012 7:22:49 AM\n> Subject: Re: [RFC/PATCH 0/3] git log --pretty=lua\n> \n> On Tue, Sep 25, 2012 at 7:23 AM, Jeff King <peff@peff.net> wrote:\n> > We've talked off and on about extending the --pretty=format\n> > specifiers to something more flexible. There's also been talk\n> > recently of more flexible commit-filtering (e.g., grepping\n> > individual notes).  Rather than invent a new Turing-complete\n> > language, I thought I'd try building on somebody else's work by\n> > embedding an existing language.\n> >\n> > Why Lua? I don't especially like it as a language. But it's designed\n> > for this purpose, which makes it very lightweight and relatively\n> > simple to embed.\n> \n> Personally I'd prefer a Scheme variant.\n\nScheme only brings up bad memories for me ;)  And while we use Lua at\n$dayjob, I, like Peff, am not a huge fan of the syntax.  So turning to\nthe internet to solve my problem, a quick Google search for embeddable\nscripting languages (assuming the heavyweights like Perl and Python are\nalready out) produces Lua, JavaScript or specifically SpiderMonkey [1]\n(yay buzzword compliance!), Ch [2] (unfortunately closed source), and\nAngelScript [3].\n\n>From a brief read of the webpage, AngelScript looks pretty interesting.\nI'm much better with (and thus preferential to) Python and Perl myself,\nbut I can understand anyone's reservation in bundling/depending on\nlibraries of that size.\n\n[1] https://developer.mozilla.org/en-US/docs/SpiderMonkey\n[2] http://www.softintegration.com/\n[3] http://angelcode.com/angelscript/\n\nThanks,\nStephen\n"},{"id":"199894","messageId":"vpqvcf2ruyv.fsf@grenoble-inp.fr","threadId":"31643","inReplyTo":"20120925002325.GA19560@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 0/3] git log --pretty=lua","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2012-09-25T15:19:20Z","receivedAt":"2012-09-25T15:19:20Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Jeff King <peff@peff.net> writes:\n\n> We've talked off and on about extending the --pretty=format specifiers\n> to something more flexible. There's also been talk recently of more\n> flexible commit-filtering (e.g., grepping individual notes).\n\nMercurial has a similar thing, which can be a source of inspiration:\n\nhttp://www.selenic.com/hg/help/revsets\n\nOn the one hand, if find it a bit overkill to have a full language for\nthis, but on the other hand, it allows expressing easily and explicitely\nboolean operators.\n\nI would find\n\n  git log 'grep(foo) or grep(bar)'\n\nvery intuitive and elegant, while I never know whether\n\n  git log --grep=foo --grep=bar\n\nis a OR or a AND (there was a patch recently to clarify the doc).\n\nNote that Mercurial's version seem to really act on sets of commits, not\njust individual commits, as it allows things like\n\n  hg log -r 'sort(date(\"May 2008\"), user)'\n\nor the equivalent of git's negative revision argument :\n\n  \"x - y\"\n      Changesets in x but not in y.\n\nSo it would be more a new \"git log --filter='some expression'\", not a\n--pretty=lua.\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"199897","messageId":"7vhaqmvywo.fsf@alter.siamese.dyndns.org","threadId":"31643","inReplyTo":"vpqvcf2ruyv.fsf@grenoble-inp.fr","subject":"Re: [RFC/PATCH 0/3] git log --pretty=lua","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-09-25T16:40:39Z","receivedAt":"2012-09-25T16:40:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:\n\n> Jeff King <peff@peff.net> writes:\n>\n>> We've talked off and on about extending the --pretty=format specifiers\n>> to something more flexible. There's also been talk recently of more\n>> flexible commit-filtering (e.g., grepping individual notes).\n>\n> Mercurial has a similar thing, which can be a source of inspiration:\n>\n> http://www.selenic.com/hg/help/revsets\n>\n> On the one hand, if find it a bit overkill to have a full language for\n> this, but on the other hand, it allows expressing easily and explicitely\n> boolean operators.\n>\n> I would find\n>\n>   git log 'grep(foo) or grep(bar)'\n>\n> very intuitive and elegant,...\n\nYou have to be careful with \"grep\", though.  It would be unclear\nwhat \"and\" there would mean if you replaced your \"or\" with.\n\nPeff's earlier examples e.g.\n\n  git log --lua-filter='\n    return\n      author().name.match(\"Junio\") &&\n      note(\"p4\").match(\"1234567\")\n  '\n\n  git log --lua-filter='return subject().len > 100'\n\nare clearly good ones that illustrate the power of filtering.\n"},{"id":"199909","messageId":"7vy5jxvm1l.fsf@alter.siamese.dyndns.org","threadId":"31643","inReplyTo":"20120925034211.GC3656@sigill.intra.peff.net","subject":"Re: [PATCH 2/3] add basic lua infrastructure","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-09-25T21:18:30Z","receivedAt":"2012-09-25T21:18:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Tue, Sep 25, 2012 at 03:21:10AM +0000, Robin H. Johnson wrote:\n>\n>> On Mon, Sep 24, 2012 at 08:25:12PM -0400,  Jeff King wrote:\n>> > +ifdef USE_LUA\n>> > +\tBASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n>> > +\tEXTLIBS += `pkg-config --libs lua5.2`\n>> > +endif\n>> Can you please hoist the packagename out to a variable? It's just plain\n>> \"lua\" on Gentoo.\n>\n> Yeah. I mentioned these patches were very rough, but I didn't go into\n> detail on all the bad points.  That is definitely one of them. I have no\n> idea what the \"normal\" name is; my debian system sticks the version\n> number in to allow multiple concurrent versions.\n\nYeah, there is no point nitpicking yet.  Even the choice of lua is\nnot all that interesting; embedding _any_ reasonable interpreter,\nand figuring out which operations and codepaths in us benefit most\nfrom such embedding, are of bigger interest at this early stage.\n\nHow about doing this on top at the minimum?  You can let pkg-config\nto tell you where -I<directory> and what -l<lib> is, or you can set\nit yourself.\n\n    $ make USE_LUA=YesPlease \\\n    \tLUA_INCLUDE_ARG=-I/usr/include/lua5.2 \\\n        LUA_LINK_ARG=-llua5.2\n\nor\n\n    $ make USE_LUA=lua5.2\n\n\n Makefile | 10 ++++++++--\n 1 file changed, 8 insertions(+), 2 deletions(-)\n\ndiff --git i/Makefile w/Makefile\nindex 620df89..90335ba 100644\n--- i/Makefile\n+++ w/Makefile\n@@ -1898,8 +1898,14 @@ ifdef USE_NED_ALLOCATOR\n endif\n \n ifdef USE_LUA\n-\tBASIC_CFLAGS += -DUSE_LUA `pkg-config --cflags lua5.2`\n-\tEXTLIBS += `pkg-config --libs lua5.2`\n+\t# You can say\n+\t# $ make USE_LUA=YesPlease LUA_INCLUDE_ARG=-I/usr/include\n+\t# or\n+\t# $ make USE_LUA=lua5.2\n+\tLUA_INCLUDE_ARG ?= $(shell pkg-config --cflags $(USE_LUA))\n+\tLUA_LINK_ARG ?= $(shell pkg-config --libs $(USE_LUA))\n+\tBASIC_CFLAGS += -DUSE_LUA $(LUA_INCLUDE_ARG)\n+\tEXTLIBS += $(LUA_LINK_ARG)\n endif\n \n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n"},{"id":"200191","messageId":"CACsJy8AvC9=6owwzMByZq2Wv1P_YS5xCUXRHH_Lr4qHA1BXyRQ@mail.gmail.com","threadId":"31643","inReplyTo":"20120925002325.GA19560@sigill.intra.peff.net","subject":"Re: [RFC/PATCH 0/3] git log --pretty=lua","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-09-30T04:31:13Z","receivedAt":"2012-09-30T04:31:13Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Sep 25, 2012 at 7:23 AM, Jeff King <peff@peff.net> wrote:\n> Why Lua? I don't especially like it as a language. But it's designed for\n> this purpose, which makes it very lightweight and relatively simple to\n> embed.\n\nAnother option is tcl. String-based approach feels more natural for\npretty.c and shell users. >=8.1 supports utf-8. We might be able to\nshare some routines between the embedded tcl interpreter and\ngitk/git-gui. The drawback is we won't get the speed of luajit if we\never need to (I'm looking at filter-branch, but I assume eliminating\nfork() should already give decent speedup).\n-- \nDuy\n"},{"id":"200675","messageId":"20121006173329.GA31012@sigill.intra.peff.net","threadId":"31643","inReplyTo":"20120925002539.GC19605@sigill.intra.peff.net","subject":"Re: [PATCH 3/3] add a \"lua\" pretty format","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-10-06T17:33:29Z","receivedAt":"2012-10-06T17:33:29Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 24, 2012 at 08:25:39PM -0400, Jeff King wrote:\n\n> @@ -1168,7 +1180,11 @@ void format_commit_message(const struct commit *commit,\n>  \t\tfree(enc);\n>  \t}\n>  \n> -\tstrbuf_expand(sb, format, format_commit_item, &context);\n> +\tif (pretty_ctx->fmt == CMIT_FMT_USERFORMAT)\n> +\t\tstrbuf_expand(sb, format, format_commit_item, &context);\n> +\telse if (pretty_ctx->fmt == CMIT_FMT_LUA)\n> +\t\tlua_commit_format(sb, &context);\n> +\n\nThis hunk breaks lots of tests. I know we are not seriously considering\nthe lua series as-is, but in case anybody wants to play with it, here is\nthe fix (and we would need the same fix regardless of language, anyway).\n\nYou might want to queue this on jk/lua-hackery (probably it would be\nsquashed in for a real series).\n\n-- >8 --\nSubject: [PATCH] pretty: fix up one-off format_commit_message calls\n\nIf the usual pretty-print code invokes format_commit_message,\nthe \"fmt\" field of the pretty_print_context will always have\neither CMIT_FMT_USERFORMAT or CMIT_FMT_LUA in it, and we can\njust choose which to use.\n\nHowever, many call sites invoke format_commit_message\ndirectly without bothering to set the \"fmt\" field of the\ncontext; they expect format_commit_message to just default\nto CMIT_FMT_USERFORMAT in that case, since previously that\nwas the only format it handled.\n\nThe recent addition of the lua formatter broke that\nassumption. Rather than require each caller to be more\nstrict, let's just default to USERFORMAT when the format is\nset to something nonsensical.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n pretty.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/pretty.c b/pretty.c\nindex fdd4258..7289590 100644\n--- a/pretty.c\n+++ b/pretty.c\n@@ -1180,10 +1180,10 @@ void format_commit_message(const struct commit *commit,\n \t\tfree(enc);\n \t}\n \n-\tif (pretty_ctx->fmt == CMIT_FMT_USERFORMAT)\n-\t\tstrbuf_expand(sb, format, format_commit_item, &context);\n-\telse if (pretty_ctx->fmt == CMIT_FMT_LUA)\n+\tif (pretty_ctx->fmt == CMIT_FMT_LUA)\n \t\tlua_commit_format(sb, &context);\n+\telse\n+\t\tstrbuf_expand(sb, format, format_commit_item, &context);\n \n \trewrap_message_tail(sb, &context, 0, 0, 0);\n \n-- \n1.8.0.rc0.22.g285fd2d\n"}]}