{"thread":{"id":"25598","subject":"[WIP RFC/PATCH] WIP: git-sh-i18n--envsubst for a correct eval_gettext","startedAt":"2010-10-30T15:22:43Z","lastAt":"2010-10-30T15:22:43Z","messageCount":1,"participants":["Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"154850","messageId":"1288452163-814-1-git-send-email-avarab@gmail.com","threadId":"25598","inReplyTo":null,"subject":"[WIP RFC/PATCH] WIP: git-sh-i18n--envsubst for a correct eval_gettext","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-10-30T15:22:43Z","receivedAt":"2010-10-30T15:22:43Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"t/t0201-gettext-fallbacks.sh: test for broken eval_gettext\n\nAdd a test for the broken eval_gettext() variable interpolation\nbehavior.\n\nReported-by: Johannes Sixt <j.sixt@viscovery.net>\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n\ngit-sh-i18n.sh: use envsubst\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n\nWIP: add a git-sh-i18n--envsubst\n\nI can't figure out how to make the Makefile read from\ngit-sh-i18n--envsubst.c and produce git-sh-i18n--envsubst, not\ngit-git-sh-i18n--envsubst.\n\nI also can't see how to make it available to git tests.\n\nIf I name it sh-i18n--envsubst.c I'll get linking errors (about\nxrealloc).\n---\n\nHeading out so I thought I'd dump a WIP patch here and ask if anyone\nknows how to solve this Makefile issue I'm having.\n\nI wanted to call the two programs\n/{git-sh-i18n.sh,git-sh-i18n--envsubst.c}, but maybe going that route\nis the wrong thing to do?\n\n .gitignore                   |    1 +\n Makefile                     |    3 +\n git-sh-i18n--envsubst.c      |  392 ++++++++++++++++++++++++++++++++++++++++++\n git-sh-i18n.sh               |    3 +-\n t/t0201-gettext-fallbacks.sh |   23 +++\n 5 files changed, 420 insertions(+), 2 deletions(-)\n create mode 100644 git-sh-i18n--envsubst.c\n\ndiff --git a/.gitignore b/.gitignore\nindex 80ca718..d2d9ec2 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -126,6 +126,7 @@\n /git-send-email\n /git-send-pack\n /git-sh-i18n\n+/git-sh-i18n--envsubst\n /git-sh-setup\n /git-shell\n /git-shortlog\ndiff --git a/Makefile b/Makefile\nindex e1650ba..cdfe6bc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -435,6 +435,7 @@ PROGRAM_OBJS += shell.o\n PROGRAM_OBJS += show-index.o\n PROGRAM_OBJS += upload-pack.o\n PROGRAM_OBJS += http-backend.o\n+PROGRAM_OBJS += git-sh-i18n--envsubst.o\n \n PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS))\n \n@@ -2016,6 +2017,8 @@ git-http-fetch$X: revision.o http.o http-walker.o http-fetch.o $(GITLIBS)\n git-http-push$X: revision.o http.o http-push.o $(GITLIBS)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(LIBS) $(CURL_LIBCURL) $(EXPAT_LIBEXPAT)\n+git-sh-i18n--envsubst$X: git-sh-i18n--envsubst.o $(GITLIBS)\n+\t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^)\n \n $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)\n \t$(QUIET_LNCP)$(RM) $@ && \\\ndiff --git a/git-sh-i18n--envsubst.c b/git-sh-i18n--envsubst.c\nnew file mode 100644\nindex 0000000..3be7e8e\n--- /dev/null\n+++ b/git-sh-i18n--envsubst.c\n@@ -0,0 +1,392 @@\n+/* This is a modified version of\n+   67d0871a8c:gettext-runtime/src/envsubst.c from the gettext.git\n+   repository that does only the envsubst that we need in the\n+   git-sh-i18n fallbacks.\n+*/\n+\n+#include \"git-compat-util.h\"\n+\n+/* Substitution of environment variables in shell format strings.\n+   Copyright (C) 2003-2007 Free Software Foundation, Inc.\n+   Written by Bruno Haible <bruno@clisp.org>, 2003.\n+\n+   This program is free software; you can redistribute it and/or modify\n+   it under the terms of the GNU General Public License as published by\n+   the Free Software Foundation; either version 2, or (at your option)\n+   any later version.\n+\n+   This program is distributed in the hope that it will be useful,\n+   but WITHOUT ANY WARRANTY; without even the implied warranty of\n+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n+   GNU General Public License for more details.\n+\n+   You should have received a copy of the GNU General Public License\n+   along with this program; if not, write to the Free Software Foundation,\n+   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */\n+\n+#include <errno.h>\n+#include <getopt.h>\n+#include <stdbool.h>\n+#include <stdio.h>\n+#include <stdlib.h>\n+#include <string.h>\n+\n+/* If true, substitution shall be performed on all variables.  */\n+static bool all_variables;\n+\n+static void print_variables (const char *string);\n+static void note_variables (const char *string);\n+static void subst_from_stdin (void);\n+\n+int\n+main (int argc, char *argv[])\n+{\n+  all_variables = true;\n+  subst_from_stdin ();\n+\n+  exit (EXIT_SUCCESS);\n+}\n+\n+/* Parse the string and invoke the callback each time a $VARIABLE or\n+   ${VARIABLE} construct is seen, where VARIABLE is a nonempty sequence\n+   of ASCII alphanumeric/underscore characters, starting with an ASCII\n+   alphabetic/underscore character.\n+   We allow only ASCII characters, to avoid dependencies w.r.t. the current\n+   encoding: While \"${\\xe0}\" looks like a variable access in ISO-8859-1\n+   encoding, it doesn't look like one in the BIG5, BIG5-HKSCS, GBK, GB18030,\n+   SHIFT_JIS, JOHAB encodings, because \\xe0\\x7d is a single character in these\n+   encodings.  */\n+static void\n+find_variables (const char *string,\n+\t\tvoid (*callback) (const char *var_ptr, size_t var_len))\n+{\n+  for (; *string != '\\0';)\n+    if (*string++ == '$')\n+      {\n+\tconst char *variable_start;\n+\tconst char *variable_end;\n+\tbool valid;\n+\tchar c;\n+\n+\tif (*string == '{')\n+\t  string++;\n+\n+\tvariable_start = string;\n+\tc = *string;\n+\tif ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_')\n+\t  {\n+\t    do\n+\t      c = *++string;\n+\t    while ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')\n+\t\t   || (c >= '0' && c <= '9') || c == '_');\n+\t    variable_end = string;\n+\n+\t    if (variable_start[-1] == '{')\n+\t      {\n+\t\tif (*string == '}')\n+\t\t  {\n+\t\t    string++;\n+\t\t    valid = true;\n+\t\t  }\n+\t\telse\n+\t\t  valid = false;\n+\t      }\n+\t    else\n+\t      valid = true;\n+\n+\t    if (valid)\n+\t      callback (variable_start, variable_end - variable_start);\n+\t  }\n+      }\n+}\n+\n+\n+/* Print a variable to stdout, followed by a newline.  */\n+static void\n+print_variable (const char *var_ptr, size_t var_len)\n+{\n+  fwrite (var_ptr, var_len, 1, stdout);\n+  putchar ('\\n');\n+}\n+\n+/* Print the variables contained in STRING to stdout, each one followed by a\n+   newline.  */\n+static void\n+print_variables (const char *string)\n+{\n+  find_variables (string, &print_variable);\n+}\n+\n+\n+/* Type describing list of immutable strings,\n+   implemented using a dynamic array.  */\n+typedef struct string_list_ty string_list_ty;\n+struct string_list_ty\n+{\n+  const char **item;\n+  size_t nitems;\n+  size_t nitems_max;\n+};\n+\n+/* Initialize an empty list of strings.  */\n+static inline void\n+string_list_init (string_list_ty *slp)\n+{\n+  slp->item = NULL;\n+  slp->nitems = 0;\n+  slp->nitems_max = 0;\n+}\n+\n+/* Append a single string to the end of a list of strings.  */\n+static inline void\n+string_list_append (string_list_ty *slp, const char *s)\n+{\n+  /* Grow the list.  */\n+  if (slp->nitems >= slp->nitems_max)\n+    {\n+      size_t nbytes;\n+\n+      slp->nitems_max = slp->nitems_max * 2 + 4;\n+      nbytes = slp->nitems_max * sizeof (slp->item[0]);\n+      slp->item = (const char **) xrealloc (slp->item, nbytes);\n+    }\n+\n+  /* Add the string to the end of the list.  */\n+  slp->item[slp->nitems++] = s;\n+}\n+\n+/* Compare two strings given by reference.  */\n+static int\n+cmp_string (const void *pstr1, const void *pstr2)\n+{\n+  const char *str1 = *(const char **)pstr1;\n+  const char *str2 = *(const char **)pstr2;\n+\n+  return strcmp (str1, str2);\n+}\n+\n+/* Sort a list of strings.  */\n+static inline void\n+string_list_sort (string_list_ty *slp)\n+{\n+  if (slp->nitems > 0)\n+    qsort (slp->item, slp->nitems, sizeof (slp->item[0]), cmp_string);\n+}\n+\n+/* Test whether a string list contains a given string.  */\n+static inline int\n+string_list_member (const string_list_ty *slp, const char *s)\n+{\n+  size_t j;\n+\n+  for (j = 0; j < slp->nitems; ++j)\n+    if (strcmp (slp->item[j], s) == 0)\n+      return 1;\n+  return 0;\n+}\n+\n+/* Test whether a sorted string list contains a given string.  */\n+static int\n+sorted_string_list_member (const string_list_ty *slp, const char *s)\n+{\n+  size_t j1, j2;\n+\n+  j1 = 0;\n+  j2 = slp->nitems;\n+  if (j2 > 0)\n+    {\n+      /* Binary search.  */\n+      while (j2 - j1 > 1)\n+\t{\n+\t  /* Here we know that if s is in the list, it is at an index j\n+\t     with j1 <= j < j2.  */\n+\t  size_t j = (j1 + j2) >> 1;\n+\t  int result = strcmp (slp->item[j], s);\n+\n+\t  if (result > 0)\n+\t    j2 = j;\n+\t  else if (result == 0)\n+\t    return 1;\n+\t  else\n+\t    j1 = j + 1;\n+\t}\n+      if (j2 > j1)\n+\tif (strcmp (slp->item[j1], s) == 0)\n+\t  return 1;\n+    }\n+  return 0;\n+}\n+\n+/* Destroy a list of strings.  */\n+static inline void\n+string_list_destroy (string_list_ty *slp)\n+{\n+  size_t j;\n+\n+  for (j = 0; j < slp->nitems; ++j)\n+    free ((char *) slp->item[j]);\n+  if (slp->item != NULL)\n+    free (slp->item);\n+}\n+\n+\n+/* Set of variables on which to perform substitution.\n+   Used only if !all_variables.  */\n+static string_list_ty variables_set;\n+\n+/* Adds a variable to variables_set.  */\n+static void\n+note_variable (const char *var_ptr, size_t var_len)\n+{\n+  char *string = xmalloc (var_len + 1);\n+  memcpy (string, var_ptr, var_len);\n+  string[var_len] = '\\0';\n+\n+  string_list_append (&variables_set, string);\n+}\n+\n+/* Stores the variables occurring in the string in variables_set.  */\n+static void\n+note_variables (const char *string)\n+{\n+  string_list_init (&variables_set);\n+  find_variables (string, &note_variable);\n+  string_list_sort (&variables_set);\n+}\n+\n+\n+static int\n+do_getc ()\n+{\n+  int c = getc (stdin);\n+\n+  if (c == EOF)\n+    {\n+      if (ferror (stdin))\n+\terror (\"error while reading standard input\");\n+    }\n+\n+  return c;\n+}\n+\n+static inline void\n+do_ungetc (int c)\n+{\n+  if (c != EOF)\n+    ungetc (c, stdin);\n+}\n+\n+/* Copies stdin to stdout, performing substitutions.  */\n+static void\n+subst_from_stdin ()\n+{\n+  static char *buffer;\n+  static size_t bufmax;\n+  static size_t buflen;\n+  int c;\n+\n+  for (;;)\n+    {\n+      c = do_getc ();\n+      if (c == EOF)\n+\tbreak;\n+      /* Look for $VARIABLE or ${VARIABLE}.  */\n+      if (c == '$')\n+\t{\n+\t  bool opening_brace = false;\n+\t  bool closing_brace = false;\n+\n+\t  c = do_getc ();\n+\t  if (c == '{')\n+\t    {\n+\t      opening_brace = true;\n+\t      c = do_getc ();\n+\t    }\n+\t  if ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_')\n+\t    {\n+\t      bool valid;\n+\n+\t      /* Accumulate the VARIABLE in buffer.  */\n+\t      buflen = 0;\n+\t      do\n+\t\t{\n+\t\t  if (buflen >= bufmax)\n+\t\t    {\n+\t\t      bufmax = 2 * bufmax + 10;\n+\t\t      buffer = xrealloc (buffer, bufmax);\n+\t\t    }\n+\t\t  buffer[buflen++] = c;\n+\n+\t\t  c = do_getc ();\n+\t\t}\n+\t      while ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')\n+\t\t     || (c >= '0' && c <= '9') || c == '_');\n+\n+\t      if (opening_brace)\n+\t\t{\n+\t\t  if (c == '}')\n+\t\t    {\n+\t\t      closing_brace = true;\n+\t\t      valid = true;\n+\t\t    }\n+\t\t  else\n+\t\t    {\n+\t\t      valid = false;\n+\t\t      do_ungetc (c);\n+\t\t    }\n+\t\t}\n+\t      else\n+\t\t{\n+\t\t  valid = true;\n+\t\t  do_ungetc (c);\n+\t\t}\n+\n+\t      if (valid)\n+\t\t{\n+\t\t  /* Terminate the variable in the buffer.  */\n+\t\t  if (buflen >= bufmax)\n+\t\t    {\n+\t\t      bufmax = 2 * bufmax + 10;\n+\t\t      buffer = xrealloc (buffer, bufmax);\n+\t\t    }\n+\t\t  buffer[buflen] = '\\0';\n+\n+\t\t  /* Test whether the variable shall be substituted.  */\n+\t\t  if (!all_variables\n+\t\t      && !sorted_string_list_member (&variables_set, buffer))\n+\t\t    valid = false;\n+\t\t}\n+\n+\t      if (valid)\n+\t\t{\n+\t\t  /* Substitute the variable's value from the environment.  */\n+\t\t  const char *env_value = getenv (buffer);\n+\n+\t\t  if (env_value != NULL)\n+\t\t    fputs (env_value, stdout);\n+\t\t}\n+\t      else\n+\t\t{\n+\t\t  /* Perform no substitution at all.  Since the buffered input\n+\t\t     contains no other '$' than at the start, we can just\n+\t\t     output all the buffered contents.  */\n+\t\t  putchar ('$');\n+\t\t  if (opening_brace)\n+\t\t    putchar ('{');\n+\t\t  fwrite (buffer, buflen, 1, stdout);\n+\t\t  if (closing_brace)\n+\t\t    putchar ('}');\n+\t\t}\n+\t    }\n+\t  else\n+\t    {\n+\t      do_ungetc (c);\n+\t      putchar ('$');\n+\t      if (opening_brace)\n+\t\tputchar ('{');\n+\t    }\n+\t}\n+      else\n+\tputchar (c);\n+    }\n+}\ndiff --git a/git-sh-i18n.sh b/git-sh-i18n.sh\nindex f8dd43a..b3b599a 100644\n--- a/git-sh-i18n.sh\n+++ b/git-sh-i18n.sh\n@@ -55,8 +55,7 @@ then\n \t\t}\n \n \t\teval_gettext () {\n-\t\t\tgettext_eval=\"printf '%s' \\\"$1\\\"\"\n-\t\t\tprintf \"%s\" \"`eval \\\"$gettext_eval\\\"`\"\n+\t\t\tprintf \"%s\" \"$1\" | /home/avar/g/git/git-git-sh-i18n--envsubst\n \t\t}\n \tfi\n else\ndiff --git a/t/t0201-gettext-fallbacks.sh b/t/t0201-gettext-fallbacks.sh\nindex 7a85d9b..682c602 100755\n--- a/t/t0201-gettext-fallbacks.sh\n+++ b/t/t0201-gettext-fallbacks.sh\n@@ -46,4 +46,27 @@ test_expect_success NO_GETTEXT_POISON 'eval_gettext: our eval_gettext() fallback\n     test_cmp expect actual\n '\n \n+test_expect_success NO_GETTEXT_POISON 'eval_gettext: our eval_gettext() fallback can interpolate whitespace variables' '\n+    git_am_cmdline=\"git am\" &&\n+    export git_am_cmdline &&\n+    printf \"test git am\" >expect &&\n+    eval_gettext \"test \\$git_am_cmdline\" >actual &&\n+    test_cmp expect actual\n+'\n+\n+test_expect_success NO_GETTEXT_POISON 'eval_gettext: git am $cmdline bug' '\n+    cmdline=\"git am -3\" &&\n+    export cmdline &&\n+    cat >expect <<EOF &&\n+When you have resolved this problem run \"git am -3 --resolved\".\n+If you would prefer to skip this patch, instead run \"git am -3 --skip\".\n+To restore the original branch and stop patching run \"git am -3 --abort\".\n+EOF\n+    eval_gettext \"When you have resolved this problem run \\\"\\$cmdline --resolved\\\".\n+If you would prefer to skip this patch, instead run \\\"\\$cmdline --skip\\\".\n+To restore the original branch and stop patching run \\\"\\$cmdline --abort\\\".\" >actual &&\n+    echo >>actual &&\n+    test_cmp expect actual\n+'\n+\n test_done\n-- \n1.7.3.2.309.g4cea8.dirty\n"}]}