git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v9 5/8] convert: add 'working-tree-encoding' attribute

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Mar 6, 2018, 20:42 UTC
Message-ID
<CAPig+cTOpBODeoHV=+4-4MEjKM=pUrZSa=BEmHh4mVP=xPpFCA@mail.gmail.com>
In-Reply-To
<20180304201418.60958-6-lars.schneider@autodesk.com>
On Sun, Mar 4, 2018 at 3:14 PM,  <lars.schneider@autodesk.com> wrote:
Show 18 quoted lines
> Git recognizes files encoded with ASCII or one of its supersets (e.g.
> UTF-8 or ISO-8859-1) as text files. All other encodings are usually
> interpreted as binary and consequently built-in Git text processing
> tools (e.g. 'git diff') as well as most Git web front ends do not
> visualize the content.
> [...]
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
> diff --git a/convert.c b/convert.c
> @@ -978,6 +1051,25 @@ static int ident_to_worktree(const char *path, const char *src, size_t len,
> +static const char *git_path_check_encoding(struct attr_check_item *check)
> +{
> +       [...]
> +       /*
> +        * Ensure encoding names are always upper case (e.g. UTF-8) to
> +        * simplify subsequent string comparisons.
> +        */
> +       return xstrdup_toupper(value);
xstrdup_toupper() allocates memory...
Show 6 quoted lines
> +}
> @@ -1033,6 +1125,7 @@ struct conv_attrs {
>         enum crlf_action attr_action; /* What attr says */
>         enum crlf_action crlf_action; /* When no attr is set, use core.autocrlf */
>         int ident;
> +       const char *working_tree_encoding; /* Supported encoding or default encoding if NULL */
...which is assigned to 'const char *'...
Show 6 quoted lines
>  };
> @@ -1064,6 +1158,7 @@ static void convert_attrs(struct conv_attrs *ca, const char *path)
>                         else if (eol_attr == EOL_CRLF)
>                                 ca->crlf_action = CRLF_TEXT_CRLF;
>                 }
> +               ca->working_tree_encoding = git_path_check_encoding(ccheck + 5);
...by this code, and eventually leaked.

It's too bad it isn't cleaned up (freed), but looking at the callers, fixing this leak would be mildly noisy (though not particularly invasive). How much do we care about this leak?

Show 7 quoted lines
>         } else {
>                 ca->drv = NULL;
>                 ca->crlf_action = CRLF_UNDEFINED;
> diff --git a/t/t0028-working-tree-encoding.sh b/t/t0028-working-tree-encoding.sh
> @@ -0,0 +1,135 @@
> +test_expect_success 'check $GIT_DIR/info/attributes support' '
> +       test_when_finished "rm -f test.utf8.raw test.utf32.raw test.utf32.git" &&

It seems weird to be cleaning up files this test didn't create (test.utf8.raw and test.utf32.raw).

Show 12 quoted lines
> +       test_when_finished "git reset --hard HEAD" &&
> +
> +       echo "*.utf32 text working-tree-encoding=utf-32" >.git/info/attributes &&
> +       git add test.utf32 &&
> +
> +       git cat-file -p :test.utf32 >test.utf32.git &&
> +       test_cmp_bin test.utf8.raw test.utf32.git
> +'
> +
> +test_expect_success 'check unsupported encodings' '
> +       test_when_finished "rm -f err.out" &&
> +       test_when_finished "git reset --hard HEAD" &&

Resetting to HEAD here is an important cleanup action, but tests don't usually clean up files such as 'err.out' since such detritus doesn't usually impact subsequent tests negatively. (Just an observation; no re-roll needed.)

Show 9 quoted lines
> +       echo "*.nothing text working-tree-encoding=" >>.gitattributes &&
> +       printf "nothing" >t.nothing &&
> +       git add t.nothing &&
> +
> +       echo "*.garbage text working-tree-encoding=garbage" >>.gitattributes &&
> +       printf "garbage" >t.garbage &&
> +       test_must_fail git add t.garbage 2>err.out &&
> +       test_i18ngrep "fatal: failed to encode" err.out
> +'
Previous: lars.schneider@autodesk.comNext: Lars Schneider
Message 15 of 26 in “convert: add support for different encodings”
  1. 0/8 convert: add support for different encodingslars.schneider@autodesk.com, Mar 4, 2018
  2. 1/8 strbuf: remove unnecessary NUL assignment in xstrdup_tolower()lars.schneider@autodesk.com, Mar 4, 2018
  3. 3/8 utf8: add function to detect prohibited UTF-16/32 BOMlars.schneider@autodesk.com, Mar 4, 2018
  4. 2/8 strbuf: add xstrdup_toupper()lars.schneider@autodesk.com, Mar 4, 2018
  5. 4/8 utf8: add function to detect a missing UTF-16/32 BOMlars.schneider@autodesk.com, Mar 4, 2018
  6. Junio C HamanoMar 6, 2018
  7. Lars SchneiderMar 6, 2018
  8. Junio C HamanoMar 6, 2018
  9. Lars SchneiderMar 6, 2018
  10. Junio C HamanoMar 6, 2018
  11. Lars SchneiderMar 6, 2018
  12. Junio C HamanoMar 6, 2018
  13. Torsten BögershausenMar 7, 2018
  14. 5/8 convert: add 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 4, 2018
  15. Eric SunshineMar 6, 2018
  16. Lars SchneiderMar 6, 2018
  17. Eric SunshineMar 6, 2018
  18. 8/8 convert: add round trip check based on 'core.checkRoundtripEncoding'lars.schneider@autodesk.com, Mar 4, 2018
  19. 6/8 convert: check for detectable errors in UTF encodingslars.schneider@autodesk.com, Mar 4, 2018
  20. Junio C HamanoMar 5, 2018
  21. Lars SchneiderMar 5, 2018
  22. Junio C HamanoMar 6, 2018
  23. Lars SchneiderMar 6, 2018
  24. Eric SunshineMar 6, 2018
  25. 7/8 convert: add tracing for 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 4, 2018
  26. Eric SunshineMar 6, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.