git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Extend sample pre-commit hook to check for non ascii filenames

From
Heiko Voigt <hvoigt@hvoigt.net>
Date
May 18, 2009, 09:50 UTC
Message-ID
<20090518095044.GB910@heiko-voigts-macbook.local>
In-Reply-To
<200905151657.47225.jnareb@gmail.com>

At the moment non-ascii encodings of filenames are not portably converted between different filesystems by git. This will most likely change in the future but to allow repositories to be portable among different file/operating systems this check is enabled by default.

Signed-off-by: Heiko <hvoigt@hvoigt.net>
---
so here is a third version ...
On Fri, May 15, 2009 at 04:57:45PM +0200, Jakub Narebski wrote:
Show 11 quoted lines
> On Thu, 14 May 2009, Heiko Voigt wrote:
> 
> > At the moment non-ascii encodings of filenames are not portably converted
> > between different filesystems by git. This will most likely change in the
> > future but to allow repositories to be portable among different file/operating
> > systems this check is enabled by default.
> 
> By the way, you might consider choosing shorter line length for commits,
> something around 70-76 characters per line; otherwise it is harder to
> reply to without linewrapping. 80 characters that you used is, IMHO,
> absolute maximum, and it is good that you kept to it.
Yeah, I admit they were a little bit long.
Show 11 quoted lines
> > +function is_ascii () {
> > +    test -z "$(cat | sed -e "s/[\ -~]*//g")"
> > +    return $?
> > +}
> 
> From CodingGuidelines for shell scripts:
>  - We do not write the noiseword "function" in front of shell
>    functions.
> 
> (in short: do not use bash-specific features... unless, of course,
> you are modifying bash-completion script).
Addressed.
Show 5 quoted lines
> Second, it would be nice to have comment about how to use this
> function (as it does not check file given by its argument, but
> rather its standard input). And perhaps also a comment that it
> works because ASCII printable characters begin with ' ' space
> (does it have to be escaped?) and end with '~' tilde[2].
Done
Show 5 quoted lines
> 
> Third, isn't it useless use of 'cat'[3]? And wouldn't it be better
> to use 'tr' to either delete printable characters and check for
> anything left (as you do; BTW. wouldn't "return test ..." be simpler?),
> or use 'tr' to count non portable characters?
Yes indeed it was useless. I also switched from sed to tr.
> 
> [1] http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html
> [2] http://en.wikipedia.org/wiki/ASCII#ASCII_printable_characters
> [3] http://partmaps.org/era/unix/award.html#cat
On Fri, May 15, 2009 at 11:11:12AM -0700, Junio C Hamano wrote:
Show 5 quoted lines
> Heiko Voigt <hvoigt@hvoigt.net> writes:
> > +function is_ascii () {
> 
> We do not say "#!/bin/bash" at the beginning (hopefully), so let's not say
> "function " here.
See above.
> > +    test -z "$(cat | sed -e "s/[\ -~]*//g")"
> 
> Do you need "cat | "?
Also above.
> Does this script run under LC_ALL=C?  Can an i18n'ized sed interfere with
> what you are trying to do?

I now explicitely set LC_ALL=C for the tr call which should now be robust against such cases.

Show 5 quoted lines
> 
> > +    return $?
> 
> Do you need this, or does the function return the result of the last
> statment anyway?
I wasn't aware of that. Removed the return.
Show 5 quoted lines
> > +		echo "Non-ascii filenames are not allowed !"
> > +		echo "Please rename the file ..."
> 
> Can we make this sound more like a _sample_ project policy?  It's not like
> we enforce that policy to other people's projects.
I've polished this so we are now more user friendly as well.
 templates/hooks--pre-commit.sample |   32 ++++++++++++++++++++++++++++++++
 1 files changed, 32 insertions(+), 0 deletions(-)
diff --git a/templates/hooks--pre-commit.sample b/templates/hooks--pre-commit.sample
index 0e49279..91ab563 100755
--- a/templates/hooks--pre-commit.sample
+++ b/templates/hooks--pre-commit.sample
@@ -7,6 +7,38 @@
 #
 # To enable this hook, rename this file to "pre-commit".
 
+# If you want to allow non-ascii filenames set this variable to true.
+allownonascii=$(git config hooks.allownonascii)
+
+# is_ascii() Tests the string given given on standard input for
+# printable ascii conformance. We exploit the fact that the printable
+# range starts at the space character and ends with tilde.
+is_ascii() {
+    test -z "$(LC_ALL=C tr -d \ -~)"
+}
+
+if [ "$allownonascii" != "true" ]
+then
+	# until git can handle non-ascii filenames gracefully
+	# prevent them to be added into the repository
+	if ! git diff --cached --name-only --diff-filter=A -z \
+	   | tr "\0" "\n" | is_ascii; then
+		echo "Error: Preventing to add a non-ascii filename."
+		echo
+		echo "This can cause problems if you want to work together"
+		echo "with people on other platforms than you."
+		echo
+		echo "To be portable it is adviseable to rename the file ..."
+		echo
+		echo "If you know what you are doing you can disable this"
+		echo "check using:"
+		echo
+		echo "  git config hooks.allownonascii true"
+		echo
+		exit 1
+	fi
+fi
+
 if git-rev-parse --verify HEAD 2>/dev/null
 then
 	against=HEAD

-- 
1.6.3
Previous: Jakub NarebskiNext: Johannes Sixt
Message 38 of 59 in “Re: Cross-Platform Version Control”
  1. Esko LuontolaMay 12, 2009
  2. Shawn O. PearceMay 12, 2009
  3. Johannes SchindelinMay 12, 2009
  4. Esko LuontolaMay 12, 2009
  5. Johannes SchindelinMay 12, 2009
  6. Esko LuontolaMay 12, 2009
  7. Johannes SchindelinMay 13, 2009
  8. Esko LuontolaMay 13, 2009
  9. Alex RiesenMay 13, 2009
  10. Johannes SchindelinMay 13, 2009
  11. Jeff KingMay 12, 2009
  12. Johannes SchindelinMay 12, 2009
  13. Linus TorvaldsMay 13, 2009
  14. Linus TorvaldsMay 13, 2009
  15. Andreas EricssonMay 13, 2009
  16. Linus TorvaldsMay 13, 2009
  17. Martin LanghoffMay 13, 2009
  18. Linus TorvaldsMay 13, 2009
  19. Theodore TsoMay 13, 2009
  20. Linus TorvaldsMay 13, 2009
  21. Daniel BarkalowMay 13, 2009
  22. Linus TorvaldsMay 13, 2009
  23. Matthias AndreeMay 13, 2009
  24. Linus TorvaldsMay 13, 2009
  25. Jay SoffianMay 13, 2009
  26. Matthias AndreeMay 13, 2009
  27. Dmitry PotapovMay 12, 2009
  28. Martin LanghoffMay 12, 2009
  29. Jakub NarebskiMay 12, 2009
  30. Extend sample pre-commit hook to check for non ascii file/usernamesHeiko Voigt, May 12, 2009
  31. Jakub NarebskiMay 12, 2009
  32. Extend sample pre-commit hook to check for non ascii filenamesHeiko Voigt, May 14, 2009
  33. Martin LanghoffMay 15, 2009
  34. Heiko VoigtMay 18, 2009
  35. Jakub NarebskiMay 18, 2009
  36. check for filenames that only differ in case to sample pre-commit hookHeiko Voigt, Jun 20, 2009
  37. Jakub NarebskiMay 15, 2009
  38. Extend sample pre-commit hook to check for non ascii filenamesHeiko Voigt, May 18, 2009
  39. Johannes SixtMay 18, 2009
  40. Heiko VoigtMay 18, 2009
  41. Johannes SixtMay 18, 2009
  42. Extend sample pre-commit hook to check for non ascii filenamesHeiko Voigt, May 19, 2009
  43. Junio C HamanoMay 18, 2009
  44. Julian PhillipsMay 18, 2009
  45. Junio C HamanoMay 15, 2009
  46. Peter KreftingMay 14, 2009
  47. Esko LuontolaMay 14, 2009
  48. Andreas EricssonMay 14, 2009
  49. Johannes SchindelinMay 14, 2009
  50. Dmitry PotapovMay 15, 2009
  51. Cross-Platform Version ControlJohn Tapsell, May 13, 2009
  52. Jay SoffianMay 13, 2009
  53. Alex RiesenMay 13, 2009
  54. Jay SoffianMay 13, 2009
  55. John TapsellMay 13, 2009
  56. Nicolas PitreMay 13, 2009
  57. Johannes SchindelinMay 13, 2009
  58. Andreas EricssonMay 13, 2009
  59. Miles BaderMay 14, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.