{"thread":{"id":"44697","subject":"[PATCH v2 6/6] unicode_width.h: update the width tables to Unicode 9.0","startedAt":"2016-12-13T23:34:28Z","lastAt":"2016-12-15T17:50:47Z","messageCount":12,"participants":["Beat Bolli","Junio C Hamano","Dennis Kaarsemaker"],"isPatch":true,"patchVersion":2,"patchTotal":6},"messages":[{"id":"307715","messageId":"1481671904-1143-7-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"[PATCH v2 6/6] unicode_width.h: update the width tables to Unicode 9.0","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:44Z","receivedAt":"2016-12-13T23:34:28Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"Rerunning update-unicode.sh that we fixed in the previous commits\nproduces these new tables.\n\nSigned-off-by: Beat Bolli \n---\n unicode_width.h | 131 +++++++++++++++++++++++++++++++++++++++++++++-----------\n 1 file changed, 107 insertions(+), 24 deletions(-)\n\ndiff --git a/unicode_width.h b/unicode_width.h\nindex 47cdd23..02207be 100644\n--- a/unicode_width.h\n+++ b/unicode_width.h\n@@ -25,7 +25,7 @@ static const struct interval zero_width[] = {\n { 0x0825, 0x0827 },\n { 0x0829, 0x082D },\n { 0x0859, 0x085B },\n-{ 0x08E4, 0x0902 },\n+{ 0x08D4, 0x0902 },\n { 0x093A, 0x093A },\n { 0x093C, 0x093C },\n { 0x0941, 0x0948 },\n@@ -120,6 +120,7 @@ static const struct interval zero_width[] = {\n { 0x17C9, 0x17D3 },\n { 0x17DD, 0x17DD },\n { 0x180B, 0x180E },\n+{ 0x1885, 0x1886 },\n { 0x18A9, 0x18A9 },\n { 0x1920, 0x1922 },\n { 0x1927, 0x1928 },\n@@ -158,7 +159,7 @@ static const struct interval zero_width[] = {\n { 0x1CF4, 0x1CF4 },\n { 0x1CF8, 0x1CF9 },\n { 0x1DC0, 0x1DF5 },\n-{ 0x1DFC, 0x1DFF },\n+{ 0x1DFB, 0x1DFF },\n { 0x200B, 0x200F },\n { 0x202A, 0x202E },\n { 0x2060, 0x2064 },\n@@ -171,13 +172,13 @@ static const struct interval zero_width[] = {\n { 0x3099, 0x309A },\n { 0xA66F, 0xA672 },\n { 0xA674, 0xA67D },\n-{ 0xA69F, 0xA69F },\n+{ 0xA69E, 0xA69F },\n { 0xA6F0, 0xA6F1 },\n { 0xA802, 0xA802 },\n { 0xA806, 0xA806 },\n { 0xA80B, 0xA80B },\n { 0xA825, 0xA826 },\n-{ 0xA8C4, 0xA8C4 },\n+{ 0xA8C4, 0xA8C5 },\n { 0xA8E0, 0xA8F1 },\n { 0xA926, 0xA92D },\n { 0xA947, 0xA951 },\n@@ -204,7 +205,7 @@ static const struct interval zero_width[] = {\n { 0xABED, 0xABED },\n { 0xFB1E, 0xFB1E },\n { 0xFE00, 0xFE0F },\n-{ 0xFE20, 0xFE2D },\n+{ 0xFE20, 0xFE2F },\n { 0xFEFF, 0xFEFF },\n { 0xFFF9, 0xFFFB },\n { 0x101FD, 0x101FD },\n@@ -228,16 +229,21 @@ static const struct interval zero_width[] = {\n { 0x11173, 0x11173 },\n { 0x11180, 0x11181 },\n { 0x111B6, 0x111BE },\n+{ 0x111CA, 0x111CC },\n { 0x1122F, 0x11231 },\n { 0x11234, 0x11234 },\n { 0x11236, 0x11237 },\n+{ 0x1123E, 0x1123E },\n { 0x112DF, 0x112DF },\n { 0x112E3, 0x112EA },\n-{ 0x11301, 0x11301 },\n+{ 0x11300, 0x11301 },\n { 0x1133C, 0x1133C },\n { 0x11340, 0x11340 },\n { 0x11366, 0x1136C },\n { 0x11370, 0x11374 },\n+{ 0x11438, 0x1143F },\n+{ 0x11442, 0x11444 },\n+{ 0x11446, 0x11446 },\n { 0x114B3, 0x114B8 },\n { 0x114BA, 0x114BA },\n { 0x114BF, 0x114C0 },\n@@ -245,6 +251,7 @@ static const struct interval zero_width[] = {\n { 0x115B2, 0x115B5 },\n { 0x115BC, 0x115BD },\n { 0x115BF, 0x115C0 },\n+{ 0x115DC, 0x115DD },\n { 0x11633, 0x1163A },\n { 0x1163D, 0x1163D },\n { 0x1163F, 0x11640 },\n@@ -252,6 +259,16 @@ static const struct interval zero_width[] = {\n { 0x116AD, 0x116AD },\n { 0x116B0, 0x116B5 },\n { 0x116B7, 0x116B7 },\n+{ 0x1171D, 0x1171F },\n+{ 0x11722, 0x11725 },\n+{ 0x11727, 0x1172B },\n+{ 0x11C30, 0x11C36 },\n+{ 0x11C38, 0x11C3D },\n+{ 0x11C3F, 0x11C3F },\n+{ 0x11C92, 0x11CA7 },\n+{ 0x11CAA, 0x11CB0 },\n+{ 0x11CB2, 0x11CB3 },\n+{ 0x11CB5, 0x11CB6 },\n { 0x16AF0, 0x16AF4 },\n { 0x16B30, 0x16B36 },\n { 0x16F8F, 0x16F92 },\n@@ -262,31 +279,59 @@ static const struct interval zero_width[] = {\n { 0x1D185, 0x1D18B },\n { 0x1D1AA, 0x1D1AD },\n { 0x1D242, 0x1D244 },\n+{ 0x1DA00, 0x1DA36 },\n+{ 0x1DA3B, 0x1DA6C },\n+{ 0x1DA75, 0x1DA75 },\n+{ 0x1DA84, 0x1DA84 },\n+{ 0x1DA9B, 0x1DA9F },\n+{ 0x1DAA1, 0x1DAAF },\n+{ 0x1E000, 0x1E006 },\n+{ 0x1E008, 0x1E018 },\n+{ 0x1E01B, 0x1E021 },\n+{ 0x1E023, 0x1E024 },\n+{ 0x1E026, 0x1E02A },\n { 0x1E8D0, 0x1E8D6 },\n+{ 0x1E944, 0x1E94A },\n { 0xE0001, 0xE0001 },\n { 0xE0020, 0xE007F },\n { 0xE0100, 0xE01EF }\n };\n static const struct interval double_width[] = {\n-{ /* plane */ 0x0, 0x1C },\n-{ /* plane */ 0x1C, 0x21 },\n-{ /* plane */ 0x21, 0x22 },\n-{ /* plane */ 0x22, 0x23 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n-{ /* plane */ 0x0, 0x0 },\n { 0x1100, 0x115F },\n+{ 0x231A, 0x231B },\n { 0x2329, 0x232A },\n+{ 0x23E9, 0x23EC },\n+{ 0x23F0, 0x23F0 },\n+{ 0x23F3, 0x23F3 },\n+{ 0x25FD, 0x25FE },\n+{ 0x2614, 0x2615 },\n+{ 0x2648, 0x2653 },\n+{ 0x267F, 0x267F },\n+{ 0x2693, 0x2693 },\n+{ 0x26A1, 0x26A1 },\n+{ 0x26AA, 0x26AB },\n+{ 0x26BD, 0x26BE },\n+{ 0x26C4, 0x26C5 },\n+{ 0x26CE, 0x26CE },\n+{ 0x26D4, 0x26D4 },\n+{ 0x26EA, 0x26EA },\n+{ 0x26F2, 0x26F3 },\n+{ 0x26F5, 0x26F5 },\n+{ 0x26FA, 0x26FA },\n+{ 0x26FD, 0x26FD },\n+{ 0x2705, 0x2705 },\n+{ 0x270A, 0x270B },\n+{ 0x2728, 0x2728 },\n+{ 0x274C, 0x274C },\n+{ 0x274E, 0x274E },\n+{ 0x2753, 0x2755 },\n+{ 0x2757, 0x2757 },\n+{ 0x2795, 0x2797 },\n+{ 0x27B0, 0x27B0 },\n+{ 0x27BF, 0x27BF },\n+{ 0x2B1B, 0x2B1C },\n+{ 0x2B50, 0x2B50 },\n+{ 0x2B55, 0x2B55 },\n { 0x2E80, 0x2E99 },\n { 0x2E9B, 0x2EF3 },\n { 0x2F00, 0x2FD5 },\n@@ -313,11 +358,49 @@ static const struct interval double_width[] = {\n { 0xFE68, 0xFE6B },\n { 0xFF01, 0xFF60 },\n { 0xFFE0, 0xFFE6 },\n+{ 0x16FE0, 0x16FE0 },\n+{ 0x17000, 0x187EC },\n+{ 0x18800, 0x18AF2 },\n { 0x1B000, 0x1B001 },\n+{ 0x1F004, 0x1F004 },\n+{ 0x1F0CF, 0x1F0CF },\n+{ 0x1F18E, 0x1F18E },\n+{ 0x1F191, 0x1F19A },\n { 0x1F200, 0x1F202 },\n-{ 0x1F210, 0x1F23A },\n+{ 0x1F210, 0x1F23B },\n { 0x1F240, 0x1F248 },\n { 0x1F250, 0x1F251 },\n+{ 0x1F300, 0x1F320 },\n+{ 0x1F32D, 0x1F335 },\n+{ 0x1F337, 0x1F37C },\n+{ 0x1F37E, 0x1F393 },\n+{ 0x1F3A0, 0x1F3CA },\n+{ 0x1F3CF, 0x1F3D3 },\n+{ 0x1F3E0, 0x1F3F0 },\n+{ 0x1F3F4, 0x1F3F4 },\n+{ 0x1F3F8, 0x1F43E },\n+{ 0x1F440, 0x1F440 },\n+{ 0x1F442, 0x1F4FC },\n+{ 0x1F4FF, 0x1F53D },\n+{ 0x1F54B, 0x1F54E },\n+{ 0x1F550, 0x1F567 },\n+{ 0x1F57A, 0x1F57A },\n+{ 0x1F595, 0x1F596 },\n+{ 0x1F5A4, 0x1F5A4 },\n+{ 0x1F5FB, 0x1F64F },\n+{ 0x1F680, 0x1F6C5 },\n+{ 0x1F6CC, 0x1F6CC },\n+{ 0x1F6D0, 0x1F6D2 },\n+{ 0x1F6EB, 0x1F6EC },\n+{ 0x1F6F4, 0x1F6F6 },\n+{ 0x1F910, 0x1F91E },\n+{ 0x1F920, 0x1F927 },\n+{ 0x1F930, 0x1F930 },\n+{ 0x1F933, 0x1F93E },\n+{ 0x1F940, 0x1F94B },\n+{ 0x1F950, 0x1F95E },\n+{ 0x1F980, 0x1F991 },\n+{ 0x1F9C0, 0x1F9C0 },\n { 0x20000, 0x2FFFD },\n { 0x30000, 0x3FFFD }\n };\n-- \n2.7.2\n"},{"id":"307716","messageId":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":null,"subject":"[PATCH v2 0/6] unicode_width.h: update the width tables to Unicode 9.0","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:38Z","receivedAt":"2016-12-13T23:34:31Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"This is v2 of my Unicode 9.0 series. After a short discussion [1], we\ndecided to move the generator script into contrib. This is what this\nseries now does first. The script is then updated in contrib.\n\nDiff to v1:\n- complete commit reordering\n- fix nits in the commit messages\n\n.gitignore | 1 -\ncontrib/update-unicode/.gitignore | 3 ++\ncontrib/update-unicode/README | 20 +++++++++++\ncontrib/update-unicode/update_unicode.sh | 33 ++++++++++++++++++\nunicode_width.h | 131 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------\nupdate_unicode.sh | 40 ----------------------\n6 files changed, 163 insertions(+), 65 deletions(-)\n\n[1] http://public-inbox.org/git/xmqqr35dm203.fsf@gitster.mtv.corp.google.com/\n"},{"id":"307717","messageId":"1481671904-1143-5-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"[PATCH v2 4/6] update-unicode.sh: automatically download newer definition files","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:42Z","receivedAt":"2016-12-13T23:44:02Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"Checking just for the unicode data files' existence is not sufficient;\nwe should also download them if a newer version exists on the Unicode\nconsortium's servers. Option -N of wget does this nicely for us.\n\nReviewed-by: Torsten Bögershausen \nSigned-off-by: Beat Bolli \n---\n contrib/update-unicode/update_unicode.sh | 8 ++------\n 1 file changed, 2 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/update-unicode/update_unicode.sh b/contrib/update-unicode/update_unicode.sh\nindex 9f1bf31..56871a1 100755\n--- a/contrib/update-unicode/update_unicode.sh\n+++ b/contrib/update-unicode/update_unicode.sh\n@@ -8,12 +8,8 @@\n cd \"$(dirname \"$0\")\"\n UNICODEWIDTH_H=$(git rev-parse --show-toplevel)/unicode_width.h\n \n-if ! test -f UnicodeData.txt; then\n-\twget http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt\n-fi &&\n-if ! test -f EastAsianWidth.txt; then\n-\twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n-fi &&\n+wget -N http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt \\\n+\thttp://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt &&\n if ! test -d uniset; then\n \tgit clone https://github.com/depp/uniset.git &&\n \t( cd uniset && git checkout 4b186196dd )\n-- \n2.7.2\n"},{"id":"307718","messageId":"1481671904-1143-3-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"[PATCH v2 2/6] update_unicode.sh: remove an unnecessary subshell level","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:40Z","receivedAt":"2016-12-13T23:44:06Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"After the move into contrib/update-unicode, we no longer create the\nunicode directory to have a clean working folder. Instead, the directory\nof the script is used. This means that the subshell can be removed.\n\nSigned-off-by: Beat Bolli \n---\n contrib/update-unicode/update_unicode.sh | 53 ++++++++++++++++----------------\n 1 file changed, 26 insertions(+), 27 deletions(-)\n\ndiff --git a/contrib/update-unicode/update_unicode.sh b/contrib/update-unicode/update_unicode.sh\nindex 7b90126..ff664ec 100755\n--- a/contrib/update-unicode/update_unicode.sh\n+++ b/contrib/update-unicode/update_unicode.sh\n@@ -7,32 +7,31 @@\n #\n cd \"$(dirname \"$0\")\"\n UNICODEWIDTH_H=$(git rev-parse --show-toplevel)/unicode_width.h\n+\n+if ! test -f UnicodeData.txt; then\n+\twget http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt\n+fi &&\n+if ! test -f EastAsianWidth.txt; then\n+\twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n+fi &&\n+if ! test -d uniset; then\n+\tgit clone https://github.com/depp/uniset.git\n+fi &&\n (\n-\tif ! test -f UnicodeData.txt; then\n-\t\twget http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt\n+\tcd uniset &&\n+\tif ! test -x uniset; then\n+\t\tautoreconf -i &&\n+\t\t./configure --enable-warnings=-Werror CFLAGS='-O0 -ggdb'\n \tfi &&\n-\tif ! test -f EastAsianWidth.txt; then\n-\t\twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n-\tfi &&\n-\tif ! test -d uniset; then\n-\t\tgit clone https://github.com/depp/uniset.git\n-\tfi &&\n-\t(\n-\t\tcd uniset &&\n-\t\tif ! test -x uniset; then\n-\t\t\tautoreconf -i &&\n-\t\t\t./configure --enable-warnings=-Werror CFLAGS='-O0 -ggdb'\n-\t\tfi &&\n-\t\tmake\n-\t) &&\n-\tUNICODE_DIR=. && export UNICODE_DIR &&\n-\tcat >$UNICODEWIDTH_H <<-EOF\n-\tstatic const struct interval zero_width[] = {\n-\t\t$(uniset/uniset --32 cat:Me,Mn,Cf + U+1160..U+11FF - U+00AD |\n-\t\t grep -v plane)\n-\t};\n-\tstatic const struct interval double_width[] = {\n-\t\t$(uniset/uniset --32 eaw:F,W)\n-\t};\n-\tEOF\n-)\n+\tmake\n+) &&\n+UNICODE_DIR=. && export UNICODE_DIR &&\n+cat >$UNICODEWIDTH_H <<-EOF\n+static const struct interval zero_width[] = {\n+\t$(uniset/uniset --32 cat:Me,Mn,Cf + U+1160..U+11FF - U+00AD |\n+\t grep -v plane)\n+};\n+static const struct interval double_width[] = {\n+\t$(uniset/uniset --32 eaw:F,W)\n+};\n+EOF\n-- \n2.7.2\n"},{"id":"307719","messageId":"1481671904-1143-6-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"[PATCH v2 5/6] update_unicode.sh: remove the plane filter","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:43Z","receivedAt":"2016-12-13T23:44:31Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"The uniset upstream has accepted my patches that eliminate the Unicode\nplane offsets from the output in '--32' mode.\n\nRemove the corresponding filter in update_unicode.sh.\n\nThis also fixes the issue that the plane offsets were not removed from\nthe second uniset call.\n\nSigned-off-by: Beat Bolli \n---\n contrib/update-unicode/update_unicode.sh | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/contrib/update-unicode/update_unicode.sh b/contrib/update-unicode/update_unicode.sh\nindex 56871a1..e05db92 100755\n--- a/contrib/update-unicode/update_unicode.sh\n+++ b/contrib/update-unicode/update_unicode.sh\n@@ -25,8 +25,7 @@ fi &&\n UNICODE_DIR=. && export UNICODE_DIR &&\n cat >$UNICODEWIDTH_H <<-EOF\n static const struct interval zero_width[] = {\n-\t$(uniset/uniset --32 cat:Me,Mn,Cf + U+1160..U+11FF - U+00AD |\n-\t grep -v plane)\n+\t$(uniset/uniset --32 cat:Me,Mn,Cf + U+1160..U+11FF - U+00AD)\n };\n static const struct interval double_width[] = {\n \t$(uniset/uniset --32 eaw:F,W)\n-- \n2.7.2\n"},{"id":"307720","messageId":"1481671904-1143-4-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"[PATCH v2 3/6] update_unicode.sh: pin the uniset repo to a known good commit","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:41Z","receivedAt":"2016-12-13T23:44:34Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"The uniset upstream has added more commits that for example change the\nhexadecimal output in '--32' mode to decimal. Let's pin the repo to a\ncommit that still outputs the width tables in the format we want.\n\nSigned-off-by: Beat Bolli \n---\n contrib/update-unicode/update_unicode.sh | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/contrib/update-unicode/update_unicode.sh b/contrib/update-unicode/update_unicode.sh\nindex ff664ec..9f1bf31 100755\n--- a/contrib/update-unicode/update_unicode.sh\n+++ b/contrib/update-unicode/update_unicode.sh\n@@ -15,7 +15,8 @@ if ! test -f EastAsianWidth.txt; then\n \twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n fi &&\n if ! test -d uniset; then\n-\tgit clone https://github.com/depp/uniset.git\n+\tgit clone https://github.com/depp/uniset.git &&\n+\t( cd uniset && git checkout 4b186196dd )\n fi &&\n (\n \tcd uniset &&\n-- \n2.7.2\n"},{"id":"307721","messageId":"1481671904-1143-2-git-send-email-dev+git@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"[PATCH v2 1/6] update_unicode.sh: move it into contrib/update-unicode","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-13T23:31:39Z","receivedAt":"2016-12-13T23:44:36Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"As it's used only by a tiny minority of the Git developer population,\nthis script does not belong into the main Git source directory.\n\nMove it into contrib/ and adjust the paths to account for the new\nlocation.\n\nSigned-off-by: Beat Bolli \n---\n .gitignore | 1 -\n contrib/update-unicode/.gitignore | 3 +++\n contrib/update-unicode/README | 20 ++++++++++++++++\n contrib/update-unicode/update_unicode.sh | 38 ++++++++++++++++++++++++++++++\n update_unicode.sh | 40 --------------------------------\n 5 files changed, 61 insertions(+), 41 deletions(-)\n create mode 100644 contrib/update-unicode/.gitignore\n create mode 100644 contrib/update-unicode/README\n create mode 100755 contrib/update-unicode/update_unicode.sh\n delete mode 100755 update_unicode.sh\n\ndiff --git a/.gitignore b/.gitignore\nindex f96e50e..5555ae0 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -204,7 +204,6 @@\n /config.mak.autogen\n /config.mak.append\n /configure\n-/unicode\n /tags\n /TAGS\n /cscope*\ndiff --git a/contrib/update-unicode/.gitignore b/contrib/update-unicode/.gitignore\nnew file mode 100644\nindex 0000000..b0ebc6a\n--- /dev/null\n+++ b/contrib/update-unicode/.gitignore\n@@ -0,0 +1,3 @@\n+uniset/\n+UnicodeData.txt\n+EastAsianWidth.txt\ndiff --git a/contrib/update-unicode/README b/contrib/update-unicode/README\nnew file mode 100644\nindex 0000000..b9e2fc8\n--- /dev/null\n+++ b/contrib/update-unicode/README\n@@ -0,0 +1,20 @@\n+TL;DR: Run update_unicode.sh after the publication of a new Unicode\n+standard and commit the resulting unicode_widths.h file.\n+\n+The long version\n+================\n+\n+The Git source code ships the file unicode_widths.h which contains\n+tables of zero and double width Unicode code points, respectively.\n+These tables are generated using update_unicode.sh in this directory.\n+update_unicode.sh itself uses a third-party tool, uniset, to query two\n+Unicode data files for the interesting code points.\n+\n+On first run, update_unicode.sh clones uniset from Github and builds it.\n+This requires a current-ish version of autoconf (2.69 works per December\n+2016).\n+\n+On each run, update_unicode.sh checks whether more recent Unicode data\n+files are available from the Unicode consortium, and rebuilds the header\n+unicode_widths.h with the new data. The new header can then be\n+committed.\ndiff --git a/contrib/update-unicode/update_unicode.sh b/contrib/update-unicode/update_unicode.sh\nnew file mode 100755\nindex 0000000..7b90126\n--- /dev/null\n+++ b/contrib/update-unicode/update_unicode.sh\n@@ -0,0 +1,38 @@\n+#!/bin/sh\n+#See http://www.unicode.org/reports/tr44/\n+#\n+#Me Enclosing_Mark an enclosing combining mark\n+#Mn Nonspacing_Mark a nonspacing combining mark (zero advance width)\n+#Cf Format a format control character\n+#\n+cd \"$(dirname \"$0\")\"\n+UNICODEWIDTH_H=$(git rev-parse --show-toplevel)/unicode_width.h\n+(\n+\tif ! test -f UnicodeData.txt; then\n+\t\twget http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt\n+\tfi &&\n+\tif ! test -f EastAsianWidth.txt; then\n+\t\twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n+\tfi &&\n+\tif ! test -d uniset; then\n+\t\tgit clone https://github.com/depp/uniset.git\n+\tfi &&\n+\t(\n+\t\tcd uniset &&\n+\t\tif ! test -x uniset; then\n+\t\t\tautoreconf -i &&\n+\t\t\t./configure --enable-warnings=-Werror CFLAGS='-O0 -ggdb'\n+\t\tfi &&\n+\t\tmake\n+\t) &&\n+\tUNICODE_DIR=. && export UNICODE_DIR &&\n+\tcat >$UNICODEWIDTH_H <<-EOF\n+\tstatic const struct interval zero_width[] = {\n+\t\t$(uniset/uniset --32 cat:Me,Mn,Cf + U+1160..U+11FF - U+00AD |\n+\t\t grep -v plane)\n+\t};\n+\tstatic const struct interval double_width[] = {\n+\t\t$(uniset/uniset --32 eaw:F,W)\n+\t};\n+\tEOF\n+)\ndiff --git a/update_unicode.sh b/update_unicode.sh\ndeleted file mode 100755\nindex 27af77c..0000000\n--- a/update_unicode.sh\n+++ /dev/null\n@@ -1,40 +0,0 @@\n-#!/bin/sh\n-#See http://www.unicode.org/reports/tr44/\n-#\n-#Me Enclosing_Mark an enclosing combining mark\n-#Mn Nonspacing_Mark a nonspacing combining mark (zero advance width)\n-#Cf Format a format control character\n-#\n-UNICODEWIDTH_H=../unicode_width.h\n-if ! test -d unicode; then\n-\tmkdir unicode\n-fi &&\n-( cd unicode &&\n-\tif ! test -f UnicodeData.txt; then\n-\t\twget http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt\n-\tfi &&\n-\tif ! test -f EastAsianWidth.txt; then\n-\t\twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n-\tfi &&\n-\tif ! test -d uniset; then\n-\t\tgit clone https://github.com/depp/uniset.git\n-\tfi &&\n-\t(\n-\t\tcd uniset &&\n-\t\tif ! test -x uniset; then\n-\t\t\tautoreconf -i &&\n-\t\t\t./configure --enable-warnings=-Werror CFLAGS='-O0 -ggdb'\n-\t\tfi &&\n-\t\tmake\n-\t) &&\n-\tUNICODE_DIR=. && export UNICODE_DIR &&\n-\tcat >$UNICODEWIDTH_H <<-EOF\n-\tstatic const struct interval zero_width[] = {\n-\t\t$(uniset/uniset --32 cat:Me,Mn,Cf + U+1160..U+11FF - U+00AD |\n-\t\t grep -v plane)\n-\t};\n-\tstatic const struct interval double_width[] = {\n-\t\t$(uniset/uniset --32 eaw:F,W)\n-\t};\n-\tEOF\n-)\n-- \n2.7.2\n"},{"id":"307725","messageId":"xmqqh967z4zn.fsf@gitster.mtv.corp.google.com","threadId":"44697","inReplyTo":"1481671904-1143-1-git-send-email-dev+git@drbeat.li","subject":"Re: [PATCH v2 0/6] unicode_width.h: update the width tables to Unicode 9.0","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-14T01:14:36Z","receivedAt":"2016-12-14T01:14:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thanks. Very much appreciated.\n"},{"id":"307790","messageId":"b137249e-728a-5d3c-4993-5ed5a1593737@drbeat.li","threadId":"44697","inReplyTo":"1481671904-1143-5-git-send-email-dev+git@drbeat.li","subject":"Re: [PATCH v2 4/6] update-unicode.sh: automatically download newer definition files","fromName":"Beat Bolli","fromEmail":"dev+git@drbeat.li","sentAt":"2016-12-14T17:40:48Z","receivedAt":"2016-12-14T17:40:59Z","isPatch":true,"sender":{"key":"dev+git@drbeat.li","avatar":"https://avatars.githubusercontent.com/u/21444?v=4"},"body":"On 14.12.16 00:31, Beat Bolli wrote:\n\n> [PATCH v2 4/6] update-unicode.sh: automatically download newer definition files\n\nDang! And again I'm not capable of putting an underline instead of the\ndash...\n\nJunio, would you please reword the subject to\n\nRe: [PATCH v2 4/6] update_unicode.sh: automatically download newer\ndefinition files\n\nThanks,\nBeat\n\n\n> we should also download them if a newer version exists on the Unicode\n> consortium's servers. Option -N of wget does this nicely for us.\n> \n> Reviewed-by: Torsten Bögershausen \n> Signed-off-by: Beat Bolli \n> ---\n> contrib/update-unicode/update_unicode.sh | 8 ++------\n> 1 file changed, 2 insertions(+), 6 deletions(-)\n> \n> diff --git a/contrib/update-unicode/update_unicode.sh b/contrib/update-unicode/update_unicode.sh\n> index 9f1bf31..56871a1 100755\n> --- a/contrib/update-unicode/update_unicode.sh\n> +++ b/contrib/update-unicode/update_unicode.sh\n> @@ -8,12 +8,8 @@\n> cd \"$(dirname \"$0\")\"\n> UNICODEWIDTH_H=$(git rev-parse --show-toplevel)/unicode_width.h\n> \n> -if ! test -f UnicodeData.txt; then\n> -\twget http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt\n> -fi &&\n> -if ! test -f EastAsianWidth.txt; then\n> -\twget http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt\n> -fi &&\n> +wget -N http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt \\\n> +\thttp://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt &&\n> if ! test -d uniset; then\n> \tgit clone https://github.com/depp/uniset.git &&\n> \t( cd uniset && git checkout 4b186196dd )\n> \n"},{"id":"307791","messageId":"xmqqvaumxuvg.fsf@gitster.mtv.corp.google.com","threadId":"44697","inReplyTo":"b137249e-728a-5d3c-4993-5ed5a1593737@drbeat.li","subject":"Re: [PATCH v2 4/6] update-unicode.sh: automatically download newer definition files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-14T17:50:43Z","receivedAt":"2016-12-14T17:50:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Beat Bolli writes:\n\n> On 14.12.16 00:31, Beat Bolli wrote:\n>\n>> [PATCH v2 4/6] update-unicode.sh: automatically download newer definition files\n>\n> Dang! And again I'm not capable of putting an underline instead of the\n> dash...\n>\n> Junio, would you please reword the subject to\n>\n> Re: [PATCH v2 4/6] update_unicode.sh: automatically download newer\n> definition files\n\nWill do.\n\nThis is an indication that the script is probably named against\npeople's expectation. We may want to rename it after the dust\nsettles.\n\nThanks.\n"},{"id":"307834","messageId":"1481795248.4090.15.camel@kaarsemaker.net","threadId":"44697","inReplyTo":"1481671904-1143-4-git-send-email-dev+git@drbeat.li","subject":"Re: [PATCH v2 3/6] update_unicode.sh: pin the uniset repo to a known good commit","fromName":"Dennis Kaarsemaker","fromEmail":"dennis@kaarsemaker.net","sentAt":"2016-12-15T09:47:28Z","receivedAt":"2016-12-15T09:47:39Z","isPatch":true,"sender":{"key":"dennis@kaarsemaker.net","avatar":"https://avatars.githubusercontent.com/u/200649?v=4"},"body":"On Wed, 2016-12-14 at 00:31 +0100, Beat Bolli wrote:\n> +       ( cd uniset && git checkout 4b186196dd )\n\nMicronit, but this is perhaps better written as\n\ngit -C uniset checkout 4b186196dd\n\nto avoid the subshell and cd.\n\nD.\n"},{"id":"307852","messageId":"xmqq4m25w07k.fsf@gitster.mtv.corp.google.com","threadId":"44697","inReplyTo":"1481795248.4090.15.camel@kaarsemaker.net","subject":"Re: [PATCH v2 3/6] update_unicode.sh: pin the uniset repo to a known good commit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-12-15T17:50:39Z","receivedAt":"2016-12-15T17:50:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Dennis Kaarsemaker writes:\n\n> On Wed, 2016-12-14 at 00:31 +0100, Beat Bolli wrote:\n>> +       ( cd uniset && git checkout 4b186196dd )\n>\n> Micronit, but this is perhaps better written as\n>\n> git -C uniset checkout 4b186196dd\n>\n> to avoid the subshell and cd.\n>\n> D.\n\nIn the context of this script, I would say that is not even a\nmicronit. It is \"you could do this if you wanted to\".\n"}]}