{"thread":{"id":"17070","subject":"[PATCH 1/2] grep -w: forward to next possible position after rejected match","startedAt":"2009-01-09T23:08:40Z","lastAt":"2009-01-17T15:50:37Z","messageCount":12,"participants":["René Scharfe","Junio C Hamano","Mikael Magnusson","Johannes Schindelin","Alex Riesen"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"99823","messageId":"4967D8F8.9070508@lsrfire.ath.cx","threadId":"17070","inReplyTo":null,"subject":"[PATCH 1/2] grep -w: forward to next possible position after rejected match","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-09T23:08:40Z","receivedAt":"2009-01-09T23:08:40Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"grep -w accepts matches between non-word characters, only.  If a match\nfrom regexec() doesn't meet this criteria, grep continues its search\nafter the first character of that match.\n\nWe can be a bit smarter here and skip all positions that follow a word\ncharacter first, as they can't match our criteria.  This way we can\nconsume characters quite cheaply and don't need to special-case the\nhandling of the beginning of a line.\n\nHere's a contrived example command on msysgit (best of five runs):\n\n\t$ time git grep -w ...... v1.6.1 >/dev/null\n\n\treal    0m1.611s\n\tuser    0m0.000s\n\tsys     0m0.015s\n\nWith the patch it's quite a bit faster:\n\n\t$ time git grep -w ...... v1.6.1 >/dev/null\n\n\treal    0m1.179s\n\tuser    0m0.000s\n\tsys     0m0.015s\n\nMore common search patterns will gain a lot less, but it's a nice clean\nup anyway.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\n grep.c |   11 +++++++----\n 1 files changed, 7 insertions(+), 4 deletions(-)\n\ndiff --git a/grep.c b/grep.c\nindex 49e9319..394703b 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -294,7 +294,6 @@ static struct {\n static int match_one_pattern(struct grep_opt *opt, struct grep_pat *p, char *bol, char *eol, enum grep_context ctx)\n {\n \tint hit = 0;\n-\tint at_true_bol = 1;\n \tint saved_ch = 0;\n \tregmatch_t pmatch[10];\n \n@@ -337,7 +336,7 @@ static int match_one_pattern(struct grep_opt *opt, struct grep_pat *p, char *bol\n \t\t * either end of the line, or at word boundary\n \t\t * (i.e. the next char must not be a word char).\n \t\t */\n-\t\tif ( ((pmatch[0].rm_so == 0 && at_true_bol) ||\n+\t\tif ( ((pmatch[0].rm_so == 0) ||\n \t\t      !word_char(bol[pmatch[0].rm_so-1])) &&\n \t\t     ((pmatch[0].rm_eo == (eol-bol)) ||\n \t\t      !word_char(bol[pmatch[0].rm_eo])) )\n@@ -349,10 +348,14 @@ static int match_one_pattern(struct grep_opt *opt, struct grep_pat *p, char *bol\n \t\t\t/* There could be more than one match on the\n \t\t\t * line, and the first match might not be\n \t\t\t * strict word match.  But later ones could be!\n+\t\t\t * Forward to the next possible start, i.e. the\n+\t\t\t * next position following a non-word char.\n \t\t\t */\n \t\t\tbol = pmatch[0].rm_so + bol + 1;\n-\t\t\tat_true_bol = 0;\n-\t\t\tgoto again;\n+\t\t\twhile (word_char(bol[-1]) && bol < eol)\n+\t\t\t\tbol++;\n+\t\t\tif (bol < eol)\n+\t\t\t\tgoto again;\n \t\t}\n \t}\n \tif (p->token == GREP_PATTERN_HEAD && saved_ch)\n-- \n1.6.1\n"},{"id":"99824","messageId":"4967DB4A.2000702@lsrfire.ath.cx","threadId":"17070","inReplyTo":"4967D8F8.9070508@lsrfire.ath.cx","subject":"[PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-09T23:18:34Z","receivedAt":"2009-01-09T23:18:34Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Add the new flag \"fixed\" to struct grep_pat and set it if the pattern\nis doesn't contain any regex control characters in addition to if the\nflag -F/--fixed-strings was specified.\n\nThis gives a nice speed up on msysgit, where regexec() seems to be\nextra slow.  Before (best of five runs):\n\n\t$ time git grep grep v1.6.1 >/dev/null\n\n\treal    0m0.552s\n\tuser    0m0.000s\n\tsys     0m0.000s\n\n\t$ time git grep -F grep v1.6.1 >/dev/null\n\n\treal    0m0.170s\n\tuser    0m0.000s\n\tsys     0m0.015s\n\nWith the patch:\n\n\t$ time git grep grep v1.6.1 >/dev/null\n\n\treal    0m0.173s\n\tuser    0m0.000s\n\tsys     0m0.000s\n\nThe difference is much smaller on Linux, but still measurable.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\n grep.c |   29 +++++++++++++++++++++++++----\n grep.h |    1 +\n 2 files changed, 26 insertions(+), 4 deletions(-)\n\ndiff --git a/grep.c b/grep.c\nindex 394703b..a1092df 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -28,9 +28,31 @@ void append_grep_pattern(struct grep_opt *opt, const char *pat,\n \tp->next = NULL;\n }\n \n+static int isregexspecial(int c)\n+{\n+\treturn isspecial(c) || c == '$' || c == '(' || c == ')' || c == '+' ||\n+\t\t\t       c == '.' || c == '^' || c == '{' || c == '|';\n+}\n+\n+static int is_fixed(const char *s)\n+{\n+\twhile (!isregexspecial(*s))\n+\t\ts++;\n+\treturn !*s;\n+}\n+\n static void compile_regexp(struct grep_pat *p, struct grep_opt *opt)\n {\n-\tint err = regcomp(&p->regexp, p->pattern, opt->regflags);\n+\tint err;\n+\n+\tif (opt->fixed || is_fixed(p->pattern))\n+\t\tp->fixed = 1;\n+\tif (opt->regflags & REG_ICASE)\n+\t\tp->fixed = 0;\n+\tif (p->fixed)\n+\t\treturn;\n+\n+\terr = regcomp(&p->regexp, p->pattern, opt->regflags);\n \tif (err) {\n \t\tchar errbuf[1024];\n \t\tchar where[1024];\n@@ -159,8 +181,7 @@ void compile_grep_patterns(struct grep_opt *opt)\n \t\tcase GREP_PATTERN: /* atom */\n \t\tcase GREP_PATTERN_HEAD:\n \t\tcase GREP_PATTERN_BODY:\n-\t\t\tif (!opt->fixed)\n-\t\t\t\tcompile_regexp(p, opt);\n+\t\t\tcompile_regexp(p, opt);\n \t\t\tbreak;\n \t\tdefault:\n \t\t\topt->extended = 1;\n@@ -314,7 +335,7 @@ static int match_one_pattern(struct grep_opt *opt, struct grep_pat *p, char *bol\n \t}\n \n  again:\n-\tif (!opt->fixed) {\n+\tif (!p->fixed) {\n \t\tregex_t *exp = &p->regexp;\n \t\thit = !regexec(exp, bol, ARRAY_SIZE(pmatch),\n \t\t\t       pmatch, 0);\ndiff --git a/grep.h b/grep.h\nindex 45a222d..5102ce3 100644\n--- a/grep.h\n+++ b/grep.h\n@@ -30,6 +30,7 @@ struct grep_pat {\n \tconst char *pattern;\n \tenum grep_header_field field;\n \tregex_t regexp;\n+\tunsigned fixed:1;\n };\n \n enum grep_expr_node {\n-- \n1.6.1\n"},{"id":"99923","messageId":"7vbpuex5hm.fsf@gitster.siamese.dyndns.org","threadId":"17070","inReplyTo":"4967DB4A.2000702@lsrfire.ath.cx","subject":"Re: [PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-01-10T20:37:09Z","receivedAt":"2009-01-10T20:37:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> Add the new flag \"fixed\" to struct grep_pat and set it if the pattern\n> is doesn't contain any regex control characters in addition to if the\n> flag -F/--fixed-strings was specified.\n>\n> This gives a nice speed up on msysgit, where regexec() seems to be\n> extra slow.  Before (best of five runs):\n\nThanks, and...\n\n>  static void compile_regexp(struct grep_pat *p, struct grep_opt *opt)\n>  {\n> -\tint err = regcomp(&p->regexp, p->pattern, opt->regflags);\n> +\tint err;\n> +\n> +\tif (opt->fixed || is_fixed(p->pattern))\n> +\t\tp->fixed = 1;\n> +\tif (opt->regflags & REG_ICASE)\n> +\t\tp->fixed = 0;\n\n... thanks again for being extra careful.  That's why I *love* your\npatches.\n"},{"id":"100115","messageId":"237967ef0901120425x79b7c1a4p238081a99694ae23@mail.gmail.com","threadId":"17070","inReplyTo":"4967DB4A.2000702@lsrfire.ath.cx","subject":"Re: [PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"Mikael Magnusson","fromEmail":"mikachu@gmail.com","sentAt":"2009-01-12T12:25:11Z","receivedAt":"2009-01-12T12:25:11Z","isPatch":true,"sender":{"key":"mikachu@gmail.com","avatar":null},"body":"2009/1/10 René Scharfe <rene.scharfe@lsrfire.ath.cx>:\n> Add the new flag \"fixed\" to struct grep_pat and set it if the pattern\n> is doesn't contain any regex control characters in addition to if the\n> flag -F/--fixed-strings was specified.\n>\n> diff --git a/grep.c b/grep.c\n> index 394703b..a1092df 100644\n> --- a/grep.c\n> +++ b/grep.c\n> @@ -28,9 +28,31 @@ void append_grep_pattern(struct grep_opt *opt, const char *pat,\n>        p->next = NULL;\n>  }\n>\n> +static int isregexspecial(int c)\n> +{\n> +       return isspecial(c) || c == '$' || c == '(' || c == ')' || c == '+' ||\n> +                              c == '.' || c == '^' || c == '{' || c == '|';\n> +}\n\nShouldn't this include '*' and '\\'?\n\n-- \nMikael Magnusson\n"},{"id":"100121","messageId":"alpine.DEB.1.00.0901121433240.3586@pacific.mpi-cbg.de","threadId":"17070","inReplyTo":"237967ef0901120425x79b7c1a4p238081a99694ae23@mail.gmail.com","subject":"Re: [PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-01-12T13:33:44Z","receivedAt":"2009-01-12T13:33:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 12 Jan 2009, Mikael Magnusson wrote:\n\n> 2009/1/10 René Scharfe <rene.scharfe@lsrfire.ath.cx>:\n> > Add the new flag \"fixed\" to struct grep_pat and set it if the pattern\n> > is doesn't contain any regex control characters in addition to if the\n> > flag -F/--fixed-strings was specified.\n> >\n> > diff --git a/grep.c b/grep.c\n> > index 394703b..a1092df 100644\n> > --- a/grep.c\n> > +++ b/grep.c\n> > @@ -28,9 +28,31 @@ void append_grep_pattern(struct grep_opt *opt, const char *pat,\n> >        p->next = NULL;\n> >  }\n> >\n> > +static int isregexspecial(int c)\n> > +{\n> > +       return isspecial(c) || c == '$' || c == '(' || c == ')' || c == '+' ||\n> > +                              c == '.' || c == '^' || c == '{' || c == '|';\n> > +}\n> \n> Shouldn't this include '*' and '\\'?\n\nThis is covered by isspecial(): see ctype.c.\n\nHth,\nDscho\n"},{"id":"100127","messageId":"81b0412b0901120732t1bd1978awdc4be47767e02863@mail.gmail.com","threadId":"17070","inReplyTo":"4967DB4A.2000702@lsrfire.ath.cx","subject":"Re: [PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2009-01-12T15:32:36Z","receivedAt":"2009-01-12T15:32:36Z","isPatch":true,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"2009/1/10 René Scharfe <rene.scharfe@lsrfire.ath.cx>:\n> +static int isregexspecial(int c)\n> +{\n> +       return isspecial(c) || c == '$' || c == '(' || c == ')' || c == '+' ||\n> +                              c == '.' || c == '^' || c == '{' || c == '|';\n> +}\n> +\n> +static int is_fixed(const char *s)\n> +{\n> +       while (!isregexspecial(*s))\n> +               s++;\n> +       return !*s;\n> +}\n\nstrchr?\n"},{"id":"100153","messageId":"496B9780.3030000@lsrfire.ath.cx","threadId":"17070","inReplyTo":"81b0412b0901120732t1bd1978awdc4be47767e02863@mail.gmail.com","subject":"Re: [PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-12T19:18:24Z","receivedAt":"2009-01-12T19:18:24Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Alex Riesen schrieb:\n> 2009/1/10 René Scharfe <rene.scharfe@lsrfire.ath.cx>:\n>> +static int isregexspecial(int c)\n>> +{\n>> +       return isspecial(c) || c == '$' || c == '(' || c == ')' || c == '+' ||\n>> +                              c == '.' || c == '^' || c == '{' || c == '|';\n>> +}\n>> +\n>> +static int is_fixed(const char *s)\n>> +{\n>> +       while (!isregexspecial(*s))\n>> +               s++;\n>> +       return !*s;\n>> +}\n> \n> strchr?\n\nOh, yes, that would look nicer.\n\nAnother option is to extend ctype.c and implement isregexspecial() --\nand while we're at it islowerxdigit() (builtin-name-rev.c::ishex()) and\niswordchar() (config.c::iskeychar(), grep.c::word_char()), too -- as\ntable lookups.  I.e., something like the following (untested).\n\nWhich of the mentioned functions are really worth of this promotion?\nThe isregexspecial() char class has more members than isspecial(), but\nit's not performance critical (unless you have a lot of patterns and\nonly a small amount of data to grep :).\n\nAre there more candidates for ctype-ification?\n\nRené\n\n\n ctype.c           |   14 ++++++++++----\n git-compat-util.h |    6 ++++++\n 2 files changed, 16 insertions(+), 4 deletions(-)\n\ndiff --git a/ctype.c b/ctype.c\nindex 9208d67..1a76586 100644\n--- a/ctype.c\n+++ b/ctype.c\n@@ -10,20 +10,26 @@\n #undef AA\n #undef DD\n #undef GS\n+#undef RR\n+#undef US\n+#undef Ah\n \n #define SS GIT_SPACE\n #define AA GIT_ALPHA\n #define DD GIT_DIGIT\n #define GS GIT_SPECIAL  /* \\0, *, ?, [, \\\\ */\n+#define RR GIT_REGEX_SPECIAL /* $, (, ), +, ., ^, {, | */\n+#define US GIT_UNDERSCORE\n+#define Ah (GIT_ALPHA | GIT_LOWER_XDIGIT)\n \n unsigned char sane_ctype[256] = {\n \tGS,  0,  0,  0,  0,  0,  0,  0,  0, SS, SS,  0,  0, SS,  0,  0,\t\t/* 0-15 */\n \t 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,\t\t/* 16-15 */\n-\tSS,  0,  0,  0,  0,  0,  0,  0,  0,  0, GS,  0,  0,  0,  0,  0,\t\t/* 32-15 */\n+\tSS,  0,  0,  0, RR,  0,  0,  0, RR, RR, GS, RR,  0,  0, RR,  0,\t\t/* 32-15 */\n \tDD, DD, DD, DD, DD, DD, DD, DD, DD, DD,  0,  0,  0,  0,  0, GS,\t\t/* 48-15 */\n \t 0, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA,\t\t/* 64-15 */\n-\tAA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, GS, GS,  0,  0,  0,\t\t/* 80-15 */\n-\t 0, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA,\t\t/* 96-15 */\n-\tAA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA,  0,  0,  0,  0,  0,\t\t/* 112-15 */\n+\tAA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, GS, GS,  0, RR, US,\t\t/* 80-15 */\n+\t 0, Ah, Ah, Ah, Ah, Ah, Ah, AA, AA, AA, AA, AA, AA, AA, AA, AA,\t\t/* 96-15 */\n+\tAA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, RR, RR,  0,  0,  0,\t\t/* 112-15 */\n \t/* Nothing in the 128.. range */\n };\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex e20b1e8..5eaa662 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -328,12 +328,18 @@ extern unsigned char sane_ctype[256];\n #define GIT_DIGIT 0x02\n #define GIT_ALPHA 0x04\n #define GIT_SPECIAL 0x08\n+#define GIT_REGEX_SPECIAL 0x10\n+#define GIT_UNDERSCORE 0x20\n+#define GIT_LOWER_XDIGIT 0x40\n #define sane_istest(x,mask) ((sane_ctype[(unsigned char)(x)] & (mask)) != 0)\n #define isspace(x) sane_istest(x,GIT_SPACE)\n #define isdigit(x) sane_istest(x,GIT_DIGIT)\n #define isalpha(x) sane_istest(x,GIT_ALPHA)\n #define isalnum(x) sane_istest(x,GIT_ALPHA | GIT_DIGIT)\n #define isspecial(x) sane_istest(x,GIT_SPECIAL)\n+#define isregexspecial(x) sane_istest(x,GIT_SPECIAL | GIT_REGEX_SPECIAL)\n+#define iswordchar(x) sane_istest(x,GIT_ALPHA | GIT_DIGIT | GIT_UNDERSCORE)\n+#define islowerxdigit(x) sane_istest(x,GIT_DIGIT | GIT_LOWER_XDIGIT)\n #define tolower(x) sane_case((unsigned char)(x), 0x20)\n #define toupper(x) sane_case((unsigned char)(x), 0)\n \n"},{"id":"100249","messageId":"7vy6xfd3oh.fsf@gitster.siamese.dyndns.org","threadId":"17070","inReplyTo":"496B9780.3030000@lsrfire.ath.cx","subject":"Re: [PATCH 2/2] grep: don't call regexec() for fixed strings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-01-13T08:13:18Z","receivedAt":"2009-01-13T08:13:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:\n\n> +#define RR GIT_REGEX_SPECIAL /* $, (, ), +, ., ^, {, | */\n> +#define US GIT_UNDERSCORE\n> +#define Ah (GIT_ALPHA | GIT_LOWER_XDIGIT)\n>  \n>  unsigned char sane_ctype[256] = {\n>  \tGS,  0,  0,  0,  0,  0,  0,  0,  0, SS, SS,  0,  0, SS,  0,  0,\t\t/* 0-15 */\n\nMental note.  NUL is marked as GIT_SPECIAL.\n\n>  #define isspecial(x) sane_istest(x,GIT_SPECIAL)\n> +#define isregexspecial(x) sane_istest(x,GIT_SPECIAL | GIT_REGEX_SPECIAL)\n\nPerhaps isspecial() is misnamed if we were to enhance git-ctype in this\nway.  It is about a byte being shell glob pattern or a NUL (!!!), and it\nshould be renamed to isglobspecial() or something.\n\ndir.c uses isspecial() in two places, and both callers rely on NUL being a\npart of special to terminate the loops they are in, like this:\n\n\tfor (;;) {\n\t\tunsigned char c = *match++;\n\t\tlen++;\n\t\tif (isspecial(c))\n\t\t\treturn len;\n\t}\n\nIt may be a cunning and cute logic, but I do not particularly like it.  It\nmight be cleaner to rename it to isglobspecial(), drop NUL from it, and\nhave these two call existing call sites to explicitly check for (c == NUL)\nfor loop termination.\n"},{"id":"100845","messageId":"1232207413.16172.104.camel@ubuntu.ubuntu-domain","threadId":"17070","inReplyTo":"7vy6xfd3oh.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 1/4] Add ctype test","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-17T15:50:13Z","receivedAt":"2009-01-17T15:50:13Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Manipulating the character class table in ctype.c by hand is error prone.\nTo ensure that typos are found quickly, add a test program and script.\n\ntest-ctype checks the output of the character class macros isspace() et.\nal. by applying them on all possible char values and consulting a list of\nall characters in the particular class.  It doesn't check tolower() and\ntoupper(); this could be added later.\n\nThe test script t0070-fundamental.sh is created because there is no good\nplace for the ctype test, yet -- except for t0000-basic.sh perhaps, but\nit doesn't run well on Windows, yet.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\n Makefile               |    3 ++\n t/t0070-fundamental.sh |   15 +++++++++++\n test-ctype.c           |   66 ++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 84 insertions(+), 0 deletions(-)\n create mode 100755 t/t0070-fundamental.sh\n create mode 100644 test-ctype.c\n\ndiff --git a/Makefile b/Makefile\nindex 4b1d488..dca61f5 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1360,6 +1360,7 @@ endif\n ### Testing rules\n \n TEST_PROGRAMS += test-chmtime$X\n+TEST_PROGRAMS += test-ctype$X\n TEST_PROGRAMS += test-date$X\n TEST_PROGRAMS += test-delta$X\n TEST_PROGRAMS += test-genrandom$X\n@@ -1379,6 +1380,8 @@ export NO_SVN_TESTS\n test: all\n \t$(MAKE) -C t/ all\n \n+test-ctype$X: ctype.o\n+\n test-date$X: date.o ctype.o\n \n test-delta$X: diff-delta.o patch-delta.o\ndiff --git a/t/t0070-fundamental.sh b/t/t0070-fundamental.sh\nnew file mode 100755\nindex 0000000..680d7d6\n--- /dev/null\n+++ b/t/t0070-fundamental.sh\n@@ -0,0 +1,15 @@\n+#!/bin/sh\n+\n+test_description='check that the most basic functions work\n+\n+\n+Verify wrappers and compatibility functions.\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'character classes (isspace, isalpha etc.)' '\n+\ttest-ctype\n+'\n+\n+test_done\ndiff --git a/test-ctype.c b/test-ctype.c\nnew file mode 100644\nindex 0000000..723eff4\n--- /dev/null\n+++ b/test-ctype.c\n@@ -0,0 +1,66 @@\n+#include \"cache.h\"\n+\n+\n+static int test_isdigit(int c)\n+{\n+\treturn isdigit(c);\n+}\n+\n+static int test_isspace(int c)\n+{\n+\treturn isspace(c);\n+}\n+\n+static int test_isalpha(int c)\n+{\n+\treturn isalpha(c);\n+}\n+\n+static int test_isalnum(int c)\n+{\n+\treturn isalnum(c);\n+}\n+\n+#define DIGIT \"0123456789\"\n+#define LOWER \"abcdefghijklmnopqrstuvwxyz\"\n+#define UPPER \"ABCDEFGHIJKLMNOPQRSTUVWXYZ\"\n+\n+static const struct ctype_class {\n+\tconst char *name;\n+\tint (*test_fn)(int);\n+\tconst char *members;\n+} classes[] = {\n+\t{ \"isdigit\", test_isdigit, DIGIT },\n+\t{ \"isspace\", test_isspace, \" \\n\\r\\t\" },\n+\t{ \"isalpha\", test_isalpha, LOWER UPPER },\n+\t{ \"isalnum\", test_isalnum, LOWER UPPER DIGIT },\n+\t{ NULL }\n+};\n+\n+static int test_class(const struct ctype_class *test)\n+{\n+\tint i, rc = 0;\n+\n+\tfor (i = 0; i < 256; i++) {\n+\t\tint expected = i ? !!strchr(test->members, i) : 0;\n+\t\tint actual = test->test_fn(i);\n+\n+\t\tif (actual != expected) {\n+\t\t\trc = 1;\n+\t\t\tprintf(\"%s classifies char %d (0x%02x) wrongly\\n\",\n+\t\t\t       test->name, i, i);\n+\t\t}\n+\t}\n+\treturn rc;\n+}\n+\n+int main(int argc, char **argv)\n+{\n+\tconst struct ctype_class *test;\n+\tint rc = 0;\n+\n+\tfor (test = classes; test->name; test++)\n+\t\trc |= test_class(test);\n+\n+\treturn rc;\n+}\n-- \n1.6.1\n"},{"id":"100848","messageId":"1232207429.16172.105.camel@ubuntu.ubuntu-domain","threadId":"17070","inReplyTo":"7vy6xfd3oh.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 2/4] Reformat ctype.c","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-17T15:50:29Z","receivedAt":"2009-01-17T15:50:29Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Enhance the readability of ctype.c by using an enum instead of macros\nto initialize the character class table.  This allows the use of a single\nletter to mark a char, making the table fit within 80 columns.\n\nAlso list the index of the last entry in each row in the following comment.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\n ctype.c |   32 ++++++++++++++------------------\n 1 files changed, 14 insertions(+), 18 deletions(-)\n\ndiff --git a/ctype.c b/ctype.c\nindex 9208d67..6528687 100644\n--- a/ctype.c\n+++ b/ctype.c\n@@ -5,25 +5,21 @@\n  */\n #include \"cache.h\"\n \n-/* Just so that no insane platform contaminate namespace with these symbols */\n-#undef SS\n-#undef AA\n-#undef DD\n-#undef GS\n-\n-#define SS GIT_SPACE\n-#define AA GIT_ALPHA\n-#define DD GIT_DIGIT\n-#define GS GIT_SPECIAL  /* \\0, *, ?, [, \\\\ */\n+enum {\n+\tS = GIT_SPACE,\n+\tA = GIT_ALPHA,\n+\tD = GIT_DIGIT,\n+\tG = GIT_SPECIAL,\t/* \\0, *, ?, [, \\\\ */\n+};\n \n unsigned char sane_ctype[256] = {\n-\tGS,  0,  0,  0,  0,  0,  0,  0,  0, SS, SS,  0,  0, SS,  0,  0,\t\t/* 0-15 */\n-\t 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,\t\t/* 16-15 */\n-\tSS,  0,  0,  0,  0,  0,  0,  0,  0,  0, GS,  0,  0,  0,  0,  0,\t\t/* 32-15 */\n-\tDD, DD, DD, DD, DD, DD, DD, DD, DD, DD,  0,  0,  0,  0,  0, GS,\t\t/* 48-15 */\n-\t 0, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA,\t\t/* 64-15 */\n-\tAA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, GS, GS,  0,  0,  0,\t\t/* 80-15 */\n-\t 0, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA,\t\t/* 96-15 */\n-\tAA, AA, AA, AA, AA, AA, AA, AA, AA, AA, AA,  0,  0,  0,  0,  0,\t\t/* 112-15 */\n+\tG, 0, 0, 0, 0, 0, 0, 0, 0, S, S, 0, 0, S, 0, 0,\t\t/*   0.. 15 */\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\t\t/*  16.. 31 */\n+\tS, 0, 0, 0, 0, 0, 0, 0, 0, 0, G, 0, 0, 0, 0, 0,\t\t/*  32.. 47 */\n+\tD, D, D, D, D, D, D, D, D, D, 0, 0, 0, 0, 0, G,\t\t/*  48.. 63 */\n+\t0, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A,\t\t/*  64.. 79 */\n+\tA, A, A, A, A, A, A, A, A, A, A, G, G, 0, 0, 0,\t\t/*  80.. 95 */\n+\t0, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A,\t\t/*  96..111 */\n+\tA, A, A, A, A, A, A, A, A, A, A, 0, 0, 0, 0, 0,\t\t/* 112..127 */\n \t/* Nothing in the 128.. range */\n };\n-- \n1.6.1\n"},{"id":"100846","messageId":"1232207434.16172.106.camel@ubuntu.ubuntu-domain","threadId":"17070","inReplyTo":"7vy6xfd3oh.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 3/4] Change NUL char handling of isspecial()","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-17T15:50:34Z","receivedAt":"2009-01-17T15:50:34Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Replace isspecial() by the new macro is_glob_special(), which is more,\nwell, specialized.  The former included the NUL char in its character\nclass, while the letter only included characters that are special to\nfile name globbing.\n\nThe new name contains underscores because they enhance readability\nconsiderably now that it's made up of three words.  Renaming the\nfunction is necessary to document its changed scope.\n\nThe call sites of isspecial() are updated to check explicitly for NUL.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\nThis patch applies to next (plus the previous ones in this series).\n\n ctype.c           |    4 ++--\n dir.c             |    4 ++--\n git-compat-util.h |    4 ++--\n grep.c            |    5 +++--\n test-ctype.c      |    6 ++++++\n 5 files changed, 15 insertions(+), 8 deletions(-)\n\ndiff --git a/ctype.c b/ctype.c\nindex 6528687..9de187c 100644\n--- a/ctype.c\n+++ b/ctype.c\n@@ -9,11 +9,11 @@ enum {\n \tS = GIT_SPACE,\n \tA = GIT_ALPHA,\n \tD = GIT_DIGIT,\n-\tG = GIT_SPECIAL,\t/* \\0, *, ?, [, \\\\ */\n+\tG = GIT_GLOB_SPECIAL,\t/* *, ?, [, \\\\ */\n };\n \n unsigned char sane_ctype[256] = {\n-\tG, 0, 0, 0, 0, 0, 0, 0, 0, S, S, 0, 0, S, 0, 0,\t\t/*   0.. 15 */\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, S, S, 0, 0, S, 0, 0,\t\t/*   0.. 15 */\n \t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\t\t/*  16.. 31 */\n \tS, 0, 0, 0, 0, 0, 0, 0, 0, 0, G, 0, 0, 0, 0, 0,\t\t/*  32.. 47 */\n \tD, D, D, D, D, D, D, D, D, D, 0, 0, 0, 0, 0, G,\t\t/*  48.. 63 */\ndiff --git a/dir.c b/dir.c\nindex 7c59829..d55a41a 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -75,7 +75,7 @@ static int match_one(const char *match, const char *name, int namelen)\n \tfor (;;) {\n \t\tunsigned char c1 = *match;\n \t\tunsigned char c2 = *name;\n-\t\tif (isspecial(c1))\n+\t\tif (c1 == '\\0' || is_glob_special(c1))\n \t\t\tbreak;\n \t\tif (c1 != c2)\n \t\t\treturn 0;\n@@ -678,7 +678,7 @@ static int simple_length(const char *match)\n \tfor (;;) {\n \t\tunsigned char c = *match++;\n \t\tlen++;\n-\t\tif (isspecial(c))\n+\t\tif (c == '\\0' || is_glob_special(c))\n \t\t\treturn len;\n \t}\n }\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex e20b1e8..7c92588 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -327,13 +327,13 @@ extern unsigned char sane_ctype[256];\n #define GIT_SPACE 0x01\n #define GIT_DIGIT 0x02\n #define GIT_ALPHA 0x04\n-#define GIT_SPECIAL 0x08\n+#define GIT_GLOB_SPECIAL 0x08\n #define sane_istest(x,mask) ((sane_ctype[(unsigned char)(x)] & (mask)) != 0)\n #define isspace(x) sane_istest(x,GIT_SPACE)\n #define isdigit(x) sane_istest(x,GIT_DIGIT)\n #define isalpha(x) sane_istest(x,GIT_ALPHA)\n #define isalnum(x) sane_istest(x,GIT_ALPHA | GIT_DIGIT)\n-#define isspecial(x) sane_istest(x,GIT_SPECIAL)\n+#define is_glob_special(x) sane_istest(x,GIT_GLOB_SPECIAL)\n #define tolower(x) sane_case((unsigned char)(x), 0x20)\n #define toupper(x) sane_case((unsigned char)(x), 0)\n \ndiff --git a/grep.c b/grep.c\nindex 6485760..f9a4525 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -30,8 +30,9 @@ void append_grep_pattern(struct grep_opt *opt, const char *pat,\n \n static int isregexspecial(int c)\n {\n-\treturn isspecial(c) || c == '$' || c == '(' || c == ')' || c == '+' ||\n-\t\t\t       c == '.' || c == '^' || c == '{' || c == '|';\n+\treturn c == '\\0' || is_glob_special(c) ||\n+\t\tc == '$' || c == '(' || c == ')' || c == '+' ||\n+\t\tc == '.' || c == '^' || c == '{' || c == '|';\n }\n \n static int is_fixed(const char *s)\ndiff --git a/test-ctype.c b/test-ctype.c\nindex 723eff4..d6425d5 100644\n--- a/test-ctype.c\n+++ b/test-ctype.c\n@@ -21,6 +21,11 @@ static int test_isalnum(int c)\n \treturn isalnum(c);\n }\n \n+static int test_is_glob_special(int c)\n+{\n+\treturn is_glob_special(c);\n+}\n+\n #define DIGIT \"0123456789\"\n #define LOWER \"abcdefghijklmnopqrstuvwxyz\"\n #define UPPER \"ABCDEFGHIJKLMNOPQRSTUVWXYZ\"\n@@ -34,6 +39,7 @@ static const struct ctype_class {\n \t{ \"isspace\", test_isspace, \" \\n\\r\\t\" },\n \t{ \"isalpha\", test_isalpha, LOWER UPPER },\n \t{ \"isalnum\", test_isalnum, LOWER UPPER DIGIT },\n+\t{ \"is_glob_special\", test_is_glob_special, \"*?[\\\\\" },\n \t{ NULL }\n };\n \n-- \n1.6.1\n"},{"id":"100847","messageId":"1232207437.16172.107.camel@ubuntu.ubuntu-domain","threadId":"17070","inReplyTo":"7vy6xfd3oh.fsf@gitster.siamese.dyndns.org","subject":"[PATCH 4/4] Add is_regex_special()","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2009-01-17T15:50:37Z","receivedAt":"2009-01-17T15:50:37Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Add is_regex_special(), a character class macro for chars that have a\nspecial meaning in regular expressions.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\nThis patch applies to next (plus the previous ones in this series).\n\n ctype.c           |    7 ++++---\n git-compat-util.h |    2 ++\n grep.c            |    9 +--------\n test-ctype.c      |    6 ++++++\n 4 files changed, 13 insertions(+), 11 deletions(-)\n\ndiff --git a/ctype.c b/ctype.c\nindex 9de187c..b90ec00 100644\n--- a/ctype.c\n+++ b/ctype.c\n@@ -10,16 +10,17 @@ enum {\n \tA = GIT_ALPHA,\n \tD = GIT_DIGIT,\n \tG = GIT_GLOB_SPECIAL,\t/* *, ?, [, \\\\ */\n+\tR = GIT_REGEX_SPECIAL,\t/* $, (, ), +, ., ^, {, | * */\n };\n \n unsigned char sane_ctype[256] = {\n \t0, 0, 0, 0, 0, 0, 0, 0, 0, S, S, 0, 0, S, 0, 0,\t\t/*   0.. 15 */\n \t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\t\t/*  16.. 31 */\n-\tS, 0, 0, 0, 0, 0, 0, 0, 0, 0, G, 0, 0, 0, 0, 0,\t\t/*  32.. 47 */\n+\tS, 0, 0, 0, R, 0, 0, 0, R, R, G, R, 0, 0, R, 0,\t\t/*  32.. 47 */\n \tD, D, D, D, D, D, D, D, D, D, 0, 0, 0, 0, 0, G,\t\t/*  48.. 63 */\n \t0, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A,\t\t/*  64.. 79 */\n-\tA, A, A, A, A, A, A, A, A, A, A, G, G, 0, 0, 0,\t\t/*  80.. 95 */\n+\tA, A, A, A, A, A, A, A, A, A, A, G, G, 0, R, 0,\t\t/*  80.. 95 */\n \t0, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A,\t\t/*  96..111 */\n-\tA, A, A, A, A, A, A, A, A, A, A, 0, 0, 0, 0, 0,\t\t/* 112..127 */\n+\tA, A, A, A, A, A, A, A, A, A, A, R, R, 0, 0, 0,\t\t/* 112..127 */\n \t/* Nothing in the 128.. range */\n };\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 7c92588..079cbe9 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -328,12 +328,14 @@ extern unsigned char sane_ctype[256];\n #define GIT_DIGIT 0x02\n #define GIT_ALPHA 0x04\n #define GIT_GLOB_SPECIAL 0x08\n+#define GIT_REGEX_SPECIAL 0x10\n #define sane_istest(x,mask) ((sane_ctype[(unsigned char)(x)] & (mask)) != 0)\n #define isspace(x) sane_istest(x,GIT_SPACE)\n #define isdigit(x) sane_istest(x,GIT_DIGIT)\n #define isalpha(x) sane_istest(x,GIT_ALPHA)\n #define isalnum(x) sane_istest(x,GIT_ALPHA | GIT_DIGIT)\n #define is_glob_special(x) sane_istest(x,GIT_GLOB_SPECIAL)\n+#define is_regex_special(x) sane_istest(x,GIT_GLOB_SPECIAL | GIT_REGEX_SPECIAL)\n #define tolower(x) sane_case((unsigned char)(x), 0x20)\n #define toupper(x) sane_case((unsigned char)(x), 0)\n \ndiff --git a/grep.c b/grep.c\nindex f9a4525..062b2b6 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -28,16 +28,9 @@ void append_grep_pattern(struct grep_opt *opt, const char *pat,\n \tp->next = NULL;\n }\n \n-static int isregexspecial(int c)\n-{\n-\treturn c == '\\0' || is_glob_special(c) ||\n-\t\tc == '$' || c == '(' || c == ')' || c == '+' ||\n-\t\tc == '.' || c == '^' || c == '{' || c == '|';\n-}\n-\n static int is_fixed(const char *s)\n {\n-\twhile (!isregexspecial(*s))\n+\twhile (*s && !is_regex_special(*s))\n \t\ts++;\n \treturn !*s;\n }\ndiff --git a/test-ctype.c b/test-ctype.c\nindex d6425d5..033c749 100644\n--- a/test-ctype.c\n+++ b/test-ctype.c\n@@ -26,6 +26,11 @@ static int test_is_glob_special(int c)\n \treturn is_glob_special(c);\n }\n \n+static int test_is_regex_special(int c)\n+{\n+\treturn is_regex_special(c);\n+}\n+\n #define DIGIT \"0123456789\"\n #define LOWER \"abcdefghijklmnopqrstuvwxyz\"\n #define UPPER \"ABCDEFGHIJKLMNOPQRSTUVWXYZ\"\n@@ -40,6 +45,7 @@ static const struct ctype_class {\n \t{ \"isalpha\", test_isalpha, LOWER UPPER },\n \t{ \"isalnum\", test_isalnum, LOWER UPPER DIGIT },\n \t{ \"is_glob_special\", test_is_glob_special, \"*?[\\\\\" },\n+\t{ \"is_regex_special\", test_is_regex_special, \"$()*+.?[\\\\^{|\" },\n \t{ NULL }\n };\n \n-- \n1.6.1\n"}]}