{"thread":{"id":"36831","subject":"[PATCH v6 1/2] refs.c: optimize check_refname_component()","startedAt":"2014-06-04T03:38:10Z","lastAt":"2014-06-05T22:02:45Z","messageCount":14,"participants":["David Turner","Torsten Bögershausen","Duy Nguyen","Junio C Hamano","Ondřej Bílka"],"isPatch":true,"patchVersion":6,"patchTotal":2},"messages":[{"id":"243275","messageId":"1401853091-15535-1-git-send-email-dturner@twitter.com","threadId":"36831","inReplyTo":null,"subject":"[PATCH v6 1/2] refs.c: optimize check_refname_component()","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-04T03:38:10Z","receivedAt":"2014-06-04T03:38:10Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"In a repository with many refs, check_refname_component can be a major\ncontributor to the runtime of some git commands. One such command is\n\ngit rev-parse HEAD\n\nTimings for one particular repo, with about 60k refs, almost all\npacked, are:\n\nOld: 35 ms\nNew: 29 ms\n\nMany other commands which read refs are also sped up.\n\nSigned-off-by: David Turner <dturner@twitter.com>\n---\n refs.c             | 67 +++++++++++++++++++++++++++++++-----------------------\n t/t5511-refspec.sh |  6 ++++-\n 2 files changed, 44 insertions(+), 29 deletions(-)\n\ndiff --git a/refs.c b/refs.c\nindex 28d5eca..46139d2 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -6,8 +6,29 @@\n #include \"string-list.h\"\n \n /*\n- * Make sure \"ref\" is something reasonable to have under \".git/refs/\";\n- * We do not like it if:\n+ * How to handle various characters in refnames:\n+ * 0: An acceptable character for refs\n+ * 1: End-of-component\n+ * 2: ., look for a preceding . to reject .. in refs\n+ * 3: {, look for a preceding @ to reject @{ in refs\n+ * 4: A bad character: ASCII control characters, \"~\", \"^\", \":\" or SP\n+ */\n+static unsigned char refname_disposition[256] = {\n+\t1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,\n+\t4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,\n+\t4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 1,\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n+\t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4\n+};\n+\n+/*\n+ * Try to read one refname component from the front of refname.\n+ * Return the length of the component found, or -1 if the component is\n+ * not legal.  It is legal if it is something reasonable to have under\n+ * \".git/refs/\"; We do not like it if:\n  *\n  * - any path component of it begins with \".\", or\n  * - it has double dots \"..\", or\n@@ -16,41 +37,31 @@\n  * - it ends with \".lock\"\n  * - it contains a \"\\\" (backslash)\n  */\n-\n-/* Return true iff ch is not allowed in reference names. */\n-static inline int bad_ref_char(int ch)\n-{\n-\tif (((unsigned) ch) <= ' ' || ch == 0x7f ||\n-\t    ch == '~' || ch == '^' || ch == ':' || ch == '\\\\')\n-\t\treturn 1;\n-\t/* 2.13 Pattern Matching Notation */\n-\tif (ch == '*' || ch == '?' || ch == '[') /* Unsupported */\n-\t\treturn 1;\n-\treturn 0;\n-}\n-\n-/*\n- * Try to read one refname component from the front of refname.  Return\n- * the length of the component found, or -1 if the component is not\n- * legal.\n- */\n static int check_refname_component(const char *refname, int flags)\n {\n \tconst char *cp;\n \tchar last = '\\0';\n \n \tfor (cp = refname; ; cp++) {\n-\t\tchar ch = *cp;\n-\t\tif (ch == '\\0' || ch == '/')\n+\t\tint ch = *cp & 255;\n+\t\tunsigned char disp = refname_disposition[ch];\n+\t\tswitch(disp) {\n+\t\tcase 1:\n+\t\t\tgoto out;\n+\t\tcase 2:\n+\t\t\tif (last == '.')\n+\t\t\t\treturn -1; /* Refname contains \"..\". */\n+\t\t\tbreak;\n+\t\tcase 3:\n+\t\t\tif (last == '@')\n+\t\t\t\treturn -1; /* Refname contains \"@{\". */\n \t\t\tbreak;\n-\t\tif (bad_ref_char(ch))\n-\t\t\treturn -1; /* Illegal character in refname. */\n-\t\tif (last == '.' && ch == '.')\n-\t\t\treturn -1; /* Refname contains \"..\". */\n-\t\tif (last == '@' && ch == '{')\n-\t\t\treturn -1; /* Refname contains \"@{\". */\n+\t\tcase 4:\n+\t\t\treturn -1;\n+\t\t}\n \t\tlast = ch;\n \t}\n+out:\n \tif (cp == refname)\n \t\treturn 0; /* Component has zero length. */\n \tif (refname[0] == '.') {\ndiff --git a/t/t5511-refspec.sh b/t/t5511-refspec.sh\nindex c289322..de6db86 100755\n--- a/t/t5511-refspec.sh\n+++ b/t/t5511-refspec.sh\n@@ -5,7 +5,6 @@ test_description='refspec parsing'\n . ./test-lib.sh\n \n test_refspec () {\n-\n \tkind=$1 refspec=$2 expect=$3\n \tgit config remote.frotz.url \".\" &&\n \tgit config --remove-section remote.frotz &&\n@@ -84,4 +83,9 @@ test_refspec push 'refs/heads/*/*/for-linus:refs/remotes/mine/*' invalid\n test_refspec fetch 'refs/heads/*/for-linus:refs/remotes/mine/*'\n test_refspec push 'refs/heads/*/for-linus:refs/remotes/mine/*'\n \n+good=$(printf '\\303\\204')\n+test_refspec fetch \"refs/heads/${good}\"\n+bad=$(printf '\\011tab')\n+test_refspec fetch \"refs/heads/${bad}\"\t\t\t\tinvalid\n+\n test_done\n-- \n2.0.0.rc1.18.gf763c0f\n"},{"id":"243276","messageId":"1401853091-15535-2-git-send-email-dturner@twitter.com","threadId":"36831","inReplyTo":"1401853091-15535-1-git-send-email-dturner@twitter.com","subject":"[PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-04T03:38:11Z","receivedAt":"2014-06-04T03:38:11Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"Optimize check_refname_component using SSE4.2, where available.\n\ngit rev-parse HEAD is a good test-case for this, since it does almost\nnothing except parse refs.  For one particular repo with about 60k\nrefs, almost all packed, the timings are:\n\nLook up table: 29 ms\nSSE4.2:        25 ms\n\nThis is about a 15% improvement.\n\nThe configure.ac changes include code from the GNU C Library written\nby Joseph S. Myers <joseph at codesourcery dot com>.\n\nSigned-off-by: David Turner <dturner@twitter.com>\n---\n Makefile           |   6 +++\n aclocal.m4         |   6 +++\n configure.ac       |  17 ++++++++\n git-compat-util.h  |  22 ++++++++++\n refs.c             | 117 ++++++++++++++++++++++++++++++++++++++++++++++-------\n t/t5511-refspec.sh |  13 ++++++\n 6 files changed, 166 insertions(+), 15 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex a53f3a8..dd2127a 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1326,6 +1326,11 @@ else\n \t\tCOMPAT_OBJS += compat/win32mmap.o\n \tendif\n endif\n+ifdef NO_SSE42\n+\tBASIC_CFLAGS += -DNO_SSE42\n+else\n+\tBASIC_CFLAGS += -msse4.2\n+endif\n ifdef OBJECT_CREATION_USES_RENAMES\n \tCOMPAT_CFLAGS += -DOBJECT_CREATION_MODE=1\n endif\n@@ -2199,6 +2204,7 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo NO_PERL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PERL)))'\\' >>$@\n \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@\n+\t@echo NO_SSE42=\\''$(subst ','\\'',$(subst ','\\'',$(NO_SSE42)))'\\' >>$@\n ifdef TEST_OUTPUT_DIRECTORY\n \t@echo TEST_OUTPUT_DIRECTORY=\\''$(subst ','\\'',$(subst ','\\'',$(TEST_OUTPUT_DIRECTORY)))'\\' >>$@\n endif\ndiff --git a/aclocal.m4 b/aclocal.m4\nindex f11bc7e..d9f3f19 100644\n--- a/aclocal.m4\n+++ b/aclocal.m4\n@@ -38,3 +38,9 @@ AC_DEFUN([TYPE_SOCKLEN_T],\n       [#include <sys/types.h>\n #include <sys/socket.h>])\n ])\n+\n+dnl Test a compiler option or options with an empty input file.\n+dnl LIBC_TRY_CC_OPTION([options], [action-if-true], [action-if-false])\n+AC_DEFUN([LIBC_TRY_CC_OPTION],\n+[AS_IF([AC_TRY_COMMAND([${CC-cc} $1 -xc /dev/null -S -o /dev/null])],\n+\t[$2], [$3])])\ndiff --git a/configure.ac b/configure.ac\nindex b711254..3a5bda9 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -382,6 +382,23 @@ AS_HELP_STRING([],[Tcl/Tk interpreter will be found in a system.]),\n GIT_PARSE_WITH(tcltk))\n #\n \n+# Declare the with-sse42/without-sse42 options.\n+AC_ARG_WITH(sse42,\n+AS_HELP_STRING([--with-sse42],[use SSE4.2 instructions])\n+AS_HELP_STRING([],[(default is YES if your compiler supports -msse4.2)]),\n+GIT_PARSE_WITH(sse42))\n+\n+if test \"$NO_SSE42\" != \"YesPlease\"; then\n+   dnl Check if -msse4.2 works.\n+   AC_CACHE_CHECK(for SSE4.2 support, cc_cv_sse42, [dnl\n+   LIBC_TRY_CC_OPTION([-msse4.2], [cc_cv_sse42=yes], [cc_cv_sse42=no])\n+   ])\n+   if test $cc_cv_sse42 = no; then\n+     NO_SSE42=1\n+   fi\n+fi\n+\n+GIT_CONF_SUBST([NO_SSE42])\n \n ## Checks for programs.\n AC_MSG_NOTICE([CHECKS for programs])\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex f6d3a46..218d510 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -668,6 +668,28 @@ void git_qsort(void *base, size_t nmemb, size_t size,\n #endif\n #endif\n \n+#ifndef NO_SSE42\n+#include <nmmintrin.h>\n+/*\n+ * Clang ships with a version of nmmintrin.h that's incomplete; if\n+ * necessary, we define the constants that we're going to use.\n+ */\n+#ifndef _SIDD_UBYTE_OPS\n+#define _SIDD_UBYTE_OPS                 0x00\n+#define _SIDD_CMP_EQUAL_ANY             0x00\n+#define _SIDD_CMP_RANGES                0x04\n+#define _SIDD_CMP_EQUAL_ORDERED         0x0c\n+#define _SIDD_NEGATIVE_POLARITY         0x10\n+#endif\n+\n+/* This is the system memory page size; it's used so that we can read\n+ * outside the bounds of an allocation without segfaulting.\n+ */\n+#ifndef PAGE_SIZE\n+#define PAGE_SIZE 4096\n+#endif\n+#endif\n+\n #ifdef UNRELIABLE_FSTAT\n #define fstat_is_reliable() 0\n #else\ndiff --git a/refs.c b/refs.c\nindex 46139d2..2fe0075 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -24,6 +24,25 @@ static unsigned char refname_disposition[256] = {\n \t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4\n };\n \n+static int check_refname_component_trailer(const char *cp, const char *refname, int flags)\n+{\n+\tif (cp == refname)\n+\t\treturn 0; /* Component has zero length. */\n+\tif (refname[0] == '.') {\n+\t\tif (!(flags & REFNAME_DOT_COMPONENT))\n+\t\t\treturn -1; /* Component starts with '.'. */\n+\t\t/*\n+\t\t * Even if leading dots are allowed, don't allow \".\"\n+\t\t * as a component (\"..\" is prevented by a rule above).\n+\t\t */\n+\t\tif (refname[1] == '\\0')\n+\t\t\treturn -1; /* Component equals \".\". */\n+\t}\n+\tif (cp - refname >= 5 && !memcmp(cp - 5, \".lock\", 5))\n+\t\treturn -1; /* Refname ends with \".lock\". */\n+\treturn cp - refname;\n+}\n+\n /*\n  * Try to read one refname component from the front of refname.\n  * Return the length of the component found, or -1 if the component is\n@@ -37,7 +56,7 @@ static unsigned char refname_disposition[256] = {\n  * - it ends with \".lock\"\n  * - it contains a \"\\\" (backslash)\n  */\n-static int check_refname_component(const char *refname, int flags)\n+static int check_refname_component_1(const char *refname, int flags)\n {\n \tconst char *cp;\n \tchar last = '\\0';\n@@ -47,7 +66,7 @@ static int check_refname_component(const char *refname, int flags)\n \t\tunsigned char disp = refname_disposition[ch];\n \t\tswitch(disp) {\n \t\tcase 1:\n-\t\t\tgoto out;\n+\t\t\treturn check_refname_component_trailer(cp, refname, flags);\n \t\tcase 2:\n \t\t\tif (last == '.')\n \t\t\t\treturn -1; /* Refname contains \"..\". */\n@@ -61,23 +80,91 @@ static int check_refname_component(const char *refname, int flags)\n \t\t}\n \t\tlast = ch;\n \t}\n-out:\n-\tif (cp == refname)\n-\t\treturn 0; /* Component has zero length. */\n-\tif (refname[0] == '.') {\n-\t\tif (!(flags & REFNAME_DOT_COMPONENT))\n-\t\t\treturn -1; /* Component starts with '.'. */\n+}\n+\n+#ifdef NO_SSE42\n+#define check_refname_component check_refname_component_1\n+#else\n+#define SSE_VECTOR_BYTES 16\n+\n+/* Vectorized version of check_refname_component */\n+static int check_refname_component(const char *refname, int flags)\n+{\n+\tconst __m128i *refname_vec = (__m128i*) refname;\n+\n+\t/* Character ranges for characters forbidden in refs; see above */\n+\tstatic const __v16qi bad = {\n+\t\t0x01, 0x20,  0x7e, 0x7f,  0x5e, 0x5e,  0x3a, 0x3a,\n+\t\t0x5b, 0x5c,  0x2a, 0x2a,  0x3f, 0x3f,  0x3f, 0x3f};\n+\n+\tstatic const __v16qi nonslashes = {\n+\t\t'\\001', '/' -1, '/' + 1, 0xff,\n+\t};\n+\n+\tstatic const __v16qi dotdot = {'.', '.', 0};\n+\tstatic const __v16qi atcurly = {'@', '{', 0};\n+\n+\tconst __m128i *vp;\n+\tconst char *cp = (const char *)refname_vec;\n+\n+\tint dotdotpos = SSE_VECTOR_BYTES, atcurlypos = SSE_VECTOR_BYTES;\n+\tfor (vp = refname_vec; ; vp++) {\n+\t\t__m128i tmp;\n+\t\tint endpos;\n+\n \t\t/*\n-\t\t * Even if leading dots are allowed, don't allow \".\"\n-\t\t * as a component (\"..\" is prevented by a rule above).\n+\t\t * Handle case of forbidden substrings .. and @{ crossing\n+\t\t * sixteen-byte boundaries\n \t\t */\n-\t\tif (refname[1] == '\\0')\n-\t\t\treturn -1; /* Component equals \".\". */\n+\t\tif (dotdotpos == 15 && *cp == '.')\n+\t\t\treturn -1;\n+\n+\t\tif (atcurlypos == 15 && *cp == '{')\n+\t\t\treturn -1;\n+\n+\t\tif (((uintptr_t) vp % PAGE_SIZE) > PAGE_SIZE - SSE_VECTOR_BYTES)\n+\t\t\t/*\n+\t\t\t * End-of-page; fall back to slow method for\n+\t\t\t * this entire component.\n+\t\t\t */\n+\t\t\treturn check_refname_component_1(refname, flags);\n+\n+\t\ttmp = _mm_lddqu_si128(vp);\n+\n+\t\t/*\n+\t\t * Find slashes or end-of-string. The double-negative\n+\t\t * (negative-polarity search for non-slashes) is\n+\t\t * necessary so that \\0 will also be counted.\n+\t\t */\n+\t\tendpos = _mm_cmpistri((__m128i) nonslashes, tmp,\n+\t\t\t\t      _SIDD_UBYTE_OPS | _SIDD_CMP_RANGES |\n+\t\t\t\t      _SIDD_NEGATIVE_POLARITY);\n+\n+\t\tif (_mm_cmpestrc((__m128i) bad, SSE_VECTOR_BYTES, tmp, endpos,\n+\t\t\t\t _SIDD_UBYTE_OPS | _SIDD_CMP_RANGES))\n+\t\t\treturn -1;\n+\n+\t\tdotdotpos = _mm_cmpestri((__m128i) dotdot, 2, tmp, endpos,\n+\t\t\t\t\t _SIDD_UBYTE_OPS |\n+\t\t\t\t\t _SIDD_CMP_EQUAL_ORDERED);\n+\t\tif (dotdotpos < 15)\n+\t\t\treturn -1;\n+\n+\t\tatcurlypos = _mm_cmpestri((__m128i) atcurly, 2, tmp, endpos,\n+\t\t\t\t\t  _SIDD_UBYTE_OPS |\n+\t\t\t\t\t  _SIDD_CMP_EQUAL_ORDERED);\n+\t\tif (atcurlypos < 15)\n+\t\t\treturn -1;\n+\n+\t\tif (endpos < SSE_VECTOR_BYTES) {\n+\t\t\tcp = ((const char*) vp) + endpos;\n+\t\t\tbreak;\n+\t\t}\n+\t\tcp = (const char*) vp + SSE_VECTOR_BYTES;\n \t}\n-\tif (cp - refname >= 5 && !memcmp(cp - 5, \".lock\", 5))\n-\t\treturn -1; /* Refname ends with \".lock\". */\n-\treturn cp - refname;\n+\treturn check_refname_component_trailer(cp, refname, flags);\n }\n+#endif\n \n int check_refname_format(const char *refname, int flags)\n {\ndiff --git a/t/t5511-refspec.sh b/t/t5511-refspec.sh\nindex de6db86..7f1bd74 100755\n--- a/t/t5511-refspec.sh\n+++ b/t/t5511-refspec.sh\n@@ -88,4 +88,17 @@ test_refspec fetch \"refs/heads/${good}\"\n bad=$(printf '\\011tab')\n test_refspec fetch \"refs/heads/${bad}\"\t\t\t\tinvalid\n \n+test_refspec fetch 'refs/heads/a-very-long-refname'\n+test_refspec fetch 'refs/heads/.a-very-long-refname'\t\tinvalid\n+test_refspec fetch 'refs/heads/abcdefgh0123..'\t\t\tinvalid\n+test_refspec fetch 'refs/heads/abcdefgh01234..'\t\t\tinvalid\n+test_refspec fetch 'refs/heads/abcdefgh012345..'\t\tinvalid\n+test_refspec fetch 'refs/heads/abcdefgh0123456..'\t\tinvalid\n+test_refspec fetch 'refs/heads/abcdefgh01234567..'\t\tinvalid\n+test_refspec fetch 'refs/heads/abcdefgh0123.a'\n+test_refspec fetch 'refs/heads/abcdefgh01234.a'\n+test_refspec fetch 'refs/heads/abcdefgh012345.a'\n+test_refspec fetch 'refs/heads/abcdefgh0123456.a'\n+test_refspec fetch 'refs/heads/abcdefgh01234567.a'\n+\n test_done\n-- \n2.0.0.rc1.18.gf763c0f\n"},{"id":"243279","messageId":"538ED2F1.9030003@web.de","threadId":"36831","inReplyTo":"1401853091-15535-2-git-send-email-dturner@twitter.com","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2014-06-04T08:04:01Z","receivedAt":"2014-06-04T08:04:01Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"\nOn 2014-06-04 05.38, David Turner wrote:\n[]\n> []\n> diff --git a/Makefile b/Makefile\n> index a53f3a8..dd2127a 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -1326,6 +1326,11 @@ else\n>  \t\tCOMPAT_OBJS += compat/win32mmap.o\n>  \tendif\n>  endif\n> +ifdef NO_SSE42\n> +\tBASIC_CFLAGS += -DNO_SSE42\n> +else\n> +\tBASIC_CFLAGS += -msse4.2\n> +endif\nThis does work for some people, but break for others, like the systems in my test-lab.\nOn 2 different systems the gcc has support for -msse4.2, but the processor has not,\nand t5511 fails with \"Illegal instruction\".\nHow can that be?\nThe maintainer of a Linux distro wants to ship gcc with all possible features,\nan the end-user can compile the code with all the features his very processor has.\n\nOn the other hand, a pre-compiled package like e.g. Git is compiled into a binary package\nwith all the latest features switched of, to be able to run the binary on as many different\nprocessor variants as possible.\n\nHe already needs to make 3 binaries only for x86:\n\n- the minimum version is a 32 bit processor like 486/586/686.\n- a \"medium\" version for systems with 4GB RAM (or more) which have 32 bit processors with PAE (686-pae)\n- a version for x86_64\n\nE.g. a maintainer wants to have SSE42 enabled, when he builds Git for his system,\nbut disabled when he builds an RPM.\nThe people compiling Git need to know what the binary is used for, how about\nusing something like this in Makefile:\n\nifdef HAVE_SSE42\n\tBASIC_CFLAGS += -msse4.2 -DHAS_SSE42\n\n+endif\n\n>  ifdef OBJECT_CREATION_USES_RENAMES\n>  \tCOMPAT_CFLAGS += -DOBJECT_CREATION_MODE=1\n>  endif\n> @@ -2199,6 +2204,7 @@ GIT-BUILD-OPTIONS: FORCE\n>  \t@echo NO_PERL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PERL)))'\\' >>$@\n>  \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@\n>  \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@\n> +\t@echo NO_SSE42=\\''$(subst ','\\'',$(subst ','\\'',$(NO_SSE42)))'\\' >>$@\nSame here: Use HAVE_SSE42 rather than NO_SSE42\n\n> diff --git a/aclocal.m4 b/aclocal.m4\n> index f11bc7e..d9f3f19 100644\n> --- a/aclocal.m4\n> +++ b/aclocal.m4\n> @@ -38,3 +38,9 @@ AC_DEFUN([TYPE_SOCKLEN_T],\n>        [#include <sys/types.h>\n>  #include <sys/socket.h>])\n>  ])\nAs the whole detection logic does not work as expected (we need to compile and test-run the code,\nnot only compile),\ncan we drop this part completely ? (at least for the first round)\n\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -668,6 +668,28 @@ void git_qsort(void *base, size_t nmemb, size_t size,\n>  #endif\n>  #endif\n>  \n> +#ifndef NO_SSE42\n> +#include <nmmintrin.h>\n> +/*\n> + * Clang ships with a version of nmmintrin.h that's incomplete; if\n> + * necessary, we define the constants that we're going to use.\n> + */\n> +#ifndef _SIDD_UBYTE_OPS\n> +#define _SIDD_UBYTE_OPS                 0x00\n> +#define _SIDD_CMP_EQUAL_ANY             0x00\n> +#define _SIDD_CMP_RANGES                0x04\n> +#define _SIDD_CMP_EQUAL_ORDERED         0x0c\n> +#define _SIDD_NEGATIVE_POLARITY         0x10\n> +#endif\nWhy do this defines end up in git-compat-util.h when they are needed by one file?\n(see even below)\n\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -24,6 +24,25 @@ static unsigned char refname_disposition[256] = {\n>  \t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4\n>  };\n>  \n> +static int check_refname_component_trailer(const char *cp, const char *refname, int flags)\n> +{\n> +\tif (cp == refname)\n> +\t\treturn 0; /* Component has zero length. */\n> +\tif (refname[0] == '.') {\n> +\t\tif (!(flags & REFNAME_DOT_COMPONENT))\n> +\t\t\treturn -1; /* Component starts with '.'. */\n> +\t\t/*\n> +\t\t * Even if leading dots are allowed, don't allow \".\"\n> +\t\t * as a component (\"..\" is prevented by a rule above).\n> +\t\t */\n> +\t\tif (refname[1] == '\\0')\n> +\t\t\treturn -1; /* Component equals \".\". */\n> +\t}\n> +\tif (cp - refname >= 5 && !memcmp(cp - 5, \".lock\", 5))\n> +\t\treturn -1; /* Refname ends with \".lock\". */\n> +\treturn cp - refname;\n> +}\n> +\n>  /*\n>   * Try to read one refname component from the front of refname.\n>   * Return the length of the component found, or -1 if the component is\n> @@ -37,7 +56,7 @@ static unsigned char refname_disposition[256] = {\n>   * - it ends with \".lock\"\n>   * - it contains a \"\\\" (backslash)\n>   */\n> -static int check_refname_component(const char *refname, int flags)\n> +static int check_refname_component_1(const char *refname, int flags)\nThe name check_refname_component_1() doesn't tell too much,\n(check_refname_component_sse42()  or check_refname_component_nonsse42() say more)\n\ncan I suggest to move all SSE code out to a file under compat/,\nlike compat/refs_sse42.c, or something similar ?\n(And here we need the missing sse4 defines from git-compat-util.h)\n>  {\n>  \tconst char *cp;\n>  \tchar last = '\\0';\n> @@ -47,7 +66,7 @@ static int check_refname_component(const char *refname, int flags)\n>  \t\tunsigned char disp = refname_disposition[ch];\n>  \t\tswitch(disp) {\n>  \t\tcase 1:\n> -\t\t\tgoto out;\n> +\t\t\treturn check_refname_component_trailer(cp, refname, flags);\n>  \t\tcase 2:\n>  \t\t\tif (last == '.')\n>  \t\t\t\treturn -1; /* Refname contains \"..\". */\n> @@ -61,23 +80,91 @@ static int check_refname_component(const char *refname, int flags)\n>  \t\t}\n>  \t\tlast = ch;\n>  \t}\n> -out:\n> -\tif (cp == refname)\n> -\t\treturn 0; /* Component has zero length. */\n> -\tif (refname[0] == '.') {\n> -\t\tif (!(flags & REFNAME_DOT_COMPONENT))\n> -\t\t\treturn -1; /* Component starts with '.'. */\n> +}\n> +\n> +#ifdef NO_SSE42\n> +#define check_refname_component check_refname_component_1\n> +#else\n> +#define SSE_VECTOR_BYTES 16\nSee above, all sse42 related stuff, should it be isolated in a seperate file?\n> +\n> +/* Vectorized version of check_refname_component */\n> +static int check_refname_component(const char *refname, int flags)\n> +{\n[]\n"},{"id":"243282","messageId":"CACsJy8CK3LNaPVNv=EfFX06uOgpujAz364ZDFL3HBPicDNF57w@mail.gmail.com","threadId":"36831","inReplyTo":"538ED2F1.9030003@web.de","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2014-06-04T11:21:30Z","receivedAt":"2014-06-04T11:21:30Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Jun 4, 2014 at 3:04 PM, Torsten Bögershausen <tboegi@web.de> wrote:\n>\n> On 2014-06-04 05.38, David Turner wrote:\n> []\n>> []\n>> diff --git a/Makefile b/Makefile\n>> index a53f3a8..dd2127a 100644\n>> --- a/Makefile\n>> +++ b/Makefile\n>> @@ -1326,6 +1326,11 @@ else\n>>               COMPAT_OBJS += compat/win32mmap.o\n>>       endif\n>>  endif\n>> +ifdef NO_SSE42\n>> +     BASIC_CFLAGS += -DNO_SSE42\n>> +else\n>> +     BASIC_CFLAGS += -msse4.2\n>> +endif\n> This does work for some people, but break for others, like the systems in my test-lab.\n> On 2 different systems the gcc has support for -msse4.2, but the processor has not,\n> and t5511 fails with \"Illegal instruction\".\n> How can that be?\n> The maintainer of a Linux distro wants to ship gcc with all possible features,\n> an the end-user can compile the code with all the features his very processor has.\n\nI think glibc code uses cpuid instruction to decide whether to use\noptimized version. May be we can do the same? If we go that route and\nhave a way to detect sse support from compiler, then we can drop\nNO_SSE42, enable all and pick one at runtime.\n-- \nDuy\n"},{"id":"243291","messageId":"538F2C6B.2030004@web.de","threadId":"36831","inReplyTo":"CACsJy8CK3LNaPVNv=EfFX06uOgpujAz364ZDFL3HBPicDNF57w@mail.gmail.com","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2014-06-04T14:25:47Z","receivedAt":"2014-06-04T14:25:47Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2014-06-04 13.21, Duy Nguyen wrote:\n> On Wed, Jun 4, 2014 at 3:04 PM, Torsten Bögershausen <tboegi@web.de> wrote:\n>>\n>> On 2014-06-04 05.38, David Turner wrote:\n>> []\n>>> []\n>>> diff --git a/Makefile b/Makefile\n>>> index a53f3a8..dd2127a 100644\n>>> --- a/Makefile\n>>> +++ b/Makefile\n>>> @@ -1326,6 +1326,11 @@ else\n>>>               COMPAT_OBJS += compat/win32mmap.o\n>>>       endif\n>>>  endif\n>>> +ifdef NO_SSE42\n>>> +     BASIC_CFLAGS += -DNO_SSE42\n>>> +else\n>>> +     BASIC_CFLAGS += -msse4.2\n>>> +endif\n>> This does work for some people, but break for others, like the systems in my test-lab.\n>> On 2 different systems the gcc has support for -msse4.2, but the processor has not,\n>> and t5511 fails with \"Illegal instruction\".\n>> How can that be?\n>> The maintainer of a Linux distro wants to ship gcc with all possible features,\n>> an the end-user can compile the code with all the features his very processor has.\n> \n> I think glibc code uses cpuid instruction to decide whether to use\n> optimized version. May be we can do the same? If we go that route and\n> have a way to detect sse support from compiler, then we can drop\n> NO_SSE42, enable all and pick one at runtime.\n> \nRunning make under a non-X86 processor like arm fails, as his gcc does not have -msse4.2\n\nOn the other hand, looking here: \nhttp://sourceware.org/ml/libc-alpha/2009-10/msg00063.html\nand looking into refs.c,\nit seems as if we can try to run \nstrcspn(refname, bad_characters)\nand \nstrstr(refname, \"@{\"\nand \nstrstr(refname, \"..\"\non each refname, instead of checking each char in a loop.\nThe library will pick the fastest version for strcspn() automatically.\n\nDavid, the repo you run the tests on, is it public?\nOr is there a public repo with this many refs ?\nOr can you make a dummy repo with 60k refs ?\n"},{"id":"243354","messageId":"1401916476.18134.165.camel@stross","threadId":"36831","inReplyTo":"538ED2F1.9030003@web.de","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-04T21:14:36Z","receivedAt":"2014-06-04T21:14:36Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Wed, 2014-06-04 at 10:04 +0200, Torsten Bögershausen wrote:\n[snip discussion of compiler flags; I'll look into a cpuid approach]\n\n> > --- a/git-compat-util.h\n> > +++ b/git-compat-util.h\n> > @@ -668,6 +668,28 @@ void git_qsort(void *base, size_t nmemb, size_t size,\n> >  #endif\n> >  #endif\n> >  \n> > +#ifndef NO_SSE42\n> > +#include <nmmintrin.h>\n> > +/*\n> > + * Clang ships with a version of nmmintrin.h that's incomplete; if\n> > + * necessary, we define the constants that we're going to use.\n> > + */\n> > +#ifndef _SIDD_UBYTE_OPS\n> > +#define _SIDD_UBYTE_OPS                 0x00\n> > +#define _SIDD_CMP_EQUAL_ANY             0x00\n> > +#define _SIDD_CMP_RANGES                0x04\n> > +#define _SIDD_CMP_EQUAL_ORDERED         0x0c\n> > +#define _SIDD_NEGATIVE_POLARITY         0x10\n> > +#endif\n> Why do this defines end up in git-compat-util.h when they are needed by one file?\n> (see even below)\n\nBecause Junio told me to:\n\"We would prefer not to add inclusion of any system header files in\nrandom *.c files, as there often are system dependencies (order of\ninclusion, definition of feature macros, etc.) we would rather want\nto encapsulate in one place, that is git-compat-util.h.\"\n\n> > --- a/refs.c\n> > +++ b/refs.c\n> > @@ -24,6 +24,25 @@ static unsigned char refname_disposition[256] = {\n> >  \t0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4\n> >  };\n> >  \n> > +static int check_refname_component_trailer(const char *cp, const char *refname, int flags)\n> > +{\n> > +\tif (cp == refname)\n> > +\t\treturn 0; /* Component has zero length. */\n> > +\tif (refname[0] == '.') {\n> > +\t\tif (!(flags & REFNAME_DOT_COMPONENT))\n> > +\t\t\treturn -1; /* Component starts with '.'. */\n> > +\t\t/*\n> > +\t\t * Even if leading dots are allowed, don't allow \".\"\n> > +\t\t * as a component (\"..\" is prevented by a rule above).\n> > +\t\t */\n> > +\t\tif (refname[1] == '\\0')\n> > +\t\t\treturn -1; /* Component equals \".\". */\n> > +\t}\n> > +\tif (cp - refname >= 5 && !memcmp(cp - 5, \".lock\", 5))\n> > +\t\treturn -1; /* Refname ends with \".lock\". */\n> > +\treturn cp - refname;\n> > +}\n> > +\n> >  /*\n> >   * Try to read one refname component from the front of refname.\n> >   * Return the length of the component found, or -1 if the component is\n> > @@ -37,7 +56,7 @@ static unsigned char refname_disposition[256] = {\n> >   * - it ends with \".lock\"\n> >   * - it contains a \"\\\" (backslash)\n> >   */\n> > -static int check_refname_component(const char *refname, int flags)\n> > +static int check_refname_component_1(const char *refname, int flags)\n> The name check_refname_component_1() doesn't tell too much,\n> (check_refname_component_sse42()  or check_refname_component_nonsse42() say more)\n\nI'll go with \"_bytewise\", since that's how it works.\n\n> can I suggest to move all SSE code out to a file under compat/,\n> like compat/refs_sse42.c, or something similar ?\n\nSince this is a relatively small section of code, I think that would be\noverkill.  Does anyone else have an opinion?\n"},{"id":"243355","messageId":"1401916560.18134.167.camel@stross","threadId":"36831","inReplyTo":"538F2C6B.2030004@web.de","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-04T21:16:00Z","receivedAt":"2014-06-04T21:16:00Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Wed, 2014-06-04 at 16:25 +0200, Torsten Bögershausen wrote:\n> On the other hand, looking here: \n> http://sourceware.org/ml/libc-alpha/2009-10/msg00063.html\n> and looking into refs.c,\n> it seems as if we can try to run \n> strcspn(refname, bad_characters)\n> and \n> strstr(refname, \"@{\"\n> and \n> strstr(refname, \"..\"\n> on each refname, instead of checking each char in a loop.\n> The library will pick the fastest version for strcspn() automatically.\n\nYes, you could try that, but I worry that it would be less efficient,\nbecause it duplicates the looping machinery.\n\n> David, the repo you run the tests on, is it public?\n\nUnfortunately, it is an internal Twitter repo.\n\n> Or is there a public repo with this many refs ?\n\nI do not know of one.\n\n> Or can you make a dummy repo with 60k refs ?\n\nSure!  I actually went with > 120k to make measurement easier:\nhttps://github.com/dturner-tw/many-refs\n"},{"id":"243356","messageId":"xmqqsinktkjv.fsf@gitster.dls.corp.google.com","threadId":"36831","inReplyTo":"1401916476.18134.165.camel@stross","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-06-04T21:46:28Z","receivedAt":"2014-06-04T21:46:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Turner <dturner@twopensource.com> writes:\n\n> On Wed, 2014-06-04 at 10:04 +0200, Torsten Bögershausen wrote:\n> [snip discussion of compiler flags; I'll look into a cpuid approach]\n\nHmmmm, I am not sure if the complexity is really worth it.\n\nIn any case, [PATCH 1/2] is fairly uncontroversial, so I am inclined\nto queue it by itself early without waiting for the discussion on\n2/2 to settle.\n\n>> The name check_refname_component_1() doesn't tell too much,\n>> (check_refname_component_sse42()  or check_refname_component_nonsse42() say more)\n>\n> I'll go with \"_bytewise\", since that's how it works.\n\nThat naming assumes that there will never be any alternative\nimplementation of the bytewise checker other than the one that uses\nsse42, no?\n\n>> can I suggest to move all SSE code out to a file under compat/,\n>> like compat/refs_sse42.c, or something similar ?\n>\n> Since this is a relatively small section of code, I think that would be\n> overkill.  Does anyone else have an opinion?\n\nIf we foresee people on other architectures to invent different\nvectorized implementations on their favourite archs, we may end up\nseparating it out into compat/.  I have no opinion on how likely\nthat will happen, though, and because this is a small piece of code\nright now, it shouldn't be too painful to reorganize when the time\ncomes.\n"},{"id":"243390","messageId":"539062D9.60000@web.de","threadId":"36831","inReplyTo":"1401916560.18134.167.camel@stross","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2014-06-05T12:30:17Z","receivedAt":"2014-06-05T12:30:17Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2014-06-04 23.16, David Turner wrote:\n> \n> Sure!  I actually went with > 120k to make measurement easier:\n> https://github.com/dturner-tw/many-refs\nHm, I didn't get so man\n\ngit remote -v\norigin  https://github.com/dturner-tw/many-refs \n\n wc .git/packed-refs \n     750    1130   38868 .git/packed-refs\n\n\n\ntime git rev-parse HEAD\n7ac416f789fd4f1e398449113f6e1ec1d699141a\n\nreal    0m0.008s\nuser    0m0.002s\nsys     0m0.004s\n\nwhere only patch 1/2 doesn't seem to speed up things on my system:\n\ntime ~/projects/git/tb.140604_DavidTurner_SSE4/git rev-parse HEAD\n7ac416f789fd4f1e398449113f6e1ec1d699141a\n\nreal    0m0.010s\nuser    0m0.002s\nsys     0m0.005s\n\n\n\nIntel Core Duo @ 2.4 Ghz, Mac OS\n\n(and I get similar values under an AMD Dual core running 2Ghz under Linux)\n"},{"id":"243391","messageId":"20140605125846.GB9316@domone.podge","threadId":"36831","inReplyTo":"539062D9.60000@web.de","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Ondřej Bílka","fromEmail":"neleai@seznam.cz","sentAt":"2014-06-05T12:58:46Z","receivedAt":"2014-06-05T12:58:46Z","isPatch":true,"sender":{"key":"neleai@seznam.cz","avatar":"https://avatars.githubusercontent.com/u/48067?v=4"},"body":"On Thu, Jun 05, 2014 at 02:30:17PM +0200, Torsten Bögershausen wrote:\n> On 2014-06-04 23.16, David Turner wrote:\n> > \n> > Sure!  I actually went with > 120k to make measurement easier:\n> > https://github.com/dturner-tw/many-refs\n> Hm, I didn't get so man\n> \n> git remote -v\n> origin  https://github.com/dturner-tw/many-refs \n> \n>  wc .git/packed-refs \n>      750    1130   38868 .git/packed-refs\n> \n> \n> \n> time git rev-parse HEAD\n> 7ac416f789fd4f1e398449113f6e1ec1d699141a\n> \n> real    0m0.008s\n> user    0m0.002s\n> sys     0m0.004s\n> \n> where only patch 1/2 doesn't seem to speed up things on my system:\n> \n> time ~/projects/git/tb.140604_DavidTurner_SSE4/git rev-parse HEAD\n> 7ac416f789fd4f1e398449113f6e1ec1d699141a\n> \n> real    0m0.010s\n> user    0m0.002s\n> sys     0m0.005s\n> \n> \nCould you run it 100 times to get better resolution? This could be just\nmeasurement error.\n"},{"id":"243416","messageId":"1401996362.18134.179.camel@stross","threadId":"36831","inReplyTo":"539062D9.60000@web.de","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-05T19:26:02Z","receivedAt":"2014-06-05T19:26:02Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2014-06-05 at 14:30 +0200, Torsten Bögershausen wrote:\n> On 2014-06-04 23.16, David Turner wrote:\n> > \n> > Sure!  I actually went with > 120k to make measurement easier:\n> > https://github.com/dturner-tw/many-refs\n> Hm, I didn't get so man\n> \n> git remote -v\n> origin  https://github.com/dturner-tw/many-refs \n> \n>  wc .git/packed-refs \n>      750    1130   38868 .git/packed-refs\n> \n\nOops.  It looks like I forgot to push all of the refs.  And when I try,\nit fails with \"fatal: cannot exec 'send-pack': Argument list too long\"\n\nI hacked git to send batches of 1000; maybe I'll actually make a real\npatch with ARG_MAX at some point. Anyway, this is uploading now, but I\nestimate that it will take at least five hours, because github is being\nreally slow about this.\n\n...\n> where only patch 1/2 doesn't seem to speed up things on my system:\n...\n\nI would not expect a noticeable change on a tiny number of refs; it's\nonly when ref parsing takes up a large percentage of runtime -- tens of\nthousands of refs.\n"},{"id":"243415","messageId":"1401996440.18134.180.camel@stross","threadId":"36831","inReplyTo":"xmqqsinktkjv.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-05T19:27:20Z","receivedAt":"2014-06-05T19:27:20Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Wed, 2014-06-04 at 14:46 -0700, Junio C Hamano wrote:\n> David Turner <dturner@twopensource.com> writes:\n> \n> > On Wed, 2014-06-04 at 10:04 +0200, Torsten Bögershausen wrote:\n> > [snip discussion of compiler flags; I'll look into a cpuid approach]\n> \n> Hmmmm, I am not sure if the complexity is really worth it.\n> \n> In any case, [PATCH 1/2] is fairly uncontroversial, so I am inclined\n> to queue it by itself early without waiting for the discussion on\n> 2/2 to settle.\n> \n> >> The name check_refname_component_1() doesn't tell too much,\n> >> (check_refname_component_sse42()  or check_refname_component_nonsse42() say more)\n> >\n> > I'll go with \"_bytewise\", since that's how it works.\n> \n> That naming assumes that there will never be any alternative\n> implementation of the bytewise checker other than the one that uses\n> sse42, no?\n\ncheck_refname_component_1 is the non-sse (LUT) one; I assume that there\nwill only be one implementation of that (and if there's later another\none we can rename it).  I guess this is strong evidence for _1 being a\nbad name.\n"},{"id":"243421","messageId":"5390E43B.3070501@web.de","threadId":"36831","inReplyTo":"1401996362.18134.179.camel@stross","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2014-06-05T21:42:19Z","receivedAt":"2014-06-05T21:42:19Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2014-06-05 21.26, David Turner wrote:\n> On Thu, 2014-06-05 at 14:30 +0200, Torsten Bögershausen wrote:\n>> On 2014-06-04 23.16, David Turner wrote:\n>>>\n>>> Sure!  I actually went with > 120k to make measurement easier:\n>>> https://github.com/dturner-tw/many-refs\n>> Hm, I didn't get so man\n>>\n>> git remote -v\n>> origin  https://github.com/dturner-tw/many-refs \n>>\n>>  wc .git/packed-refs \n>>      750    1130   38868 .git/packed-refs\n>>\n> \n> Oops.  It looks like I forgot to push all of the refs.  And when I try,\n> it fails with \"fatal: cannot exec 'send-pack': Argument list too long\"\n\nI just noticed that I may be able to re-use code from t5551 to create a repo\nwith 50000 tags.\n\n\nAnd how about renaming check_refname_component_1() into\ncheck_refname_component_slow() ;-)\n"},{"id":"243426","messageId":"1402005765.18134.181.camel@stross","threadId":"36831","inReplyTo":"5390E43B.3070501@web.de","subject":"Re: [PATCH v6 2/2] refs.c: SSE4.2 optimizations for check_refname_component","fromName":"David Turner","fromEmail":"dturner@twopensource.com","sentAt":"2014-06-05T22:02:45Z","receivedAt":"2014-06-05T22:02:45Z","isPatch":true,"sender":{"key":"novalis@novalis.org","avatar":"https://avatars.githubusercontent.com/u/77003?v=4"},"body":"On Thu, 2014-06-05 at 23:42 +0200, Torsten Bögershausen wrote:\n> On 2014-06-05 21.26, David Turner wrote:\n> > On Thu, 2014-06-05 at 14:30 +0200, Torsten Bögershausen wrote:\n> >> On 2014-06-04 23.16, David Turner wrote:\n> >>>\n> >>> Sure!  I actually went with > 120k to make measurement easier:\n> >>> https://github.com/dturner-tw/many-refs\n> >> Hm, I didn't get so man\n> >>\n> >> git remote -v\n> >> origin  https://github.com/dturner-tw/many-refs \n> >>\n> >>  wc .git/packed-refs \n> >>      750    1130   38868 .git/packed-refs\n> >>\n> > \n> > Oops.  It looks like I forgot to push all of the refs.  And when I try,\n> > it fails with \"fatal: cannot exec 'send-pack': Argument list too long\"\n> \n> I just noticed that I may be able to re-use code from t5551 to create a repo\n> with 50000 tags.\n\nThat's good, because github really didn't like me having a repo with\nthousands of refs.  Try 100k, tho, because the difference is easier to\nsee.\n"}]}