{"thread":{"id":"43285","subject":"[PATCH] Store peeled refs in packed-refs (take 2).","startedAt":"2006-11-22T07:36:35Z","lastAt":"2006-11-25T08:09:25Z","messageCount":3,"participants":["Junio C Hamano","Marco Costalba"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"294224","messageId":"7vac2kvsos.fsf@assigned-by-dhcp.cox.net","threadId":"43285","inReplyTo":null,"subject":"[PATCH] Store peeled refs in packed-refs (take 2).","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-22T07:36:35Z","receivedAt":"2006-11-22T07:36:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This fixes the previous implementation which failed to optimize\nrepositories with tons of lightweight tags.  The updated\npacked-refs format begins with \"# packed-refs with:\" line that\nlists the kind of extended data the file records.  Currently,\nthere is only one such extension defined, \"peeled\".  This stores\nthe \"peeled tag\" on a line that immediately follows a line for a\ntag object itself in the format \"^<sha-1>\".\n\nThe header line itself and any extended data are ignored by\nolder implementation, so packed-refs file generated with this\nversion can still be used by older git.  packed-refs made by\nolder git can of course be used with this version.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n builtin-pack-refs.c |    8 +++-\n builtin-show-ref.c  |    6 ++-\n refs.c              |  111 +++++++++++++++++++++++++++++---------------------\n refs.h              |    7 +--\n 4 files changed, 77 insertions(+), 55 deletions(-)\n\ndiff --git a/builtin-pack-refs.c b/builtin-pack-refs.c\nindex ee5a556..8dc5b9e 100644\n--- a/builtin-pack-refs.c\n+++ b/builtin-pack-refs.c\n@@ -46,8 +46,8 @@ static int handle_one_ref(const char *pa\n \t\tif (o->type == OBJ_TAG) {\n \t\t\to = deref_tag(o, path, 0);\n \t\t\tif (o)\n-\t\t\t\tfprintf(cb->refs_file, \"%s  %s^{}\\n\",\n-\t\t\t\t\tsha1_to_hex(o->sha1), path);\n+\t\t\t\tfprintf(cb->refs_file, \"^%s\\n\",\n+\t\t\t\t\tsha1_to_hex(o->sha1));\n \t\t}\n \t}\n \n@@ -111,6 +111,10 @@ int cmd_pack_refs(int argc, const char *\n \tif (!cbdata.refs_file)\n \t\tdie(\"unable to create ref-pack file structure (%s)\",\n \t\t    strerror(errno));\n+\n+\t/* perhaps other traits later as well */\n+\tfprintf(cbdata.refs_file, \"# pack-refs with: peeled \\n\");\n+\n \tfor_each_ref(handle_one_ref, &cbdata);\n \tfflush(cbdata.refs_file);\n \tfsync(fd);\ndiff --git a/builtin-show-ref.c b/builtin-show-ref.c\nindex 9ae3d08..0739798 100644\n--- a/builtin-show-ref.c\n+++ b/builtin-show-ref.c\n@@ -67,8 +67,10 @@ match:\n \t\treturn 0;\n \n \tif ((flag & REF_ISPACKED) && !peel_ref(refname, peeled)) {\n-\t\thex = find_unique_abbrev(peeled, abbrev);\n-\t\tprintf(\"%s %s^{}\\n\", hex, refname);\n+\t\tif (!is_null_sha1(peeled)) {\n+\t\t\thex = find_unique_abbrev(peeled, abbrev);\n+\t\t\tprintf(\"%s %s^{}\\n\", hex, refname);\n+\t\t}\n \t}\n \telse {\n \t\tobj = parse_object(sha1);\ndiff --git a/refs.c b/refs.c\nindex 75cbc0e..96ea8b6 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -5,14 +5,18 @@\n \n #include <errno.h>\n \n+/* ISSYMREF=01 and ISPACKED=02 are public interfaces */\n+#define REF_KNOWS_PEELED 04\n+\n struct ref_list {\n \tstruct ref_list *next;\n-\tunsigned char flag; /* ISSYMREF? ISPACKED? ISPEELED? */\n+\tunsigned char flag; /* ISSYMREF? ISPACKED? */\n \tunsigned char sha1[20];\n+\tunsigned char peeled[20];\n \tchar name[FLEX_ARRAY];\n };\n \n-static const char *parse_ref_line(char *line, unsigned char *sha1, int *flag)\n+static const char *parse_ref_line(char *line, unsigned char *sha1)\n {\n \t/*\n \t * 42: the answer to everything.\n@@ -23,7 +27,6 @@ static const char *parse_ref_line(char *\n \t *  +1 (newline at the end of the line)\n \t */\n \tint len = strlen(line) - 42;\n-\tint peeled = 0;\n \n \tif (len <= 0)\n \t\treturn NULL;\n@@ -32,29 +35,18 @@ static const char *parse_ref_line(char *\n \tif (!isspace(line[40]))\n \t\treturn NULL;\n \tline += 41;\n-\n-\tif (isspace(*line)) {\n-\t\t/* \"SHA-1 SP SP refs/tags/tagname^{} LF\"? */\n-\t\tline++;\n-\t\tlen--;\n-\t\tpeeled = 1;\n-\t}\n+\tif (isspace(*line))\n+\t\treturn NULL;\n \tif (line[len] != '\\n')\n \t\treturn NULL;\n \tline[len] = 0;\n \n-\tif (peeled && (len < 3 || strcmp(line + len - 3, \"^{}\")))\n-\t\treturn NULL;\n-\n-\tif (!peeled)\n-\t\t*flag &= ~REF_ISPEELED;\n-\telse\n-\t\t*flag |= REF_ISPEELED;\n \treturn line;\n }\n \n static struct ref_list *add_ref(const char *name, const unsigned char *sha1,\n-\t\t\t\tint flag, struct ref_list *list)\n+\t\t\t\tint flag, struct ref_list *list,\n+\t\t\t\tstruct ref_list **new_entry)\n {\n \tint len;\n \tstruct ref_list **p = &list, *entry;\n@@ -66,8 +58,11 @@ static struct ref_list *add_ref(const ch\n \t\t\tbreak;\n \n \t\t/* Same as existing entry? */\n-\t\tif (!cmp)\n+\t\tif (!cmp) {\n+\t\t\tif (new_entry)\n+\t\t\t\t*new_entry = entry;\n \t\t\treturn list;\n+\t\t}\n \t\tp = &entry->next;\n \t}\n \n@@ -75,10 +70,13 @@ static struct ref_list *add_ref(const ch\n \tlen = strlen(name) + 1;\n \tentry = xmalloc(sizeof(struct ref_list) + len);\n \thashcpy(entry->sha1, sha1);\n+\thashclr(entry->peeled);\n \tmemcpy(entry->name, name, len);\n \tentry->flag = flag;\n \tentry->next = *p;\n \t*p = entry;\n+\tif (new_entry)\n+\t\t*new_entry = entry;\n \treturn list;\n }\n \n@@ -114,27 +112,50 @@ static void invalidate_cached_refs(void)\n \tca->did_loose = ca->did_packed = 0;\n }\n \n+static void read_packed_refs(FILE *f, struct cached_refs *cached_refs)\n+{\n+\tstruct ref_list *list = NULL;\n+\tstruct ref_list *last = NULL;\n+\tchar refline[PATH_MAX];\n+\tint flag = REF_ISPACKED;\n+\n+\twhile (fgets(refline, sizeof(refline), f)) {\n+\t\tunsigned char sha1[20];\n+\t\tconst char *name;\n+\t\tstatic const char header[] = \"# pack-refs with:\";\n+\n+\t\tif (!strncmp(refline, header, sizeof(header)-1)) {\n+\t\t\tconst char *traits = refline + sizeof(header) - 1;\n+\t\t\tif (strstr(traits, \" peeled \"))\n+\t\t\t\tflag |= REF_KNOWS_PEELED;\n+\t\t\t/* perhaps other traits later as well */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tname = parse_ref_line(refline, sha1);\n+\t\tif (name) {\n+\t\t\tlist = add_ref(name, sha1, flag, list, &last);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (last &&\n+\t\t    refline[0] == '^' &&\n+\t\t    strlen(refline) == 42 &&\n+\t\t    refline[41] == '\\n' &&\n+\t\t    !get_sha1_hex(refline + 1, sha1))\n+\t\t\thashcpy(last->peeled, sha1);\n+\t}\n+\tcached_refs->packed = list;\n+}\n+\n static struct ref_list *get_packed_refs(void)\n {\n \tif (!cached_refs.did_packed) {\n-\t\tstruct ref_list *refs = NULL;\n \t\tFILE *f = fopen(git_path(\"packed-refs\"), \"r\");\n+\t\tcached_refs.packed = NULL;\n \t\tif (f) {\n-\t\t\tstruct ref_list *list = NULL;\n-\t\t\tchar refline[PATH_MAX];\n-\t\t\twhile (fgets(refline, sizeof(refline), f)) {\n-\t\t\t\tunsigned char sha1[20];\n-\t\t\t\tint flag = REF_ISPACKED;\n-\t\t\t\tconst char *name =\n-\t\t\t\t\tparse_ref_line(refline, sha1, &flag);\n-\t\t\t\tif (!name)\n-\t\t\t\t\tcontinue;\n-\t\t\t\tlist = add_ref(name, sha1, flag, list);\n-\t\t\t}\n+\t\t\tread_packed_refs(f, &cached_refs);\n \t\t\tfclose(f);\n-\t\t\trefs = list;\n \t\t}\n-\t\tcached_refs.packed = refs;\n \t\tcached_refs.did_packed = 1;\n \t}\n \treturn cached_refs.packed;\n@@ -177,7 +198,7 @@ static struct ref_list *get_ref_dir(cons\n \t\t\t\terror(\"%s points nowhere!\", ref);\n \t\t\t\tcontinue;\n \t\t\t}\n-\t\t\tlist = add_ref(ref, sha1, flag, list);\n+\t\t\tlist = add_ref(ref, sha1, flag, list, NULL);\n \t\t}\n \t\tfree(ref);\n \t\tclosedir(dir);\n@@ -225,8 +246,7 @@ const char *resolve_ref(const char *ref,\n \t\tif (lstat(path, &st) < 0) {\n \t\t\tstruct ref_list *list = get_packed_refs();\n \t\t\twhile (list) {\n-\t\t\t\tif (!(list->flag & REF_ISPEELED) &&\n-\t\t\t\t    !strcmp(ref, list->name)) {\n+\t\t\t\tif (!strcmp(ref, list->name)) {\n \t\t\t\t\thashcpy(sha1, list->sha1);\n \t\t\t\t\tif (flag)\n \t\t\t\t\t\t*flag |= REF_ISPACKED;\n@@ -348,8 +368,6 @@ static int do_one_ref(const char *base,\n \t\treturn 0;\n \tif (is_null_sha1(entry->sha1))\n \t\treturn 0;\n-\tif (entry->flag & REF_ISPEELED)\n-\t\treturn 0;\n \tif (!has_sha1_file(entry->sha1)) {\n \t\terror(\"%s does not point to a valid object!\", entry->name);\n \t\treturn 0;\n@@ -368,22 +386,21 @@ int peel_ref(const char *ref, unsigned c\n \n \tif ((flag & REF_ISPACKED)) {\n \t\tstruct ref_list *list = get_packed_refs();\n-\t\tint len = strlen(ref);\n \n \t\twhile (list) {\n-\t\t\tif ((list->flag & REF_ISPEELED) &&\n-\t\t\t    !strncmp(list->name, ref, len) &&\n-\t\t\t    strlen(list->name) == len + 3 &&\n-\t\t\t    !strcmp(list->name + len, \"^{}\")) {\n-\t\t\t\thashcpy(sha1, list->sha1);\n-\t\t\t\treturn 0;\n+\t\t\tif (!strcmp(list->name, ref)) {\n+\t\t\t\tif (list->flag & REF_KNOWS_PEELED) {\n+\t\t\t\t\thashcpy(sha1, list->peeled);\n+\t\t\t\t\treturn 0;\n+\t\t\t\t}\n+\t\t\t\t/* older pack-refs did not leave peeled ones */\n+\t\t\t\tbreak;\n \t\t\t}\n \t\t\tlist = list->next;\n \t\t}\n-\t\t/* older pack-refs did not leave peeled ones in */\n \t}\n \n-\t/* otherwise ... */\n+\t/* fallback - callers should not call this for unpacked refs */\n \to = parse_object(base);\n \tif (o->type == OBJ_TAG) {\n \t\to = deref_tag(o, ref, 0);\ndiff --git a/refs.h b/refs.h\nindex 40048a6..cd1e1d6 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -10,14 +10,13 @@ struct ref_lock {\n \tint force_write;\n };\n \n+#define REF_ISSYMREF 01\n+#define REF_ISPACKED 02\n+\n /*\n  * Calls the specified function for each ref file until it returns nonzero,\n  * and returns the value\n  */\n-#define REF_ISSYMREF 01\n-#define REF_ISPACKED 02\n-#define REF_ISPEELED 04 /* internal use */\n-\n typedef int each_ref_fn(const char *refname, const unsigned char *sha1, int flags, void *cb_data);\n extern int head_ref(each_ref_fn, void *);\n extern int for_each_ref(each_ref_fn, void *);\n-- \n1.4.4.g26805\n\n"},{"id":"295542","messageId":"e5bfff550611220425x5cf9e33ds7b7b98229c6b408a@mail.gmail.com","threadId":"43285","inReplyTo":"7vac2kvsos.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Store peeled refs in packed-refs (take 2).","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-22T12:25:36Z","receivedAt":"2006-11-22T12:25:36Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/22/06, Junio C Hamano <junkio@cox.net> wrote:\n> there is only one such extension defined, \"peeled\".  This stores\n> the \"peeled tag\" on a line that immediately follows a line for a\n> tag object itself in the format \"^<sha-1>\".\n>\n\nThanks, I will try to do some performance tests this week end.\n\n"},{"id":"295932","messageId":"e5bfff550611250009x70a16c66h8077a2dccc1273e7@mail.gmail.com","threadId":"43285","inReplyTo":"e5bfff550611220425x5cf9e33ds7b7b98229c6b408a@mail.gmail.com","subject":"Re: [PATCH] Store peeled refs in packed-refs (take 2).","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-25T08:09:25Z","receivedAt":"2006-11-25T08:09:25Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/22/06, Marco Costalba <mcostalba@gmail.com> wrote:\n> On 11/22/06, Junio C Hamano <junkio@cox.net> wrote:\n> > there is only one such extension defined, \"peeled\".  This stores\n> > the \"peeled tag\" on a line that immediately follows a line for a\n> > tag object itself in the format \"^<sha-1>\".\n> >\n>\n> Thanks, I will try to do some performance tests this week end.\n>\n\nCold cache tests with \"git show-ref -d\" on packed refs (after\n\"git-pack-refs --prune\")\n\nBefore patch:\n\n- git tree 2374ms\n- linux tree 2225ms\n\n\n\nAfter patch:\n\n- git tree 1108ms\n"}]}