{"thread":{"id":"43080","subject":"Re: [WISH] Store also tag dereferences in packed-refs","startedAt":"2006-11-18T09:15:12Z","lastAt":"2006-11-20T19:32:19Z","messageCount":29,"participants":["Marco Costalba","Jakub Narebski","Linus Torvalds","Petr Baudis","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"296527","messageId":"e5bfff550611180115j135746a1h916e8ae029d1374d@mail.gmail.com","threadId":"43080","inReplyTo":null,"subject":"[WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-18T09:15:12Z","receivedAt":"2006-11-18T09:15:12Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Time needed to execute git-peek-remote in my box with cold cache currently is:\n\n- git tree 2.347 ms\n- linux tree 2.144 ms\n\nAnd refs are *already* packed in both repos.\n\nLooking at packed-refs file, it contains something like:\n\nd9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n\nwhile I would need something like git-peek-remote output,\n\nd9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\ne0b0830726286287744cc9e1a629a534bbe75452 refs/tags/v1.4.3^{}\n4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n1965efb1599f59b8e3380335d1fa395e2008a30b refs/tags/v1.4.3-rc1^{}\n\nBecause the sha value a tag points to is needed to match against\ngit-rev-list output so to identify tagged revisions.\n\nWould be possible to store in packed-refs also the dereferenced tag\ninfo, so that cold opening of a repository would be much faster?\n\nJust to give an idea, with warmed up cache, refs reading times are:\n\n- git tree 43 ms\n- linux tree 28 ms\n\nThanks\nMarco\n\nP.S: In case it's not clear I don't suggest to read directly the\npacked-refs file with the added info, but always to use\n"},{"id":"296632","messageId":"7vmz6oeh2k.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"e5bfff550611180115j135746a1h916e8ae029d1374d@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-18T18:38:11Z","receivedAt":"2006-11-18T18:38:11Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Marco Costalba\" writes:\n\n> Time needed to execute git-peek-remote in my box with cold cache currently is:\n>\n> - git tree 2.347 ms\n> - linux tree 2.144 ms\n>\n> And refs are *already* packed in both repos.\n>\n> Looking at packed-refs file, it contains something like:\n>\n> d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n>\n> while I would need something like git-peek-remote output,\n>\n> d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> e0b0830726286287744cc9e1a629a534bbe75452 refs/tags/v1.4.3^{}\n> 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n> 1965efb1599f59b8e3380335d1fa395e2008a30b refs/tags/v1.4.3-rc1^{}\n>\n> Because the sha value a tag points to is needed to match against\n> git-rev-list output so to identify tagged revisions.\n>\n> Would be possible to store in packed-refs also the dereferenced tag\n> info, so that cold opening of a repository would be much faster?\n>\n> Just to give an idea, with warmed up cache, refs reading times are:\n>\n> - git tree 43 ms\n> - linux tree 28 ms\n>\n> Thanks\n> Marco\n>\n> P.S: In case it's not clear I don't suggest to read directly the\n> packed-refs file with the added info, but always to use\n> git-peek-remote that _would_ became much faster.\n\nI think the question is why you would want to run peek-remote.\nDo you use the ^{} peeled-onion information and if so how and\nwhy?\n\n"},{"id":"295970","messageId":"20061118184345.GO7201@pasky.or.cz","threadId":"43080","inReplyTo":"7vmz6oeh2k.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-11-18T18:43:46Z","receivedAt":"2006-11-18T18:43:46Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Sat, Nov 18, 2006 at 07:38:11PM CET, Junio C Hamano wrote:\n> \"Marco Costalba\" writes:\n> \n> > Time needed to execute git-peek-remote in my box with cold cache currently is:\n> >\n> > - git tree 2.347 ms\n> > - linux tree 2.144 ms\n> >\n> > And refs are *already* packed in both repos.\n> >\n> > Looking at packed-refs file, it contains something like:\n> >\n> > d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> > 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n> >\n> > while I would need something like git-peek-remote output,\n> >\n> > d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> > e0b0830726286287744cc9e1a629a534bbe75452 refs/tags/v1.4.3^{}\n> > 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n> > 1965efb1599f59b8e3380335d1fa395e2008a30b refs/tags/v1.4.3-rc1^{}\n> >\n> > Because the sha value a tag points to is needed to match against\n> > git-rev-list output so to identify tagged revisions.\n> >\n> > Would be possible to store in packed-refs also the dereferenced tag\n> > info, so that cold opening of a repository would be much faster?\n> >\n> > Just to give an idea, with warmed up cache, refs reading times are:\n> >\n> > - git tree 43 ms\n> > - linux tree 28 ms\n> >\n> > Thanks\n> > Marco\n> >\n> > P.S: In case it's not clear I don't suggest to read directly the\n> > packed-refs file with the added info, but always to use\n> > git-peek-remote that _would_ became much faster.\n> \n> I think the question is why you would want to run peek-remote.\n> Do you use the ^{} peeled-onion information and if so how and\n> why?\n\nMy wild guess would be to attach tags to the right commits in qgit?\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nThe meaning of Stonehenge in Traflamadorian, when viewed from above, is:\n\"Replacement part being rushed with all possible speed.\"\n"},{"id":"298032","messageId":"e5bfff550611181047w6712774fkccc697d312b87c7e@mail.gmail.com","threadId":"43080","inReplyTo":"20061118184345.GO7201@pasky.or.cz","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-18T18:47:40Z","receivedAt":"2006-11-18T18:47:40Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/18/06, Petr Baudis wrote:\n> On Sat, Nov 18, 2006 at 07:38:11PM CET, Junio C Hamano wrote:\n> > \"Marco Costalba\" writes:\n> >\n> > > Time needed to execute git-peek-remote in my box with cold cache currently is:\n> > >\n> > > - git tree 2.347 ms\n> > > - linux tree 2.144 ms\n> > >\n> > > And refs are *already* packed in both repos.\n> > >\n> > > Looking at packed-refs file, it contains something like:\n> > >\n> > > d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> > > 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n> > >\n> > > while I would need something like git-peek-remote output,\n> > >\n> > > d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> > > e0b0830726286287744cc9e1a629a534bbe75452 refs/tags/v1.4.3^{}\n> > > 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n> > > 1965efb1599f59b8e3380335d1fa395e2008a30b refs/tags/v1.4.3-rc1^{}\n> > >\n> > > Because the sha value a tag points to is needed to match against\n> > > git-rev-list output so to identify tagged revisions.\n> > >\n> > > Would be possible to store in packed-refs also the dereferenced tag\n> > > info, so that cold opening of a repository would be much faster?\n> > >\n> > > Just to give an idea, with warmed up cache, refs reading times are:\n> > >\n> > > - git tree 43 ms\n> > > - linux tree 28 ms\n> > >\n> > > Thanks\n> > > Marco\n> > >\n> > > P.S: In case it's not clear I don't suggest to read directly the\n> > > packed-refs file with the added info, but always to use\n> > > git-peek-remote that _would_ became much faster.\n> >\n> > I think the question is why you would want to run peek-remote.\n> > Do you use the ^{} peeled-onion information and if so how and\n> > why?\n>\n> My wild guess would be to attach tags to the right commits in qgit?\n>\nYes. It is. From a list like\n\n> > > d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n> > > e0b0830726286287744cc9e1a629a534bbe75452 refs/tags/v1.4.3^{}\n> > > 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n> > > 1965efb1599f59b8e3380335d1fa395e2008a30b refs/tags/v1.4.3-rc1^{}\n>\nqgit (but also gitk FWIK) extracts\n\ne0b0830726286287744cc9e1a629a534bbe75452\n1965efb1599f59b8e3380335d1fa395e2008a30b\n\nStores in a quick look-up container and then checks against loaded\ncommits to, as Pasky says, attach the nice green markers to tags.\n\n"},{"id":"298058","messageId":"7vac2oefuz.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"e5bfff550611181047w6712774fkccc697d312b87c7e@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-18T19:04:20Z","receivedAt":"2006-11-18T19:04:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Marco Costalba\" writes:\n\n> On 11/18/06, Petr Baudis wrote:\n>> On Sat, Nov 18, 2006 at 07:38:11PM CET, Junio C Hamano wrote:\n>> >\n>> > I think the question is why you would want to run peek-remote.\n>> > Do you use the ^{} peeled-onion information and if so how and\n>> > why?\n>>..\n> Yes. It is. From a list like\n>\n>> > > d9b0f913ce0508fcc83e642e0241f373428368e5 refs/tags/v1.4.3\n>> > > e0b0830726286287744cc9e1a629a534bbe75452 refs/tags/v1.4.3^{}\n>> > > 4314f5982d2aac08001a977fc0b1b611e858e025 refs/tags/v1.4.3-rc1\n>> > > 1965efb1599f59b8e3380335d1fa395e2008a30b refs/tags/v1.4.3-rc1^{}\n>>\n> qgit (but also gitk FWIK) extracts\n>\n> e0b0830726286287744cc9e1a629a534bbe75452\n> 1965efb1599f59b8e3380335d1fa395e2008a30b\n>\n> Stores in a quick look-up container and then checks against loaded\n> commits to, as Pasky says, attach the nice green markers to tags.\n\nTwo answers.\n\nA quick one that is to the point to solve \"your\" problem.\n\n\tshow-ref -d\n\nNot a quick one but that may lead to solution of a similar issue\nfor wider audiences is...\n\nI wonder how fast update-server-info is under the same condition\nwith your earlier timing.\n\nI am not suggesting you to use update-server-info. The reason I\nam wondering about it are:\n\n (1) traditionally, \"peek-remote .\" has been the only way to get\n to that information, so you have every right to keep doing\n so;\n\n (2) however, even with presense of packed-refs, upload-pack\n that is invoked by peek-remote needs to consult unpacked\n refs first and then fall back to packed-refs, and only\n using the ^{} information \"cached\" in packed-refs file and\n computing ^{} itself when dealing with unpacked refs means\n more code, which we need to assess the pros-and-cons.\n\n (3) another inefficiency of using \"peek-remote .\" is that it\n spawns another process in the \"remote\" repo and talks with\n it.\n\nSo storing this information making upload-pack to reuse it when\nit can might make things go faster for other applications but\nfirst I wanted to know how much overhead is incurred in the\nextra upload-pack process, and time update-server-info needs to\nprepare the info in .git/info/refs would be a way to get a rough\nestimate for that (you subtract that from \"peek-remote .\" time).\n\n\n"},{"id":"296501","messageId":"e5bfff550611181628o41e11652ycd17ddad5dd21225@mail.gmail.com","threadId":"43080","inReplyTo":"7vac2oefuz.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-19T00:28:50Z","receivedAt":"2006-11-19T00:28:50Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":">\n> A quick one that is to the point to solve \"your\" problem.\n>\n> show-ref -d\n>\n\nI was out for dinner, just come back home.\n\nSome quick tests with show-ref -d instead of peek-remote:\n\n- git tree 2374ms\n- linux tree 2225ms\n\nNot a big change. I reboot before each test to have a guaranteed cold cache.\n\nTested also with show-ref only, not useful to me, but just as a comparison.\n\n- git tree 1420ms\n- linux tree 1021ms\n\nBetter, but still slower then what I would expected.\n\nIn both case CPU load is low, processes are heavily I/O bound, so\navoiding some fork does not save the day.\n\nPlease, tell me if you want me to run some kind of additional test.\n\n> I wonder how fast update-server-info is under the same condition\n> with your earlier timing.\n>\n> I am not suggesting you to use update-server-info. The reason I\n> am wondering about it are:\n>\n> (1) traditionally, \"peek-remote .\" has been the only way to get\n> to that information, so you have every right to keep doing\n> so;\n>\n> (2) however, even with presense of packed-refs, upload-pack\n> that is invoked by peek-remote needs to consult unpacked\n> refs first and then fall back to packed-refs, and only\n> using the ^{} information \"cached\" in packed-refs file and\n> computing ^{} itself when dealing with unpacked refs means\n> more code, which we need to assess the pros-and-cons.\n>\n> (3) another inefficiency of using \"peek-remote .\" is that it\n> spawns another process in the \"remote\" repo and talks with\n> it.\n>\n> So storing this information making upload-pack to reuse it when\n> it can might make things go faster for other applications but\n> first I wanted to know how much overhead is incurred in the\n> extra upload-pack process, and time update-server-info needs to\n> prepare the info in .git/info/refs would be a way to get a rough\n> estimate for that (you subtract that from \"peek-remote .\" time).\n>\n\nIt's to late to understand this part of your e-mail ;-) I will read\nbetter tomorrow.\n\nThanks\n"},{"id":"298095","messageId":"Pine.LNX.4.64.0611181706250.3692@woody.osdl.org","threadId":"43080","inReplyTo":"e5bfff550611181628o41e11652ycd17ddad5dd21225@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T01:11:28Z","receivedAt":"2006-11-19T01:11:28Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Marco Costalba wrote:\n> \n> Tested also with show-ref only, not useful to me, but just as a comparison.\n> \n> - git tree 1420ms\n> - linux tree 1021ms\n> \n> Better, but still slower then what I would expected.\n\nMUCH slower than expected.\n\nCan you do\n\n\tstrace -o tracefile -Ttt git show-ref\n\nand send out the tracefile?\n\nWhat I _suspect_ is going on is that when you packed your refs, you either \ndidn't prune them (which means that packing didn't actually help you), or \nif you did prune them, since we don't seem to remove the refs \n_directories_ when packing, you still have an old and big directory for \n.git/refs/tags, and just reading that (empty, but non-shrunken) directory \ntakes time.\n\nThe -Ttt thing should show quite clearly what takes time.\n\n(This, btw, is one _huge_ reason why the old shell script things with \npiping sucked. Doing things like performance analysis is so much easier \nwith a builtin thing and really shows what's going on).\n\n"},{"id":"297462","messageId":"7vfycg9pu5.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611181706250.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-19T01:40:02Z","receivedAt":"2006-11-19T01:40:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> On Sun, 19 Nov 2006, Marco Costalba wrote:\n>> \n>> Tested also with show-ref only, not useful to me, but just as a comparison.\n>> \n>> - git tree 1420ms\n>> - linux tree 1021ms\n>> \n>> Better, but still slower then what I would expected.\n>\n> MUCH slower than expected.\n\nThis is still a WIP and for discussion only (I have to leave to\npick up my wife now so I won't be making any further progress\ntonight), but I think parse_object() in show-ref is costing us.\n\nOne worry is that existing git will misinterpret packed-refs\nfile prepared with this version.\n\nIf you say \"git show-ref -d --tags\" to this version, it takes\nthe fast path which seems to shave about 1/3 of the runtime.\n\n---\n\ndiff --git a/builtin-pack-refs.c b/builtin-pack-refs.c\nindex 042d271..0c770f5 100644\n--- a/builtin-pack-refs.c\n+++ b/builtin-pack-refs.c\n@@ -1,5 +1,7 @@\n #include \"cache.h\"\n #include \"refs.h\"\n+#include \"object.h\"\n+#include \"tag.h\"\n \n static const char builtin_pack_refs_usage[] =\n \"git-pack-refs [--all] [--prune]\";\n@@ -29,12 +31,24 @@ static int handle_one_ref(const char *pa\n \t\t\t int flags, void *cb_data)\n {\n \tstruct pack_refs_cb_data *cb = cb_data;\n+\tint is_tag_ref;\n \n-\tif (!cb->all && strncmp(path, \"refs/tags/\", 10))\n-\t\treturn 0;\n \t/* Do not pack the symbolic refs */\n-\tif (!(flags & REF_ISSYMREF))\n-\t\tfprintf(cb->refs_file, \"%s %s\\n\", sha1_to_hex(sha1), path);\n+\tif ((flags & REF_ISSYMREF))\n+\t\treturn 0;\n+\tis_tag_ref = !strncmp(path, \"refs/tags/\", 10);\n+\tif (!cb->all && !is_tag_ref)\n+\t\treturn 0;\n+\n+\tfprintf(cb->refs_file, \"%s %s\\n\", sha1_to_hex(sha1), path);\n+\tif (is_tag_ref) {\n+\t\tstruct object *o = parse_object(sha1);\n+\t\to = deref_tag(o, path, 0);\n+\t\tif (o)\n+\t\t\tfprintf(cb->refs_file, \"%s %s^{}\\n\",\n+\t\t\t\tsha1_to_hex(o->sha1), path);\n+\t}\n+\n \tif (cb->prune && !do_not_prune(flags)) {\n \t\tint namelen = strlen(path) + 1;\n \t\tstruct ref_to_prune *n = xcalloc(1, sizeof(*n) + namelen);\ndiff --git a/builtin-show-ref.c b/builtin-show-ref.c\nindex 06ec400..32cbcb8 100644\n--- a/builtin-show-ref.c\n+++ b/builtin-show-ref.c\n@@ -13,6 +13,7 @@ static int show_ref(const char *refname,\n {\n \tstruct object *obj;\n \tconst char *hex;\n+\tunsigned char peeled[20];\n \n \tif (tags_only || heads_only) {\n \t\tint match;\n@@ -44,6 +45,17 @@ static int show_ref(const char *refname,\n \n match:\n \tfound_match++;\n+\n+\t/* Fastpath for demonstration only */\n+\tif (deref_tags && (flag & REF_ISPACKED) &&\n+\t !peel_ref(refname, peeled) && !quiet && !hash_only) {\n+\t\thex = find_unique_abbrev(sha1, abbrev);\n+\t\tprintf(\"%s %s\\n\", hex, refname);\n+\t\thex = find_unique_abbrev(peeled, abbrev);\n+\t\tprintf(\"%s %s^{}\\n\", hex, refname);\n+\t\treturn 0;\n+\t}\n+\n \tobj = parse_object(sha1);\n \tif (!obj) {\n \t\tif (quiet)\n@@ -59,8 +71,13 @@ match:\n \telse\n \t\tprintf(\"%s %s\\n\", hex, refname);\n \tif (deref_tags && obj->type == OBJ_TAG) {\n-\t\tobj = deref_tag(obj, refname, 0);\n-\t\thex = find_unique_abbrev(obj->sha1, abbrev);\n+\t\tunsigned char sha1[20];\n+\t\tif ((flag & REF_ISPACKED) && !peel_ref(refname, sha1))\n+\t\t\thex = find_unique_abbrev(sha1, abbrev);\n+\t\telse {\n+\t\t\tobj = deref_tag(obj, refname, 0);\n+\t\t\thex = find_unique_abbrev(obj->sha1, abbrev);\n+\t\t}\n \t\tprintf(\"%s %s^{}\\n\", hex, refname);\n \t}\n \treturn 0;\ndiff --git a/refs.c b/refs.c\nindex f003a0b..347c9b7 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1,16 +1,18 @@\n #include \"refs.h\"\n #include \"cache.h\"\n+#include \"object.h\"\n+#include \"tag.h\"\n \n #include \n \n struct ref_list {\n \tstruct ref_list *next;\n-\tunsigned char flag; /* ISSYMREF? ISPACKED? */\n+\tunsigned char flag; /* ISSYMREF? ISPACKED? ISPEELED? */\n \tunsigned char sha1[20];\n \tchar name[FLEX_ARRAY];\n };\n \n-static const char *parse_ref_line(char *line, unsigned char *sha1)\n+static const char *parse_ref_line(char *line, unsigned char *sha1, int *flag)\n {\n \t/*\n \t * 42: the answer to everything.\n@@ -34,6 +36,10 @@ static const char *parse_ref_line(char *\n \tif (line[len] != '\\n')\n \t\treturn NULL;\n \tline[len] = 0;\n+\tif (len < 3 || strcmp(line + len - 3, \"^{}\"))\n+\t *flag &= ~REF_ISPEELED;\n+\telse\n+\t *flag |= REF_ISPEELED;\n \treturn line;\n }\n \n@@ -108,10 +114,12 @@ static struct ref_list *get_packed_refs(\n \t\t\tchar refline[PATH_MAX];\n \t\t\twhile (fgets(refline, sizeof(refline), f)) {\n \t\t\t\tunsigned char sha1[20];\n-\t\t\t\tconst char *name = parse_ref_line(refline, sha1);\n+\t\t\t\tint flag = REF_ISPACKED;\n+\t\t\t\tconst char *name =\n+\t\t\t\t\tparse_ref_line(refline, sha1, &flag);\n \t\t\t\tif (!name)\n \t\t\t\t\tcontinue;\n-\t\t\t\tlist = add_ref(name, sha1, REF_ISPACKED, list);\n+\t\t\t\tlist = add_ref(name, sha1, flag, list);\n \t\t\t}\n \t\t\tfclose(f);\n \t\t\trefs = list;\n@@ -207,7 +215,8 @@ const char *resolve_ref(const char *ref,\n \t\tif (lstat(path, &st) < 0) {\n \t\t\tstruct ref_list *list = get_packed_refs();\n \t\t\twhile (list) {\n-\t\t\t\tif (!strcmp(ref, list->name)) {\n+\t\t\t\tif (!(list->flag & REF_ISPEELED) &&\n+\t\t\t\t !strcmp(ref, list->name)) {\n \t\t\t\t\thashcpy(sha1, list->sha1);\n \t\t\t\t\tif (flag)\n \t\t\t\t\t\t*flag |= REF_ISPACKED;\n@@ -322,6 +331,41 @@ int read_ref(const char *ref, unsigned c\n \treturn -1;\n }\n \n+int peel_ref(const char *ref, unsigned char *sha1)\n+{\n+\tint flag;\n+\tunsigned char base[20];\n+\tstruct object *o;\n+\n+\tif (!resolve_ref(ref, base, 1, &flag))\n+\t\treturn -1;\n+\n+\tif ((flag & REF_ISPACKED)) {\n+\t\tstruct ref_list *list = get_packed_refs();\t\t\n+\t\tint len = strlen(ref);\n+\t\twhile (list) {\n+\t\t\tif ((list->flag & REF_ISPEELED) &&\n+\t\t\t !strncmp(list->name, ref, len) &&\n+\t\t\t strlen(list->name) == len + 3 &&\n+\t\t\t !strcmp(list->name + len, \"^{}\")) {\n+\t\t\t\thashcpy(sha1, list->sha1);\n+\t\t\t\treturn 0;\n+\t\t\t}\n+\t\t\tlist = list->next;\n+\t\t}\n+\t\t/* older pack-refs did not leave peeled ones in */\n+\t}\n+\n+\t/* otherwise ... */\n+\to = parse_object(sha1);\n+\to = deref_tag(o, ref, 0);\n+\tif (o) {\n+\t\thashcpy(sha1, o->sha1);\n+\t\treturn 0;\n+\t}\n+\treturn -1;\n+}\n+\n static int do_for_each_ref(const char *base, each_ref_fn fn, int trim,\n \t\t\t void *cb_data)\n {\n@@ -347,6 +391,8 @@ static int do_for_each_ref(const char *b\n \t\t\tcontinue;\n \t\tif (is_null_sha1(entry->sha1))\n \t\t\tcontinue;\n+\t\tif (entry->flag & REF_ISPEELED)\n+\t\t\tcontinue;\n \t\tif (!has_sha1_file(entry->sha1)) {\n \t\t\terror(\"%s does not point to a valid object!\", entry->name);\n \t\t\tcontinue;\n@@ -357,15 +403,22 @@ static int do_for_each_ref(const char *b\n \t\t\treturn retval;\n \t}\n \n-\tpacked = packed ? packed : loose;\n-\twhile (packed) {\n-\t\tif (!strncmp(base, packed->name, trim)) {\n-\t\t\tretval = fn(packed->name + trim, packed->sha1,\n-\t\t\t\t packed->flag, cb_data);\n-\t\t\tif (retval)\n-\t\t\t\treturn retval;\n+\tfor (packed = packed ? packed : loose; packed; packed = packed->next) {\n+\t\tif (strncmp(base, packed->name, trim))\n+\t\t\tcontinue;\n+\t\tif (is_null_sha1(packed->sha1))\n+\t\t\tcontinue;\n+\t\tif (packed->flag & REF_ISPEELED)\n+\t\t\tcontinue;\n+\t\tif (!has_sha1_file(packed->sha1)) {\n+\t\t\terror(\"%s does not point to a valid object!\",\n+\t\t\t packed->name);\n+\t\t\tcontinue;\n \t\t}\n-\t\tpacked = packed->next;\n+\t\tretval = fn(packed->name + trim, packed->sha1,\n+\t\t\t packed->flag, cb_data);\n+\t\tif (retval)\n+\t\t\treturn retval;\n \t}\n \treturn 0;\n }\ndiff --git a/refs.h b/refs.h\nindex a57d437..40048a6 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -16,6 +16,8 @@ struct ref_lock {\n */\n #define REF_ISSYMREF 01\n #define REF_ISPACKED 02\n+#define REF_ISPEELED 04 /* internal use */\n+\n typedef int each_ref_fn(const char *refname, const unsigned char *sha1, int flags, void *cb_data);\n extern int head_ref(each_ref_fn, void *);\n extern int for_each_ref(each_ref_fn, void *);\n@@ -23,6 +25,8 @@ extern int for_each_tag_ref(each_ref_fn,\n extern int for_each_branch_ref(each_ref_fn, void *);\n extern int for_each_remote_ref(each_ref_fn, void *);\n \n+extern int peel_ref(const char *, unsigned char *);\n+\n /** Reads the refs file specified into sha1 **/\n extern int get_ref_sha1(const char *ref, unsigned char *sha1);\n \n"},{"id":"298469","messageId":"7vac2o9pln.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"7vfycg9pu5.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-19T01:45:08Z","receivedAt":"2006-11-19T01:45:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Linus Torvalds writes:\n>\n>> On Sun, 19 Nov 2006, Marco Costalba wrote:\n>>> \n>>> Tested also with show-ref only, not useful to me, but just as a comparison.\n>>> \n>>> - git tree 1420ms\n>>> - linux tree 1021ms\n>>> \n>>> Better, but still slower then what I would expected.\n>>\n>> MUCH slower than expected.\n\nThere is something seriously wrong about Marco's number with or\nwithout my patch. I am getting something like this from\nlinux-2.6 with fully packed and pruned refs;\n\n$ /usr/bin/time git-show-ref -d >/dev/null\n0.08user 0.00system 0:00.08elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+3314minor)pagefaults 0swaps\n\n$ /usr/bin/time ../git.junio/git-show-ref -d >/dev/null\n0.05user 0.01system 0:00.06elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+2724minor)pagefaults 0swaps\n\nThe second one is with the patch, the first one is from\nyesterday's master.\n"},{"id":"297055","messageId":"Pine.LNX.4.64.0611181758290.3692@woody.osdl.org","threadId":"43080","inReplyTo":"7vac2o9pln.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T01:59:08Z","receivedAt":"2006-11-19T01:59:08Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 18 Nov 2006, Junio C Hamano wrote:\n> \n> There is something seriously wrong about Marco's number with or\n> without my patch. I am getting something like this from\n> linux-2.6 with fully packed and pruned refs;\n\nWell, Marco is testing cold-cache numbers. But I agree, even with \ncold-cache, it shouldn't be anywhere close to that bad. \n\n"},{"id":"294937","messageId":"e5bfff550611190140n3277ee26v95feba26dd3348fa@mail.gmail.com","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611181706250.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-19T09:40:10Z","receivedAt":"2006-11-19T09:40:10Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":">\n> Can you do\n>\n> strace -o tracefile -Ttt git show-ref\n>\n> and send out the tracefile?\n>\n\nSure. File ran against git tree attached.\n\n> What I _suspect_ is going on is that when you packed your refs, you either\n> didn't prune them (which means that packing didn't actually help you), or\n> if you did prune them, since we don't seem to remove the refs\n> _directories_ when packing, you still have an old and big directory for\n> .git/refs/tags, and just reading that (empty, but non-shrunken) directory\n> takes time.\n\nIndeed it is not empty, it has 4 refs, all the remaining are in packed-refs:\n\n$ ls .git/refs/tags\nv1.4.3.4 v1.4.3.5 v1.4.4 v1.4.4-rc1 v1.4.4-rc2\n\n$ ls -l .git/packed-refs\n-rw-r--r-- 1 marco marco 6469 nov 3 21:03 .git/packed-refs\n\n$ cat .git/packed-refs |wc\n 109 218 6469\n\nIf you want I can repack and prune, but for now I just wait to avoid\nto corrupt this test case.\n\nIMHO if just 4 entry in a directory out of 109 could slow down in this\nway, at least should be documented/warned in some place. Also because\nthis it seems to me, perhaps wrongly, a quite common scenario of an\nuser that repacks the tags seldom.\n\n\n Marco\n\n\n10:19:02.935227 execve(\"/home/marco/bin/git\", [\"git\", \"show-ref\"], [/* 75 vars */]) = 0 <0.045394>\n10:19:02.981333 brk(0) = 0x80de000 <0.000011>\n10:19:02.981498 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fcb000 <0.000014>\n10:19:02.981618 access(\"/etc/ld.so.preload\", R_OK) = -1 ENOENT (No such file or directory) <0.000015>\n10:19:02.981760 open(\"/etc/ld.so.cache\", O_RDONLY) = 3 <0.000020>\n10:19:02.981849 fstat64(3, {st_mode=S_IFREG|0644, st_size=79652, ...}) = 0 <0.000011>\n10:19:02.981975 mmap2(NULL, 79652, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fb7000 <0.000016>\n10:19:02.982056 close(3) = 0 <0.000011>\n10:19:02.982166 open(\"/lib/libz.so.1\", O_RDONLY) = 3 <0.000019>\n10:19:02.982253 read(3, \"\\177ELF\\1\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0\\3\\0\\1\\0\\0\\0\\20\\27\\0\"..., 512) = 512 <0.000015>\n10:19:02.982368 fstat64(3, {st_mode=S_IFREG|0755, st_size=73156, ...}) = 0 <0.000011>\n10:19:02.982474 mmap2(NULL, 75980, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7fa4000 <0.000014>\n10:19:02.982558 mmap2(0xb7fb6000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x11) = 0xb7fb6000 <0.000025>\n10:19:02.982666 close(3) = 0 <0.000010>\n10:19:02.982762 open(\"/usr/lib/libcrypto.so.0.9.8\", O_RDONLY) = 3 <0.000017>\n10:19:02.982851 read(3, \"\\177ELF\\1\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0\\3\\0\\1\\0\\0\\0@U\\3\\000\"..., 512) = 512 <0.000014>\n10:19:02.982958 fstat64(3, {st_mode=S_IFREG|0755, st_size=1353744, ...}) = 0 <0.000011>\n10:19:02.983062 mmap2(NULL, 1370456, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7e55000 <0.000014>\n10:19:02.983146 mmap2(0xb7f8c000, 86016, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x136) = 0xb7f8c000 <0.000020>\n10:19:02.983254 mmap2(0xb7fa1000, 10584, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fa1000 <0.000015>\n10:19:02.983354 close(3) = 0 <0.000011>\n10:19:02.983441 open(\"/lib/i686/libc.so.6\", O_RDONLY) = 3 <0.000019>\n10:19:02.983530 read(3, \"\\177ELF\\1\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0\\3\\0\\1\\0\\0\\0\\240X\\1\"..., 512) = 512 <0.000013>\n10:19:02.983635 fstat64(3, {st_mode=S_IFREG|0644, st_size=1220244, ...}) = 0 <0.000011>\n10:19:02.983745 mmap2(NULL, 1230204, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7d28000 <0.000014>\n10:19:02.983828 mmap2(0xb7e4f000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x126) = 0xb7e4f000 <0.000022>\n10:19:02.983936 mmap2(0xb7e52000, 9596, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7e52000 <0.000015>\n10:19:02.984161 close(3) = 0 <0.000012>\n10:19:02.984261 open(\"/lib/libdl.so.2\", O_RDONLY) = 3 <0.000020>\n10:19:02.984350 read(3, \"\\177ELF\\1\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0\\3\\0\\1\\0\\0\\0p\\f\\0\\000\"..., 512) = 512 <0.000014>\n10:19:02.984457 fstat64(3, {st_mode=S_IFREG|0755, st_size=9700, ...}) = 0 <0.000011>\n10:19:02.984562 mmap2(NULL, 12412, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7d24000 <0.000016>\n10:19:02.984647 mmap2(0xb7d26000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb7d26000 <0.000020>\n10:19:02.984766 close(3) = 0 <0.000011>\n10:19:02.984883 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7d23000 <0.000016>\n10:19:02.984995 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7d22000 <0.000012>\n10:19:02.985086 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7d226c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 <0.000011>\n10:19:02.985642 mprotect(0xb7e4f000, 4096, PROT_READ) = 0 <0.000019>\n10:19:02.986216 mprotect(0xb7fe5000, 4096, PROT_READ) = 0 <0.000021>\n10:19:02.986324 munmap(0xb7fb7000, 79652) = 0 <0.000027>\n10:19:02.988091 brk(0) = 0x80de000 <0.000012>\n10:19:02.988180 brk(0x80ff000) = 0x80ff000 <0.000015>\n10:19:02.988343 getcwd(\"/home/marco/programmi/git\", 4097) = 26 <0.000025>\n10:19:02.988456 access(\".git/refs/\", X_OK) = 0 <0.016158>\n10:19:03.004751 access(\".git/objects/\", X_OK) = 0 <0.008759>\n10:19:03.013670 lstat64(\".git/HEAD\", {st_mode=S_IFREG|0644, st_size=23, ...}) = 0 <0.000023>\n10:19:03.013812 open(\".git/HEAD\", O_RDONLY) = 3 <0.000021>\n10:19:03.013910 read(3, \"ref: refs/heads/origin\\n\", 255) = 23 <0.008989>\n10:19:03.023027 close(3) = 0 <0.000015>\n10:19:03.023257 access(\"/home/marco/.gitconfig\", R_OK) = -1 ENOENT (No such file or directory) <0.000030>\n10:19:03.023401 open(\".git/config\", O_RDONLY) = 3 <0.000027>\n10:19:03.023527 fstat64(3, {st_mode=S_IFREG|0664, st_size=84, ...}) = 0 <0.000012>\n10:19:03.023639 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fca000 <0.000019>\n10:19:03.023730 read(3, \"[core]\\n\\trepositoryformatversion \"..., 4096) = 84 <0.004629>\n10:19:03.028488 read(3, \"\", 4096) = 0 <0.000012>\n10:19:03.028583 close(3) = 0 <0.000014>\n10:19:03.028659 munmap(0xb7fca000, 4096) = 0 <0.000023>\n10:19:03.029585 open(\".git/packed-refs\", O_RDONLY) = 3 <0.000031>\n10:19:03.029711 fstat64(3, {st_mode=S_IFREG|0644, st_size=6469, ...}) = 0 <0.000011>\n10:19:03.029818 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fca000 <0.000015>\n10:19:03.029903 read(3, \"dcc22ee22803fe091761047d69d92529\"..., 4096) = 4096 <0.013330>\n10:19:03.043674 read(3, \"f416f8f2bc74d6d0a refs/tags/v1.1\"..., 4096) = 2373 <0.003832>\n10:19:03.047952 read(3, \"\", 4096) = 0 <0.000012>\n10:19:03.048142 close(3) = 0 <0.000013>\n10:19:03.048221 munmap(0xb7fca000, 4096) = 0 <0.000022>\n10:19:03.048343 open(\".git/refs\", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3 <0.000018>\n10:19:03.048436 fstat64(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000011>\n10:19:03.048541 fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 <0.000012>\n10:19:03.048641 getdents(3, /* 4 entries */, 4096) = 68 <0.011642>\n10:19:03.060486 stat64(\".git/refs/heads\", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000034>\n10:19:03.060658 open(\".git/refs/heads\", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 4 <0.000021>\n10:19:03.060752 fstat64(4, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000011>\n10:19:03.060854 fcntl64(4, F_SETFD, FD_CLOEXEC) = 0 <0.000012>\n10:19:03.060946 getdents(4, /* 10 entries */, 4096) = 172 <0.000179>\n10:19:03.061242 stat64(\".git/refs/heads/pu\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.006570>\n10:19:03.067925 lstat64(\".git/refs/heads/pu\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.068152 open(\".git/refs/heads/pu\", O_RDONLY) = 5 <0.000019>\n10:19:03.068248 read(5, \"69e98b538aeb47f1a5602b00f1972bfd\"..., 255) = 41 <0.007676>\n10:19:03.076003 close(5) = 0 <0.000541>\n10:19:03.076680 stat64(\".git/refs/heads/man\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000030>\n10:19:03.076831 lstat64(\".git/refs/heads/man\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.076944 open(\".git/refs/heads/man\", O_RDONLY) = 5 <0.000019>\n10:19:03.077031 read(5, \"d0f877fb601ac0076fd69ec4f48f71f9\"..., 255) = 41 <0.009018>\n10:19:03.086137 close(5) = 0 <0.000013>\n10:19:03.086232 stat64(\".git/refs/heads/todo\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000025>\n10:19:03.086361 lstat64(\".git/refs/heads/todo\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.086475 open(\".git/refs/heads/todo\", O_RDONLY) = 5 <0.000017>\n10:19:03.086562 read(5, \"16a4568a8292605cb688d0e26c65aea8\"..., 255) = 41 <0.008006>\n10:19:03.094684 close(5) = 0 <0.000013>\n10:19:03.094782 stat64(\".git/refs/heads/maint\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000028>\n10:19:03.094916 lstat64(\".git/refs/heads/maint\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000016>\n10:19:03.095030 open(\".git/refs/heads/maint\", O_RDONLY) = 5 <0.000017>\n10:19:03.095118 read(5, \"fe142b3a4577a6692a39e2386ed64966\"..., 255) = 41 <0.006822>\n10:19:03.102017 close(5) = 0 <0.000013>\n10:19:03.102103 stat64(\".git/refs/heads/next\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000031>\n10:19:03.102236 lstat64(\".git/refs/heads/next\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.102382 open(\".git/refs/heads/next\", O_RDONLY) = 5 <0.000017>\n10:19:03.102471 read(5, \"8f89a06b2529346f2b05bd44bd6a61a7\"..., 255) = 41 <0.008473>\n10:19:03.111052 close(5) = 0 <0.000013>\n10:19:03.111150 stat64(\".git/refs/heads/v1.4.0\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000029>\n10:19:03.111284 lstat64(\".git/refs/heads/v1.4.0\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.111398 open(\".git/refs/heads/v1.4.0\", O_RDONLY) = 5 <0.000018>\n10:19:03.111485 read(5, \"41292ddd37202ff6dce34986c87a6000\"..., 255) = 41 <0.008847>\n10:19:03.120410 close(5) = 0 <0.000012>\n10:19:03.120495 stat64(\".git/refs/heads/html\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000023>\n10:19:03.120621 lstat64(\".git/refs/heads/html\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.120734 open(\".git/refs/heads/html\", O_RDONLY) = 5 <0.000017>\n10:19:03.120821 read(5, \"fb2ead308920336d55720031034442c1\"..., 255) = 41 <0.015274>\n10:19:03.136258 close(5) = 0 <0.000016>\n10:19:03.136383 stat64(\".git/refs/heads/origin\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000033>\n10:19:03.136524 lstat64(\".git/refs/heads/origin\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.136638 open(\".git/refs/heads/origin\", O_RDONLY) = 5 <0.000020>\n10:19:03.136728 read(5, \"e267c2f6f0784e242883b7d3fe5f36ef\"..., 255) = 41 <0.000725>\n10:19:03.137529 close(5) = 0 <0.000013>\n10:19:03.137611 getdents(4, /* 0 entries */, 4096) = 0 <0.000013>\n10:19:03.137702 close(4) = 0 <0.000012>\n10:19:03.137781 stat64(\".git/refs/tags\", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000025>\n10:19:03.137909 open(\".git/refs/tags\", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 4 <0.000016>\n10:19:03.137997 fstat64(4, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000012>\n10:19:03.138098 fcntl64(4, F_SETFD, FD_CLOEXEC) = 0 <0.000013>\n10:19:03.138177 getdents(4, /* 7 entries */, 4096) = 140 <0.010973>\n10:19:03.149276 stat64(\".git/refs/tags/v1.4.3.4\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.004994>\n10:19:03.154448 lstat64(\".git/refs/tags/v1.4.3.4\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000017>\n10:19:03.154585 open(\".git/refs/tags/v1.4.3.4\", O_RDONLY) = 5 <0.000022>\n10:19:03.154677 read(5, \"100490cfcd50d0f9ddadeb90cf5d95b2\"..., 255) = 41 <0.005124>\n10:19:03.159877 close(5) = 0 <0.000015>\n10:19:03.159966 stat64(\".git/refs/tags/v1.4.4-rc1\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.006757>\n10:19:03.166861 lstat64(\".git/refs/tags/v1.4.4-rc1\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000017>\n10:19:03.166995 open(\".git/refs/tags/v1.4.4-rc1\", O_RDONLY) = 5 <0.000019>\n10:19:03.167085 read(5, \"3a41a48d139d1425c1d27e3fbe4f511f\"..., 255) = 41 <0.005877>\n10:19:03.173113 close(5) = 0 <0.000015>\n10:19:03.173236 stat64(\".git/refs/tags/v1.4.3.5\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.006382>\n10:19:03.179730 lstat64(\".git/refs/tags/v1.4.3.5\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.179846 open(\".git/refs/tags/v1.4.3.5\", O_RDONLY) = 5 <0.000019>\n10:19:03.179936 read(5, \"a8d8dc4c6cd6d4681746fd6919318640\"..., 255) = 41 <0.007536>\n10:19:03.187558 close(5) = 0 <0.000013>\n10:19:03.187655 stat64(\".git/refs/tags/v1.4.4-rc2\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000026>\n10:19:03.187785 lstat64(\".git/refs/tags/v1.4.4-rc2\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.187899 open(\".git/refs/tags/v1.4.4-rc2\", O_RDONLY) = 5 <0.000018>\n10:19:03.187988 read(5, \"b9b0b3d0020c3559d4d9e6d07e3316c3\"..., 255) = 41 <0.010095>\n10:19:03.198218 close(5) = 0 <0.000015>\n10:19:03.198330 stat64(\".git/refs/tags/v1.4.4\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.003882>\n10:19:03.202322 lstat64(\".git/refs/tags/v1.4.4\", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0 <0.000015>\n10:19:03.202437 open(\".git/refs/tags/v1.4.4\", O_RDONLY) = 5 <0.000019>\n10:19:03.202527 read(5, \"fab4f17015ccb2c6ea978b45386d45dd\"..., 255) = 41 <0.014246>\n10:19:03.216949 close(5) = 0 <0.000014>\n10:19:03.217063 getdents(4, /* 0 entries */, 4096) = 0 <0.000013>\n10:19:03.217140 close(4) = 0 <0.000012>\n10:19:03.217213 getdents(3, /* 0 entries */, 4096) = 0 <0.000011>\n10:19:03.217287 close(3) = 0 <0.000011>\n10:19:03.217397 open(\".git/objects/pack\", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3 <0.015266>\n10:19:03.232796 fstat64(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000011>\n10:19:03.232928 fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 <0.000012>\n10:19:03.233009 getdents(3, /* 4 entries */, 4096) = 160 <0.000216>\n10:19:03.233356 open(\".git/objects/pack/pack-a587adeec6a06edc539cc73c0a526e93b2982022.idx\", O_RDONLY) = 4 <0.000032>\n10:19:03.233486 fstat64(4, {st_mode=S_IFREG|0444, st_size=431864, ...}) = 0 <0.000011>\n10:19:03.233598 mmap2(NULL, 431864, PROT_READ, MAP_PRIVATE, 4, 0) = 0xb7cb8000 <0.000022>\n10:19:03.233687 close(4) = 0 <0.000012>\n10:19:03.252324 stat64(\".git/objects/pack/pack-a587adeec6a06edc539cc73c0a526e93b2982022.pack\", {st_mode=S_IFREG|0444, st_size=7118252, ...}) = 0 <0.000057>\n10:19:03.252570 getdents(3, /* 0 entries */, 4096) = 0 <0.000013>\n10:19:03.252652 close(3) = 0 <0.000014>\n10:19:03.252741 open(\".git/objects/info/alternates\", O_RDONLY) = -1 ENOENT (No such file or directory) <0.006948>\n10:19:03.271644 stat64(\".git/objects/fb/2ead308920336d55720031034442c1ff3a2088\", {st_mode=S_IFREG|0444, st_size=193, ...}) = 0 <0.029404>\n10:19:03.301358 stat64(\".git/objects/fb/2ead308920336d55720031034442c1ff3a2088\", {st_mode=S_IFREG|0444, st_size=193, ...}) = 0 <0.000018>\n10:19:03.301514 open(\".git/objects/fb/2ead308920336d55720031034442c1ff3a2088\", O_RDONLY|O_NOATIME) = 3 <0.000023>\n10:19:03.301620 mmap2(NULL, 193, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fca000 <0.000019>\n10:19:03.301706 close(3) = 0 <0.000012>\n10:19:03.310337 munmap(0xb7fca000, 193) = 0 <0.000030>\n10:19:03.310740 open(\".git/info/grafts\", O_RDONLY) = -1 ENOENT (No such file or directory) <0.008295>\n10:19:03.319252 fstat64(1, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 <0.000013>\n10:19:03.319386 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fca000 <0.000018>\n10:19:03.319500 stat64(\".git/objects/fe/142b3a4577a6692a39e2386ed649664ad8bd20\", {st_mode=S_IFREG|0444, st_size=316, ...}) = 0 <0.010379>\n10:19:03.330019 stat64(\".git/objects/fe/142b3a4577a6692a39e2386ed649664ad8bd20\", {st_mode=S_IFREG|0444, st_size=316, ...}) = 0 <0.000016>\n10:19:03.330156 open(\".git/objects/fe/142b3a4577a6692a39e2386ed649664ad8bd20\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:03.330260 mmap2(NULL, 316, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:03.330344 close(3) = 0 <0.000011>\n10:19:03.342077 munmap(0xb7fc9000, 316) = 0 <0.000058>\n10:19:03.355754 stat64(\".git/objects/d0/f877fb601ac0076fd69ec4f48f71f9247771c8\", {st_mode=S_IFREG|0444, st_size=190, ...}) = 0 <0.046244>\n10:19:03.402234 stat64(\".git/objects/d0/f877fb601ac0076fd69ec4f48f71f9247771c8\", {st_mode=S_IFREG|0444, st_size=190, ...}) = 0 <0.000019>\n10:19:03.402392 open(\".git/objects/d0/f877fb601ac0076fd69ec4f48f71f9247771c8\", O_RDONLY|O_NOATIME) = 3 <0.000023>\n10:19:03.402498 mmap2(NULL, 190, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000021>\n10:19:03.402586 close(3) = 0 <0.000012>\n10:19:03.408581 munmap(0xb7fc9000, 190) = 0 <0.000022>\n10:19:03.427950 stat64(\".git/objects/8f/89a06b2529346f2b05bd44bd6a61a76898c639\", {st_mode=S_IFREG|0444, st_size=262, ...}) = 0 <0.018769>\n10:19:03.446963 stat64(\".git/objects/8f/89a06b2529346f2b05bd44bd6a61a76898c639\", {st_mode=S_IFREG|0444, st_size=262, ...}) = 0 <0.000017>\n10:19:03.447109 open(\".git/objects/8f/89a06b2529346f2b05bd44bd6a61a76898c639\", O_RDONLY|O_NOATIME) = 3 <0.000018>\n10:19:03.447209 mmap2(NULL, 262, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:03.447297 close(3) = 0 <0.000010>\n10:19:03.455682 munmap(0xb7fc9000, 262) = 0 <0.000023>\n10:19:03.455835 stat64(\".git/objects/e2/67c2f6f0784e242883b7d3fe5f36ef63d6950d\", {st_mode=S_IFREG|0444, st_size=179, ...}) = 0 <0.033849>\n10:19:03.489904 stat64(\".git/objects/e2/67c2f6f0784e242883b7d3fe5f36ef63d6950d\", {st_mode=S_IFREG|0444, st_size=179, ...}) = 0 <0.000018>\n10:19:03.490059 open(\".git/objects/e2/67c2f6f0784e242883b7d3fe5f36ef63d6950d\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:03.490162 mmap2(NULL, 179, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.490249 close(3) = 0 <0.000011>\n10:19:03.504780 munmap(0xb7fc9000, 179) = 0 <0.000023>\n10:19:03.504931 stat64(\".git/objects/69/e98b538aeb47f1a5602b00f1972bfd05ac27cc\", {st_mode=S_IFREG|0444, st_size=517, ...}) = 0 <0.026898>\n10:19:03.531966 stat64(\".git/objects/69/e98b538aeb47f1a5602b00f1972bfd05ac27cc\", {st_mode=S_IFREG|0444, st_size=517, ...}) = 0 <0.000016>\n10:19:03.532337 open(\".git/objects/69/e98b538aeb47f1a5602b00f1972bfd05ac27cc\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:03.532458 mmap2(NULL, 517, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.532542 close(3) = 0 <0.000012>\n10:19:03.542325 munmap(0xb7fc9000, 517) = 0 <0.000022>\n10:19:03.542485 stat64(\".git/objects/16/a4568a8292605cb688d0e26c65aea8850f22d4\", {st_mode=S_IFREG|0444, st_size=184, ...}) = 0 <0.015860>\n10:19:03.558508 stat64(\".git/objects/16/a4568a8292605cb688d0e26c65aea8850f22d4\", {st_mode=S_IFREG|0444, st_size=184, ...}) = 0 <0.000017>\n10:19:03.558654 open(\".git/objects/16/a4568a8292605cb688d0e26c65aea8850f22d4\", O_RDONLY|O_NOATIME) = 3 <0.000019>\n10:19:03.558755 mmap2(NULL, 184, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:03.558839 close(3) = 0 <0.000012>\n10:19:03.566339 munmap(0xb7fc9000, 184) = 0 <0.000024>\n10:19:03.566511 stat64(\".git/objects/41/292ddd37202ff6dce34986c87a6000c5d3fbfa\", {st_mode=S_IFREG|0444, st_size=180, ...}) = 0 <0.020396>\n10:19:03.587058 stat64(\".git/objects/41/292ddd37202ff6dce34986c87a6000c5d3fbfa\", {st_mode=S_IFREG|0444, st_size=180, ...}) = 0 <0.000016>\n10:19:03.587197 open(\".git/objects/41/292ddd37202ff6dce34986c87a6000c5d3fbfa\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:03.587299 mmap2(NULL, 180, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.587383 close(3) = 0 <0.000012>\n10:19:03.600927 munmap(0xb7fc9000, 180) = 0 <0.000022>\n10:19:03.601072 open(\".git/objects/pack/pack-a587adeec6a06edc539cc73c0a526e93b2982022.pack\", O_RDONLY) = 3 <0.000022>\n10:19:03.601180 fstat64(3, {st_mode=S_IFREG|0444, st_size=7118252, ...}) = 0 <0.000011>\n10:19:03.601284 mmap2(NULL, 7118252, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb75ee000 <0.000018>\n10:19:03.601365 close(3) = 0 <0.000011>\n10:19:03.645053 brk(0x8124000) = 0x8124000 <0.000019>\n10:19:03.662737 write(1, \"fb2ead308920336d55720031034442c1\"..., 4096) = 4096 <0.000054>\n10:19:03.663479 stat64(\".git/objects/b0/517166ae2ad92f3b17638cbdee0f04b8170d99\", {st_mode=S_IFREG|0444, st_size=277, ...}) = 0 <0.022552>\n10:19:03.686177 stat64(\".git/objects/b0/517166ae2ad92f3b17638cbdee0f04b8170d99\", {st_mode=S_IFREG|0444, st_size=277, ...}) = 0 <0.000017>\n10:19:03.686315 open(\".git/objects/b0/517166ae2ad92f3b17638cbdee0f04b8170d99\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:03.686418 mmap2(NULL, 277, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000021>\n10:19:03.686504 close(3) = 0 <0.000012>\n10:19:03.690535 munmap(0xb7fc9000, 277) = 0 <0.000026>\n10:19:03.690897 stat64(\".git/objects/86/a9bf69164090c91474bb7bcdde7d76966b7d40\", {st_mode=S_IFREG|0444, st_size=282, ...}) = 0 <0.015918>\n10:19:03.706991 stat64(\".git/objects/86/a9bf69164090c91474bb7bcdde7d76966b7d40\", {st_mode=S_IFREG|0444, st_size=282, ...}) = 0 <0.000017>\n10:19:03.707140 open(\".git/objects/86/a9bf69164090c91474bb7bcdde7d76966b7d40\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:03.707243 mmap2(NULL, 282, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:03.707328 close(3) = 0 <0.000012>\n10:19:03.721897 munmap(0xb7fc9000, 282) = 0 <0.000026>\n10:19:03.722067 stat64(\".git/objects/fa/7bd724f61bcdcd71ef579f7b94b72bda1298cf\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.007090>\n10:19:03.729292 stat64(\".git/objects/fa/7bd724f61bcdcd71ef579f7b94b72bda1298cf\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.000017>\n10:19:03.729456 open(\".git/objects/fa/7bd724f61bcdcd71ef579f7b94b72bda1298cf\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:03.729560 mmap2(NULL, 279, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.729645 close(3) = 0 <0.000012>\n10:19:03.730857 munmap(0xb7fc9000, 279) = 0 <0.000019>\n10:19:03.730979 stat64(\".git/objects/43/2567e79be81bc91f4af5bcfe5ac51330d2d349\", {st_mode=S_IFREG|0444, st_size=281, ...}) = 0 <0.035298>\n10:19:03.766440 stat64(\".git/objects/43/2567e79be81bc91f4af5bcfe5ac51330d2d349\", {st_mode=S_IFREG|0444, st_size=281, ...}) = 0 <0.000018>\n10:19:03.766586 open(\".git/objects/43/2567e79be81bc91f4af5bcfe5ac51330d2d349\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:03.766687 mmap2(NULL, 281, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.766771 close(3) = 0 <0.000012>\n10:19:03.780679 munmap(0xb7fc9000, 281) = 0 <0.000023>\n10:19:03.780829 stat64(\".git/objects/ff/a22c1601a782de4f54c7667d772936da123ef0\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.020505>\n10:19:03.801496 stat64(\".git/objects/ff/a22c1601a782de4f54c7667d772936da123ef0\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.000017>\n10:19:03.801642 open(\".git/objects/ff/a22c1601a782de4f54c7667d772936da123ef0\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:03.801744 mmap2(NULL, 279, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:03.801829 close(3) = 0 <0.000011>\n10:19:03.811789 munmap(0xb7fc9000, 279) = 0 <0.000022>\n10:19:03.811920 stat64(\".git/objects/12/0530fb2e10679d9095e278c95c0e98c96f3080\", {st_mode=S_IFREG|0444, st_size=277, ...}) = 0 <0.008756>\n10:19:03.820801 stat64(\".git/objects/12/0530fb2e10679d9095e278c95c0e98c96f3080\", {st_mode=S_IFREG|0444, st_size=277, ...}) = 0 <0.000015>\n10:19:03.820927 open(\".git/objects/12/0530fb2e10679d9095e278c95c0e98c96f3080\", O_RDONLY|O_NOATIME) = 3 <0.000019>\n10:19:03.821027 mmap2(NULL, 277, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000017>\n10:19:03.821109 close(3) = 0 <0.000011>\n10:19:03.838061 munmap(0xb7fc9000, 277) = 0 <0.000029>\n10:19:03.838263 stat64(\".git/objects/92/d7605b6d9312a29efa30b1baa4d7c880a0be41\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.032662>\n10:19:03.871093 stat64(\".git/objects/92/d7605b6d9312a29efa30b1baa4d7c880a0be41\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.000017>\n10:19:03.871238 open(\".git/objects/92/d7605b6d9312a29efa30b1baa4d7c880a0be41\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:03.871339 mmap2(NULL, 280, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000021>\n10:19:03.871426 close(3) = 0 <0.000012>\n10:19:03.885472 munmap(0xb7fc9000, 280) = 0 <0.000046>\n10:19:03.885707 stat64(\".git/objects/58/9ee4d580d34a76238e90795a070a7372de18a2\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.021423>\n10:19:03.907288 stat64(\".git/objects/58/9ee4d580d34a76238e90795a070a7372de18a2\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.000017>\n10:19:03.907429 open(\".git/objects/58/9ee4d580d34a76238e90795a070a7372de18a2\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:03.907531 mmap2(NULL, 279, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:03.907617 close(3) = 0 <0.000012>\n10:19:03.912980 munmap(0xb7fc9000, 279) = 0 <0.000020>\n10:19:03.913107 stat64(\".git/objects/4c/e643674c24dbda1d0fa0a0da60163ff9ea666f\", {st_mode=S_IFREG|0444, st_size=275, ...}) = 0 <0.013600>\n10:19:03.926830 stat64(\".git/objects/4c/e643674c24dbda1d0fa0a0da60163ff9ea666f\", {st_mode=S_IFREG|0444, st_size=275, ...}) = 0 <0.000016>\n10:19:03.926958 open(\".git/objects/4c/e643674c24dbda1d0fa0a0da60163ff9ea666f\", O_RDONLY|O_NOATIME) = 3 <0.000019>\n10:19:03.927056 mmap2(NULL, 275, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000016>\n10:19:03.927137 close(3) = 0 <0.000011>\n10:19:03.939741 munmap(0xb7fc9000, 275) = 0 <0.000023>\n10:19:03.939895 stat64(\".git/objects/73/8dec0ead6a96f87cf71989a1dad7bb5d1faba4\", {st_mode=S_IFREG|0444, st_size=747, ...}) = 0 <0.020027>\n10:19:03.960098 stat64(\".git/objects/73/8dec0ead6a96f87cf71989a1dad7bb5d1faba4\", {st_mode=S_IFREG|0444, st_size=747, ...}) = 0 <0.000292>\n10:19:03.960542 open(\".git/objects/73/8dec0ead6a96f87cf71989a1dad7bb5d1faba4\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:03.960649 mmap2(NULL, 747, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.960734 close(3) = 0 <0.000012>\n10:19:03.965837 munmap(0xb7fc9000, 747) = 0 <0.000027>\n10:19:03.966016 stat64(\".git/objects/33/c9db007159db11c1ad5fa7101ea95853740acf\", {st_mode=S_IFREG|0444, st_size=469, ...}) = 0 <0.015332>\n10:19:03.981493 stat64(\".git/objects/33/c9db007159db11c1ad5fa7101ea95853740acf\", {st_mode=S_IFREG|0444, st_size=469, ...}) = 0 <0.000016>\n10:19:03.981629 open(\".git/objects/33/c9db007159db11c1ad5fa7101ea95853740acf\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:03.981730 mmap2(NULL, 469, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:03.981815 close(3) = 0 <0.000012>\n10:19:03.988857 munmap(0xb7fc9000, 469) = 0 <0.000023>\n10:19:03.989010 stat64(\".git/objects/84/19a453dc088b25b63ab1746d3d7e679caf686d\", {st_mode=S_IFREG|0444, st_size=382, ...}) = 0 <0.020436>\n10:19:04.009621 stat64(\".git/objects/84/19a453dc088b25b63ab1746d3d7e679caf686d\", {st_mode=S_IFREG|0444, st_size=382, ...}) = 0 <0.000016>\n10:19:04.009759 open(\".git/objects/84/19a453dc088b25b63ab1746d3d7e679caf686d\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:04.009860 mmap2(NULL, 382, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.009944 close(3) = 0 <0.000012>\n10:19:04.017654 munmap(0xb7fc9000, 382) = 0 <0.000021>\n10:19:04.017786 stat64(\".git/objects/85/ba32e27db65314ad4ea25f5afcaad684d68811\", {st_mode=S_IFREG|0444, st_size=275, ...}) = 0 <0.013548>\n10:19:04.031456 stat64(\".git/objects/85/ba32e27db65314ad4ea25f5afcaad684d68811\", {st_mode=S_IFREG|0444, st_size=275, ...}) = 0 <0.000015>\n10:19:04.031583 open(\".git/objects/85/ba32e27db65314ad4ea25f5afcaad684d68811\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:04.031681 mmap2(NULL, 275, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000016>\n10:19:04.031763 close(3) = 0 <0.000011>\n10:19:04.043476 munmap(0xb7fc9000, 275) = 0 <0.000022>\n10:19:04.043619 stat64(\".git/objects/91/82013b4241fefc455f07fc78a922c9c149a38b\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.008078>\n10:19:04.051857 stat64(\".git/objects/91/82013b4241fefc455f07fc78a922c9c149a38b\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.000017>\n10:19:04.052001 open(\".git/objects/91/82013b4241fefc455f07fc78a922c9c149a38b\", O_RDONLY|O_NOATIME) = 3 <0.000019>\n10:19:04.052102 mmap2(NULL, 280, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000097>\n10:19:04.052284 close(3) = 0 <0.000011>\n10:19:04.058917 munmap(0xb7fc9000, 280) = 0 <0.000022>\n10:19:04.059041 stat64(\".git/objects/d2/d79edaf81ec78ad11c8daeb91ee3e82a4aaa19\", {st_mode=S_IFREG|0444, st_size=282, ...}) = 0 <0.013396>\n10:19:04.072584 stat64(\".git/objects/d2/d79edaf81ec78ad11c8daeb91ee3e82a4aaa19\", {st_mode=S_IFREG|0444, st_size=282, ...}) = 0 <0.000016>\n10:19:04.072725 open(\".git/objects/d2/d79edaf81ec78ad11c8daeb91ee3e82a4aaa19\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:04.072827 mmap2(NULL, 282, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.072911 close(3) = 0 <0.000012>\n10:19:04.084973 munmap(0xb7fc9000, 282) = 0 <0.000024>\n10:19:04.085140 stat64(\".git/objects/bc/3aec1567d09218d319e8053d58fd740e3f6d4a\", {st_mode=S_IFREG|0444, st_size=281, ...}) = 0 <0.005028>\n10:19:04.090289 stat64(\".git/objects/bc/3aec1567d09218d319e8053d58fd740e3f6d4a\", {st_mode=S_IFREG|0444, st_size=281, ...}) = 0 <0.000016>\n10:19:04.090414 open(\".git/objects/bc/3aec1567d09218d319e8053d58fd740e3f6d4a\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:04.090515 mmap2(NULL, 281, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000017>\n10:19:04.090597 close(3) = 0 <0.000011>\n10:19:04.101560 munmap(0xb7fc9000, 281) = 0 <0.000019>\n10:19:04.101683 stat64(\".git/objects/8c/7a107d8eeba18ae56befb6142daaba91aa3ecf\", {st_mode=S_IFREG|0444, st_size=378, ...}) = 0 <0.014772>\n10:19:04.116612 stat64(\".git/objects/8c/7a107d8eeba18ae56befb6142daaba91aa3ecf\", {st_mode=S_IFREG|0444, st_size=378, ...}) = 0 <0.000016>\n10:19:04.116745 open(\".git/objects/8c/7a107d8eeba18ae56befb6142daaba91aa3ecf\", O_RDONLY|O_NOATIME) = 3 <0.000019>\n10:19:04.116845 mmap2(NULL, 378, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.116928 close(3) = 0 <0.000011>\n10:19:04.117185 munmap(0xb7fc9000, 378) = 0 <0.000019>\n10:19:04.117305 stat64(\".git/objects/ad/86a7fc13a820a56957f41f81c4763f734676f4\", {st_mode=S_IFREG|0444, st_size=324, ...}) = 0 <0.029156>\n10:19:04.146625 stat64(\".git/objects/ad/86a7fc13a820a56957f41f81c4763f734676f4\", {st_mode=S_IFREG|0444, st_size=324, ...}) = 0 <0.000017>\n10:19:04.146769 open(\".git/objects/ad/86a7fc13a820a56957f41f81c4763f734676f4\", O_RDONLY|O_NOATIME) = 3 <0.000018>\n10:19:04.146868 mmap2(NULL, 324, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.146952 close(3) = 0 <0.000011>\n10:19:04.156903 munmap(0xb7fc9000, 324) = 0 <0.000023>\n10:19:04.157043 stat64(\".git/objects/27/da5785a8fec4e462aeee4dc41281ac89233675\", {st_mode=S_IFREG|0444, st_size=397, ...}) = 0 <0.027180>\n10:19:04.184386 stat64(\".git/objects/27/da5785a8fec4e462aeee4dc41281ac89233675\", {st_mode=S_IFREG|0444, st_size=397, ...}) = 0 <0.000017>\n10:19:04.184532 open(\".git/objects/27/da5785a8fec4e462aeee4dc41281ac89233675\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:04.184635 mmap2(NULL, 397, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.184720 close(3) = 0 <0.000011>\n10:19:04.189421 munmap(0xb7fc9000, 397) = 0 <0.000025>\n10:19:04.189583 stat64(\".git/objects/32/578a62ce8469465504509360aed5f1a6dc5b2b\", {st_mode=S_IFREG|0444, st_size=319, ...}) = 0 <0.011685>\n10:19:04.201394 stat64(\".git/objects/32/578a62ce8469465504509360aed5f1a6dc5b2b\", {st_mode=S_IFREG|0444, st_size=319, ...}) = 0 <0.000016>\n10:19:04.201521 open(\".git/objects/32/578a62ce8469465504509360aed5f1a6dc5b2b\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:04.201621 mmap2(NULL, 319, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:04.201702 close(3) = 0 <0.000012>\n10:19:04.216765 munmap(0xb7fc9000, 319) = 0 <0.000027>\n10:19:04.216944 stat64(\".git/objects/ab/71dde01e310ed582e86446d216531f792468bf\", {st_mode=S_IFREG|0444, st_size=325, ...}) = 0 <0.004873>\n10:19:04.221941 stat64(\".git/objects/ab/71dde01e310ed582e86446d216531f792468bf\", {st_mode=S_IFREG|0444, st_size=325, ...}) = 0 <0.000016>\n10:19:04.222067 open(\".git/objects/ab/71dde01e310ed582e86446d216531f792468bf\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:04.222169 mmap2(NULL, 325, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:04.222253 close(3) = 0 <0.000012>\n10:19:04.230433 munmap(0xb7fc9000, 325) = 0 <0.000019>\n10:19:04.230556 stat64(\".git/objects/d9/b0f913ce0508fcc83e642e0241f373428368e5\", {st_mode=S_IFREG|0444, st_size=275, ...}) = 0 <0.014431>\n10:19:04.245143 stat64(\".git/objects/d9/b0f913ce0508fcc83e642e0241f373428368e5\", {st_mode=S_IFREG|0444, st_size=275, ...}) = 0 <0.000017>\n10:19:04.245282 open(\".git/objects/d9/b0f913ce0508fcc83e642e0241f373428368e5\", O_RDONLY|O_NOATIME) = 3 <0.000018>\n10:19:04.245380 mmap2(NULL, 275, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.245464 close(3) = 0 <0.000011>\n10:19:04.259086 munmap(0xb7fc9000, 275) = 0 <0.000021>\n10:19:04.259218 stat64(\".git/objects/43/14f5982d2aac08001a977fc0b1b611e858e025\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.012914>\n10:19:04.272282 stat64(\".git/objects/43/14f5982d2aac08001a977fc0b1b611e858e025\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.000017>\n10:19:04.272425 open(\".git/objects/43/14f5982d2aac08001a977fc0b1b611e858e025\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:04.272528 mmap2(NULL, 280, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:04.272611 close(3) = 0 <0.000011>\n10:19:04.284930 munmap(0xb7fc9000, 280) = 0 <0.000028>\n10:19:04.285142 stat64(\".git/objects/87/30a82b9f00b8debab717ec384eedbd0f36a54f\", {st_mode=S_IFREG|0444, st_size=294, ...}) = 0 <0.006320>\n10:19:04.291672 stat64(\".git/objects/87/30a82b9f00b8debab717ec384eedbd0f36a54f\", {st_mode=S_IFREG|0444, st_size=294, ...}) = 0 <0.000017>\n10:19:04.291825 open(\".git/objects/87/30a82b9f00b8debab717ec384eedbd0f36a54f\", O_RDONLY|O_NOATIME) = 3 <0.000023>\n10:19:04.291929 mmap2(NULL, 294, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:04.292015 close(3) = 0 <0.000012>\n10:19:04.303292 munmap(0xb7fc9000, 294) = 0 <0.000023>\n10:19:04.303442 stat64(\".git/objects/a4/97d102d6961ded91b60dc3524eb63679d7dc08\", {st_mode=S_IFREG|0444, st_size=282, ...}) = 0 <0.012219>\n10:19:04.315813 stat64(\".git/objects/a4/97d102d6961ded91b60dc3524eb63679d7dc08\", {st_mode=S_IFREG|0444, st_size=282, ...}) = 0 <0.000016>\n10:19:04.315955 open(\".git/objects/a4/97d102d6961ded91b60dc3524eb63679d7dc08\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:04.316057 mmap2(NULL, 282, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.316141 close(3) = 0 <0.000082>\n10:19:04.316530 munmap(0xb7fc9000, 282) = 0 <0.000021>\n10:19:04.316655 stat64(\".git/objects/95/f1e36cc34e8738e5c431aac47d6fea18913db0\", {st_mode=S_IFREG|0444, st_size=395, ...}) = 0 <0.023411>\n10:19:04.340297 stat64(\".git/objects/95/f1e36cc34e8738e5c431aac47d6fea18913db0\", {st_mode=S_IFREG|0444, st_size=395, ...}) = 0 <0.000017>\n10:19:04.340444 open(\".git/objects/95/f1e36cc34e8738e5c431aac47d6fea18913db0\", O_RDONLY|O_NOATIME) = 3 <0.000022>\n10:19:04.340547 mmap2(NULL, 395, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:04.340632 close(3) = 0 <0.000012>\n10:19:04.349413 munmap(0xb7fc9000, 395) = 0 <0.000024>\n10:19:04.349568 stat64(\".git/objects/84/13d9ffefd1fde6d8a04f9e22503ab910fc36de\", {st_mode=S_IFREG|0444, st_size=1061, ...}) = 0 <0.015285>\n10:19:04.365008 stat64(\".git/objects/84/13d9ffefd1fde6d8a04f9e22503ab910fc36de\", {st_mode=S_IFREG|0444, st_size=1061, ...}) = 0 <0.000017>\n10:19:04.365150 open(\".git/objects/84/13d9ffefd1fde6d8a04f9e22503ab910fc36de\", O_RDONLY|O_NOATIME) = 3 <0.000021>\n10:19:04.365252 mmap2(NULL, 1061, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:04.365335 close(3) = 0 <0.000012>\n10:19:04.369839 munmap(0xb7fc9000, 1061) = 0 <0.000021>\n10:19:04.369984 stat64(\".git/objects/d1/eebeecb839749b1559ec7d44b9a31b2513cfb3\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.011146>\n10:19:04.381282 stat64(\".git/objects/d1/eebeecb839749b1559ec7d44b9a31b2513cfb3\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.000017>\n10:19:04.381422 open(\".git/objects/d1/eebeecb839749b1559ec7d44b9a31b2513cfb3\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:04.381523 mmap2(NULL, 279, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000018>\n10:19:04.381607 close(3) = 0 <0.000011>\n10:19:04.381899 munmap(0xb7fc9000, 279) = 0 <0.000020>\n10:19:04.382020 stat64(\".git/objects/10/0490cfcd50d0f9ddadeb90cf5d95b28e2397ce\", {st_mode=S_IFREG|0444, st_size=1641, ...}) = 0 <0.023306>\n10:19:04.405476 open(\".git/objects/10/0490cfcd50d0f9ddadeb90cf5d95b28e2397ce\", O_RDONLY|O_NOATIME) = 3 <0.000023>\n10:19:04.405599 mmap2(NULL, 1641, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:04.405684 close(3) = 0 <0.000012>\n10:19:04.420508 munmap(0xb7fc9000, 1641) = 0 <0.000025>\n10:19:04.420725 stat64(\".git/objects/a8/d8dc4c6cd6d4681746fd6919318640ee97a059\", {st_mode=S_IFREG|0444, st_size=279, ...}) = 0 <0.007644>\n10:19:04.428492 open(\".git/objects/a8/d8dc4c6cd6d4681746fd6919318640ee97a059\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:04.428599 mmap2(NULL, 279, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000017>\n10:19:04.428681 close(3) = 0 <0.000011>\n10:19:04.439591 munmap(0xb7fc9000, 279) = 0 <0.000022>\n10:19:04.439727 stat64(\".git/objects/fa/b4f17015ccb2c6ea978b45386d45dd008df42d\", {st_mode=S_IFREG|0444, st_size=278, ...}) = 0 <0.000039>\n10:19:04.439879 open(\".git/objects/fa/b4f17015ccb2c6ea978b45386d45dd008df42d\", O_RDONLY|O_NOATIME) = 3 <0.000020>\n10:19:04.439978 mmap2(NULL, 278, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000016>\n10:19:04.440060 close(3) = 0 <0.000012>\n10:19:04.449022 munmap(0xb7fc9000, 278) = 0 <0.000028>\n10:19:04.449236 stat64(\".git/objects/3a/41a48d139d1425c1d27e3fbe4f511fb7e09e94\", {st_mode=S_IFREG|0444, st_size=278, ...}) = 0 <0.817989>\n10:19:05.267499 open(\".git/objects/3a/41a48d139d1425c1d27e3fbe4f511fb7e09e94\", O_RDONLY|O_NOATIME) = 3 <0.000027>\n10:19:05.267643 mmap2(NULL, 278, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000019>\n10:19:05.267730 close(3) = 0 <0.000012>\n10:19:05.298154 munmap(0xb7fc9000, 278) = 0 <0.000024>\n10:19:05.298362 stat64(\".git/objects/b9/b0b3d0020c3559d4d9e6d07e3316c3ebcd6e85\", {st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.025652>\n10:19:05.324191 open(\".git/objects/b9/b0b3d0020c3559d4d9e6d07e3316c3ebcd6e85\", O_RDONLY|O_NOATIME) = 3 <0.000095>\n10:19:05.324395 mmap2(NULL, 280, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fc9000 <0.000020>\n10:19:05.324483 close(3) = 0 <0.000011>\n10:19:05.336149 munmap(0xb7fc9000, 280) = 0 <0.000025>\n10:19:05.336371 write(1, \"f416f8f2bc74d6d0a refs/tags/v1.1\"..., 2675) = 2675 <0.000049>\n10:19:05.336516 exit_group(0) = ?\n"},{"id":"297575","messageId":"Pine.LNX.4.64.0611190945500.3692@woody.osdl.org","threadId":"43080","inReplyTo":"e5bfff550611190140n3277ee26v95feba26dd3348fa@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T18:05:44Z","receivedAt":"2006-11-19T18:05:44Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Marco Costalba wrote:\n> \n> Sure. File ran against git tree attached.\n\nOk. Nothing really strange stands out - it's a nice trace with just over \n400 system calls. I'd expect it to finish in no time at all (tracing will \nadd some overhead, since you context switch back and forth between the \ntracer and the tracee, but it's really not doing a lot, so even with \ntracing it should execute almost instantaneously).\n\nSo it all looks _almost_ fine..\n\nExcept for this one:\n\n 10:19:04.449236 stat64(\".git/objects/3a/41a48d139d1425c1d27e3fbe4f511fb7e09e94\", {st_mode=S_IFREG|0444, st_size=278, ...}) = 0 <0.817989>\n\nThat's a _single_ \"stat64()\" system call that takes almost a second to \nexecute. All the rest are in the millisecond range, and sometimes a \nhundreth of a second or two. Ie doing\n\n\tgrep -v ' <0.0[012]' tracefile_git_tree.txt\n\non your tracefile, there's really not a lot of system calls that take a \nlong time, and that one stat _really_ stands out (the others are 3 or four \nhundredths of a second, and then suddenly you have one that is 20 times \nlonger than even the slowest other ones.\n\nBasically, you seem to have a _single_ object access that takes up half \nthe time of the whole program.\n\nIt's the object for 'refs/tags/v1.4.4-rc1' in case you care, btw. \n\n> If you want I can repack and prune, but for now I just wait to avoid\n> to corrupt this test case.\n\nWhat you could try to do is to re-run it a few times (cold-cache) and see \nif those numbers really are stable, and if it's always the same object \nthat takes that long.\n\nIn fact, you could even do a simple\n\n\ttime ls -l .git/objects/3a/41a48d139d1425c1d27e3fbe4f511fb7e09e94\n\nfor the cold-cache case, and see if just even _that_ takes almost a \nsecond.\n\nIf it _is_ stable, there's two possibilities:\n\n - you have a large and slow disk, and that one object really is way out \n there on the other side of the disk, and seeking really takes almost a \n second.\n\n Quite frankly, I expected that the time when a single stat() took \n almost a second was a decade or more in the past, back in the days of \n floppy-disks. But what do I know?\n\n - your disk is failing, and ends up doing error recovery etc.\n\n Maybe worth running \"smartctl -a /dev/hda\" or whatever, to see if the \n disk already knows it is having problems.\n\nAnyway, repacking will fix this, but quite frankly, you might have a \nreason to be a bit nervous about that disk if I were you.\n\n(NOTE NOTE NOTE! There could be other reasons for that second delay. If \nthe machine was under heavy load, or was running low on memory, maybe the \nlong delay was just due to havign to swap things out or run other things \ninstead. That's why it might be interesting to see if the number is \n\"stable\" in that it's always that same object..)\n\n"},{"id":"293821","messageId":"e5bfff550611191107o63d89d8bp5ff4080803a0d784@mail.gmail.com","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611190945500.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-19T19:07:07Z","receivedAt":"2006-11-19T19:07:07Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/19/06, Linus Torvalds wrote:\n>\n> So it all looks _almost_ fine..\n>\n> Except for this one:\n>\n> 10:19:04.449236 stat64(\".git/objects/3a/41a48d139d1425c1d27e3fbe4f511fb7e09e94\", {st_mode=S_IFREG|0444, st_size=278, ...}) = 0 <0.817989>\n>\n> That's a _single_ \"stat64()\" system call that takes almost a second to\n> execute. All the rest are in the millisecond range, and sometimes a\n> hundreth of a second or two. Ie doing\n>\n> grep -v ' <0.0[012]' tracefile_git_tree.txt\n>\n> on your tracefile, there's really not a lot of system calls that take a\n> long time, and that one stat _really_ stands out (the others are 3 or four\n> hundredths of a second, and then suddenly you have one that is 20 times\n> longer than even the slowest other ones.\n>\n> Basically, you seem to have a _single_ object access that takes up half\n> the time of the whole program.\n>\n\nBut why my numbers are bad both in git, in Linux and also qgit (not\nposted) local repo? If it is a single case other repos should load\nfast.\n\n> It's the object for 'refs/tags/v1.4.4-rc1' in case you care, btw.\n>\n> > If you want I can repack and prune, but for now I just wait to avoid\n> > to corrupt this test case.\n>\n> What you could try to do is to re-run it a few times (cold-cache) and see\n> if those numbers really are stable, and if it's always the same object\n> that takes that long.\n>\n\nRight now I'm running smartctl -t long /dev/hda, I was a little bit scared ;-)\n\nWhen finished I will do additional cold chache (reboot) tests.\n\n> In fact, you could even do a simple\n>\n> time ls -l .git/objects/3a/41a48d139d1425c1d27e3fbe4f511fb7e09e94\n>\n> for the cold-cache case, and see if just even _that_ takes almost a\n> second.\n>\n> If it _is_ stable, there's two possibilities:\n>\n> - you have a large and slow disk, and that one object really is way out\n> there on the other side of the disk, and seeking really takes almost a\n> second.\n>\n\nIts a Thinkpad 2.5 inches HD, 2 years old (IBM/Hitachi Travelstar 40GNX family)\n\n> Quite frankly, I expected that the time when a single stat() took\n> almost a second was a decade or more in the past, back in the days of\n> floppy-disks. But what do I know?\n>\n> - your disk is failing, and ends up doing error recovery etc.\n>\n\nNo recent errors are reported:\n\nStripped from smartctl -a /dev/hda\n\nSMART Attributes Data Structure revision number: 16\nVendor Specific SMART Attributes with Thresholds:\nID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE\nUPDATED WHEN_FAILED RAW_VALUE\n 1 Raw_Read_Error_Rate 0x000b 100 100 062 Pre-fail\nAlways - 0\n 2 Throughput_Performance 0x0005 100 100 040 Pre-fail\nOffline - 0\n 3 Spin_Up_Time 0x0007 165 165 033 Pre-fail\nAlways - 1\n 4 Start_Stop_Count 0x0012 097 097 000 Old_age\nAlways - 4928\n 5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail\nAlways - 0\n 7 Seek_Error_Rate 0x000b 100 100 067 Pre-fail\nAlways - 0\n 8 Seek_Time_Performance 0x0005 100 100 040 Pre-fail\nOffline - 0\n 9 Power_On_Hours 0x0012 073 073 000 Old_age\nAlways - 11832\n 10 Spin_Retry_Count 0x0013 100 100 060 Pre-fail\nAlways - 0\n 12 Power_Cycle_Count 0x0032 099 099 000 Old_age\nAlways - 2563\n191 G-Sense_Error_Rate 0x000a 100 100 000 Old_age\nAlways - 0\n192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age\nAlways - 117\n193 Load_Cycle_Count 0x0012 045 045 000 Old_age\nAlways - 558210\n194 Temperature_Celsius 0x0002 130 130 000 Old_age\nAlways - 42 (Lifetime Min/Max 5/59)\n196 Reallocated_Event_Count 0x0032 100 100 000 Old_age\nAlways - 1\n197 Current_Pending_Sector 0x0022 100 100 000 Old_age\nAlways - 0\n198 Offline_Uncorrectable 0x0008 100 100 000 Old_age\nOffline - 0\n199 UDMA_CRC_Error_Count 0x000a 200 200 000 Old_age\nAlways - 0\n210 Unknown_Attribute 0x0023 100 100 001 Pre-fail\nAlways - 0\n\nSMART Error Log Version: 1\nATA Error Count: 2\n CR = Command Register [HEX]\n FR = Features Register [HEX]\n SC = Sector Count Register [HEX]\n SN = Sector Number Register [HEX]\n CL = Cylinder Low Register [HEX]\n CH = Cylinder High Register [HEX]\n DH = Device/Head Register [HEX]\n DC = Device Command Register [HEX]\n ER = Error register [HEX]\n ST = Status register [HEX]\nPowered_Up_Time is measured from power on, and printed as\nDDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,\nSS=sec, and sss=millisec. It \"wraps\" after 49.710 days.\n\nError 2 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)\n When the command that caused the error occurred, the device was\nactive or idle.\n\n After command completion occurred, registers were:\n ER ST SC SN CL CH DH\n -- -- -- -- -- -- --\n 10 51 01 0f 8e a8 e4 Error: IDNF at LBA = 0x04a88e0f = 78155279\n\n Commands leading to the command that caused the error were:\n CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name\n -- -- -- -- -- -- -- -- ---------------- --------------------\n 20 ff 01 0f 8e a8 e4 00 00:05:26.850 READ SECTOR(S)\n c8 ff 01 0f 8e a8 e4 00 00:05:26.775 READ DMA\n c8 ff 01 00 00 00 e0 00 00:05:26.625 READ DMA\n ca 04 01 ff 52 a8 e4 00 00:03:40.575 WRITE DMA\n c8 04 01 ff 52 a8 e4 00 00:03:40.275 READ DMA\n\nError 1 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)\n When the command that caused the error occurred, the device was\nactive or idle.\n\n After command completion occurred, registers were:\n ER ST SC SN CL CH DH\n -- -- -- -- -- -- --\n 10 51 01 0f 8e a8 e4 Error: IDNF 1 sectors at LBA = 0x04a88e0f = 78155279\n\n Commands leading to the command that caused the error were:\n CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name\n -- -- -- -- -- -- -- -- ---------------- --------------------\n c8 ff 01 0f 8e a8 e4 00 00:05:26.775 READ DMA\n c8 ff 01 00 00 00 e0 00 00:05:26.625 READ DMA\n ca 04 01 ff 52 a8 e4 00 00:03:40.575 WRITE DMA\n c8 04 01 ff 52 a8 e4 00 00:03:40.275 READ DMA\n ca 04 7d 73 49 02 e0 00 00:03:40.275 WRITE DMA\n\nSMART Self-test log structure revision number 1\nNo self-tests have been logged. [To run self-tests, use: smartctl -t]\n\n>\n> (NOTE NOTE NOTE! There could be other reasons for that second delay. If\n> the machine was under heavy load, or was running low on memory, maybe the\n> long delay was just due to havign to swap things out or run other things\n> instead. That's why it might be interesting to see if the number is\n> \"stable\" in that it's always that same object..)\n>\n\nNo load, no low memory: I quitted everything before to test. I will\ntest again as soon smartctl finishes.\n\n"},{"id":"294020","messageId":"e5bfff550611191209s63982818vd3999b543e68e8df@mail.gmail.com","threadId":"43080","inReplyTo":"e5bfff550611191107o63d89d8bp5ff4080803a0d784@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-19T20:09:01Z","receivedAt":"2006-11-19T20:09:01Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/19/06, Marco Costalba wrote:\n>\n> When finished I will do additional cold chache (reboot) tests.\n>\n\nPrevious delay was not reproduced. This time I tested also with git\nshow-ref -d (dereferencing tags), always with cold-cache.\n\nIt does not seems there are strange delays, but total time it's high\n(very I/O bound)\n\n\n- LINUX TREE\n\n$ pwd\n/git/linux-2.6\n\n$ time strace -o tracefile -Ttt git show-ref -d >> /dev/null\n0.02user 0.01system 0:02.39elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (127major+894minor)pagefaults 0swaps\n\n$ grep -v ' <0.0[012]' tracefile >> tracefile_slowest.txt\n$ cat tracefile_slowest.txt\n20:51:49.695511 execve(\"/home/marco/bin/git\", [\"git\", \"show-ref\",\n\"-d\"], [/* 78 vars */]) = 0 <0.065006>\n20:51:49.946258 open(\".git/objects/info/alternates\", O_RDONLY) = -1\nENOENT (No such file or directory) <0.091447>\n20:51:50.070555\nstat64(\".git/objects/e0/30f8294a5b9f8179dae10cdbf9dcf32aa64110\",\n{st_mode=S_IFREG|0444, st_size=284, ...}) = 0 <0.037669>\n20:51:51.822469\nstat64(\".git/objects/7a/9d289b6650bf78df77ab463bedc2919df89833\",\n{st_mode=S_IFREG|0444, st_size=285, ...}) = 0 <0.048868>\n20:51:51.930582\nstat64(\".git/objects/80/c218812786f619c9a1ce50d0e7c32c7afde4de\",\n{st_mode=S_IFREG|0444, st_size=210, ...}) = 0 <0.030057>\n20:51:51.981951\nstat64(\".git/objects/44/597f65f6af3c692560a639f61d25398d13d1b6\",\n{st_mode=S_IFREG|0444, st_size=249, ...}) = 0 <0.037440>\n20:51:52.023469 exit_group(0) = ?\n\n\n- GIT TREE\n\n$cd /home/marco/programmi/git\n$ time strace -o tracefile -Ttt git show-ref -d >> /dev/null\n0.04user 0.04system 0:02.47elapsed 3%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (91major+794minor)pagefaults 0swaps\n\n$ grep -v ' <0.0[012]' tracefile >> tracefile_slowest.txt\n$ cat tracefile_slowest.txt\n20:55:04.844584\nstat64(\".git/objects/d0/f877fb601ac0076fd69ec4f48f71f9247771c8\",\n{st_mode=S_IFREG|0444, st_size=190, ...}) = 0 <0.046100>\n20:55:04.944839\nstat64(\".git/objects/e2/67c2f6f0784e242883b7d3fe5f36ef63d6950d\",\n{st_mode=S_IFREG|0444, st_size=179, ...}) = 0 <0.033536>\n20:55:05.057966\nstat64(\".git/objects/41/292ddd37202ff6dce34986c87a6000c5d3fbfa\",\n{st_mode=S_IFREG|0444, st_size=180, ...}) = 0 <0.072937>\n20:55:05.432107\nstat64(\".git/objects/7d/09fbe4ab7f080a8f8f5dcef7e0f3edf5e26019\",\n{st_mode=S_IFREG|0444, st_size=396, ...}) = 0 <0.035952>\n20:55:05.592877\nstat64(\".git/objects/92/d7605b6d9312a29efa30b1baa4d7c880a0be41\",\n{st_mode=S_IFREG|0444, st_size=280, ...}) = 0 <0.032139>\n20:55:05.738173\nstat64(\".git/objects/05/56a11a0df6b4119e01aa77dfb795561e62eb34\",\n{st_mode=S_IFREG|0444, st_size=557, ...}) = 0 <0.031572>\n20:55:06.176786\nstat64(\".git/objects/88/3653babd8ee7ea23e6a5c392bb739348b1eb61\",\n{st_mode=S_IFREG|0444, st_size=442, ...}) = 0 <0.030382>\n20:55:06.549025\nstat64(\".git/objects/95/f1e36cc34e8738e5c431aac47d6fea18913db0\",\n{st_mode=S_IFREG|0444, st_size=395, ...}) = 0 <0.031731>\n20:55:06.844432\nstat64(\".git/objects/ba/f0bfcb4b335438e9359835f2c27cccf20e54a3\",\n{st_mode=S_IFREG|0444, st_size=182, ...}) = 0 <0.033688>\n20:55:06.922993 exit_group(0) = ?\n\n\n"},{"id":"296099","messageId":"Pine.LNX.4.64.0611191156520.3692@woody.osdl.org","threadId":"43080","inReplyTo":"e5bfff550611191107o63d89d8bp5ff4080803a0d784@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T20:18:30Z","receivedAt":"2006-11-19T20:18:30Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Marco Costalba wrote:\n> \n> But why my numbers are bad both in git, in Linux and also qgit (not\n> posted) local repo? If it is a single case other repos should load\n> fast.\n\nWell, it's almost guaranteed to not really be a \"single\" case.\n\nAn inode on disk is usually ~128 bytes or so, which means that you'll be \nable to fit quite a number of inodes in a page-sized allocation of disk \ncache. If _any_ of the sectors was slow to access (because of IO retries, \ndisk sector remapping, whatever), all those inodes will be slow to access.\n\n(It might also be a specific area on the disk, or something).\n\n> Its a Thinkpad 2.5 inches HD, 2 years old (IBM/Hitachi Travelstar 40GNX\n> family)\n\nThat's a 5400rpm disk with an average seektime of 12ms, and full stroke \ntypical seek of 23ms.\n\nA \"stat64()\" will do more than a single IO when it's cold-cache (the \nkernel will have to look up the directory entry and the inode, of course), \nbut you _should_ see just a few IO's (2-3), so quite frankly, normally I'd \nexpect to see times in the 0.03s - 0.05s range _maximum_. You'd see less \nif even just part of it was cached (which is normal, exactly because \nthings like directory entries are contiguous on disk etc).\n\nAnd that seems to match your other numbers.\n\nThe 0.8s number really is an outlier. It sounds like the drive didn't find \nthe sector at first, perhaps had to go to its remapping table, seek around \na bit more, take a break just to calm down, and then perhaps recalibrate \nitself (or write a SMART record entry).\n\n> > - your disk is failing, and ends up doing error recovery etc.\n> \n> No recent errors are reported:\n> \n> Stripped from smartctl -a /dev/hda\n\nOk, I'm not actually all that familiar with all the SMART error codes, but \nit _looks_ healthy. You do have one Spin_Up_Time event (that is \nmarked pre-fail), and those two IDNF errors you have:\n\n> 10 51 01 0f 8e a8 e4 Error: IDNF at LBA = 0x04a88e0f = 78155279\n> 10 51 01 0f 8e a8 e4 Error: IDNF 1 sectors at LBA = 0x04a88e0f = 78155279\n\nmakes me wonder a bit, but for all I know, it's not actually serious. I \n_think_ the IDNF error means that it couldn't find a sector, but..\n\n"},{"id":"296971","messageId":"Pine.LNX.4.64.0611191219350.3692@woody.osdl.org","threadId":"43080","inReplyTo":"e5bfff550611191209s63982818vd3999b543e68e8df@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T20:36:46Z","receivedAt":"2006-11-19T20:36:46Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Marco Costalba wrote:\n> \n> It does not seems there are strange delays, but total time it's high\n> (very I/O bound)\n\nThis looks more normal. No truly horrid IO times. With your disk, having \nan uncached \"stat64()\" taking ~50ms is not at all impossible, if you just \nend up having to do a few seeks for directory/inode information.\n\n> $ time strace -o tracefile -Ttt git show-ref -d >> /dev/null\n> 0.02user 0.01system 0:02.39elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k\n> 0inputs+0outputs (127major+894minor)pagefaults 0swaps\n\nSo in addition to the \"stat()\" calls on all the objects you have \nreferenced, you also had 127 page faults that needed to do IO (probably a \ncombination of executable and pack-file accesses). \n\nI think the only way to avoid this is likely to try to either not do the \nobject lookups at all (which you really cannot currently avoid with \"-d\", \nsince the whole point is to dereference the objects if they are tags), or \nto do some silly optimizations like fsck does.\n\nFor example, it's often (but not always) faster to do all the readdir's \nseparately, and then sort the thing by inode number, and try to avoid \nback-and-forth movement. But quite frankly, that kind of stuff probably \nisn't sane to do in \"git show-refs\".\n\nSo the optimizations that _can_ be done are:\n\n - add dereference info to .git/packed-refs\n\n This would allow us to simply not do the expensive object lookup for \n every single tag. We'd still have to do it for non-packed objects, of \n course, but the cost here tends to be that over time you might have \n hundreds of tags, and even if each tag only takes 0.02s to look up, \n you're going to be slow.\n\n - avoid the references for \"heads/\" (which we know are supposed to be \n commits, and cannot be tags) and when not specifying \"-d\". This won't \n help your case very much, though. If you want \"-d\", you want it, and \n the _big_ number of refs tends to be in tags, not branches, anyway.\n\n - using a filesystem wih nicer locality behaviour for directory entries \n and inodes. This can cut down costs of cold-cache case by a factor of \n two, but right now there are no good filesystems that do this (but see \n for example \"spadfs\" that Mikulas Patocka announced a few weeks ago on \n linux-kernel - it would seem to have the possibility of being better in \n this area. I looked at the code and it looked like it could become \n very reasonable, but I've not actually _tested_ it, soo...)\n\nAnyway, I think that if we really want to make \"git show-refs\" go fast \nwhen things are cold in the cache, and with lots ot tags and \"-d\" (which \nis a reasonable case to optimize for: it's probably exactly what we end up \ndoing both for gitweb _and_ for \"git-send-pack\"), we'd need to expand the \npacked-refs file with the deref cache.\n\nJunio?\n\n"},{"id":"295935","messageId":"Pine.LNX.4.64.0611191242000.3692@woody.osdl.org","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611191219350.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T20:44:11Z","receivedAt":"2006-11-19T20:44:11Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Linus Torvalds wrote:\n> \n> So the optimizations that _can_ be done are:\n\nOf course, doing a \"git repack -a -d && git prune\" is always the #1 \noptimization. It will still look up the objects, but it will be a lot \nfaster about it, and avoid a lot of the really bad IO patterns.\n\nIf you don't want to repack, you could try just doing a \"git clone\" and \nthen doing the \"git pack-refs --prune\" in the clone too. And compare \ncold-cache times between the two \"identical\" repositories.\n\n"},{"id":"298293","messageId":"7vbqn38831.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611191219350.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-19T21:01:06Z","receivedAt":"2006-11-19T21:01:06Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> So the optimizations that _can_ be done are:\n>\n> - add dereference info to .git/packed-refs\n>\n> This would allow us to simply not do the expensive object lookup for \n> every single tag. We'd still have to do it for non-packed objects, of \n> course, but the cost here tends to be that over time you might have \n> hundreds of tags, and even if each tag only takes 0.02s to look up, \n> you're going to be slow.\n>\n> - avoid the references for \"heads/\" (which we know are supposed to be \n> commits, and cannot be tags) and when not specifying \"-d\". This won't \n> help your case very much, though. If you want \"-d\", you want it, and \n> the _big_ number of refs tends to be in tags, not branches, anyway.\n>\n> - using a filesystem wih nicer locality behaviour for directory entries \n> and inodes. This can cut down costs of cold-cache case by a factor of \n> two, but right now there are no good filesystems that do this (but see \n> for example \"spadfs\" that Mikulas Patocka announced a few weeks ago on \n> linux-kernel - it would seem to have the possibility of being better in \n> this area. I looked at the code and it looked like it could become \n> very reasonable, but I've not actually _tested_ it, soo...)\n>\n> Anyway, I think that if we really want to make \"git show-refs\" go fast \n> when things are cold in the cache, and with lots ot tags and \"-d\" (which \n> is a reasonable case to optimize for: it's probably exactly what we end up \n> doing both for gitweb _and_ for \"git-send-pack\"), we'd need to expand the \n> packed-refs file with the deref cache.\n>\n> Junio?\n\nYes; the \"for discussion only\" WIP patch from yesterday was\nabout the first point and half of the second point.\n\nOne downside about storing the peeled refs in .git/packed-refs\nis that the code in the wild does not have safety guard against\nthem, so the version of show-ref from v1.4.4 recently released\nwill list tags/v2.6.19^{} as if it is a true tag and worse yet\nit might even show tags/v2.6.19^{}^{} under -d option after the\nrepository's refs are packed in the new format.\n\nBut I think we can avoid that problem by using a format slightly\ndifferent from the WIP patch. We can for example use two spaces\nbetween SHA-1 and the name for them. refs.c::parse_ref_line()\nin existing code will say \"oh, this line is nonsense\" and skip\nthem, while the updated code can say \"ok, this is a peeled ref\",\nsince it has a seemingly incomplete \"check for valid refname\"\nthere that only checks \"isspace(*line)\".\n\nDid you do this \"if (isspace(*line)) return NULL\" to have\nextensibility later, I wonder...\n\n\n\n"},{"id":"296879","messageId":"Pine.LNX.4.64.0611191310430.3692@woody.osdl.org","threadId":"43080","inReplyTo":"7vbqn38831.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T21:14:34Z","receivedAt":"2006-11-19T21:14:34Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Junio C Hamano wrote:\n> \n> Did you do this \"if (isspace(*line)) return NULL\" to have\n> extensibility later, I wonder...\n\nNo, but I try to have a policy of not accepting anything that is even \nslightly questionable, so it was unintentional in the details but a result \nof a higher-level principle..\n\nOf course, even when I wrote it I _also_ knew that I should check \neverything else too (not just the first character), but I was lazy. That \nwould have been even better - then we could have made the thing be \nsomething like\n\n\t[]*\n\ninstead.\n\n"},{"id":"296251","messageId":"ejqhvi$9kc$1@sea.gmane.org","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611191310430.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-19T21:24:27Z","receivedAt":"2006-11-19T21:24:27Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Linus Torvalds wrote:\n\n> On Sun, 19 Nov 2006, Junio C Hamano wrote:\n>> \n>> Did you do this \"if (isspace(*line)) return NULL\" to have\n>> extensibility later, I wonder...\n> \n> No, but I try to have a policy of not accepting anything that is even \n> slightly questionable, so it was unintentional in the details but a result \n> of a higher-level principle..\n> \n> Of course, even when I wrote it I _also_ knew that I should check \n> everything else too (not just the first character), but I was lazy. That \n> would have been even better - then we could have made the thing be \n> something like\n> \n> []*\n> \n> instead.\n\nSo the only reason against this format (IMHO more reasonable, more readable\nand more readable, and even easy extendable to the whole chain of derefs)\nis code in the wild? \n\nBy the way, do we copy packed refs literaly when fetching or cloning?\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"294278","messageId":"e5bfff550611191425g30d02b98l71cfdc6439d2e09f@mail.gmail.com","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611191219350.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-19T22:25:49Z","receivedAt":"2006-11-19T22:25:49Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"On 11/19/06, Linus Torvalds wrote:\n>\n>\n>\n> For example, it's often (but not always) faster to do all the readdir's\n> separately, and then sort the thing by inode number, and try to avoid\n> back-and-forth movement. But quite frankly, that kind of stuff probably\n> isn't sane to do in \"git show-refs\".\n\nExcuse me for my ignorance, but isn' it the job of OS disk schedulers?\n\n>\n> Anyway, I think that if we really want to make \"git show-refs\" go fast\n> when things are cold in the cache, and with lots ot tags and \"-d\" (which\n> is a reasonable case to optimize for: it's probably exactly what we end up\n> doing both for gitweb _and_ for \"git-send-pack\"), we'd need to expand the\n> packed-refs file with the deref cache.\n>\n\nProbably also gitk, currently it uses (the slower) \"git ls-remote\"\ninstead of \"git peek-remote\" or \"git show-refs\" but the concept seems\nthe same.\n\n"},{"id":"294422","messageId":"Pine.LNX.4.64.0611191522580.3692@woody.osdl.org","threadId":"43080","inReplyTo":"e5bfff550611191425g30d02b98l71cfdc6439d2e09f@mail.gmail.com","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T23:26:12Z","receivedAt":"2006-11-19T23:26:12Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Marco Costalba wrote:\n> \n> Excuse me for my ignorance, but isn' it the job of OS disk schedulers?\n\nThe OS disk scheduler does exactly that but\n\n\tONLY IF IT'S GIVEN DATA TO WORK ON IN PARALLEL\n\nIf an application gives it read requests one by one, the OS has no choice \nbut to just do the accesses one by one.\n\nWrites are easier, since you can just buffer them. But you can't \"buffer\" \na read. When the user asks for a readdir(), you'd better give it to the \nuser, and there's not anything you can do about it.\n\nSo disk schedulers only work for \n - parallel workloads\n - writes\n\n(where \"parallel workloads\" can be asynchronous reads - where the user \nsays \"I will _start_ this read, notify me when it's done\" and then gives \nmultiple independent reads to do in parallel).\n\nSo the OS cannot add parallelism - it can only take advantage of what \nparallelism the application gives it.\n\n"},{"id":"294330","messageId":"Pine.LNX.4.64.0611191527270.3692@woody.osdl.org","threadId":"43080","inReplyTo":"ejqhvi$9kc$1@sea.gmane.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-19T23:36:36Z","receivedAt":"2006-11-19T23:36:36Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Jakub Narebski wrote:\n> \n> So the only reason against this format (IMHO more reasonable, more readable\n> and more readable, and even easy extendable to the whole chain of derefs)\n> is code in the wild? \n> \n> By the way, do we copy packed refs literaly when fetching or cloning?\n\nThe packed refs format is _purely_ a local format. So in that sense, we \ncan change it any way we want, and nobody really cares.\n\nHOWEVER, even on a local machine, we generally want to be able to upgrade \nand downgrade git versions without having to worry about things like this, \nand having to convert one format to another. So using a format that \"just \nworks\" with any version of git that understands packed refs is _wildly_ \nmore preferable over changing the format.\n\nBtw, the simplest and best format is probably to make the new extension \nlook something like\n\n\t8ba130df4b67fa40878ccf80d54615132d24bc68 refs/tags/v2.6.17\n\t^427abfa28afedffadfca9dd8b067eb6d36bac53f\n\td882e0c80e6e3c60640492b83395e6fbbae04276 refs/tags/v2.6.17-rc1\n\t^6246b6128bbe34d0752f119cf7c5111c85fe481d\n\nwhich is basically almost as dense as having a space on the same line, and \nwill also trigger the old \"that's not a valid line, just ignore it\" \nreaction from older versions.\n\nSo the parsing rules for that would simply be:\n\n - if you see a line that starts with a \"^\", then that is the \n \"unpeeling\" of the previous packed entry (which in turn might have been \n an unpeeling itself)\n\nSo if you were to have tags pointing to tags, you migt have\n\n\t refs/tags/tagname\n\t^\n\t^\n\t...\n\t refs/tags/othertag\n\nIn addition, we'd need a line at the top of the file that says \"this has \nunpeeled information\", because otherwise we have no way to distinguish \nbetween the case of \"no actual tag objects\" and \"old-fashioned ref-pack \nfile without any unpeeling info\" - since they'd look identical.\n\nSo I'd suggest adding - at the very top of the ref-pack file - a line line\n\n\t# Ref-pack version 2\n\nwhich will be ignored by the current ref-pack reader (again, because it's \nnot a valid ref line), but we can use it in the future to specify further \nextensions if we want to.\n\nNow somebody would just need to implement that ;)\n\n"},{"id":"296003","messageId":"7vr6vy7smi.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611191527270.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-20T02:35:01Z","receivedAt":"2006-11-20T02:35:01Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> So I'd suggest adding - at the very top of the ref-pack file - a line line\n>\n> \t# Ref-pack version 2\n>\n> which will be ignored by the current ref-pack reader (again, because it's \n> not a valid ref line), but we can use it in the future to specify further \n> extensions if we want to.\n>\n> Now somebody would just need to implement that ;)\n\nFor this particular one, there is no need for version 2.\n\nMy current wip does:\n\n\tSHA-1 SP name LF\n\tSHA-1 SP SP name^{} LF\n\nthe latter of which is ignored by code in the wild and the new\ncode can take advantage of (and fall back the usual deref_tag\nwhen it is not available).\n\nI need to rebase it on top of a minor update to refs.c before\npushing it out.\n"},{"id":"297595","messageId":"ejrt37$fg9$1@sea.gmane.org","threadId":"43080","inReplyTo":"7vr6vy7smi.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-20T09:40:15Z","receivedAt":"2006-11-20T09:40:15Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n\n> Linus Torvalds writes:\n> \n>> So I'd suggest adding - at the very top of the ref-pack file - a line line\n>>\n>> # Ref-pack version 2\n>>\n>> which will be ignored by the current ref-pack reader (again, because it's \n>> not a valid ref line), but we can use it in the future to specify further \n>> extensions if we want to.\n>>\n>> Now somebody would just need to implement that ;)\n> \n> For this particular one, there is no need for version 2.\n\nActually, I think it is both true and untrue. True, because we need some\nindicator that we trust packed-refs file to provide tag dereferences to\ndistinguish between the case when there are no tag objects at all, so there\nare no tag dereferences in packed-refs, and the situation where we use\npacked-refs generated by older git, and there are no tag dereferences in\npacked-refs because git didn't saved it.\n\nUntrue, because it is not enough. In the case[*1*] when packed-refs was\ncreated with tag dereferences, then some \"heavyweight\" tags were added\nby older version of git (adding references doesn't rewrite packed-refs\nif I understand correctly), then we use new git again and trust that there\nare no derefs...\n\n[*1*] For example when git repository is on the network filesystem, but\nprograms are installed locally, and perhaps computers in the network are\nheterogenic (perhaps even different architectures: PC vs. Sun and/or\ndifferent operating systems: Linux vs. FreeBSD vs. Solaris vs.\nWindows+Cygwin) and have different versions of git installed (perhaps\none of them is \"your\" machine, where you have admin rights, and you have\nnewest git installed there). Or for example using git repository on USB\nstick, again on different computers with different version of git installed.\n \n---------------------------------------------------------------------\n\nTo summarize, we have the following proposals of the packed-refs format\nextension\n\n\nThe unusable Linux Torvalds proposal (unusable because of requiring\nnewer packed-refs work with older git, for example in the case [*1*]\nor the case of git downgrade):\n\nlt> []*\n\n\nLinus Torvalds \"Now somebody would just need to implement that ;)\"\nproposal:\n\nlt> refs/tags/tagname\nlt> ^\nlt> ^\nlt> ...\nlt> refs/tags/othertag\n\n\nJunio C Hamano proposal _with code_ (proposal with code usually wins).\nLess elegant IMVHO, but perhaps better.\n\njc> My current wip does:\njc> \njc> SHA-1 SP name LF\njc> SHA-1 SP SP name^{} LF\njc> \njc> the latter of which is ignored by code in the wild and the new\njc> code can take advantage of (and fall back the usual deref_tag\njc> when it is not available).\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"294249","messageId":"200611201233.02581.jnareb@gmail.com","threadId":"43080","inReplyTo":"7vu00u2wln.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-11-20T11:33:02Z","receivedAt":"2006-11-20T11:33:02Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n> Jakub Narebski writes:\n> \n>> ... I mean you trust (use) reference\n>> info from packed-refs, but don't trust lack of dereference in\n>> packed-refs.\n> \n> That is exactly what the code does (at least that was the intent;\n> there could be bugs since I am not Linus ;-).\n\nThe question is: is it more common case to have very large number\nof heavyweight tags, or is it more common case to have very large\nnumber of lightweight tags (refs to commit objects).\n\nIn the latter case the solution to not trust lack of dereference\nmeans no gain in performance (although for the core checking type\nof object is faster (much faster?) than depeeling tag, so the gain\nwouldn't be large), although the solution is probably safer.\n\nStill, the decision: do not trust the lack of dereference in\npacked-refs, or mark packed-refs as having dereferences and trust\nlack of dereferences, is fairly orthogonal to the format for depeel\nin packed-refs.\n\n\nP.S. I have just noticed that you have taken the discussion\noff-list...\n-- \nJakub Narebski\n"},{"id":"298394","messageId":"e5bfff550611200456l4bc730fcvd91c01843aeb6c64@mail.gmail.com","threadId":"43080","inReplyTo":"ejrt37$fg9$1@sea.gmane.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2006-11-20T12:56:10Z","receivedAt":"2006-11-20T12:56:10Z","isPatch":false,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"> >\n> > For this particular one, there is no need for version 2.\n>\n> Actually, I think it is both true and untrue. True, because we need some\n> indicator that we trust packed-refs file to provide tag dereferences to\n> distinguish between the case when there are no tag objects at all, so there\n> are no tag dereferences in packed-refs, and the situation where we use\n> packed-refs generated by older git, and there are no tag dereferences in\n> packed-refs because git didn't saved it.\n>\n\nWe should be able to handle the ambiguous/malformed lines correctly\nand gracefully _always_, without 'trust' a version number to avoid to\nbe prone to attacks with a malicious malformed file.\n\nAnyway document versioning it's a common and savy practice.\n\n\n"},{"id":"297910","messageId":"Pine.LNX.4.64.0611200817330.3692@woody.osdl.org","threadId":"43080","inReplyTo":"7vr6vy7smi.fsf@assigned-by-dhcp.cox.net","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-11-20T16:29:38Z","receivedAt":"2006-11-20T16:29:38Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 19 Nov 2006, Junio C Hamano wrote:\n\n> Linus Torvalds writes:\n> \n> > So I'd suggest adding - at the very top of the ref-pack file - a line line\n> >\n> > \t# Ref-pack version 2\n> >\n> > which will be ignored by the current ref-pack reader (again, because it's \n> > not a valid ref line), but we can use it in the future to specify further \n> > extensions if we want to.\n> >\n> > Now somebody would just need to implement that ;)\n> \n> For this particular one, there is no need for version 2.\n\nI don't think you understand.\n\n> My current wip does:\n> \n> \tSHA-1 SP name LF\n> \tSHA-1 SP SP name^{} LF\n\nI think that's ugly and redundant (if \"name\" is ever different from the \nlien above it, that would be a bug), but that's not the real problem.\n\nThe real problem is (go back to the mail that you answered, and snipped \nthe explanation from) this:\n\n - you have a thousand tags\n\n - NONE of them are \"tag objects\".\n\n - as a result your ref-pack file doesn't have a _single_ of the ^{} lines\n\nThink about it. How do you know whether you should look up the tag objects \nfor \"-d\" or not?\n\nThe answer is: you don't. You can't tell a \"version 1\" and \"version 2\" \nfile apart. It might be an old \"version 1\" file that simply doesn't _have_ \ndereference information. Or it might be a \"version 2\" file that _does_ \nhave dereference information, but nothing to dereference.\n\nSo you either have to:\n\n - look up each object again to see if it's a tag that should be \n dereferenced\n\nOR:\n\n - add a \"# ref-pack version 2\" flag at the top of the file.\n\nSo it's not about \"parsing\" the new file structure. I realize that parsing \nit is trivial. It's simply about knowing whether the new information \n_could_ be there or not.\n\nAnd once you have that flag, your _future_ extensions can add their own \nversion, which is an added bonus. But that means that \"version 2\" parsing \nshould _also_ ignore lines that it cannot match, so you'd better have an \nescape from the new format. I personally think that using\n\n\t^\n\ninstead of \"^{}\" is better partly for that \nreason: it's not only denser, it is \"stricter\" in the sense that there's \nless room for some future extended version that could be mistaken for a \n\"version 2 unpeeling\" line. \n\n(But you can do the same thing with your version too. You should:\n - check that there is just _one_ extra space\n - verify that the name matches the previous one\n - verify that it ends exactly with \"^{}\", so that any future extension \n could add their own flags at the end.)\n\nBut regardless of which format chosen, you need the flag of \"this format \nis in use\", exactly because the extended unpeeling information might not \n_exist_.\n\nOh, and regardless of which format chosen, you'd need to verify that the \nunpeeled object in the pack wasn't overridden, of course. \n\n\t\tLinus\n"},{"id":"296243","messageId":"7v7ixp3oe4.fsf@assigned-by-dhcp.cox.net","threadId":"43080","inReplyTo":"Pine.LNX.4.64.0611200817330.3692@woody.osdl.org","subject":"Re: [WISH] Store also tag dereferences in packed-refs","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-11-20T19:32:19Z","receivedAt":"2006-11-20T19:32:19Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n>> For this particular one, there is no need for version 2.\n>\n> I don't think you understand.\n\nThat is true. I was not thinking about optimizing the\nlightweight tag case -- we would want to be able to tell that\nthey are not tag objects and skip peel_ref altogether, and in\norder to do that we do need a way to tell us that we can trust\nthe absense of peeled representation in the packed-refs file.\n\nIt is my day job day so I do not expect I can continue til later\ntoday, though...\n"}]}