{"thread":{"id":"7190","subject":"[PATCH] prepare_packed_git(): sort packs by age and localness.","startedAt":"2007-03-11T02:29:20Z","lastAt":"2007-03-11T03:07:13Z","messageCount":3,"participants":["Junio C Hamano","Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"36776","messageId":"7vtzwsy0lb.fsf@assigned-by-dhcp.cox.net","threadId":"7190","inReplyTo":null,"subject":"[PATCH] prepare_packed_git(): sort packs by age and localness.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-03-11T02:29:20Z","receivedAt":"2007-03-11T02:29:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When accessing objects, we first look for them in packs that\nare linked together in the reverse order of discovery.\n\nSince younger packs tend to contain more recent objects, which\nare more likely to be accessed often, and local packs tend to\ncontain objects more relevant to our specific projects, sort the\nlist of packs before starting to access them.  In addition,\nfavoring local packs over the ones borrowed from alternates can\nbe a win when alternates are mounted on network file systems.\n\nSigned-off-by: Junio C Hamano <junkio@cox.net>\n---\n cache.h     |    2 ++\n sha1_file.c |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 60 insertions(+), 0 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex ae25759..db3a000 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -373,6 +373,7 @@ extern struct packed_git {\n \tstruct packed_git *next;\n \tstruct pack_window *windows;\n \tuint32_t *index_base;\n+\ttime_t mtime;\n \toff_t index_size;\n \toff_t pack_size;\n \tint pack_fd;\n@@ -431,6 +432,7 @@ extern off_t find_pack_entry_one(const unsigned char *, struct packed_git *);\n extern void *unpack_entry(struct packed_git *, off_t, enum object_type *, unsigned long *);\n extern unsigned long unpack_object_header_gently(const unsigned char *buf, unsigned long len, enum object_type *type, unsigned long *sizep);\n extern const char *packed_object_info_detail(struct packed_git *, off_t, unsigned long *, unsigned long *, unsigned int *, unsigned char *);\n+void rearrange_packed_git(void);\n \n /* Dumb servers support */\n extern int update_server_info(int);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 219a10f..3036817 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -739,6 +739,7 @@ struct packed_git *add_packed_git(char *path, int path_len, int local)\n \tp->windows = NULL;\n \tp->pack_fd = -1;\n \tp->pack_local = local;\n+\tp->mtime = st.st_mtime;\n \tif ((path_len > 44) && !get_sha1_hex(path + path_len - 44, sha1))\n \t\thashcpy(p->sha1, sha1);\n \treturn p;\n@@ -823,6 +824,62 @@ static void prepare_packed_git_one(char *objdir, int local)\n \tclosedir(dir);\n }\n \n+static int sort_pack(const void *a_, const void *b_)\n+{\n+\tstruct packed_git *a = *((struct packed_git **)a_);\n+\tstruct packed_git *b = *((struct packed_git **)b_);\n+\tint st;\n+\n+\t/*\n+\t * Local packs tend to contain objects specific to our\n+\t * variant of the project than remote ones.  In addition,\n+\t * remote ones could be on a network mounted filesystem.\n+\t * Favor local ones for these reasons.\n+\t */\n+\tst = a->pack_local - b->pack_local;\n+\tif (st)\n+\t\treturn -st;\n+\n+\t/*\n+\t * Younger packs tend to contain more recent objects,\n+\t * and more recent objects tend to get accessed more\n+\t * often.\n+\t */\n+\tif (a->mtime < b->mtime)\n+\t\treturn 1;\n+\telse if (a->mtime == b->mtime)\n+\t\treturn 0;\n+\treturn -1;\n+}\n+\n+void rearrange_packed_git(void)\n+{\n+\tstruct packed_git **ary, *p;\n+\tint i, n;\n+\tint (*sort_fn)(const void *, const void *);\n+\n+\tfor (n = 0, p = packed_git; p; p = p->next)\n+\t\tn++;\n+\tif (n < 2)\n+\t\treturn;\n+\n+\t/* prepare an array of packed_git for easier sorting */\n+\tary = xcalloc(n, sizeof(struct packed_git *));\n+\tfor (n = 0, p = packed_git; p; p = p->next)\n+\t\tary[n++] = p;\n+\n+\tsort_fn = sort_pack;\n+\tqsort(ary, n, sizeof(struct packed_git *), sort_fn);\n+\n+\t/* link them back again */\n+\tfor (i = 0; i < n - 1; i++)\n+\t\tary[i]->next = ary[i + 1];\n+\tary[n - 1]->next = NULL;\n+\tpacked_git = ary[0];\n+\n+\tfree(ary);\n+}\n+\n static int prepare_packed_git_run_once = 0;\n void prepare_packed_git(void)\n {\n@@ -837,6 +894,7 @@ void prepare_packed_git(void)\n \t\tprepare_packed_git_one(alt->base, 0);\n \t\talt->name[-1] = '/';\n \t}\n+\trearrange_packed_git();\n \tprepare_packed_git_run_once = 1;\n }\n \n-- \n1.5.0.3.942.g299fc\n"},{"id":"36777","messageId":"20070311023422.GE10343@spearce.org","threadId":"7190","inReplyTo":"7vtzwsy0lb.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] prepare_packed_git(): sort packs by age and localness.","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-03-11T02:34:22Z","receivedAt":"2007-03-11T02:34:22Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano <junkio@cox.net> wrote:\n> When accessing objects, we first look for them in packs that\n> are linked together in the reverse order of discovery.\n> \n> Since younger packs tend to contain more recent objects, which\n> are more likely to be accessed often, and local packs tend to\n> contain objects more relevant to our specific projects, sort the\n> list of packs before starting to access them.  In addition,\n> favoring local packs over the ones borrowed from alternates can\n> be a win when alternates are mounted on network file systems.\n\nACK.  But...\n \n> diff --git a/cache.h b/cache.h\n> @@ -431,6 +432,7 @@ extern off_t find_pack_entry_one(const unsigned char *, struct packed_git *);\n> +void rearrange_packed_git(void);\n\nIs that really necessary to make public?\n\n-- \nShawn.\n"},{"id":"36778","messageId":"7vps7gxyu6.fsf@assigned-by-dhcp.cox.net","threadId":"7190","inReplyTo":"20070311023422.GE10343@spearce.org","subject":"Re: [PATCH] prepare_packed_git(): sort packs by age and localness.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-03-11T03:07:13Z","receivedAt":"2007-03-11T03:07:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Shawn O. Pearce\" <spearce@spearce.org> writes:\n\n> Junio C Hamano <junkio@cox.net> wrote:\n>> When accessing objects, we first look for them in packs that\n>> are linked together in the reverse order of discovery.\n>> \n>> Since younger packs tend to contain more recent objects, which\n>> are more likely to be accessed often, and local packs tend to\n>> contain objects more relevant to our specific projects, sort the\n>> list of packs before starting to access them.  In addition,\n>> favoring local packs over the ones borrowed from alternates can\n>> be a win when alternates are mounted on network file systems.\n>\n> ACK.  But...\n>  \n>> diff --git a/cache.h b/cache.h\n>> @@ -431,6 +432,7 @@ extern off_t find_pack_entry_one(const unsigned char *, struct packed_git *);\n>> +void rearrange_packed_git(void);\n>\n> Is that really necessary to make public?\n\nThe next patch in the series was going to sort in a different\norder (giving it a parameter to control the sort order) that is\nmore suitable while repacking, but it seems that the patch is\nnot needed.\n"}]}