{"thread":{"id":"33324","subject":"[PATCH] rev-list: preallocate object hash table in --all --objects","startedAt":"2013-03-29T13:20:10Z","lastAt":"2013-04-01T18:33:45Z","messageCount":6,"participants":["Nguyễn Thái Ngọc Duy","Jeff King","Duy Nguyen","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"212619","messageId":"1364563210-28813-1-git-send-email-pclouds@gmail.com","threadId":"33324","inReplyTo":null,"subject":"[PATCH] rev-list: preallocate object hash table in --all --objects","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2013-03-29T13:20:10Z","receivedAt":"2013-03-29T13:20:10Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Every time the object hash table grows, all entries must be\nrearranged. The few last times could be really expensive when the\ntable contains a lot of entries.\n\nWhen we do \"--all --objects\" we know in advance we may need to hash\nall objects. Just prepare the hash table big enough, so there won't be\nany resizing later on. The hash table is resized a couple times before\nprehash_objects() is called. But that's ok because there aren't many\nobjects by that time (unless you have tons of refs, likely tags..)\n\nOn linux-2.6.git:\n\n        before       after\nreal    0m34.402s    0m33.288s\nuser    0m34.111s    0m32.863s\nsys     0m0.205s     0m0.352s\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n object.c   | 21 +++++++++++++++++++--\n object.h   |  2 ++\n revision.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++----\n 3 files changed, 76 insertions(+), 6 deletions(-)\n\ndiff --git a/object.c b/object.c\nindex 20703f5..bcfd2c6 100644\n--- a/object.c\n+++ b/object.c\n@@ -88,10 +88,10 @@ struct object *lookup_object(const unsigned char *sha1)\n \treturn obj;\n }\n \n-static void grow_object_hash(void)\n+void grow_object_hash_to(unsigned long nr)\n {\n \tint i;\n-\tint new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;\n+\tint new_hash_size = nr < 32 ? 32 : nr * 2;\n \tstruct object **new_hash;\n \n \tnew_hash = xcalloc(new_hash_size, sizeof(struct object *));\n@@ -106,6 +106,11 @@ static void grow_object_hash(void)\n \tobj_hash_size = new_hash_size;\n }\n \n+static void grow_object_hash(void)\n+{\n+\tgrow_object_hash_to(obj_hash_size);\n+}\n+\n void *create_object(const unsigned char *sha1, int type, void *o)\n {\n \tstruct object *obj = o;\n@@ -307,3 +312,15 @@ void clear_object_flags(unsigned flags)\n \t\t\tobj->flags &= ~flags;\n \t}\n }\n+\n+int has_object_flags(unsigned flags)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i < obj_hash_size; i++) {\n+\t\tstruct object *obj = obj_hash[i];\n+\t\tif (obj && (obj->flags & flags))\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\ndiff --git a/object.h b/object.h\nindex 97d384b..1e8fee8 100644\n--- a/object.h\n+++ b/object.h\n@@ -52,6 +52,7 @@ extern struct object *get_indexed_object(unsigned int);\n  */\n struct object *lookup_object(const unsigned char *sha1);\n \n+extern void grow_object_hash_to(unsigned long nr);\n extern void *create_object(const unsigned char *sha1, int type, void *obj);\n \n /*\n@@ -87,6 +88,7 @@ void add_object_array(struct object *obj, const char *name, struct object_array\n void add_object_array_with_mode(struct object *obj, const char *name, struct object_array *array, unsigned mode);\n void object_array_remove_duplicates(struct object_array *);\n \n+int has_object_flags(unsigned flags);\n void clear_object_flags(unsigned flags);\n \n #endif /* OBJECT_H */\ndiff --git a/revision.c b/revision.c\nindex 71e62d8..f9ea2d1 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1665,8 +1665,9 @@ static int for_each_good_bisect_ref(const char *submodule, each_ref_fn fn, void\n }\n \n static int handle_revision_pseudo_opt(const char *submodule,\n-\t\t\t\tstruct rev_info *revs,\n-\t\t\t\tint argc, const char **argv, int *flags)\n+\t\t\t\t      struct rev_info *revs,\n+\t\t\t\t      int argc, const char **argv,\n+\t\t\t\t      int *flags, int *all)\n {\n \tconst char *arg = argv[0];\n \tconst char *optarg;\n@@ -1685,6 +1686,7 @@ static int handle_revision_pseudo_opt(const char *submodule,\n \tif (!strcmp(arg, \"--all\")) {\n \t\thandle_refs(submodule, revs, *flags, for_each_ref_submodule);\n \t\thandle_refs(submodule, revs, *flags, head_ref_submodule);\n+\t\t*all = 1;\n \t} else if (!strcmp(arg, \"--branches\")) {\n \t\thandle_refs(submodule, revs, *flags, for_each_branch_ref_submodule);\n \t} else if (!strcmp(arg, \"--bisect\")) {\n@@ -1738,6 +1740,49 @@ static int handle_revision_pseudo_opt(const char *submodule,\n \treturn 1;\n }\n \n+static void preallocate_hash_table(void)\n+{\n+\tunsigned long cnt = 0;\n+\tstruct packed_git *p;\n+\tint i;\n+\n+\tif (has_object_flags(UNINTERESTING))\n+\t\t/*\n+\t\t * nope this is not simply \"--all --objects\"\n+\t\t * not worth preallocation.\n+\t\t */\n+\t\treturn;\n+\n+\tfor (i = 0; i < 256; i++) {\n+\t\tstruct dirent *ent;\n+\t\tDIR *d = opendir(git_path(\"objects/%02x\", i));\n+\t\tif (!d)\n+\t\t\tcontinue;\n+\t\twhile ((ent = readdir(d)) != NULL)\n+\t\t\t/*\n+\t\t\t * We only worry about insufficient size which\n+\t\t\t * leads to expensive growths later on. A few\n+\t\t\t * extra slots in the hash table would not hurt.\n+\t\t\t */\n+\t\t\tcnt++;\n+\t\tclosedir(d);\n+\t}\n+\n+\tif (!packed_git)\n+\t\tprepare_packed_git();\n+\n+\tfor (p = packed_git; p; p = p->next) {\n+\t\tif (!p->pack_local)\n+\t\t\t/* this may lead to extra growths later */\n+\t\t\tcontinue;\n+\t\tif (open_pack_index(p))\n+\t\t\tcontinue;\n+\t\tcnt += p->num_objects;\n+\t}\n+\n+\tgrow_object_hash_to(cnt);\n+}\n+\n /*\n  * Parse revision information, filling in the \"rev_info\" structure,\n  * and removing the used arguments from the argument list.\n@@ -1750,6 +1795,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s\n \tint i, flags, left, seen_dashdash, read_from_stdin, got_rev_arg = 0, revarg_opt;\n \tstruct cmdline_pathspec prune_data;\n \tconst char *submodule = NULL;\n+\tint all = 0;\n \n \tmemset(&prune_data, 0, sizeof(prune_data));\n \tif (opt)\n@@ -1785,8 +1831,9 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s\n \t\t\tint opts;\n \n \t\t\topts = handle_revision_pseudo_opt(submodule,\n-\t\t\t\t\t\trevs, argc - i, argv + i,\n-\t\t\t\t\t\t&flags);\n+\t\t\t\t\t\t\t  revs, argc - i,\n+\t\t\t\t\t\t\t  argv + i,\n+\t\t\t\t\t\t\t  &flags, &all);\n \t\t\tif (opts > 0) {\n \t\t\t\ti += opts - 1;\n \t\t\t\tcontinue;\n@@ -1856,6 +1903,10 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s\n \t\t\t      get_pathspec(revs->prefix, prune_data.path));\n \t}\n \n+\tif (all && revs->tag_objects &&\n+\t    revs->tree_objects && revs->blob_objects)\n+\t\tpreallocate_hash_table();\n+\n \tif (revs->def == NULL)\n \t\trevs->def = opt ? opt->def : NULL;\n \tif (opt && opt->tweak)\n-- \n1.8.2.83.gc99314b\n"},{"id":"212621","messageId":"20130329151208.GA22744@sigill.intra.peff.net","threadId":"33324","inReplyTo":"1364563210-28813-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] rev-list: preallocate object hash table in --all --objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-03-29T15:12:08Z","receivedAt":"2013-03-29T15:12:08Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 29, 2013 at 08:20:10PM +0700, Nguyen Thai Ngoc Duy wrote:\n\n> Every time the object hash table grows, all entries must be\n> rearranged. The few last times could be really expensive when the\n> table contains a lot of entries.\n> \n> When we do \"--all --objects\" we know in advance we may need to hash\n> all objects. Just prepare the hash table big enough, so there won't be\n> any resizing later on. The hash table is resized a couple times before\n> prehash_objects() is called. But that's ok because there aren't many\n> objects by that time (unless you have tons of refs, likely tags..)\n> \n> On linux-2.6.git:\n> \n>         before       after\n> real    0m34.402s    0m33.288s\n> user    0m34.111s    0m32.863s\n> sys     0m0.205s     0m0.352s\n\nThis feels weirdly specific, and like we should just be tuning our hash\ntable growth better. You show a 3.2% speedup here. I was able to get a\n2.8% speedup just by doing this:\n\ndiff --git a/object.c b/object.c\nindex 20703f5..8e5e12c 100644\n--- a/object.c\n+++ b/object.c\n@@ -91,7 +91,7 @@ static void grow_object_hash(void)\n static void grow_object_hash(void)\n {\n \tint i;\n-\tint new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;\n+\tint new_hash_size = obj_hash_size < 32 ? 32 : 3 * obj_hash_size;\n \tstruct object **new_hash;\n \n \tnew_hash = xcalloc(new_hash_size, sizeof(struct object *));\n\nIt might be worth trying to figure out what the optimium growth rate is\nfirst, which would help this use case and others. With less fragile\ncode.\n\n-Peff\n"},{"id":"212622","messageId":"CACsJy8AXUUz3=-pWeK-y0va-=d_-aCeNgH8rAtMZdq0PE+X97g@mail.gmail.com","threadId":"33324","inReplyTo":"20130329151208.GA22744@sigill.intra.peff.net","subject":"Re: [PATCH] rev-list: preallocate object hash table in --all --objects","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-03-29T15:29:52Z","receivedAt":"2013-03-29T15:29:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Mar 29, 2013 at 10:12 PM, Jeff King <peff@peff.net> wrote:\n> This feels weirdly specific, and like we should just be tuning our hash\n> table growth better. You show a 3.2% speedup here. I was able to get a\n> 2.8% speedup just by doing this:\n\nIt also uses a lot more memory. 5.8m entries for \".. * 2\" and 8.8m for\n\"... * 3\". Probably no big deal for modern machines..\n\n> It might be worth trying to figure out what the optimium growth rate is\n> first, which would help this use case and others. With less fragile\n> code.\n\nAgreed. Although I think it's getting out of my domain. I'm not even\nsure how many factors are involved.\n-- \nDuy\n"},{"id":"212624","messageId":"7vsj3ep4zn.fsf@alter.siamese.dyndns.org","threadId":"33324","inReplyTo":"20130329151208.GA22744@sigill.intra.peff.net","subject":"Re: [PATCH] rev-list: preallocate object hash table in --all --objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-03-29T16:04:12Z","receivedAt":"2013-03-29T16:04:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> This feels weirdly specific, and like we should just be tuning our hash\n> table growth better. You show a 3.2% speedup here. I was able to get a\n> 2.8% speedup just by doing this:\n>\n> diff --git a/object.c b/object.c\n> index 20703f5..8e5e12c 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -91,7 +91,7 @@ static void grow_object_hash(void)\n>  static void grow_object_hash(void)\n>  {\n>  \tint i;\n> -\tint new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;\n> +\tint new_hash_size = obj_hash_size < 32 ? 32 : 3 * obj_hash_size;\n>  \tstruct object **new_hash;\n>  \n>  \tnew_hash = xcalloc(new_hash_size, sizeof(struct object *));\n>\n> It might be worth trying to figure out what the optimium growth rate is\n> first, which would help this use case and others. With less fragile\n> code.\n\nI agree with the general principle to avoid heuristics that is too\nspecific to the use case.  Thanks.\n"},{"id":"212647","messageId":"20130329203200.GA32155@sigill.intra.peff.net","threadId":"33324","inReplyTo":"CACsJy8AXUUz3=-pWeK-y0va-=d_-aCeNgH8rAtMZdq0PE+X97g@mail.gmail.com","subject":"Re: [PATCH] rev-list: preallocate object hash table in --all --objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-03-29T20:32:00Z","receivedAt":"2013-03-29T20:32:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 29, 2013 at 10:29:52PM +0700, Nguyen Thai Ngoc Duy wrote:\n\n> On Fri, Mar 29, 2013 at 10:12 PM, Jeff King <peff@peff.net> wrote:\n> > This feels weirdly specific, and like we should just be tuning our hash\n> > table growth better. You show a 3.2% speedup here. I was able to get a\n> > 2.8% speedup just by doing this:\n> \n> It also uses a lot more memory. 5.8m entries for \".. * 2\" and 8.8m for\n> \"... * 3\". Probably no big deal for modern machines..\n\nYeah, it will use more, but it's not going to waste more than half again\nmore than we already were.\n\n> > It might be worth trying to figure out what the optimium growth rate is\n> > first, which would help this use case and others. With less fragile\n> > code.\n> \n> Agreed. Although I think it's getting out of my domain. I'm not even\n> sure how many factors are involved.\n\nThere's the load factor that causes us to grow, and the growth factor of\nhow aggressively we expand when we do need to grow. I fooled around with\na few numbers and the patch below seemed to give good results. Probably\nvarying both numbers over a range and graphing the result would make\nsense, but I don't have time to do it at the moment (each run takes a\nwhile, if I do best-of-five).\n\n[before]\nreal    0m46.255s\nuser    0m45.812s\nsys     0m0.276s\n\n[after]\nreal    0m43.729s\nuser    0m43.204s\nsys     0m0.356s\n\ndiff --git a/object.c b/object.c\nindex 20703f5..c3be886 100644\n--- a/object.c\n+++ b/object.c\n@@ -91,7 +91,7 @@ static void grow_object_hash(void)\n static void grow_object_hash(void)\n {\n \tint i;\n-\tint new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;\n+\tint new_hash_size = obj_hash_size < 32 ? 32 : obj_hash_size * 5/2;\n \tstruct object **new_hash;\n \n \tnew_hash = xcalloc(new_hash_size, sizeof(struct object *));\n@@ -116,7 +116,7 @@ void *create_object(const unsigned char *sha1, int type, void *o)\n \tobj->flags = 0;\n \thashcpy(obj->sha1, sha1);\n \n-\tif (obj_hash_size - 1 <= nr_objs * 2)\n+\tif (nr_objs + 1 > obj_hash_size * 1/3)\n \t\tgrow_object_hash();\n \n \tinsert_obj_hash(obj, obj_hash, obj_hash_size);\n"},{"id":"212808","messageId":"20130401183345.GA2779@sigill.intra.peff.net","threadId":"33324","inReplyTo":"20130329203200.GA32155@sigill.intra.peff.net","subject":"Re: [PATCH] rev-list: preallocate object hash table in --all --objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-04-01T18:33:45Z","receivedAt":"2013-04-01T18:33:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 29, 2013 at 04:32:00PM -0400, Jeff King wrote:\n\n> > Agreed. Although I think it's getting out of my domain. I'm not even\n> > sure how many factors are involved.\n> \n> There's the load factor that causes us to grow, and the growth factor of\n> how aggressively we expand when we do need to grow. I fooled around with\n> a few numbers and the patch below seemed to give good results. Probably\n> varying both numbers over a range and graphing the result would make\n> sense, but I don't have time to do it at the moment (each run takes a\n> while, if I do best-of-five).\n\nSo I did that. I'm still not quite sure what it means, but here's the\ndata (all times are best-of-five wall-clock times to complete `git\nrev-list --objects --all` on linux-2.6, in seconds).\n\n Load  | Growth |\nFactor | Factor | Time\n-------+--------+--------\n 0.17  |  1.50  |  44.104\n 0.17  |  2.00  |  43.373\n 0.17  |  2.50  |  45.662\n 0.17  |  3.00  |  44.909\n 0.17  |  3.50  |  42.733\n 0.17  |  4.00  |  42.659\n 0.33  |  1.50  |  44.806\n 0.33  |  2.00  |  44.876\n 0.33  |  2.50  |  47.991\n 0.33  |  3.00  |  44.940\n 0.33  |  3.50  |  43.615\n 0.33  |  4.00  |  43.707\n 0.50  |  1.50  |  46.459\n 0.50  |  2.00  |  45.872\n 0.50  |  2.50  |  47.844\n 0.50  |  3.00  |  47.466\n 0.50  |  3.50  |  44.063\n 0.50  |  4.00  |  43.807\n 0.67  |  1.50  |  50.178\n 0.67  |  2.00  |  48.692\n 0.67  |  2.50  |  50.458\n 0.67  |  3.00  |  47.307\n 0.67  |  3.50  |  45.114\n 0.67  |  4.00  |  45.114\n 0.83  |  1.50  |  54.337\n 0.83  |  2.00  |  51.286\n 0.83  |  2.50  |  50.110\n 0.83  |  3.00  |  47.736\n 0.83  |  3.50  |  47.617\n 0.83  |  4.00  |  47.282\n\nI'm attaching a graph of the results, too, which I think makes it easier\nto look at (and you probably want to look at it now, or the rest of this\nwon't make any sense). The interesting things I see are:\n\n  1. The benefits of increasing the growth factor flatten out around\n     3x-4x.\n\n  2. Obviously having a smaller load factor increases efficiency.\n\n  3. Increasing the growth factor compensates for a worse load factor\n     (e.g., a growth rate of 3 performs about the same with a load\n     factor of 1/2 to 5/6).\n\nIt makes sense that one could compensate for the other. Our pattern of\ngrowth for the hash is to add a lot at first, and then more and more\nfrequently hit objects that we have already seen (because the number we\nhave seen is going up). So we do many more queries on the hash when it\nis at size X than when it is at X/2.\n\nOr another way of thinking about it is: it doesn't matter that much how\nwe get there, but when we reach our final size (where most of our\nlookups are going to happen), how crowded is the hash table (i.e., how\nmany times are we going to see collisions and have to do extra\nhashcmps?).\n\nWith a load factor of 0.17, we know it never goes over that. But with a\nconfigured max load factor of 0.5, right after we double, we know the\nload factor is now only 0.25; it will rise again from there, but not\nnecessarily even back up to 0.5 (if we never allocate again).\n\nAnd I think that explains the weird spikes. Why, when we have a load\nfactor of 0.33, do we perform worse with a growth factor of 2.5 than\nwith 2? The hash should be more sparse. And I think the answer is: for\nthe number of objects we have, it so happens that the growth factor of 2\ncauses us to end up with a more sparsely populated table, and we see\na lot of queries on the table in that state. Whereas with 2.5, we do\nfewer growth iterations, but end in a state that is slightly less\noptimal.\n\nGiven this conjecture, I added an atexit() to determine the final state\nof the hash. Here are the final states for a max load factor of 0.33,\nand a few growth rates:\n\ngrow | objects | objects  | final\nrate | used    | alloc    | load\n-----+---------+----------+------\n2    | 3005531 | 16777216 | 0.179\n2.5  | 3005531 | 11920105 | 0.252\n3    | 3005531 | 17006112 | 0.177\n\nI think that supports the conjecture; the final load factor is much\nworse with 2.5 than with 2 or 3. Not for any good reason; it just\nhappens to match the growth pattern we see given the number of objects\nwe have. Of course the tradeoff is that we waste more memory (37M with\n8-byte pointers).\n\nSo what should we do? I don't think increasing the growth rate makes\nsense. Whether we end up helping or hurting is somewhat random, as it is\nreally all about where we end up in terms of the final load factor,\nwhere most of our queries happen.\n\nWe would do much better to tweak the max load factor, which ensures that\nthe final load factor (and the intermediate ones) is below a certain\nvalue. Of course that comes at the cost of wasted memory. Moving from\nthe current load factor of 0.5 to 0.33 saves us about 1 second\nprocessing time. But it means our memory usage jumps (in this case, it\ndoubles from 64M to 128M). So there are small savings to be had, but\nbigger memory losses; I guess the question is how much we would care\nabout those memory losses (on a modern machine, using an extra 64M for\nthe kernel repo is not that big a deal).\n\nAnd of course the other thing to do is to use a slotting mechanism that\nreduces conflicts. Junio experimented with cuckoo hashing, and after\nreading his attempt, I tried quadratic stepping. As I recall, neither\nexperiment yielded anything impressive (though I may simply have looked\nat 1s speedup and considered it \"not impressive\"; I don't remember).\n\nSo I dunno. We are close to as good as it will get, I think. We might\nsteal a few percent by making a memory tradeoff, or doing something\nclever with the hash stepping (cuckoo, quadratic, etc). But those are\nbig-ish jumps in complexity or memory use for not much gain.\n\nMy test harness patch is below in case anybody wants to play with.\n\n-Peff\n\n---\ndiff --git a/object.c b/object.c\nindex 20703f5..dd04009 100644\n--- a/object.c\n+++ b/object.c\n@@ -88,12 +88,26 @@ static void grow_object_hash(void)\n \treturn obj;\n }\n \n+static void print_hash_size(void)\n+{\n+\tfprintf(stderr, \"final hash size is %d/%d = %f\\n\",\n+\t\tnr_objs, obj_hash_size, ((double)nr_objs)/obj_hash_size);\n+}\n+\n static void grow_object_hash(void)\n {\n+\tstatic int rate;\n \tint i;\n-\tint new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;\n+\tint new_hash_size;\n \tstruct object **new_hash;\n \n+\tif (!rate) {\n+\t\t/* in units of 1/2 to give more resolution and avoid floats */\n+\t\trate = atoi(getenv(\"GIT_GROW_RATE\"));\n+\t\tatexit(print_hash_size);\n+\t}\n+\n+\tnew_hash_size = obj_hash_size < 32 ? 32 : obj_hash_size * rate / 2;\n \tnew_hash = xcalloc(new_hash_size, sizeof(struct object *));\n \tfor (i = 0; i < obj_hash_size; i++) {\n \t\tstruct object *obj = obj_hash[i];\n@@ -109,6 +123,7 @@ void *create_object(const unsigned char *sha1, int type, void *o)\n void *create_object(const unsigned char *sha1, int type, void *o)\n {\n \tstruct object *obj = o;\n+\tstatic int factor;\n \n \tobj->parsed = 0;\n \tobj->used = 0;\n@@ -116,7 +131,11 @@ void *create_object(const unsigned char *sha1, int type, void *o)\n \tobj->flags = 0;\n \thashcpy(obj->sha1, sha1);\n \n-\tif (obj_hash_size - 1 <= nr_objs * 2)\n+\t/* in units of 1/6 to give more resolution and avoid floats */\n+\tif (!factor)\n+\t\tfactor = atoi(getenv(\"GIT_LOAD_FACTOR\"));\n+\n+\tif (nr_objs + 1 > obj_hash_size * factor / 6)\n \t\tgrow_object_hash();\n \n \tinsert_obj_hash(obj, obj_hash, obj_hash_size);\n"}]}