git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 08/11] object: try 5-way cuckoo -- use all 20-bytes of SHA-1

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 11, 2011, 17:53 UTC
Message-ID
<1313085196-13249-9-git-send-email-gitster@pobox.com>
In-Reply-To
<1313085196-13249-1-git-send-email-gitster@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
../+v/ced878c983ddc9a256b22535aa8af006529ec3d0/git-pack-objects
Counting objects: 2139209, done.
31.66user 2.14system 0:33.91elapsed 99%CPU (0avgtext+0avgdata 2874176maxresident)k
0inputs+0outputs (0major+225342minor)pagefaults 0swaps
Counting objects: 2139209, done.
31.79user 2.03system 0:33.93elapsed 99%CPU (0avgtext+0avgdata 2875808maxresident)k
0inputs+0outputs (0major+225444minor)pagefaults 0swaps
Counting objects: 2139209, done.
31.80user 1.94system 0:33.84elapsed 99%CPU (0avgtext+0avgdata 2875792maxresident)k
0inputs+0outputs (0major+225443minor)pagefaults 0swaps
---
 object.c |   43 ++++++++++++++++++++++++++++++-------------
 1 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/object.c b/object.c
index 7624c48..c777520 100644
--- a/object.c
+++ b/object.c
@@ -43,27 +43,27 @@ struct object *get_indexed_object(unsigned int idx)
 	return obj_hash[idx];
 }
 
-static unsigned int hash_val(const unsigned char *sha1)
-{
-	unsigned int hash;
-	memcpy(&hash, sha1, sizeof(unsigned int));
-	return hash;
-}
-
 
-#define H1(sha1) (hash_val(sha1) % obj_hash_size)
-#define H2(sha1) (hash_val((sha1) + sizeof(unsigned int)) % obj_hash_size)
+#define H(hv,ix) ((hv[ix]) & (obj_hash_size-1))
 
 struct object *lookup_object(const unsigned char *sha1)
 {
 	struct object *obj;
+	unsigned int hashval[5];
 
 	if (!obj_hash)
 		return NULL;
 
-	if ((obj = obj_hash[H1(sha1)]) && !hashcmp(sha1, obj->sha1))
+	memcpy(hashval, sha1, 20);
+	if ((obj = obj_hash[H(hashval, 0)]) && !hashcmp(sha1, obj->sha1))
+		return obj;
+	if ((obj = obj_hash[H(hashval, 1)]) && !hashcmp(sha1, obj->sha1))
+		return obj;
+	if ((obj = obj_hash[H(hashval, 2)]) && !hashcmp(sha1, obj->sha1))
 		return obj;
-	if ((obj = obj_hash[H2(sha1)]) && !hashcmp(sha1, obj->sha1))
+	if ((obj = obj_hash[H(hashval, 3)]) && !hashcmp(sha1, obj->sha1))
+		return obj;
+	if ((obj = obj_hash[H(hashval, 4)]) && !hashcmp(sha1, obj->sha1))
 		return obj;
 	return NULL;
 }
@@ -84,13 +84,30 @@ static struct object *insert_obj_hash(struct object *obj)
 	for (loop = obj_hash_size / 2; 0 <= loop; loop--) {
 		struct object *tmp_obj;
 		unsigned int ix;
+		unsigned int hashval[5];
 
-		ix = H1(obj->sha1);
+		memcpy(hashval, obj->sha1, 20);
+		ix = H(hashval, 0);
+		if (!obj_hash[ix]) {
+			obj_hash[ix] = obj;
+			return NULL;
+		}
+		ix = H(hashval, 1);
+		if (!obj_hash[ix]) {
+			obj_hash[ix] = obj;
+			return NULL;
+		}
+		ix = H(hashval, 2);
+		if (!obj_hash[ix]) {
+			obj_hash[ix] = obj;
+			return NULL;
+		}
+		ix = H(hashval, 3);
 		if (!obj_hash[ix]) {
 			obj_hash[ix] = obj;
 			return NULL;
 		}
-		ix = H2(obj->sha1);
+		ix = H(hashval, 4);
 		tmp_obj = obj_hash[ix];
 		obj_hash[ix] = obj;
 		if (!tmp_obj)
-- 
1.7.6.433.g1421f
Previous: Junio C HamanoNext: Junio C Hamano
Message 9 of 15 in “Micro-optimizing lookup_object()”
  1. 00/11 Micro-optimizing lookup_object()Junio C Hamano, Aug 11, 2011
  2. 01/11 object.c: code movement for readabilityJunio C Hamano, Aug 11, 2011
  3. 02/11 object.c: remove duplicated code for object hashingJunio C Hamano, Aug 11, 2011
  4. 03/11 pack-objects --count-onlyJunio C Hamano, Aug 11, 2011
  5. 04/11 object: next_size() helper for readabilityJunio C Hamano, Aug 11, 2011
  6. 05/11 object hash: we know the table size is a power of twoJunio C Hamano, Aug 11, 2011
  7. 06/11 object: growing the hash-table more aggressively does not help muchJunio C Hamano, Aug 11, 2011
  8. 07/11 object: try naive cuckoo hashingJunio C Hamano, Aug 11, 2011
  9. 08/11 object: try 5-way cuckoo -- use all 20-bytes of SHA-1Junio C Hamano, Aug 11, 2011
  10. 09/11 object: try 4-way cuckooJunio C Hamano, Aug 11, 2011
  11. 10/11 object: try 3-way cuckooJunio C Hamano, Aug 11, 2011
  12. 11/11 object: try 2-way cuckoo againJunio C Hamano, Aug 11, 2011
  13. 12/11 object.c: make object hash implementation more opaqueJunio C Hamano, Aug 11, 2011
  14. git_checkattr() is inefficient when repeated [Re: [PATCH 00/11] Micro-optimizing lookup_object()]Thomas Rast, Aug 12, 2011
  15. Junio C HamanoAug 15, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.