git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/6] introduce a commit metapack

From
Jeff King <peff@peff.net>
Date
Feb 1, 2013, 10:40 UTC
Message-ID
<20130201104010.GH30644@sigill.intra.peff.net>
In-Reply-To
<20130131110656.GA28093@lanh>
On Thu, Jan 31, 2013 at 06:06:56PM +0700, Nguyen Thai Ngoc Duy wrote:
Show 14 quoted lines
> On Wed, Jan 30, 2013 at 09:16:29PM +0700, Duy Nguyen wrote:
> > Perhaps we could store abbrev sha-1 instead of full sha-1. Nice
> > space/time trade-off.
> 
> Following the on-disk format experiment yesterday, I changed the
> format to:
> 
>  - a list a _short_ SHA-1 of cached commits
>  - a list of cache entries, each (5 uint32_t) consists of:
>    - uint32_t for the index in .idx sha-1 table to get full SHA-1 of
>      the commit
>    - uint32_t for timestamp
>    - uint32_t for tree, 1st and 2nd parents for the index in .idx
>      table

BTW, I needed the minor fixups below to silence some warnings from your patch. Here are the cold and warm cache timings I got, as compared to stock git and my implementation:

 Pack  |  Cold Revs   |  Warm Revs
-------+--------------+------------
 stock | 12.54        | 4.14
    me |  4.76 (-62%) | 0.66 (-84%)
   duy |  4.36 (-65%) | 0.55 (-86%)

Not surprising; yours is just a bit faster in terms of CPU, and even gains a little more in the cold cache case. Nice. Of course that is just gravy on top of the smaller disk usage, too. :)

---
diff --git a/commit-metapack.c b/commit-metapack.c
index c984b8e..78fd961 100644
--- a/commit-metapack.c
+++ b/commit-metapack.c
@@ -106,7 +106,7 @@ int commit_metapack(unsigned char *sha1,
 	for (p = commit_metapacks; p; p = p->next) {
 		struct commit_entry *data;
 		uint32_t p1, p2;
-		unsigned lo, hi, mi;
+		unsigned lo, hi;
 		int pos;
 
 		/* sha1_entry_pos does not work with abbreviated sha-1 */
@@ -161,7 +161,7 @@ static void get_commits(struct metapack_writer *mw,
 	struct write_cb *write_cb = (struct write_cb *)data;
 	enum object_type type = sha1_object_info(sha1, NULL);
 	struct commit *c;
-	int p1, p2;
+	int p1 = -1, p2 = -1;
 
 	if (type != OBJ_COMMIT)
 		return;
diff --git a/commit.c b/commit.c
index b326201..5b776f8 100644
--- a/commit.c
+++ b/commit.c
@@ -309,7 +309,7 @@ static int parse_commit_metapack(struct commit *item)
 
 static int parse_commit_metapack(struct commit *item)
 {
-	unsigned char *tree, *p1, *p2;
+	const unsigned char *tree, *p1, *p2;
 	uint32_t ts;
 
 	if (commit_metapack(item->object.sha1, &ts, &tree, &p1, &p2) < 0)
Previous: Duy NguyenNext: Duy Nguyen
Message 32 of 43 in “commit caching”
  1. 0/6 commit cachingJeff King, Jan 29, 2013
  2. 1/6 csum-file: make sha1write const-correctJeff King, Jan 29, 2013
  3. 2/6 strbuf: add string-chomping functionsJeff King, Jan 29, 2013
  4. Michael HaggertyJan 29, 2013
  5. Jeff KingJan 29, 2013
  6. Michael HaggertyJan 30, 2013
  7. 3/6 introduce pack metadata cache filesJeff King, Jan 29, 2013
  8. Junio C HamanoJan 29, 2013
  9. Jeff KingJan 30, 2013
  10. Duy NguyenJan 30, 2013
  11. Jeff KingJan 30, 2013
  12. 4/6 introduce a commit metapackJeff King, Jan 29, 2013
  13. Michael HaggertyJan 29, 2013
  14. Jeff KingJan 29, 2013
  15. Junio C HamanoJan 29, 2013
  16. Junio C HamanoJan 29, 2013
  17. Jeff KingJan 30, 2013
  18. Junio C HamanoJan 30, 2013
  19. Jeff KingFeb 1, 2013
  20. Junio C HamanoJan 30, 2013
  21. Shawn PearceJan 31, 2013
  22. Jeff KingFeb 1, 2013
  23. Junio C HamanoFeb 2, 2013
  24. Jeff KingJan 30, 2013
  25. Duy NguyenJan 30, 2013
  26. Jeff KingJan 30, 2013
  27. Duy NguyenJan 30, 2013
  28. Duy NguyenJan 30, 2013
  29. Duy NguyenJan 31, 2013
  30. Jeff KingFeb 1, 2013
  31. Duy NguyenFeb 2, 2013
  32. Jeff KingFeb 1, 2013
  33. Duy NguyenMar 17, 2013
  34. Jeff KingMar 18, 2013
  35. Jeff KingFeb 1, 2013
  36. 5/6 add git-metapack commandJeff King, Jan 29, 2013
  37. 6/6 commit: look up commit info in metapackJeff King, Jan 29, 2013
  38. Duy NguyenJan 30, 2013
  39. Jeff KingJan 30, 2013
  40. Duy NguyenJan 30, 2013
  41. Shawn PearceJan 31, 2013
  42. Jeff KingFeb 1, 2013
  43. Shawn PearceFeb 2, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.