git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 5/4] pack v4: convert v4 tree to canonical format if found in base cache

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Sep 12, 2013, 13:29 UTC
Message-ID
<1378992550-11771-1-git-send-email-pclouds@gmail.com>
In-Reply-To
<1378982284-7848-1-git-send-email-pclouds@gmail.com>

"git log --stat -10000 v1.4.8 >/dev/null" takes 13s with v4 (8s with v2). Of course we could do better when v4-aware tree-diff interface is in place..

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Oops.. forgot this and broke git.
 Another option is change cache_or_unpack_entry() to force
 OBJ_PV4_TREE to go through unpack_entry(), then update pv4_get_tree() to
 lookup the base cache at the first decode_entries() call. Right now
 it does not (hdr == 0) so we need more processing.
 packv4-parse.c | 22 ++++++++++++++++++++++
 packv4-parse.h |  2 ++
 sha1_file.c    | 11 +++++++++++
 3 files changed, 35 insertions(+)
diff --git a/packv4-parse.c b/packv4-parse.c
index b8855b0..448c91e 100644
--- a/packv4-parse.c
+++ b/packv4-parse.c
@@ -461,6 +461,10 @@ static int decode_entries(struct packed_git *p, struct pack_window **w_curs,
 	avail -= scp - src;
 	src = scp;
 
+	/* special case for pv4_cached_tree_to_canonical() */
+	if (!count && cached)
+		count = nb_entries;
+
 	while (count) {
 		unsigned int what;
 
@@ -648,3 +652,21 @@ unsigned long pv4_unpack_object_header_buffer(const unsigned char *base,
 	*sizep = val >> 4;
 	return cp - base;
 }
+
+/* offset must already be cached! */
+void *pv4_cached_tree_to_canonical(struct packed_git *p, off_t offset,
+				   unsigned long size)
+{
+	int ret;
+	unsigned char *dst, *dcp;
+	unsigned char *v4_dstp = NULL;
+	dst = xmallocz(size);
+	dcp = dst;
+	ret = decode_entries(p, NULL, offset, 0, 0,
+			     &dcp, &size, &v4_dstp, NULL, NULL, 1);
+	if (ret < 0 || size != 0) {
+		free(dst);
+		return NULL;
+	}
+	return dst;
+}
diff --git a/packv4-parse.h b/packv4-parse.h
index f584c31..ad21e19 100644
--- a/packv4-parse.h
+++ b/packv4-parse.h
@@ -24,5 +24,7 @@ void *pv4_get_commit(struct packed_git *p, struct pack_window **w_curs,
 void *pv4_get_tree(struct packed_git *p, struct pack_window **w_curs,
 		   off_t offset, unsigned long size,
 		   void **v4_data, unsigned long *v4_size);
+void *pv4_cached_tree_to_canonical(struct packed_git *p, off_t offset,
+				   unsigned long size);
 
 #endif
diff --git a/sha1_file.c b/sha1_file.c
index 82570be..0944ef6 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -2000,6 +2000,17 @@ static void *cache_or_unpack_entry(struct packed_git *p, off_t base_offset,
 	if (!eq_delta_base_cache_entry(ent, p, base_offset))
 		return unpack_entry(p, base_offset, type, base_size);
 
+	if (ent->type == OBJ_PV4_TREE) {
+		ret = pv4_cached_tree_to_canonical(p, base_offset, ent->size);
+		if (!ret)
+			return NULL;
+		if (!keep_cache)
+			clear_delta_base_cache_entry(ent);
+		*type = OBJ_TREE;
+		*base_size = ent->size;
+		return ret;
+	}
+
 	ret = ent->data;
 
 	if (!keep_cache)
-- 
1.8.2.83.gc99314b
Previous: Nguyễn Thái Ngọc Duy
Message 12 of 12 in “pack v4: avoid strlen() in tree_entry_prefix”
  1. 1/4 pack v4: avoid strlen() in tree_entry_prefixNguyễn Thái Ngọc Duy, Sep 12, 2013
  2. 2/4 pack v4: add v4_size to struct delta_base_cache_entryNguyễn Thái Ngọc Duy, Sep 12, 2013
  3. Nicolas PitreSep 13, 2013
  4. Duy NguyenSep 13, 2013
  5. Nicolas PitreSep 14, 2013
  6. Nicolas PitreSep 14, 2013
  7. Duy NguyenSep 15, 2013
  8. Nicolas PitreSep 16, 2013
  9. Duy NguyenSep 16, 2013
  10. 3/4 pack v4: cache flattened v4 trees in delta base cacheNguyễn Thái Ngọc Duy, Sep 12, 2013
  11. 4/4 pack v4: make use of cached v4 trees when unpackingNguyễn Thái Ngọc Duy, Sep 12, 2013
  12. 5/4 pack v4: convert v4 tree to canonical format if found in base cacheNguyễn Thái Ngọc Duy, Sep 12, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.