git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 4/7] delta_base_cache: use list.h for LRU

From
Jeff King <peff@peff.net>
Date
Aug 22, 2016, 21:59 UTC
Message-ID
<20160822215942.nm2jcrkyib7acr4u@sigill.intra.peff.net>
In-Reply-To
<20160822215725.qdikfcaz3smhulau@sigill.intra.peff.net>

We keep an LRU list of entries for when we need to drop something from an over-full cache. The list is implemented as a circular doubly-linked list, which is exactly what list.h provides. We can save a few lines by using the list.h macros and functions. More importantly, this makes the code easier to follow, as the reader sees explicit concepts like "list_add_tail()" instead of pointer manipulation.

As a bonus, the list_entry() macro lets us place the lru pointers anywhere inside the delta_base_cache_entry struct (as opposed to just casting the pointer, which requires it at the front of the struct). This will be useful in later patches when we need to place other items at the front of the struct (e.g., our hashmap implementation requires this).

Signed-off-by: Jeff King <peff@peff.net>
---
I think the result is much nicer, but I found list_entry() a little
disappointing, because we lack typeof(). So you are stuck writing:
  struct delta_base_cache_entry *f =
    list_entry(p, struct delta_base_cache_entry, lru);
I waffled on adding something like:
  LIST_ITEM(struct delta_bas_cache_entry, f, p, lru);
to declare "f" as above. But it's getting rather magical and un-C-like.
 sha1_file.c | 38 ++++++++++++++++----------------------
 1 file changed, 16 insertions(+), 22 deletions(-)
diff --git a/sha1_file.c b/sha1_file.c
index 8264b39..c02e785 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -24,6 +24,7 @@
 #include "streaming.h"
 #include "dir.h"
 #include "mru.h"
+#include "list.h"
 
 #ifndef O_NOATIME
 #if defined(__linux__) && (defined(__i386__) || defined(__PPC__))
@@ -2077,13 +2078,10 @@ static void *unpack_compressed_entry(struct packed_git *p,
 
 static size_t delta_base_cached;
 
-static struct delta_base_cache_lru_list {
-	struct delta_base_cache_lru_list *prev;
-	struct delta_base_cache_lru_list *next;
-} delta_base_cache_lru = { &delta_base_cache_lru, &delta_base_cache_lru };
+static LIST_HEAD(delta_base_cache_lru);
 
 static struct delta_base_cache_entry {
-	struct delta_base_cache_lru_list lru;
+	struct list_head lru;
 	void *data;
 	struct packed_git *p;
 	off_t base_offset;
@@ -2128,8 +2126,7 @@ static int in_delta_base_cache(struct packed_git *p, off_t base_offset)
 static void detach_delta_base_cache_entry(struct delta_base_cache_entry *ent)
 {
 	ent->data = NULL;
-	ent->lru.next->prev = ent->lru.prev;
-	ent->lru.prev->next = ent->lru.next;
+	list_del(&ent->lru);
 	delta_base_cached -= ent->size;
 }
 
@@ -2168,24 +2165,24 @@ static void add_delta_base_cache(struct packed_git *p, off_t base_offset,
 {
 	unsigned long hash = pack_entry_hash(p, base_offset);
 	struct delta_base_cache_entry *ent = delta_base_cache + hash;
-	struct delta_base_cache_lru_list *lru;
+	struct list_head *lru;
 
 	release_delta_base_cache(ent);
 	delta_base_cached += base_size;
 
-	for (lru = delta_base_cache_lru.next;
-	     delta_base_cached > delta_base_cache_limit
-	     && lru != &delta_base_cache_lru;
-	     lru = lru->next) {
-		struct delta_base_cache_entry *f = (void *)lru;
+	list_for_each(lru, &delta_base_cache_lru) {
+		struct delta_base_cache_entry *f =
+			list_entry(lru, struct delta_base_cache_entry, lru);
+		if (delta_base_cached <= delta_base_cache_limit)
+			break;
 		if (f->type == OBJ_BLOB)
 			release_delta_base_cache(f);
 	}
-	for (lru = delta_base_cache_lru.next;
-	     delta_base_cached > delta_base_cache_limit
-	     && lru != &delta_base_cache_lru;
-	     lru = lru->next) {
-		struct delta_base_cache_entry *f = (void *)lru;
+	list_for_each(lru, &delta_base_cache_lru) {
+		struct delta_base_cache_entry *f =
+			list_entry(lru, struct delta_base_cache_entry, lru);
+		if (delta_base_cached <= delta_base_cache_limit)
+			break;
 		release_delta_base_cache(f);
 	}
 
@@ -2194,10 +2191,7 @@ static void add_delta_base_cache(struct packed_git *p, off_t base_offset,
 	ent->type = type;
 	ent->data = base;
 	ent->size = base_size;
-	ent->lru.next = &delta_base_cache_lru;
-	ent->lru.prev = delta_base_cache_lru.prev;
-	delta_base_cache_lru.prev->next = &ent->lru;
-	delta_base_cache_lru.prev = &ent->lru;
+	list_add_tail(&ent->lru, &delta_base_cache_lru);
 }
 
 static void *read_object(const unsigned char *sha1, enum object_type *type,
-- 
2.10.0.rc1.118.ge2299eb
Previous: Junio C HamanoNext: Eric Wong
Message 10 of 17 in “tweaking the delta base cache”
  1. 0/7 tweaking the delta base cacheJeff King, Aug 22, 2016
  2. 1/7 cache_or_unpack_entry: drop keep_cache parameterJeff King, Aug 22, 2016
  3. Junio C HamanoAug 23, 2016
  4. 2/7 clear_delta_base_cache_entry: use a more descriptive nameJeff King, Aug 22, 2016
  5. Junio C HamanoAug 23, 2016
  6. 3/7 release_delta_base_cache: reuse existing detach functionJeff King, Aug 22, 2016
  7. Junio C HamanoAug 23, 2016
  8. Jeff KingAug 24, 2016
  9. Junio C HamanoAug 24, 2016
  10. 4/7 delta_base_cache: use list.h for LRUJeff King, Aug 22, 2016
  11. Eric WongAug 22, 2016
  12. Jeff KingAug 23, 2016
  13. 5/7 delta_base_cache: drop special treatment of blobsJeff King, Aug 22, 2016
  14. Junio C HamanoAug 23, 2016
  15. 6/7 delta_base_cache: use hashmap.hJeff King, Aug 22, 2016
  16. Junio C HamanoAug 23, 2016
  17. 7/7 t/perf: add basic perf tests for delta base cacheJeff King, Aug 22, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.