git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 3/7] add generic most-recently-used list

From
Jeff King <peff@peff.net>
Date
Jul 29, 2016, 04:06 UTC
Message-ID
<20160729040659.GC22408@sigill.intra.peff.net>
In-Reply-To
<20160729040422.GA19678@sigill.intra.peff.net>

There are a few places in Git that would benefit from a fast most-recently-used cache (e.g., the list of packs, which we search linearly but would like to order based on locality). This patch introduces a generic list that can be used to store arbitrary pointers in most-recently-used order.

The implementation is just a doubly-linked list, where "marking" an item as used moves it to the front of the list. Insertion and marking are O(1), and iteration is O(n).

There's no lookup support provided; if you need fast lookups, you are better off with a different data structure in the first place.

There is also no deletion support. This would not be hard to do, but it's not necessary for handling pack structs, which are created and never removed.

Signed-off-by: Jeff King <peff@peff.net>
---
 Makefile |  1 +
 mru.c    | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 mru.h    | 45 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 96 insertions(+)
 create mode 100644 mru.c
 create mode 100644 mru.h
diff --git a/Makefile b/Makefile
index 6a13386..ad3624d 100644
--- a/Makefile
+++ b/Makefile
@@ -755,6 +755,7 @@ LIB_OBJS += merge.o
 LIB_OBJS += merge-blobs.o
 LIB_OBJS += merge-recursive.o
 LIB_OBJS += mergesort.o
+LIB_OBJS += mru.o
 LIB_OBJS += name-hash.o
 LIB_OBJS += notes.o
 LIB_OBJS += notes-cache.o
diff --git a/mru.c b/mru.c
new file mode 100644
index 0000000..9dedae0
--- /dev/null
+++ b/mru.c
@@ -0,0 +1,50 @@
+#include "cache.h"
+#include "mru.h"
+
+void mru_append(struct mru *mru, void *item)
+{
+	struct mru_entry *cur = xmalloc(sizeof(*cur));
+	cur->item = item;
+	cur->prev = mru->tail;
+	cur->next = NULL;
+
+	if (mru->tail)
+		mru->tail->next = cur;
+	else
+		mru->head = cur;
+	mru->tail = cur;
+}
+
+void mru_mark(struct mru *mru, struct mru_entry *entry)
+{
+	/* If we're already at the front of the list, nothing to do */
+	if (mru->head == entry)
+		return;
+
+	/* Otherwise, remove us from our current slot... */
+	if (entry->prev)
+		entry->prev->next = entry->next;
+	if (entry->next)
+		entry->next->prev = entry->prev;
+	else
+		mru->tail = entry->prev;
+
+	/* And insert us at the beginning. */
+	entry->prev = NULL;
+	entry->next = mru->head;
+	if (mru->head)
+		mru->head->prev = entry;
+	mru->head = entry;
+}
+
+void mru_clear(struct mru *mru)
+{
+	struct mru_entry *p = mru->head;
+
+	while (p) {
+		struct mru_entry *to_free = p;
+		p = p->next;
+		free(to_free);
+	}
+	mru->head = mru->tail = NULL;
+}
diff --git a/mru.h b/mru.h
new file mode 100644
index 0000000..42e4aea
--- /dev/null
+++ b/mru.h
@@ -0,0 +1,45 @@
+#ifndef MRU_H
+#define MRU_H
+
+/**
+ * A simple most-recently-used cache, backed by a doubly-linked list.
+ *
+ * Usage is roughly:
+ *
+ *   // Create a list.  Zero-initialization is required.
+ *   static struct mru cache;
+ *   mru_append(&cache, item);
+ *   ...
+ *
+ *   // Iterate in MRU order.
+ *   struct mru_entry *p;
+ *   for (p = cache.head; p; p = p->next) {
+ *	if (matches(p->item))
+ *		break;
+ *   }
+ *
+ *   // Mark an item as used, moving it to the front of the list.
+ *   mru_mark(&cache, p);
+ *
+ *   // Reset the list to empty, cleaning up all resources.
+ *   mru_clear(&cache);
+ *
+ * Note that you SHOULD NOT call mru_mark() and then continue traversing the
+ * list; it reorders the marked item to the front of the list, and therefore
+ * you will begin traversing the whole list again.
+ */
+
+struct mru_entry {
+	void *item;
+	struct mru_entry *prev, *next;
+};
+
+struct mru {
+	struct mru_entry *head, *tail;
+};
+
+void mru_append(struct mru *mru, void *item);
+void mru_mark(struct mru *mru, struct mru_entry *entry);
+void mru_clear(struct mru *mru);
+
+#endif /* MRU_H */
-- 
2.9.2.607.g98dce7b
Previous: Jeff KingNext: Jeff King
Message 4 of 35 in “speed up pack-objects counting with many packs”
  1. 0/7 speed up pack-objects counting with many packsJeff King, Jul 29, 2016
  2. 1/7 t/perf: add tests for many-pack scenariosJeff King, Jul 29, 2016
  3. 2/7 sha1_file: drop free_pack_by_nameJeff King, Jul 29, 2016
  4. 3/7 add generic most-recently-used listJeff King, Jul 29, 2016
  5. 4/7 find_pack_entry: replace last_found_pack with MRU cacheJeff King, Jul 29, 2016
  6. 5/7 pack-objects: break out of want_object loop earlyJeff King, Jul 29, 2016
  7. 6/7 pack-objects: compute local/ignore_pack_keep earlyJeff King, Jul 29, 2016
  8. 7/7 pack-objects: use mru list when iterating over packsJeff King, Jul 29, 2016
  9. Jeff KingJul 29, 2016
  10. Junio C HamanoJul 29, 2016
  11. Jeff KingAug 8, 2016
  12. Junio C HamanoAug 8, 2016
  13. Jeff KingAug 8, 2016
  14. Junio C HamanoAug 8, 2016
  15. Jeff KingAug 9, 2016
  16. Jeff KingAug 9, 2016
  17. Junio C HamanoAug 9, 2016
  18. Junio C HamanoAug 9, 2016
  19. 0/2 pack-objects mruJeff King, Aug 10, 2016
  20. Junio C HamanoAug 10, 2016
  21. Jeff KingAug 11, 2016
  22. 1/2 pack-objects: break delta cycles before delta-search phaseJeff King, Aug 10, 2016
  23. Junio C HamanoAug 10, 2016
  24. Jeff KingAug 11, 2016
  25. 1/2 pack-objects: break delta cycles before delta-search phaseJeff King, Aug 11, 2016
  26. Jeff KingAug 11, 2016
  27. pack-objects mruJeff King, Aug 11, 2016
  28. 1/4 provide an initializer for "struct object_info"Jeff King, Aug 11, 2016
  29. 2/4 sha1_file: make packed_object_info publicJeff King, Aug 11, 2016
  30. 3/4 pack-objects: break delta cycles before delta-search phaseJeff King, Aug 11, 2016
  31. 4/4 pack-objects: use mru list when iterating over packsJeff King, Aug 11, 2016
  32. Jeff KingAug 11, 2016
  33. Junio C HamanoAug 11, 2016
  34. Jeff KingAug 11, 2016
  35. 2/2 pack-objects: use mru list when iterating over packsJeff King, Aug 10, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.