git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 12/16] sha1_file: add for_each iterators for loose and packed objects

From
Jeff King <peff@peff.net>
Date
Oct 3, 2014, 20:32 UTC
Message-ID
<20141003203219.GL16293@peff.net>
In-Reply-To
<20141003202045.GA15205@peff.net>

We typically iterate over the reachable objects in a repository by starting at the tips and walking the graph. There's no easy way to iterate over all of the objects, including unreachable ones. Let's provide a way of doing so.

Signed-off-by: Jeff King <peff@peff.net>
---
 cache.h     | 11 +++++++++++
 sha1_file.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 73 insertions(+)
diff --git a/cache.h b/cache.h
index 7abe7f6..3826b4b 100644
--- a/cache.h
+++ b/cache.h
@@ -1270,6 +1270,17 @@ int for_each_loose_file_in_objdir(const char *path,
 				  each_loose_subdir_fn subdir_cb,
 				  void *data);
 
+/*
+ * Iterate over loose and packed objects in both the local
+ * repository and any alternates repositories.
+ */
+typedef int each_packed_object_fn(const unsigned char *sha1,
+				  struct packed_git *pack,
+				  uint32_t pos,
+				  void *data);
+extern int for_each_loose_object(each_loose_object_fn, void *);
+extern int for_each_packed_object(each_packed_object_fn, void *);
+
 struct object_info {
 	/* Request */
 	enum object_type *typep;
diff --git a/sha1_file.c b/sha1_file.c
index 9fdad47..d017289 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3313,3 +3313,65 @@ int for_each_loose_file_in_objdir(const char *path,
 	strbuf_release(&buf);
 	return r;
 }
+
+struct loose_alt_odb_data {
+	each_loose_object_fn *cb;
+	void *data;
+};
+
+static int loose_from_alt_odb(struct alternate_object_database *alt,
+			      void *vdata)
+{
+	struct loose_alt_odb_data *data = vdata;
+	return for_each_loose_file_in_objdir(alt->base,
+					     data->cb, NULL, NULL,
+					     data->data);
+}
+
+int for_each_loose_object(each_loose_object_fn cb, void *data)
+{
+	struct loose_alt_odb_data alt;
+	int r;
+
+	r = for_each_loose_file_in_objdir(get_object_directory(),
+					  cb, NULL, NULL, data);
+	if (r)
+		return r;
+
+	alt.cb = cb;
+	alt.data = data;
+	return foreach_alt_odb(loose_from_alt_odb, &alt);
+}
+
+int for_each_object_in_pack(struct packed_git *p, each_packed_object_fn cb, void *data)
+{
+	uint32_t i;
+	int r = 0;
+
+	for (i = 0; i < p->num_objects; i++) {
+		const unsigned char *sha1 = nth_packed_object_sha1(p, i);
+
+		if (!sha1)
+			return error("unable to get sha1 of object %u in %s",
+				     i, p->pack_name);
+
+		r = cb(sha1, p, i, data);
+		if (r)
+			break;
+	}
+	return r;
+}
+
+int for_each_packed_object(each_packed_object_fn cb, void *data)
+{
+	struct packed_git *p;
+	int r = 0;
+
+	prepare_packed_git();
+	for (p = packed_git; p; p = p->next) {
+		r = for_each_object_in_pack(p, cb, data);
+		if (r)
+			break;
+	}
+	return 0;
+}
-- 
2.1.1.566.gdb1f904
Previous: Jeff KingNext: René Scharfe
Message 36 of 58 in “make prune mtime-checking more careful”
  1. 0/16 make prune mtime-checking more carefulJeff King, Oct 3, 2014
  2. 01/16 foreach_alt_odb: propagate return value from callbackJeff King, Oct 3, 2014
  3. René ScharfeOct 3, 2014
  4. Jeff KingOct 4, 2014
  5. 02/16 isxdigit: cast input to unsigned charJeff King, Oct 3, 2014
  6. 03/16 object_array: factor out slopbuf-freeing logicJeff King, Oct 3, 2014
  7. Michael HaggertyOct 7, 2014
  8. Jeff KingOct 8, 2014
  9. Michael HaggertyOct 8, 2014
  10. Jeff KingOct 8, 2014
  11. 04/16 object_array: add a "clear" functionJeff King, Oct 3, 2014
  12. 05/16 clean up name allocation in prepare_revision_walkJeff King, Oct 3, 2014
  13. 06/16 reachable: clear pending array after walking itJeff King, Oct 3, 2014
  14. 07/16 t5304: use test_path_is_* instead of "test -f"Jeff King, Oct 3, 2014
  15. Junio C HamanoOct 3, 2014
  16. 08/16 t5304: use helper to report failure of "test foo = bar"Jeff King, Oct 3, 2014
  17. Junio C HamanoOct 3, 2014
  18. Jeff KingOct 4, 2014
  19. Michael HaggertyOct 7, 2014
  20. Junio C HamanoOct 7, 2014
  21. Jeff KingOct 7, 2014
  22. Junio C HamanoOct 7, 2014
  23. Junio C HamanoOct 7, 2014
  24. Jeff KingOct 7, 2014
  25. Junio C HamanoOct 7, 2014
  26. Michael HaggertyOct 7, 2014
  27. Jeff KingOct 8, 2014
  28. Junio C HamanoOct 8, 2014
  29. 09/16 prune: factor out loose-object directory traversalJeff King, Oct 3, 2014
  30. Junio C HamanoOct 3, 2014
  31. Jeff KingOct 4, 2014
  32. Michael HaggertyOct 7, 2014
  33. Jeff KingOct 8, 2014
  34. 10/16 count-objects: do not use xsize_t when counting object sizeJeff King, Oct 3, 2014
  35. 11/16 count-objects: use for_each_loose_file_in_objdirJeff King, Oct 3, 2014
  36. 12/16 sha1_file: add for_each iterators for loose and packed objectsJeff King, Oct 3, 2014
  37. René ScharfeOct 5, 2014
  38. Ramsay JonesOct 5, 2014
  39. 13/16 prune: keep objects reachable from recent objectsJeff King, Oct 3, 2014
  40. Junio C HamanoOct 3, 2014
  41. Jeff KingOct 4, 2014
  42. Jeff KingOct 4, 2014
  43. Junio C HamanoOct 4, 2014
  44. Michael HaggertyOct 7, 2014
  45. Jeff KingOct 8, 2014
  46. Michael HaggertyOct 8, 2014
  47. 14/16 pack-objects: refactor unpack-unreachable expiration checkJeff King, Oct 3, 2014
  48. 15/16 pack-objects: match prune logic for discarding objectsJeff King, Oct 3, 2014
  49. 16/16 write_sha1_file: freshen existing objectsJeff King, Oct 3, 2014
  50. Junio C HamanoOct 3, 2014
  51. Jeff KingOct 4, 2014
  52. René ScharfeOct 5, 2014
  53. Junio C HamanoOct 3, 2014
  54. Junio C HamanoOct 4, 2014
  55. René ScharfeOct 5, 2014
  56. Jeff KingOct 6, 2014
  57. Jeff KingOct 8, 2014
  58. Junio C HamanoOct 8, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.