git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 13/25] sha1_file: add for_each iterators for loose and packed objects

From
Jeff King <peff@peff.net>
Date
Oct 15, 2014, 22:41 UTC
Message-ID
<20141015224121.GM25630@peff.net>
In-Reply-To
<20141015223244.GA25368@peff.net>

We typically iterate over the reachable objects in a repository by starting at the tips and walking the graph. There's no easy way to iterate over all of the objects, including unreachable ones. Let's provide a way of doing so.

Signed-off-by: Jeff King <peff@peff.net>
---
 cache.h     | 11 +++++++++++
 sha1_file.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 73 insertions(+)
diff --git a/cache.h b/cache.h
index 8ffefaa..d24dd32 100644
--- a/cache.h
+++ b/cache.h
@@ -1254,6 +1254,17 @@ int for_each_loose_file_in_objdir(const char *path,
 				  each_loose_subdir_fn subdir_cb,
 				  void *data);
 
+/*
+ * Iterate over loose and packed objects in both the local
+ * repository and any alternates repositories.
+ */
+typedef int each_packed_object_fn(const unsigned char *sha1,
+				  struct packed_git *pack,
+				  uint32_t pos,
+				  void *data);
+extern int for_each_loose_object(each_loose_object_fn, void *);
+extern int for_each_packed_object(each_packed_object_fn, void *);
+
 struct object_info {
 	/* Request */
 	enum object_type *typep;
diff --git a/sha1_file.c b/sha1_file.c
index a20240b..eb410a2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3349,3 +3349,65 @@ int for_each_loose_file_in_objdir(const char *path,
 	strbuf_release(&buf);
 	return r;
 }
+
+struct loose_alt_odb_data {
+	each_loose_object_fn *cb;
+	void *data;
+};
+
+static int loose_from_alt_odb(struct alternate_object_database *alt,
+			      void *vdata)
+{
+	struct loose_alt_odb_data *data = vdata;
+	return for_each_loose_file_in_objdir(alt->base,
+					     data->cb, NULL, NULL,
+					     data->data);
+}
+
+int for_each_loose_object(each_loose_object_fn cb, void *data)
+{
+	struct loose_alt_odb_data alt;
+	int r;
+
+	r = for_each_loose_file_in_objdir(get_object_directory(),
+					  cb, NULL, NULL, data);
+	if (r)
+		return r;
+
+	alt.cb = cb;
+	alt.data = data;
+	return foreach_alt_odb(loose_from_alt_odb, &alt);
+}
+
+static int for_each_object_in_pack(struct packed_git *p, each_packed_object_fn cb, void *data)
+{
+	uint32_t i;
+	int r = 0;
+
+	for (i = 0; i < p->num_objects; i++) {
+		const unsigned char *sha1 = nth_packed_object_sha1(p, i);
+
+		if (!sha1)
+			return error("unable to get sha1 of object %u in %s",
+				     i, p->pack_name);
+
+		r = cb(sha1, p, i, data);
+		if (r)
+			break;
+	}
+	return r;
+}
+
+int for_each_packed_object(each_packed_object_fn cb, void *data)
+{
+	struct packed_git *p;
+	int r = 0;
+
+	prepare_packed_git();
+	for (p = packed_git; p; p = p->next) {
+		r = for_each_object_in_pack(p, cb, data);
+		if (r)
+			break;
+	}
+	return r;
+}
-- 
2.1.2.596.g7379948
Previous: Jeff KingNext: Jeff King
Message 20 of 48 in “prune-safety”
  1. 0/25 prune-safetyJeff King, Oct 15, 2014
  2. 01/25 foreach_alt_odb: propagate return value from callbackJeff King, Oct 15, 2014
  3. 02/25 isxdigit: cast input to unsigned charJeff King, Oct 15, 2014
  4. Junio C HamanoOct 16, 2014
  5. 03/25 object_array: factor out slopbuf-freeing logicJeff King, Oct 15, 2014
  6. Junio C HamanoOct 16, 2014
  7. git-bundle rev handling and de-dupingJeff King, Oct 17, 2014
  8. Philip OakleyOct 17, 2014
  9. Junio C HamanoOct 17, 2014
  10. 04/25 object_array: add a "clear" functionJeff King, Oct 15, 2014
  11. 05/25 clean up name allocation in prepare_revision_walkJeff King, Oct 15, 2014
  12. 06/25 reachable: use traverse_commit_list instead of custom walkJeff King, Oct 15, 2014
  13. Junio C HamanoOct 16, 2014
  14. 07/25 reachable: reuse revision.c "add all reflogs" codeJeff King, Oct 15, 2014
  15. 08/25 prune: factor out loose-object directory traversalJeff King, Oct 15, 2014
  16. 09/25 reachable: mark index blobs as SEENJeff King, Oct 15, 2014
  17. 10/25 prune-packed: use for_each_loose_file_in_objdirJeff King, Oct 15, 2014
  18. 11/25 count-objects: do not use xsize_t when counting object sizeJeff King, Oct 15, 2014
  19. 12/25 count-objects: use for_each_loose_file_in_objdirJeff King, Oct 15, 2014
  20. 13/25 sha1_file: add for_each iterators for loose and packed objectsJeff King, Oct 15, 2014
  21. 14/25 prune: keep objects reachable from recent objectsJeff King, Oct 15, 2014
  22. 15/25 pack-objects: refactor unpack-unreachable expiration checkJeff King, Oct 15, 2014
  23. 16/25 pack-objects: match prune logic for discarding objectsJeff King, Oct 15, 2014
  24. 17/25 write_sha1_file: freshen existing objectsJeff King, Oct 15, 2014
  25. 18/25 make add_object_array_with_context interface more saneJeff King, Oct 15, 2014
  26. 19/25 traverse_commit_list: support pending blobs/trees with pathsJeff King, Oct 15, 2014
  27. 20/25 rev-list: document --reflog optionJeff King, Oct 15, 2014
  28. 21/25 rev-list: add --index-objects optionJeff King, Oct 15, 2014
  29. Junio C HamanoOct 16, 2014
  30. Jeff KingOct 17, 2014
  31. Jeff KingOct 17, 2014
  32. 22/26 rev-list: add --indexed-objects optionJeff King, Oct 17, 2014
  33. 23/26 reachable: use revision machinery's --indexed-objects codeJeff King, Oct 17, 2014
  34. 24/26 pack-objects: use argv_arrayJeff King, Oct 17, 2014
  35. 25/26 repack: pack objects mentioned by the indexJeff King, Oct 17, 2014
  36. 26/26 pack-objects: double-check options before discarding objectsJeff King, Oct 17, 2014
  37. 22/25 reachable: use revision machinery's --index-objects codeJeff King, Oct 15, 2014
  38. 23/25 pack-objects: use argv_arrayJeff King, Oct 15, 2014
  39. 24/25 repack: pack objects mentioned by the indexJeff King, Oct 15, 2014
  40. 25/25 pack-objects: double-check options before discarding objectsJeff King, Oct 15, 2014
  41. Junio C HamanoOct 16, 2014
  42. Junio C HamanoOct 16, 2014
  43. Jeff KingOct 16, 2014
  44. Jeff KingOct 16, 2014
  45. Junio C HamanoOct 16, 2014
  46. Jeff KingOct 17, 2014
  47. Jeff KingOct 18, 2014
  48. Jeff KingOct 17, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.