git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/7] for_each_packed_object: automatically open pack index

From
Jeff King <peff@peff.net>
Date
Jun 22, 2015, 10:40 UTC
Message-ID
<20150622104049.GA14475@peff.net>
In-Reply-To
<20150622103321.GB12584@peff.net>

When for_each_packed_object is called, we call prepare_packed_git() to make sure we have the actual list of packs. But the latter does not actually open the pack indices, meaning that pack->nr_objects may simply be 0 if the pack has not otherwise been used since the program started.

In practice, this didn't come up for the current callers, because they iterate the packed objects only after iterating all reachable objects (so for it to matter you would have to have a pack consisting only of unreachable objects). But it is a dangerous and confusing interface that should be fixed for future callers.

Note that we do not end the iteration when a pack cannot be opened, but we do return an error. That lets you complete the iteration even in actively-repacked repository where an .idx file may racily go away, but it also lets callers know that they may not have gotten the complete list (which the current reachability-check caller does care about).

We have to tweak one of the prune tests due to the changed return value; an earlier test creates bogus .idx files and does not clean them up. Having to make this tweak is a good thing; it means we will not prune in a broken repository, and the test confirms that we do not negatively impact a more lenient caller, count-objects.

Signed-off-by: Jeff King <peff@peff.net>
---
 sha1_file.c      | 7 ++++++-
 t/t5304-prune.sh | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/sha1_file.c b/sha1_file.c
index 5038475..f1f0efb 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -3573,14 +3573,19 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)
 {
 	struct packed_git *p;
 	int r = 0;
+	int pack_errors = 0;
 
 	prepare_packed_git();
 	for (p = packed_git; p; p = p->next) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
+		if (open_pack_index(p)) {
+			pack_errors = 1;
+			continue;
+		}
 		r = for_each_object_in_pack(p, cb, data);
 		if (r)
 			break;
 	}
-	return r;
+	return r ? r : pack_errors;
 }
diff --git a/t/t5304-prune.sh b/t/t5304-prune.sh
index 0794d33..023d7c6 100755
--- a/t/t5304-prune.sh
+++ b/t/t5304-prune.sh
@@ -218,6 +218,7 @@ test_expect_success 'gc: prune old objects after local clone' '
 '
 
 test_expect_success 'garbage report in count-objects -v' '
+	test_when_finished "rm -f .git/objects/pack/fake*" &&
 	: >.git/objects/pack/foo &&
 	: >.git/objects/pack/foo.bar &&
 	: >.git/objects/pack/foo.keep &&
-- 
2.4.4.719.g3984bc6
Previous: Jeff KingNext: Jeff King
Message 32 of 51 in “Improvements to parse-options and a new filter-objects command”
  1. Charles BaileyJun 19, 2015
  2. 1/3 Correct test-parse-options to handle negative intsCharles Bailey, Jun 19, 2015
  3. Junio C HamanoJun 19, 2015
  4. 2/3 Move unsigned long option parsing out of pack-objects.cCharles Bailey, Jun 19, 2015
  5. Remi Galan AlfonsoJun 19, 2015
  6. Charles BaileyJun 19, 2015
  7. Junio C HamanoJun 19, 2015
  8. Junio C HamanoJun 19, 2015
  9. Jakub NarębskiJun 20, 2015
  10. Jakub NarębskiJun 19, 2015
  11. Charles BaileyJun 20, 2015
  12. Junio C HamanoJun 20, 2015
  13. 3/3 Add filter-objects commandCharles Bailey, Jun 19, 2015
  14. Jeff KingJun 19, 2015
  15. Charles BaileyJun 19, 2015
  16. Jeff KingJun 19, 2015
  17. Junio C HamanoJun 19, 2015
  18. John KeepingJun 19, 2015
  19. Charles BaileyJun 19, 2015
  20. Improvements to integer option parsingCharles Bailey, Jun 21, 2015
  21. 1/2 Correct test-parse-options to handle negative intsCharles Bailey, Jun 21, 2015
  22. 2/2 Move unsigned long option parsing out of pack-objects.cCharles Bailey, Jun 21, 2015
  23. Charles BaileyJun 21, 2015
  24. Junio C HamanoJun 22, 2015
  25. Junio C HamanoJun 22, 2015
  26. Junio C HamanoJun 22, 2015
  27. Charles BaileyJun 22, 2015
  28. Fast enumeration of objectsCharles Bailey, Jun 21, 2015
  29. Add list-all-objects commandCharles Bailey, Jun 21, 2015
  30. Jeff KingJun 22, 2015
  31. Jeff KingJun 22, 2015
  32. 1/7 for_each_packed_object: automatically open pack indexJeff King, Jun 22, 2015
  33. 2/7 cat-file: minor style fix in options listJeff King, Jun 22, 2015
  34. 3/7 cat-file: move batch_options definition to top of fileJeff King, Jun 22, 2015
  35. 4/7 cat-file: add --buffer optionJeff King, Jun 22, 2015
  36. 5/7 cat-file: stop returning value from batch_one_objectJeff King, Jun 22, 2015
  37. 6/7 cat-file: split batch_one_object into two stagesJeff King, Jun 22, 2015
  38. 7/7 cat-file: add --batch-all-objects optionJeff King, Jun 22, 2015
  39. Eric SunshineJun 26, 2015
  40. Jeff KingJun 26, 2015
  41. 8/7 cat-file: sort and de-dup output of --batch-all-objectsJeff King, Jun 22, 2015
  42. Charles BaileyJun 22, 2015
  43. Jeff KingJun 22, 2015
  44. Charles BaileyJun 22, 2015
  45. Junio C HamanoJun 22, 2015
  46. Jeff KingJun 22, 2015
  47. Charles BaileyJun 22, 2015
  48. Duy NguyenJun 22, 2015
  49. Jeff KingJun 22, 2015
  50. Jeff KingJun 22, 2015
  51. Junio C HamanoJun 22, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.