git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] t5304: add a test for pruning with bitmaps

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 20, 2019, 01:01 UTC
Message-ID
<ba16afb4-25c1-8148-602e-130b0e17fc89@gmail.com>
In-Reply-To
<20190418200827.GB15249@sigill.intra.peff.net>
On 4/18/2019 4:08 PM, Jeff King wrote:
Show 6 quoted lines
> On Thu, Apr 18, 2019 at 03:49:53PM -0400, Jeff King wrote:
>> I dunno. I guess it does not hurt to at least to at least make sure this
>> code is running in the normal suite. I don't think that will find the
>> more interesting regressions, but at least saves us the from the most
>> egregious ones ("oops, the whole thing segfaults because somebody
>> changed the API" kinds of things).

That's the level of coverage I was hoping to see. I won't be picky if the OBJ_TAG case isn't hit or something.

> So here's a test. This goes on top of jk/prune-optim (which is also
> already in master).
[snip]
Show 9 quoted lines
> +test_expect_success 'trivial prune with bitmaps enabled' '
> +	git repack -adb &&
> +	blob=$(echo bitmap-unreachable-blob | git hash-object -w --stdin) &&
> +	git prune --expire=now &&
> +	git cat-file -e HEAD &&
> +	test_must_fail git cat-file -e $blob
> +'
> +
>  test_done

This test does cover the 'mark_object_seen()' method, which I tested by removing the "!" in the "if (!obj) die();" condition.

However, my first inclination was to remove the line
	obj->flags |= SEEN;

from the method, and I found that it still worked! This was confusing, so I looked for places where the SEEN flag was added during bitmap walks, and it turns out that the objects are marked as SEEN by prepare_bitmap_walk().

This means that we can remove a lot of the implementation from reachable.c and have the same effect (and drop an iteration through the objects). See the diff below.

Thoughts? -Stolee

-->8--
diff --git a/reachable.c b/reachable.c
index 0d00a91de4..7d2762d47f 100644
--- a/reachable.c
+++ b/reachable.c
@@ -159,39 +159,6 @@ int add_unseen_recent_objects_to_traversal(struct rev_info *revs,
                                      FOR_EACH_OBJECT_LOCAL_ONLY);
 }

-static void *lookup_object_by_type(struct repository *r,
-                                  const struct object_id *oid,
-                                  enum object_type type)
-{
-       switch (type) {
-       case OBJ_COMMIT:
-               return lookup_commit(r, oid);
-       case OBJ_TREE:
-               return lookup_tree(r, oid);
-       case OBJ_TAG:
-               return lookup_tag(r, oid);
-       case OBJ_BLOB:
-               return lookup_blob(r, oid);
-       default:
-               die("BUG: unknown object type %d", type);
-       }
-}
-
-static int mark_object_seen(const struct object_id *oid,
-                            enum object_type type,
-                            int exclude,
-                            uint32_t name_hash,
-                            struct packed_git *found_pack,
-                            off_t found_offset)
-{
-       struct object *obj = lookup_object_by_type(the_repository, oid, type);
-       if (!obj)
-               die("unable to create object '%s'", oid_to_hex(oid));
-
-       obj->flags |= SEEN;
-       return 0;
-}
-
 void mark_reachable_objects(struct rev_info *revs, int mark_reflog,
                            timestamp_t mark_recent, struct progress *progress)
 {
@@ -225,7 +192,10 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog,

        bitmap_git = prepare_bitmap_walk(revs);
        if (bitmap_git) {
-               traverse_bitmap_commit_list(bitmap_git, mark_object_seen);
+               /*
+                * reachable objects are marked as SEEN
+                * by prepare_bitmap_walk(revs).
+                */
                free_bitmap_index(bitmap_git);
                return;
        }
Previous: Jeff KingNext: Jeff King
Message 54 of 57 in “some prune optimizations”
  1. 0/3 some prune optimizationsJeff King, Feb 14, 2019
  2. 1/3 prune: lazily perform reachability traversalJeff King, Feb 14, 2019
  3. Eric SunshineFeb 14, 2019
  4. Jeff KingFeb 14, 2019
  5. 2/3 prune: use bitmaps for reachability traversalJeff King, Feb 14, 2019
  6. bitmaps by default? [was: prune: use bitmaps for reachability traversal]Eric Wong, Mar 9, 2019
  7. Jeff KingMar 10, 2019
  8. repack: enable bitmaps by default on bare reposEric Wong, Mar 12, 2019
  9. Ævar Arnfjörð BjarmasonMar 12, 2019
  10. Jeff KingMar 12, 2019
  11. Jeff KingMar 12, 2019
  12. Eric WongMar 13, 2019
  13. Jeff KingMar 13, 2019
  14. repack: enable bitmaps by default on bare reposEric Wong, Mar 14, 2019
  15. Jeff KingMar 14, 2019
  16. 0/2 enable bitmap hash-cache by defaultJeff King, Mar 15, 2019
  17. 1/2 t5310: correctly remove bitmaps for jgit testJeff King, Mar 15, 2019
  18. SZEDER GáborMar 15, 2019
  19. Jeff KingMar 15, 2019
  20. 2/2 pack-objects: default to writing bitmap hash-cacheJeff King, Mar 15, 2019
  21. Ævar Arnfjörð BjarmasonApr 9, 2019
  22. Jeff KingApr 10, 2019
  23. Junio C HamanoApr 25, 2019
  24. Jeff KingMay 4, 2019
  25. Ævar Arnfjörð BjarmasonMay 4, 2019
  26. SZEDER GáborMay 4, 2019
  27. Ævar Arnfjörð BjarmasonMay 8, 2019
  28. Junio C HamanoMay 9, 2019
  29. Jeff KingMay 7, 2019
  30. Ævar Arnfjörð BjarmasonMay 7, 2019
  31. Jeff KingMay 8, 2019
  32. Derrick StoleeMay 8, 2019
  33. Ævar Arnfjörð BjarmasonMay 8, 2019
  34. Jeff KingMay 8, 2019
  35. Jeff KingMay 23, 2019
  36. Derrick StoleeMay 23, 2019
  37. Jeff KingMay 24, 2019
  38. Derrick StoleeMay 24, 2019
  39. Ævar Arnfjörð BjarmasonMay 23, 2019
  40. Jeff KingMay 24, 2019
  41. Ævar Arnfjörð BjarmasonMay 24, 2019
  42. Jeff KingMay 24, 2019
  43. Ævar Arnfjörð BjarmasonMay 24, 2019
  44. SZEDER GáborMay 24, 2019
  45. Ævar Arnfjörð BjarmasonMay 24, 2019
  46. SZEDER GáborMay 24, 2019
  47. Ævar Arnfjörð BjarmasonMay 24, 2019
  48. SZEDER GáborMay 24, 2019
  49. Ævar Arnfjörð BjarmasonMay 24, 2019
  50. pack-bitmap: look for an uninteresting bitmapDerrick Stolee, May 24, 2019
  51. Derrick StoleeApr 15, 2019
  52. Jeff KingApr 18, 2019
  53. t5304: add a test for pruning with bitmapsJeff King, Apr 18, 2019
  54. Derrick StoleeApr 20, 2019
  55. Jeff KingApr 20, 2019
  56. Derrick StoleeApr 20, 2019
  57. 3/3 prune: check SEEN flag for reachabilityJeff King, Feb 14, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.