git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/3] builtin/pack-objects.c: respect 'pack.preferBitmapTips'

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 1, 2021, 13:05 UTC
Message-ID
<924b5815-f7c6-7451-92e5-19bb936c54b1@gmail.com>
In-Reply-To
<5e10199dae5e342c0482497b038b3299a8ed046b.1617240723.git.me@ttaylorr.com>
On 3/31/2021 9:32 PM, Taylor Blau wrote:
Show 14 quoted lines
> When writing a new pack with a bitmap, it is sometimes convenient to
> indicate some reference prefixes which should receive priority when
> selecting which commits to receive bitmaps.
> 
> A truly motivated caller could accomplish this by setting
> 'pack.islandCore', (since all commits in the core island are similarly
> marked as preferred) but this requires callers to opt into using delta
> islands, which they may or may not want to do.
> 
> Introduce a new multi-valued configuration, 'pack.preferBitmapTips' to
> allow callers to specify a list of reference prefixes. All references
> which have a prefix contained in 'pack.preferBitmapTips' will mark their
> tips as "preferred" in the same way as commits are marked as preferred
> for selection by 'pack.islandCore'.

I almost hit send on an email saying "this doesn't seem to work with prefixes" but I have discovered the subtle reason why. I'll point it out below.

Show 16 quoted lines
> +static int mark_bitmap_preferred_tip(const char *refname,
> +				     const struct object_id *oid, int flags,
> +				     void *_data)
> +{
> +	struct object_id peeled;
> +	struct object *object;
> +
> +	if (!peel_iterated_oid(oid, &peeled))
> +		oid = &peeled;
> +
> +	object = parse_object_or_die(oid, refname);
> +	if (object->type == OBJ_COMMIT)
> +		object->flags |= NEEDS_BITMAP;
> +
> +	return 0;
> +}

This takes a refname and peels it (in case of a tag) but doesn't do any prefix matching.

Show 13 quoted lines
> +static void mark_bitmap_preferred_tips(void)
> +{
> +	struct string_list_item *item;
> +	const struct string_list *preferred_tips;
> +
> +	preferred_tips = bitmap_preferred_tips(the_repository);
> +	if (!preferred_tips)
> +		return;
> +
> +	for_each_string_list_item(item, preferred_tips) {
> +		for_each_ref_in(item->string, mark_bitmap_preferred_tip, NULL);
> +	}
> +}

This iterates on the list returned by bitmap_preferred_tips(), but specifically for_each_ref_in() does the prefix matching for us. I missed that point on my first read.

> +const struct string_list *bitmap_preferred_tips(struct repository *r)
> +{
> +	return repo_config_get_value_multi(r, "pack.preferbitmaptips");
> +}

And this just returns the config values, which are fed into for_each_ref_in().

Show 28 quoted lines
> +test_expect_success 'pack.preferBitmapTips' '
> +	git init repo &&
> +	test_when_finished "rm -fr repo" &&
> +	(
> +		cd repo &&
> +
> +		# create enough commits that not all are receive bitmap
> +		# coverage even if they are all at the tip of some reference.
> +		test_commit_bulk --message="%s" 103 &&
> +
> +		git rev-list HEAD >commits.raw &&
> +		sort <commits.raw >commits &&
> +
> +		git log --format="create refs/tags/%s %H" HEAD >refs &&
> +		git update-ref --stdin <refs &&
> +
> +		git repack -adb &&
> +		test-tool bitmap list-commits | sort >bitmaps &&
> +
> +		# remember which commits did not receive bitmaps
> +		comm -13 bitmaps commits >before &&
> +		test_file_not_empty before &&
> +
> +		# mark the commits which did not receive bitmaps as preferred,
> +		# and generate the bitmap again
> +		perl -pe "s{^}{create refs/tags/include/$. }" <before |
> +			git update-ref --stdin &&
> +		git -c pack.preferBitmapTips=refs/tags/include repack -adb &&

And this test is a clever way of creating a new ref category to check the prefix matching. Since you created refs pointing to every commit earlier, this verifies that the config is the reason they are picked now. Good.

Sorry for all of my thinking out loud. These are good patches and I have no recommended changes.

Thanks, -Stolee

Previous: Taylor Blau
Message 13 of 13 in “pack-objects: introduce pack.preferBitmapTips”
  1. 0/3 pack-objects: introduce pack.preferBitmapTipsTaylor Blau, Apr 1, 2021
  2. 1/3 pack-bitmap: add 'test_bitmap_commits()' helperTaylor Blau, Apr 1, 2021
  3. 2/3 t/helper/test-bitmap.c: initial commitTaylor Blau, Apr 1, 2021
  4. SunCC doesn't compile v2.32.0-rc* anymore (was "Re: [PATCH 2/3] t/helper/test-bitmap.c: initial commit")Ævar Arnfjörð Bjarmason, May 26, 2021
  5. Taylor BlauMay 26, 2021
  6. Ævar Arnfjörð BjarmasonMay 26, 2021
  7. pack-objects: move builtin-only code to its own headerÆvar Arnfjörð Bjarmason, May 27, 2021
  8. Junio C HamanoMay 27, 2021
  9. Junio C HamanoMay 27, 2021
  10. Junio C HamanoMay 27, 2021
  11. Ævar Arnfjörð BjarmasonMay 27, 2021
  12. 3/3 builtin/pack-objects.c: respect 'pack.preferBitmapTips'Taylor Blau, Apr 1, 2021
  13. Derrick StoleeApr 1, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.