Re: [PATCH 1/3] pack-objects: fix handling of multiple --filter options

From: René Scharfe <l.s.r@web.de>
Date: Nov 13, 2022, 17:31 UTC
Message-ID: <be477f52-33e0-b2e8-9bcc-9264f1b50460@web.de>
In-Reply-To: <221112.86bkpcmm6i.gmgdl@evledraar.gmail.com>

Am 12.11.22 um 12:41 schrieb Ævar Arnfjörð Bjarmason:

Show 16 quoted lines

>
> On Sat, Nov 12 2022, René Scharfe wrote:
>
>> Since 5cb28270a1 (pack-objects: lazily set up "struct rev_info", don't
>> leak, 2022-03-28) --filter options given to git pack-objects overrule
>> earlier ones, letting only the leftmost win and leaking the memory
>> allocated for earlier ones.  Fix that by only initializing the rev_info
>> struct once.
>
> If I do e.g. this with SANITIZE=leak:
>
> 	echo e83c5163316f89bfbde7d9ab23ca2e25604af290 | ./git pack-objects --revs --filter=blob:limit=1001 --filter=object:type=blob --stdout  >/dev/null
>
> I see one leak that wasn't there, but two that are gone now. I haven't
> looked into it, but I think the commit message should discuss what leaks
> are fixed, which remain/are new etc.

The leak is insubstantial; I mentioned it just because of the irony. It is caused by initializing an already initialized struct rev_info without releasing it in between, as mentioned in the commit message.

.filter_data allocated by filter_combine__init() is not released by filter_combine__init() in list-objects-filter.c. Plugging that leak allows your example command to run with SANITIZE=leak. That is a matter for a separate patch, though.

Show 28 quoted lines

>
>> @@ -4158,7 +4158,8 @@ static struct list_objects_filter_options *po_filter_revs_init(void *value)
>>  {
>>  	struct po_filter_data *data = value;
>>
>> -	repo_init_revisions(the_repository, &data->revs, NULL);
>> +	if (!data->have_revs)
>> +		repo_init_revisions(the_repository, &data->revs, NULL);
>>  	data->have_revs = 1;
>>
>>  	return &data->revs.filter;
>
> FWIW as this goes away in your 2/3 I think just squashing the two with a
> leak fix would be nice, if...
>
>> diff --git a/t/t5317-pack-objects-filter-objects.sh b/t/t5317-pack-objects-filter-objects.sh
>> index bb633c9b09..bd8983bb56 100755
>> --- a/t/t5317-pack-objects-filter-objects.sh
>> +++ b/t/t5317-pack-objects-filter-objects.sh
>> @@ -178,6 +178,25 @@ test_expect_success 'verify blob:limit=1001' '
>>  	test_cmp expected observed
>>  '
>>
>> +test_expect_success 'verify blob:limit=1001+object:type=blob' '
>> +	git -C r2 ls-files -s large.1000 |
>
> Aside: Should do "git >tmp && test_parse... <tmp", we lose the exit code
> of "ls-files" here.

OK. Copied that line from the surrounding tests. They used temporary files before fb2d0db502 (test-lib-functions: add parsing helpers for ls-files and ls-tree, 2022-04-04).

Show 33 quoted lines

>
>> +	test_parse_ls_files_stage_oids |
>> +	sort >expected &&
>> +
>> +	git -C r2 pack-objects --revs --stdout --filter=blob:limit=1001 \
>> +		--filter=object:type=blob >filter.pack <<-EOF &&
>> +	HEAD
>> +	EOF
>> +	git -C r2 index-pack ../filter.pack &&
>> +
>> +	git -C r2 verify-pack -v ../filter.pack >verify_result &&
>> +	grep blob verify_result |
>> +	parse_verify_pack_blob_oid |
>
> Whereas this one's not a problem, no "git".
>
>> +	sort >observed &&
>> +
>> +	test_cmp expected observed
>
> Aside: It would be nice if we had a "test_cmp_sort", but some other day...
>
>> +'
>> +
>>  test_expect_success 'verify blob:limit=10001' '
>>  	git -C r2 ls-files -s large.1000 large.10000 |
>>  	test_parse_ls_files_stage_oids |
>
> ...we can test it, but this test is in a file that's not part of "linux-leaks".
>
> If that one leak I mentioned above can be fixed (or maybe it's not new?)
> this could be tested if we put it in a file of its own with
> TEST_PASSES_SANITIZE_LEAK=true.

Plugging the leak in your example command is not enough to make t5317 leak free.

René