git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git Test Coverage Report (v2.20.0-rc0)

From
Derrick Stolee <stolee@gmail.com>
Date
Nov 20, 2018, 12:17 UTC
Message-ID
<80690e4b-4fc4-7846-009f-59dee9fbdd86@gmail.com>
In-Reply-To
<20181120113423.GA29319@sigill.intra.peff.net>
On 11/20/2018 6:34 AM, Jeff King wrote:
Show 12 quoted lines
> On Mon, Nov 19, 2018 at 10:40:53AM -0500, Derrick Stolee wrote:
>
>
> Since depth is never incremented, we are not covering this block. Is it
> possible to test?
> This should be covered by the fix in:
>
>    https://public-inbox.org/git/20181120095053.GC22742@sigill.intra.peff.net/
>
> because now entries at the top-level are depth "1". The "depth++" line
> is still never executed in our test suite. I'm not sure how much that
> matters.
Thanks! I'll go take a look at your patch.
Show 26 quoted lines
>>> delta-islands.c
>>> c8d521faf7  53) memcpy(b, old, size);
>> This memcpy happens when the 'old' island_bitmap is passed to
>> island_bitmap_new(), but...
>>
>>> c8d521faf7 187) b->refcount--;
>>> c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);
>> This block has the only non-NULL caller to island_bitmap_new().
> This is another case where it triggers a lot for a reasonably-sized
> repo, but it's hard to construct a small case. This code implements a
> copy-on-write of the bitmap, which means the same objects have to be
> accessible from two different paths through the reachability graph, each
> with different island marks. And then a test would I guess make sure
> that the correct subsets of objects never become deltas, which gets
> complicated.
>
> And I think that's a pattern with the delta-island code. What we really
> care about most is that if we throw a real fork-network repository at
> it, it produces faster clones with fewer un-reusable deltas. So I think
> a much more interesting approach here would be perf tests. But:
>
>    - we'd want to count those as coverage, and that likely makes your
>      coverage tests prohibitively expensive
>
>    - it requires a specialized repo to demonstrate, which most people
>      aren't going to have handy

Do you have regularly-running tests that check this in your infrastructure? As long as someone would notice if this code starts failing, that would be enough.

Show 13 quoted lines
>>> c8d521faf7 212) obj = ((struct tag *)obj)->tagged;
>>> c8d521faf7 213) if (obj) {
>>> c8d521faf7 214) parse_object(the_repository, &obj->oid);
>>> c8d521faf7 215) marks = create_or_get_island_marks(obj);
>>> c8d521faf7 216) island_bitmap_set(marks, island_counter);
>> It appears that this block would happen if we placed a tag in the delta
>> island.
> Yep. Again, exercised by real repositories.
>
> I'm not sure how far we want to go in the blind pursuit of coverage.
> Certainly we could add a tag to the repo in t5320, and this code would
> get executed. But verifying that it's doing the right thing is much
> harder (and is more easily done with a perf test).

Blind coverage goals are definitely not worth the effort. My goal here was to re-check all of the new code (since last release) that is not covered, because it's easier to hide a bug there.

Show 7 quoted lines
>
>>> c8d521faf7 397) strbuf_addch(&island_name, '-');
>> This block is inside the following patch:
>> [...]
> Yeah, this triggers if your regex has more than one capture group (and
> likewise, we almost certainly don't run the "you have too many groups"
> warning).

Did you know that regexes are notoriously under-tested [1]? When looking at this code, I didn't even know regexes were involved (but I didn't look enough at the context).

[1] https://dl.acm.org/citation.cfm?id=3236072
Show 26 quoted lines
>
>>> c8d521faf7 433) continue;
>>> c8d521faf7 436) list[dst] = list[src];
>> These blocks are inside the following nested loop in deduplicate_islands():
>>
>> +       for (ref = 0; ref + 1 < island_count; ref++) {
>> +               for (src = ref + 1, dst = src; src < island_count; src++) {
>> +                       if (list[ref]->hash == list[src]->hash)
>> +                               continue;
>> +
>> +                       if (src != dst)
>> +                               list[dst] = list[src];
>> +
>> +                       dst++;
>> +               }
>> +               island_count = dst;
>> +       }
>>
>> This means that our "deduplication" logic is never actually doing anything
>> meaningful.
> Sorry, I don't even remember what this code is trying to do. The island
> code is 5+ years old, and just recently ported to upstream Git by
> Christian. And that's perhaps part of my laziness in the above tests; it
> would be a significant effort to re-figure out all these corner cases.
> It's a big part of why I hadn't been sending the patches upstream
> myself.

Sure. Hopefully pointing out these blocks gives us more motivation to manually inspect them and avoid silly bugs.

Thanks, -Stolee

Previous: Jeff KingNext: Jeff King
Message 8 of 18 in “Git Test Coverage Report (v2.20.0-rc0)”
  1. Derrick StoleeNov 19, 2018
  2. Derrick StoleeNov 19, 2018
  3. Jeff KingNov 19, 2018
  4. Jeff KingNov 19, 2018
  5. Ben PeartNov 19, 2018
  6. Derrick StoleeNov 19, 2018
  7. Jeff KingNov 20, 2018
  8. Derrick StoleeNov 20, 2018
  9. Jeff KingNov 20, 2018
  10. Ævar Arnfjörð BjarmasonNov 19, 2018
  11. Jonathan NiederNov 19, 2018
  12. Ævar Arnfjörð BjarmasonNov 19, 2018
  13. Derrick StoleeNov 19, 2018
  14. Ævar Arnfjörð BjarmasonNov 19, 2018
  15. Derrick StoleeNov 19, 2018
  16. Derrick StoleeNov 19, 2018
  17. Ævar Arnfjörð BjarmasonNov 19, 2018
  18. Johannes SchindelinNov 20, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.