git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/4] pack-bitmap: pass object filter to fill-in traversal

From
Jeff King <peff@peff.net>
Date
Apr 24, 2020, 05:42 UTC
Message-ID
<20200424054227.GE1648190@coredump.intra.peff.net>
In-Reply-To
<65467a058e7dca6cf1e2db9cdab81513989b5db0.1587597151.git.me@ttaylorr.com>
On Wed, Apr 22, 2020 at 05:13:35PM -0600, Taylor Blau wrote:
Show 18 quoted lines
> From: Jeff King <peff@peff.net>
> 
> Sometimes a bitmap traversal still has to walk some commits manually,
> because those commits aren't included in the bitmap packfile (e.g., due
> to a push or commit since the last full repack). If we're given an
> object filter, we don't pass it down to this traversal. It's not
> necessary for correctness because the bitmap code has its own filters to
> post-process the bitmap result (which it must, to filter out the objects
> that _are_ mentioned in the bitmapped packfile).
> 
> And with blob filters, there was no performance reason to pass along
> those filters, either. The fill-in traversal could omit them from the
> result, but it wouldn't save us any time to do so, since we'd still have
> to walk each tree entry to see if it's a blob or not.
> 
> But now that we support tree filters, there's opportunity for savings. A
> tree:depth=0 filter means we can avoid accessing trees entirely, since
> we know we won't them (or any of the subtrees or blobs they point to).
s/won't them/won't include them/ perhaps
Show 12 quoted lines
> diff --git a/t/perf/p5310-pack-bitmaps.sh b/t/perf/p5310-pack-bitmaps.sh
> index b629a211f9..95379b1d4e 100755
> --- a/t/perf/p5310-pack-bitmaps.sh
> +++ b/t/perf/p5310-pack-bitmaps.sh
> @@ -95,4 +95,9 @@ test_perf 'pack to file (partial bitmap)' '
>  	git pack-objects --use-bitmap-index --all pack2b </dev/null >/dev/null
>  '
>  
> +test_perf 'rev-list with tree filter (partial bitmap)' '
> +	git rev-list --use-bitmap-index --count --objects --all \
> +		--filter=tree:0 >/dev/null
> +'

This covers perf testing of this partial-bitmap state, but we shoudl make sure that we are covering correctness, too. I think so, because t6113 creates a similar state for all of its tests.

-Peff
Previous: Taylor BlauNext: Taylor Blau
Message 6 of 8 in “pack-bitmap: use bitmaps for traversals with '--filter=tree:0'”
  1. 0/4 pack-bitmap: use bitmaps for traversals with '--filter=tree:0'Taylor Blau, Apr 22, 2020
  2. 1/4 list-objects-filter: treat NULL filter_options as "disabled"Taylor Blau, Apr 22, 2020
  3. 2/4 pack-bitmap.c: make object filtering functions genericTaylor Blau, Apr 22, 2020
  4. 3/4 pack-bitmap.c: support 'tree:0' filteringTaylor Blau, Apr 22, 2020
  5. 4/4 pack-bitmap: pass object filter to fill-in traversalTaylor Blau, Apr 22, 2020
  6. Jeff KingApr 24, 2020
  7. Taylor BlauApr 24, 2020
  8. Jeff KingApr 24, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.