git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] commit: don't rewrite shared index unnecessarily

From
David Turner <dturner@twopensource.com>
Date
Aug 31, 2015, 18:41 UTC
Message-ID
<1441046491.25570.13.camel@twopensource.com>
In-Reply-To
<CACsJy8D=KyWBYf1LW=n928rmG=9xavSaR+1G1g+F7mMTKFCpfg@mail.gmail.com>
On Mon, 2015-08-31 at 17:02 +0700, Duy Nguyen wrote:
Show 31 quoted lines
> On Fri, Aug 28, 2015 at 12:07 AM, David Turner <dturner@twopensource.com> wrote:
> > Remove a cache invalidation which would cause the shared index to be
> > rewritten on as-is commits.
> >
> > When the cache-tree has changed, we need to update it.  But we don't
> > necessarily need to update the shared index.  So setting
> > active_cache_changed to SOMETHING_CHANGED is unnecessary.  Instead, we
> > let update_main_cache_tree just update the CACHE_TREE_CHANGED bit.
> >
> > In order to test this, make test-dump-split-index not segfault on
> > missing replace_bitmap/delete_bitmap.  This new codepath is not called
> > now that the test passes, but is necessary to avoid a segfault when the
> > new test is run with the old builtin/commit.c code.
> >
> > Signed-off-by: David Turner <dturner@twopensource.com>
> 
> Ack.
> 
> I made SOMETHING_CHANGED "1" for catching these cases (there were a
> few on-flight topics when this series was being cooked and I was
> afraid I could not cache all active_cache_changed sites).
> 
> > ---
> >
> > I introduced this bug last year while improving the cache-tree code.
> > I guess I probably didn't notice that active_cache_changed wasn't a
> > boolean.
> 
> So.. you did you split-index? Cool. Never heard anyone using it for
> real. It needs the other part to improve reading/refresh side to get
> to full potential though..

I am experimenting with an "unskipped index", to reduce the cost of index loading and processing in sparse checkouts. The unskipped index is an index on the (v3) index -- that is, it lists the byte offsets of all entries in the sharedindex where CE_SKIP_WORKTREE is 0. In addition, there is a list of the offsets of *all* entries, which we can binary search in. There's also some other metadata that I'm figuring out as I go along. This might allow us to avoid loading the full shared index in most cases. Or it might be a total dead end. Not sure yet.

Previous: Junio C Hamano
Message 4 of 4 in “commit: don't rewrite shared index unnecessarily”
  1. commit: don't rewrite shared index unnecessarilyDavid Turner, Aug 27, 2015
  2. Duy NguyenAug 31, 2015
  3. Junio C HamanoAug 31, 2015
  4. David TurnerAug 31, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.