git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v6 03/19] index-helper: new daemon for caching index and related stuff

From
Duy Nguyen <pclouds@gmail.com>
Date
May 1, 2016, 00:22 UTC
Message-ID
<CACsJy8AZihi2j=LLqhPvvLm2PPR4KZne2y9ENR4YDgY+tOXo0A@mail.gmail.com>
In-Reply-To
<1461955617.4123.37.camel@twopensource.com>
On Sat, Apr 30, 2016 at 1:46 AM, David Turner <dturner@twopensource.com> wrote:
Show 26 quoted lines
> On Thu, 2016-04-28 at 11:58 -0700, Junio C Hamano wrote:
>> David Turner <dturner@twopensource.com> writes:
>>
>> > From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> > ...
>> > The biggest gain is not having to verify the trailing SHA-1, which
>> > takes lots of time especially on large index files. But this also
>> > opens doors for further optimiztions:
>>
>> optimizAtion
>>
>> > Git can poke the daemon via unix domain sockets to tell it to
>> > refresh
>> > the index cache, or to keep it alive some more minutes. It can't
>> > give
>> > any real index data directly to the daemon. Real data goes to disk
>> > first, then the daemon reads and verifies it from there. Poking
>> > only
>> > happens for $GIT_DIR/index, not temporary index files.
>>
>> Is this limited to "poking", or the helper daemon is not involved in
>> codepaths that handle temporary index at all?  It makes sense if it
>> is the latter, and it doesn't if it were the former, but it is
>> unclear in this paragraph.
>
> It is in fact the latter.  Will clarify.

The intention is so. The execution is less than perfect. On the server side, the index-helper only concerns itself with .git/index only. Good. On the client side, is_main_index() is used to detect if it's $GIT_DIR/index. It checks if the given index_state pointer points to the_index and if so assumes that it is the main index.

That is not true when read_cache_from() is used with something other than git_index_file() as argument. Which is exactly what builtin/commit.c does in prepare_index(). .git/index.lock gets passed in instead. read-cache.c may "poke" or "refresh" (probably should rename these to "read" and "write") because is_main_index() incorrectly returns true. Luckily it will soon find out index-helper does not prepare shm for this particular index file and fall back to reading from disk. We waste some context switches though.

-- 
Duy
Previous: Junio C HamanoNext: David Turner
Message 8 of 24 in “index-helper/watchman”
  1. 00/19 index-helper/watchmanDavid Turner, Apr 27, 2016
  2. 01/19 read-cache.c: fix constness of verify_hdr()David Turner, Apr 27, 2016
  3. 02/19 read-cache: allow to keep mmap'd memory after readingDavid Turner, Apr 27, 2016
  4. 03/19 index-helper: new daemon for caching index and related stuffDavid Turner, Apr 27, 2016
  5. Junio C HamanoApr 28, 2016
  6. David TurnerApr 29, 2016
  7. Junio C HamanoApr 29, 2016
  8. Duy NguyenMay 1, 2016
  9. 04/19 index-helper: add --strictDavid Turner, Apr 27, 2016
  10. 05/19 index-helper: log warningsDavid Turner, Apr 27, 2016
  11. 06/19 daemonize(): set a flag before exiting the main processDavid Turner, Apr 27, 2016
  12. 07/19 index-helper: add --detachDavid Turner, Apr 27, 2016
  13. 08/19 read-cache: add watchman 'WAMA' extensionDavid Turner, Apr 27, 2016
  14. 09/19 Add watchman support to reduce index refresh costDavid Turner, Apr 27, 2016
  15. 10/19 index-helper: use watchman to avoid refreshing index with lstat()David Turner, Apr 27, 2016
  16. 11/19 update-index: enable/disable watchman supportDavid Turner, Apr 27, 2016
  17. 12/19 unpack-trees: preserve index extensionsDavid Turner, Apr 27, 2016
  18. 13/19 watchman: add a config option to enable the extensionDavid Turner, Apr 27, 2016
  19. 14/19 index-helper: kill modeDavid Turner, Apr 27, 2016
  20. 15/19 index-helper: don't run if already runningDavid Turner, Apr 27, 2016
  21. 16/19 index-helper: autorun modeDavid Turner, Apr 27, 2016
  22. 17/19 index-helper: optionally automatically runDavid Turner, Apr 27, 2016
  23. 18/19 Add tracing to measure where most of the time is spentDavid Turner, Apr 27, 2016
  24. 19/19 untracked-cache: config optionDavid Turner, Apr 27, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.