git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 0/9] Reduce index load time

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Nov 1, 2015, 13:42 UTC
Message-ID
<1446385369-8669-1-git-send-email-pclouds@gmail.com>

This is the rebased version since last time [1] with s/free_index_shm/release_index_shm/ as suggested by David Turner. It introduces a daemon that can cache index data in memory so that subsequent git processes can avoid reading (and more importantly, verifying) the index from disk. Together with split-index it should keep index I/O cost down to minimum. The series can also be found at [2].

One of the factors that affected my design was Windows support. We now have Dscho back, he can evaluate my approach for Windows.

This daemon is the foundation for watchman support later to reduce refresh time. To be posted shortly after this.

[0] http://mid.gmane.org/1406548995-28549-1-git-send-email-pclouds@gmail.com [2] http://github.com/pclouds/git/commits/index-helper

Nguyễn Thái Ngọc Duy (9):
  trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics
  read-cache.c: fix constness of verify_hdr()
  read-cache: allow to keep mmap'd memory after reading
  index-helper: new daemon for caching index and related stuff
  trace.c: add GIT_TRACE_INDEX_STATS for index  statistics
  index-helper: add --strict
  daemonize(): set a flag before exiting the main process
  index-helper: add --detach
  index-helper: add Windows support
 .gitignore                               |   1 +
 Documentation/git-index-helper.txt (new) |  56 +++++++
 Documentation/git.txt                    |   4 +
 Makefile                                 |   9 ++
 builtin/gc.c                             |   2 +-
 cache.h                                  |  12 +-
 config.mak.uname                         |   3 +
 daemon.c                                 |   2 +-
 git-compat-util.h                        |   1 +
 git.c                                    |   1 +
 index-helper.c (new)                     | 264 +++++++++++++++++++++++++++++++
 read-cache.c                             | 147 ++++++++++++++++-
 setup.c                                  |   4 +-
 sha1_file.c                              |  24 +++
 shm.c (new)                              | 163 +++++++++++++++++++
 shm.h (new)                              |  23 +++
 trace.c                                  |  16 ++
 trace.h                                  |   1 +
 18 files changed, 721 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/git-index-helper.txt
 create mode 100644 index-helper.c
 create mode 100644 shm.c
 create mode 100644 shm.h
-- 
2.2.0.513.g477eb31
Next: Nguyễn Thái Ngọc Duy
Message 1 of 12 in “Reduce index load time”
  1. 0/9 Reduce index load timeNguyễn Thái Ngọc Duy, Nov 1, 2015
  2. 1/9 trace.c: add GIT_TRACE_PACK_STATS for pack usage statisticsNguyễn Thái Ngọc Duy, Nov 1, 2015
  3. 2/9 read-cache.c: fix constness of verify_hdr()Nguyễn Thái Ngọc Duy, Nov 1, 2015
  4. 3/9 read-cache: allow to keep mmap'd memory after readingNguyễn Thái Ngọc Duy, Nov 1, 2015
  5. 4/9 index-helper: new daemon for caching index and related stuffNguyễn Thái Ngọc Duy, Nov 1, 2015
  6. David TurnerNov 2, 2015
  7. 5/9 trace.c: add GIT_TRACE_INDEX_STATS for index statisticsNguyễn Thái Ngọc Duy, Nov 1, 2015
  8. 6/9 index-helper: add --strictNguyễn Thái Ngọc Duy, Nov 1, 2015
  9. 7/9 daemonize(): set a flag before exiting the main processNguyễn Thái Ngọc Duy, Nov 1, 2015
  10. 8/9 index-helper: add --detachNguyễn Thái Ngọc Duy, Nov 1, 2015
  11. 9/9 index-helper: add Windows supportNguyễn Thái Ngọc Duy, Nov 1, 2015
  12. Johannes SchindelinNov 16, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.