git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Does git load index file into memory?

From
Thomas Gummerer <t.gummerer@gmail.com>
Date
Oct 14, 2018, 09:59 UTC
Message-ID
<20181014095949.GB22611@hank.intra.tgummerer.com>
In-Reply-To
<CAFd4kYCTuLMZ6UDk49MASbZfhwjgUcmfeyd6jZS5QO9-LnzLCw@mail.gmail.com>
On 10/12, Farhan Khan wrote:
Show 5 quoted lines
> Hi all,
> 
> Does git load the entire index file into memory when it wants to
> edit/view it? I ask because I wonder if this can become a problem with
> the index file becomes arbitrarily large, like for the Linux kernel.

Yes, currently git always loads the entire index file for any operation on it. This is not particularly a problem for projects like the linux kernel, as the index file for it is relatively small, ~6MB at the moment.

It is more of a problem for larger repositories, such as the windows repository, which has and index file of ~300-400MB, iirc, where loading the index has a significant cost. There's some patch series in progress to improve the performance, e.g. Ben Pearts series to load the index in parallel [*1*].

For writing the index to disk again, the split index feature can help improve performance. See also 'man git-update-index' and "core.splitIndex" in 'man git-config'.

[*1*]: https://public-inbox.org/git/20181010155938.20996-1-peartben@gmail.com/
> Thanks,
> --
> Farhan Khan
> PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE
Previous: Farhan Khan
Message 2 of 2 in “Does git load index file into memory?”
  1. Farhan KhanOct 12, 2018
  2. Thomas GummererOct 14, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.