Does git load index file into memory?

Farhan Khan· Oct 12, 2018, 16:22 UTC · lore

Hi all,

Does git load the entire index file into memory when it wants to edit/view it? I ask because I wonder if this can become a problem with the index file becomes arbitrarily large, like for the Linux kernel.

Thanks, -- Farhan Khan PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE

Thomas Gummerer· Oct 14, 2018, 09:59 UTC · re: Farhan Khan · lore

Re: Does git load index file into memory?

On 10/12, Farhan Khan wrote:

Show 5 quoted lines

> Hi all,
> 
> Does git load the entire index file into memory when it wants to
> edit/view it? I ask because I wonder if this can become a problem with
> the index file becomes arbitrarily large, like for the Linux kernel.

Yes, currently git always loads the entire index file for any operation on it. This is not particularly a problem for projects like the linux kernel, as the index file for it is relatively small, ~6MB at the moment.

It is more of a problem for larger repositories, such as the windows repository, which has and index file of ~300-400MB, iirc, where loading the index has a significant cost. There's some patch series in progress to improve the performance, e.g. Ben Pearts series to load the index in parallel [*1*].

For writing the index to disk again, the split index feature can help improve performance. See also 'man git-update-index' and "core.splitIndex" in 'man git-config'.

[*1*]: https://public-inbox.org/git/20181010155938.20996-1-peartben@gmail.com/

> Thanks,
> --
> Farhan Khan
> PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE