{"thread":{"id":"49550","subject":"Does git load index file into memory?","startedAt":"2018-10-12T16:23:15Z","lastAt":"2018-10-14T09:59:54Z","messageCount":2,"participants":["Farhan Khan","Thomas Gummerer"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"360324","messageId":"CAFd4kYCTuLMZ6UDk49MASbZfhwjgUcmfeyd6jZS5QO9-LnzLCw@mail.gmail.com","threadId":"49550","inReplyTo":null,"subject":"Does git load index file into memory?","fromName":"Farhan Khan","fromEmail":"khanzf@gmail.com","sentAt":"2018-10-12T16:22:19Z","receivedAt":"2018-10-12T16:23:15Z","isPatch":false,"sender":{"key":"khanzf@gmail.com","avatar":null},"body":"Hi all,\n\nDoes git load the entire index file into memory when it wants to\nedit/view it? I ask because I wonder if this can become a problem with\nthe index file becomes arbitrarily large, like for the Linux kernel.\n\nThanks,\n--\nFarhan Khan\nPGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE\n"},{"id":"360402","messageId":"20181014095949.GB22611@hank.intra.tgummerer.com","threadId":"49550","inReplyTo":"CAFd4kYCTuLMZ6UDk49MASbZfhwjgUcmfeyd6jZS5QO9-LnzLCw@mail.gmail.com","subject":"Re: Does git load index file into memory?","fromName":"Thomas Gummerer","fromEmail":"t.gummerer@gmail.com","sentAt":"2018-10-14T09:59:49Z","receivedAt":"2018-10-14T09:59:54Z","isPatch":false,"sender":{"key":"t.gummerer@gmail.com","avatar":"https://avatars.githubusercontent.com/u/191004?v=4"},"body":"On 10/12, Farhan Khan wrote:\n> Hi all,\n> \n> Does git load the entire index file into memory when it wants to\n> edit/view it? I ask because I wonder if this can become a problem with\n> the index file becomes arbitrarily large, like for the Linux kernel.\n\nYes, currently git always loads the entire index file for any\noperation on it.  This is not particularly a problem for projects like\nthe linux kernel, as the index file for it is relatively small, ~6MB\nat the moment.\n\nIt is more of a problem for larger repositories, such as the windows\nrepository, which has and index file of ~300-400MB, iirc, where\nloading the index has a significant cost.  There's some patch series\nin progress to improve the performance, e.g. Ben Pearts series to load\nthe index in parallel [*1*].\n\nFor writing the index to disk again, the split index feature can help\nimprove performance.  See also 'man git-update-index' and\n\"core.splitIndex\" in 'man git-config'.\n\n[*1*]: https://public-inbox.org/git/20181010155938.20996-1-peartben@gmail.com/\n\n> Thanks,\n> --\n> Farhan Khan\n> PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE\n"}]}