{"thread":{"id":"58049","subject":"Bug report. Out of memory about git checkout.","startedAt":"2022-06-22T13:56:15Z","lastAt":"2022-06-22T13:56:15Z","messageCount":1,"participants":["void f"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"457696","messageId":"CADRR3BHq4ZmjHh9HJfh-nGrKR5X5TUPq3-eyr0Atbz8zM8oOxA@mail.gmail.com","threadId":"58049","inReplyTo":null,"subject":"Bug report. Out of memory about git checkout.","fromName":"void f","fromEmail":"fv729164860@gmail.com","sentAt":"2022-06-22T13:55:56Z","receivedAt":"2022-06-22T13:56:15Z","isPatch":false,"sender":{"key":"fv729164860@gmail.com","avatar":null},"body":"—env:\n\ngit version: 2.19.1\nos : center os\nmemory 8G\n\n— how to repeat this bug:\n\nBuild a repository with large lfs-file use GIT_LFS_SKIP_SMUDGE like\nthis:  (I can't push a 10G lfs-file to github, So I can’t give you an\nexample repostiory)\n\n```\nhecanwei@MacBook-Pro lfs-test % git st\nOn branch master\nnothing to commit, working tree clean\nhecanwei@MacBook-Pro lfs-test % echo \"$(cat Xcode_13.4.1.xip )\"\nversion https://git-lfs.github.com/spec/v1\noid sha256:a1e0dbd6d5a96c4a6d3d63600b58486759aa836c2d9f7e8fa6d7da4c7399638b\nsize 10783587696\n```\n\n\nRm Xcode_13.4.1.xip\n\nGit checkout .\n\nYou will see “Out of memory, realloc failed”\n\nIt also use too much memory above version: 2.36.1 macOS\n\n\n— reason of the bug:\n\n\nWhen you execute git checkout, Which have to checkout a lfs-file to\nthe worktree. Git would execute **convert.c\napply_multi_file_filter()** to convert lfs pointer from git-object to\nthe lfs file. It will execute a subprocess to convert this file. But\nIt is strange that git would read all of the file into memory When\nfinish the git-lfs subprocess. (The code is about pkt-line.c\nread_packetized_to_strbut()). Lfs usually is a very large file even\nmore than the memory. So it would throw out of memory exception.\n\nWith this bug, it would have trouble to use sparse-chekout in a\nrepository with large lfs-file. Because you must init the repository\nfirst and set the sparse-chekout config, than use git\npull/merge/checkout to checkout your subset worktree. It would out of\nmemory when you checkout it.\n\nI think git don’t need to read all of the file in memory. It can use a\nstream to finish the checkout.\n"}]}