{"thread":{"id":"56633","subject":"Bug: Failed checkout causes index to be in incorrect state","startedAt":"2021-10-02T20:41:37Z","lastAt":"2021-10-04T09:23:48Z","messageCount":2,"participants":["some-java-user-99206970363698485155@vodafonemail.de","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"437789","messageId":"07ae9b61a42346a0b49744586d4496bd@vodafonemail.de","threadId":"56633","inReplyTo":null,"subject":"Bug: Failed checkout causes index to be in incorrect state","fromName":"","fromEmail":"some-java-user-99206970363698485155@vodafonemail.de","sentAt":"2021-10-02T20:41:32Z","receivedAt":"2021-10-02T20:41:37Z","isPatch":false,"sender":{"key":"some-java-user-99206970363698485155@vodafonemail.de","avatar":null},"body":"Hello,\n\nthis originally occurred with Git for Windows (see https://github.com/git-for-windows/git/issues/3411)\nbut it appears the underlying issue is within Git itself, see Johannes Schindelin's comment\n(https://github.com/git-for-windows/git/issues/3411#issuecomment-914188040).\n\n## Description\nWhen Git fails to check out files (during `git clone`) the index seems to be in an incomplete state afterwards.\nThe files which could not be checked out are erroneously staged as \"deleted\", and files which were checked out\nsuccessfully are marked as untracked. This renders the suggested `git restore --source=HEAD :/` command to retry\nthe checkout ineffective.\n\n## Reproduction steps (Windows)\nPreconditions:\n- Long Windows file paths are not enabled\n  (https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later)\n- `git config core.longpaths` is not set\n\n1. `git clone https://github.com/dependabot/dependabot-core --depth=1 file-name-test`\n   (should fail due to too long file names)\n2. `cd file-name-test`\n3. `git status`\n   (should claim that files for which checkout failed are staged as \"deleted\")\n4. `git config core.longpaths true`\n5. Run the command suggested by the error message from step 1: `git restore --source=HEAD :/`\n6. `git status`\n   (you should see that `git restore` had no effect)\n\n## System information\nMicrosoft Windows [Version 10.0.19043.1237]\ngit version 2.33.0.windows.2\n\nAll reproduction steps up to, including, step 3 can also be reproduced with WSL 2.\nYou might need to use a different Git repository with longer files names though, such as\nhttps://github.com/Marcono1234/git-file-length-test\nWSL 2: 5.4.72-microsoft-standard-WSL2\ngit version 2.25.1\n\n\n"},{"id":"437863","messageId":"YVrIIBSG7BF1Pyy5@coredump.intra.peff.net","threadId":"56633","inReplyTo":"07ae9b61a42346a0b49744586d4496bd@vodafonemail.de","subject":"Re: Bug: Failed checkout causes index to be in incorrect state","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-04T09:23:44Z","receivedAt":"2021-10-04T09:23:48Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Oct 02, 2021 at 10:41:32PM +0200, some-java-user-99206970363698485155@vodafonemail.de wrote:\n\n> ## Description\n> When Git fails to check out files (during `git clone`) the index seems to be in an incomplete state afterwards.\n> The files which could not be checked out are erroneously staged as \"deleted\", and files which were checked out\n> successfully are marked as untracked. This renders the suggested `git restore --source=HEAD :/` command to retry\n> the checkout ineffective.\n\nHmm. I guess I'm not too surprised that we might leave things in an\ninconsistent state, but I'm having trouble getting the same outcome on\nLinux.\n\nIf I clone a repo with a very long file name, I do indeed get a failure.\nBut there is no index written at all, nor any files checked out. So all\nfile are marked as to-be-deleted in the index (compared to the HEAD\ncommit), but nothing is untracked, nor are there differences between the\nindex and working tree.\n\nMy reproduction is:\n\n\tgit init repo\n\tcd repo\n\n\techo before >0\n\techo after >2\n\tgit add .\n\tgit update-index --index-info <<-EOF\n\t100644 $(git rev-parse :0)\t1-$(perl -e 'print \"a\" x 5000')\n\tEOF\n\tgit commit -qm whatever\n\n\tgit clone --no-local . foo.git\n\tcd foo.git\n\tgit status | cut -c -60\n\nwhere the final status yields (note the intentional truncation on the\nfilename):\n\n  On branch main\n  Your branch is up to date with 'origin/main'.\n  \n  Changes to be committed:\n  \tdeleted:    0\n  \tdeleted:    1-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  \tdeleted:    2\n\nOne interesting thing is that my error during clone is:\n\n  error: cannot stat '1-aaaaaaaaaaa...\n\nwhich happens before we even start putting things into the filesystem.\nWhereas in the sample output you linked, Dscho had:\n\n  error: unable to create file...\n\nfollowed by more \"Unpacking objects\".\n\nSo there seems to be some difference in how the checkout occurs. The\nbacktrace for my error is:\n\n  #0  error_errno (fmt=0x5555558bcf02 \"cannot stat '%s'\") at usage.c:220\n  #1  0x0000555555804950 in verify_absent_1 (ce=0x7ffff7390098, error_type=ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, \n      o=0x7fffffffdba0) at unpack-trees.c:2249\n  #2  0x0000555555804a56 in verify_absent (ce=0x7ffff7390098, error_type=ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, \n      o=0x7fffffffdba0) at unpack-trees.c:2268\n  #3  0x0000555555804af8 in merged_entry (ce=0x5555559a8ae0, old=0x0, o=0x7fffffffdba0) at unpack-trees.c:2301\n  #4  0x0000555555805a50 in oneway_merge (src=0x7fffffffd450, o=0x7fffffffdba0) at unpack-trees.c:2701\n  #5  0x0000555555800896 in call_unpack_fn (src=0x7fffffffd450, o=0x7fffffffdba0) at unpack-trees.c:582\n  #6  0x0000555555801f04 in unpack_single_entry (n=1, mask=1, dirmask=0, src=0x7fffffffd450, names=0x7fffffffd790, \n      info=0x7fffffffd9e0) at unpack-trees.c:1132\n  #7  0x00005555558028c0 in unpack_callback (n=1, mask=1, dirmask=0, names=0x7fffffffd790, info=0x7fffffffd9e0)\n      at unpack-trees.c:1404\n  #8  0x00005555557fd0d5 in traverse_trees (istate=0x55555596f0e0 <the_index>, n=1, t=0x7fffffffdb50, \n      info=0x7fffffffd9e0) at tree-walk.c:532\n  #9  0x0000555555803654 in unpack_trees (len=1, t=0x7fffffffdb50, o=0x7fffffffdba0) at unpack-trees.c:1784\n  #10 0x0000555555599a7e in checkout (submodule_progress=1) at builtin/clone.c:699\n  #11 0x000055555559b56a in cmd_clone (argc=2, argv=0x7fffffffe4f0, prefix=0x0) at builtin/clone.c:1299\n\nI wonder why we get further than that on Windows. If I hack\nverify_absent_1() to treat this as success, I do get \"unable to create\"\nmessage later, along with some checked-out files. My status output is:\n\n  On branch main\n  Your branch is up to date with 'origin/main'.\n  \n  Changes to be committed:\n  \tdeleted:    0\n  \tdeleted:    1-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  \tdeleted:    2\n  \n  Untracked files:\n  \t0\n  \t2\n\nWe did not create the index at all, so comparing to HEAD, _every_ file\nappears to be deleted. And likewise comparing to the working tree,\nanything we did create is untracked.\n\nI think it would probably be nicer if we wrote out the index in this\ncase, if there were no problems creating the index itself, and just in\nthe actual file-checkout stage. unpack_trees() tries to make this\ndistinction, but we don't pay any attention. So perhaps something like:\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex ff1d3d447a..84dca96dad 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -696,14 +696,27 @@ static int checkout(int submodule_progress)\n \ttree = parse_tree_indirect(&oid);\n \tparse_tree(tree);\n \tinit_tree_desc(&t, tree->buffer, tree->size);\n-\tif (unpack_trees(1, &t, &opts) < 0)\n-\t\tdie(_(\"unable to checkout working tree\"));\n+\tswitch (unpack_trees(1, &t, &opts)) {\n+\tcase 0: break;\n+\tcase -2:\n+\t\t/*\n+\t\t * failed to checkout; install index so that tools like\n+\t\t * git-status give sensible results, but die below.\n+\t\t */\n+\t\terr = -1;\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(_(\"unable to create index\"));\n+\t}\n \n \tfree(head);\n \n \tif (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))\n \t\tdie(_(\"unable to write new index file\"));\n \n+\tif (err)\n+\t\tdie(_(\"unable to checkout working tree\"));\n+\n \terr |= run_hook_le(NULL, \"post-checkout\", oid_to_hex(null_oid()),\n \t\t\t   oid_to_hex(&oid), \"1\", NULL);\n \n\nwould help. But the outcome from git-status is weird. Because the long\nname chokes stat(), it doesn't report it as a change, even though the\nfile is missing! This seems like a bug on the reading side.\n\nI'm also still puzzled at the fact that on Linux we barf in\nverify_absent(), but somehow don't on Windows.\n\n-Peff\n"}]}