{"thread":{"id":"54891","subject":"[BUG] Destructive access to an \"objects/info/alternates\" repository","startedAt":"2020-12-28T18:51:37Z","lastAt":"2021-01-04T23:48:02Z","messageCount":9,"participants":["Stefan Monnier","Konstantin Ryabitsev"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"413084","messageId":"jwvpn2tdb0r.fsf-monnier+gmane.comp.version-control.git@gnu.org","threadId":"54891","inReplyTo":null,"subject":"[BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2020-12-28T18:50:43Z","receivedAt":"2020-12-28T18:51:37Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":"[ Resending with a subject that clarifies that it's a bug report.  ]\n\n> Thank you for filling out a Git bug report!\n> Please answer the following questions to help us understand your issue.\n\n> What did you do before the bug happened? (Steps to reproduce your\n> issue)\n\nI did basically:\n\n    git clone --reference ~otheruser/nongnu\n    [ Days passed and I did various things.  ]\n    cd nongnu; git gc\n\nNote that `otheruser` is [..wait for it..] another user, so I only have\nread access to the referenced repository.\n\n> What did you expect to happen? (Expected behavior)\n\nThat it would GC the local part of the repository and that it wouldn't\ntry to modify the referenced repository.\n\n> What happened instead? (Actual behavior)\n\nI got warnings indicating that Git was trying to delete files from\nthe referenced repository:\n\n    Enumerating objects: 35095, done.\n    Nothing new to pack.\n    warning: unable to unlink\n'/home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx':\nPermission denied\n    warning: unable to unlink\n'/home/otheruser/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx':\nPermission denied\n\n> What's different between what you expected and what actually happened?\n\nThe warnings.\n\n> Anything else you want to add:\n\nThe warnings make me wonder if I should worry about other possible\nmisbehaviors when using such a setup.\n\n> Please review the rest of the bug report below.\n> You can delete any lines you don't wish to share.\n\nThis is on a Debian stable system but with Git from Debian testing\n(because the version from Debian stable crashed in a `git log` command\nI was using in my scripts).\n\n[System Info]\ngit version:\ngit version 2.29.2\ncpu: x86_64\nno commit associated with this build\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nuname: Linux 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) x86_64\ncompiler info: gnuc: 10.2\nlibc info: glibc: 2.28\n$SHELL (typically, interactive shell): /bin/bash\n\n\n[Enabled Hooks]\n\n\n\n"},{"id":"413085","messageId":"20201228190036.vnkgeu6puxmvgt5s@chatter.i7.local","threadId":"54891","inReplyTo":"jwvpn2tdb0r.fsf-monnier+gmane.comp.version-control.git@gnu.org","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-12-28T19:00:36Z","receivedAt":"2020-12-28T19:01:21Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Dec 28, 2020 at 01:50:43PM -0500, Stefan Monnier wrote:\n> [ Resending with a subject that clarifies that it's a bug report.  ]\n> \n> > Thank you for filling out a Git bug report!\n> > Please answer the following questions to help us understand your issue.\n> \n> > What did you do before the bug happened? (Steps to reproduce your\n> > issue)\n> \n> I did basically:\n> \n>     git clone --reference ~otheruser/nongnu\n\nThis doesn't appear to be a proper command, because it's missing actual\nrepository to clone, only the repository to use as reference. This is the\nproper usage of --reference:\n\ngit clone --reference ~/some/repo https://some/remote/repo.git\n\nPerhaps you meant --shared?\n\nWithout knowing exactly what you did, it's hard to tell what went wrong. You\ncan try running \"find nongnu\" to see if what you think is your own checkout is\nactually a symlink or some combination thereof.\n\n-K\n"},{"id":"413094","messageId":"jwv8s9hd9cg.fsf-monnier+Inbox@gnu.org","threadId":"54891","inReplyTo":"20201228190036.vnkgeu6puxmvgt5s@chatter.i7.local","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2020-12-28T19:29:03Z","receivedAt":"2020-12-28T19:30:04Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":">>     git clone --reference ~otheruser/nongnu\n>\n> This doesn't appear to be a proper command, because it's missing actual\n> repository to clone, only the repository to use as reference. This is the\n> proper usage of --reference:\n>\n> git clone --reference ~/some/repo https://some/remote/repo.git\n\nIndeed, thanks:\n\ngit clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git\n\n> Without knowing exactly what you did, it's hard to tell what went wrong. You\n> can try running \"find nongnu\" to see if what you think is your own checkout is\n> actually a symlink or some combination thereof.\n\nNot sure what you mean by that.\n\nIn any case, I just reproduced the problem:\n\n    % git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git\n    Cloning into 'nongnu'...\n    Fetching objects: 36974, done.\n    % cd nongnu \n    % git gc\n    Enumerating objects: 36974, done.\n    Nothing new to pack.\n    warning: unable to unlink '/home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx': Permission denied\n    warning: unable to unlink '/home/otheruser/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx': Permission denied\n    % ls -l /home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx\n    -rw-r--r-- 1 otheruser otheruser 975612 Dec 11 16:08 /home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx\n    % ls -ld /home/otheruser/nongnu/.git/objects/pack\n    drwxr-xr-x 2 otheruser otheruser 4096 Dec 27 17:57 /home/otheruser/nongnu/.git/objects/pack/\n    %\n\n\n-- Stefan\n\n"},{"id":"413107","messageId":"jwvlfdhbsbs.fsf-monnier+gmane.comp.version-control.git@gnu.org","threadId":"54891","inReplyTo":"jwv8s9hd9cg.fsf-monnier+Inbox@gnu.org","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2020-12-28T20:17:01Z","receivedAt":"2020-12-28T23:23:03Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":">>>     git clone --reference ~otheruser/nongnu\n>>\n>> This doesn't appear to be a proper command, because it's missing actual\n>> repository to clone, only the repository to use as reference. This is the\n>> proper usage of --reference:\n>>\n>> git clone --reference ~/some/repo https://some/remote/repo.git\n>\n> Indeed, thanks:\n>\n> git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git\n>\n>> Without knowing exactly what you did, it's hard to tell what went wrong. You\n>> can try running \"find nongnu\" to see if what you think is your own checkout is\n>> actually a symlink or some combination thereof.\n>\n> Not sure what you mean by that.\n>\n> In any case, I just reproduced the problem:\n\nFWIW, I just tried it with\n\n    git clone --shared ~otheruser/nongnu\n\nand got the same result.\nI also tried with another repository, still with the same result (tho\ndifferent pack.idx names, of course).\n\n\n        Stefan\n\n"},{"id":"413129","messageId":"20201229154403.xutnk2aoawdrjfwx@chatter.i7.local","threadId":"54891","inReplyTo":"jwvlfdhbsbs.fsf-monnier+gmane.comp.version-control.git@gnu.org","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2020-12-29T15:44:03Z","receivedAt":"2020-12-29T15:44:48Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Dec 28, 2020 at 03:17:01PM -0500, Stefan Monnier wrote:\n> FWIW, I just tried it with\n> \n>     git clone --shared ~otheruser/nongnu\n> \n> and got the same result.\n> I also tried with another repository, still with the same result (tho\n> different pack.idx names, of course).\n\nI've tried several times to reproduce this, but I am unable to do so. You may\nneed to provide the tarball with ~otheruser/nongnu contents for someone to be\nable to properly debug this.\n\n-K\n"},{"id":"413338","messageId":"jwv1rf26k9n.fsf-monnier+gmane.comp.version-control.git@gnu.org","threadId":"54891","inReplyTo":"20201229154403.xutnk2aoawdrjfwx@chatter.i7.local","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2021-01-03T16:50:35Z","receivedAt":"2021-01-03T16:56:04Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":">> FWIW, I just tried it with\n>> \n>>     git clone --shared ~otheruser/nongnu\n>> \n>> and got the same result.\n>> I also tried with another repository, still with the same result (tho\n>> different pack.idx names, of course).\n>\n> I've tried several times to reproduce this, but I am unable to do so. You may\n> need to provide the tarball with ~otheruser/nongnu contents for someone to be\n> able to properly debug this.\n\nOK, you can try with the following tarball:\n\n    https://www.iro.umontreal.ca/~monnier/nongnu.tar.gz\n\nuntar it (as \"otheruser\") into ~otheruser (which will create\n~otheruser/nongnu) and then do (as your normal user):\n\n    git clone --shared ~otheruser/nongnu\n    cd nongnu\n    git gc\n\nIt \"worked\" (i.e. gave me the unlink failure warnings) for me, so\nhopefully it will \"work\" for others as well.\n\nThe `git clone` will likely give you some message about HEAD not being\nfound, which is normal.\n\n\n        Stefan\n\n"},{"id":"413372","messageId":"20210104135410.myjaygaulqnxcnsc@chatter.i7.local","threadId":"54891","inReplyTo":"jwv1rf26k9n.fsf-monnier+gmane.comp.version-control.git@gnu.org","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2021-01-04T13:54:10Z","receivedAt":"2021-01-04T13:54:55Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Sun, Jan 03, 2021 at 11:50:35AM -0500, Stefan Monnier wrote:\n> >> FWIW, I just tried it with\n> >> \n> >>     git clone --shared ~otheruser/nongnu\n> >> \n> >> and got the same result.\n> >> I also tried with another repository, still with the same result (tho\n> >> different pack.idx names, of course).\n> >\n> > I've tried several times to reproduce this, but I am unable to do so. You may\n> > need to provide the tarball with ~otheruser/nongnu contents for someone to be\n> > able to properly debug this.\n> \n> OK, you can try with the following tarball:\n> \n>     https://www.iro.umontreal.ca/~monnier/nongnu.tar.gz\n\nGreat, thank you for providing that. At this point, I can verify the behaviour\nyou are seeing:\n\n$ cat .git/objects/info/alternates\n/tmp/test/nongnu/.git/objects\n$ git gc\nEnumerating objects: 119, done.\nNothing new to pack.\nwarning: unable to unlink '/tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx': Permission denied\nwarning: unable to unlink '/tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx': Permission denied\n\nNotably, if I run repack/prune by themselves, this does not happen:\n\n$ git repack -Adln\nEnumerating objects: 119, done.\nNothing new to pack.\n$ git prune --expire=now\n$\n\nI would guess that this is happening in builtin/gc.c#686 [1], where it calls\nclean_pack_garbage(). Running \"git count-objects -v\" shows:\n\n$ git count-objects -v\nwarning: no corresponding .pack: /tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx\nwarning: no corresponding .pack: /tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx\ncount: 0\nsize: 0\nin-pack: 0\npacks: 0\nsize-pack: 0\nprune-packable: 0\ngarbage: 2\nsize-garbage: 6573\nalternate: /tmp/test/nongnu/.git/objects\n\nI agree with Stefan that this is undesired behaviour, even when it only\nhappens when attempting to clean up garbage -- git-gc (and git-count-objects)\nshould distinguish between garbage in the parent repository and its own\nrepository and not attempt any modification of the parent repository during\ngc or any other operation.\n\n[1] https://git.kernel.org/pub/scm/git/git.git/tree/builtin/gc.c#n686\n\n-K\n"},{"id":"413386","messageId":"jwvlfd81le8.fsf-monnier+gmane.comp.version-control.git@gnu.org","threadId":"54891","inReplyTo":"20210104135410.myjaygaulqnxcnsc@chatter.i7.local","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Stefan Monnier","fromEmail":"monnier@iro.umontreal.ca","sentAt":"2021-01-04T20:47:53Z","receivedAt":"2021-01-04T20:48:55Z","isPatch":false,"sender":{"key":"monnier@iro.umontreal.ca","avatar":null},"body":"Konstantin Ryabitsev [2021-01-04 08:54:10] wrote:\n> I would guess that this is happening in builtin/gc.c#686 [1], where it calls\n> clean_pack_garbage(). Running \"git count-objects -v\" shows:\n>\n> $ git count-objects -v\n> warning: no corresponding .pack:\n> /tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx\n> warning: no corresponding .pack:\n> /tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx\n> count: 0\n> size: 0\n> in-pack: 0\n> packs: 0\n> size-pack: 0\n> prune-packable: 0\n> garbage: 2\n> size-garbage: 6573\n> alternate: /tmp/test/nongnu/.git/objects\n\nAh, that'd make sense, yes.\n\n> I agree with Stefan that this is undesired behaviour, even when it only\n> happens when attempting to clean up garbage -- git-gc (and git-count-objects)\n> should distinguish between garbage in the parent repository and its own\n> repository and not attempt any modification of the parent repository during\n> gc or any other operation.\n\nGreat, so IIUC:\n- this is a harmless warning\n- it's considered as a bug\n- so my use case should work fine in practice and is not considered as\n  \"too weird to be supported\"\nThank you,\n\n\n        Stefan\n\n"},{"id":"413389","messageId":"20210104212020.qnokgnpvsoxlm77j@chatter.i7.local","threadId":"54891","inReplyTo":"jwvlfd81le8.fsf-monnier+gmane.comp.version-control.git@gnu.org","subject":"Re: [BUG] Destructive access to an \"objects/info/alternates\" repository","fromName":"Konstantin Ryabitsev","fromEmail":"konstantin@linuxfoundation.org","sentAt":"2021-01-04T21:20:20Z","receivedAt":"2021-01-04T23:48:02Z","isPatch":false,"sender":{"key":"konstantin@linuxfoundation.org","avatar":"https://gravatar.com/avatar/7cb8827c6de56e1bd2dea16508c6708aa43feed3bf3813bcdacecdf96ceadd79?d=mp&s=160"},"body":"On Mon, Jan 04, 2021 at 03:47:53PM -0500, Stefan Monnier wrote:\n> > I agree with Stefan that this is undesired behaviour, even when it only\n> > happens when attempting to clean up garbage -- git-gc (and git-count-objects)\n> > should distinguish between garbage in the parent repository and its own\n> > repository and not attempt any modification of the parent repository during\n> > gc or any other operation.\n> \n> Great, so IIUC:\n> - this is a harmless warning\n\nWell, this is a benign situation in general, since git only tries to delete\nfiles that it determined to be garbage. Even if it succeeded in your case, it\nwouldn't really modify the actual shared repository state/contents.\n\n> - it's considered as a bug\n\nI'm not really someone who decides if it's a bug or not. :) I *think* it's a\nbug, since git modifies the repository from which it should be merely\nborrowing objects (a read-only operation). Even if the results are benign,\nit's not something that should be happening, in my opinion.\n\n> - so my use case should work fine in practice and is not considered as\n>   \"too weird to be supported\"\n\nIt's easy to replicate, if someone feels like fixing it:\n\nmkdir repo1\ncd repo1\ngit init\necho test > test\ngit add test\ngit commit -am test\ntouch .git/objects/pack/pack-abcd.idx\ncd ..\ngit clone -s repo1 repo2\ncd repo2\ngit gc\nls -al ../repo1/.git/objects/pack/pack-abcd.idx\n/bin/ls: cannot access '../repo1/.git/objects/pack/pack-abcd.idx': No such file or directory\n\nHope this helps.\n\n-K\n"}]}