threads / bug / 54891

[BUG] Destructive access to an "objects/info/alternates" repository

Subject: [BUG] Destructive access to an "objects/info/alternates" repository

## tl;dr

9 messages between Dec 28, 2020 and Jan 4, 2021.

replies: 8people: 2as markdown or json

Stefan Monnier· Dec 28, 2020, 18:50 UTC · lore
[ Resending with a subject that clarifies that it's a bug report.  ]
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> What did you do before the bug happened? (Steps to reproduce your
> issue)
I did basically:
    git clone --reference ~otheruser/nongnu
    [ Days passed and I did various things.  ]
    cd nongnu; git gc

Note that `otheruser` is [..wait for it..] another user, so I only have read access to the referenced repository.

> What did you expect to happen? (Expected behavior)

That it would GC the local part of the repository and that it wouldn't try to modify the referenced repository.

> What happened instead? (Actual behavior)

I got warnings indicating that Git was trying to delete files from the referenced repository:

    Enumerating objects: 35095, done.
    Nothing new to pack.
    warning: unable to unlink
'/home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx':
Permission denied
    warning: unable to unlink
'/home/otheruser/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx':
Permission denied
> What's different between what you expected and what actually happened?
The warnings.
> Anything else you want to add:

The warnings make me wonder if I should worry about other possible misbehaviors when using such a setup.

> Please review the rest of the bug report below.
> You can delete any lines you don't wish to share.

This is on a Debian stable system but with Git from Debian testing (because the version from Debian stable crashed in a `git log` command I was using in my scripts).

[System Info] git version: git version 2.29.2 cpu: x86_64 no commit associated with this build sizeof-long: 8 sizeof-size_t: 8 shell-path: /bin/sh uname: Linux 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) x86_64 compiler info: gnuc: 10.2 libc info: glibc: 2.28 $SHELL (typically, interactive shell): /bin/bash

[Enabled Hooks]
Konstantin Ryabitsev· Dec 28, 2020, 19:00 UTC · re: Stefan Monnier · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

On Mon, Dec 28, 2020 at 01:50:43PM -0500, Stefan Monnier wrote:
Show 11 quoted lines
> [ Resending with a subject that clarifies that it's a bug report.  ]
> 
> > Thank you for filling out a Git bug report!
> > Please answer the following questions to help us understand your issue.
> 
> > What did you do before the bug happened? (Steps to reproduce your
> > issue)
> 
> I did basically:
> 
>     git clone --reference ~otheruser/nongnu

This doesn't appear to be a proper command, because it's missing actual repository to clone, only the repository to use as reference. This is the proper usage of --reference:

git clone --reference ~/some/repo https://some/remote/repo.git
Perhaps you meant --shared?

Without knowing exactly what you did, it's hard to tell what went wrong. You can try running "find nongnu" to see if what you think is your own checkout is actually a symlink or some combination thereof.

-K
Stefan Monnier· Dec 28, 2020, 19:29 UTC · re: Konstantin Ryabitsev · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

Show 7 quoted lines
>>     git clone --reference ~otheruser/nongnu
>
> This doesn't appear to be a proper command, because it's missing actual
> repository to clone, only the repository to use as reference. This is the
> proper usage of --reference:
>
> git clone --reference ~/some/repo https://some/remote/repo.git
Indeed, thanks:
git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git
> Without knowing exactly what you did, it's hard to tell what went wrong. You
> can try running "find nongnu" to see if what you think is your own checkout is
> actually a symlink or some combination thereof.
Not sure what you mean by that.
In any case, I just reproduced the problem:
    % git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git
    Cloning into 'nongnu'...
    Fetching objects: 36974, done.
    % cd nongnu 
    % git gc
    Enumerating objects: 36974, done.
    Nothing new to pack.
    warning: unable to unlink '/home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx': Permission denied
    warning: unable to unlink '/home/otheruser/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx': Permission denied
    % ls -l /home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
    -rw-r--r-- 1 otheruser otheruser 975612 Dec 11 16:08 /home/otheruser/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
    % ls -ld /home/otheruser/nongnu/.git/objects/pack
    drwxr-xr-x 2 otheruser otheruser 4096 Dec 27 17:57 /home/otheruser/nongnu/.git/objects/pack/
    %
-- Stefan
Stefan Monnier· Dec 28, 2020, 20:17 UTC · re: Stefan Monnier · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

Show 19 quoted lines
>>>     git clone --reference ~otheruser/nongnu
>>
>> This doesn't appear to be a proper command, because it's missing actual
>> repository to clone, only the repository to use as reference. This is the
>> proper usage of --reference:
>>
>> git clone --reference ~/some/repo https://some/remote/repo.git
>
> Indeed, thanks:
>
> git clone --reference ~otheruser/nongnu https://git.savannah.gnu.org/r/emacs/nongnu.git
>
>> Without knowing exactly what you did, it's hard to tell what went wrong. You
>> can try running "find nongnu" to see if what you think is your own checkout is
>> actually a symlink or some combination thereof.
>
> Not sure what you mean by that.
>
> In any case, I just reproduced the problem:
FWIW, I just tried it with
    git clone --shared ~otheruser/nongnu

and got the same result. I also tried with another repository, still with the same result (tho different pack.idx names, of course).

        Stefan
Konstantin Ryabitsev· Dec 29, 2020, 15:44 UTC · re: Stefan Monnier · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

On Mon, Dec 28, 2020 at 03:17:01PM -0500, Stefan Monnier wrote:
Show 7 quoted lines
> FWIW, I just tried it with
> 
>     git clone --shared ~otheruser/nongnu
> 
> and got the same result.
> I also tried with another repository, still with the same result (tho
> different pack.idx names, of course).

I've tried several times to reproduce this, but I am unable to do so. You may need to provide the tarball with ~otheruser/nongnu contents for someone to be able to properly debug this.

-K
Stefan Monnier· Jan 3, 2021, 16:50 UTC · re: Konstantin Ryabitsev · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

Show 11 quoted lines
>> FWIW, I just tried it with
>> 
>>     git clone --shared ~otheruser/nongnu
>> 
>> and got the same result.
>> I also tried with another repository, still with the same result (tho
>> different pack.idx names, of course).
>
> I've tried several times to reproduce this, but I am unable to do so. You may
> need to provide the tarball with ~otheruser/nongnu contents for someone to be
> able to properly debug this.
OK, you can try with the following tarball:
    https://www.iro.umontreal.ca/~monnier/nongnu.tar.gz

untar it (as "otheruser") into ~otheruser (which will create ~otheruser/nongnu) and then do (as your normal user):

    git clone --shared ~otheruser/nongnu
    cd nongnu
    git gc

It "worked" (i.e. gave me the unlink failure warnings) for me, so hopefully it will "work" for others as well.

The `git clone` will likely give you some message about HEAD not being found, which is normal.

        Stefan
Konstantin Ryabitsev· Jan 4, 2021, 13:54 UTC · re: Stefan Monnier · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

On Sun, Jan 03, 2021 at 11:50:35AM -0500, Stefan Monnier wrote:
Show 15 quoted lines
> >> FWIW, I just tried it with
> >> 
> >>     git clone --shared ~otheruser/nongnu
> >> 
> >> and got the same result.
> >> I also tried with another repository, still with the same result (tho
> >> different pack.idx names, of course).
> >
> > I've tried several times to reproduce this, but I am unable to do so. You may
> > need to provide the tarball with ~otheruser/nongnu contents for someone to be
> > able to properly debug this.
> 
> OK, you can try with the following tarball:
> 
>     https://www.iro.umontreal.ca/~monnier/nongnu.tar.gz

Great, thank you for providing that. At this point, I can verify the behaviour you are seeing:

$ cat .git/objects/info/alternates /tmp/test/nongnu/.git/objects $ git gc Enumerating objects: 119, done. Nothing new to pack. warning: unable to unlink '/tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx': Permission denied warning: unable to unlink '/tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx': Permission denied

Notably, if I run repack/prune by themselves, this does not happen:

$ git repack -Adln Enumerating objects: 119, done. Nothing new to pack. $ git prune --expire=now $

I would guess that this is happening in builtin/gc.c#686 [1], where it calls clean_pack_garbage(). Running "git count-objects -v" shows:

$ git count-objects -v warning: no corresponding .pack: /tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx warning: no corresponding .pack: /tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx count: 0 size: 0 in-pack: 0 packs: 0 size-pack: 0 prune-packable: 0 garbage: 2 size-garbage: 6573 alternate: /tmp/test/nongnu/.git/objects

I agree with Stefan that this is undesired behaviour, even when it only happens when attempting to clean up garbage -- git-gc (and git-count-objects) should distinguish between garbage in the parent repository and its own repository and not attempt any modification of the parent repository during gc or any other operation.

[1] https://git.kernel.org/pub/scm/git/git.git/tree/builtin/gc.c#n686
-K
Stefan Monnier· Jan 4, 2021, 20:47 UTC · re: Konstantin Ryabitsev · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

Konstantin Ryabitsev [2021-01-04 08:54:10] wrote:
Show 17 quoted lines
> I would guess that this is happening in builtin/gc.c#686 [1], where it calls
> clean_pack_garbage(). Running "git count-objects -v" shows:
>
> $ git count-objects -v
> warning: no corresponding .pack:
> /tmp/test/nongnu/.git/objects/pack/pack-0d85e74ac2f7e51ce26f281e64eb738e8182fa95.idx
> warning: no corresponding .pack:
> /tmp/test/nongnu/.git/objects/pack/pack-5fe14feff49ccdee5469af9dc94f6784e8464a6b.idx
> count: 0
> size: 0
> in-pack: 0
> packs: 0
> size-pack: 0
> prune-packable: 0
> garbage: 2
> size-garbage: 6573
> alternate: /tmp/test/nongnu/.git/objects
Ah, that'd make sense, yes.
Show 5 quoted lines
> I agree with Stefan that this is undesired behaviour, even when it only
> happens when attempting to clean up garbage -- git-gc (and git-count-objects)
> should distinguish between garbage in the parent repository and its own
> repository and not attempt any modification of the parent repository during
> gc or any other operation.
Great, so IIUC:
- this is a harmless warning
- it's considered as a bug
- so my use case should work fine in practice and is not considered as
  "too weird to be supported"
Thank you,
        Stefan
Konstantin Ryabitsev· Jan 4, 2021, 21:20 UTC · re: Stefan Monnier · lore

Re: [BUG] Destructive access to an "objects/info/alternates" repository

On Mon, Jan 04, 2021 at 03:47:53PM -0500, Stefan Monnier wrote:
Show 8 quoted lines
> > I agree with Stefan that this is undesired behaviour, even when it only
> > happens when attempting to clean up garbage -- git-gc (and git-count-objects)
> > should distinguish between garbage in the parent repository and its own
> > repository and not attempt any modification of the parent repository during
> > gc or any other operation.
> 
> Great, so IIUC:
> - this is a harmless warning

Well, this is a benign situation in general, since git only tries to delete files that it determined to be garbage. Even if it succeeded in your case, it wouldn't really modify the actual shared repository state/contents.

> - it's considered as a bug

I'm not really someone who decides if it's a bug or not. :) I *think* it's a bug, since git modifies the repository from which it should be merely borrowing objects (a read-only operation). Even if the results are benign, it's not something that should be happening, in my opinion.

> - so my use case should work fine in practice and is not considered as
>   "too weird to be supported"
It's easy to replicate, if someone feels like fixing it:

mkdir repo1 cd repo1 git init echo test > test git add test git commit -am test touch .git/objects/pack/pack-abcd.idx cd .. git clone -s repo1 repo2 cd repo2 git gc ls -al ../repo1/.git/objects/pack/pack-abcd.idx /bin/ls: cannot access '../repo1/.git/objects/pack/pack-abcd.idx': No such file or directory

Hope this helps.
-K

← back to recent threads