threads / discuss / 61639

Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Subject: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

## tl;dr

20 messages between Jun 17, 2024 and Sep 25, 2024.

replies: 19people: 6as markdown or json

David C. Rankin· Jun 17, 2024, 00:36 UTC · lore

All,

   Security enhancement in 2.45.1 have broken ability to serve git over https 
and ssh from local git server running Apache. (web server runs as http:http on 
Archlinux)

   The fix of adding the following to gitconfig (system-wide and per-user in 
~/.gitconfig) does not solve the problem:

[safe]
	directory = *

(* or the actual /srv/git/reponame.git makes no difference)

   On Archlinux, all repos are served from /srv/git. This has worked well for 
both https and ssh allowing repos under /srv/git to be owned by the user with 
public-private ssh key. (they are members of group owning /srv/git with write 
privileges)

   After 2.45.1,

   - git will not allow https unless the repositories are OWNED by http.
   - git will not allow ssh   unless the repositories are OWNED by user.

   A catch-22.

   I've tried every possible file repository ownership of http:user and 
user:http with permissions of 0775, but no luck, it is either one or the 
other. I've even tried making users members of the http group, but ssh still 
refuses push unless the repository is OWNED by user.

   The errors run the gamut from https attmepts:

$ git pull fatal: unable to access 'https://www.mydomain.com/git/examples.git/': The requested URL returned error: 500

to ssh attempts:

$ git push
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 1.01 KiB | 1.01 MiB/s, done.
Total 5 (delta 2), reused 0 (delta 0), pack-reused 0
remote: error: cannot lock ref 'HEAD': Unable to create 
'/srv/git/pico.git/./HEAD.lock': Permission denied
To valkyrie:/srv/git/pico.git
  ! [remote rejected] master -> master (failed to update ref)
error: failed to push some refs to 'valkyrie:/srv/git/pico.git'

   This was discussed on the arch-general mailing list under the thread "git 
server changes - how to allow https AND ssh now that /srv/git/xxx.git must be 
owned by http?" at 
https://lists.archlinux.org/archives/list/arch-general@lists.archlinux.org/thread/3GCCU6QZNGRY45WMQAQEVF572AIHN646/

   There was a suggestion to try a bind mount for the repositories, but that 
seems like it may introduce other issues, but if that is the correct approach 
I'm happy to try it.

   What is the correct local-server way to now serve repositories over https 
AND ssh from Apache without running into this either-or ownership problem?

(gitweb continues to work fine)

-- 
David C. Rankin, J.D.,P.E.

Junio C Hamano· Jun 17, 2024, 18:47 UTC · re: David C. Rankin · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

"David C. Rankin" <drankinatty@gmail.com> writes:

Show 9 quoted lines

>   Security enhancement in 2.45.1 have broken ability to serve git over
>   https and ssh from local git server running Apache. (web server runs
>   as http:http on Archlinux)
>
>   The fix of adding the following to gitconfig (system-wide and
>   per-user in ~/.gitconfig) does not solve the problem:
>
> [safe]
> 	directory = *

It is not clear what you exactly meant "per-user" above, so just to make sure. Is this set in the global configuration file for the httpd (or whoever Apache runs as) user?

The purpose of "dubious ownershop" thing is to protect the user who runs Git from random repositories' with potentially malicious hooks and configuration files, so the user being protected (in this case, whoever Apache runs as) needs to declare "I trust these repositories" in its ~/.gitconfig file. What individual owners of /srv/my-repo.git/ project has in their ~/.gitconfig file does not matter when deciding if Apache trusts these repositories.

Michal Suchánek· Jun 17, 2024, 21:15 UTC · re: Junio C Hamano · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Hello,

On Mon, Jun 17, 2024 at 11:47:20AM -0700, Junio C Hamano wrote:

Show 23 quoted lines

> "David C. Rankin" <drankinatty@gmail.com> writes:
> 
> >   Security enhancement in 2.45.1 have broken ability to serve git over
> >   https and ssh from local git server running Apache. (web server runs
> >   as http:http on Archlinux)
> >
> >   The fix of adding the following to gitconfig (system-wide and
> >   per-user in ~/.gitconfig) does not solve the problem:
> >
> > [safe]
> > 	directory = *
> 
> It is not clear what you exactly meant "per-user" above, so just to
> make sure.  Is this set in the global configuration file for the
> httpd (or whoever Apache runs as) user?
> 
> The purpose of "dubious ownershop" thing is to protect the user who
> runs Git from random repositories' with potentially malicious hooks
> and configuration files, so the user being protected (in this case,
> whoever Apache runs as) needs to declare "I trust these
> repositories" in its ~/.gitconfig file.  What individual owners of
> /srv/my-repo.git/ project has in their ~/.gitconfig file does not
> matter when deciding if Apache trusts these repositories.

looks like the semantic of 'dubious ownershop' changed recently.

Disro backport of fixes for CVE-2024-32002 CVE-2024-32004 CVE-2024-32020 CVE-2024-32021 CVE-2024-32465 to 2.35.3 broke git-daemon. No amount of whitelisting makes the 'fixed' git serve the repository.

Of course, this might be bug in the backpotrt as much as but in the upstream logic, not analyzed yet.

However, serving repositories is a gaping security hole right now. At least on some 'stable' distributions.

Thanks

Michal

Michal Suchánek· Jun 25, 2024, 07:24 UTC · re: Michal Suchánek · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

On Mon, Jun 17, 2024 at 11:15:13PM +0200, Michal Suchánek wrote:

Show 33 quoted lines

> Hello,
> 
> On Mon, Jun 17, 2024 at 11:47:20AM -0700, Junio C Hamano wrote:
> > "David C. Rankin" <drankinatty@gmail.com> writes:
> > 
> > >   Security enhancement in 2.45.1 have broken ability to serve git over
> > >   https and ssh from local git server running Apache. (web server runs
> > >   as http:http on Archlinux)
> > >
> > >   The fix of adding the following to gitconfig (system-wide and
> > >   per-user in ~/.gitconfig) does not solve the problem:
> > >
> > > [safe]
> > > 	directory = *
> > 
> > It is not clear what you exactly meant "per-user" above, so just to
> > make sure.  Is this set in the global configuration file for the
> > httpd (or whoever Apache runs as) user?
> > 
> > The purpose of "dubious ownershop" thing is to protect the user who
> > runs Git from random repositories' with potentially malicious hooks
> > and configuration files, so the user being protected (in this case,
> > whoever Apache runs as) needs to declare "I trust these
> > repositories" in its ~/.gitconfig file.  What individual owners of
> > /srv/my-repo.git/ project has in their ~/.gitconfig file does not
> > matter when deciding if Apache trusts these repositories.
> 
> 
> looks like the semantic of 'dubious ownershop' changed recently.
> 
> Disro backport of fixes for CVE-2024-32002 CVE-2024-32004 CVE-2024-32020
> CVE-2024-32021 CVE-2024-32465 to 2.35.3 broke git-daemon. No amount of
> whitelisting makes the 'fixed' git serve the repository.

Same regression between 2.45.0 and 2.45.2 which allegedly fixes the same CVEs.

Looks like downgrading to gaping hole version is needed to serve repositories in general.

Please consider adjusting the fix so that repositories can still be served.

Thanks

Michal

To reproduce:

cat /usr/local/bin/git-ping #!/bin/sh -e

# Try connecting to one or more remote repository URLs

while true ; do
        git ls-remote -h "$1" >/dev/null
        shift
        [ -n "$1" ] || break
done

mkdir -p /srv/git/some chown hramrach /srv/git/some su hramrach -c "git init --bare /srv/git/some/repo.git" su hramrach -c "touch /srv/git/some/repo.git/git-daemon-export-ok" version=2.35.3-150300.10.36.1 ; zypper in --oldpackage git-core-$version git-daemon-$version systemctl start git-daemon.service git ping git://localhost/some/repo.git <nothing>

version=2.35.3-150300.10.39.1 ; zypper in --oldpackage git-core-$version git-daemon-$version systemctl restart git-daemon.service git ping git://localhost/some/repo.git fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

systemctl status git-daemon.service
● git-daemon.service - Git Daemon
     Loaded: loaded (/usr/lib/systemd/system/git-daemon.service; disabled; vendor preset: disabled)
     Active: active (running) since Thu 2024-06-06 08:29:28 CEST; 6min ago
   Main PID: 31742 (git)
      Tasks: 2 (limit: 4915)
     CGroup: /system.slice/git-daemon.service
             ├─ 31742 git daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
             └─ 31749 /usr/lib/git/git-daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup

Jun 06 08:29:28 localhost.localdomain systemd[1]: Started Git Daemon. Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: fatal: detected dubious ownership in repository at '/srv/git//some/repo.git' Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: To add an exception for this directory, call: Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: git config --global --add safe.directory /srv/git//some/repo.git

git config --global --add safe.directory /srv/git//some/repo.git mv ~/.gitconfig /etc/gitconfig git ping git://localhost/some/repo.git fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

git config --global --add safe.directory /srv/git/some/repo.git mv ~/.gitconfig /etc/gitconfig git ping git://localhost/some/repo.git fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

Junio C Hamano· Jun 25, 2024, 16:12 UTC · re: Michal Suchánek · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Dscho, the f4aa8c8b (fetch/clone: detect dubious ownership of local repositories, 2024-04-10) is your brainchild and people seem to be unhappy about having to adjust their settings. Are there any advice you can offer them?

Michal Suchánek <msuchanek@suse.de> writes:

Show 108 quoted lines

> On Mon, Jun 17, 2024 at 11:15:13PM +0200, Michal Suchánek wrote:
>> Hello,
>> 
>> On Mon, Jun 17, 2024 at 11:47:20AM -0700, Junio C Hamano wrote:
>> > "David C. Rankin" <drankinatty@gmail.com> writes:
>> > 
>> > >   Security enhancement in 2.45.1 have broken ability to serve git over
>> > >   https and ssh from local git server running Apache. (web server runs
>> > >   as http:http on Archlinux)
>> > >
>> > >   The fix of adding the following to gitconfig (system-wide and
>> > >   per-user in ~/.gitconfig) does not solve the problem:
>> > >
>> > > [safe]
>> > > 	directory = *
>> > 
>> > It is not clear what you exactly meant "per-user" above, so just to
>> > make sure.  Is this set in the global configuration file for the
>> > httpd (or whoever Apache runs as) user?
>> > 
>> > The purpose of "dubious ownershop" thing is to protect the user who
>> > runs Git from random repositories' with potentially malicious hooks
>> > and configuration files, so the user being protected (in this case,
>> > whoever Apache runs as) needs to declare "I trust these
>> > repositories" in its ~/.gitconfig file.  What individual owners of
>> > /srv/my-repo.git/ project has in their ~/.gitconfig file does not
>> > matter when deciding if Apache trusts these repositories.
>> 
>> 
>> looks like the semantic of 'dubious ownershop' changed recently.
>> 
>> Disro backport of fixes for CVE-2024-32002 CVE-2024-32004 CVE-2024-32020
>> CVE-2024-32021 CVE-2024-32465 to 2.35.3 broke git-daemon. No amount of
>> whitelisting makes the 'fixed' git serve the repository.
>
> Same regression between 2.45.0 and 2.45.2 which allegedly fixes the
> same CVEs.
>
> Looks like downgrading to gaping hole version is needed to serve repositories
> in general.
>
> Please consider adjusting the fix so that repositories can still be served.
>
> Thanks
>
> Michal
>
> To reproduce:
>
> cat /usr/local/bin/git-ping
> #!/bin/sh -e
>
> # Try connecting to one or more remote repository URLs
>
> while true ; do
>         git ls-remote -h "$1" >/dev/null
>         shift
>         [ -n "$1" ] || break
> done
>
> mkdir -p /srv/git/some
> chown hramrach /srv/git/some
> su hramrach -c "git init --bare /srv/git/some/repo.git"
> su hramrach -c "touch /srv/git/some/repo.git/git-daemon-export-ok"
> version=2.35.3-150300.10.36.1 ; zypper in --oldpackage git-core-$version git-daemon-$version
> systemctl start git-daemon.service
> git ping git://localhost/some/repo.git
> <nothing>
>
> version=2.35.3-150300.10.39.1 ; zypper in --oldpackage git-core-$version git-daemon-$version
> systemctl restart git-daemon.service
> git ping git://localhost/some/repo.git
> fatal: Could not read from remote repository.
>
> Please make sure you have the correct access rights
> and the repository exists.
>
>
> systemctl status git-daemon.service
> ● git-daemon.service - Git Daemon
>      Loaded: loaded (/usr/lib/systemd/system/git-daemon.service; disabled; vendor preset: disabled)
>      Active: active (running) since Thu 2024-06-06 08:29:28 CEST; 6min ago
>    Main PID: 31742 (git)
>       Tasks: 2 (limit: 4915)
>      CGroup: /system.slice/git-daemon.service
>              ├─ 31742 git daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
>              └─ 31749 /usr/lib/git/git-daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
>
> Jun 06 08:29:28 localhost.localdomain systemd[1]: Started Git Daemon.
> Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: fatal: detected dubious ownership in repository at '/srv/git//some/repo.git'
> Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: To add an exception for this directory, call:
> Jun 06 08:29:39 localhost.localdomain git-daemon[31756]:         git config --global --add safe.directory /srv/git//some/repo.git
>
> git config --global --add safe.directory /srv/git//some/repo.git
> mv ~/.gitconfig /etc/gitconfig
> git ping git://localhost/some/repo.git
> fatal: Could not read from remote repository.
>
> Please make sure you have the correct access rights
> and the repository exists.
>
> git config --global --add safe.directory /srv/git/some/repo.git
> mv ~/.gitconfig /etc/gitconfig
> git ping git://localhost/some/repo.git
> fatal: Could not read from remote repository.
>
> Please make sure you have the correct access rights
> and the repository exists.

Michal Suchánek· Jun 25, 2024, 18:34 UTC · re: Junio C Hamano · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Hello,

On Tue, Jun 25, 2024 at 09:12:38AM -0700, Junio C Hamano wrote:

> Dscho, the f4aa8c8b (fetch/clone: detect dubious ownership of local
> repositories, 2024-04-10) is your brainchild and people seem to be
> unhappy about having to adjust their settings.  Are there any advice
> you can offer them?

while I am somewhat unhappy about the need to adjust settings this is not really about that.

It is undesirable to implement breaking changes, especially in point release. At the same time it's sometimes necessary, and if the change is announced, documented, and actually works it can be acceptable.

Here git-daemon requests that repository be added to list of safe repositories to be able to serve it. Clearly this is a security requirement. A user that has write access to a repository can touch the export-ok file, and then the daemon would potentially serve the repository under some circumstances, potentially exposing the system administrator to some other things that the user did to the repository.

Adding the repository to the list of safe repositories is a known concept that was already required for gitweb and for working with the repository locally, and applying it to git-daemon as well is consistent although it does require configuration changes for some users.

The real problem here is that adding the repository to the list of safe repositories does not make it possible to serve it by git-daemon.

Thanks

Michal

Show 111 quoted lines

> 
> Michal Suchánek <msuchanek@suse.de> writes:
> 
> > On Mon, Jun 17, 2024 at 11:15:13PM +0200, Michal Suchánek wrote:
> >> Hello,
> >> 
> >> On Mon, Jun 17, 2024 at 11:47:20AM -0700, Junio C Hamano wrote:
> >> > "David C. Rankin" <drankinatty@gmail.com> writes:
> >> > 
> >> > >   Security enhancement in 2.45.1 have broken ability to serve git over
> >> > >   https and ssh from local git server running Apache. (web server runs
> >> > >   as http:http on Archlinux)
> >> > >
> >> > >   The fix of adding the following to gitconfig (system-wide and
> >> > >   per-user in ~/.gitconfig) does not solve the problem:
> >> > >
> >> > > [safe]
> >> > > 	directory = *
> >> > 
> >> > It is not clear what you exactly meant "per-user" above, so just to
> >> > make sure.  Is this set in the global configuration file for the
> >> > httpd (or whoever Apache runs as) user?
> >> > 
> >> > The purpose of "dubious ownershop" thing is to protect the user who
> >> > runs Git from random repositories' with potentially malicious hooks
> >> > and configuration files, so the user being protected (in this case,
> >> > whoever Apache runs as) needs to declare "I trust these
> >> > repositories" in its ~/.gitconfig file.  What individual owners of
> >> > /srv/my-repo.git/ project has in their ~/.gitconfig file does not
> >> > matter when deciding if Apache trusts these repositories.
> >> 
> >> 
> >> looks like the semantic of 'dubious ownershop' changed recently.
> >> 
> >> Disro backport of fixes for CVE-2024-32002 CVE-2024-32004 CVE-2024-32020
> >> CVE-2024-32021 CVE-2024-32465 to 2.35.3 broke git-daemon. No amount of
> >> whitelisting makes the 'fixed' git serve the repository.
> >
> > Same regression between 2.45.0 and 2.45.2 which allegedly fixes the
> > same CVEs.
> >
> > Looks like downgrading to gaping hole version is needed to serve repositories
> > in general.
> >
> > Please consider adjusting the fix so that repositories can still be served.
> >
> > Thanks
> >
> > Michal
> >
> > To reproduce:
> >
> > cat /usr/local/bin/git-ping
> > #!/bin/sh -e
> >
> > # Try connecting to one or more remote repository URLs
> >
> > while true ; do
> >         git ls-remote -h "$1" >/dev/null
> >         shift
> >         [ -n "$1" ] || break
> > done
> >
> > mkdir -p /srv/git/some
> > chown hramrach /srv/git/some
> > su hramrach -c "git init --bare /srv/git/some/repo.git"
> > su hramrach -c "touch /srv/git/some/repo.git/git-daemon-export-ok"
> > version=2.35.3-150300.10.36.1 ; zypper in --oldpackage git-core-$version git-daemon-$version
> > systemctl start git-daemon.service
> > git ping git://localhost/some/repo.git
> > <nothing>
> >
> > version=2.35.3-150300.10.39.1 ; zypper in --oldpackage git-core-$version git-daemon-$version
> > systemctl restart git-daemon.service
> > git ping git://localhost/some/repo.git
> > fatal: Could not read from remote repository.
> >
> > Please make sure you have the correct access rights
> > and the repository exists.
> >
> >
> > systemctl status git-daemon.service
> > ● git-daemon.service - Git Daemon
> >      Loaded: loaded (/usr/lib/systemd/system/git-daemon.service; disabled; vendor preset: disabled)
> >      Active: active (running) since Thu 2024-06-06 08:29:28 CEST; 6min ago
> >    Main PID: 31742 (git)
> >       Tasks: 2 (limit: 4915)
> >      CGroup: /system.slice/git-daemon.service
> >              ├─ 31742 git daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
> >              └─ 31749 /usr/lib/git/git-daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
> >
> > Jun 06 08:29:28 localhost.localdomain systemd[1]: Started Git Daemon.
> > Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: fatal: detected dubious ownership in repository at '/srv/git//some/repo.git'
> > Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: To add an exception for this directory, call:
> > Jun 06 08:29:39 localhost.localdomain git-daemon[31756]:         git config --global --add safe.directory /srv/git//some/repo.git
> >
> > git config --global --add safe.directory /srv/git//some/repo.git
> > mv ~/.gitconfig /etc/gitconfig
> > git ping git://localhost/some/repo.git
> > fatal: Could not read from remote repository.
> >
> > Please make sure you have the correct access rights
> > and the repository exists.
> >
> > git config --global --add safe.directory /srv/git/some/repo.git
> > mv ~/.gitconfig /etc/gitconfig
> > git ping git://localhost/some/repo.git
> > fatal: Could not read from remote repository.
> >
> > Please make sure you have the correct access rights
> > and the repository exists.

Phillip Wood· Jun 26, 2024, 13:03 UTC · re: Michal Suchánek · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Hi Michal

On 25/06/2024 19:34, Michal Suchánek wrote:

Show 8 quoted lines

>
> Adding the repository to the list of safe repositories is a known
> concept that was already required for gitweb and for working with the
> repository locally, and applying it to git-daemon as well is consistent
> although it does require configuration changes for some users.
> 
> The real problem here is that adding the repository to the list of safe
> repositories does not make it possible to serve it by git-daemon.

That is indeed unexpected. I set up git-daemon on my laptop this morning and in order to get it to work one has to add "." as well as the repository paths one wants to serve to the list of safe directories. Clearly that is undesirable and does not really feel any safer that using "safe.directory=*". What is happening is that git-daemon checks that the repository path is listed as safe and then changes into that directory and forks

	git upload-pack --strict .

"git upload-pack" then checks "." against the list of safe directories which fails. It fails because the safe directory check does not do any normalization such as cleaning up "//" elements (as seen in your example) or expanding relative paths on $git_dir before checking it against the list of safe directories.

I think the fix is probably to make the safe directory check use the absolute path of $git_dir. In the mean time there is a workaround if you're happy to add "." to the list of safe directories.

Best Wishes

Phillip

Show 117 quoted lines

> Thanks
> 
> Michal
> 
> 
>>
>> Michal Suchánek <msuchanek@suse.de> writes:
>>
>>> On Mon, Jun 17, 2024 at 11:15:13PM +0200, Michal Suchánek wrote:
>>>> Hello,
>>>>
>>>> On Mon, Jun 17, 2024 at 11:47:20AM -0700, Junio C Hamano wrote:
>>>>> "David C. Rankin" <drankinatty@gmail.com> writes:
>>>>>
>>>>>>    Security enhancement in 2.45.1 have broken ability to serve git over
>>>>>>    https and ssh from local git server running Apache. (web server runs
>>>>>>    as http:http on Archlinux)
>>>>>>
>>>>>>    The fix of adding the following to gitconfig (system-wide and
>>>>>>    per-user in ~/.gitconfig) does not solve the problem:
>>>>>>
>>>>>> [safe]
>>>>>> 	directory = *
>>>>>
>>>>> It is not clear what you exactly meant "per-user" above, so just to
>>>>> make sure.  Is this set in the global configuration file for the
>>>>> httpd (or whoever Apache runs as) user?
>>>>>
>>>>> The purpose of "dubious ownershop" thing is to protect the user who
>>>>> runs Git from random repositories' with potentially malicious hooks
>>>>> and configuration files, so the user being protected (in this case,
>>>>> whoever Apache runs as) needs to declare "I trust these
>>>>> repositories" in its ~/.gitconfig file.  What individual owners of
>>>>> /srv/my-repo.git/ project has in their ~/.gitconfig file does not
>>>>> matter when deciding if Apache trusts these repositories.
>>>>
>>>>
>>>> looks like the semantic of 'dubious ownershop' changed recently.
>>>>
>>>> Disro backport of fixes for CVE-2024-32002 CVE-2024-32004 CVE-2024-32020
>>>> CVE-2024-32021 CVE-2024-32465 to 2.35.3 broke git-daemon. No amount of
>>>> whitelisting makes the 'fixed' git serve the repository.
>>>
>>> Same regression between 2.45.0 and 2.45.2 which allegedly fixes the
>>> same CVEs.
>>>
>>> Looks like downgrading to gaping hole version is needed to serve repositories
>>> in general.
>>>
>>> Please consider adjusting the fix so that repositories can still be served.
>>>
>>> Thanks
>>>
>>> Michal
>>>
>>> To reproduce:
>>>
>>> cat /usr/local/bin/git-ping
>>> #!/bin/sh -e
>>>
>>> # Try connecting to one or more remote repository URLs
>>>
>>> while true ; do
>>>          git ls-remote -h "$1" >/dev/null
>>>          shift
>>>          [ -n "$1" ] || break
>>> done
>>>
>>> mkdir -p /srv/git/some
>>> chown hramrach /srv/git/some
>>> su hramrach -c "git init --bare /srv/git/some/repo.git"
>>> su hramrach -c "touch /srv/git/some/repo.git/git-daemon-export-ok"
>>> version=2.35.3-150300.10.36.1 ; zypper in --oldpackage git-core-$version git-daemon-$version
>>> systemctl start git-daemon.service
>>> git ping git://localhost/some/repo.git
>>> <nothing>
>>>
>>> version=2.35.3-150300.10.39.1 ; zypper in --oldpackage git-core-$version git-daemon-$version
>>> systemctl restart git-daemon.service
>>> git ping git://localhost/some/repo.git
>>> fatal: Could not read from remote repository.
>>>
>>> Please make sure you have the correct access rights
>>> and the repository exists.
>>>
>>>
>>> systemctl status git-daemon.service
>>> ● git-daemon.service - Git Daemon
>>>       Loaded: loaded (/usr/lib/systemd/system/git-daemon.service; disabled; vendor preset: disabled)
>>>       Active: active (running) since Thu 2024-06-06 08:29:28 CEST; 6min ago
>>>     Main PID: 31742 (git)
>>>        Tasks: 2 (limit: 4915)
>>>       CGroup: /system.slice/git-daemon.service
>>>               ├─ 31742 git daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
>>>               └─ 31749 /usr/lib/git/git-daemon --reuseaddr --base-path=/srv/git/ --user=git-daemon --group=nogroup
>>>
>>> Jun 06 08:29:28 localhost.localdomain systemd[1]: Started Git Daemon.
>>> Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: fatal: detected dubious ownership in repository at '/srv/git//some/repo.git'
>>> Jun 06 08:29:39 localhost.localdomain git-daemon[31756]: To add an exception for this directory, call:
>>> Jun 06 08:29:39 localhost.localdomain git-daemon[31756]:         git config --global --add safe.directory /srv/git//some/repo.git
>>>
>>> git config --global --add safe.directory /srv/git//some/repo.git
>>> mv ~/.gitconfig /etc/gitconfig
>>> git ping git://localhost/some/repo.git
>>> fatal: Could not read from remote repository.
>>>
>>> Please make sure you have the correct access rights
>>> and the repository exists.
>>>
>>> git config --global --add safe.directory /srv/git/some/repo.git
>>> mv ~/.gitconfig /etc/gitconfig
>>> git ping git://localhost/some/repo.git
>>> fatal: Could not read from remote repository.
>>>
>>> Please make sure you have the correct access rights
>>> and the repository exists.
>

Junio C Hamano· Jun 26, 2024, 18:14 UTC · re: Phillip Wood · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Phillip Wood <phillip.wood123@gmail.com> writes:

Show 15 quoted lines

> ... What is happening is that
> git-daemon checks that the repository path is listed as safe and then
> changes into that directory and forks
>
> 	git upload-pack --strict .
>
> "git upload-pack" then checks "." against the list of safe directories
> which fails. It fails because the safe directory check does not do any
> normalization such as cleaning up "//" elements (as seen in your
> example) or expanding relative paths on $git_dir before checking it
> against the list of safe directories.
> ...
> I think the fix is probably to make the safe directory check use the
> absolute path of $git_dir. In the mean time there is a workaround if
> you're happy to add "." to the list of safe directories.

It still is curious why unnormalized "." does not pass "*" (which is not even a pattern matching, but is a declaration that says "don't bother which path we are talking about"), though. As long as the value of that configuration is found to be '*' literally, safe directory data is marked as "is_safe" (cf. setup.c:safe_directory_cb and setup.c:ensure_valid_ownership; notice that data.path is not even consulted if the value of the configuration variable is '*').

Anyway, thanks for digging.

Phillip Wood· Jun 26, 2024, 18:35 UTC · re: Junio C Hamano · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

On 26/06/2024 19:14, Junio C Hamano wrote:

Show 19 quoted lines

> Phillip Wood <phillip.wood123@gmail.com> writes:
> 
>> ... What is happening is that
>> git-daemon checks that the repository path is listed as safe and then
>> changes into that directory and forks
>>
>> 	git upload-pack --strict .
>>
>> "git upload-pack" then checks "." against the list of safe directories
>> which fails. It fails because the safe directory check does not do any
>> normalization such as cleaning up "//" elements (as seen in your
>> example) or expanding relative paths on $git_dir before checking it
>> against the list of safe directories.
>> ...
>> I think the fix is probably to make the safe directory check use the
>> absolute path of $git_dir. In the mean time there is a workaround if
>> you're happy to add "." to the list of safe directories.
> 
> It still is curious why unnormalized "." does not pass "*"

Sorry if I wasn't clear. "." is considered safe with "safe.directory = *" but I was looking at why it was not considered safe when using repository paths in safe.directory.

Best Wishes

Phillip

Show 10 quoted lines

> (which is
> not even a pattern matching, but is a declaration that says "don't
> bother which path we are talking about"), though.  As long as the
> value of that configuration is found to be '*' literally, safe
> directory data is marked as "is_safe" (cf. setup.c:safe_directory_cb
> and setup.c:ensure_valid_ownership; notice that data.path is not
> even consulted if the value of the configuration variable is '*').
> 
> Anyway, thanks for digging.
>

Junio C Hamano· Jun 26, 2024, 18:51 UTC · re: Phillip Wood · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Phillip Wood <phillip.wood123@gmail.com> writes:

> Sorry if I wasn't clear. "." is considered safe with "safe.directory =
> *" but I was looking at why it was not considered safe when using
> repository paths in safe.directory.

Ahh, OK. We tell them /home/wood/repo/git is a good repository to visit, but daemon's call to ensure_valid_ownership() asks "I am in the '.' directory; does that match /home/wood/repo/git?"

That sucks. I agree with your idea that we'd need to canonicalize both before comparing.

THanks.

Michal Suchánek· Sep 25, 2024, 11:34 UTC · re: Junio C Hamano · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

On Wed, Jun 26, 2024 at 11:51:46AM -0700, Junio C Hamano wrote:

Show 12 quoted lines

> Phillip Wood <phillip.wood123@gmail.com> writes:
> 
> > Sorry if I wasn't clear. "." is considered safe with "safe.directory =
> > *" but I was looking at why it was not considered safe when using
> > repository paths in safe.directory.
> 
> Ahh, OK.  We tell them /home/wood/repo/git is a good repository to
> visit, but daemon's call to ensure_valid_ownership() asks "I am in
> the '.' directory; does that match /home/wood/repo/git?"  
> 
> That sucks.  I agree with your idea that we'd need to canonicalize
> both before comparing.

And it's addressed with 0ed3dde067a5dc5141e3a31d950972f00a08b35d at least since 2.46.1

Thanks

Michal

Joey Hess· Aug 29, 2024, 20:34 UTC · re: Phillip Wood · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Phillip Wood wrote:

Show 5 quoted lines

> That is indeed unexpected. I set up git-daemon on my laptop this morning and
> in order to get it to work one has to add "."  as well as the repository
> paths one wants to serve to the list of safe directories. Clearly that is
> undesirable and does not really feel any safer that using
> "safe.directory=*".

Having also now run into this new landmine, I don't see how safe.directory adds any security to git-daemon at all.

git-daemon is already told which directory contains the repositories to export, and they have to have git-daemon-export-ok in them as well. So the user has already specified what repositories it is safe for git-daemon to use.

Show 5 quoted lines

> What is happening is that git-daemon checks that the
> repository path is listed as safe and then changes into that directory and
> forks
> 
> 	git upload-pack --strict .

git-daemon might as well run children with safe.directory=* itself.

-- 
see shy jo

Jamie Landeg-Jones· Jul 26, 2024, 00:38 UTC · re: Junio C Hamano · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Junio C Hamano <gitster@pobox.com> wrote:

> Dscho, the f4aa8c8b (fetch/clone: detect dubious ownership of local
> repositories, 2024-04-10) is your brainchild and people seem to be
> unhappy about having to adjust their settings.  Are there any advice
> you can offer them?

This thread has gone quiet. Is it going to be fixed? The issue isn't that settings need to be adjusted, it's that it no longer works whatever your settings are.

I had to go back to.2.45.0

Cheers

David C. Rankin· Jul 26, 2024, 05:58 UTC · re: Jamie Landeg-Jones · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

On 7/25/24 7:38 PM, Jamie Landeg-Jones wrote:

Show 15 quoted lines

> Junio C Hamano <gitster@pobox.com> wrote:
> 
>> Dscho, the f4aa8c8b (fetch/clone: detect dubious ownership of local
>> repositories, 2024-04-10) is your brainchild and people seem to be
>> unhappy about having to adjust their settings.  Are there any advice
>> you can offer them?
> 
> This thread has gone quiet. Is it going to be fixed? The issue isn't that
> settings need to be adjusted, it's that it no longer works whatever
> your settings are.
> 
> I had to go back to.2.45.0
> 
> Cheers
>

I too am stuck. It's a damned if you do, damned if you don't Hobson's choice of git repo ownership on the server. All worked fine serving both https and ssh prior to the change, now it is either one or the other but not both. So either your read-only pulls and clones are hosed, or your read/write are.

If more info is needed, let me know and I'm happy to privide it.

-- 
David C. Rankin, J.D.,P.E.

Jamie Landeg-Jones· Jul 28, 2024, 03:46 UTC · re: David C. Rankin · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

"David C. Rankin" <drankinatty@gmail.com> wrote:

> I too am stuck. It's a damned if you do, damned if you don't Hobson's choice 
> of git repo ownership on the server. All worked fine serving both https and 
> ssh prior to the change, now it is either one or the other but not both. So 
> either your read-only pulls and clones are hosed, or your read/write are.

Thanks for the reply. I guess I'm potentially not as stuck as you, because I don't require ssh access to the repos - they are just mirrors. I have one unix user that updates the repository, and the cgit and git https backends run as another suexec user exclusively for that task. However, I'm not going to compromise security by giving the web user effective write-access to the files it only needs read access to.

> If more info is needed, let me know and I'm happy to privide it.

If you have any solution, I'd be grateful to hear. I'm a bit puzzled to what's going on though.

From what I gather, the idea was to stop a client user from unintentionally running potential hook scripts that could be evil. A similar protection was already present, I note, for other methods (I couldn't run git log and other commands directly on a repo i had unix-level read access to, but were owned by another user until I added the safe directory thing in .gitconfig)

Closing this hole to make security more consistent is fair enough... however, in the case of git-http-backend the "safe directory" method doesn't work at all!

Is this an accurate summary of the situation?

As an aside, do you know why this security mechanism blocks ALL access, and not just the execution of hooks? - my mirrors don't have any hook-scripts, and as long as I give them read access at the unix level, why are they still blocked from doing things like "git log" on a repo without having to first mark it as safe?

Cheers, Jamie

David C. Rankin· Jul 28, 2024, 06:57 UTC · re: Jamie Landeg-Jones · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

On 7/27/24 10:46 PM, Jamie Landeg-Jones wrote:

Show 11 quoted lines

>  From what I gather, the idea was to stop a client user from unintentionally
> running potential hook scripts that could be evil. A similar protection was
> already present, I note, for other methods (I couldn't run git log and other
> commands directly on a repo i had unix-level read access to, but were owned
> by another user until I added the safe directory thing in .gitconfig)
> 
> Closing this hole to make security more consistent is fair enough... however,
> in the case of git-http-backend the "safe directory" method doesn't work at
> all!
> 
> Is this an accurate summary of the situation?

Yes it is.

   Prior to the change, the repositories on my git server were owned by the 
user with push privileges and all repositories would still be cloned over 
https. Now unless the files are owned by http (what the web server runs as on 
Archlinux), https users cannot update their repositories anymore. Changes 
pushed to the server by the user with ssh access (and owner of the files) 
cannot be pulled over https.

   On the other side of the issue, if I make the webserver the owner of the 
repos, the ssh user cannot push or pull.

   Gitweb remains happy no matter the ownership.

   I'm all for more security and welcome it, but you have to provide a way to 
configure it so that prior functionality is simply not broken. I'll have to 
search and find the little truth-table style of repo user:group ownership and 
the resulting succeed:fail with push/pull from the server.

   I'd be happy if the docs were just updated with a concise explanation of 
how to support both ssh and https to the same repo running under Apache -- if 
that is doable. It must be, github does it. I haven't had the time I'd like to 
to further investigate. Next month looks better.

-- 
David C. Rankin, J.D.,P.E.

Jamie Landeg-Jones· Aug 1, 2024, 00:15 UTC · re: David C. Rankin · lore

[SOLVED] Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

"David C. Rankin" <drankinatty@gmail.com> wrote:

>    I'd be happy if the docs were just updated with a concise explanation of 
> how to support both ssh and https to the same repo running under Apache -- if 
> that is doable.

Hi. I've done some investigating, and have discovered what the problem is (with my setup, at least)

It only tries to access /etc/gitconfig (or /usr/local/etc/gitconfig) - putting the rules in there allow it to work, but with the caveat mentioned by an earlier poster - you need to add a path of "." as well as the required directories. (or enable everything with a path of "*")

Now, I don't want to do that globally: "*" it's too risky, and I haven't delved deep enough to determine if "." is a security risk too.

However, there is a better solution:

The problem is, the environment variable HOME isn't defined when the http server calls cgi-http-backend.

I actually have an intermediate script that is called by httpd, that itself execs git-http-backend, so all I needed to do was add the line

export HOME=/.../git-web-user/

before the exec, and it now works as expected, picking up .gitconfig from the .../git-web-user/ directory.

If you call git-http-backend directly, you may be able to set the environment variable in the httpd config. If not, just create an intermediate script, something like

#!/bin/sh export HOME=... exec <path-to-git-http-backend>

, and change http to call that.

Now, I don't think that if not set, git should determine HOME itself (from the password file etc.), as someone may want that disabled for other reasons, HOWEVER, this should really be documented.

So, in my mind, the fix is to fix the "." directory requirement, and document the potential need to set HOME.

Ler me know if this works for you!

Cheers, Jamie

Junio C Hamano· Aug 2, 2024, 19:31 UTC · re: Jamie Landeg-Jones · lore

Re: [SOLVED] Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

Jamie Landeg-Jones <jamie@catflap.org> writes:

Show 7 quoted lines

> I actually have an intermediate script that is called by httpd, that itself
> execs git-http-backend, so all I needed to do was add the line
>
> export HOME=/.../git-web-user/
>
> before the exec, and it now works as expected, picking up .gitconfig from the
> .../git-web-user/ directory.

Good.

> So, in my mind, the fix is to fix the "." directory requirement, and document
> the potential need to set HOME.

Good, too.

Thanks for a success report.

David C. Rankin· Jun 18, 2024, 01:08 UTC · re: Junio C Hamano · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

On 6/17/24 13:47, Junio C Hamano wrote:

Show 25 quoted lines

> "David C. Rankin" <drankinatty@gmail.com> writes:
> 
>>    Security enhancement in 2.45.1 have broken ability to serve git over
>>    https and ssh from local git server running Apache. (web server runs
>>    as http:http on Archlinux)
>>
>>    The fix of adding the following to gitconfig (system-wide and
>>    per-user in ~/.gitconfig) does not solve the problem:
>>
>> [safe]
>> 	directory = *
> 
> It is not clear what you exactly meant "per-user" above, so just to
> make sure.  Is this set in the global configuration file for the
> httpd (or whoever Apache runs as) user?
> 
> The purpose of "dubious ownershop" thing is to protect the user who
> runs Git from random repositories' with potentially malicious hooks
> and configuration files, so the user being protected (in this case,
> whoever Apache runs as) needs to declare "I trust these
> repositories" in its ~/.gitconfig file.  What individual owners of
> /srv/my-repo.git/ project has in their ~/.gitconfig file does not
> matter when deciding if Apache trusts these repositories.
> 
>

   I apologize for being unclear. The directory on the server (valkyrie) that 
hosts the repositories has the current ownerships:

19:53 valkyrie:/srv/git> l total 60 drwxr-xr-x 15 http http 4096 Feb 13 02:37 . drwxr-xr-x 6 root root 4096 Jan 4 2017 .. drwxr-xr-x 7 http david 4096 Feb 1 06:54 arm.git drwxr-xr-x 7 david david 4096 Nov 13 2023 c_datastructs.git drwxr-xr-x 7 david david 4096 Jun 15 02:42 embedded.git drwxr-xr-x 7 http http 4096 Jan 26 01:34 examples.git drwxr-xr-x 7 http david 4096 Aug 13 2023 gtkedit.git drwxr-xr-x 7 http david 4096 Aug 13 2023 gtkeditphmirror.git drwxr-xr-x 7 http david 4096 Aug 13 2023 gtkwritemirror.git drwxr-xr-x 7 http david 4096 Aug 13 2023 mtrxsolv.git drwxr-xr-x 7 http devs 4096 Aug 13 2023 notesdlg.git drwxr-xr-x 7 http david 4096 Aug 13 2023 objstate.git drwxr-xr-x 7 http devs 4096 Nov 13 2023 pgauth.git drwxr-xr-x 7 david david 4096 Jun 16 19:14 pico.git drwxr-xr-x 7 http david 4096 Aug 13 2023 vect_t.git

   Prior to this dubious ownership issue, everything was owned by david:david, 
other than notesdg.git and pgauth.git which were david:devs with other 
developers made members of the devs group.

   After 2.45.1 update, those with read-only access over https could not pull 
updates anymore. I didn't notice right away because I pull everything over ssh.

   I do have other machines where I pull https copies of repos, and that too 
broke after 2.45.1.

   I read the error from the server log and then attempted to add the 
suggested config to ~/.gitconfig, e.g.:

[safe]
	directory = /srv/git/embedded.git

   That failed. I then scoured the internet and many posts said use * for all 
instead of individual directory names, e.g.

[safe]
	directory = *

   Same error.

   I then changed ownership over everything in /srv/git to http:http and https 
started working again! But ... when I tried to push over ssh it failed, e.g.

19:14 wizard:~/dev/arm/pico> git push
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 1.01 KiB | 1.01 MiB/s, done.
Total 5 (delta 2), reused 0 (delta 0), pack-reused 0
remote: error: cannot lock ref 'HEAD': Unable to create 
'/srv/git/pico.git/./HEAD.lock': Permission denied
To valkyrie:/srv/git/pico.git
  ! [remote rejected] master -> master (failed to update ref)
error: failed to push some refs to 'valkyrie:/srv/git/pico.git'

   I have tried every combination of ownership, e.g. david:david, david:http, 
http:david and the bottom line is:

   - https won't work unless http owns the repository (http:dontcare), and
   - ssh won't work unless the user owns the repository (david:dontcare)

   Somehow I need a configuration that allows both ssh and https to work 
together. My apache config is setup per the scm-git.com book adapted to use my 
server auth:

## git directory for https access

SetEnv GIT_PROJECT_ROOT /srv/git ScriptAlias /git/ /usr/lib/git-core/git-http-backend/ ScriptAlias /git /usr/lib/git-core/git-http-backend/

<Directory /usr/lib/git-core>
   Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
   AllowOverride None
   Require all granted
</Directory>

<Files "git-http-backend">
   # authentication
   AuthType Basic
   AuthName "Skyline_Restricted"
   AuthBasicProvider dbm
   AuthDBMType DB
   AuthDBMUserFile /usr/local/lib/apache2/dcraccess
   Require expr !(%{QUERY_STRING} -strmatch '*service=git-receive-pack*' || 
%{REQUEST_URI} =~ m#/git-receive-pack$#)
   Require valid-user
</Files>

   This has always worked great. It's just the side effect of the latest 
dubious ownership security changes that have killed it.

   Hopefully this is just a temporary regression in the CVE fixes? Let me know 
if you need any more information. I'm happy to provide it.

-- 
David C. Rankin, J.D.,P.E.

Junio C Hamano· Jun 24, 2024, 14:53 UTC · re: David C. Rankin · lore

Re: Local git server can't serve https until repos owned by http, can't serve ssh unless repos owned by user after 2.45.1

"David C. Rankin" <drankinatty@gmail.com> writes:

Show 7 quoted lines

>   I read the error from the server log and then attempted to add the
>   suggested config to ~/.gitconfig, e.g.:
>
> [safe]
> 	directory = /srv/git/embedded.git
> ...
> drwxr-xr-x  7 david david 4096 Jun 15 02:42 embedded.git

You say ~/.gitconfig here.  Whose home directory does "~" refer to?

The primary reason I wrote the message you are responding to was to find this piece of information out (and tell you that it would be different from the repository owner's), and I am still not sure if I extracted that piece of information out of you, after reading the message I am responding to twice.

In short, the above has to be in ~http/.gitconfig (if 'http' is the user Apache runs as). Listing it in ~david/.gitconfig would not work. It has to come from the user who runs Git in a directory the user does not own (in this case, that is the 'http' user. That user runs, via the CGI interface, the server side Git programs to respond to pushes and fetches, serving a repository not owned by 'http'),

The mechanism is about protecting the user that runs Git (i.e. 'http' in the context of "git push/fetch is responded by the Apache web server via the CGI mechanism spawning 'git http-backend'") from potentially malicious repositories owned by users that are *not* 'http' (i.e. /srv/git/embedded.git that is owned by david:david in this case), as such repositories can contain hooks and other configuration settings that would end up running programs as 'http' user with 'http' user's priviledge, potentially writing into files owned by 'http' user to attack 'http' user.

Hence, the "I am OK to run Git as myself in these repositories, even though they are not owned by me" acceptance, which is an snippet in gitconfig file you showed above, has to come from the 'http' user, and certainly not from the 'david' user. The configuration is about protecting 'http' from 'david' and not about protecting 'david' from 'http', so configuration files in 'david' user's home directory should have no say in who 'http' user is willing to trust.

So, back to the original question. You said ~/.gitconfig/ has the safe.directory settings. Whose ~/.gitconfig/?

I would have expected that the only thing you'd need to do is to let your 'http' user tell Git that it does not want the paranoid safety more recent versions of Git offers with the safe.directory setting in ~http/.gitconfig, and have repositories owned by individual users that come over ssh as they themselves. For those who are accessing their own repositories over ssh as themselves, everything should work just as before.