threads / bug / 57043

BUG: git bash / python interaction with absolute paths as environment variables in Windows

Subject: BUG: git bash / python interaction with absolute paths as environment variables in Windows

## tl;dr

4 messages between Dec 7, 2021 and Dec 7, 2021.

replies: 3people: 3as markdown or json

Leland Weathers· Dec 7, 2021, 03:00 UTC · lore
Issue: Using Git Bash for Windows (2.34.1-64) and Python 3.9.9, a git
path is incorrectly prepended to environment variables in Python code.

Sample Output of a simple Python script pulling an environment variable - this is the expected output, where Python is given the correct environment variables for both absolute directory paths and non-directory paths - OS should not matter:

(gitfu-XXGUYdp7) C:\Users\Leland\code\gitfu>type foobar.py from os import environ from sys import stdout

stdout.write(f'environ: {environ["TEST_DIR_BROKEN"]}\n') stdout.write(f'environ: {environ["TEST_DIR_WORKING"]}\n') (gitfu-XXGUYdp7) C:\Users\Leland\code\gitfu>echo %TEST_DIR_BROKEN% /foo/bar

(gitfu-XXGUYdp7) C:\Users\Leland\code\gitfu>echo %TEST_DIR_WORKING% foo/bar

(gitfu-XXGUYdp7) C:\Users\Leland\code\gitfu>python foobar.py environ: /foo/bar environ: foo/bar

(gitfu-XXGUYdp7) C:\Users\Leland\code\gitfu>powershell Get-Command python.exe

CommandType Name Version Source ----------- ---- ------- ------ Application python.exe 3.9.915... C:\Users\Leland\.virtualenvs\gitfu-XXGUYdp7\Scripts\python.exe

(gitfu-XXGUYdp7) C:\Users\Leland\code\gitfu>

Sample output when the same set of commands is run in git bash - absolute path has been altered the relative path has not been:

$ cat foobar.py from os import environ from sys import stdout

stdout.write(f'environ: {environ["TEST_DIR_BROKEN"]}\n') stdout.write(f'environ: {environ["TEST_DIR_WORKING"]}\n') Leland@local MINGW64 ~/code/gitfu $ echo $TEST_DIR_BROKEN /foo/bar

Leland@local MINGW64 ~/code/gitfu $ echo $TEST_DIR_WORKING foo/bar

Leland@local MINGW64 ~/code/gitfu $ python foobar.py environ: C:/Users/Leland/AppData/Local/Programs/Git/foo/bar environ: foo/bar

Leland@local MINGW64 ~/code/gitfu $ which python /c/Users/Leland/.virtualenvs/gitfu-XXGUYdp7/Scripts/python

Leland@local MINGW64 ~/code/gitfu $

Is there anything else I'm missing on why the same Python script would read environment variables differently than what is read from Git Bash itself or why the exact same Python code reads the environment variable correctly when run from a command prompt and not in Git Bash?

In both cases I am using the same Python virtual environment. Other environment variables (e.g. non-absolute directory paths) appear to be read correctly. I'm assuming that this is a git issue given the

Eric Sunshine· Dec 7, 2021, 03:15 UTC · re: Leland Weathers · lore

Re: BUG: git bash / python interaction with absolute paths as environment variables in Windows

On Mon, Dec 6, 2021 at 10:01 PM Leland Weathers <leland@lcweathers.net> wrote:
Show 19 quoted lines
> Issue: Using Git Bash for Windows (2.34.1-64) and Python 3.9.9, a git
> path is incorrectly prepended to environment variables in Python code.
>
> $ echo $TEST_DIR_BROKEN
> /foo/bar
> $ echo $TEST_DIR_WORKING
> foo/bar
> $ python foobar.py
> environ: C:/Users/Leland/AppData/Local/Programs/Git/foo/bar
> environ: foo/bar
>
> Is there anything else I'm missing on why the same Python script would
> read environment variables differently than what is read from Git Bash
> itself or why the exact same Python code reads the environment
> variable correctly when run from a command prompt and not in Git Bash?
>
> In both cases I am using the same Python virtual environment. Other
> environment variables (e.g. non-absolute directory paths) appear to be
> read correctly. I'm assuming that this is a git issue given the

This is probably not specific to Git, but rather a "feature" of MSYS2, which Git for Windows happens to employ for its Bash shell. When invoking Windows commands from within MSYS2, command-line arguments and environment variables on the Unix side which appear to be paths will be converted to Windows paths for the sake of the native Windows program (since it won't know anything about the Unix paths coming out of the MSYS2 environment).

This behavior is documented at [1]; in particular, see the "Environment Variables" section.

[1]: https://www.msys2.org/docs/filesystem-paths/
Leland Weathers· Dec 7, 2021, 13:10 UTC · re: Eric Sunshine · lore

Re: BUG: git bash / python interaction with absolute paths as environment variables in Windows

Eric, thanks for the pointer. Got it working as expected now.
On Mon, Dec 6, 2021 at 9:16 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 34 quoted lines
>
> On Mon, Dec 6, 2021 at 10:01 PM Leland Weathers <leland@lcweathers.net> wrote:
> > Issue: Using Git Bash for Windows (2.34.1-64) and Python 3.9.9, a git
> > path is incorrectly prepended to environment variables in Python code.
> >
> > $ echo $TEST_DIR_BROKEN
> > /foo/bar
> > $ echo $TEST_DIR_WORKING
> > foo/bar
> > $ python foobar.py
> > environ: C:/Users/Leland/AppData/Local/Programs/Git/foo/bar
> > environ: foo/bar
> >
> > Is there anything else I'm missing on why the same Python script would
> > read environment variables differently than what is read from Git Bash
> > itself or why the exact same Python code reads the environment
> > variable correctly when run from a command prompt and not in Git Bash?
> >
> > In both cases I am using the same Python virtual environment. Other
> > environment variables (e.g. non-absolute directory paths) appear to be
> > read correctly. I'm assuming that this is a git issue given the
>
> This is probably not specific to Git, but rather a "feature" of MSYS2,
> which Git for Windows happens to employ for its Bash shell. When
> invoking Windows commands from within MSYS2, command-line arguments
> and environment variables on the Unix side which appear to be paths
> will be converted to Windows paths for the sake of the native Windows
> program (since it won't know anything about the Unix paths coming out
> of the MSYS2 environment).
>
> This behavior is documented at [1]; in particular, see the
> "Environment Variables" section.
>
> [1]: https://www.msys2.org/docs/filesystem-paths/
Johannes Schindelin· Dec 7, 2021, 21:39 UTC · re: Leland Weathers · lore

Re: BUG: git bash / python interaction with absolute paths as environment variables in Windows

Hi Leland,
On Mon, 6 Dec 2021, Leland Weathers wrote:
> Issue: Using Git Bash for Windows (2.34.1-64) and Python 3.9.9, a git
> path is incorrectly prepended to environment variables in Python code.

From Git for Windows' [Release Notes' "Known Issues" section](https://github.com/git-for-windows/build-extra/blob/master/ReleaseNotes.md#known-issues):

	* If you specify command-line options starting with a slash,
	  POSIX-to-Windows path conversion will kick in converting e.g.
	  "`/usr/bin/bash.exe`" to "`C:\Program Files\Git\usr\bin\bash.exe`".
	 When that is not desired -- e.g.
	 "`--upload-pack=/opt/git/bin/git-upload-pack`" or "`-L/regex/`" --
	 you need to set the environment variable `MSYS_NO_PATHCONV`
	 temporarily, like so:
	 > `MSYS_NO_PATHCONV=1 git blame -L/pathconv/ msys2_path_conv.cc`
	 Alternatively, you can double the first slash to avoid
	 POSIX-to-Windows path conversion, e.g. "`//usr/bin/bash.exe`".

Those release notes are shown by default when you install Git for Windows, and they are also accessible via the Start Menu as "Git Release Notes".

Show 5 quoted lines
> [... snip ...]
>
> In both cases I am using the same Python virtual environment. Other
> environment variables (e.g. non-absolute directory paths) appear to be
> read correctly. I'm assuming that this is a git issue given the
Premature end-of-mail?

Ciao, Johannes

← back to recent threads