threads / bug / 61308

[BUG] Clones from local repositories do not work correctly under Windows

Subject: [BUG] Clones from local repositories do not work correctly under Windows

## tl;dr

9 messages between Apr 11, 2024 and Apr 12, 2024.

replies: 8people: 4as markdown or json

L. E. Segovia· Apr 11, 2024, 17:03 UTC · lore

Thank you for filling out a Git bug report! Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

Under Windows 10 21H2, I cloned a Git repository off an existing local copy, then applied a patch to it using `git am`.

What did you expect to happen? (Expected behavior)

Operations on the repo like `git status` and `git log` should work out of the box.

What happened instead? (Actual behavior)
I keep getting this error:
> error: unable to normalize alternate object path:
/c/Users/Amalia/.cache/cerbero-sources/librsvg-2.40.20/.git/objects

With `git status` it happens only once, but with `git log` it's a wall worth of that error before the log is actually generated and paged.

This error makes Git complain, but tools depending on it (in particular Cargo) outright crash. For instance, this is the error I got when attempting to build the Rust crate above:

> Error: CliError { error: Some(failed to determine package fingerprint

for build script for librsvg v2.58.0-beta.1 (E:\cerbero\build\sources\msvc_x86_64\librsvg-2.40.20\rsvg)

>
> Caused by:
>     0: failed to determine the most recently modified file in
E:\cerbero\build\sources\msvc_x86_64\librsvg-2.40.20\rsvg
>     1: failed to determine list of files in
E:\cerbero\build\sources\msvc_x86_64\librsvg-2.40.20\rsvg
>     2: object not found - no match for id

(32467a0191907fd571b502e395d033a06dfee655); class=Odb (9); code=NotFound (-3)), exit_code: 101 }

What's different between what you expected and what actually happened?

It seems that under Windows, the code in https://github.com/git/git/blame/436d4e5b14df49870a897f64fe92c0ddc7017e4c/object-file.c#L525 is not able to resolve the absolute path of the "remote" (the original copy) at all. I checked if it was a filesystem traversal issue by issuing the clone against a folder in the same drive, but there was no change.

Anything else you want to add:

Possibly introduced in 2.39.0: https://github.com/git/git/commit/199337d6ec5c656e52b914b5dac3820cc5e363f3

Please review the rest of the bug report below. You can delete any lines you don't wish to share.

[System Info] git version: git version 2.44.0.windows.1 cpu: x86_64 built from commit: ad0bbfffa543db6979717be96df630d3e5741331 sizeof-long: 4 sizeof-size_t: 8 shell-path: /bin/sh feature: fsmonitor--daemon uname: Windows 10.0 19045 compiler info: gnuc: 13.2 libc info: no libc information available $SHELL (typically, interactive shell): <unset>

[Enabled Hooks]
-- 
amyspark 🌸 https://www.amyspark.me
Johannes Sixt· Apr 12, 2024, 06:56 UTC · re: L. E. Segovia · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Am 11.04.24 um 19:03 schrieb L. E. Segovia:
Show 7 quoted lines
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> 
> What did you do before the bug happened? (Steps to reproduce your issue)
> 
> Under Windows 10 21H2, I cloned a Git repository off an existing local
> copy, then applied a patch to it using `git am`.

This is an imprecise description of what you did. I did my best to follow these instructions, but could not reproduce the problem mentioned below.

You should post the commands you have given with their output.
Show 12 quoted lines
> 
> What did you expect to happen? (Expected behavior)
> 
> Operations on the repo like `git status` and `git log` should work out
> of the box.
> 
> What happened instead? (Actual behavior)
> 
> I keep getting this error:
> 
>> error: unable to normalize alternate object path:
> /c/Users/Amalia/.cache/cerbero-sources/librsvg-2.40.20/.git/objects
What is this .cache here?
-- Hannes
L. E. Segovia· Apr 12, 2024, 13:05 UTC · re: Johannes Sixt · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Hi Johannes,

Under a MSYS2 shell, follow these steps (feel free to replace the choice of remote repo with any other supporting a Rust crate):

1. `cd` to a directory of your choice (let's call it `dirA`).
2. `git clone https://gitlab.gnome.org/GNOME/librsvg.git .`
3. `git checkout -b cerbero_build`.
4. `cd ..`
5. `git clone <full path to dirA> -s -b cerbero_build dirB`
6. `cd dirB`
7. `git status`

Now switch to any other shell of your choice, such as Git Bash, CMD, or PowerShell, and try issuing commands such as `git status`. It will result in a wall of "error: unable to normalize..."

To further the error state, try `git am` the attached diff to the repo under the MSYS2 shell. It will succeed, but afterwards you'll also start to get fatal errors from `git status`:

> error: unable to normalize alternate object path: <dirA>
> error: Could not read 819d81aefc255dee9d8845363038f7fdec809673
> fatal: Failed to traverse parents of commit
90042c71ef79cebeba782c98b26f0532de0e6a25

Attempting to build the repo using the well-known pair `meson setup build; meson compile -C build` (assuming you have all the dependencies around) will, at this step, crash at the very start with the following Rust stacktrace:

Show 6 quoted lines
> Error: CliError { error: Some(failed to determine package fingerprint for build script for librsvg v2.58.0-beta.1 (<dirB>/rsvg)
> 
> Caused by:
>     0: failed to determine the most recently modified file in <dirB>\rsvg
>     1: failed to determine list of files in <dirB>\rsvg
>     2: object not found - no match for id (32467a0191907fd571b502e395d033a06dfee655); class=Odb (9); code=NotFound (-3)), exit_code: 101 }

It should be noted, however, than exchanging the shells (cloning in Git Bash, but then checking and patching under MSYS) appears to not cause any errors. That leads me to believe there's some part of Git that is not handling line separators under MSYS specifically.

> What is this .cache here?

This is part of the Cerbero package management system, used by GStreamer to craft its SDK:

https://gitlab.freedesktop.org/gstreamer/cerbero.git
The above mentioned steps were reverse engineered from its caching steps.
Hope this is of use.
Best,
amyspark
On 12/04/2024 03:56, Johannes Sixt wrote:
Show 32 quoted lines
> Am 11.04.24 um 19:03 schrieb L. E. Segovia:
>> Thank you for filling out a Git bug report!
>> Please answer the following questions to help us understand your issue.
>>
>> What did you do before the bug happened? (Steps to reproduce your issue)
>>
>> Under Windows 10 21H2, I cloned a Git repository off an existing local
>> copy, then applied a patch to it using `git am`.
> 
> This is an imprecise description of what you did. I did my best to
> follow these instructions, but could not reproduce the problem mentioned
> below.
> 
> You should post the commands you have given with their output.
> 
>>
>> What did you expect to happen? (Expected behavior)
>>
>> Operations on the repo like `git status` and `git log` should work out
>> of the box.
>>
>> What happened instead? (Actual behavior)
>>
>> I keep getting this error:
>>
>>> error: unable to normalize alternate object path:
>> /c/Users/Amalia/.cache/cerbero-sources/librsvg-2.40.20/.git/objects
> 
> What is this .cache here?
> 
> -- Hannes
> 
-- 
amyspark 🌸 https://www.amyspark.me


From 6519af2fbab17cd4b1861e6b9cbfbb2c76906edf Mon Sep 17 00:00:00 2001
From: "L. E. Segovia" <amy@amyspark.me>
Date: Fri, 12 Apr 2024 09:43:51 -0300
Subject: [PATCH] test

---
 AUTHORS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/AUTHORS b/AUTHORS
index 9971ce58..7b2a9266 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -68,6 +68,7 @@ Kurosawa Takeshi
 liucougar
 LRN
 Laszlo Peter
+L. E. Segovia
 Luis Menina
 Maarten Bosmans
 Maciej Piechotka
-- 
2.44.0.windows.1
Johannes Sixt· Apr 12, 2024, 16:16 UTC · re: L. E. Segovia · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Am 12.04.24 um 15:05 schrieb L. E. Segovia:
Show 14 quoted lines
> Under a MSYS2 shell, follow these steps (feel free to replace the choice
> of remote repo with any other supporting a Rust crate):
> 
> 1. `cd` to a directory of your choice (let's call it `dirA`).
> 2. `git clone https://gitlab.gnome.org/GNOME/librsvg.git .`
> 3. `git checkout -b cerbero_build`.
> 4. `cd ..`
> 5. `git clone <full path to dirA> -s -b cerbero_build dirB`
> 6. `cd dirB`
> 7. `git status`
> 
> Now switch to any other shell of your choice, such as Git Bash, CMD, or
> PowerShell, and try issuing commands such as `git status`. It will
> result in a wall of "error: unable to normalize..."

Thanks, that is much more helpful. Here are my assumption, my explanation of the symptoms, and the conclusion (which is basically "if it hurts, don't do it"):

I assume that you have two installations of Git. One installation is used when you use the git command from the MSYS2 shell; it is the MSYS2 version. The other installation is Git for Windows or equivalent that is used by all the other shells.

When you clone a repository with -s, a .git/objects/info/alternates file is initialized that contains the path to the original repository. You can see its contents with `git count-objects -v`. I tested the reproduction recipe with my own build (which is a MinGW build, much like Git for Windows), and it looks like this:

C:\Temp\repoclone>git count-objects -v count: 0 size: 0 in-pack: 0 packs: 0 size-pack: 0 prune-packable: 0 garbage: 0 size-garbage: 0 alternate: C:/Temp/repoorig/.git/objects

See the last line? It begins with a drive letter. When you call the command with the MSYS2 version, it will look more like this:

alternate: /c/Temp/repoorig/.git/objects

This is the MSYS2/Cygwin absolute path with a "drive letter". Unfortunately, this kind of path is unintelligible for Git for Windows. It expects absolute paths to begin with drive letter-colon or a double-slash or double-backslash. For this reason, it reports "unable to normalize alternate object path".

The conclusion is: Do not use two different flavors of Git on a repository that is set up with a link to an alternate repository.

BTW, I did not do the followup test with `git am`. (Actually, I did, but there were no errors.) So, I cannot tell why your MSYS2 version suddenly also reports the "unable to normalize" error.

-- Hannes
L. E. Segovia· Apr 12, 2024, 17:19 UTC · re: Johannes Sixt · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Hi Johannes,

Indeed there are two -- one lives under the MSYS2 prefix, the other under Program Files. However, Cargo seems to access the Git data directly, so I believe you're right -- it's not able to understand the path syntax.

As for MSYS2, so far my testing points to an inability to chain back to the original clone. Given the mixup of installs, it's best to focus on fixing that.

Thanks for the explanation!
amyspark
On 12/04/2024 13:16, Johannes Sixt wrote:
Show 62 quoted lines
> Am 12.04.24 um 15:05 schrieb L. E. Segovia:
>> Under a MSYS2 shell, follow these steps (feel free to replace the choice
>> of remote repo with any other supporting a Rust crate):
>>
>> 1. `cd` to a directory of your choice (let's call it `dirA`).
>> 2. `git clone https://gitlab.gnome.org/GNOME/librsvg.git .`
>> 3. `git checkout -b cerbero_build`.
>> 4. `cd ..`
>> 5. `git clone <full path to dirA> -s -b cerbero_build dirB`
>> 6. `cd dirB`
>> 7. `git status`
>>
>> Now switch to any other shell of your choice, such as Git Bash, CMD, or
>> PowerShell, and try issuing commands such as `git status`. It will
>> result in a wall of "error: unable to normalize..."
> 
> Thanks, that is much more helpful. Here are my assumption, my
> explanation of the symptoms, and the conclusion (which is basically "if
> it hurts, don't do it"):
> 
> I assume that you have two installations of Git. One installation is
> used when you use the git command from the MSYS2 shell; it is the MSYS2
> version. The other installation is Git for Windows or equivalent that is
> used by all the other shells.
> 
> When you clone a repository with -s, a .git/objects/info/alternates file
> is initialized that contains the path to the original repository. You
> can see its contents with `git count-objects -v`. I tested the
> reproduction recipe with my own build (which is a MinGW build, much like
> Git for Windows), and it looks like this:
> 
> C:\Temp\repoclone>git count-objects -v
> count: 0
> size: 0
> in-pack: 0
> packs: 0
> size-pack: 0
> prune-packable: 0
> garbage: 0
> size-garbage: 0
> alternate: C:/Temp/repoorig/.git/objects
> 
> See the last line? It begins with a drive letter. When you call the
> command with the MSYS2 version, it will look more like this:
> 
> alternate: /c/Temp/repoorig/.git/objects
> 
> This is the MSYS2/Cygwin absolute path with a "drive letter".
> Unfortunately, this kind of path is unintelligible for Git for Windows.
> It expects absolute paths to begin with drive letter-colon or a
> double-slash or double-backslash. For this reason, it reports "unable to
> normalize alternate object path".
> 
> The conclusion is: Do not use two different flavors of Git on a
> repository that is set up with a link to an alternate repository.
> 
> BTW, I did not do the followup test with `git am`. (Actually, I did, but
> there were no errors.) So, I cannot tell why your MSYS2 version suddenly
> also reports the "unable to normalize" error.
> 
> -- Hannes
> 
-- 
amyspark 🌸 https://www.amyspark.me
Junio C Hamano· Apr 12, 2024, 17:32 UTC · re: Johannes Sixt · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Johannes Sixt <j6t@kdbg.org> writes:
Show 11 quoted lines
> alternate: C:/Temp/repoorig/.git/objects
> alternate: /c/Temp/repoorig/.git/objects
>
> This is the MSYS2/Cygwin absolute path with a "drive letter".
> Unfortunately, this kind of path is unintelligible for Git for Windows.
> It expects absolute paths to begin with drive letter-colon or a
> double-slash or double-backslash. For this reason, it reports "unable to
> normalize alternate object path".
>
> The conclusion is: Do not use two different flavors of Git on a
> repository that is set up with a link to an alternate repository.

I do not complain to the conclusion, and I do not use Windows, but it makes me wonder if there is a way to spell that full path that can be understood by both implementations. The two implementations are not incompatible in the actual object contents and refnames and other things in .git/ directory, are they?

In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style, and if so, writing that out, even though it may not what it considers the native format, would make the world a happier place.

rsbecker@nexbridge.com· Apr 12, 2024, 17:41 UTC · re: Junio C Hamano · lore

RE: [BUG] Clones from local repositories do not work correctly under Windows

On Friday, April 12, 2024 1:33 PM, Junio C Hamano wrote:
Show 15 quoted lines
>Johannes Sixt <j6t@kdbg.org> writes:
>
>> alternate: C:/Temp/repoorig/.git/objects
>> alternate: /c/Temp/repoorig/.git/objects
>>
>> This is the MSYS2/Cygwin absolute path with a "drive letter".
>> Unfortunately, this kind of path is unintelligible for Git for Windows.
>> It expects absolute paths to begin with drive letter-colon or a
>> double-slash or double-backslash. For this reason, it reports "unable
>> to normalize alternate object path".
>>
>> The conclusion is: Do not use two different flavors of Git on a
>> repository that is set up with a link to an alternate repository.
>
>I do not complain to the conclusion, and I do not use Windows, but it makes
me
>wonder if there is a way to spell that full path that can be understood by
both
>implementations.  The two implementations are not incompatible in the
actual
>object contents and refnames and other things in .git/ directory, are they?
>
>In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style, and if
so,
>writing that out, even though it may not what it considers the native
format, would
>make the world a happier place.

I'm my environment (<2 month old Cygwin update), the path would be /cygdrive/c/temp... I have not had issues using that style with git. Cygwin itself does not understand C:\Temp (resolves to C:temp), but does understand C:/Temp in bash and various utilities. It also understands (and expects) ~ to mean /home/user (C:\Cygwin64\home\user) in the Cygwin space, not relative to C:\Users\user for resolving .gitconfig.

L. E. Segovia· Apr 12, 2024, 20:10 UTC · re: rsbecker@nexbridge.com · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Hi all,

In Cerbero, we use cygpath or a bespoke conversion mechanism, depending on availability:

https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/main/cerbero/utils/__init__.py#L142-149
I'm not sure if there's a function in MSYS2's libc exposing that.
amyspark
On 12/04/2024 14:41, rsbecker@nexbridge.com wrote:
Show 36 quoted lines
> On Friday, April 12, 2024 1:33 PM, Junio C Hamano wrote:
>> Johannes Sixt <j6t@kdbg.org> writes:
>>
>>> alternate: C:/Temp/repoorig/.git/objects
>>> alternate: /c/Temp/repoorig/.git/objects
>>>
>>> This is the MSYS2/Cygwin absolute path with a "drive letter".
>>> Unfortunately, this kind of path is unintelligible for Git for Windows.
>>> It expects absolute paths to begin with drive letter-colon or a
>>> double-slash or double-backslash. For this reason, it reports "unable
>>> to normalize alternate object path".
>>>
>>> The conclusion is: Do not use two different flavors of Git on a
>>> repository that is set up with a link to an alternate repository.
>>
>> I do not complain to the conclusion, and I do not use Windows, but it makes
> me
>> wonder if there is a way to spell that full path that can be understood by
> both
>> implementations.  The two implementations are not incompatible in the
> actual
>> object contents and refnames and other things in .git/ directory, are they?
>>
>> In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style, and if
> so,
>> writing that out, even though it may not what it considers the native
> format, would
>> make the world a happier place.
> 
> I'm my environment (<2 month old Cygwin update), the path would be
> /cygdrive/c/temp... I have not had issues using that style with git. Cygwin
> itself does not understand C:\Temp (resolves to C:temp), but does understand
> C:/Temp in bash and various utilities. It also understands (and expects) ~
> to mean /home/user (C:\Cygwin64\home\user) in the Cygwin space, not relative
> to C:\Users\user for resolving .gitconfig.
> 
-- 
amyspark 🌸 https://www.amyspark.me
Johannes Sixt· Apr 12, 2024, 21:53 UTC · re: Junio C Hamano · lore

Re: [BUG] Clones from local repositories do not work correctly under Windows

Am 12.04.24 um 19:32 schrieb Junio C Hamano:
Show 23 quoted lines
> Johannes Sixt <j6t@kdbg.org> writes:
> 
>> alternate: C:/Temp/repoorig/.git/objects
>> alternate: /c/Temp/repoorig/.git/objects
>>
>> This is the MSYS2/Cygwin absolute path with a "drive letter".
>> Unfortunately, this kind of path is unintelligible for Git for Windows.
>> It expects absolute paths to begin with drive letter-colon or a
>> double-slash or double-backslash. For this reason, it reports "unable to
>> normalize alternate object path".
>>
>> The conclusion is: Do not use two different flavors of Git on a
>> repository that is set up with a link to an alternate repository.
> 
> I do not complain to the conclusion, and I do not use Windows, but
> it makes me wonder if there is a way to spell that full path that
> can be understood by both implementations.  The two implementations
> are not incompatible in the actual object contents and refnames and
> other things in .git/ directory, are they?
> 
> In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style,
> and if so, writing that out, even though it may not what it
> considers the native format, would make the world a happier place.

Yes, MSYS2 should understand C:/Temp. This can be achieved easily by doing the clone with Git for Windows. A small complication is, though, that the actual contents of the .git/objects/info/alternates is constructed from the literal command line arguments. In my test I used

   git clone -s C:\Temp\repoorig repoclone
which writes
   C:\Temp\repoorig/.git/objects

to the file. The path we see in the output of count-objects -v is obviously a normalized version. Users will have to say

   git clone -s C:/Temp/repoorig repoclone
to get the more widely understood version into the file.
-- Hannes

← back to recent threads