From: Philippe Blain Date: Thu, 07 Jan 2021 16:21:56 GMT Subject: Re: git-bugreport-2021-01-06-1209.txt (git can't deal with special characters) Message-ID: <613fb31c-c23d-0568-f311-74eb14bba2ca@gmail.com> In-Reply-To: <20210107154926.6tb7ukgemn7kmpnl@tb-raspi4> Hi Torsten, Le 2021-01-07 à 10:49, Torsten Bögershausen a écrit : > On Thu, Jan 07, 2021 at 09:34:35AM -0500, Philippe Blain wrote: >> Hi everyone, >> >> Le 2021-01-06 à 18:07, Randall S. Becker a écrit : >>> On January 6, 2021 5:22 PM, Daniel Troger wrote: >>>> Hi, maybe this helps you reproduce. I think I should have committed before >>>> doing the second changes but I still got the error message and the two >>>> names for one folder: >>>> >>>> me@iMac:/tmp$ mkdir git_bug >>>> me@iMac:/tmp$ cd git_bug >>>> me@iMac:/tmp/git_bug$ git init >>>> hint: Using 'master' as the name for the initial branch. This default branch >>>> name >>>> hint: is subject to change. To configure the initial branch name to use in all >>>> hint: of your new repositories, which will suppress this warning, call: >>>> hint: >>>> hint: git config --global init.defaultBranch >>>> hint: >>>> hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and >>>> hint: 'development'. The just-created branch can be renamed via this >>>> command: >>>> hint: >>>> hint: git branch -m >>>> Initialized empty Git repository in /private/tmp/git_bug/.git/ >>>> me@iMac:/tmp/git_bug$ ls -la total 8 >>>> drwxr-xr-x 4 daniel wheel 128 Jan 6 23:13 . >>>> drwxrwxrwt 27 root wheel 864 Jan 6 23:13 .. >>>> drwxr-xr-x 9 daniel wheel 288 Jan 6 23:12 .git >>>> -rw-r--r--@ 1 daniel staff 1283 Jan 6 23:13 paulbrunngård-springyard.zip >>>> me@iMac:/tmp/git_bug$ unzip paulbrunngård-springyard.zip >>>> Archive: paulbrunngård-springyard.zip >>>> creating: paulbrunnga??rd-springyard/ >>>> inflating: paulbrunnga??rd-springyard/.DS_Store >>>> creating: __MACOSX/ >>>> creating: __MACOSX/paulbrunnga??rd-springyard/ >>>> inflating: __MACOSX/paulbrunnga??rd-springyard/._.DS_Store >>>> extracting: paulbrunnga??rd-springyard/empty me@iMac:/tmp/git_bug$ rm >>>> -rf __MACOSX/ *.zip me@iMac:/tmp/git_bug$ ls -la total 0 >>>> drwxr-xr-x 4 daniel wheel 128 Jan 6 23:15 . >>>> drwxrwxrwt 27 root wheel 864 Jan 6 23:13 .. >>>> drwxr-xr-x 9 daniel wheel 288 Jan 6 23:15 .git >>>> drwxr-xr-x@ 4 daniel wheel 128 Jan 6 12:20 paulbrunngård-springyard >>>> me@iMac:/tmp/git_bug$ cd paulbrunngård-springyard/ >>>> me@iMac:/tmp/git_bug/paulbrunngård-springyard$ nano empty >>>> me@iMac:/tmp/git_bug/paulbrunngård-springyard$ cat empty Initial >>>> content me@iMac:/tmp/git_bug/paulbrunngård-springyard$ git add empty >>>> me@iMac:/tmp/git_bug/paulbrunngård-springyard$ nano empty >>>> me@iMac:/tmp/git_bug/paulbrunngård-springyard$ cat empty Initial >>>> content >>>> >>>> >>>> Line I want to keep >>>> >>>> Line I want gone >>>> me@iMac:/tmp/git_bug/paulbrunngård-springyard$ git restore -p . >>>> BUG: pathspec.c:495: error initializing pathspec_item Cannot close git diff- >>>> index --cached --numstat --summary >>>> 4b825dc642cb6eb9a060e54bf8d69288fbee4904 -- >>>> :(,prefix:27)paulbrunngård-springyard/ () at >>>> /usr/local/Cellar/git/2.30.0/libexec/git-core/git-add--interactive line 183. >>>> me@iMac:/tmp/git_bug/paulbrunngård-springyard$ cd .. >>>> me@iMac:/tmp/git_bug$ git status >>>> On branch master >>>> >>>> No commits yet >>>> >>>> Changes to be committed: >>>> (use "git rm --cached ..." to unstage) >>>> new file: "paulbrunnga\314\212rd-springyard/empty" >>>> >>>> Changes not staged for commit: >>>> (use "git add ..." to update what will be committed) >>>> (use "git restore ..." to discard changes in working directory) >>>> modified: "paulbrunnga\314\212rd-springyard/empty" >>>> >>>> Untracked files: >>>> (use "git add ..." to include in what will be committed) >>>> .DS_Store >>>> "paulbrunng\303\245rd-springyard/" >>>> >>>> me@iMac:/tmp/git_bug$ >>> >>> Is it possible that the å character is coming from a UTF-16 encoding and is not representable in UTF-8? I'm wondering whether the name has a double-byte representation where one of the bytes is null, resulting in a truncated file name coming from readdir(). The file name would not be representable on some platforms that do not support UTF-16 path names. >> >> I don't think that's the case (the angstrom is present in UTF-8 [1]). >> I think it's another UTF-8 precomposed/decomposed bug. As far as I >> was able to test it happens as soon as you have a precomposed character >> in the folder name. I observed the same behaviour with a folder named >> "folderü", for example. I also tried 'git -c add.interactive.usebuiltin restore -p .' >> to see if the new experimental builtin§ add-interactive has the same problem, >> and it does (though the error is less verbose). >> >> Anyway as you show with 'git status', it's not just 'git add -p' that is >> faulty, it's deeper than that, I would say. >> >> Cheers, >> >> Philippe. >> >> [1] https://en.wikipedia.org/wiki/%C3%85#On_computers > > Folks, I can not reproduce anything here. > - The zip file mentioned earlier does not include a decomposed "å" > Neither when running unzip under Linux nor under Mac > - Trying to write a test script does not show anything special > > My attempt looks like this: With a few modifications, you should see things start to get strange: > cat test.sh > #!/bin/sh > DIR=git-test-restore-p > GIT=/usr/local/bin/git > > Adiarnfc=$(printf '\303\204') > Adiarnfd=$(printf 'A\314\210') > DIRNAME=xx${Adiarnfd}yy > FILENAME=$DIRNAME/file change to: FILENAME=file > > rm -rf $DIR && > mkdir $DIR && > cd $DIR && > $GIT init && > mkdir $DIRNAME && add: cd $DIRNAME && > echo "Initial" >$FILENAME && add: $GIT status && > $GIT add $FILENAME && add: $GIT status && > echo >>$FILENAME && > echo >>$FILENAME && > echo "One more line" >>$FILENAME && > echo >>$FILENAME && > echo >>$FILENAME && > echo "Last line" >$FILENAME && add: $GIT status && > $GIT restore -p . && > $GIT restore -p . && > echo git=$GIT && > $GIT --version && > echo "OK" > > ================================== > It points out Git from brew, I think. I don't think it's a packaging bug; I tried with /usr/bin/git on macOS 10.11 (git version 2.10.1 (Apple Git-78)) and I also get a (different) error but the command does not fail (I needed to change restore to reset): $ ./test.sh Initialized empty Git repository in /private/var/folders/lr/r6n2057j0dzd4gdb614fp0740000gp/T/repro/git-test-restore-p/.git/ Assertion failed: (item->nowildcard_len <= item->len && item->prefix <= item->len), function prefix_pathspec, file pathspec.c, line 317. No changes. Assertion failed: (item->nowildcard_len <= item->len && item->prefix <= item->len), function prefix_pathspec, file pathspec.c, line 317. No changes. git=/usr/bin/git git version 2.10.1 (Apple Git-78) OK