Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

Hello,

Recently I installed 1.9.5 git version and faced the problem that one of the files in my cloned repository with a name ICON~714.PNG is marked as deleted even repository was freshly cloned. Trying to do anything with that file resulted in constant "Invalid Path" errors. Reverting back to 1.9.4. fixed that problem.

Thanks, Dmitry

On Wed, Jan 7, 2015 at 3:26 PM, Dmitry Bykov <pvrt74@gmail.com> wrote:

Git had a security issue with filenames which look similar to the .git repository. Please see the announcement at http://article.gmane.org/gmane.linux.kernel/1853266 (That also updated 1.9.4 -> 1.9.5)

I'm not sure if I can advice though.

Dscho, this sounds to me like the additional "8.3 ambiguity protection" (which is only in Git for Windows) in action. Any thoughts?

On Wed, Jan 7, 2015 at 3:26 PM, Dmitry Bykov <pvrt74@gmail.com> wrote:

Hi Junio,

On Wed, 7 Jan 2015, Junio C Hamano wrote:

First thought: the Git for Windows mailing list should be Cc:ed (I was traveling yesterday and somebody else might have been able to address Dmitry's problem).

Second thought below:

ICON~714.PNG is a valid short name for a long name (such as 'icon.background.png') because it fits the shortening scheme (8.3 format, the base name ends in ~<n>). As this can clash with a validly shortened long name, Git for Windows refuses to check out such paths by default.

If you want the old -- unsafe -- behavior back, just set your core.protectNTFS to false (this means that you agree that the incurred problems are your own responsibility and cannot be blamed on anybody else ;-))

Ciao, Johannes

On Thu, Jan 08, 2015 at 11:06:18AM +0100, Johannes Schindelin wrote:

I wonder if it is worth having a "git-only" mode for core.protectNTFS. Turning it off entirely would make him susceptible to GIT~1 attacks.

-Peff

Hi Peff,

On Thu, 8 Jan 2015, Jeff King wrote:

That is a good idea!

Ciao, Dscho

On 01/08/2015 11:28 AM, Jeff King wrote:

There is something more then just the "tilde" protection going on, it seems:

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master) $ ~/projects/git/msysgit/git ls-files ICON~714.PNG

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master) $ ~/projects/git/msysgit/git status On branch master nothing to commit, working directory clean

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master) $ ~/projects/git/tb/git status On branch master nothing to commit, working directory clean

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master) $ ~/projects/git/msysgit/git --version git version 1.9.2.msysgit.0.2273.gc47d6ec

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master) $ ~/projects/git/tb/git --version git version 2.2.1.268.g1e6f5b2

Neither latest msygit nor latest git.git reports a problem with a single file within a short path. It may be, that the '~' is not accepted in a long path or there are "problems" when a directory is filled with many files, but that is speculation.

Dimitry, is there a way to make the problem reproducible for other people ? Either a public demo-repo, or a step-by-step receipe ?

Hi Torsten,

On Thu, 8 Jan 2015, Torsten Bögershausen wrote:

Indeed. What is going on is that you build Git yourself, from git.git, while Dmitry obviously used Git for Windows -- which carries a couple of patches on top of upstream git.git.

In this particular case, the tilde protection was introduced in https://github.com/msysgit/git/commit/2e2a2d12.

Ciao, Johannes

On 01/08/2015 04:58 PM, Johannes Schindelin wrote:

My bad, I was building the msysgit master branch, which didn't have the 2e2a2d12.

However, I am wondering if 2e2a2d12 is really needed that much ? In the same spirit that we do not prevent the checkout of "FILE" when "file" is present, why do we prevent people from using "~" in file names ? When a file is visible under different file names, I assume that the user chooses only one representation to run "git add" on. But that is another discussion.

Jeff King <peff@peff.net> writes:

Yes, I think having distinctions would make sense, but I am not sure if "git-only" vs "every questionable included" should be the classification.

To classify various funniness of paths by the extent of damage they may cause for Windows:

 - ".Git" and "GIT~1" would be the worst level; 

 - "CON.c" and the like would be the next bad level; and

 - "ICON~714.PNG" would be in "to be avoided in an ideal world but
   as long as the project is comfortable with having it in, Git is
   in no position to complain" level.

that is just my knee-jerk reaction.

And if we were to go with just two level, I'd throw "CON.c" into the same level as ".Git"; they both are to break the host that checks out such paths. The former may break Git on the host, and the latter may break something that is not Git on the host, but they are not that different in that the end user on that host is harmed.

"8.3 ambiguity" does not directly harm individual hosts, but the reason to flag is primarily because such a path may make the project's contents unreliable (e.g. "icon1234234.png" may alias "ICON~714.PNG" on somebody's machine, causing confusion). The same reason should make us flag a tree object as suspect if it contains two paths that are the same case-insensitively (e.g. a tree that represents the "net/netfilter" part of the Linux kernel source would be flagged because it contains both "xt_tcpmss.c" and "xt_TCPMSS.c").

Let's call the former Harmful and the latter Questionable. "CON.c" is Harmful on Windows, while "net/netfilter" is Questionable on Windows. Orthogonal to this, we earlier discussed what to do in fsck and receive.fsckObjects. I think we want to be express different shades of grays between the two extremes:

 - A repository of a mono-culture project would want to flag
   Harmfuls and Questionables that apply to the project's intended
   platform as errors.  By definition, a mono-culture project would
   not care about paths that may only be Harmful on others'
   platforms.

 - A repository for a cross-platform project would want to flag
   paths that may be Harmful or Questionable on some platform, and
   does not care if these paths are Harmful or Questionable on the
   platform that happens to host the repository.  By definition, a
   cross-platform project wants to make sure everybody involved will
   not get hurt.

On Thu, Jan 08, 2015 at 11:08:47AM -0800, Junio C Hamano wrote:

Yeah, I agree with everything you said below (I should have said "security-relevant" instead of "git-only" originally). I also agree that there may be more than two classifications, but I do not know enough about the severity of the bad effects of each type to judge. I'd leave that to the folks on each platform.

-Peff