threads / patch / 18402

patchDefine a version of lstat(2) with posix semantics

Subject: [PATCH] Define a version of lstat(2) with posix semantics

## tl;dr

13 messages between Mar 19, 2009 and Mar 20, 2009. Diffs are folded; open one to read it.

replies: 12people: 4as markdown or json

Alex Riesen· Mar 19, 2009, 10:27 UTC · lore

So that Cygwin port can continue work around its supporting library and get access to its faked file attributes.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
2009/3/18 Junio C Hamano <gitster@pobox.com>:
> Yuck; that's a bit too ugly for generic code.  Will there be other places
> that this needs to be used?  If so, we'd probably need to encourage its
> use where appropriate, which is even uglier but we cannot avoid it...

You are damn right! I suggest providing posix_lstat, which has the posix semantics regarding x-bit in st_mode. Maybe even posix_stat is need, too.

There are other places which went broken recently in this regard (lstat cache patches?). Some will have to use this posix_lstat (the copy_templates_1), the others have to get a careful core.filemode handling (like check_remove in diff-lib.c, which isn't just about removed files).

 git-compat-util.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
From b0e3710e7e60d8f81bf252ff9987e17ece4a3be3 Mon Sep 17 00:00:00 2001
From: Alex Riesen <raa.lkml@gmail.com>
Date: Thu, 19 Mar 2009 11:17:01 +0100
Subject: [PATCH] Define a version of lstat(2) with posix semantics

So that Cygwin port can continue work around its supporting library and get access to its faked file attributes.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
 git-compat-util.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
Show changes to git-compat-util.h +5 −0
diff --git a/git-compat-util.h b/git-compat-util.h
index 1906253..2401f84 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -85,11 +85,16 @@
 #undef _XOPEN_SOURCE
 #include <grp.h>
 #define _XOPEN_SOURCE 600
+static inline int posix_lstat(const char *file_name, struct stat *buf)
+{
+	return lstat(file_name, buf);
+}
 #include "compat/cygwin.h"
 #else
 #undef _ALL_SOURCE /* AIX 5.3L defines a struct list with _ALL_SOURCE. */
 #include <grp.h>
 #define _ALL_SOURCE 1
+#define posix_lstat lstat
 #endif
 #else 	/* __MINGW32__ */
 /* pull in Windows compatibility stuff */

-- 
1.6.2.142.gaf8db
Johannes Schindelin· Mar 19, 2009, 10:57 UTC · re: Alex Riesen · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

Hi,
On Thu, 19 Mar 2009, Alex Riesen wrote:
Show 5 quoted lines
> So that Cygwin port can continue work around its supporting
> library and get access to its faked file attributes.
> 
> Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
> ---
[patch not inlined: therefore you'll have to guess what I am referring to]

It seems quite wrong to define something for other platforms when only Cygwin is affected.

I'd rather just disable WIN32_STAT for Cygwin, because otherwise, we will keep running into issues.

Ciao, Dscho

Alex Riesen· Mar 19, 2009, 21:40 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

So that Cygwin port can continue work around its supporting library and get access to its faked file attributes.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
Johannes Schindelin, Thu, Mar 19, 2009 11:57:01 +0100:
Show 6 quoted lines
> On Thu, 19 Mar 2009, Alex Riesen wrote:
> > So that Cygwin port can continue work around its supporting
> > library and get access to its faked file attributes.
> > 
> 
> [patch not inlined: therefore you'll have to guess what I am referring to]
I resend.
> It seems quite wrong to define something for other platforms when only 
> Cygwin is affected.

I know. Didn't I already said I'm sorry? I even said that it is fine if the patches wont be applied. Just wanted to share the pain.

> I'd rather just disable WIN32_STAT for Cygwin, because otherwise, we will 
> keep running into issues.

I'd rather not. The thing is just so unbelievably slow and being stuck on it I'm just trying my damnedest to squeeze every last bit of performance out of it.

 git-compat-util.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
Show changes to git-compat-util.h +5 −0
diff --git a/git-compat-util.h b/git-compat-util.h
index f09f244..c99549d 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -85,11 +85,16 @@
 #undef _XOPEN_SOURCE
 #include <grp.h>
 #define _XOPEN_SOURCE 600
+static inline int posix_lstat(const char *file_name, struct stat *buf)
+{
+	return lstat(file_name, buf);
+}
 #include "compat/cygwin.h"
 #else
 #undef _ALL_SOURCE /* AIX 5.3L defines a struct list with _ALL_SOURCE. */
 #include <grp.h>
 #define _ALL_SOURCE 1
+#define posix_lstat lstat
 #endif
 #else 	/* __MINGW32__ */
 /* pull in Windows compatibility stuff */

-- 
1.6.2.1.237.g7206c6
Alex Riesen· Mar 19, 2009, 21:43 UTC · re: Alex Riesen · lore

[PATCH] git clone needs to know executability of template files

Use posix_lstat for template source. The copy_templates_1 needs to know the file mode information for the copy of the template. The copying of the executable bit always work on POSIX systems and sometimes even on Cygwin.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
 builtin-init-db.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Show changes to builtin-init-db.c +1 −1
diff --git a/builtin-init-db.c b/builtin-init-db.c
index ee3911f..0b6dc16 100644
--- a/builtin-init-db.c
+++ b/builtin-init-db.c
@@ -66,7 +66,7 @@ static void copy_templates_1(char *path, int baselen,
 		else
 			exists = 1;
 
-		if (lstat(template, &st_template))
+		if (posix_lstat(template, &st_template))
 			die("cannot stat template %s", template);
 
 		if (S_ISDIR(st_template.st_mode)) {

-- 
1.6.2.1.237.g7206c6
Johannes Schindelin· Mar 19, 2009, 23:30 UTC · re: Alex Riesen · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

Hi,
On Thu, 19 Mar 2009, Alex Riesen wrote:
Show 8 quoted lines
> Johannes Schindelin, Thu, Mar 19, 2009 11:57:01 +0100:
> 
> > I'd rather just disable WIN32_STAT for Cygwin, because otherwise, we 
> > will keep running into issues.
> 
> I'd rather not. The thing is just so unbelievably slow and being stuck 
> on it I'm just trying my damnedest to squeeze every last bit of 
> performance out of it.

If you are serious about performance, you will not stay with Cygwin -- for the purposes of Git.

Do not get me wrong: Cygwin is a wonderful thing if your goal is to spare yourself a lot of trouble with that seriously challenged win32 API.

But if your goal is to get the most out of the Win32 API in terms of speed, you _will_ have to go with MinGW (at least, as long as you are unwilling to shell out big bucks in the vague direction of Redmond, and add some time tax to that).

Now, we _do_ have msysGit, you _do_ have shown the capability to fix issues when they arise, so I do _not_ see any obstacle why you should not go msysGit, rather than staying with the pain of trying to stay POSIX-compatible, but not quite all the time.

Ciao, Dscho

Alex Riesen· Mar 20, 2009, 08:30 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
Show 12 quoted lines
> On Thu, 19 Mar 2009, Alex Riesen wrote:
>> Johannes Schindelin, Thu, Mar 19, 2009 11:57:01 +0100:
>>
>> > I'd rather just disable WIN32_STAT for Cygwin, because otherwise, we
>> > will keep running into issues.
>>
>> I'd rather not. The thing is just so unbelievably slow and being stuck
>> on it I'm just trying my damnedest to squeeze every last bit of
>> performance out of it.
>
> If you are serious about performance, you will not stay with Cygwin -- for
> the purposes of Git.

I'm staying with Cygwin port not _for_ Git, but _because_ of Cygwin. If it were for me, I'd gladly let the Cygwin port die (frankly, any Windows port). I just use Linux everywhere possible. Sometimes it's not...

> Do not get me wrong: Cygwin is a wonderful thing if your goal is to spare
> yourself a lot of trouble with that seriously challenged win32 API.

I beg to differ: Cygwin is a piece of crap and performs dangerously hideous in what it tries to do.

> But if your goal is to get the most out of the Win32 API in terms of
> speed, you _will_ have to go with MinGW (at least, as long as you are
> unwilling to shell out big bucks in the vague direction of Redmond, and
> add some time tax to that).

My goal in this environment is to make it bearable. I think that both Cygwin and MinGW ports are dead-on-arrival, because the underlying platform is so stupidly and hopelessly broken, and no amount of no bucks can help it.

> Now, we _do_ have msysGit, you _do_ have shown the capability to fix
> issues when they arise, so I do _not_ see any obstacle why you should not
> go msysGit, rather than staying with the pain of trying to stay
> POSIX-compatible, but not quite all the time.

I understand. It is not pure POSIX compatibility I seek. I just can't use MinGW port, because I absolutely must use the cygwin environment (for "hysterical" reasons) and they don't play well together (tried, yes. Conflicting libraries, but you already know that).

I don't have the Windows sickness anywhere else but at my work, and sadly it is a good place in all respects, even if absolute crazy in this regard.

So I'm sorry, if I ever made an impression that I'm motivated to do any Windows-related development. I'm _absolutely_ not. I just happen to have that itch of Perforce (may it burn in hell for thousand years), have a know how, and have a little time to spare (hopefully no one of management sees that).

Johannes Schindelin· Mar 20, 2009, 09:17 UTC · re: Alex Riesen · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

Hi,
On Fri, 20 Mar 2009, Alex Riesen wrote:
Show 10 quoted lines
> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> > Now, we _do_ have msysGit, you _do_ have shown the capability to fix 
> > issues when they arise, so I do _not_ see any obstacle why you should 
> > not go msysGit, rather than staying with the pain of trying to stay 
> > POSIX-compatible, but not quite all the time.
> 
> I understand. It is not pure POSIX compatibility I seek. I just can't 
> use MinGW port, because I absolutely must use the cygwin environment 
> (for "hysterical" reasons) and they don't play well together (tried, 
> yes. Conflicting libraries, but you already know that).

Maybe we can work on those conflicting libraries? After all, we do have a "rebase.exe" tool now (for all those as puzzled by the naming as I was: the rebase.exe tool can shift the memory range used by a .dll so that it does not overlap with that one of another .dll).

Ciao, Dscho

Alex Riesen· Mar 20, 2009, 13:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
Show 17 quoted lines
> On Fri, 20 Mar 2009, Alex Riesen wrote:
>
>> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>> > Now, we _do_ have msysGit, you _do_ have shown the capability to fix
>> > issues when they arise, so I do _not_ see any obstacle why you should
>> > not go msysGit, rather than staying with the pain of trying to stay
>> > POSIX-compatible, but not quite all the time.
>>
>> I understand. It is not pure POSIX compatibility I seek. I just can't
>> use MinGW port, because I absolutely must use the cygwin environment
>> (for "hysterical" reasons) and they don't play well together (tried,
>> yes. Conflicting libraries, but you already know that).
>
> Maybe we can work on those conflicting libraries?  After all, we do have a
> "rebase.exe" tool now (for all those as puzzled by the naming as I was:
> the rebase.exe tool can shift the memory range used by a .dll so that it
> does not overlap with that one of another .dll).

As long as they can be made to coexist I'm fine. Wasn't the problem that MinGW/MSYS used cygwin1.dll if it were in PATH? Or was it something else with their supporting libraries?

My other problem is that the cygwin programs, and the worst of all - a proprietary compiler based on cygwin, must be in PATH. AFAIR, the presence of cygwin in PATH broken shell scripting.

Rogan Dawes· Mar 20, 2009, 13:49 UTC · re: Alex Riesen · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

Alex Riesen wrote:
Show 24 quoted lines
> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>> On Fri, 20 Mar 2009, Alex Riesen wrote:
>>
>>> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>>>> Now, we _do_ have msysGit, you _do_ have shown the capability to fix
>>>> issues when they arise, so I do _not_ see any obstacle why you should
>>>> not go msysGit, rather than staying with the pain of trying to stay
>>>> POSIX-compatible, but not quite all the time.
>>> I understand. It is not pure POSIX compatibility I seek. I just can't
>>> use MinGW port, because I absolutely must use the cygwin environment
>>> (for "hysterical" reasons) and they don't play well together (tried,
>>> yes. Conflicting libraries, but you already know that).
>> Maybe we can work on those conflicting libraries?  After all, we do have a
>> "rebase.exe" tool now (for all those as puzzled by the naming as I was:
>> the rebase.exe tool can shift the memory range used by a .dll so that it
>> does not overlap with that one of another .dll).
> 
> As long as they can be made to coexist I'm fine. Wasn't the problem
> that MinGW/MSYS used cygwin1.dll if it were in PATH? Or was it
> something else with their supporting libraries?
> 
> My other problem is that the cygwin programs, and the worst of all - a
> proprietary compiler based on cygwin, must be in PATH. AFAIR, the
> presence of cygwin in PATH broken shell scripting.

How about a wrapper that fixes the PATH before exec'ing git? i.e. removes cygwin and the compiler.

Rogan
Alex Riesen· Mar 20, 2009, 14:17 UTC · re: Rogan Dawes · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

2009/3/20 Rogan Dawes <lists@dawes.za.net>:
Show 29 quoted lines
> Alex Riesen wrote:
>> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>>> On Fri, 20 Mar 2009, Alex Riesen wrote:
>>>
>>>> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>>>>> Now, we _do_ have msysGit, you _do_ have shown the capability to fix
>>>>> issues when they arise, so I do _not_ see any obstacle why you should
>>>>> not go msysGit, rather than staying with the pain of trying to stay
>>>>> POSIX-compatible, but not quite all the time.
>>>> I understand. It is not pure POSIX compatibility I seek. I just can't
>>>> use MinGW port, because I absolutely must use the cygwin environment
>>>> (for "hysterical" reasons) and they don't play well together (tried,
>>>> yes. Conflicting libraries, but you already know that).
>>> Maybe we can work on those conflicting libraries?  After all, we do have a
>>> "rebase.exe" tool now (for all those as puzzled by the naming as I was:
>>> the rebase.exe tool can shift the memory range used by a .dll so that it
>>> does not overlap with that one of another .dll).
>>
>> As long as they can be made to coexist I'm fine. Wasn't the problem
>> that MinGW/MSYS used cygwin1.dll if it were in PATH? Or was it
>> something else with their supporting libraries?
>>
>> My other problem is that the cygwin programs, and the worst of all - a
>> proprietary compiler based on cygwin, must be in PATH. AFAIR, the
>> presence of cygwin in PATH broken shell scripting.
>
> How about a wrapper that fixes the PATH before exec'ing git? i.e.
> removes cygwin and the compiler.
>
For shame... I never tried :-/
Johannes Schindelin· Mar 20, 2009, 13:52 UTC · re: Alex Riesen · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

Hi,
On Fri, 20 Mar 2009, Alex Riesen wrote:
Show 22 quoted lines
> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> > On Fri, 20 Mar 2009, Alex Riesen wrote:
> >
> >> 2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> >> > Now, we _do_ have msysGit, you _do_ have shown the capability to 
> >> > fix issues when they arise, so I do _not_ see any obstacle why you 
> >> > should not go msysGit, rather than staying with the pain of trying 
> >> > to stay POSIX-compatible, but not quite all the time.
> >>
> >> I understand. It is not pure POSIX compatibility I seek. I just can't 
> >> use MinGW port, because I absolutely must use the cygwin environment 
> >> (for "hysterical" reasons) and they don't play well together (tried, 
> >> yes. Conflicting libraries, but you already know that).
> >
> > Maybe we can work on those conflicting libraries?  After all, we do 
> > have a "rebase.exe" tool now (for all those as puzzled by the naming 
> > as I was: the rebase.exe tool can shift the memory range used by a 
> > .dll so that it does not overlap with that one of another .dll).
> 
> As long as they can be made to coexist I'm fine. Wasn't the problem that 
> MinGW/MSYS used cygwin1.dll if it were in PATH? Or was it something else 
> with their supporting libraries?

I think it is the fact that msys-1.0.dll is a stripped-down fork of cygwin1.dll.

It is needed to run shell and perl scripts.

(Unfortunately, a few important parts of Git are still implemented as shell scripts: bisect, pull and rebase being the most obvious to me, but repack, stash and submodule are not too unimportant, either.)

> My other problem is that the cygwin programs, and the worst of all - a 
> proprietary compiler based on cygwin, must be in PATH. AFAIR, the 
> presence of cygwin in PATH broken shell scripting.

If it is a PATH issue, then it should be fixable by teaching msysGit to prepend $GIT_ROOT/bin and $GIT_ROOT/libexec/git-core to the PATH, but AFAIR we already do that.

*clicketyclick*

Yep, from reading setup_path() in exec_cmd.c, it appears that we prepend the PATH correctly.

Traditionally, we did have problems with Cygwin, that is correct, but I think with your help we can resolve the interaction issues.

Ciao, Dscho

Alex Riesen· Mar 20, 2009, 14:20 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

2009/3/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> On Fri, 20 Mar 2009, Alex Riesen wrote:
> (Unfortunately, a few important parts of Git are still implemented as
> shell scripts: bisect, pull and rebase being the most obvious to me, but
> repack, stash and submodule are not too unimportant, either.)
I can't imagine not using bisect or rebase.
Show 15 quoted lines
>> My other problem is that the cygwin programs, and the worst of all - a
>> proprietary compiler based on cygwin, must be in PATH. AFAIR, the
>> presence of cygwin in PATH broken shell scripting.
>
> If it is a PATH issue, then it should be fixable by teaching msysGit to
> prepend $GIT_ROOT/bin and $GIT_ROOT/libexec/git-core to the PATH, but
> AFAIR we already do that.
>
> *clicketyclick*
>
> Yep, from reading setup_path() in exec_cmd.c, it appears that we prepend
> the PATH correctly.
>
> Traditionally, we did have problems with Cygwin, that is correct, but I
> think with your help we can resolve the interaction issues.
Ok. I guess it is a time for me to take another look at mingw port.
Junio C Hamano· Mar 19, 2009, 22:08 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Define a version of lstat(2) with posix semantics

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 17 quoted lines
> Hi,
>
> On Thu, 19 Mar 2009, Alex Riesen wrote:
>
>> So that Cygwin port can continue work around its supporting
>> library and get access to its faked file attributes.
>> 
>> Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
>> ---
>
> [patch not inlined: therefore you'll have to guess what I am referring to]
>
> It seems quite wrong to define something for other platforms when only 
> Cygwin is affected.
>
> I'd rather just disable WIN32_STAT for Cygwin, because otherwise, we will 
> keep running into issues.
I am inclined to agree with this.

Back when Cygwin was the only choice, it was a way to bring benefit of git to folks who have to work on Windows, but with the recent advances in msysgit, probably Cygwin port should return to a role more in line with the overall Cygwin theme of bringing the more POSIXy sanity into Windows world. I personally see a Cygwin port as a vehicle for people who care about having a POSIXly-correct world where files have executable bits and lines are terminated with LF on Windows. If you want to have a system that is closer to Window's world view, there is (or will be, as msysgit is still officially marked as alpha) a viable alternative, and the current "selective cheating" Cygwin port does may benefit nobody.

But I do not work on Windows myself, so please take this only as a mere uninformed opinion, nothing more.

← back to recent threads