threads / patch / 44695

patchfix pushing to //server/share/dir paths on Windows

Subject: [PATCH] fix pushing to //server/share/dir paths on Windows

## tl;dr

8 messages between Dec 13, 2016 and Dec 19, 2016. Diffs are folded; open one to read it.

replies: 7people: 5as markdown or json

Johannes Sixt· Dec 13, 2016, 21:32 UTC · lore

normalize_path_copy() is not prepared to keep the double-slash of a //server/share/dir kind of path, but treats it like a regular POSIX style path and transforms it to /server/share/dir.

The bug manifests when 'git push //server/share/dir master' is run, because tmp_objdir_add_as_alternate() uses the path in normalized form when it registers the quarantine object database via link_alt_odb_entries(). Needless to say that the directory cannot be accessed using the wrongly normalized path.

Fix it by skipping all of the root part, not just a potential drive prefix. offset_1st_component takes care of this, see the implementation in compat/mingw.c::mingw_offset_1st_component().

There is a change in behavior: \\server\share is not transformed into //server/share anymore, but all subsequent directory separators are rewritten to '/'. This should not make a difference; Windows can handle the mix. In the context of 'git push' this cannot be verified, though, as there seems to be an independent bug that transforms the double '\\' to a single '\' on the way.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 Another long-standing bug uncovered by the quarantine series.
 Dscho, it looks like this could fix the original report at
 https://github.com/git-for-windows/git/issues/979
 This patch should cook well because of the change in behavior.
 I would not be surprised if there is some fall-out.
 The other bug I'm alluding to, I still have to investigate. I do
 not think that it can be counted as fall-out.
 path.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)
Show changes to path.c +15 −9
diff --git a/path.c b/path.c
index 52d889c88e..02dc70fb92 100644
--- a/path.c
+++ b/path.c
@@ -991,7 +991,7 @@ const char *remove_leading_path(const char *in, const char *prefix)
  *
  * Performs the following normalizations on src, storing the result in dst:
  * - Ensures that components are separated by '/' (Windows only)
- * - Squashes sequences of '/'.
+ * - Squashes sequences of '/' except "//server/share" on Windows
  * - Removes "." components.
  * - Removes ".." components, and the components the precede them.
  * Returns failure (non-zero) if a ".." component appears as first path
@@ -1014,17 +1014,23 @@ const char *remove_leading_path(const char *in, const char *prefix)
 int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
 {
 	char *dst0;
-	int i;
-
-	for (i = has_dos_drive_prefix(src); i > 0; i--)
-		*dst++ = *src++;
-	dst0 = dst;
+	int offset;
 
-	if (is_dir_sep(*src)) {
+	/*
+	 * Handle initial part of absolute path: "/", "C:/", "\\server\share/".
+	 */
+	offset = offset_1st_component(src);
+	if (offset) {
+		/* Convert the trailing separator to '/' on Windows. */
+		memcpy(dst, src, offset - 1);
+		dst += offset - 1;
 		*dst++ = '/';
-		while (is_dir_sep(*src))
-			src++;
+		src += offset;
 	}
+	dst0 = dst;
+
+	while (is_dir_sep(*src))
+		src++;
 
 	for (;;) {
 		char c = *src;

-- 
2.11.0.79.g263f27a
Junio C Hamano· Dec 13, 2016, 22:48 UTC · re: Johannes Sixt · lore

Re: [PATCH] fix pushing to //server/share/dir paths on Windows

Johannes Sixt <j6t@kdbg.org> writes:
> There is a change in behavior: \\server\share is not transformed
> into //server/share anymore, but all subsequent directory separators
> are rewritten to '/'. This should not make a difference; Windows can
> handle the mix.

I saw Dscho had a similar "windows can handle the mix" change in an earlier development cycle, I think, and this is being consistent.

Show 13 quoted lines
>  Another long-standing bug uncovered by the quarantine series.
>
>  Dscho, it looks like this could fix the original report at
>  https://github.com/git-for-windows/git/issues/979
>
>  This patch should cook well because of the change in behavior.
>  I would not be surprised if there is some fall-out.
>
>  The other bug I'm alluding to, I still have to investigate. I do
>  not think that it can be counted as fall-out.
>
>  path.c | 24 +++++++++++++++---------
>  1 file changed, 15 insertions(+), 9 deletions(-)
Thanks.
Show 10 quoted lines
> diff --git a/path.c b/path.c
> index 52d889c88e..02dc70fb92 100644
> --- a/path.c
> +++ b/path.c
> @@ -991,7 +991,7 @@ const char *remove_leading_path(const char *in, const char *prefix)
>   *
>   * Performs the following normalizations on src, storing the result in dst:
>   * - Ensures that components are separated by '/' (Windows only)
> - * - Squashes sequences of '/'.
> + * - Squashes sequences of '/' except "//server/share" on Windows

"on windows" because offset_1st_component() does the magic only there? Makes sense.

Show 29 quoted lines
>   * - Removes "." components.
>   * - Removes ".." components, and the components the precede them.
>   * Returns failure (non-zero) if a ".." component appears as first path
> @@ -1014,17 +1014,23 @@ const char *remove_leading_path(const char *in, const char *prefix)
>  int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
>  {
>  	char *dst0;
> -	int i;
> -
> -	for (i = has_dos_drive_prefix(src); i > 0; i--)
> -		*dst++ = *src++;
> -	dst0 = dst;
> +	int offset;
>  
> -	if (is_dir_sep(*src)) {
> +	/*
> +	 * Handle initial part of absolute path: "/", "C:/", "\\server\share/".
> +	 */
> +	offset = offset_1st_component(src);
> +	if (offset) {
> +		/* Convert the trailing separator to '/' on Windows. */
> +		memcpy(dst, src, offset - 1);
> +		dst += offset - 1;
>  		*dst++ = '/';
> -		while (is_dir_sep(*src))
> -			src++;
> +		src += offset;
>  	}
> +	dst0 = dst;

By resetting dst0 here, we ensure that up_one that is triggered by seeing "../" will not escape the \\server\share\ part, which makes sense to me.

Show 5 quoted lines
> +	while (is_dir_sep(*src))
> +		src++;
>  
>  	for (;;) {
>  		char c = *src;
Jeff King· Dec 14, 2016, 17:30 UTC · re: Johannes Sixt · lore

Re: [PATCH] fix pushing to //server/share/dir paths on Windows

On Tue, Dec 13, 2016 at 10:32:01PM +0100, Johannes Sixt wrote:
Show 9 quoted lines
> normalize_path_copy() is not prepared to keep the double-slash of a
> //server/share/dir kind of path, but treats it like a regular POSIX
> style path and transforms it to /server/share/dir.
> 
> The bug manifests when 'git push //server/share/dir master' is run,
> because tmp_objdir_add_as_alternate() uses the path in normalized
> form when it registers the quarantine object database via
> link_alt_odb_entries(). Needless to say that the directory cannot be
> accessed using the wrongly normalized path.

Thanks for digging this up! I had a feeling that the problem was going to be in the underlying path code, but I didn't want to just pass the buck without evidence. :)

Show 10 quoted lines
> -	if (is_dir_sep(*src)) {
> +	/*
> +	 * Handle initial part of absolute path: "/", "C:/", "\\server\share/".
> +	 */
> +	offset = offset_1st_component(src);
> +	if (offset) {
> +		/* Convert the trailing separator to '/' on Windows. */
> +		memcpy(dst, src, offset - 1);
> +		dst += offset - 1;
>  		*dst++ = '/';

Hmm. So this is the "change-of-behavior" bit. Would it be reasonable to write:

  /* Copy initial part of absolute path, converting separators on Windows */
  const char *end = src + offset_1st_component(src);
  while (src < end) {
	  char c = *src++;
	  if (c == '\\')
		  c = '/';
	  *dst++ = c;
  }

? I'm not sure if it's wrong to convert backslashes in that first component or not (but certainly we were before). I don't think we'd need is_dir_sep() in that "if()", because we can leave slashes as-is. But maybe it would make the code easier to read.

-Peff
Johannes Sixt· Dec 14, 2016, 19:37 UTC · re: Jeff King · lore

[PATCH v2] fix pushing to //server/share/dir on Windows

normalize_path_copy() is not prepared to keep the double-slash of a //server/share/dir kind of path, but treats it like a regular POSIX style path and transforms it to /server/share/dir.

The bug manifests when 'git push //server/share/dir master' is run, because tmp_objdir_add_as_alternate() uses the path in normalized form when it registers the quarantine object database via link_alt_odb_entries(). Needless to say that the directory cannot be accessed using the wrongly normalized path.

Fix it by skipping all of the root part, not just a potential drive prefix. offset_1st_component takes care of this, see the implementation in compat/mingw.c::mingw_offset_1st_component().

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
Am 14.12.2016 um 18:30 schrieb Jeff King:
Show 11 quoted lines
> Would it be reasonable to
> write:
> 
>   /* Copy initial part of absolute path, converting separators on Windows */
>   const char *end = src + offset_1st_component(src);
>   while (src < end) {
> 	  char c = *src++;
> 	  if (c == '\\')
> 		  c = '/';
> 	  *dst++ = c;
>   }

Makes a lot of sense! I haven't had an opportunity, though, to test on Windows.

> ? I'm not sure if it's wrong to convert backslashes in that first
> component or not (but certainly we were before). I don't think we'd need
> is_dir_sep() in that "if()", because we can leave slashes as-is. But
> maybe it would make the code easier to read.
is_dir_sep() is preferable, IMO.
I also changed the commit message and subject line slightly.
 path.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)
Show changes to path.c +14 −9
diff --git a/path.c b/path.c
index 52d889c88e..efcedafba6 100644
--- a/path.c
+++ b/path.c
@@ -991,7 +991,7 @@ const char *remove_leading_path(const char *in, const char *prefix)
  *
  * Performs the following normalizations on src, storing the result in dst:
  * - Ensures that components are separated by '/' (Windows only)
- * - Squashes sequences of '/'.
+ * - Squashes sequences of '/' except "//server/share" on Windows
  * - Removes "." components.
  * - Removes ".." components, and the components the precede them.
  * Returns failure (non-zero) if a ".." component appears as first path
@@ -1014,17 +1014,22 @@ const char *remove_leading_path(const char *in, const char *prefix)
 int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
 {
 	char *dst0;
-	int i;
+	const char *end;
 
-	for (i = has_dos_drive_prefix(src); i > 0; i--)
-		*dst++ = *src++;
+	/*
+	 * Copy initial part of absolute path: "/", "C:/", "//server/share/".
+	 */
+	end = src + offset_1st_component(src);
+	while (src < end) {
+		char c = *src++;
+		if (is_dir_sep(c))
+			c = '/';
+		*dst++ = c;
+	}
 	dst0 = dst;
 
-	if (is_dir_sep(*src)) {
-		*dst++ = '/';
-		while (is_dir_sep(*src))
-			src++;
-	}
+	while (is_dir_sep(*src))
+		src++;
 
 	for (;;) {
 		char c = *src;

-- 
2.11.0.79.g263f27a
Torsten Bögershausen· Dec 15, 2016, 07:30 UTC · re: Johannes Sixt · lore

Re: [PATCH v2] fix pushing to //server/share/dir on Windows

On 14/12/16 20:37, Johannes Sixt wrote:
Show 30 quoted lines
> normalize_path_copy() is not prepared to keep the double-slash of a
> //server/share/dir kind of path, but treats it like a regular POSIX
> style path and transforms it to /server/share/dir.
>
> The bug manifests when 'git push //server/share/dir master' is run,
> because tmp_objdir_add_as_alternate() uses the path in normalized
> form when it registers the quarantine object database via
> link_alt_odb_entries(). Needless to say that the directory cannot be
> accessed using the wrongly normalized path.
>
> Fix it by skipping all of the root part, not just a potential drive
> prefix. offset_1st_component takes care of this, see the
> implementation in compat/mingw.c::mingw_offset_1st_component().
>
> Signed-off-by: Johannes Sixt <j6t@kdbg.org>
> ---
> Am 14.12.2016 um 18:30 schrieb Jeff King:
>> Would it be reasonable to
>> write:
>>
>>    /* Copy initial part of absolute path, converting separators on Windows */
>>    const char *end = src + offset_1st_component(src);
>>    while (src < end) {
>> 	  char c = *src++;
>> 	  if (c == '\\')
>> 		  c = '/';
>> 	  *dst++ = c;
>>    }
> Makes a lot of sense! I haven't had an opportunity, though, to test
> on Windows.

I'm not sure, if a conversion should be done here, in this part of code. To my knowledge,

C:\dir1\file is the same as C:/dir1/file and that is handled by windows.

The \\server\share\dir1\file is native to windows, and I can't see good reasons to change '\' into '/' somewhere in Git, when UNC is used.

Cygwin does a translation from //server/share/dir1/file into \\server\share\dir1\file

In other words: The patch looks good as is, and once I get a Windows machine, may be able to do some testing and come up with test cases

<https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx> []

Jeff King· Dec 15, 2016, 11:01 UTC · re: Torsten Bögershausen · lore

Re: [PATCH v2] fix pushing to //server/share/dir on Windows

On Thu, Dec 15, 2016 at 08:30:52AM +0100, Torsten Bögershausen wrote:
Show 21 quoted lines
> > > Would it be reasonable to
> > > write:
> > > 
> > >    /* Copy initial part of absolute path, converting separators on Windows */
> > >    const char *end = src + offset_1st_component(src);
> > >    while (src < end) {
> > > 	  char c = *src++;
> > > 	  if (c == '\\')
> > > 		  c = '/';
> > > 	  *dst++ = c;
> > >    }
> > Makes a lot of sense! I haven't had an opportunity, though, to test
> > on Windows.
> I'm not sure, if a conversion should be done here, in this part of code.
> To my knowledge,
> 
> C:\dir1\file
> is the same
> as
> C:/dir1/file
> and that is handled by windows.

I don't have an opinion either way on what Windows would want, but note that the function already _does_ convert separators to slashes. With Johannes's original patch, you'd end up with a mix, like:

  \\server\share/dir1/file

So this conversion is really just retaining the original behavior, and making it consistent throughout the path.

Which isn't to say that the function as it currently exists isn't a little bit buggy. :)

One of the points of normalizing, though, is that Git can then do textual comparisons between the output. So I think there's value in having a canonical internal representation, even if the OS could handle more exotic ones.

-Peff
Junio C Hamano· Dec 15, 2016, 17:49 UTC · re: Jeff King · lore

Re: [PATCH v2] fix pushing to //server/share/dir on Windows

Jeff King <peff@peff.net> writes:
Show 8 quoted lines
> I don't have an opinion either way on what Windows would want, but note
> that the function already _does_ convert separators to slashes. With
> Johannes's original patch, you'd end up with a mix, like:
>
>   \\server\share/dir1/file
>
> So this conversion is really just retaining the original behavior, and
> making it consistent throughout the path.
Show 8 quoted lines
>
> Which isn't to say that the function as it currently exists isn't a
> little bit buggy. :)
>
> One of the points of normalizing, though, is that Git can then do
> textual comparisons between the output. So I think there's value in
> having a canonical internal representation, even if the OS could handle
> more exotic ones.

E.g. the log message of d53c2c6738 ("mingw: fix t9700's assumption about directory separators", 2016-01-27) says the two kinds of slashes are equivalent over there, but the patch text ends up doing exactly that normalization.

5ca6b7bb47 ("config --show-origin: report paths with forward slashes", 2016-03-23) is an example of us trying to normalize in order to give consistent output to the users.

Having said all that, I do not have an opinion either way on what Windows would want, either ;-)

Johannes Schindelin· Dec 19, 2016, 17:18 UTC · re: Johannes Sixt · lore

Re: [PATCH] fix pushing to //server/share/dir paths on Windows

Hi Hannes,
On Tue, 13 Dec 2016, Johannes Sixt wrote:
>  Dscho, it looks like this could fix the original report at
>  https://github.com/git-for-windows/git/issues/979
Yes, thank you so much for fixing it!

I think your v2 is safe enough to cherry-pick directly into Git for Windows' `master`.

Thanks, Dscho

← back to recent threads