threads / discuss / 30599

Interpretation of '/' changed sparse-checkout

Subject: Interpretation of '/' changed sparse-checkout

## tl;dr

4 messages between May 23, 2012 and May 25, 2012.

replies: 3people: 2as markdown or json

Caleb Marchent· May 23, 2012, 13:40 UTC · lore

I have a system where I checkout by default everything except one directory. Then the rules in the top-level sources can selectivly reintroduce subdirectories as required for the build target. I have deployed, using git 1.7.2.5, a set of git trees which use the following sparse-checkout rules to achieve this.

/ !exclude/ exclude/reinclude

I have however discovered that if the reincluded directory is the same as the first part of any other path then that path will also be checked out.

/ !exlcude exclude/wanted

will checkout both:

exclude/wanted exclude/wanted_not

This appears to have bee fixed in git 1.7.10.2 but from that version of git onwards using '/' to request everything and then exclude a specific directory from it no longer works.

While I have been able to contrive the same effect with recent versions of git using the following updated sparse-checkout rules

/* !exclude/ exclude/reinclude

Even with this work-around there are two problems:

1) If I have a path: exclude/reinclude/exclude, this will now be exluded where I 
would expect it to be included as it was with 1.7.2.5

2) I have a large number of deployed trees across a number of machines that have 
the existing format which will break when git operations are performed after a 
upgrade to the latest git.

I believe that the interpretation of '/' should be 'everything in the repo', from which later rules can exclude. I think this is a bug in recent versions of git. I have been working on some ideas for a fix to the git code based along these lines. Can anyone confirm if my analysis is correct or provide an alternate solution?

Also I would like to know if anyone knows the solution to point 1) above as this also blocks upgrading to the latest version of git.

Caleb Marchent· May 23, 2012, 14:09 UTC · re: Caleb Marchent · lore

Re: Interpretation of '/' changed sparse-checkout

Caleb Marchent <cmarchent <at> aminocom.com> writes:

Corrected cut-paste errors on first submission, seems I missed a bunch of terminating '/' when hand-copying the text in from the sparse-checkout files :-( Sorry.

/ !exclude/ exclude/reinclude/

I have however discovered that if the reincluded directory is the same as the 
first part of any other path then that path will also be checked out.
 
/
!exlcude/
exclude/wanted/

 
will checkout both:

exclude/wanted/ exclude/wanted_not/

This appears to have bee fixed in git 1.7.10.2 but from that version of git 
onwards using '/' to request everything and then exclude a specific directory 
from it no longer works.
 
While I have been able to contrive the same effect with recent versions of git 
using the following updated sparse-checkout rules
 
/*
!exclude/
exclude/reinclude/

 
Even with this work-around there are two problems:

1) If I have a path: exclude/reinclude/exclude, this will now be exluded where 
I would expect it to be included as it was with 1.7.2.5
 
2) I have a large number of deployed trees across a number of machines that 
have the existing format which will break when git operations are performed 
after a upgrade to the latest git.
 
I believe that the interpretation of '/' should be 'everything in the repo', 
from which later rules can exclude. I think this is a bug in recent versions 
of git. I have been working on some ideas for a fix to the git code based along 
these lines. Can anyone confirm if my analysis is correct or provide an 
alternate solution?
 
Also I would like to know if anyone knows the solution to point 1) above as 
this also blocks upgrading to the latest version of git.

Nguyen Thai Ngoc Duy· May 24, 2012, 14:28 UTC · re: Caleb Marchent · lore

Re: Interpretation of '/' changed sparse-checkout

Hi,

On Wed, May 23, 2012 at 02:09:24PM +0000, Caleb Marchent wrote:

Show 34 quoted lines

> Caleb Marchent <cmarchent <at> aminocom.com> writes:
> 
> Corrected cut-paste errors on first submission, seems I missed a bunch of 
> terminating '/' when hand-copying the text in from the sparse-checkout files :-( 
> Sorry.
> 
> I have a system where I checkout by default everything except one directory. 
> Then the rules in the top-level sources can selectivly reintroduce 
> subdirectories as required for the build target.
> I have deployed, using git 1.7.2.5, a set of git trees which use the following 
> sparse-checkout rules to achieve this.
> 
> /
> !exclude/
> exclude/reinclude/
> 
> 
> I have however discovered that if the reincluded directory is the same as the 
> first part of any other path then that path will also be checked out.
>  
> /
> !exlcude/
> exclude/wanted/
> 
>  
> will checkout both:
> 
> exclude/wanted/
> exclude/wanted_not/
> 
> 
> This appears to have bee fixed in git 1.7.10.2 but from that version of git 
> onwards using '/' to request everything and then exclude a specific directory 
> from it no longer works.

As a user I agree that "/" should mean "everything under the sky" and your use case should work (i.e. it is broken in 1.7.10.2). If you can build git, you can try the patch below. I'll submit a patch for inclusion to git later (perhaps with optimization for this special case).

-- 8< --

diff --git a/unpack-trees.c b/unpack-trees.c
index bcee99c..188c196 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -838,7 +838,8 @@ static int clear_ce_flags_dir(struct cache_entry **cache, int nr,
 	int dtype = DT_DIR;
 	int ret = excluded_from_list(prefix, prefix_len, basename, &dtype, el);
 
-	prefix[prefix_len++] = '/';
+	if (prefix_len)
+		prefix[prefix_len++] = '/';
 
 	/* If undecided, use matching result of parent dir in defval */
 	if (ret < 0)
@@ -953,10 +954,12 @@ static int clear_ce_flags(struct cache_entry **cache, int nr,
 			    struct exclude_list *el)
 {
 	char prefix[PATH_MAX];
-	return clear_ce_flags_1(cache, nr,
-				prefix, 0,
-				select_mask, clear_mask,
-				el, 0);
+	*prefix = '\0';
+	return clear_ce_flags_dir(cache, nr,
+				  prefix, 0,
+				  "",
+				  select_mask, clear_mask,
+				  el, 0);
 }
 
 /*

-- 8< --

>  
> While I have been able to contrive the same effect with recent versions of git 
> using the following updated sparse-checkout rules
>  
> /*
> !exclude/
> exclude/reinclude/
> 
>  
> Even with this work-around there are two problems:
> 
> 1) If I have a path: exclude/reinclude/exclude, this will now be exluded where 
> I would expect it to be included as it was with 1.7.2.5
>  
> 2) I have a large number of deployed trees across a number of machines that 
> have the existing format which will break when git operations are performed 
> after a upgrade to the latest git.
>  
> I believe that the interpretation of '/' should be 'everything in the repo', 
> from which later rules can exclude. I think this is a bug in recent versions 
> of git. I have been working on some ideas for a fix to the git code based along 
> these lines. Can anyone confirm if my analysis is correct or provide an 
> alternate solution?
>  
> Also I would like to know if anyone knows the solution to point 1) above as 
> this also blocks upgrading to the latest version of git.

-- 
Duy

Caleb Marchent· May 25, 2012, 10:21 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: Interpretation of '/' changed sparse-checkout

Nguyen Thai Ngoc Duy <pclouds <at> gmail.com> writes:

Show 6 quoted lines

> 
> As a user I agree that "/" should mean "everything under the sky" and
> your use case should work (i.e. it is broken in 1.7.10.2). If you can
> build git, you can try the patch below. I'll submit a patch for
> inclusion to git later (perhaps with optimization for this special case).
>

Many Thanks Duy!, that patch works so please do submit the patch.

That just leaves the other issue I noted in my original posting - if I 
exclude a path from the top-level with a 1.7.2.5 format sparse-checkout
 file, git 1.7.10.2 will exclude a sub-directory with the same name:

/ !exclude/

Unfortunately (for me), I think this is correct behaviour of 1.7.10.2 because in 1.7.10.2 you can now do:

/ !/exclude/ /exlucde/reinclude/

which makes sense and correctly excludes "/exclude", but not "/exclude/reinclude/exclude". The flexibility to have:

/ !CVS/

or something like that is required; so I don't believe should be changed?