threads / discuss / 28348

git checkout under 1.7.6 does not properly list untracked files and aborts

Subject: git checkout under 1.7.6 does not properly list untracked files and aborts

## tl;dr

23 messages between Sep 9, 2011 and Sep 22, 2011.

replies: 22people: 4as markdown or json

Joshua Jensen· Sep 9, 2011, 20:04 UTC · lore

This may be an msysGit 1.7.6 issue, as that is what I am using. It also occurs in msysGit 1.7.5, but I am almost certain it did not happen in msysGit 1.7.2.

Given an untracked file in the working directory that has been added to an alternate branch, when switching to that alternate branch, 'git checkout' exits with an error code but does not print anything to the console. It should print an untracked file error.

I have been trying to track this down in code. The point where the error messages are printed, display_error_msgs, is never hit.

I don't have a debugger to trace this on Windows, so I've just been adding lots of debug output. So far, I have not found the catalyst.

I am out on vacation now for the next week and a half. However, I thought I would send this along to see if someone else can confirm the the problem. I'll continue investigation when I'm back.

Thanks.
Josh
Joshua Jensen· Sep 19, 2011, 19:44 UTC · re: Joshua Jensen · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

----- Original Message -----
From: Joshua Jensen
Date: 9/9/2011 2:04 PM
Show 11 quoted lines
> This may be an msysGit 1.7.6 issue, as that is what I am using.  It 
> also occurs in msysGit 1.7.5, but I am almost certain it did not 
> happen in msysGit 1.7.2.
>
> Given an untracked file in the working directory that has been added 
> to an alternate branch, when switching to that alternate branch, 'git 
> checkout' exits with an error code but does not print anything to the 
> console.  It should print an untracked file error.
>
> I have been trying to track this down in code.  The point where the 
> error messages are printed, display_error_msgs, is never hit.
Okay, so I've tracked this down, but I am unsure what the correct fix is.
In unpack-trees.c's unpack_trees() function, there are some lines that read:
     if (ce->ce_flags & CE_ADDED &&
         verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
             return -1;

If the 'return -1' is changed to 'goto return_failed', then a proper error message appears:

     error: The following untracked working tree files would be 
overwritten by checkout:
         one/file/listed/here.txt
     Please move or remove them before you can switch branches.
     Aborting

The thing is, there are multiple files that would be overwritten by checkout, and I believe an older version of Git showed me the entire list before aborting.

What would be the proper fix here?  What am I doing wrong?
Thanks.
Josh
Joshua Jensen· Sep 19, 2011, 20:06 UTC · re: Joshua Jensen · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

----- Original Message -----
From: Joshua Jensen
Date: 9/19/2011 1:44 PM
Show 35 quoted lines
> ----- Original Message -----
> From: Joshua Jensen
> Date: 9/9/2011 2:04 PM
>> This may be an msysGit 1.7.6 issue, as that is what I am using.  It 
>> also occurs in msysGit 1.7.5, but I am almost certain it did not 
>> happen in msysGit 1.7.2.
>>
>> Given an untracked file in the working directory that has been added 
>> to an alternate branch, when switching to that alternate branch, 'git 
>> checkout' exits with an error code but does not print anything to the 
>> console.  It should print an untracked file error.
>>
>> I have been trying to track this down in code.  The point where the 
>> error messages are printed, display_error_msgs, is never hit.
> Okay, so I've tracked this down, but I am unsure what the correct fix is.
>
> In unpack-trees.c's unpack_trees() function, there are some lines that 
> read:
>
>     if (ce->ce_flags & CE_ADDED &&
>         verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
>             return -1;
>
> If the 'return -1' is changed to 'goto return_failed', then a proper 
> error message appears:
>
>     error: The following untracked working tree files would be 
> overwritten by checkout:
>         one/file/listed/here.txt
>     Please move or remove them before you can switch branches.
>     Aborting
>
> The thing is, there are multiple files that would be overwritten by 
> checkout, and I believe an older version of Git showed me the entire 
> list before aborting.

While probably not correct, the following patch collects all files that would be overwritten and logs them to the output:

---
  unpack-trees.c |   11 +++++++----
  1 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/unpack-trees.c b/unpack-trees.c
index cc616c3..e1f7263 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -993,7 +993,7 @@ static int verify_absent(struct cache_entry *, enum 
unpack_trees_error_types, st
   */
  int unpack_trees(unsigned len, struct tree_desc *t, struct 
unpack_trees_options *o)
  {
-    int i, ret;
+    int i, ret = 0;
      static struct cache_entry *dfc;
      struct exclude_list el;

@@ -1100,9 +1100,10 @@ int unpack_trees(unsigned len, struct tree_desc 
*t, struct unpack_trees_options
               * Do the real check now because we have had
               * correct CE_NEW_SKIP_WORKTREE
               */
-            if (ce->ce_flags & CE_ADDED &&
-                verify_absent(ce, 
ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
-                    return -1;
+            if (ce->ce_flags & CE_ADDED) {
+                int result = verify_absent(ce, 
ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o);
+                ret = min(ret, result);
+            }

              if (apply_sparse_checkout(ce, o)) {
                  ret = -1;
@@ -1112,6 +1113,8 @@ int unpack_trees(unsigned len, struct tree_desc 
*t, struct unpack_trees_options
                  empty_worktree = 0;

          }
+        if (ret == -1)
+            goto return_failed;
          if (o->result.cache_nr && empty_worktree) {
              /* dubious---why should this fail??? */
              ret = unpack_failed(o, "Sparse checkout leaves no entry on 
working directory");

-- 
1.7.6.msysgit.1
Michael J Gruber· Sep 20, 2011, 15:10 UTC · re: Joshua Jensen · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

Joshua Jensen venit, vidit, dixit 19.09.2011 21:44:
Show 36 quoted lines
> ----- Original Message -----
> From: Joshua Jensen
> Date: 9/9/2011 2:04 PM
>> This may be an msysGit 1.7.6 issue, as that is what I am using.  It 
>> also occurs in msysGit 1.7.5, but I am almost certain it did not 
>> happen in msysGit 1.7.2.
>>
>> Given an untracked file in the working directory that has been added 
>> to an alternate branch, when switching to that alternate branch, 'git 
>> checkout' exits with an error code but does not print anything to the 
>> console.  It should print an untracked file error.
>>
>> I have been trying to track this down in code.  The point where the 
>> error messages are printed, display_error_msgs, is never hit.
> Okay, so I've tracked this down, but I am unsure what the correct fix is.
> 
> In unpack-trees.c's unpack_trees() function, there are some lines that read:
> 
>      if (ce->ce_flags & CE_ADDED &&
>          verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
>              return -1;
> 
> If the 'return -1' is changed to 'goto return_failed', then a proper 
> error message appears:
> 
>      error: The following untracked working tree files would be 
> overwritten by checkout:
>          one/file/listed/here.txt
>      Please move or remove them before you can switch branches.
>      Aborting
> 
> The thing is, there are multiple files that would be overwritten by 
> checkout, and I believe an older version of Git showed me the entire 
> list before aborting.
> 
> What would be the proper fix here?  What am I doing wrong?
Can you provide a simple test case, such as this one:

---%<--- #!/bin/sh

rm -Rf utest || exit 1 mkdir utest || exit 1 cd utest || exit 1 git init echo tracked >a git add a git commit -m a a git branch side echo tracked >b git add b git commit -m b cat b git checkout side cat b echo untracked >b cat b git checkout master cat b ---%<---

With 1.7.6 and current next this gives (Linux):
Initialized empty Git repository in /tmp/t/utest/.git/
[master (root-commit) b462c80] a
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 a
[master 22d8f2f] b
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 b
tracked
Switched to branch 'side'
cat: b: No such file or directory
untracked
error: The following untracked working tree files would be overwritten
by checkout:
        b
Please move or remove them before you can switch branches.
Aborting
untracked
Does this test reproduce your problem on msysgit?
Michael
Joshua Jensen· Sep 20, 2011, 18:26 UTC · re: Michael J Gruber · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

----- Original Message -----
From: Michael J Gruber
Date: 9/20/2011 9:10 AM
Show 26 quoted lines
> Can you provide a simple test case, such as this one:
>
> ---%<---
> #!/bin/sh
>
> rm -Rf utest || exit 1
> mkdir utest || exit 1
> cd utest || exit 1
> git init
> echo tracked>a
> git add a
> git commit -m a a
> git branch side
> echo tracked>b
> git add b
> git commit -m b
> cat b
> git checkout side
> cat b
> echo untracked>b
> cat b
> git checkout master
> cat b
> ---%<---
>
> Does this test reproduce your problem on msysgit?
As listed, your script produces the same results on msysGit.
It appears the issue is related to sparse checkouts:

---%<--- #!/bin/sh

rm -Rf utest || exit 1 mkdir utest || exit 1 cd utest || exit 1 git init git config core.sparseCheckout true echo *>.git/info/sparse-checkout echo tracked>a git add a git commit -m a a git branch side echo tracked>b git add b git commit -m b cat b git checkout side cat b echo untracked>b cat b git checkout master cat b ---%<---

-Josh
Michael J Gruber· Sep 21, 2011, 07:47 UTC · re: Joshua Jensen · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

Joshua Jensen venit, vidit, dixit 20.09.2011 20:26:
Show 59 quoted lines
> ----- Original Message -----
> From: Michael J Gruber
> Date: 9/20/2011 9:10 AM
>> Can you provide a simple test case, such as this one:
>>
>> ---%<---
>> #!/bin/sh
>>
>> rm -Rf utest || exit 1
>> mkdir utest || exit 1
>> cd utest || exit 1
>> git init
>> echo tracked>a
>> git add a
>> git commit -m a a
>> git branch side
>> echo tracked>b
>> git add b
>> git commit -m b
>> cat b
>> git checkout side
>> cat b
>> echo untracked>b
>> cat b
>> git checkout master
>> cat b
>> ---%<---
>>
>> Does this test reproduce your problem on msysgit?
> As listed, your script produces the same results on msysGit.
> 
> It appears the issue is related to sparse checkouts:
> 
> ---%<---
> #!/bin/sh
> 
> rm -Rf utest || exit 1
> mkdir utest || exit 1
> cd utest || exit 1
> git init
> git config core.sparseCheckout true
> echo *>.git/info/sparse-checkout
> echo tracked>a
> git add a
> git commit -m a a
> git branch side
> echo tracked>b
> git add b
> git commit -m b
> cat b
> git checkout side
> cat b
> echo untracked>b
> cat b
> git checkout master
> cat b
> ---%<---
> 
> -Josh

Interesting. So, it is related to sparse, it seems, and there is also a mistake in the documentation (which contradicts itself, btw). With your script, since all files are requested to be checked out ('*'), none should have skip-worktree set, and the result should be the same as with my version. But I get this:

Initialized empty Git repository in /tmp/t/utest/.git/
[master (root-commit) 5929c8b] a
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 a
[master 60ad69e] b
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 b
tracked
Switched to branch 'side'
cat: b: No such file or directory
untracked
untracked

That is probably the same output as for you. The final "git checkout master" errored out without doing anything and without message. I'll send a few cleanups now, and do some bisecting and thinking later; though there are people whose sparse-knowledge is way less sparse than mine ;)

Michael
Michael J Gruber· Sep 21, 2011, 07:48 UTC · re: Michael J Gruber · lore

[PATCH 0/3] read-tree cleanups

These are a few cleanups I noticed while checking Joshua's sparse checkout problem when swithcing branches with a dirty sparse tree. They are independent of a possible fix.

Michael J Gruber (3):
  unpack-trees: print "Aborting" to stderr
  git-read-tree.txt: language and typography fixes
  git-read-tree.txt: correct sparse-checkout and skip-worktree
    description
 Documentation/git-read-tree.txt |   48 +++++++++++++++++++-------------------
 unpack-trees.c                  |    2 +-
 2 files changed, 25 insertions(+), 25 deletions(-)
-- 
1.7.7.rc0.469.g9eb94
Michael J Gruber· Sep 21, 2011, 07:51 UTC · re: Michael J Gruber · lore

Re: [PATCH 0/3] read-tree cleanups

Michael J Gruber venit, vidit, dixit 21.09.2011 09:48:
Show 14 quoted lines
> These are a few cleanups I noticed while checking Joshua's sparse checkout
> problem when swithcing branches with a dirty sparse tree.
> They are independent of a possible fix.
> 
> Michael J Gruber (3):
>   unpack-trees: print "Aborting" to stderr
>   git-read-tree.txt: language and typography fixes
>   git-read-tree.txt: correct sparse-checkout and skip-worktree
>     description
> 
>  Documentation/git-read-tree.txt |   48 +++++++++++++++++++-------------------
>  unpack-trees.c                  |    2 +-
>  2 files changed, 25 insertions(+), 25 deletions(-)
>

Uhm, I forgot about the dont-cc-jc-in-rc policy, sorry. This is not a late-minute regression fix and can wait.

Michael
Junio C Hamano· Sep 21, 2011, 12:14 UTC · re: Michael J Gruber · lore

Re: [PATCH 0/3] read-tree cleanups

Michael J Gruber <git@drmicha.warpmail.net> writes:
> Uhm, I forgot about the dont-cc-jc-in-rc policy, sorry. This is not a
> late-minute regression fix and can wait.

Thanks but that is not a policy; merely my preference. It also is a bit unnecessarily stronger than my actual preferences, which is

 (1) I do not want to see a patch sent "To:" me as the maintainer the
     first round of a new, undiscussed patch, as if it is ready for
     inclusion, at any time, not just during rc.
 (2) I do appreciate a patch sent "Cc:" me as a member of the development
     community patches for discussion, especially to areas in which I am
     an area expert, at any time, including during rc.
Michael J Gruber· Sep 21, 2011, 07:48 UTC · re: Michael J Gruber · lore

[PATCH 1/3] unpack-trees: print "Aborting" to stderr

display_error_msgs() prints all the errors to stderr already (if any), followed by "Aborting" (if any) to stdout. Make the latter go to stderr instead.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 unpack-trees.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/unpack-trees.c b/unpack-trees.c
index 670b464..237aed8 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -159,7 +159,7 @@ static void display_error_msgs(struct unpack_trees_options *o)
 		string_list_clear(rejects, 0);
 	}
 	if (something_displayed)
-		printf("Aborting\n");
+		fprintf(stderr, "Aborting\n");
 }
 
 /*

-- 
1.7.7.rc0.469.g9eb94
Junio C Hamano· Sep 21, 2011, 22:40 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/3] unpack-trees: print "Aborting" to stderr

Junio C Hamano <gitster@pobox.com> writes:
> Forgot to update a test or two that this breaks?
In the meantime I've squashed this in.
 t/t7607-merge-overwrite.sh     |    1 +
 t/t7609-merge-co-error-msgs.sh |    5 +++++
 2 files changed, 6 insertions(+), 0 deletions(-)
diff --git a/t/t7607-merge-overwrite.sh b/t/t7607-merge-overwrite.sh
index 72a8731..aa74184 100755
--- a/t/t7607-merge-overwrite.sh
+++ b/t/t7607-merge-overwrite.sh
@@ -107,6 +107,7 @@ error: The following untracked working tree files would be overwritten by merge:
 	sub
 	sub2
 Please move or remove them before you can merge.
+Aborting
 EOF
 
 test_expect_success 'will not overwrite untracked file in leading path' '
diff --git a/t/t7609-merge-co-error-msgs.sh b/t/t7609-merge-co-error-msgs.sh
index c994836..0e4a682 100755
--- a/t/t7609-merge-co-error-msgs.sh
+++ b/t/t7609-merge-co-error-msgs.sh
@@ -32,6 +32,7 @@ error: The following untracked working tree files would be overwritten by merge:
 	three
 	two
 Please move or remove them before you can merge.
+Aborting
 EOF
 
 test_expect_success 'untracked files overwritten by merge (fast and non-fast forward)' '
@@ -56,6 +57,7 @@ Please, commit your changes or stash them before you can merge.
 error: The following untracked working tree files would be overwritten by merge:
 	five
 Please move or remove them before you can merge.
+Aborting
 EOF
 
 test_expect_success 'untracked files or local changes ovewritten by merge' '
@@ -71,6 +73,7 @@ error: Your local changes to the following files would be overwritten by checkou
 	rep/one
 	rep/two
 Please, commit your changes or stash them before you can switch branches.
+Aborting
 EOF
 
 test_expect_success 'cannot switch branches because of local changes' '
@@ -92,6 +95,7 @@ error: Your local changes to the following files would be overwritten by checkou
 	rep/one
 	rep/two
 Please, commit your changes or stash them before you can switch branches.
+Aborting
 EOF
 
 test_expect_success 'not uptodate file porcelain checkout error' '
@@ -105,6 +109,7 @@ error: Updating the following directories would lose untracked files in it:
 	rep
 	rep2
 
+Aborting
 EOF
 
 test_expect_success 'not_uptodate_dir porcelain checkout error' '
Michael J Gruber· Sep 22, 2011, 05:58 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/3] unpack-trees: print "Aborting" to stderr

Junio C Hamano venit, vidit, dixit 22.09.2011 00:40:
Show 5 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> Forgot to update a test or two that this breaks?
> 
> In the meantime I've squashed this in.
Yikes. Sorry and thanks!
Show 61 quoted lines
> 
>  t/t7607-merge-overwrite.sh     |    1 +
>  t/t7609-merge-co-error-msgs.sh |    5 +++++
>  2 files changed, 6 insertions(+), 0 deletions(-)
> 
> diff --git a/t/t7607-merge-overwrite.sh b/t/t7607-merge-overwrite.sh
> index 72a8731..aa74184 100755
> --- a/t/t7607-merge-overwrite.sh
> +++ b/t/t7607-merge-overwrite.sh
> @@ -107,6 +107,7 @@ error: The following untracked working tree files would be overwritten by merge:
>  	sub
>  	sub2
>  Please move or remove them before you can merge.
> +Aborting
>  EOF
>  
>  test_expect_success 'will not overwrite untracked file in leading path' '
> diff --git a/t/t7609-merge-co-error-msgs.sh b/t/t7609-merge-co-error-msgs.sh
> index c994836..0e4a682 100755
> --- a/t/t7609-merge-co-error-msgs.sh
> +++ b/t/t7609-merge-co-error-msgs.sh
> @@ -32,6 +32,7 @@ error: The following untracked working tree files would be overwritten by merge:
>  	three
>  	two
>  Please move or remove them before you can merge.
> +Aborting
>  EOF
>  
>  test_expect_success 'untracked files overwritten by merge (fast and non-fast forward)' '
> @@ -56,6 +57,7 @@ Please, commit your changes or stash them before you can merge.
>  error: The following untracked working tree files would be overwritten by merge:
>  	five
>  Please move or remove them before you can merge.
> +Aborting
>  EOF
>  
>  test_expect_success 'untracked files or local changes ovewritten by merge' '
> @@ -71,6 +73,7 @@ error: Your local changes to the following files would be overwritten by checkou
>  	rep/one
>  	rep/two
>  Please, commit your changes or stash them before you can switch branches.
> +Aborting
>  EOF
>  
>  test_expect_success 'cannot switch branches because of local changes' '
> @@ -92,6 +95,7 @@ error: Your local changes to the following files would be overwritten by checkou
>  	rep/one
>  	rep/two
>  Please, commit your changes or stash them before you can switch branches.
> +Aborting
>  EOF
>  
>  test_expect_success 'not uptodate file porcelain checkout error' '
> @@ -105,6 +109,7 @@ error: Updating the following directories would lose untracked files in it:
>  	rep
>  	rep2
>  
> +Aborting
>  EOF
>  
>  test_expect_success 'not_uptodate_dir porcelain checkout error' '
Michael J Gruber· Sep 21, 2011, 07:48 UTC · re: Michael J Gruber · lore

[PATCH 2/3] git-read-tree.txt: language and typography fixes

Fix a few missing articles and such, and mark-up 'commands' and `files` appropriately.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 Documentation/git-read-tree.txt |   40 +++++++++++++++++++-------------------
 1 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index c45d53c..0004f4b 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -47,7 +47,7 @@ OPTIONS
 
 -i::
 	Usually a merge requires the index file as well as the
-	files in the working tree are up to date with the
+	files in the working tree to be up to date with the
 	current head commit, in order not to lose local
 	changes.  This flag disables the check with the working
 	tree and is meant to be used when creating a merge of
@@ -71,21 +71,21 @@ OPTIONS
 --aggressive::
 	Usually a three-way merge by 'git read-tree' resolves
 	the merge for really trivial cases and leaves other
-	cases unresolved in the index, so that Porcelains can
+	cases unresolved in the index, so that porcelains can
 	implement different merge policies.  This flag makes the
-	command to resolve a few more cases internally:
+	command resolve a few more cases internally:
 +
 * when one side removes a path and the other side leaves the path
   unmodified.  The resolution is to remove that path.
 * when both sides remove a path.  The resolution is to remove that path.
-* when both sides adds a path identically.  The resolution
+* when both sides add a path identically.  The resolution
   is to add that path.
 
 --prefix=<prefix>/::
 	Keep the current index contents, and read the contents
-	of named tree-ish under directory at `<prefix>`.  The
+	of the named tree-ish under the directory at `<prefix>`. The
 	original index file cannot have anything at the path
-	`<prefix>` itself, and have nothing in `<prefix>/`
+	`<prefix>` itself, nor anything in the `<prefix>/`
 	directory.  Note that the `<prefix>/` value must end
 	with a slash.
 
@@ -379,15 +379,15 @@ have finished your work-in-progress), attempt the merge again.
 Sparse checkout
 ---------------
 
-"Sparse checkout" allows to sparsely populate working directory.
-It uses skip-worktree bit (see linkgit:git-update-index[1]) to tell
-Git whether a file on working directory is worth looking at.
+"Sparse checkout" allows populating the working directory sparsely.
+It uses the skip-worktree bit (see linkgit:git-update-index[1]) to tell
+Git whether a file in the working directory is worth looking at.
 
-"git read-tree" and other merge-based commands ("git merge", "git
-checkout"...) can help maintaining skip-worktree bitmap and working
+'git read-tree' and other merge-based commands ('git merge', 'git
+checkout'...) can help maintaining the skip-worktree bitmap and working
 directory update. `$GIT_DIR/info/sparse-checkout` is used to
-define the skip-worktree reference bitmap. When "git read-tree" needs
-to update working directory, it will reset skip-worktree bit in index
+define the skip-worktree reference bitmap. When 'git read-tree' needs
+to update the working directory, it resets the skip-worktree bit in the index
 based on this file, which uses the same syntax as .gitignore files.
 If an entry matches a pattern in this file, skip-worktree will be
 set on that entry. Otherwise, skip-worktree will be unset.
@@ -397,18 +397,18 @@ skip-worktree turns from unset to set, it will add the corresponding
 file back. If it turns from set to unset, that file will be removed.
 
 While `$GIT_DIR/info/sparse-checkout` is usually used to specify what
-files are in. You can also specify what files are _not_ in, using
-negate patterns. For example, to remove file "unwanted":
+files are in, you can also specify what files are _not_ in, using
+negate patterns. For example, to remove the file `unwanted`:
 
 ----------------
 *
 !unwanted
 ----------------
 
-Another tricky thing is fully repopulating working directory when you
+Another tricky thing is fully repopulating the working directory when you
 no longer want sparse checkout. You cannot just disable "sparse
-checkout" because skip-worktree are still in the index and you working
-directory is still sparsely populated. You should re-populate working
+checkout" because skip-worktree bits are still in the index and your working
+directory is still sparsely populated. You should re-populate the working
 directory with the `$GIT_DIR/info/sparse-checkout` file content as
 follows:
 
@@ -416,8 +416,8 @@ follows:
 *
 ----------------
 
-Then you can disable sparse checkout. Sparse checkout support in "git
-read-tree" and similar commands is disabled by default. You need to
+Then you can disable sparse checkout. Sparse checkout support in 'git
+read-tree' and similar commands is disabled by default. You need to
 turn `core.sparseCheckout` on in order to have sparse checkout
 support.
 

-- 
1.7.7.rc0.469.g9eb94
Michael J Gruber· Sep 21, 2011, 07:48 UTC · re: Michael J Gruber · lore

[PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

The description of .git/info/sparse-checkout and skip-worktree is exactly the opposite of what is true, which is:

If a file matches a pattern in sparse-checkout, then (it is to be checked out and therefore) skip-worktree is unset for that file; otherwise, it is set (so that it is not checked out).

Currently, the opposite is documented, and (consistently) read-tree's behavior with respect to bit flips is descibed incorrectly.

Fix it.

In hindsight, it would have been much better to have a "sparse-ignore" or "sparse-skip" file so that an empty file would mean a full checkout, and the file logic would be analogous to that of .gitignore, excludes and skip-worktree.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 Documentation/git-read-tree.txt |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index 0004f4b..1bd0317 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -389,12 +389,12 @@ directory update. `$GIT_DIR/info/sparse-checkout` is used to
 define the skip-worktree reference bitmap. When 'git read-tree' needs
 to update the working directory, it resets the skip-worktree bit in the index
 based on this file, which uses the same syntax as .gitignore files.
-If an entry matches a pattern in this file, skip-worktree will be
-set on that entry. Otherwise, skip-worktree will be unset.
+If an entry matches a pattern in this file, skip-worktree will not be
+set on that entry. Otherwise, skip-worktree will be set.
 
 Then it compares the new skip-worktree value with the previous one. If
-skip-worktree turns from unset to set, it will add the corresponding
-file back. If it turns from set to unset, that file will be removed.
+skip-worktree turns from set to unset, it will add the corresponding
+file back. If it turns from unset to set, that file will be removed.
 
 While `$GIT_DIR/info/sparse-checkout` is usually used to specify what
 files are in, you can also specify what files are _not_ in, using

-- 
1.7.7.rc0.469.g9eb94
Nguyen Thai Ngoc Duy· Sep 21, 2011, 09:26 UTC · re: Michael J Gruber · lore

Re: [PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

On Wed, Sep 21, 2011 at 5:48 PM, Michael J Gruber <git@drmicha.warpmail.net> wrote:

Show 11 quoted lines
> The description of .git/info/sparse-checkout and
> skip-worktree is exactly the opposite of what is true, which is:
>
> If a file matches a pattern in sparse-checkout, then (it is to be
> checked out and therefore) skip-worktree is unset for that file;
> otherwise, it is set (so that it is not checked out).
>
> Currently, the opposite is documented, and (consistently) read-tree's
> behavior with respect to bit flips is descibed incorrectly.
>
> Fix it.
Ack.
> In hindsight, it would have been much better to have a "sparse-ignore"
> or "sparse-skip" file so that an empty file would mean a full checkout,
> and the file logic would be analogous to that of .gitignore, excludes
> and skip-worktree.

.gitignore works towards excluding files. No rule means no excluding. sparse-checkout file works towards including files, no rule means no inclusion.

-- 
Duy
Michael J Gruber· Sep 21, 2011, 09:33 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: [PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

Nguyen Thai Ngoc Duy venit, vidit, dixit 21.09.2011 11:26:
Show 24 quoted lines
> On Wed, Sep 21, 2011 at 5:48 PM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> The description of .git/info/sparse-checkout and
>> skip-worktree is exactly the opposite of what is true, which is:
>>
>> If a file matches a pattern in sparse-checkout, then (it is to be
>> checked out and therefore) skip-worktree is unset for that file;
>> otherwise, it is set (so that it is not checked out).
>>
>> Currently, the opposite is documented, and (consistently) read-tree's
>> behavior with respect to bit flips is descibed incorrectly.
>>
>> Fix it.
> 
> Ack.
> 
>> In hindsight, it would have been much better to have a "sparse-ignore"
>> or "sparse-skip" file so that an empty file would mean a full checkout,
>> and the file logic would be analogous to that of .gitignore, excludes
>> and skip-worktree.
> 
> .gitignore works towards excluding files. No rule means no excluding.
> sparse-checkout file works towards including files, no rule means no
> inclusion.

Sure, but with a "sparse-skip" rather than "sparse-checkout", we would not even need an additional config variable, and the skip-worktree centered explanations would follow the same logic (no need for the additional negation) as the ignore files and the new sparse-skip file. Also, I'm not sure whether more sparse users say "I want only that subdir." than "I don't want that subdir."

But it's there to stay, of course.

Cheers, Michael

Nguyen Thai Ngoc Duy· Sep 21, 2011, 09:39 UTC · re: Michael J Gruber · lore

Re: [PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

On Wed, Sep 21, 2011 at 7:33 PM, Michael J Gruber <git@drmicha.warpmail.net> wrote:

Show 13 quoted lines
>>> In hindsight, it would have been much better to have a "sparse-ignore"
>>> or "sparse-skip" file so that an empty file would mean a full checkout,
>>> and the file logic would be analogous to that of .gitignore, excludes
>>> and skip-worktree.
>>
>> .gitignore works towards excluding files. No rule means no excluding.
>> sparse-checkout file works towards including files, no rule means no
>> inclusion.
>
> Sure, but with a "sparse-skip" rather than "sparse-checkout", we would
> not even need an additional config variable, and the skip-worktree
> centered explanations would follow the same logic (no need for the
> additional negation) as the ignore files and the new sparse-skip file.

I'll kill that config variable some day when sparse checkout code has no overhead over normal case ("include all").

-- 
Duy
Michael J Gruber· Sep 21, 2011, 08:28 UTC · re: Michael J Gruber · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

Michael J Gruber venit, vidit, dixit 21.09.2011 09:47:
Show 24 quoted lines
> Interesting. So, it is related to sparse, it seems, and there is also a
> mistake in the documentation (which contradicts itself, btw). With your
> script, since all files are requested to be checked out ('*'), none
> should have skip-worktree set, and the result should be the same as with
> my version. But I get this:
> 
> Initialized empty Git repository in /tmp/t/utest/.git/
> [master (root-commit) 5929c8b] a
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 a
> [master 60ad69e] b
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 b
> tracked
> Switched to branch 'side'
> cat: b: No such file or directory
> untracked
> untracked
> 
> That is probably the same output as for you. The final "git checkout
> master" errored out without doing anything and without message. I'll
> send a few cleanups now, and do some bisecting and thinking later;
> though there are people whose sparse-knowledge is way less sparse than
> mine ;)
So, I bisected it. The first bad commit is

9037026 (unpack-trees: fix sparse checkout's "unable to match directories", 2010-11-27)

although the real culprit may be its predecessor

2431afb (unpack-trees: move all skip-worktree checks back to unpack_trees(), 2010-11-27)

which does not compile:
    CC unpack-trees.o
unpack-trees.c: In function 'mark_new_skip_worktree':
unpack-trees.c:852:75: error: 'o' undeclared (first use in this function)
unpack-trees.c:852:75: note: each undeclared identifier is reported only
once for each function it appears in
make: *** [unpack-trees.o] Error 1

So, this problem was introduced after v1.7.3.2. Given the extent of those two commits, I can't help further (but your proposed fix may be fine).

Duy, sorry for prodding you again. My bisect script was (based on Joshua's, based on mine):

#!/bin/sh GIT_EXEC_DIR=~/src/git PATH=$GIT_EXEC_DIR:$PATH export GIT_EXEC_DIR d=/tmp/t/utest rm -Rf $d || exit 1 mkdir $d || exit 1 cd $d || exit 1 git init git config core.sparseCheckout true echo *>.git/info/sparse-checkout echo tracked>a git add a git commit -m a a git branch side echo tracked>b git add b git commit -m b git checkout side echo untracked>b git checkout master 2>&1 | grep "would be"

Hope this helps.
Michael
Nguyen Thai Ngoc Duy· Sep 21, 2011, 08:58 UTC · re: Michael J Gruber · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

On Wed, Sep 21, 2011 at 10:28:59AM +0200, Michael J Gruber wrote:
Show 18 quoted lines
> So, I bisected it. The first bad commit is
> 
> 9037026 (unpack-trees: fix sparse checkout's "unable to match
> directories", 2010-11-27)
> 
> although the real culprit may be its predecessor
> 
> 2431afb (unpack-trees: move all skip-worktree checks back to
> unpack_trees(), 2010-11-27)
> 
> which does not compile:
> 
>     CC unpack-trees.o
> unpack-trees.c: In function 'mark_new_skip_worktree':
> unpack-trees.c:852:75: error: 'o' undeclared (first use in this function)
> unpack-trees.c:852:75: note: each undeclared identifier is reported only
> once for each function it appears in
> make: *** [unpack-trees.o] Error 1
This may help
--8<--
diff --git a/unpack-trees.c b/unpack-trees.c
index a6518db..a239af7 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -245,13 +245,13 @@ static int check_updates(struct unpack_trees_options *o)
 static int verify_uptodate_sparse(struct cache_entry *ce, struct unpack_trees_options *o);
 static int verify_absent_sparse(struct cache_entry *ce, enum unpack_trees_error_types, struct unpack_trees_options *o);
 
-static int will_have_skip_worktree(const struct cache_entry *ce, struct unpack_trees_options *o)
+static int will_have_skip_worktree(const struct cache_entry *ce, struct exclude_list *el)
 {
 	const char *basename;
 
 	basename = strrchr(ce->name, '/');
 	basename = basename ? basename+1 : ce->name;
-	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, o->el) <= 0;
+	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, el) <= 0;
 }

 static int apply_sparse_checkout(struct cache_entry *ce, struct unpack_trees_options *o)
@@ -849,7 +849,7 @@ static void mark_new_skip_worktree(struct exclude_list *el,
 		if (select_flag && !(ce->ce_flags & select_flag))
 			continue;
 
-		if (!ce_stage(ce) && will_have_skip_worktree(ce, o))
+		if (!ce_stage(ce) && will_have_skip_worktree(ce, el))
 			ce->ce_flags |= skip_wt_flag;
 		else
 			ce->ce_flags &= ~skip_wt_flag;
--8<--

> Duy, sorry for prodding you again.

No problem (and sorry for breaking the build). I'll also have a look
at this problem.
Michael J Gruber· Sep 21, 2011, 09:16 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

Nguyen Thai Ngoc Duy venit, vidit, dixit 21.09.2011 10:58:
Show 58 quoted lines
> On Wed, Sep 21, 2011 at 10:28:59AM +0200, Michael J Gruber wrote:
>> So, I bisected it. The first bad commit is
>>
>> 9037026 (unpack-trees: fix sparse checkout's "unable to match
>> directories", 2010-11-27)
>>
>> although the real culprit may be its predecessor
>>
>> 2431afb (unpack-trees: move all skip-worktree checks back to
>> unpack_trees(), 2010-11-27)
>>
>> which does not compile:
>>
>>     CC unpack-trees.o
>> unpack-trees.c: In function 'mark_new_skip_worktree':
>> unpack-trees.c:852:75: error: 'o' undeclared (first use in this function)
>> unpack-trees.c:852:75: note: each undeclared identifier is reported only
>> once for each function it appears in
>> make: *** [unpack-trees.o] Error 1
> 
> This may help
> 
> --8<--
> diff --git a/unpack-trees.c b/unpack-trees.c
> index a6518db..a239af7 100644
> --- a/unpack-trees.c
> +++ b/unpack-trees.c
> @@ -245,13 +245,13 @@ static int check_updates(struct unpack_trees_options *o)
>  static int verify_uptodate_sparse(struct cache_entry *ce, struct unpack_trees_options *o);
>  static int verify_absent_sparse(struct cache_entry *ce, enum unpack_trees_error_types, struct unpack_trees_options *o);
>  
> -static int will_have_skip_worktree(const struct cache_entry *ce, struct unpack_trees_options *o)
> +static int will_have_skip_worktree(const struct cache_entry *ce, struct exclude_list *el)
>  {
>  	const char *basename;
>  
>  	basename = strrchr(ce->name, '/');
>  	basename = basename ? basename+1 : ce->name;
> -	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, o->el) <= 0;
> +	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, el) <= 0;
>  }
> 
>  static int apply_sparse_checkout(struct cache_entry *ce, struct unpack_trees_options *o)
> @@ -849,7 +849,7 @@ static void mark_new_skip_worktree(struct exclude_list *el,
>  		if (select_flag && !(ce->ce_flags & select_flag))
>  			continue;
>  
> -		if (!ce_stage(ce) && will_have_skip_worktree(ce, o))
> +		if (!ce_stage(ce) && will_have_skip_worktree(ce, el))
>  			ce->ce_flags |= skip_wt_flag;
>  		else
>  			ce->ce_flags &= ~skip_wt_flag;
> --8<--
> 
>> Duy, sorry for prodding you again.
> 
> No problem (and sorry for breaking the build). I'll also have a look
> at this problem.

Thanks. I can confirm that with the above patch, the code compiles and fails my test. So it's the earlier of the two commits which introduces this.

Michael
Nguyen Thai Ngoc Duy· Sep 21, 2011, 10:28 UTC · re: Michael J Gruber · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

On Wed, Sep 21, 2011 at 11:16:15AM +0200, Michael J Gruber wrote:
> Thanks. I can confirm that with the above patch, the code compiles and
> fails my test. So it's the earlier of the two commits which introduces this.

I was afraid some logic had gone horribly wrong. Turns out I did not catch up with unpack_trees() coding style and drop the error messages so "git checkout" in this case becomes "git checkout -q".

This patch should fix it. Need another look before I submit a real patch though.

diff --git a/unpack-trees.c b/unpack-trees.c
index cc616c3..79e9e88 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1102,12 +1102,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
 			 */
 			if (ce->ce_flags & CE_ADDED &&
 			    verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
-					return -1;
+				goto return_failed;
 
-			if (apply_sparse_checkout(ce, o)) {
-				ret = -1;
-				goto done;
-			}
+			if (apply_sparse_checkout(ce, o))
+				goto return_failed;
 			if (!ce_skip_worktree(ce))
 				empty_worktree = 0;
Nguyen Thai Ngoc Duy· Sep 21, 2011, 10:31 UTC · re: Nguyen Thai Ngoc Duy · lore

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

On Wed, Sep 21, 2011 at 8:28 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> I was afraid some logic had gone horribly wrong. Turns out I did not
> catch up with unpack_trees() coding style and drop the error messages
> so "git checkout" in this case becomes "git checkout -q".

Gaah.. if I did read Joshua's problem description, it would not need to worry about the logic. He said it returned error code correctly. Silly me.

-- 
Duy

← back to recent threads