threads / discuss / 22439

v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Subject: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

## tl;dr

13 messages between Jan 29, 2010 and Jan 31, 2010.

replies: 12people: 3as markdown or json

Jonathan del Strother· Jan 29, 2010, 15:12 UTC · lore
Heya,
git-stash in v1.7.0-rc0 gives me several hundred lines of :

error: unable to find dd4a345a770f5c82a668932b7289f42b1ae38103 error: unable to find dd4a345a770f5c82a668932b7289f42b1ae38103 error: unable to find dd4a345a770f5c82a668932b7289f42b1ae38103 .... error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0 error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0 error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0 error: unable to find 0dc67edc22017de22161dde93baa615767b9c6e0 .... error: unable to find 980f084e71097df3d4086aaab574972321df80f2 error: unable to find 980f084e71097df3d4086aaab574972321df80f2 error: unable to find 980f084e71097df3d4086aaab574972321df80f2 error: unable to find 980f084e71097df3d4086aaab574972321df80f2

Other than that, it appears to work fine.  'git show <sha1>' works too
- they're all a bunch of file contents, with no particular pattern.

This only happens in my main (private) repo. I can't seem to come up with a reproducible example for anyone else...

The previous build I was using - 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show this problem. Want me to try and bisect further?

-Jonathan
Jonathan del Strother· Jan 30, 2010, 00:46 UTC · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

On 29 January 2010 17:44, Junio C Hamano <gitster@pobox.com> wrote:
Show 11 quoted lines
> Jonathan del Strother <maillist@steelskies.com> writes:
>
>> Heya,
>>
>> git-stash in v1.7.0-rc0 gives me several hundred lines of :
>> The previous build I was using -
>> 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
>> this problem.  Want me to try and bisect further?
>
> Yes, please.
>

ee6fc514f2df821c2719cc49499a56ef2fb136b0 (Show submodules as modified when they contain a dirty work tree) seems to be the first bad commit.

Jens Lehmann· Jan 30, 2010, 12:31 UTC · re: Jonathan del Strother · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Am 30.01.2010 01:46, schrieb Jonathan del Strother:
Show 15 quoted lines
> On 29 January 2010 17:44, Junio C Hamano <gitster@pobox.com> wrote:
>> Jonathan del Strother <maillist@steelskies.com> writes:
>>
>>> Heya,
>>>
>>> git-stash in v1.7.0-rc0 gives me several hundred lines of :
>>> The previous build I was using -
>>> 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
>>> this problem.  Want me to try and bisect further?
>>
>> Yes, please.
>>
> 
> ee6fc514f2df821c2719cc49499a56ef2fb136b0 (Show submodules as modified
> when they contain a dirty work tree) seems to be the first bad commit.
Thanks for pinning that down.

I assume you have one or more submodules, maybe even with untracked or yet uncommitted modified files in your tree? If so, what does git status say in the superproject and in the submodule(s)?

(If you want to keep some information off the list, you can mail me privately)

Jonathan del Strother· Jan 30, 2010, 13:23 UTC · re: Jens Lehmann · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
Show 23 quoted lines
> Am 30.01.2010 01:46, schrieb Jonathan del Strother:
>> On 29 January 2010 17:44, Junio C Hamano <gitster@pobox.com> wrote:
>>> Jonathan del Strother <maillist@steelskies.com> writes:
>>>
>>>> Heya,
>>>>
>>>> git-stash in v1.7.0-rc0 gives me several hundred lines of :
>>>> The previous build I was using -
>>>> 5b15950ac414a8a2d4f5eb480712abcc9fe176d2 from Jan 19th - didn't show
>>>> this problem.  Want me to try and bisect further?
>>>
>>> Yes, please.
>>>
>>
>> ee6fc514f2df821c2719cc49499a56ef2fb136b0 (Show submodules as modified
>> when they contain a dirty work tree) seems to be the first bad commit.
>
> Thanks for pinning that down.
>
> I assume you have one or more submodules, maybe even with untracked
> or yet uncommitted modified files in your tree? If so, what does git
> status say in the superproject and in the submodule(s)?
>

Yep, I have 10 submodules. However, they're all completely clean with no tracked or untracked changes shown in git status. Anything else I can investigate?

Jens Lehmann· Jan 30, 2010, 14:10 UTC · re: Jonathan del Strother · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Am 30.01.2010 14:23, schrieb Jonathan del Strother:
Show 8 quoted lines
> On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>> I assume you have one or more submodules, maybe even with untracked
>> or yet uncommitted modified files in your tree? If so, what does git
>> status say in the superproject and in the submodule(s)?
> 
> Yep, I have 10 submodules.  However, they're all completely clean with
> no tracked or untracked changes shown in git status.  Anything else I
> can investigate?
The change in behavior my patch introduced is that "git status" is
called inside each submodule. So i would expect getting the same
errors when using this command:
   git submodule foreach git status -s
It should just show
   Entering '<submodule 1>'
   Entering '<submodule 2>'
   Entering '<submodule 3>'
   Entering '<submodule 4>'
   Entering '<submodule 5>'
   Entering '<submodule 6>'
   Entering '<submodule 7>'
   Entering '<submodule 8>'
   Entering '<submodule 9>'
   Entering '<submodule 10>'
when the submodules are not dirty. What do you get?

The error message "error: unable to find <sha1>" is only generated in sha1_loose_object_info() when map_sha1_file() fails. Its single caller being sha1_object_info(), which is called by many git commands. Does that ring a bell for somebody?

Jonathan del Strother· Jan 30, 2010, 16:36 UTC · re: Jens Lehmann · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

On 30 January 2010 14:10, Jens Lehmann <Jens.Lehmann@web.de> wrote:
Show 28 quoted lines
> Am 30.01.2010 14:23, schrieb Jonathan del Strother:
>> On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>>> I assume you have one or more submodules, maybe even with untracked
>>> or yet uncommitted modified files in your tree? If so, what does git
>>> status say in the superproject and in the submodule(s)?
>>
>> Yep, I have 10 submodules.  However, they're all completely clean with
>> no tracked or untracked changes shown in git status.  Anything else I
>> can investigate?
>
> The change in behavior my patch introduced is that "git status" is
> called inside each submodule. So i would expect getting the same
> errors when using this command:
>   git submodule foreach git status -s
>
> It should just show
>   Entering '<submodule 1>'
>   Entering '<submodule 2>'
>   Entering '<submodule 3>'
>   Entering '<submodule 4>'
>   Entering '<submodule 5>'
>   Entering '<submodule 6>'
>   Entering '<submodule 7>'
>   Entering '<submodule 8>'
>   Entering '<submodule 9>'
>   Entering '<submodule 10>'
> when the submodules are not dirty. What do you get?
>
Correct - I just get that output.
Junio C Hamano· Jan 30, 2010, 20:01 UTC · re: Jonathan del Strother · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Please try this.

The thing is that "add -u" internally runs "diff-files" equivalent, but the code to check the submodule freshness doesn't reset the GIT_INDEX_FILE exported from the caller and tries to use that even when checking the submodule repository.

diff --git a/submodule.c b/submodule.c
index ca0527f..8bd0a30 100644
--- a/submodule.c
+++ b/submodule.c
@@ -126,7 +126,7 @@ int is_submodule_modified(const char *path)
 		"--porcelain",
 		NULL,
 	};
-	char *env[3];
+	char *env[4];
 	struct strbuf buf = STRBUF_INIT;
 
 	strbuf_addf(&buf, "%s/.git/", path);
@@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
 	env[0] = strbuf_detach(&buf, NULL);
 	strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
 	env[1] = strbuf_detach(&buf, NULL);
-	env[2] = NULL;
+	strbuf_addf(&buf, "GIT_INDEX_FILE=");
+	env[2] = strbuf_detach(&buf, NULL);
+	env[3] = NULL;
 
 	memset(&cp, 0, sizeof(cp));
 	cp.argv = argv;
@@ -161,6 +163,7 @@ int is_submodule_modified(const char *path)
 
 	free(env[0]);
 	free(env[1]);
+	free(env[2]);
 	strbuf_release(&buf);
 	return len != 0;
 }
Junio C Hamano· Jan 30, 2010, 20:25 UTC · re: Junio C Hamano · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Junio C Hamano <gitster@pobox.com> writes:
> Please try this.
oops, but with this, too.
Show 11 quoted lines
> diff --git a/submodule.c b/submodule.c
> index ca0527f..8bd0a30 100644
> --- a/submodule.c
> +++ b/submodule.c
> ...
> @@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
>  	env[0] = strbuf_detach(&buf, NULL);
>  	strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
>  	env[1] = strbuf_detach(&buf, NULL);
> -	env[2] = NULL;
> +	strbuf_addf(&buf, "GIT_INDEX_FILE=");
This should be:
	strbuf_addf(&buf, "GIT_INDEX_FILE");
Show 5 quoted lines
> +	env[2] = strbuf_detach(&buf, NULL);
> +	env[3] = NULL;
>  
>  	memset(&cp, 0, sizeof(cp));
>  	cp.argv = argv;
Junio C Hamano· Jan 30, 2010, 20:30 UTC · re: Junio C Hamano · lore

[PATCH] is_submodule_modified(): fix breakage with external GIT_INDEX_FILE

Even when the environment was given for the top-level process, checking in the submodule work tree should use the index file associated with the work tree of the submodule. Do not export it to the environment.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 * This seems to pass all the tests, so I'm inclined to push it out as
   a part of 1.7.0-rc1.  There might be some other enviroment variables
   we shouldn't be leaking out to the submodule environment, but that can
   be spotted and fixed in later patches before the final.
 submodule.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/submodule.c b/submodule.c
index ca0527f..6f7c210 100644
--- a/submodule.c
+++ b/submodule.c
@@ -126,7 +126,7 @@ int is_submodule_modified(const char *path)
 		"--porcelain",
 		NULL,
 	};
-	char *env[3];
+	char *env[4];
 	struct strbuf buf = STRBUF_INIT;
 
 	strbuf_addf(&buf, "%s/.git/", path);
@@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
 	env[0] = strbuf_detach(&buf, NULL);
 	strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
 	env[1] = strbuf_detach(&buf, NULL);
-	env[2] = NULL;
+	strbuf_addf(&buf, "GIT_INDEX_FILE");
+	env[2] = strbuf_detach(&buf, NULL);
+	env[3] = NULL;
 
 	memset(&cp, 0, sizeof(cp));
 	cp.argv = argv;
@@ -161,6 +163,7 @@ int is_submodule_modified(const char *path)
 
 	free(env[0]);
 	free(env[1]);
+	free(env[2]);
 	strbuf_release(&buf);
 	return len != 0;
 }

-- 
1.7.0.rc1.141.gbc8ce
Jonathan del Strother· Jan 31, 2010, 19:16 UTC · re: Junio C Hamano · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

On 30 January 2010 20:25, Junio C Hamano <gitster@pobox.com> wrote:
Show 28 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Please try this.
>
> oops, but with this, too.
>
>> diff --git a/submodule.c b/submodule.c
>> index ca0527f..8bd0a30 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> ...
>> @@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
>>       env[0] = strbuf_detach(&buf, NULL);
>>       strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
>>       env[1] = strbuf_detach(&buf, NULL);
>> -     env[2] = NULL;
>> +     strbuf_addf(&buf, "GIT_INDEX_FILE=");
>
> This should be:
>
>        strbuf_addf(&buf, "GIT_INDEX_FILE");
>
>> +     env[2] = strbuf_detach(&buf, NULL);
>> +     env[3] = NULL;
>>
>>       memset(&cp, 0, sizeof(cp));
>>       cp.argv = argv;
>

Sorry, I'm not sure I follow your amendment - it looks exactly like the patch you originally supplied?

With that original patch applied, I no longer get the 'unable to find' errors on stashing. However, git-status shows all my submodules as being modified, but there appear to be no local changes :

[jon@gir:Developer/AudioBooWeb]$ git status # On branch giterror # Changed but not updated: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: shared/vendor/plugins/acts_as_list # modified: shared/vendor/plugins/cucumber # modified: shared/vendor/plugins/delayed_job # modified: shared/vendor/plugins/haml # modified: shared/vendor/plugins/hoptoad_notifier # modified: shared/vendor/plugins/machinist # modified: shared/vendor/plugins/newrelic_rpm # modified: shared/vendor/plugins/rspec # modified: shared/vendor/plugins/rspec-rails # modified: shared/vendor/rails # no changes added to commit (use "git add" and/or "git commit -a")

[jon@gir:Developer/AudioBooWeb]$ git diff
diff --git a/shared/vendor/plugins/acts_as_list
b/shared/vendor/plugins/acts_as_list
diff --git a/shared/vendor/plugins/cucumber b/shared/vendor/plugins/cucumber
diff --git a/shared/vendor/plugins/delayed_job
b/shared/vendor/plugins/delayed_job
diff --git a/shared/vendor/plugins/haml b/shared/vendor/plugins/haml
diff --git a/shared/vendor/plugins/hoptoad_notifier
b/shared/vendor/plugins/hoptoad_notifier
diff --git a/shared/vendor/plugins/machinist b/shared/vendor/plugins/machinist
diff --git a/shared/vendor/plugins/newrelic_rpm
b/shared/vendor/plugins/newrelic_rpm
diff --git a/shared/vendor/plugins/rspec b/shared/vendor/plugins/rspec
diff --git a/shared/vendor/plugins/rspec-rails
b/shared/vendor/plugins/rspec-rails
diff --git a/shared/vendor/rails b/shared/vendor/rails
Jonathan del Strother· Jan 31, 2010, 19:21 UTC · re: Jonathan del Strother · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

On 31 January 2010 19:16, Jonathan del Strother <maillist@steelskies.com> wrote:
Show 73 quoted lines
> On 30 January 2010 20:25, Junio C Hamano <gitster@pobox.com> wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> Please try this.
>>
>> oops, but with this, too.
>>
>>> diff --git a/submodule.c b/submodule.c
>>> index ca0527f..8bd0a30 100644
>>> --- a/submodule.c
>>> +++ b/submodule.c
>>> ...
>>> @@ -142,7 +142,9 @@ int is_submodule_modified(const char *path)
>>>       env[0] = strbuf_detach(&buf, NULL);
>>>       strbuf_addf(&buf, "GIT_DIR=%s/.git", path);
>>>       env[1] = strbuf_detach(&buf, NULL);
>>> -     env[2] = NULL;
>>> +     strbuf_addf(&buf, "GIT_INDEX_FILE=");
>>
>> This should be:
>>
>>        strbuf_addf(&buf, "GIT_INDEX_FILE");
>>
>>> +     env[2] = strbuf_detach(&buf, NULL);
>>> +     env[3] = NULL;
>>>
>>>       memset(&cp, 0, sizeof(cp));
>>>       cp.argv = argv;
>>
>
> Sorry, I'm not sure I follow your amendment - it looks exactly like
> the patch you originally supplied?
>
> With that original patch applied, I no longer get the 'unable to find'
> errors on stashing.  However, git-status shows all my submodules as
> being modified, but there appear to be no local changes :
>
> [jon@gir:Developer/AudioBooWeb]$ git status
> # On branch giterror
> # Changed but not updated:
> #   (use "git add <file>..." to update what will be committed)
> #   (use "git checkout -- <file>..." to discard changes in working directory)
> #
> #       modified:   shared/vendor/plugins/acts_as_list
> #       modified:   shared/vendor/plugins/cucumber
> #       modified:   shared/vendor/plugins/delayed_job
> #       modified:   shared/vendor/plugins/haml
> #       modified:   shared/vendor/plugins/hoptoad_notifier
> #       modified:   shared/vendor/plugins/machinist
> #       modified:   shared/vendor/plugins/newrelic_rpm
> #       modified:   shared/vendor/plugins/rspec
> #       modified:   shared/vendor/plugins/rspec-rails
> #       modified:   shared/vendor/rails
> #
> no changes added to commit (use "git add" and/or "git commit -a")
>
> [jon@gir:Developer/AudioBooWeb]$ git diff
> diff --git a/shared/vendor/plugins/acts_as_list
> b/shared/vendor/plugins/acts_as_list
> diff --git a/shared/vendor/plugins/cucumber b/shared/vendor/plugins/cucumber
> diff --git a/shared/vendor/plugins/delayed_job
> b/shared/vendor/plugins/delayed_job
> diff --git a/shared/vendor/plugins/haml b/shared/vendor/plugins/haml
> diff --git a/shared/vendor/plugins/hoptoad_notifier
> b/shared/vendor/plugins/hoptoad_notifier
> diff --git a/shared/vendor/plugins/machinist b/shared/vendor/plugins/machinist
> diff --git a/shared/vendor/plugins/newrelic_rpm
> b/shared/vendor/plugins/newrelic_rpm
> diff --git a/shared/vendor/plugins/rspec b/shared/vendor/plugins/rspec
> diff --git a/shared/vendor/plugins/rspec-rails
> b/shared/vendor/plugins/rspec-rails
> diff --git a/shared/vendor/rails b/shared/vendor/rails
>

... however, the latest version of master (b0883aa6c77111e88496bd0afe073caf68ab9f99) gives me no errors, and (correctly) doesn't show my submodules as being modified. I still don't quite follow exactly what was wrong, but thanks :)

Junio C Hamano· Jan 31, 2010, 19:21 UTC · re: Jonathan del Strother · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Jonathan del Strother <maillist@steelskies.com> writes:
Show 8 quoted lines
>>> +     strbuf_addf(&buf, "GIT_INDEX_FILE=");
>>
>> This should be:
>>
>>        strbuf_addf(&buf, "GIT_INDEX_FILE");
>
> Sorry, I'm not sure I follow your amendment - it looks exactly like
> the patch you originally supplied?

Lack of the last "=" tells: "Instead of exporting GIT_INDEX_FILE with an empty string as its value, unexport it."

Jens Lehmann· Jan 30, 2010, 20:44 UTC · re: Jonathan del Strother · lore

Re: v1.7.0-rc0 shows lots of "unable to find <sha1>" on git-stash

Am 30.01.2010 17:36, schrieb Jonathan del Strother:
Show 31 quoted lines
> On 30 January 2010 14:10, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>> Am 30.01.2010 14:23, schrieb Jonathan del Strother:
>>> On 30 January 2010 12:31, Jens Lehmann <Jens.Lehmann@web.de> wrote:
>>>> I assume you have one or more submodules, maybe even with untracked
>>>> or yet uncommitted modified files in your tree? If so, what does git
>>>> status say in the superproject and in the submodule(s)?
>>>
>>> Yep, I have 10 submodules.  However, they're all completely clean with
>>> no tracked or untracked changes shown in git status.  Anything else I
>>> can investigate?
>>
>> The change in behavior my patch introduced is that "git status" is
>> called inside each submodule. So i would expect getting the same
>> errors when using this command:
>>   git submodule foreach git status -s
>>
>> It should just show
>>   Entering '<submodule 1>'
>>   Entering '<submodule 2>'
>>   Entering '<submodule 3>'
>>   Entering '<submodule 4>'
>>   Entering '<submodule 5>'
>>   Entering '<submodule 6>'
>>   Entering '<submodule 7>'
>>   Entering '<submodule 8>'
>>   Entering '<submodule 9>'
>>   Entering '<submodule 10>'
>> when the submodules are not dirty. What do you get?
>>
> 
> Correct - I just get that output.

Hm, so nothing unusual there. I really wonder what is the problem here, as calling "git status" inside the submodules works fine when issued via "git submodule foreach", but not when done via run_command()!?

So i would like to ask some more questions:
- Under what operating system and on what filesystem is this happening?
- Is there anything unusual about your repo (e.g. using GIT_WORK_TREE
  or having the object database somewhere else that in .git in the
  superproject or any of the submodules)?
- You are just issuing a "git stash" to stash some changes in the
  superproject when that happens, right?
- The hashes that show up as "unable to find" are reachable via "git
  show" in the superproject, not in the submodules, right?
- Do these hashes have any relation to the contents you are stashing?
- The following patch should suppress (but not solve) this problem when
  applied to a version of git that contains
  4d34477f4c5dbebc55aa1362fd705440590a85f1 (git diff: Don't test
  submodule dirtiness with --ignore-submodules), e.g. current next.
  Could you please verify that?
---8<---
diff --git a/git-stash.sh b/git-stash.sh
index 3a0685f..e9b47b4 100755
--- a/git-stash.sh
+++ b/git-stash.sh
@@ -105,7 +105,7 @@ create_stash () {
                w_tree=$(GIT_INDEX_FILE="$TMP-index" git write-tree) ||
                die "Cannot save the current worktree state"

-               git diff-tree -p HEAD $w_tree > "$TMP-patch" &&
+               git diff-tree --ignore-submodules -p HEAD $w_tree > "$TMP-patch"
                test -s "$TMP-patch" ||
                die "No changes selected"

← back to recent threads