{"thread":{"id":"416","subject":"Should git-prune-script warn about dircache?","startedAt":"2005-05-01T10:50:44Z","lastAt":"2005-05-01T21:29:53Z","messageCount":6,"participants":["Junio C Hamano","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"2289","messageId":"7vwtqjxlrv.fsf@assigned-by-dhcp.cox.net","threadId":"416","inReplyTo":null,"subject":"Should git-prune-script warn about dircache?","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-05-01T10:50:44Z","receivedAt":"2005-05-01T10:50:44Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"When the user has a change that is recorded in the dircache but\nthe change has not been committed, git-fsck-cache --unreachable\nwould report blobs and trees (if you have a subdirectory) that\nare involved.\n\n $ git-init-db\n defaulting to local storage area\n $ date >foo\n $ git-update-cache --add foo\n $ date | git-commit-tree $(git-write-tree) >.git/HEAD\n Committing initial tree 9061076c3ec10fef339c84f4e0c9e06576d3b7db\n $ date >foo\n $ git-update-cache foo\n $ git-fsck-cache --unreachable $(cat .git/HEAD)\n unreachable blob d8c61e975a591ad5905c7defc567a77fca58f092\n $ git-ls-files --stage\n 100644 d8c61e975a591ad5905c7defc567a77fca58f092 0 foo\n\nAt this point, if we run git-prune-script with the current head,\nwe would remove the blob, and after that, code that assumes the\nobjects pointed by SHA1s recorded in the dircache are locally\navailable start failing.\n\n $ git-prune-script\n $ touch foo\n $ git-diff-files -p\n .git/objects/d8/c61e975a591....: No such file or directory\n fatal: unable to read blob object for foo (d8c61e975a591....)\n\nI have a feeling somewhere in the chain from git-prune-script we\nshould have a mechanism to prevent us from losing such blob and\ntrees. I know it should not be in git-fsck-cache, but I have\nnot decided what my suggestions would be for us to do yet.\n\nMaybe a big warning in red ugly bold blinking typeface somewhere\nin the doc?\n\n"},{"id":"2297","messageId":"Pine.LNX.4.58.0505010916510.2296@ppc970.osdl.org","threadId":"416","inReplyTo":"7vwtqjxlrv.fsf@assigned-by-dhcp.cox.net","subject":"Re: Should git-prune-script warn about dircache?","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-05-01T16:18:07Z","receivedAt":"2005-05-01T16:18:07Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 1 May 2005, Junio C Hamano wrote:\n> \n> Maybe a big warning in red ugly bold blinking typeface somewhere\n> in the doc?\n\nHow about making git-prune-script first run \"git-update-cache --refresh\",\nand checking the return value of it (this, of course, assumes that\ngit-update-cache --refresh would return non-zero if it can't refresh a\nfile, which is currently not true, but should be easily fixable).\n\n\t\tLinus\n"},{"id":"2302","messageId":"7vll6yyiax.fsf@assigned-by-dhcp.cox.net","threadId":"416","inReplyTo":"Pine.LNX.4.58.0505010916510.2296@ppc970.osdl.org","subject":"Re: Should git-prune-script warn about dircache?","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-05-01T17:20:22Z","receivedAt":"2005-05-01T17:20:22Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"LT\" == Linus Torvalds writes:\n\nLT> How about making git-prune-script first run \"git-update-cache --refresh\",\nLT> and checking the return value of it (this, of course, assumes that\nLT> git-update-cache --refresh would return non-zero if it can't refresh a\nLT> file, which is currently not true, but should be easily fixable).\n\nOr just check if it sees anything in the output, especially\n\"needs update\" line.\n\nI do not think it is such a big deal [*1*] but I should point\nout that, \"git-update-cache --refresh\" needs to be run on all of\nthem if the user (or the porcelain layer) is using more than one\nGIT_INDEX_FILEs [*2*].\n\n\n[Footnotes]\n\n*1* Because git-prune-script is just an example and it already\nassumes it knows where the valid heads are; right now it looks\nonly at .git/HEAD and not .git/refs/*/*. Each Porcelain layer\nimplementation should provide its own prune script anyway.\n\n*2* I do not do this anymore but an earlier incarnation of my\nlittle SCM on GIT [*3*] allowed a user to keep snapshots of work\ntree state and switch between them by juggling multiple\nGIT_INDEX_FILE. I just create commits off of the current state\nwhen making a snapshot in the latest version so it is not a\nproblem anymore for me.\n\n*3* (PLUG) found in http://members.cox.net/junkio/.\n\n\n"},{"id":"2316","messageId":"7vwtqismpn.fsf@assigned-by-dhcp.cox.net","threadId":"416","inReplyTo":"7vll6yyiax.fsf@assigned-by-dhcp.cox.net","subject":"Re: Should git-prune-script warn about dircache?","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-05-01T20:41:40Z","receivedAt":"2005-05-01T20:41:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":">>>>> \"JCH\" == Junio C Hamano writes:\n\n>>>>> \"LT\" == Linus Torvalds writes:\nLT> How about making git-prune-script first run \"git-update-cache --refresh\",\nLT> and checking the return value of it (this, of course, assumes that\nLT> git-update-cache --refresh would return non-zero if it can't refresh a\nLT> file, which is currently not true, but should be easily fixable).\n\nJCH> Or just check if it sees anything in the output, especially\nJCH> \"needs update\" line.\n\nWell, we were both wrong. The problem is not about the work\ntree changes since the last git-update-cache, but about the\nblobs recorded in the cache but still not committed.\n\nI think we should do something like this.\n\n git-ls-files --cached | \"sed to SHA1 only\" | sort >,,1\n git-fsck-cache --unreachable | \"sed to SHA1 only\" | sort >,,2\n comm -13 ,,1 ,,2 | \"sed to .git/object/ path\" | xargs -r rm -f\n\n"},{"id":"2322","messageId":"7vk6miskl4.fsf_-_@assigned-by-dhcp.cox.net","threadId":"416","inReplyTo":"7vwtqismpn.fsf@assigned-by-dhcp.cox.net","subject":"[PATCH] Make git-prune-script a bit more careful.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-05-01T21:27:35Z","receivedAt":"2005-05-01T21:27:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"As discussed on the git list, this patch makes the sample script\ngit-prune-script a bit more careful by keeping the objects\nreferenced from the current cache from being reclaimed.\n\nAt the same time it also fixes and enhances the following:\n\n - Optional command line parameters can specify which commit\n heads to start reachablity test from. Earlier we had a\n hardcoded .git/HEAD. It now defaults to the contents of\n .git/HEAD and .git/refs/*/* files.\n\n - It does not assume SHA1_FILE_DIRECTORY is .git/objects/ but\n uses the value from the environment if there is one.\n\n - It runs xargs with \"-r\" option to prevent it from making \"rm\"\n command barf if there is nothing to remove.\n\nSigned-off-by: Junio C Hamano \n---\n\ngit-prune-script | 32 +++++++++++++++++++++++++++++++-\n1 files changed, 31 insertions(+), 1 deletion(-)\n\n# - date handling: handle \"AM\"/\"PM\" on time\n# + [PATCH] Make git-prune-script a bit more careful.\n--- k/git-prune-script\n+++ l/git-prune-script\n@@ -1,2 +1,32 @@\n #!/bin/sh\n-git-fsck-cache --unreachable $(cat .git/HEAD ) | grep unreachable | cut -d' ' -f3 | sed 's:^\\(..\\):.git/objects/\\1/:' | xargs rm\n+\n+tmp=.git-prune-script-$$\n+trap 'rm -f $tmp-*' 0 1 2 3 15 \n+\n+# Defaulting to include .git/refs/*/* may be debatable from the\n+# purist POV but power users can always give explicit parameters\n+# to the script anyway.\n+case \"$#\" in\n+0) set x $(cat .git/HEAD .git/refs/*/*); shift ;;\n+esac\n+\n+git-fsck-cache --unreachable \"$@\" |\n+sed -ne 's/unreachable [^ ][^ ]* //p' |\n+sort >$tmp-unreachable\n+\n+# This makes extra objects to be kept if the cache has an entry\n+# with an unusual name like \"this\\n0 0123...abcdef 0 file\", but\n+# we are trying not to discard information and keeping extra in\n+# an unusual situation would be OK.\n+git-ls-files --stage |\n+sed -ne 's|^[0-7][0-7]* \\([0-9a-f][0-9a-f]*\\) [0-3] .*|\\1|p' |\n+sort >$tmp-keep\n+\n+comm -23 $tmp-unreachable $tmp-keep |\n+sed -e 's|\\(..\\)|\\1/|' | {\n+\tcase \"$SHA1_FILE_DIRECTORY\" in\n+\t'') cd .git/objects/ ;;\n+\t*) cd \"$SHA1_FILE_DIRECTORY\" ;;\n+\tesac || exit\n+\txargs -r echo rm -f\n+}\n\n"},{"id":"2326","messageId":"7vbr7uskha.fsf_-_@assigned-by-dhcp.cox.net","threadId":"416","inReplyTo":"Pine.LNX.4.58.0505010916510.2296@ppc970.osdl.org","subject":"[PATCH] Make git-update-cache --refresh fail if update/merge needed.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-05-01T21:29:53Z","receivedAt":"2005-05-01T21:29:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Scripts may find it useful if they do not have to parse the\noutput from the command but just can rely on its exit status.\n\nEarlier both Linus and myself thought this would be necessary to\nmake git-prune-script safer but it turns out that the issue was\nsomewhere else and not related to what this patch addresses.\n\nSigned-off-by: Junio C Hamano \n---\n\nupdate-cache.c | 15 ++++++++++-----\n1 files changed, 10 insertions(+), 5 deletions(-)\n\n# - date handling: handle \"AM\"/\"PM\" on time\n# + [PATCH] Make git-prune-script a bit more careful.\n--- k/update-cache.c\n+++ l/update-cache.c\n@@ -212,15 +212,17 @@ static struct cache_entry *refresh_entry\n \treturn updated;\n }\n \n-static void refresh_cache(void)\n+static int refresh_cache(void)\n {\n \tint i;\n+\tint has_errors = 0;\n \n \tfor (i = 0; i < active_nr; i++) {\n \t\tstruct cache_entry *ce, *new;\n \t\tce = active_cache[i];\n \t\tif (ce_stage(ce)) {\n \t\t\tprintf(\"%s: needs merge\\n\", ce->name);\n+\t\t\thas_errors = 1;\n \t\t\twhile ((i < active_nr) &&\n \t\t\t ! strcmp(active_cache[i]->name, ce->name))\n \t\t\t\ti++;\n@@ -230,12 +232,15 @@ static void refresh_cache(void)\n \n \t\tnew = refresh_entry(ce);\n \t\tif (IS_ERR(new)) {\n-\t\t\tif (!(not_new && PTR_ERR(new) == -ENOENT))\n+\t\t\tif (!(not_new && PTR_ERR(new) == -ENOENT)) {\n \t\t\t\tprintf(\"%s: needs update\\n\", ce->name);\n+\t\t\t\thas_errors = 1;\n+\t\t\t}\n \t\t\tcontinue;\n \t\t}\n \t\tactive_cache[i] = new;\n \t}\n+\treturn has_errors;\n }\n \n /*\n@@ -307,7 +312,7 @@ static void remove_lock_file_on_signal(i\n \n int main(int argc, char **argv)\n {\n-\tint i, newfd, entries;\n+\tint i, newfd, entries, has_errors = 0;\n \tint allow_options = 1;\n \tstatic char lockfile[MAXPATHLEN+1];\n \tconst char *indexfile = get_index_file();\n@@ -343,7 +348,7 @@ int main(int argc, char **argv)\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tif (!strcmp(path, \"--refresh\")) {\n-\t\t\t\trefresh_cache();\n+\t\t\t\thas_errors |= refresh_cache();\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tif (!strcmp(path, \"--cacheinfo\")) {\n@@ -369,5 +374,5 @@ int main(int argc, char **argv)\n \t\tdie(\"Unable to write new cachefile\");\n \n \tlockfile_name = NULL;\n-\treturn 0;\n+\treturn has_errors;\n }\n\n"}]}