{"thread":{"id":"57601","subject":"[RFC PATCH 0/1] invalid outdated temporary files","startedAt":"2022-03-23T06:33:23Z","lastAt":"2022-03-23T20:36:28Z","messageCount":3,"participants":["Han Xin","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":1},"messages":[{"id":"451950","messageId":"cover.1648007488.git.hanxin.hx@alibaba-inc.com","threadId":"57601","inReplyTo":null,"subject":"[RFC PATCH 0/1] invalid outdated temporary files","fromName":"Han Xin","fromEmail":"chiyutianyi@gmail.com","sentAt":"2022-03-23T06:33:12Z","receivedAt":"2022-03-23T06:33:23Z","isPatch":true,"sender":{"key":"chiyutianyi@gmail.com","avatar":null},"body":"From: Han Xin \n\nWhen I use git on network storage, there may be some temporary files left in\ngit repositories due to the program quitting unexpectedly, and they also\nprevent me from continuing to use the git command line until I manually\ndelete them.\n\nFor some obviously expired temporary files, manual deletion is an extra\nexpensive job, and one that I think can be avoided. So I wrote this patch.\nWhen creating a temporary file with the same name, if the mtime of the target\nfile exceeds one day, unlink it first, and then try to recreate it.\n\nHan Xin (1):\n tempfile: invalid outdated temporary files\n\n Documentation/config/core.txt | 6 ++++++\n cache.h | 1 +\n config.c | 5 +++++\n environment.c | 1 +\n t/t3210-pack-refs.sh | 10 ++++++++++\n tempfile.c | 21 +++++++++++++++++++++\n 6 files changed, 44 insertions(+)\n\n-- \n2.35.1.55.gde4365fc52.agit.6.5.7\n\n"},{"id":"451951","messageId":"61af2576cd94b1958584da2b231aa75ecffb3cb4.1648007488.git.hanxin.hx@alibaba-inc.com","threadId":"57601","inReplyTo":"cover.1648007488.git.hanxin.hx@alibaba-inc.com","subject":"[RFC PATCH 1/1] tempfile: invalid outdated temporary files","fromName":"Han Xin","fromEmail":"chiyutianyi@gmail.com","sentAt":"2022-03-23T06:33:13Z","receivedAt":"2022-03-23T06:33:27Z","isPatch":true,"sender":{"key":"chiyutianyi@gmail.com","avatar":null},"body":"From: Han Xin \n\nWhen git quits unexpectedly, some temporary files(e.g. HEAD.lock,\npacked-refs.lock, packed-refs.new) may remain in the repository. These\nfiles will prevent us from performing the corresponding operations\nagain, even if they were created a long time ago before, until we\nmanually remove them.\n\nIn order for git to automatically fix this situation, let's add a config\nnamed \"core.tempfileExpire\". When an attempt is made to create a\ntemporary file that exists and is older than this config value, the file\nwill be unlinked and recreated.\n\nSigned-off-by: Han Xin \n---\n Documentation/config/core.txt | 6 ++++++\n cache.h | 1 +\n config.c | 5 +++++\n environment.c | 1 +\n t/t3210-pack-refs.sh | 10 ++++++++++\n tempfile.c | 21 +++++++++++++++++++++\n 6 files changed, 44 insertions(+)\n\ndiff --git a/Documentation/config/core.txt b/Documentation/config/core.txt\nindex c04f62a54a..9907c9e6f3 100644\n--- a/Documentation/config/core.txt\n+++ b/Documentation/config/core.txt\n@@ -424,6 +424,12 @@ be delta compressed, but larger binary media files won't be.\n +\n Common unit suffixes of 'k', 'm', or 'g' are supported.\n \n+core.tempfileExpire::\n+\tWhen an attempt is made to create a temporary file that exists\n+\tand is older than this config value, the file will be unlinked\n+\tand recreated. This feature is used to ignore temporary files\n+\t(e.g. *.lock and *.new) remaining due to abnormal exits.\n+\n core.excludesFile::\n \tSpecifies the pathname to the file that contains patterns to\n \tdescribe paths that are not meant to be tracked, in addition\ndiff --git a/cache.h b/cache.h\nindex 0bc0a37cec..4264fe7960 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -972,6 +972,7 @@ extern size_t packed_git_window_size;\n extern size_t packed_git_limit;\n extern size_t delta_base_cache_limit;\n extern unsigned long big_file_threshold;\n+extern const char *tempfile_expire;\n extern unsigned long pack_size_limit_cfg;\n \n /*\ndiff --git a/config.c b/config.c\nindex e78397725c..36a5bc1960 100644\n--- a/config.c\n+++ b/config.c\n@@ -1518,6 +1518,11 @@ static int git_default_core_config(const char *var, const char *value, void *cb)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(var, \"core.tempfileexpire\")) {\n+\t\tgit_config_get_expiry(var, &tempfile_expire);\n+\t\treturn 0;\n+\t}\n+\n \tif (!strcmp(var, \"core.packedgitlimit\")) {\n \t\tpacked_git_limit = git_config_ulong(var, value);\n \t\treturn 0;\ndiff --git a/environment.c b/environment.c\nindex fb55bf6129..20b928e71b 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -48,6 +48,7 @@ size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE;\n size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT;\n size_t delta_base_cache_limit = 96 * 1024 * 1024;\n unsigned long big_file_threshold = 512 * 1024 * 1024;\n+const char *tempfile_expire = \"1.day.ago\";\n int pager_use_color = 1;\n const char *editor_program;\n const char *askpass_program;\ndiff --git a/t/t3210-pack-refs.sh b/t/t3210-pack-refs.sh\nindex 577f32dc71..7d443c1cfd 100755\n--- a/t/t3210-pack-refs.sh\n+++ b/t/t3210-pack-refs.sh\n@@ -231,6 +231,16 @@ test_expect_success 'timeout if packed-refs.lock exists' '\n \ttest_must_fail git pack-refs --all --prune\n '\n \n+test_expect_success 'success if packed-refs.lock expires' '\n+\tLOCK=.git/packed-refs.lock &&\n+\t>\"$LOCK\" &&\n+\ttest_when_finished \"rm -f $LOCK\" &&\n+\ttest-tool chmtime -86000 $LOCK &&\n+\ttest_must_fail git pack-refs --all --prune &&\n+\ttest-tool chmtime -400 $LOCK &&\n+\tgit pack-refs --all --prune\n+'\n+\n test_expect_success 'retry acquiring packed-refs.lock' '\n \tLOCK=.git/packed-refs.lock &&\n \t>\"$LOCK\" &&\ndiff --git a/tempfile.c b/tempfile.c\nindex 94aa18f3f7..f18e0121d0 100644\n--- a/tempfile.c\n+++ b/tempfile.c\n@@ -51,6 +51,7 @@\n */\n \n #include \"cache.h\"\n+#include \"date.h\"\n #include \"tempfile.h\"\n #include \"sigchain.h\"\n \n@@ -137,6 +138,26 @@ struct tempfile *create_tempfile_mode(const char *path, int mode)\n \tstrbuf_add_absolute_path(&tempfile->filename, path);\n \ttempfile->fd = open(tempfile->filename.buf,\n \t\t\t O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, mode);\n+\tif (tempfile->fd < 0 && errno == EEXIST) {\n+\t\tstruct stat st;\n+\t\tint err = errno;\n+\t\tif (lstat(tempfile->filename.buf, &st) < 0) {\n+\t\t\twarning_errno(_(\"failed to stat %s\"),\n+\t\t\t\t tempfile->filename.buf);\n+\t\t\terrno = err;\n+\t\t} else if (st.st_mtime <= approxidate(tempfile_expire)) {\n+\t\t\t/*\n+\t\t\t * If the file is older than core.tmpfileExpire, it is\n+\t\t\t * probably outdated. We'll try to remove it and try\n+\t\t\t * again.\n+\t\t\t */\n+\t\t\twarning(_(\"remove outdated tempfile %s\"), tempfile->filename.buf);\n+\t\t\tunlink_or_warn(tempfile->filename.buf);\n+\t\t\ttempfile->fd = open(tempfile->filename.buf,\n+\t\t\t\t\t O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC,\n+\t\t\t\t\t mode);\n+\t\t}\n+\t}\n \tif (O_CLOEXEC && tempfile->fd < 0 && errno == EINVAL)\n \t\t/* Try again w/o O_CLOEXEC: the kernel might not support it */\n \t\ttempfile->fd = open(tempfile->filename.buf,\n-- \n2.35.1.55.gde4365fc52.agit.6.5.7\n\n"},{"id":"452059","messageId":"xmqq5yo4s8xp.fsf@gitster.g","threadId":"57601","inReplyTo":"61af2576cd94b1958584da2b231aa75ecffb3cb4.1648007488.git.hanxin.hx@alibaba-inc.com","subject":"Re: [RFC PATCH 1/1] tempfile: invalid outdated temporary files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-23T20:36:18Z","receivedAt":"2022-03-23T20:36:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Han Xin writes:\n\n> From: Han Xin \n>\n> When git quits unexpectedly, some temporary files(e.g. HEAD.lock,\n> packed-refs.lock, packed-refs.new) may remain in the repository. These\n> files will prevent us from performing the corresponding operations\n> again, even if they were created a long time ago before, until we\n> manually remove them.\n>\n> In order for git to automatically fix this situation, let's add a config\n\nI am not sure if we want to automatically \"fix\" this situation in the\nfirst place.\n\nUnless the user is sure of the reason why these lock files are left\nbehind (e.g. \"ah, I saw a power failure last night while I was\ntrying to commit\"), we are looking at a potential data loss.\n"}]}