{"thread":{"id":"14664","subject":"[PATCH] Teach fsck and prune about the new location of temporary objects","startedAt":"2008-07-24T22:41:12Z","lastAt":"2008-07-27T04:15:23Z","messageCount":5,"participants":["Brandon Casey","Johannes Schindelin","Junio C Hamano","Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"84801","messageId":"6ruv3Y98_kSSVnJFTkV0PNdiNcQ3g-a3M4BhGoT7S1PorElp5tJAkw@cipher.nrlssc.navy.mil","threadId":"14664","inReplyTo":null,"subject":"[PATCH] Teach fsck and prune about the new location of temporary objects","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-07-24T22:41:12Z","receivedAt":"2008-07-24T22:41:12Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Since 5723fe7e, temporary objects are now created in their final destination\ndirectories, rather than in .git/objects/. Teach fsck to recognize and\nignore the temporary objects it encounters, and teach prune to remove them.\n\nSigned-off-by: Brandon Casey \n---\n builtin-fsck.c | 4 ++++\n builtin-prune.c | 41 ++++++++++++++++++++++++-----------------\n 2 files changed, 28 insertions(+), 17 deletions(-)\n\ndiff --git a/builtin-fsck.c b/builtin-fsck.c\nindex 7326dc3..7a4a4f1 100644\n--- a/builtin-fsck.c\n+++ b/builtin-fsck.c\n@@ -377,6 +377,10 @@ static void fsck_dir(int i, char *path)\n \t\t\tif (de->d_name[0] != '.')\n \t\t\t\tbreak;\n \t\t\tcontinue;\n+\t\tcase 14:\n+\t\t\tif (prefixcmp(de->d_name, \"tmp_obj_\"))\n+\t\t\t\tbreak;\n+\t\t\tcontinue;\n \t\tcase 38:\n \t\t\tsprintf(name, \"%02x\", i);\n \t\t\tmemcpy(name+2, de->d_name, len+1);\ndiff --git a/builtin-prune.c b/builtin-prune.c\nindex 7de4cab..947de8c 100644\n--- a/builtin-prune.c\n+++ b/builtin-prune.c\n@@ -13,6 +13,22 @@ static const char * const prune_usage[] = {\n static int show_only;\n static unsigned long expire;\n \n+static int prune_tmp_object(char *path, const char *filename)\n+{\n+\tconst char *fullpath = mkpath(\"%s/%s\", path, filename);\n+\tif (expire) {\n+\t\tstruct stat st;\n+\t\tif (lstat(fullpath, &st))\n+\t\t\treturn error(\"Could not stat '%s'\", fullpath);\n+\t\tif (st.st_mtime > expire)\n+\t\t\treturn 0;\n+\t}\n+\tprintf(\"Removing stale temporary file %s\\n\", fullpath);\n+\tif (!show_only)\n+\t\tunlink(fullpath);\n+\treturn 0;\n+}\n+\n static int prune_object(char *path, const char *filename, const unsigned char *sha1)\n {\n \tconst char *fullpath = mkpath(\"%s/%s\", path, filename);\n@@ -53,6 +69,11 @@ static int prune_dir(int i, char *path)\n \t\t\tif (de->d_name[0] != '.')\n \t\t\t\tbreak;\n \t\t\tcontinue;\n+\t\tcase 14:\n+\t\t\tif (prefixcmp(de->d_name, \"tmp_obj_\"))\n+\t\t\t\tbreak;\n+\t\t\tprune_tmp_object(path, de->d_name);\n+\t\t\tcontinue;\n \t\tcase 38:\n \t\t\tsprintf(name, \"%02x\", i);\n \t\t\tmemcpy(name+2, de->d_name, len+1);\n@@ -105,23 +126,9 @@ static void remove_temporary_files(void)\n \t\t\tdirname);\n \t\treturn;\n \t}\n-\twhile ((de = readdir(dir)) != NULL) {\n-\t\tif (!prefixcmp(de->d_name, \"tmp_\")) {\n-\t\t\tchar name[PATH_MAX];\n-\t\t\tint c = snprintf(name, PATH_MAX, \"%s/%s\",\n-\t\t\t\t\t dirname, de->d_name);\n-\t\t\tif (c < 0 || c >= PATH_MAX)\n-\t\t\t\tcontinue;\n-\t\t\tif (expire) {\n-\t\t\t\tstruct stat st;\n-\t\t\t\tif (stat(name, &st) != 0 || st.st_mtime >= expire)\n-\t\t\t\t\tcontinue;\n-\t\t\t}\n-\t\t\tprintf(\"Removing stale temporary file %s\\n\", name);\n-\t\t\tif (!show_only)\n-\t\t\t\tunlink(name);\n-\t\t}\n-\t}\n+\twhile ((de = readdir(dir)) != NULL)\n+\t\tif (!prefixcmp(de->d_name, \"tmp_\"))\n+\t\t\tprune_tmp_object(dirname, de->d_name);\n \tclosedir(dir);\n }\n \n-- \n1.6.0.rc0.55.g160ae.dirty\n"},{"id":"84816","messageId":"alpine.DEB.1.00.0807250233031.4140@eeepc-johanness","threadId":"14664","inReplyTo":"6ruv3Y98_kSSVnJFTkV0PNdiNcQ3g-a3M4BhGoT7S1PorElp5tJAkw@cipher.nrlssc.navy.mil","subject":"Re: [PATCH] Teach fsck and prune about the new location of temporary objects","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-25T00:33:52Z","receivedAt":"2008-07-25T00:33:52Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 24 Jul 2008, Brandon Casey wrote:\n\n> Since 5723fe7e, temporary objects are now created in their final destination\n> directories, rather than in .git/objects/. Teach fsck to recognize and\n> ignore the temporary objects it encounters,\n\nIt somehow feels wrong for fsck to ignore anything that could be used.\n\nCiao,\nDscho\n"},{"id":"84836","messageId":"7vd4l2xzrd.fsf@gitster.siamese.dyndns.org","threadId":"14664","inReplyTo":"alpine.DEB.1.00.0807250233031.4140@eeepc-johanness","subject":"Re: [PATCH] Teach fsck and prune about the new location of temporary objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-25T06:07:50Z","receivedAt":"2008-07-25T06:07:50Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> On Thu, 24 Jul 2008, Brandon Casey wrote:\n>\n>> Since 5723fe7e, temporary objects are now created in their final destination\n>> directories, rather than in .git/objects/. Teach fsck to recognize and\n>> ignore the temporary objects it encounters,\n>\n> It somehow feels wrong for fsck to ignore anything that could be used.\n\nBack when we created temporaries in .git/objects/, these tmp_obj files\nwere not even checked nor looked at, because fsck_object_dir() checked\nonly those 256 fan-out directories, so this is nothing new.\n\nAnd these tmp_obj files are something we _expect_ to be in the object\ndirectory. The user can ^C out object creation codepath anytime, and we\ndo not reference the final destination objects from refs nor the index to\npreserve the integrity of the repository when that happens. In other\nwords, they are crufts, but they are not something unusual nor get alarmed\nabout.\n\nAnd it also is the right thing to do to remove them in prune.\n"},{"id":"85133","messageId":"20080727023300.GB17425@spearce.org","threadId":"14664","inReplyTo":"6ruv3Y98_kSSVnJFTkV0PNdiNcQ3g-a3M4BhGoT7S1PorElp5tJAkw@cipher.nrlssc.navy.mil","subject":"[PATCH] fsck: Don't require tmp_obj_ file names are 14 bytes in length","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-07-27T02:33:00Z","receivedAt":"2008-07-27T02:33:00Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Not all temporary file creation routines will ensure 14 bytes are\nused to generate the temporary file name. In C Git this may be\ntrue, but alternate implementations such as jgit are not always\nable to generate a temporary file name with a specific prefix and\nalso ensure the file name length is 14 bytes long.\n\nSince temporary files in a directory we are fsck'ing should be\nuncommon (as they are short lived only long enough for an active\nwriter to finish writing the file and rename it) we shouldn't see\nthese show up very often. Always using a prefixcmp() call and\nignoring the length opens up room for other implementations to use\ndifferent name generation schemes.\n\nSigned-off-by: Shawn O. Pearce \n---\n\n Brandon Casey wrote:\n > Since 5723fe7e, temporary objects are now created in their final destination\n > directories, rather than in .git/objects/. Teach fsck to recognize and\n > ignore the temporary objects it encounters, and teach prune to remove them.\n\n jgit can't exactly follow the 14 character naming rule.\n This should be a safe way around it.\n\n builtin-fsck.c | 6 ++----\n 1 files changed, 2 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin-fsck.c b/builtin-fsck.c\nindex 7a4a4f1..6eb7da8 100644\n--- a/builtin-fsck.c\n+++ b/builtin-fsck.c\n@@ -377,10 +377,6 @@ static void fsck_dir(int i, char *path)\n \t\t\tif (de->d_name[0] != '.')\n \t\t\t\tbreak;\n \t\t\tcontinue;\n-\t\tcase 14:\n-\t\t\tif (prefixcmp(de->d_name, \"tmp_obj_\"))\n-\t\t\t\tbreak;\n-\t\t\tcontinue;\n \t\tcase 38:\n \t\t\tsprintf(name, \"%02x\", i);\n \t\t\tmemcpy(name+2, de->d_name, len+1);\n@@ -389,6 +385,8 @@ static void fsck_dir(int i, char *path)\n \t\t\tadd_sha1_list(sha1, DIRENT_SORT_HINT(de));\n \t\t\tcontinue;\n \t\t}\n+\t\tif (prefixcmp(de->d_name, \"tmp_obj_\"))\n+\t\t\tcontinue;\n \t\tfprintf(stderr, \"bad sha1 file: %s/%s\\n\", path, de->d_name);\n \t}\n \tclosedir(dir);\n-- \n1.6.0.rc0.182.gb96c7\n\n-- \nShawn.\n"},{"id":"85139","messageId":"7vwsj83quc.fsf@gitster.siamese.dyndns.org","threadId":"14664","inReplyTo":"20080727023300.GB17425@spearce.org","subject":"Re: [PATCH] fsck: Don't require tmp_obj_ file names are 14 bytes in length","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-27T04:15:23Z","receivedAt":"2008-07-27T04:15:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Shawn O. Pearce\" writes:\n\n> Brandon Casey wrote:\n> > Since 5723fe7e, temporary objects are now created in their final destination\n> > directories, rather than in .git/objects/. Teach fsck to recognize and\n> > ignore the temporary objects it encounters, and teach prune to remove them.\n>\n> jgit can't exactly follow the 14 character naming rule.\n> This should be a safe way around it.\n\nMore importantly, we might want to change the temporary filename to ensure\nthat it is 38 bytes in length in the future (cf. $gmane/85106).\n"}]}