{"thread":{"id":"6429","subject":"[PATCH] prune-packed: new option --min-age=N","startedAt":"2007-01-18T17:18:30Z","lastAt":"2007-01-22T09:32:35Z","messageCount":30,"participants":["Matthias Lederhofer","Shawn O. Pearce","Junio C Hamano","Nicolas Pitre","Simon 'corecode' Schubert","Jeff King","Steven Grimm","Linus Torvalds","Jakub Narebski"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"32007","messageId":"20070118171830.GA13521@moooo.ath.cx","threadId":"6429","inReplyTo":null,"subject":"[PATCH] prune-packed: new option --min-age=N","fromName":"Matthias Lederhofer","fromEmail":"matled@gmx.net","sentAt":"2007-01-18T17:18:30Z","receivedAt":"2007-01-18T17:18:30Z","isPatch":true,"sender":{"key":"matled@gmx.net","avatar":null},"body":"This option makes prune-packed ignore all packs younger than N seconds\n(using mtime).\n\nSigned-off-by: Matthias Lederhofer \n---\nThis option allows to remove the unpacked version of an object only if\nit has been packed for a minimum time. It is intended to work around\nthe problem of freshly packed objects which should not be deleted\nbecause someone might still try to open the unpacked version.\n\nif [ $(git count-objects | cut -d ' ' -f 3) -gt 10240 ]; then\n git repack\nfi\ngit prune-packed --min-age=$((24*60*60))\n---\n builtin-prune-packed.c | 48 +++++++++++++++++++++++++++++++++++++++++++-----\n builtin-prune.c | 2 +-\n builtin.h | 2 +-\n 3 files changed, 45 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin-prune-packed.c b/builtin-prune-packed.c\nindex a57b76d..4359a41 100644\n--- a/builtin-prune-packed.c\n+++ b/builtin-prune-packed.c\n@@ -7,7 +7,8 @@ static const char prune_packed_usage[] =\n #define DRY_RUN 01\n #define VERBOSE 02\n \n-static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)\n+static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts,\n+\t\t char **ignore_packed)\n {\n \tstruct dirent *de;\n \tchar hex[40];\n@@ -20,7 +21,7 @@ static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)\n \t\tmemcpy(hex+2, de->d_name, 38);\n \t\tif (get_sha1_hex(hex, sha1))\n \t\t\tcontinue;\n-\t\tif (!has_sha1_pack(sha1, NULL))\n+\t\tif (!has_sha1_pack(sha1, ignore_packed))\n \t\t\tcontinue;\n \t\tmemcpy(pathname + len, de->d_name, 38);\n \t\tif (opts & DRY_RUN)\n@@ -32,12 +33,46 @@ static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)\n \trmdir(pathname);\n }\n \n-void prune_packed_objects(int opts)\n+static char **prune_ignore(int min_age, int opts)\n+{\n+\tint ignore_max = 10;\n+\tint ignore_count = 0;\n+\tchar **ignore_packed = xcalloc(ignore_max, sizeof(char*));\n+\tint now = time(NULL);\n+\tstruct packed_git *p;\n+\n+\tprepare_packed_git();\n+\tfor (p = packed_git; p; p = p->next) {\n+\t\tstruct stat s;\n+\t\tif (stat(p->pack_name, &s)) {\n+\t\t\twarn(\"unable to stat %s\", p->pack_name);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (now-s.st_mtime >= min_age)\n+\t\t\tcontinue;\n+\t\tif (opts & VERBOSE)\n+\t\t\tfprintf(stderr, \"ignoring pack %s\\n\", p->pack_name);\n+\t\tif (ignore_count >= ignore_max-1) {\n+\t\t\tignore_max += 10;\n+\t\t\tignore_packed = xrealloc(ignore_packed,\n+\t\t\t\tignore_max*sizeof(char*));\n+\t\t}\n+\t\tignore_packed[ignore_count++] = p->pack_name;\n+\t}\n+\tignore_packed[ignore_count] = NULL;\n+\treturn ignore_packed;\n+}\n+\n+void prune_packed_objects(int opts, int min_age)\n {\n \tint i;\n \tstatic char pathname[PATH_MAX];\n \tconst char *dir = get_object_directory();\n \tint len = strlen(dir);\n+\tchar **ignore_packed = NULL;\n+\n+\tif (min_age > 0)\n+\t\tignore_packed = prune_ignore(min_age, opts);\n \n \tif (len > PATH_MAX - 42)\n \t\tdie(\"impossible object directory\");\n@@ -54,7 +89,7 @@ void prune_packed_objects(int opts)\n \t\t\t\t((i+1) * 100) / 256);\n \t\tif (!d)\n \t\t\tcontinue;\n-\t\tprune_dir(i, d, pathname, len + 3, opts);\n+\t\tprune_dir(i, d, pathname, len + 3, opts, ignore_packed);\n \t\tclosedir(d);\n \t}\n \tif (opts == VERBOSE)\n@@ -65,6 +100,7 @@ int cmd_prune_packed(int argc, const char **argv, const char *prefix)\n {\n \tint i;\n \tint opts = VERBOSE;\n+\tint min_age = 0;\n \n \tfor (i = 1; i < argc; i++) {\n \t\tconst char *arg = argv[i];\n@@ -74,6 +110,8 @@ int cmd_prune_packed(int argc, const char **argv, const char *prefix)\n \t\t\t\topts |= DRY_RUN;\n \t\t\telse if (!strcmp(arg, \"-q\"))\n \t\t\t\topts &= ~VERBOSE;\n+\t\t\telse if (!strncmp(arg, \"--min-age=\", 10))\n+\t\t\t\tmin_age = atoi(arg+10);\n \t\t\telse\n \t\t\t\tusage(prune_packed_usage);\n \t\t\tcontinue;\n@@ -82,6 +120,6 @@ int cmd_prune_packed(int argc, const char **argv, const char *prefix)\n \t\tusage(prune_packed_usage);\n \t}\n \tsync();\n-\tprune_packed_objects(opts);\n+\tprune_packed_objects(opts, min_age);\n \treturn 0;\n }\ndiff --git a/builtin-prune.c b/builtin-prune.c\nindex 6f0ba0d..46a0fe5 100644\n--- a/builtin-prune.c\n+++ b/builtin-prune.c\n@@ -100,6 +100,6 @@ int cmd_prune(int argc, const char **argv, const char *prefix)\n \tprune_object_dir(get_object_directory());\n \n \tsync();\n-\tprune_packed_objects(show_only);\n+\tprune_packed_objects(show_only, 0);\n \treturn 0;\n }\ndiff --git a/builtin.h b/builtin.h\nindex 818c7bf..e9929da 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -11,7 +11,7 @@ extern int mailinfo(FILE *in, FILE *out, int ks, const char *encoding, const cha\n extern int split_mbox(const char **mbox, const char *dir, int allow_bare, int nr_prec, int skip);\n extern void stripspace(FILE *in, FILE *out);\n extern int write_tree(unsigned char *sha1, int missing_ok, const char *prefix);\n-extern void prune_packed_objects(int);\n+extern void prune_packed_objects(int, int);\n \n extern int cmd_add(int argc, const char **argv, const char *prefix);\n extern int cmd_annotate(int argc, const char **argv, const char *prefix);\n-- \n1.4.4.4\n"},{"id":"32008","messageId":"20070118172408.GG15428@spearce.org","threadId":"6429","inReplyTo":"20070118171830.GA13521@moooo.ath.cx","subject":"Re: [PATCH] prune-packed: new option --min-age=N","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-01-18T17:24:08Z","receivedAt":"2007-01-18T17:24:08Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Matthias Lederhofer wrote:\n> This option allows to remove the unpacked version of an object only if\n> it has been packed for a minimum time. It is intended to work around\n> the problem of freshly packed objects which should not be deleted\n> because someone might still try to open the unpacked version.\n\nAre we not rescanning for new packs if we fail to find an\nobject in the object directory? I thought we were doing this in\nread_sha1_file. *looks at code* Indeed, we are...\n\nIs this somehow not working?\n\n-- \nShawn.\n"},{"id":"32009","messageId":"20070118174244.GA14287@moooo.ath.cx","threadId":"6429","inReplyTo":"20070118172408.GG15428@spearce.org","subject":"Re: [PATCH] prune-packed: new option --min-age=N","fromName":"Matthias Lederhofer","fromEmail":"matled@gmx.net","sentAt":"2007-01-18T17:42:44Z","receivedAt":"2007-01-18T17:42:44Z","isPatch":true,"sender":{"key":"matled@gmx.net","avatar":null},"body":"Shawn O. Pearce wrote:\n> Matthias Lederhofer wrote:\n> > This option allows to remove the unpacked version of an object only if\n> > it has been packed for a minimum time. It is intended to work around\n> > the problem of freshly packed objects which should not be deleted\n> > because someone might still try to open the unpacked version.\n> \n> Are we not rescanning for new packs if we fail to find an\n> object in the object directory? I thought we were doing this in\n> read_sha1_file. *looks at code* Indeed, we are...\n> \n> Is this somehow not working?\nI just remembered people were warning about automatic prune-packed\n(e.g. cron), so this patch does not make much sense anymore.\n\nAny comments about adding such an option to git prune for unpacked\nobjects? This would allow to run git prune automatically on\nrepositories while new objects are created.\n"},{"id":"32010","messageId":"20070118175134.GH15428@spearce.org","threadId":"6429","inReplyTo":"20070118174244.GA14287@moooo.ath.cx","subject":"Re: [PATCH] prune-packed: new option --min-age=N","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-01-18T17:51:34Z","receivedAt":"2007-01-18T17:51:34Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Matthias Lederhofer wrote:\n> I just remembered people were warning about automatic prune-packed\n> (e.g. cron), so this patch does not make much sense anymore.\n\nI think we've resolved it. There may be some hole we aren't aware\nof, but I doubt one exists. The last hole was packs could be\nremoved by repack while they were still being brought in during\npush or 'fetch -k', but that was fixed with the .keep file.\n \n> Any comments about adding such an option to git prune for unpacked\n> objects? This would allow to run git prune automatically on\n> repositories while new objects are created.\n\nAn age makes sense on just plain prune. If the loose object was\nlast modified several hours ago and its not being referenced by\nanything currently, its probably safe to remove.\n\nIf you are going to implement this I would suggest making the default\nage 2 hours and allow the user to configure it from a gc.pruneexpire\nconfiguration option, much like gc.reflogexpire.\n\n-- \nShawn.\n"},{"id":"32025","messageId":"20070118222919.GA22060@moooo.ath.cx","threadId":"6429","inReplyTo":"20070118175134.GH15428@spearce.org","subject":"[RFC] prune: --expire=seconds","fromName":"Matthias Lederhofer","fromEmail":"matled@gmx.net","sentAt":"2007-01-18T22:29:19Z","receivedAt":"2007-01-18T22:29:19Z","isPatch":false,"sender":{"key":"matled@gmx.net","avatar":null},"body":"This option specifies the minimum age of an object before it\nmay be removed by prune. The default value is 2 hours and\nmay be changed using gc.pruneexpire.\n\nSigned-off-by: Matthias Lederhofer \n---\nShawn O. Pearce wrote:\n> If you are going to implement this I would suggest making the default\n> age 2 hours and allow the user to configure it from a gc.pruneexpire\n> configuration option, much like gc.reflogexpire.\n\nHere it is, I've set the default value to 2 hours as you suggested.\nAny other comments if the default should be a value >0 or 0 to keep\nthe old behaviour?\n---\n builtin-prune.c | 25 ++++++++++++++++++++++++-\n 1 files changed, 24 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-prune.c b/builtin-prune.c\nindex 6f0ba0d..f46892d 100644\n--- a/builtin-prune.c\n+++ b/builtin-prune.c\n@@ -5,8 +5,10 @@\n #include \"builtin.h\"\n #include \"reachable.h\"\n \n-static const char prune_usage[] = \"git-prune [-n]\";\n+static const char prune_usage[] = \"git-prune [-n] [--expire=seconds]\";\n static int show_only;\n+static int prune_expire;\n+static time_t now;\n \n static int prune_object(char *path, const char *filename, const unsigned char *sha1)\n {\n@@ -38,6 +40,7 @@ static int prune_dir(int i, char *path)\n \t\tchar name[100];\n \t\tunsigned char sha1[20];\n \t\tint len = strlen(de->d_name);\n+\t\tstruct stat st;\n \n \t\tswitch (len) {\n \t\tcase 2:\n@@ -60,6 +63,11 @@ static int prune_dir(int i, char *path)\n \t\t\tif (lookup_object(sha1))\n \t\t\t\tcontinue;\n \n+\t\t\tif (prune_expire > 0 &&\n+\t\t\t !stat(mkpath(\"%s/%s\", path, de->d_name), &st) &&\n+\t\t\t now-st.st_mtime < prune_expire)\n+\t\t\t\tcontinue;\n+\n \t\t\tprune_object(path, de->d_name, sha1);\n \t\t\tcontinue;\n \t\t}\n@@ -79,10 +87,21 @@ static void prune_object_dir(const char *path)\n \t}\n }\n \n+static void git_prune_config(const char *var, const char *value)\n+{\n+\tif (!strcmp(var, \"gc.pruneexpire\")) {\n+\t\tprune_expire = git_config_int(var, value);\n+\t\treturn 0;\n+\t}\n+\treturn git_default_config(var, value);\n+}\n+\n int cmd_prune(int argc, const char **argv, const char *prefix)\n {\n \tint i;\n \tstruct rev_info revs;\n+\tprune_expire = 2*60*60;\n+\tnow = time(NULL);\n \n \tfor (i = 1; i < argc; i++) {\n \t\tconst char *arg = argv[i];\n@@ -90,6 +109,10 @@ int cmd_prune(int argc, const char **argv, const char *prefix)\n \t\t\tshow_only = 1;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!strncmp(arg, \"--expire=\", 9)) {\n+\t\t\tprune_expire = atoi(arg+9);\n+\t\t\tcontinue;\n+\t\t}\n \t\tusage(prune_usage);\n \t}\n \n-- \n1.4.4.4\n"},{"id":"32026","messageId":"7vy7o0klt1.fsf@assigned-by-dhcp.cox.net","threadId":"6429","inReplyTo":"20070118222919.GA22060@moooo.ath.cx","subject":"Re: [RFC] prune: --expire=seconds","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-01-18T22:32:58Z","receivedAt":"2007-01-18T22:32:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthias Lederhofer writes:\n\n> This option specifies the minimum age of an object before it\n> may be removed by prune. The default value is 2 hours and\n> may be changed using gc.pruneexpire.\n>\n> Signed-off-by: Matthias Lederhofer \n> ---\n> Shawn O. Pearce wrote:\n>> If you are going to implement this I would suggest making the default\n>> age 2 hours and allow the user to configure it from a gc.pruneexpire\n>> configuration option, much like gc.reflogexpire.\n>\n> Here it is, I've set the default value to 2 hours as you suggested.\n> Any other comments if the default should be a value >0 or 0 to keep\n> the old behaviour?\n\nI am not sure if this is needed, as Shawn explained earlier\nrounds of loose-objects safety work.\n\nIf this is something we would want, it might make sense if we\nallowed \"prune --expire='1.day'\" syntax ;-).\n"},{"id":"32033","messageId":"20070119034404.GA17521@spearce.org","threadId":"6429","inReplyTo":"7vy7o0klt1.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] prune: --expire=seconds","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-01-19T03:44:04Z","receivedAt":"2007-01-19T03:44:04Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> Matthias Lederhofer writes:\n> \n> > This option specifies the minimum age of an object before it\n> > may be removed by prune. The default value is 2 hours and\n> > may be changed using gc.pruneexpire.\n> \n> I am not sure if this is needed, as Shawn explained earlier\n> rounds of loose-objects safety work.\n\nI think we need this fix. We still have a race condition between\nthe loose object creation and the ref update.\n\nWe've closed this hole completely in the large push case (objects\n>=receive.unpackLimit) and 'fetch -k' case by creating .keep files\nbefore the .pack file, updating refs, then deleting the .keep file;\nand by making sure git-repack leaves packs with .keeps alone. So\nwe cannot lose an object here.\n\nBut update-index/add/merge-recursive/write-tree/commit-tree, etc.\nas well as small pushes (objects If this is something we would want, it might make sense if we\n> allowed \"prune --expire='1.day'\" syntax ;-).\n\nYes, I agree.\n\nMatthias you can take a look at builtin-reflog.c's argument handling\nfor an example. I think you just need to use approxidate() in both\nyour config function and in your command line argument handling.\nThen the default becomes '2.hours.ago' instead of just \"2\" (at\nleast from a documentation perspective).\n\nThough the more I think about this perhaps the default should be\n'1.day'. 24 hours is a hellva large window for any current ref\nupdate to complete in, even if the ref update was some massive rsync\nwhich is doing a such a large volume of data on a small bandwidth\nlink that it takes 20 hours to complete. Besides, users could\nalways force it to be much lower with the command line option if\nthey really need to prune _right_now_.\n\n-- \nShawn.\n"},{"id":"32054","messageId":"20070119104935.GA5189@moooo.ath.cx","threadId":"6429","inReplyTo":"20070119034404.GA17521@spearce.org","subject":"[PATCH] prune: --expire=time","fromName":"Matthias Lederhofer","fromEmail":"matled@gmx.net","sentAt":"2007-01-19T10:49:35Z","receivedAt":"2007-01-19T10:49:35Z","isPatch":true,"sender":{"key":"matled@gmx.net","avatar":null},"body":"This option specifies the minimum age of an object before it\nmay be removed by prune. The default value is 24 hours and\nmay be changed using gc.pruneexpire.\n\nSigned-off-by: Matthias Lederhofer \n---\nShawn O. Pearce wrote:\n> Given that 'git gc' is the encouraged way to maintain a repository,\n> and that 'repack -a -d' is safe, and prune-packed is equally safe,\n> I think we should try to make prune safe too. Matthias' patch\n> does this by giving the ref update process a fairly large window\n> to perform its action within.\nAh, git repack -a -d is safe now too?\n\n> Junio C Hamano wrote:\n> > If this is something we would want, it might make sense if we\n> > allowed \"prune --expire='1.day'\" syntax ;-).\n> \n> Yes, I agree.\n> \n> Matthias you can take a look at builtin-reflog.c's argument handling\n> for an example. I think you just need to use approxidate() in both\n> your config function and in your command line argument handling.\n> Then the default becomes '2.hours.ago' instead of just \"2\" (at\n> least from a documentation perspective).\n> \n> Though the more I think about this perhaps the default should be\n> '1.day'. 24 hours is a hellva large window for any current ref\n> update to complete in, even if the ref update was some massive rsync\n> which is doing a such a large volume of data on a small bandwidth\n> link that it takes 20 hours to complete. Besides, users could\n> always force it to be much lower with the command line option if\n> they really need to prune _right_now_.\nThanks for the advice, this is much better than specifying seconds.\nHere is the new version.\n\nThings I'm not sure about, any further comments/discussion?\n- default value for gc.pruneexpire\n- special value(s) for gc.pruneexpire/--expire which mean 'do not\n check for the age', currently it is 'off'\n---\n builtin-prune.c | 31 ++++++++++++++++++++++++++++++-\n 1 files changed, 30 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-prune.c b/builtin-prune.c\nindex 6f0ba0d..410b3b2 100644\n--- a/builtin-prune.c\n+++ b/builtin-prune.c\n@@ -5,8 +5,9 @@\n #include \"builtin.h\"\n #include \"reachable.h\"\n \n-static const char prune_usage[] = \"git-prune [-n]\";\n+static const char prune_usage[] = \"git-prune [-n] [--expire=time]\";\n static int show_only;\n+static int prune_expire;\n \n static int prune_object(char *path, const char *filename, const unsigned char *sha1)\n {\n@@ -38,6 +39,7 @@ static int prune_dir(int i, char *path)\n \t\tchar name[100];\n \t\tunsigned char sha1[20];\n \t\tint len = strlen(de->d_name);\n+\t\tstruct stat st;\n \n \t\tswitch (len) {\n \t\tcase 2:\n@@ -60,6 +62,11 @@ static int prune_dir(int i, char *path)\n \t\t\tif (lookup_object(sha1))\n \t\t\t\tcontinue;\n \n+\t\t\tif (prune_expire > 0 &&\n+\t\t\t !stat(mkpath(\"%s/%s\", path, de->d_name), &st) &&\n+\t\t\t st.st_mtime > prune_expire)\n+\t\t\t\tcontinue;\n+\n \t\t\tprune_object(path, de->d_name, sha1);\n \t\t\tcontinue;\n \t\t}\n@@ -79,10 +86,25 @@ static void prune_object_dir(const char *path)\n \t}\n }\n \n+static int git_prune_config(const char *var, const char *value)\n+{\n+\tif (!strcmp(var, \"gc.pruneexpire\")) {\n+\t\tif (!strcmp(value, \"off\"))\n+\t\t\tprune_expire = 0;\n+\t\telse\n+\t\t\tprune_expire = approxidate(value);\n+\t\treturn 0;\n+\t}\n+\treturn git_default_config(var, value);\n+}\n+\n int cmd_prune(int argc, const char **argv, const char *prefix)\n {\n \tint i;\n \tstruct rev_info revs;\n+\tprune_expire = time(NULL)-24*60*60;\n+\n+\tgit_config(git_prune_config);\n \n \tfor (i = 1; i < argc; i++) {\n \t\tconst char *arg = argv[i];\n@@ -90,6 +112,13 @@ int cmd_prune(int argc, const char **argv, const char *prefix)\n \t\t\tshow_only = 1;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!strncmp(arg, \"--expire=\", 9)) {\n+\t\t\tif (!strcmp(arg+9, \"off\"))\n+\t\t\t\tprune_expire = 0;\n+\t\t\telse\n+\t\t\t\tprune_expire = approxidate(arg+9);\n+\t\t\tcontinue;\n+\t\t}\n \t\tusage(prune_usage);\n \t}\n \n-- \n1.4.4.4\n"},{"id":"32069","messageId":"Pine.LNX.4.64.0701191039420.3011@xanadu.home","threadId":"6429","inReplyTo":"20070119104935.GA5189@moooo.ath.cx","subject":"Re: [PATCH] prune: --expire=time","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-01-19T15:41:34Z","receivedAt":"2007-01-19T15:41:34Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 19 Jan 2007, Matthias Lederhofer wrote:\n\n> Ah, git repack -a -d is safe now too?\n\nYes.\n\n\nNicolas\n"},{"id":"32074","messageId":"7vfya6hll3.fsf@assigned-by-dhcp.cox.net","threadId":"6429","inReplyTo":"20070119104935.GA5189@moooo.ath.cx","subject":"Re: [PATCH] prune: --expire=time","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-01-19T19:18:16Z","receivedAt":"2007-01-19T19:18:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthias Lederhofer writes:\n\n> Shawn O. Pearce wrote:\n>\n>> Junio C Hamano wrote:\n>> > If this is something we would want, it might make sense if we\n>> > allowed \"prune --expire='1.day'\" syntax ;-).\n>> \n>> Yes, I agree.\n>\n> Here is the new version.\n\nThanks.\n\n> Things I'm not sure about, any further comments/discussion?\n> - default value for gc.pruneexpire\n> - special value(s) for gc.pruneexpire/--expire which mean 'do not\n> check for the age', currently it is 'off'\n\nNo single timeout value can be the right timeout for everybody,\nso a big debate is not useful here. I think 1 day as you and\nShawn did makes sense.\n\n\tgit prune --expire=off\n\nfelt a bit confusing to me at the first glance. Does it turn\noff the expiration mechanism, retaining all cruft, or turns off\nthe mechanism to give grace period for recent objects? The\nanswer is obviosuly the latter as \"retain all cruft\" is\nmeaningless, but still it somehow feels funny. It might be\neasier to explain if it was:\n\n\tgit prune --expire=now\n\nMaybe an alternative:\n\n\tgit prune --retain=1.day\n\tgit prune --retain=off\n\nperhaps? I dunno.\n\nWe seem to use \"unsigned long\" as a stand-in type for time_t in\nthe rest of the code. I'd feel better if prune_expire was also\n\"unsigned long\". We probably should talk about making them all\ntime_t at some point but not right now.\n\nI was tempted to say that we might want to teach approxidate\n\"now\" (add one entry to struct special in date.c), but I do not\nthink it is useful for this application (you want prune_expire\nset to zero not the time we run time(NULL)), and I do not think\nof any other application that wants \"now\".\n"},{"id":"32150","messageId":"20070120111832.GA30368@moooo.ath.cx","threadId":"6429","inReplyTo":"7vfya6hll3.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] prune: --expire=time","fromName":"Matthias Lederhofer","fromEmail":"matled@gmx.net","sentAt":"2007-01-20T11:18:32Z","receivedAt":"2007-01-20T11:18:32Z","isPatch":true,"sender":{"key":"matled@gmx.net","avatar":null},"body":"Junio C Hamano wrote:\n> \tgit prune --expire=off\n> \n> felt a bit confusing to me at the first glance. Does it turn\n> off the expiration mechanism, retaining all cruft, or turns off\n> the mechanism to give grace period for recent objects? The\n> answer is obviosuly the latter as \"retain all cruft\" is\n> meaningless, but still it somehow feels funny. It might be\n> easier to explain if it was:\n> \n> \tgit prune --expire=now\n> \n> Maybe an alternative:\n> \n> \tgit prune --retain=1.day\n> \tgit prune --retain=off\n> \n> perhaps? I dunno.\nPerhaps we could also use 'none' or 'all, e.g. --retain=none or\n--expire=all.\n"},{"id":"32152","messageId":"45B205BB.7010101@fs.ei.tum.de","threadId":"6429","inReplyTo":"7vfya6hll3.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] prune: --expire=time","fromName":"Simon 'corecode' Schubert","fromEmail":"corecode@fs.ei.tum.de","sentAt":"2007-01-20T12:06:19Z","receivedAt":"2007-01-20T12:06:19Z","isPatch":true,"sender":{"key":"corecode@fs.ei.tum.de","avatar":"https://gravatar.com/avatar/eff9dbf0cdac0d1e6a6cd7ed0e50763edcb376b493b5253a35ff167918ad79e1?d=mp&s=160"},"body":"Junio C Hamano wrote:\n>> Things I'm not sure about, any further comments/discussion?\n>> - default value for gc.pruneexpire\n>> - special value(s) for gc.pruneexpire/--expire which mean 'do not\n>> check for the age', currently it is 'off'\n> \n> No single timeout value can be the right timeout for everybody,\n> so a big debate is not useful here. I think 1 day as you and\n> Shawn did makes sense.\n\nNot that I want to sabotage this discussion, but you have a very valid point. A timeout can always be crossed, and then bad things[tm] happen.\n\nMy idea is to create a marker file when creating (yet) unconnected loose objects, i.e. on commit/push/fetch. After the ref was updated or on abort, this marker would be removed. Prune then can simply search for the oldest marker and only remove objects older than this marker.\n\nOf course this also can mean that a marker file somehow stays and prune fails to clean properly, but that's still better than accidentially cleaning too much. In the case of dangling marker files, the admin would simply remove them. rm .git/marker/* when the repo is quiet.\n\ncheers\n simon\n\n-- \nServe - BSD +++ RENT this banner advert +++ ASCII Ribbon /\"\\\nWork - Mac +++ space for low €€€ NOW!1 +++ Campaign \\ /\nParty Enjoy Relax | http://dragonflybsd.org Against HTML \\\nDude 2c 2 the max ! http://golden-apple.biz Mail + News / \\\n\n"},{"id":"32189","messageId":"7vlkjw50nl.fsf@assigned-by-dhcp.cox.net","threadId":"6429","inReplyTo":"20070120111832.GA30368@moooo.ath.cx","subject":"Re: [PATCH] prune: --expire=time","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-01-21T06:55:42Z","receivedAt":"2007-01-21T06:55:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthias Lederhofer writes:\n\n> Junio C Hamano wrote:\n>...\n>> Maybe an alternative:\n>> \n>> \tgit prune --retain=1.day\n>> \tgit prune --retain=off\n>> \n>> perhaps? I dunno.\n>\n> Perhaps we could also use 'none' or 'all, e.g. --retain=none or\n> --expire=all.\n\nI am considering to commit the attached instead.\n\nThe command \"prune\" is about expiring loose objects that are\nunreachable, and the option introduces a grace period for the\nexpiration process. Other places that we do use the word\n'expire' to specify the time do mean the expiration timeout.\n\nAs long as you do not rewind/rebase too much, there is not much\nyou can gain from 'git-prune' ('git-repack -a -d' would give you\nmuch more disk savings and performance gain). I do not think it\nmakes sense to run uncontrolled 'git-prune' from automated cron\njobs. Even if you rewind/rebase often, 1.5.0 will protect the\nobjects reflog entries refer to, so there will be even less to\nbe gained from 'git-prune'. I am having a feeling that it might\neven make sense not to run 'git-prune' from 'git-gc'.\n\nWhile I sympathize with what Simon says to certain degree, I\ntend to think the complication it needs to introduce is really\nnot worth it. Perfect is the enemy of the good.\n\nBy the way.\n\nWhile updating the documentation, I noticed that we lost the\n'extra heads' support when git-prune was rewritten as a built-in\nin commit ba84a797 (July 6th 2006). The example (commented out\nin the patch) is a valid way to safely prune a repository whose\nobjects are borrowed via the alternates mechanism by some other\nrepository, albeit not really scalable.\n\nThe way we might want to address this would be when 'clone -s'\nmakes a new repository that borrows from an existing repository,\nwe could make a symlink under .git/refs/borrowers/ in the\noriginal repository that points at .git/refs directory of the\ncloned repository -- you can do that by hand today and it would\nbe much nicer than having to specify the other repository when\nrunning 'git prune' as the example suggests.\n\nFor this reason, I would say losing that 'extra heads' support\nfrom git-prune, which happened half year ago, was Ok.\n\nSo far, we have been telling not to rewind refs in repositories\nwhose objects are borrowed by other repositories via the\nalternates mechanism, and I think that advice is still a very\nreasonable one to give, but we probably should make it more\nprominent. As long as we do that, we would not even need the\n'refs/borrowers/*' symlinks.\n\n-- >8 --\nFrom: Matthias Lederhofer \n[PATCH] prune: --grace=time\n\nThis option gives grace period to objects that are unreachable\nfrom the refs from getting pruned.\n\nThe default value is 24 hours and may be changed using\ngc.prunegrace.\n\nSigned-off-by: Matthias Lederhofer \nSigned-off-by: Junio C Hamano \n---\n Documentation/git-prune.txt | 9 ++++++++-\n builtin-prune.c | 31 ++++++++++++++++++++++++++++++-\n 2 files changed, 38 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-prune.txt b/Documentation/git-prune.txt\nindex a11e303..fbd344d 100644\n--- a/Documentation/git-prune.txt\n+++ b/Documentation/git-prune.txt\n@@ -8,7 +8,7 @@ git-prune - Prunes all unreachable objects from the object database\n \n SYNOPSIS\n --------\n-'git-prune' [-n] [--] [...]\n+'git-prune' [-n] [--grace=