{"thread":{"id":"20891","subject":"[PATCH] preserve mtime of local clone","startedAt":"2009-09-09T19:51:58Z","lastAt":"2009-09-14T03:39:58Z","messageCount":13,"participants":["Clemens Buchacher","Junio C Hamano","Jeff King","Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"122774","messageId":"20090909195158.GA12968@localhost","threadId":"20891","inReplyTo":null,"subject":"[PATCH] preserve mtime of local clone","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-09T19:51:58Z","receivedAt":"2009-09-09T19:51:58Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"A local clone without hardlinks copies all objects, including dangling\nones, to the new repository. Since the mtimes are renewed, those\ndangling objects cannot be pruned by \"git gc --prune\", even if they\nwould have been old enough for pruning in the original repository.\n\nInstead, preserve mtime during copy. \"git gc --prune\" will then work\nin the clone just like it would have in the original.\n\nSigned-off-by: Clemens Buchacher <drizzd@aon.at>\n---\n\nI noticed this problem when I cloned a repo with lots of old dangling\nobjects onto a windows machine. git-gui immediately recommended running\ngit-gc, and I did. But each time I restarted git-gui, it recommended git-gc\nagain, because there were still plenty of dangling objects lying around\nwhich could not be removed due to their recent mtimes.\n\nSo there is actually a problem with git-gui's recommendation. Especially on\nWindows, where it only checks for 1 or more files in .git/objects/42 (as\nopposed to 8 files on other platforms). The probability of that happening if\nthe repo contains about 100 loose objects is 1-(254/255)^100 = 32%. The\nprobability for the same to happen with at least 2 files is only 6% [*].\nMaybe that would be a good compromise?\n\nAlternatively, git-gc could remember the number of dangling objects, and\ngit-gui can adjust its recommendation accordingly, taking that number and\nthe date of the lastest repack into account.\n\nClemens\n\n[*] The following octave script shows the probability for m or more objects\nto be in .git/objects/42 for a total of n objects.\n\nm = [1 2 8];\nn = 100:100:3000;\n\nP = zeros(length(n), length(m));\nfor k = 1:length(n)\n\tP(n(k), :) = 1-binocdf(m-1, n(k), 1/255);\nend\nplot(n, P);\n\nn \\ m\t1\t2\t8\n100\t32%\t6%\t0%\n500\t86%\t58%\t0%\n1000\t98%\t90%\t5%\n2000\t100%\t100%\t55%\n\n---\n builtin-clone.c   |    2 +-\n builtin-init-db.c |    2 +-\n cache.h           |    6 ++++--\n copy.c            |   25 ++++++++++++++++++++++---\n lockfile.c        |    2 +-\n rerere.c          |    2 +-\n 6 files changed, 30 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin-clone.c b/builtin-clone.c\nindex ad04808..cb3c895 100644\n--- a/builtin-clone.c\n+++ b/builtin-clone.c\n@@ -269,7 +269,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest)\n \t\t\t\tdie_errno(\"failed to create link '%s'\", dest->buf);\n \t\t\toption_no_hardlinks = 1;\n \t\t}\n-\t\tif (copy_file(dest->buf, src->buf, 0666))\n+\t\tif (copy_file(dest->buf, src->buf, 0666, 1))\n \t\t\tdie_errno(\"failed to copy file to '%s'\", dest->buf);\n \t}\n \tclosedir(dir);\ndiff --git a/builtin-init-db.c b/builtin-init-db.c\nindex dd84cae..5deb81d 100644\n--- a/builtin-init-db.c\n+++ b/builtin-init-db.c\n@@ -100,7 +100,7 @@ static void copy_templates_1(char *path, int baselen,\n \t\t\t\tdie_errno(\"cannot symlink '%s' '%s'\", lnk, path);\n \t\t}\n \t\telse if (S_ISREG(st_template.st_mode)) {\n-\t\t\tif (copy_file(path, template, st_template.st_mode))\n+\t\t\tif (copy_file(path, template, st_template.st_mode, 0))\n \t\t\t\tdie_errno(\"cannot copy '%s' to '%s'\", template,\n \t\t\t\t\t  path);\n \t\t}\ndiff --git a/cache.h b/cache.h\nindex 5fad24c..1875c97 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -921,8 +921,10 @@ extern const char *git_mailmap_file;\n \n /* IO helper functions */\n extern void maybe_flush_or_die(FILE *, const char *);\n-extern int copy_fd(int ifd, int ofd);\n-extern int copy_file(const char *dst, const char *src, int mode);\n+extern int copy_fd(int ifd, int ofd, int preserve_times);\n+extern int copy_file(const char *dst, const char *src, int mode, int\n+\t\tpreserve_times);\n+extern int copy_times(int ofd, int ifd);\n extern ssize_t read_in_full(int fd, void *buf, size_t count);\n extern ssize_t write_in_full(int fd, const void *buf, size_t count);\n extern void write_or_die(int fd, const void *buf, size_t count);\ndiff --git a/copy.c b/copy.c\nindex e54d15a..fe0380e 100644\n--- a/copy.c\n+++ b/copy.c\n@@ -1,6 +1,6 @@\n #include \"cache.h\"\n \n-int copy_fd(int ifd, int ofd)\n+int copy_fd(int ifd, int ofd, int preserve_times)\n {\n \twhile (1) {\n \t\tchar buffer[8192];\n@@ -31,11 +31,18 @@ int copy_fd(int ifd, int ofd)\n \t\t\t}\n \t\t}\n \t}\n+\tif (preserve_times && copy_times(ofd, ifd)) {\n+\t\tint time_error = errno;\n+\t\tclose(ifd);\n+\t\treturn error(\"copy-fd: failed to preserve times: %s\",\n+\t\t\t\tstrerror(time_error));\n+\t}\n \tclose(ifd);\n \treturn 0;\n }\n \n-int copy_file(const char *dst, const char *src, int mode)\n+int copy_file(const char *dst, const char *src, int mode,\n+\t\tint preserve_times)\n {\n \tint fdi, fdo, status;\n \n@@ -46,7 +53,7 @@ int copy_file(const char *dst, const char *src, int mode)\n \t\tclose(fdi);\n \t\treturn fdo;\n \t}\n-\tstatus = copy_fd(fdi, fdo);\n+\tstatus = copy_fd(fdi, fdo, preserve_times);\n \tif (close(fdo) != 0)\n \t\treturn error(\"%s: close error: %s\", dst, strerror(errno));\n \n@@ -55,3 +62,15 @@ int copy_file(const char *dst, const char *src, int mode)\n \n \treturn status;\n }\n+\n+int copy_times(int ofd, int ifd)\n+{\n+\tstruct stat st;\n+\tstruct timespec times[2];\n+\tif (fstat(ifd, &st))\n+\t\treturn -1;\n+\ttimes[0].tv_nsec = UTIME_OMIT;\n+\ttimes[1].tv_sec = st.st_mtime;\n+\ttimes[1].tv_nsec = ST_MTIME_NSEC(st);\n+\treturn futimens(ofd, times);\n+}\ndiff --git a/lockfile.c b/lockfile.c\nindex eb931ed..c7bbd4d 100644\n--- a/lockfile.c\n+++ b/lockfile.c\n@@ -196,7 +196,7 @@ int hold_lock_file_for_append(struct lock_file *lk, const char *path, int flags)\n \t\t\tclose(fd);\n \t\t\treturn error(\"cannot open '%s' for copying\", path);\n \t\t}\n-\t} else if (copy_fd(orig_fd, fd)) {\n+\t} else if (copy_fd(orig_fd, fd, 0)) {\n \t\tif (flags & LOCK_DIE_ON_ERROR)\n \t\t\texit(128);\n \t\tclose(fd);\ndiff --git a/rerere.c b/rerere.c\nindex 87360dc..d25f5f1 100644\n--- a/rerere.c\n+++ b/rerere.c\n@@ -326,7 +326,7 @@ static int do_plain_rerere(struct string_list *rr, int fd)\n \t\t\tcontinue;\n \n \t\tfprintf(stderr, \"Recorded resolution for '%s'.\\n\", path);\n-\t\tcopy_file(rerere_path(name, \"postimage\"), path, 0666);\n+\t\tcopy_file(rerere_path(name, \"postimage\"), path, 0666, 0);\n \tmark_resolved:\n \t\trr->items[i].util = NULL;\n \t}\n-- \n1.6.4.2.266.gbaa17\n"},{"id":"122933","messageId":"7vmy50lpfr.fsf@alter.siamese.dyndns.org","threadId":"20891","inReplyTo":"20090909195158.GA12968@localhost","subject":"Re: [PATCH] preserve mtime of local clone","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-09-12T05:09:12Z","receivedAt":"2009-09-12T05:09:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\nClemens Buchacher <drizzd@aon.at> writes:\n\n> +int copy_times(int ofd, int ifd)\n> +{\n> +\tstruct stat st;\n> +\tstruct timespec times[2];\n> +\tif (fstat(ifd, &st))\n> +\t\treturn -1;\n> +\ttimes[0].tv_nsec = UTIME_OMIT;\n> +\ttimes[1].tv_sec = st.st_mtime;\n> +\ttimes[1].tv_nsec = ST_MTIME_NSEC(st);\n> +\treturn futimens(ofd, times);\n> +}\n\nHmm, futimens() is relatively new.  Are minority platforms folks Ok with\nthis patch?\n\nAt least SunOS 5.11 (OpenSolaris 0811) seems to barf on UTIME_OMIT.\n"},{"id":"122935","messageId":"20090912082624.GA9654@localhost","threadId":"20891","inReplyTo":"7vmy50lpfr.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] preserve mtime of local clone","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-12T08:26:24Z","receivedAt":"2009-09-12T08:26:24Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"On Fri, Sep 11, 2009 at 10:09:12PM -0700, Junio C Hamano wrote:\n> Clemens Buchacher <drizzd@aon.at> writes:\n> \n> > +int copy_times(int ofd, int ifd)\n> > +{\n> > +\tstruct stat st;\n> > +\tstruct timespec times[2];\n> > +\tif (fstat(ifd, &st))\n> > +\t\treturn -1;\n> > +\ttimes[0].tv_nsec = UTIME_OMIT;\n> > +\ttimes[1].tv_sec = st.st_mtime;\n> > +\ttimes[1].tv_nsec = ST_MTIME_NSEC(st);\n> > +\treturn futimens(ofd, times);\n> > +}\n> \n> Hmm, futimens() is relatively new.  Are minority platforms folks Ok with\n> this patch?\n> \n> At least SunOS 5.11 (OpenSolaris 0811) seems to barf on UTIME_OMIT.\n\nIf it's a problem we can use utime() instead. I was just trying to use the\nfile descriptors, since they were available. But the patch would be a little\nsmaller if I didn't touch copy_fd().\n\nClemens\n"},{"id":"122937","messageId":"20090912090348.GB9654@localhost","threadId":"20891","inReplyTo":"20090912082624.GA9654@localhost","subject":"[PATCH] preserve mtime of local clone","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-12T09:03:48Z","receivedAt":"2009-09-12T09:03:48Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"A local clone without hardlinks copies all objects, including dangling\nones, to the new repository. Since the mtimes are renewed, those\ndangling objects cannot be pruned by \"git gc --prune\", even if they\nwould have been old enough for pruning in the original repository.\n\nInstead, preserve mtime during copy. \"git gc --prune\" will then work\nin the clone just like it did in the original.\n\nSigned-off-by: Clemens Buchacher <drizzd@aon.at>\n---\n\nOn Sat, Sep 12, 2009 at 10:26:24AM +0200, Clemens Buchacher wrote:\n\n> If it's a problem we can use utime() instead. I was just trying to use the\n> file descriptors, since they were available. But the patch would be a little\n> smaller if I didn't touch copy_fd().\n\nHere we go.\n\n builtin-clone.c   |    2 +-\n builtin-init-db.c |    2 +-\n cache.h           |    4 +++-\n copy.c            |   18 +++++++++++++++++-\n rerere.c          |    2 +-\n 5 files changed, 23 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin-clone.c b/builtin-clone.c\nindex ad04808..cb3c895 100644\n--- a/builtin-clone.c\n+++ b/builtin-clone.c\n@@ -269,7 +269,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest)\n \t\t\t\tdie_errno(\"failed to create link '%s'\", dest->buf);\n \t\t\toption_no_hardlinks = 1;\n \t\t}\n-\t\tif (copy_file(dest->buf, src->buf, 0666))\n+\t\tif (copy_file(dest->buf, src->buf, 0666, 1))\n \t\t\tdie_errno(\"failed to copy file to '%s'\", dest->buf);\n \t}\n \tclosedir(dir);\ndiff --git a/builtin-init-db.c b/builtin-init-db.c\nindex dd84cae..5deb81d 100644\n--- a/builtin-init-db.c\n+++ b/builtin-init-db.c\n@@ -100,7 +100,7 @@ static void copy_templates_1(char *path, int baselen,\n \t\t\t\tdie_errno(\"cannot symlink '%s' '%s'\", lnk, path);\n \t\t}\n \t\telse if (S_ISREG(st_template.st_mode)) {\n-\t\t\tif (copy_file(path, template, st_template.st_mode))\n+\t\t\tif (copy_file(path, template, st_template.st_mode, 0))\n \t\t\t\tdie_errno(\"cannot copy '%s' to '%s'\", template,\n \t\t\t\t\t  path);\n \t\t}\ndiff --git a/cache.h b/cache.h\nindex 5fad24c..ac692d0 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -922,7 +922,9 @@ extern const char *git_mailmap_file;\n /* IO helper functions */\n extern void maybe_flush_or_die(FILE *, const char *);\n extern int copy_fd(int ifd, int ofd);\n-extern int copy_file(const char *dst, const char *src, int mode);\n+extern int copy_file(const char *dst, const char *src, int mode, int\n+\t\tpreserve_times);\n+extern int copy_times(const char *dst, const char *src);\n extern ssize_t read_in_full(int fd, void *buf, size_t count);\n extern ssize_t write_in_full(int fd, const void *buf, size_t count);\n extern void write_or_die(int fd, const void *buf, size_t count);\ndiff --git a/copy.c b/copy.c\nindex e54d15a..fb5e946 100644\n--- a/copy.c\n+++ b/copy.c\n@@ -35,7 +35,21 @@ int copy_fd(int ifd, int ofd)\n \treturn 0;\n }\n \n-int copy_file(const char *dst, const char *src, int mode)\n+int copy_times(const char *dst, const char *src)\n+{\n+\tstruct stat st;\n+\tstruct utimbuf times;\n+\tif (stat(src, &st) < 0)\n+\t\treturn -1;\n+\ttimes.actime = st.st_atime;\n+\ttimes.modtime = st.st_mtime;\n+\tif (utime(dst, &times) < 0)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+int copy_file(const char *dst, const char *src, int mode,\n+\t\tint preserve_times)\n {\n \tint fdi, fdo, status;\n \n@@ -52,6 +66,8 @@ int copy_file(const char *dst, const char *src, int mode)\n \n \tif (!status && adjust_shared_perm(dst))\n \t\treturn -1;\n+\tif (!status && preserve_times && copy_times(dst, src))\n+\t\treturn -1;\n \n \treturn status;\n }\ndiff --git a/rerere.c b/rerere.c\nindex 87360dc..d25f5f1 100644\n--- a/rerere.c\n+++ b/rerere.c\n@@ -326,7 +326,7 @@ static int do_plain_rerere(struct string_list *rr, int fd)\n \t\t\tcontinue;\n \n \t\tfprintf(stderr, \"Recorded resolution for '%s'.\\n\", path);\n-\t\tcopy_file(rerere_path(name, \"postimage\"), path, 0666);\n+\t\tcopy_file(rerere_path(name, \"postimage\"), path, 0666, 0);\n \tmark_resolved:\n \t\trr->items[i].util = NULL;\n \t}\n-- \n1.6.5.rc0.164.g5f6b0\n"},{"id":"123022","messageId":"7vvdjn4k6s.fsf@alter.siamese.dyndns.org","threadId":"20891","inReplyTo":"20090912090348.GB9654@localhost","subject":"Re: [PATCH] preserve mtime of local clone","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-09-13T03:06:51Z","receivedAt":"2009-09-13T03:06:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\nClemens Buchacher <drizzd@aon.at> writes:\n\n> A local clone without hardlinks copies all objects, including dangling\n> ones, to the new repository. Since the mtimes are renewed, those\n> dangling objects cannot be pruned by \"git gc --prune\", even if they\n> would have been old enough for pruning in the original repository.\n>\n> Instead, preserve mtime during copy. \"git gc --prune\" will then work\n> in the clone just like it did in the original.\n>\n> Signed-off-by: Clemens Buchacher <drizzd@aon.at>\n> ---\n>\n> On Sat, Sep 12, 2009 at 10:26:24AM +0200, Clemens Buchacher wrote:\n>\n>> If it's a problem we can use utime() instead. I was just trying to use the\n>> file descriptors, since they were available. But the patch would be a little\n>> smaller if I didn't touch copy_fd().\n>\n> Here we go.\n\nThanks.\n\nThis new feature is not wanted by most of the callers of copy_file(), and\nit is not like they may want to conditionally pass 1 in preserve_times\nsomeday.  I'd limit the damage like this replacement patch, instead.\n\nIs it unreasonable to have a test for this one, by the way?\n\n builtin-clone.c |    2 +-\n cache.h         |    1 +\n copy.c          |   21 +++++++++++++++++++++\n 3 files changed, 23 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin-clone.c b/builtin-clone.c\nindex ad04808..bab2d84 100644\n--- a/builtin-clone.c\n+++ b/builtin-clone.c\n@@ -269,7 +269,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest)\n \t\t\t\tdie_errno(\"failed to create link '%s'\", dest->buf);\n \t\t\toption_no_hardlinks = 1;\n \t\t}\n-\t\tif (copy_file(dest->buf, src->buf, 0666))\n+\t\tif (copy_file_with_time(dest->buf, src->buf, 0666))\n \t\t\tdie_errno(\"failed to copy file to '%s'\", dest->buf);\n \t}\n \tclosedir(dir);\ndiff --git a/cache.h b/cache.h\nindex 867918d..1668f28 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -923,6 +923,7 @@ extern const char *git_mailmap_file;\n extern void maybe_flush_or_die(FILE *, const char *);\n extern int copy_fd(int ifd, int ofd);\n extern int copy_file(const char *dst, const char *src, int mode);\n+extern int copy_file_with_time(const char *dst, const char *src, int mode);\n extern void write_or_die(int fd, const void *buf, size_t count);\n extern int write_or_whine(int fd, const void *buf, size_t count, const char *msg);\n extern int write_or_whine_pipe(int fd, const void *buf, size_t count, const char *msg);\ndiff --git a/copy.c b/copy.c\nindex e54d15a..a7f58fd 100644\n--- a/copy.c\n+++ b/copy.c\n@@ -35,6 +35,19 @@ int copy_fd(int ifd, int ofd)\n \treturn 0;\n }\n \n+static int copy_times(const char *dst, const char *src)\n+{\n+\tstruct stat st;\n+\tstruct utimbuf times;\n+\tif (stat(src, &st) < 0)\n+\t\treturn -1;\n+\ttimes.actime = st.st_atime;\n+\ttimes.modtime = st.st_mtime;\n+\tif (utime(dst, &times) < 0)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n int copy_file(const char *dst, const char *src, int mode)\n {\n \tint fdi, fdo, status;\n@@ -55,3 +68,11 @@ int copy_file(const char *dst, const char *src, int mode)\n \n \treturn status;\n }\n+\n+int copy_file_with_time(const char *dst, const char *src, int mode)\n+{\n+\tint status = copy_file(dst, src, mode);\n+\tif (!status)\n+\t\treturn copy_times(dst, src);\n+\treturn status;\n+}\n"},{"id":"123048","messageId":"20090913104945.GA22098@localhost","threadId":"20891","inReplyTo":"7vvdjn4k6s.fsf@alter.siamese.dyndns.org","subject":"[PATCH v3] preserve mtime of local clone","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-13T10:49:45Z","receivedAt":"2009-09-13T10:49:45Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"A local clone without hardlinks copies all objects, including dangling\nones, to the new repository. Since the mtimes are renewed, those\ndangling objects cannot be pruned by \"git gc --prune\", even if they\nwould have been old enough for pruning in the original repository.\n\nInstead, preserve mtime during copy. \"git gc --prune\" will then work\nin the clone just like it did in the original.\n\nSigned-off-by: Clemens Buchacher <drizzd@aon.at>\n---\n\nOn Sat, Sep 12, 2009 at 08:06:51PM -0700, Junio C Hamano wrote:\n\n> This new feature is not wanted by most of the callers of copy_file(), and\n> it is not like they may want to conditionally pass 1 in preserve_times\n> someday.  I'd limit the damage like this replacement patch, instead.\n\nYou're right. That's much better.\n\n> Is it unreasonable to have a test for this one, by the way?\n\nIt's not. Test added.\n\nClemens\n\n builtin-clone.c  |    2 +-\n cache.h          |    2 ++\n copy.c           |   21 +++++++++++++++++++++\n t/t5304-prune.sh |   54 +++++++++++++++++++++++++++++++++---------------------\n 4 files changed, 57 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin-clone.c b/builtin-clone.c\nindex ad04808..bab2d84 100644\n--- a/builtin-clone.c\n+++ b/builtin-clone.c\n@@ -269,7 +269,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest)\n \t\t\t\tdie_errno(\"failed to create link '%s'\", dest->buf);\n \t\t\toption_no_hardlinks = 1;\n \t\t}\n-\t\tif (copy_file(dest->buf, src->buf, 0666))\n+\t\tif (copy_file_with_time(dest->buf, src->buf, 0666))\n \t\t\tdie_errno(\"failed to copy file to '%s'\", dest->buf);\n \t}\n \tclosedir(dir);\ndiff --git a/cache.h b/cache.h\nindex 5fad24c..ee9c98c 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -923,6 +923,8 @@ extern const char *git_mailmap_file;\n extern void maybe_flush_or_die(FILE *, const char *);\n extern int copy_fd(int ifd, int ofd);\n extern int copy_file(const char *dst, const char *src, int mode);\n+extern int copy_times(const char *dst, const char *src);\n+extern int copy_file_with_time(const char *dst, const char *src, int mode);\n extern ssize_t read_in_full(int fd, void *buf, size_t count);\n extern ssize_t write_in_full(int fd, const void *buf, size_t count);\n extern void write_or_die(int fd, const void *buf, size_t count);\ndiff --git a/copy.c b/copy.c\nindex e54d15a..963b481 100644\n--- a/copy.c\n+++ b/copy.c\n@@ -55,3 +55,24 @@ int copy_file(const char *dst, const char *src, int mode)\n \n \treturn status;\n }\n+\n+int copy_times(const char *dst, const char *src)\n+{\n+\tstruct stat st;\n+\tstruct utimbuf times;\n+\tif (stat(src, &st) < 0)\n+\t\treturn -1;\n+\ttimes.actime = st.st_atime;\n+\ttimes.modtime = st.st_mtime;\n+\tif (utime(dst, &times) < 0)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+int copy_file_with_time(const char *dst, const char *src, int mode)\n+{\n+     int status = copy_file(dst, src, mode);\n+     if (!status)\n+\t\treturn copy_times(dst, src);\n+     return status;\n+}\ndiff --git a/t/t5304-prune.sh b/t/t5304-prune.sh\nindex 55ed7c7..15fd90a 100755\n--- a/t/t5304-prune.sh\n+++ b/t/t5304-prune.sh\n@@ -6,6 +6,17 @@\n test_description='prune'\n . ./test-lib.sh\n \n+day=$((60*60*24))\n+week=$(($day*7))\n+\n+add_blob() {\n+\tbefore=$(git count-objects | sed \"s/ .*//\") &&\n+\tBLOB=$(echo aleph_0 | git hash-object -w --stdin) &&\n+\tBLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n+\ttest $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n+\ttest -f $BLOB_FILE\n+}\n+\n test_expect_success setup '\n \n \t: > file &&\n@@ -31,11 +42,7 @@ test_expect_success 'prune stale packs' '\n \n test_expect_success 'prune --expire' '\n \n-\tbefore=$(git count-objects | sed \"s/ .*//\") &&\n-\tBLOB=$(echo aleph | git hash-object -w --stdin) &&\n-\tBLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n-\ttest $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n-\ttest -f $BLOB_FILE &&\n+\tadd_blob &&\n \tgit prune --expire=1.hour.ago &&\n \ttest $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n \ttest -f $BLOB_FILE &&\n@@ -48,16 +55,12 @@ test_expect_success 'prune --expire' '\n \n test_expect_success 'gc: implicit prune --expire' '\n \n-\tbefore=$(git count-objects | sed \"s/ .*//\") &&\n-\tBLOB=$(echo aleph_0 | git hash-object -w --stdin) &&\n-\tBLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n-\ttest $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n-\ttest -f $BLOB_FILE &&\n-\ttest-chmtime =-$((86400*14-30)) $BLOB_FILE &&\n+\tadd_blob &&\n+\ttest-chmtime =-$((2*$week-30)) $BLOB_FILE &&\n \tgit gc &&\n \ttest $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n \ttest -f $BLOB_FILE &&\n-\ttest-chmtime =-$((86400*14+1)) $BLOB_FILE &&\n+\ttest-chmtime =-$((2*$week+1)) $BLOB_FILE &&\n \tgit gc &&\n \ttest $before = $(git count-objects | sed \"s/ .*//\") &&\n \t! test -f $BLOB_FILE\n@@ -114,12 +117,8 @@ test_expect_success 'prune: do not prune heads listed as an argument' '\n \n test_expect_success 'gc --no-prune' '\n \n-\tbefore=$(git count-objects | sed \"s/ .*//\") &&\n-\tBLOB=$(echo aleph_0 | git hash-object -w --stdin) &&\n-\tBLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n-\ttest $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n-\ttest -f $BLOB_FILE &&\n-\ttest-chmtime =-$((86400*5001)) $BLOB_FILE &&\n+\tadd_blob &&\n+\ttest-chmtime =-$((5001*$day)) $BLOB_FILE &&\n \tgit config gc.pruneExpire 2.days.ago &&\n \tgit gc --no-prune &&\n \ttest 1 = $(git count-objects | sed \"s/ .*//\") &&\n@@ -140,9 +139,8 @@ test_expect_success 'gc respects gc.pruneExpire' '\n \n test_expect_success 'gc --prune=<date>' '\n \n-\tBLOB=$(echo aleph_0 | git hash-object -w --stdin) &&\n-\tBLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n-\ttest-chmtime =-$((86400*5001)) $BLOB_FILE &&\n+\tadd_blob &&\n+\ttest-chmtime =-$((5001*$day)) $BLOB_FILE &&\n \tgit gc --prune=5002.days.ago &&\n \ttest -f $BLOB_FILE &&\n \tgit gc --prune=5000.days.ago &&\n@@ -150,4 +148,18 @@ test_expect_success 'gc --prune=<date>' '\n \n '\n \n+test_expect_success 'gc: prune old objects after local clone' '\n+\n+\tadd_blob &&\n+\ttest-chmtime =-$((2*$week+1)) $BLOB_FILE &&\n+\tgit clone --no-hardlinks . aclone &&\n+\tcd aclone &&\n+\ttest 1 = $(git count-objects | sed \"s/ .*//\") &&\n+\ttest -f $BLOB_FILE &&\n+\tgit gc --prune &&\n+\ttest 0 = $(git count-objects | sed \"s/ .*//\") &&\n+\t! test -f $BLOB_FILE\n+\n+'\n+\n test_done\n-- \n1.6.5.rc0.164.g5f6b0\n"},{"id":"123066","messageId":"20090913160637.GA15256@localhost","threadId":"20891","inReplyTo":"20090909195158.GA12968@localhost","subject":"[PATCH] git-gui: suggest gc only when counting at least 2 objects","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-13T16:06:37Z","receivedAt":"2009-09-13T16:06:37Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"On Windows, git-gui suggests running the garbage collector if it finds\n1 or more files in .git/objects/42 (as opposed to 8 files on other\nplatforms). The probability of that happening if the repo contains\nonly about 100 loose objects is 32%. The probability for the same to happen\nwith at least 2 files is only 6%, which is bit more reasonable.\n\nThe following octave script shows the probability for m or more\nobjects to be in .git/objects/42 for a total of n objects.\n\nm = [1 2 8];\nn = 100:100:3000;\n\nP = zeros(length(n), length(m));\nfor k = 1:length(n)\n        P(k, :) = 1-binocdf(m-1, n(k), 1/255);\nend\nplot(n, P);\n\nn \\ m   1       2       8\n100     32%     6%      0%\n500     86%     58%     0%\n1000    98%     90%     5%\n2000    100%    100%    55%\n\nSigned-off-by: Clemens Buchacher <drizzd@aon.at>\n---\n\nSo here is the patch for my other complaint. Note that I fixed a bug in the\noctave script above, in case someone wants to check the numbers.\n\nI even tested it on a windows VM (as if it wasn't slow enough already).\n\nClemens\n\n git-gui/lib/database.tcl |    2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/git-gui/lib/database.tcl b/git-gui/lib/database.tcl\nindex a18ac8b..44099e5 100644\n--- a/git-gui/lib/database.tcl\n+++ b/git-gui/lib/database.tcl\n@@ -91,7 +91,7 @@ proc do_fsck_objects {} {\n proc hint_gc {} {\n \tset object_limit 8\n \tif {[is_Windows]} {\n-\t\tset object_limit 1\n+\t\tset object_limit 2\n \t}\n \n \tset objects_current [llength [glob \\\n-- \n1.6.5.rc0.164.g5f6b0\n"},{"id":"123073","messageId":"7vr5uasp4a.fsf@alter.siamese.dyndns.org","threadId":"20891","inReplyTo":"20090913160637.GA15256@localhost","subject":"Re: [PATCH] git-gui: suggest gc only when counting at least 2 objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-09-13T17:58:45Z","receivedAt":"2009-09-13T17:58:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Clemens Buchacher <drizzd@aon.at> writes:\n\n> On Windows, git-gui suggests running the garbage collector if it finds\n> 1 or more files in .git/objects/42 (as opposed to 8 files on other\n> platforms).\n\nSomebody cares to explain why this threashold number has to be different\nper platform in the first place?  Instead of bumping up to 2 like your\npatch did, what bad things would happen if you increased it to 8 on\nWindows?  Doesn't the same badness happen on non-Windows because they have\nthe threashold set to 8 already?\n"},{"id":"123077","messageId":"20090913184150.GA19209@localhost","threadId":"20891","inReplyTo":"7vr5uasp4a.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] git-gui: suggest gc only when counting at least 2 objects","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-13T18:41:50Z","receivedAt":"2009-09-13T18:41:50Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"On Sun, Sep 13, 2009 at 10:58:45AM -0700, Junio C Hamano wrote:\n> Somebody cares to explain why this threashold number has to be different\n> per platform in the first place? \n\nI really don't know. I vaguely remember someone claim that performance on\nWindows suffered from many loose objects more than on other platforms. I\ncan't find any discussion of it though.\n"},{"id":"123106","messageId":"20090913204433.GA8796@coredump.intra.peff.net","threadId":"20891","inReplyTo":"20090913184150.GA19209@localhost","subject":"Re: [PATCH] git-gui: suggest gc only when counting at least 2 objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-09-13T20:44:33Z","receivedAt":"2009-09-13T20:44:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Sep 13, 2009 at 08:41:50PM +0200, Clemens Buchacher wrote:\n\n> On Sun, Sep 13, 2009 at 10:58:45AM -0700, Junio C Hamano wrote:\n> > Somebody cares to explain why this threashold number has to be different\n> > per platform in the first place? \n> \n> I really don't know. I vaguely remember someone claim that performance on\n> Windows suffered from many loose objects more than on other platforms. I\n> can't find any discussion of it though.\n\nMaybe 8ff487c?\n\n-Peff\n"},{"id":"123112","messageId":"20090913211916.GA5029@localhost","threadId":"20891","inReplyTo":"20090913204433.GA8796@coredump.intra.peff.net","subject":"Re: [PATCH] git-gui: suggest gc only when counting at least 2 objects","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-13T21:19:25Z","receivedAt":"2009-09-13T21:19:25Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"On Sun, Sep 13, 2009 at 04:44:33PM -0400, Jeff King wrote:\n> On Sun, Sep 13, 2009 at 08:41:50PM +0200, Clemens Buchacher wrote:\n> \n> > On Sun, Sep 13, 2009 at 10:58:45AM -0700, Junio C Hamano wrote:\n> > > Somebody cares to explain why this threashold number has to be different\n> > > per platform in the first place? \n> > \n> > I really don't know. I vaguely remember someone claim that performance on\n> > Windows suffered from many loose objects more than on other platforms. I\n> > can't find any discussion of it though.\n> \n> Maybe 8ff487c?\n\nOk. But it's been 2 years since then and if I'm not mistaken, there have\nbeen a number of performance improvements to msysgit. So maybe it's time to\nrevisit that threshold.\n\nIf, on the other hand, requiring 2 objects really is too many, we should\nmaybe check at least two or four directories, which would greatly improve\nthe statistic.\n\nFor example, the probability of q directories containing q objects, for n\nobjects total is\n\nn \\ q\t1\t4\n50\t18%\t1%\n100\t32%\t7%\n200\t54%\t38%\n500\t86%\t95%\n\nClemens\n"},{"id":"123119","messageId":"20090913222044.GA17146@localhost","threadId":"20891","inReplyTo":"20090913211916.GA5029@localhost","subject":"[PATCH] git-gui: search 4 directories to improve statistic of gc hint","fromName":"Clemens Buchacher","fromEmail":"drizzd@aon.at","sentAt":"2009-09-13T22:20:44Z","receivedAt":"2009-09-13T22:20:44Z","isPatch":true,"sender":{"key":"drizzd@gmx.net","avatar":"https://avatars.githubusercontent.com/u/59082?v=4"},"body":"On Windows, git-gui suggests running the garbage collector if it finds\n1 or more files in .git/objects/42 (as opposed to 8 files on other\nplatforms). The probability of that happening if the repo contains\nabout 100 loose objects is 32%. The probability for the same to happen\nwhen searching 4 directories is only 8%, which is bit more reasonable.\n\nAlso remove $objects_limit from the message, because we already know\nthat we are above (or close to) that limit. Telling the user about\nthat number does not really give him any useful information.\n\nThe following octave script shows the probability for at least m*q\nobjects to be found in q subdirectories of .git/objects if n is the\ntotal number of objects.\n\nq = 4;\nm = [1 2 8];\nn = 0:10:2000;\n\nP = zeros(length(n), length(m));\nfor k = 1:length(n)\n        P(k, :) = 1-binocdf(q*m-1, n(k), q/(256-q));\nend\nplot(n, P);\n\nn \\ q   1       4\n50      18%     1%\n100     32%     8%\n200     54%     39%\n500     86%     96%\n\nSigned-off-by: Clemens Buchacher <drizzd@aon.at>\n---\n\n git-gui/lib/database.tcl |   21 ++++++++++-----------\n 1 files changed, 10 insertions(+), 11 deletions(-)\n\ndiff --git a/git-gui/lib/database.tcl b/git-gui/lib/database.tcl\nindex a18ac8b..d4e0bed 100644\n--- a/git-gui/lib/database.tcl\n+++ b/git-gui/lib/database.tcl\n@@ -89,27 +89,26 @@ proc do_fsck_objects {} {\n }\n \n proc hint_gc {} {\n-\tset object_limit 8\n+\tset ndirs 1\n+\tset limit 8\n \tif {[is_Windows]} {\n-\t\tset object_limit 1\n+\t\tset ndirs 4\n+\t\tset limit 1\n \t}\n \n-\tset objects_current [llength [glob \\\n-\t\t-directory [gitdir objects 42] \\\n+\tset count [llength [glob \\\n \t\t-nocomplain \\\n-\t\t-tails \\\n \t\t-- \\\n-\t\t*]]\n+\t\t[gitdir objects 4\\[0-[expr {$ndirs-1}]\\]/*]]]\n \n-\tif {$objects_current >= $object_limit} {\n-\t\tset objects_current [expr {$objects_current * 250}]\n-\t\tset object_limit    [expr {$object_limit    * 250}]\n+\tif {$count >= $limit * $ndirs} {\n+\t\tset objects_current [expr {$count * 256/$ndirs}]\n \t\tif {[ask_popup \\\n \t\t\t[mc \"This repository currently has approximately %i loose objects.\n \n-To maintain optimal performance it is strongly recommended that you compress the database when more than %i loose objects exist.\n+To maintain optimal performance it is strongly recommended that you compress the database.\n \n-Compress the database now?\" $objects_current $object_limit]] eq yes} {\n+Compress the database now?\" $objects_current]] eq yes} {\n \t\t\tdo_gc\n \t\t}\n \t}\n-- \n1.6.5.rc0.164.g5f6b0\n"},{"id":"123131","messageId":"20090914033958.GT1033@spearce.org","threadId":"20891","inReplyTo":"20090913211916.GA5029@localhost","subject":"Re: [PATCH] git-gui: suggest gc only when counting at least 2 objects","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-09-14T03:39:58Z","receivedAt":"2009-09-14T03:39:58Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Clemens Buchacher <drizzd@aon.at> wrote:\n> On Sun, Sep 13, 2009 at 04:44:33PM -0400, Jeff King wrote:\n> > On Sun, Sep 13, 2009 at 08:41:50PM +0200, Clemens Buchacher wrote:\n> > > On Sun, Sep 13, 2009 at 10:58:45AM -0700, Junio C Hamano wrote:\n> > > > Somebody cares to explain why this threashold number has to be different\n> > > > per platform in the first place? \n> > > \n> > > I really don't know. I vaguely remember someone claim that performance on\n> > > Windows suffered from many loose objects more than on other platforms. I\n> > > can't find any discussion of it though.\n> > \n> > Maybe 8ff487c?\n\nYes, it was 8ff487c.  Back then I was using Windows on a daily\nbasis and this was put into git-gui because Aunt Tillie couldn't\nremember do run a git-gc every so often, and performance would just\ndrop in the bucket.  It also quite a bit predates `git gc --auto`\nbeing sprinkled throughout the code base on various operations.\n\nAs to why its been 200 as the loose count estimate, that was just\na WAG based on what I observed on my desktop.  2000 on UNIX is\nusually fine, 2000 on Windows meant you waited an extra 30 seconds\nto perform an operation.\n \n> Ok. But it's been 2 years since then and if I'm not mistaken, there have\n> been a number of performance improvements to msysgit. So maybe it's time to\n> revisit that threshold.\n\nmsysgit may have improved, but at the time I was running Git on\nCygwin, and I doubt NTFS has really improved that much.\n \n> If, on the other hand, requiring 2 objects really is too many, we should\n> maybe check at least two or four directories, which would greatly improve\n> the statistic.\n\nI'm concerned about the FS cost of checking more directories, but\nthis is a one-time penalty on startup of git-gui so it might not\nbe too bad if it gets us a better estimate.\n \n-- \nShawn.\n"}]}