{"thread":{"id":"28841","subject":"[PATCH 0/4] fsck improvements","startedAt":"2011-11-04T15:47:46Z","lastAt":"2011-11-05T05:26:04Z","messageCount":10,"participants":["Nguyễn Thái Ngọc Duy","Junio C Hamano","Jeff King","Nguyen Thai Ngoc Duy"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"178829","messageId":"1320421670-518-1-git-send-email-pclouds@gmail.com","threadId":"28841","inReplyTo":null,"subject":"[PATCH 0/4] fsck improvements","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-04T15:47:46Z","receivedAt":"2011-11-04T15:47:46Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"So I looked at fsck to see if it's easy to support multithread and\nfound out I could reduce fsck time by half. So here it is. I think I\nhaven't made any mistakes but fsck is not my domain.\n\nThe last patch is reposted because it conflicts with the series. It\nalso prints progress when checking connnectivity.\n\nNguyễn Thái Ngọc Duy (4):\n fsck: return error code when verify_pack() goes wrong\n Stop verify_packfile() as soon as an error occurs\n fsck: avoid reading every object twice\n fsck: print progress\n\n Documentation/git-fsck.txt | 12 ++++++-\n builtin/fsck.c | 78 +++++++++++++++++++++++++++++++++++---------\n pack-check.c | 27 +++++++++++++--\n pack.h | 6 +++-\n 4 files changed, 101 insertions(+), 22 deletions(-)\n\n-- \n1.7.4.74.g639db\n"},{"id":"178830","messageId":"1320421670-518-2-git-send-email-pclouds@gmail.com","threadId":"28841","inReplyTo":"1320421670-518-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 1/4] fsck: return error code when verify_pack() goes wrong","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-04T15:47:47Z","receivedAt":"2011-11-04T15:47:47Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/fsck.c | 4 +++-\n 1 files changed, 3 insertions(+), 1 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex df1a88b..4ead98d 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -29,6 +29,7 @@ static int write_lost_and_found;\n static int verbose;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n+#define ERROR_PACK 04\n \n #ifdef NO_D_INO_IN_DIRENT\n #define SORT_DIRENT 0\n@@ -626,7 +627,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\tprepare_packed_git();\n \t\tfor (p = packed_git; p; p = p->next)\n \t\t\t/* verify gives error messages itself */\n-\t\t\tverify_pack(p);\n+\t\t\tif (verify_pack(p))\n+\t\t\t\terrors_found |= ERROR_PACK;\n \n \t\tfor (p = packed_git; p; p = p->next) {\n \t\t\tuint32_t j, num;\n-- \n1.7.4.74.g639db\n"},{"id":"178831","messageId":"1320421670-518-3-git-send-email-pclouds@gmail.com","threadId":"28841","inReplyTo":"1320421670-518-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 2/4] Stop verify_packfile() as soon as an error occurs","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-04T15:47:48Z","receivedAt":"2011-11-04T15:47:48Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n pack-check.c | 6 +++++-\n 1 files changed, 5 insertions(+), 1 deletions(-)\n\ndiff --git a/pack-check.c b/pack-check.c\nindex 0c19b6e..e33ea79 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -79,6 +79,8 @@ static int verify_packfile(struct packed_git *p,\n \t\terr = error(\"%s SHA1 does not match its index\",\n \t\t\t p->pack_name);\n \tunuse_pack(w_curs);\n+\tif (err)\n+\t\treturn err;\n \n \t/* Make sure everything reachable from idx is valid. Since we\n \t * have verified that nr_objects matches between idx and pack,\n@@ -106,11 +108,13 @@ static int verify_packfile(struct packed_git *p,\n \t\t\toff_t offset = entries[i].offset;\n \t\t\toff_t len = entries[i+1].offset - offset;\n \t\t\tunsigned int nr = entries[i].nr;\n-\t\t\tif (check_pack_crc(p, w_curs, offset, len, nr))\n+\t\t\tif (check_pack_crc(p, w_curs, offset, len, nr)) {\n \t\t\t\terr = error(\"index CRC mismatch for object %s \"\n \t\t\t\t\t \"from %s at offset %\"PRIuMAX\"\",\n \t\t\t\t\t sha1_to_hex(entries[i].sha1),\n \t\t\t\t\t p->pack_name, (uintmax_t)offset);\n+\t\t\t\tbreak;\n+\t\t\t}\n \t\t}\n \t\tdata = unpack_entry(p, entries[i].offset, &type, &size);\n \t\tif (!data) {\n-- \n1.7.4.74.g639db\n"},{"id":"178832","messageId":"1320421670-518-4-git-send-email-pclouds@gmail.com","threadId":"28841","inReplyTo":"1320421670-518-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 3/4] fsck: avoid reading every object twice","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-04T15:47:49Z","receivedAt":"2011-11-04T15:47:49Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"During verify_pack() all objects are read for SHA-1 check. Then\nfsck_sha1() is called on every object, which read the object again\n(fsck_sha1 -> parse_object -> read_sha1_file).\n\nAvoid reading an object twice, do fsck_sha1 while we have an object\nuncompressed data in verify_pack.\n\nOn git.git, with this patch I got:\n\n$ /usr/bin/time ./git fsck >/dev/null\n98.97user 0.90system 1:40.01elapsed 99%CPU (0avgtext+0avgdata 616624maxresident)k\n0inputs+0outputs (0major+194186minor)pagefaults 0swaps\n\nWithout it:\n\n$ /usr/bin/time ./git fsck >/dev/null\n231.23user 2.35system 3:53.82elapsed 99%CPU (0avgtext+0avgdata 636688maxresident)k\n0inputs+0outputs (0major+461629minor)pagefaults 0swaps\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/fsck.c | 42 +++++++++++++++++++++++++-----------------\n pack-check.c | 13 ++++++++++---\n pack.h | 5 ++++-\n 3 files changed, 39 insertions(+), 21 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 4ead98d..0603f64 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -282,14 +282,8 @@ static void check_connectivity(void)\n \t}\n }\n \n-static int fsck_sha1(const unsigned char *sha1)\n+static int fsck_obj(struct object *obj)\n {\n-\tstruct object *obj = parse_object(sha1);\n-\tif (!obj) {\n-\t\terrors_found |= ERROR_OBJECT;\n-\t\treturn error(\"%s: object corrupt or missing\",\n-\t\t\t sha1_to_hex(sha1));\n-\t}\n \tif (obj->flags & SEEN)\n \t\treturn 0;\n \tobj->flags |= SEEN;\n@@ -332,6 +326,29 @@ static int fsck_sha1(const unsigned char *sha1)\n \treturn 0;\n }\n \n+static int fsck_sha1(const unsigned char *sha1)\n+{\n+\tstruct object *obj = parse_object(sha1);\n+\tif (!obj) {\n+\t\terrors_found |= ERROR_OBJECT;\n+\t\treturn error(\"%s: object corrupt or missing\",\n+\t\t\t sha1_to_hex(sha1));\n+\t}\n+\treturn fsck_obj(obj);\n+}\n+\n+static int fsck_obj_buffer(const unsigned char *sha1, enum object_type type,\n+\t\t\t unsigned long size, void *buffer, int *eaten)\n+{\n+\tstruct object *obj;\n+\tobj = parse_object_buffer(sha1, type, size, buffer, eaten);\n+\tif (!obj) {\n+\t\terrors_found |= ERROR_OBJECT;\n+\t\treturn error(\"%s: object corrupt or missing\", sha1_to_hex(sha1));\n+\t}\n+\treturn fsck_obj(obj);\n+}\n+\n /*\n * This is the sorting chunk size: make it reasonably\n * big so that we can sort well..\n@@ -627,17 +644,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\tprepare_packed_git();\n \t\tfor (p = packed_git; p; p = p->next)\n \t\t\t/* verify gives error messages itself */\n-\t\t\tif (verify_pack(p))\n+\t\t\tif (verify_pack(p, fsck_obj_buffer))\n \t\t\t\terrors_found |= ERROR_PACK;\n-\n-\t\tfor (p = packed_git; p; p = p->next) {\n-\t\t\tuint32_t j, num;\n-\t\t\tif (open_pack_index(p))\n-\t\t\t\tcontinue;\n-\t\t\tnum = p->num_objects;\n-\t\t\tfor (j = 0; j < num; j++)\n-\t\t\t\tfsck_sha1(nth_packed_object_sha1(p, j));\n-\t\t}\n \t}\n \n \theads = 0;\ndiff --git a/pack-check.c b/pack-check.c\nindex e33ea79..372d6b2 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -42,7 +42,8 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,\n }\n \n static int verify_packfile(struct packed_git *p,\n-\t\tstruct pack_window **w_curs)\n+\t\t\t struct pack_window **w_curs,\n+\t\t\t verify_fn fn)\n {\n \toff_t index_size = p->index_size;\n \tconst unsigned char *index_base = p->index_data;\n@@ -129,6 +130,12 @@ static int verify_packfile(struct packed_git *p,\n \t\t\tfree(data);\n \t\t\tbreak;\n \t\t}\n+\t\tif (fn) {\n+\t\t\tint eaten = 0;\n+\t\t\tfn(entries[i].sha1, type, size, data, &eaten);\n+\t\t\tif (eaten)\n+\t\t\t\tdata = NULL;\n+\t\t}\n \t\tfree(data);\n \t}\n \tfree(entries);\n@@ -159,7 +166,7 @@ int verify_pack_index(struct packed_git *p)\n \treturn err;\n }\n \n-int verify_pack(struct packed_git *p)\n+int verify_pack(struct packed_git *p, verify_fn fn)\n {\n \tint err = 0;\n \tstruct pack_window *w_curs = NULL;\n@@ -168,7 +175,7 @@ int verify_pack(struct packed_git *p)\n \tif (!p->index_data)\n \t\treturn -1;\n \n-\terr |= verify_packfile(p, &w_curs);\n+\terr |= verify_packfile(p, &w_curs, fn);\n \tunuse_pack(&w_curs);\n \n \treturn err;\ndiff --git a/pack.h b/pack.h\nindex 722a54e..70f3c29 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -70,10 +70,13 @@ struct pack_idx_entry {\n \toff_t offset;\n };\n \n+\n+typedef int (*verify_fn)(const unsigned char*, enum object_type, unsigned long, void*, int*);\n+\n extern const char *write_idx_file(const char *index_name, struct pack_idx_entry **objects, int nr_objects, const struct pack_idx_option *, unsigned char *sha1);\n extern int check_pack_crc(struct packed_git *p, struct pack_window **w_curs, off_t offset, off_t len, unsigned int nr);\n extern int verify_pack_index(struct packed_git *);\n-extern int verify_pack(struct packed_git *);\n+extern int verify_pack(struct packed_git *, verify_fn fn);\n extern void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t);\n extern char *index_pack_lockfile(int fd);\n extern int encode_in_pack_object_header(enum object_type, uintmax_t, unsigned char *);\n-- \n1.7.4.74.g639db\n"},{"id":"178833","messageId":"1320421670-518-5-git-send-email-pclouds@gmail.com","threadId":"28841","inReplyTo":"1320421670-518-1-git-send-email-pclouds@gmail.com","subject":"[PATCH 4/4] fsck: print progress","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-04T15:47:50Z","receivedAt":"2011-11-04T15:47:50Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"fsck is usually a long process and it would be nice if it prints\nprogress from time to time.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n Documentation/git-fsck.txt | 12 +++++++++++-\n builtin/fsck.c | 40 ++++++++++++++++++++++++++++++++++++++--\n pack-check.c | 14 +++++++++++---\n pack.h | 3 ++-\n 4 files changed, 62 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt\nindex a2a508d..5245101 100644\n--- a/Documentation/git-fsck.txt\n+++ b/Documentation/git-fsck.txt\n@@ -10,7 +10,8 @@ SYNOPSIS\n --------\n [verse]\n 'git fsck' [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n-\t [--[no-]full] [--strict] [--verbose] [--lost-found] [*]\n+\t [--[no-]full] [--strict] [--verbose] [--lost-found]\n+\t [--[no-]progress] [*]\n \n DESCRIPTION\n -----------\n@@ -72,6 +73,15 @@ index file, all SHA1 references in .git/refs/*, and all reflogs (unless\n \ta blob, the contents are written into the file, rather than\n \tits object name.\n \n+--progress::\n+--no-progress::\n+\tWhen fsck is run in a terminal, it will show the progress.\n+\tThese options can force progress to be shown or not\n+\tregardless terminal check.\n++\n+Progress is not shown when --verbose is used. --progress is ignored\n+in this case.\n+\n It tests SHA1 and general object sanity, and it does full tracking of\n the resulting reachability and everything else. It prints out any\n corruption it finds (missing or bad objects), and if you use the\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 0603f64..c4b1ca6 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -11,6 +11,7 @@\n #include \"fsck.h\"\n #include \"parse-options.h\"\n #include \"dir.h\"\n+#include \"progress.h\"\n \n #define REACHABLE 0x0001\n #define SEEN 0x0002\n@@ -27,6 +28,7 @@ static const char *head_points_at;\n static int errors_found;\n static int write_lost_and_found;\n static int verbose;\n+static int show_progress = -1;\n #define ERROR_OBJECT 01\n #define ERROR_REACHABLE 02\n #define ERROR_PACK 04\n@@ -138,7 +140,11 @@ static int traverse_one_object(struct object *obj)\n \n static int traverse_reachable(void)\n {\n+\tstruct progress *progress = NULL;\n+\tunsigned int nr = 0;\n \tint result = 0;\n+\tif (show_progress)\n+\t\tprogress = start_progress_delay(\"Checking connectivity\", 0, 0, 2);\n \twhile (pending.nr) {\n \t\tstruct object_array_entry *entry;\n \t\tstruct object *obj;\n@@ -146,7 +152,9 @@ static int traverse_reachable(void)\n \t\tentry = pending.objects + --pending.nr;\n \t\tobj = entry->item;\n \t\tresult |= traverse_one_object(obj);\n+\t\tdisplay_progress(progress, ++nr);\n \t}\n+\tstop_progress(&progress);\n \treturn !!result;\n }\n \n@@ -530,15 +538,20 @@ static void get_default_heads(void)\n static void fsck_object_dir(const char *path)\n {\n \tint i;\n+\tstruct progress *progress = NULL;\n \n \tif (verbose)\n \t\tfprintf(stderr, \"Checking object directory\\n\");\n \n+\tif (show_progress)\n+\t\tprogress = start_progress(\"Checking object directories\", 256);\n \tfor (i = 0; i < 256; i++) {\n \t\tstatic char dir[4096];\n \t\tsprintf(dir, \"%s/%02x\", path, i);\n \t\tfsck_dir(i, dir);\n+\t\tdisplay_progress(progress, i+1);\n \t}\n+\tstop_progress(&progress);\n \tfsck_sha1_list();\n }\n \n@@ -609,6 +622,7 @@ static struct option fsck_opts[] = {\n \tOPT_BOOLEAN(0, \"strict\", &check_strict, \"enable more strict checking\"),\n \tOPT_BOOLEAN(0, \"lost-found\", &write_lost_and_found,\n \t\t\t\t\"write dangling objects in .git/lost-found\"),\n+\tOPT_BOOL (0, \"progress\", &show_progress, \"show progress\"),\n \tOPT_END(),\n };\n \n@@ -621,6 +635,12 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \tread_replace_refs = 0;\n \n \targc = parse_options(argc, argv, prefix, fsck_opts, fsck_usage, 0);\n+\n+\tif (show_progress == -1)\n+\t\tshow_progress = isatty(2);\n+\tif (verbose)\n+\t\tshow_progress = 0;\n+\n \tif (write_lost_and_found) {\n \t\tcheck_full = 1;\n \t\tinclude_reflogs = 0;\n@@ -640,12 +660,28 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \n \tif (check_full) {\n \t\tstruct packed_git *p;\n+\t\tuint32_t total = 0, count = 0;\n+\t\tstruct progress *progress = NULL;\n \n \t\tprepare_packed_git();\n-\t\tfor (p = packed_git; p; p = p->next)\n+\n+\t\tif (show_progress) {\n+\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\t\tif (open_pack_index(p))\n+\t\t\t\t\tcontinue;\n+\t\t\t\ttotal += p->num_objects;\n+\t\t\t}\n+\n+\t\t\tprogress = start_progress(\"Checking objects\", total);\n+\t\t}\n+\t\tfor (p = packed_git; p; p = p->next) {\n \t\t\t/* verify gives error messages itself */\n-\t\t\tif (verify_pack(p, fsck_obj_buffer))\n+\t\t\tif (verify_pack(p, fsck_obj_buffer,\n+\t\t\t\t\tprogress, count))\n \t\t\t\terrors_found |= ERROR_PACK;\n+\t\t\tcount += p->num_objects;\n+\t\t}\n+\t\tstop_progress(&progress);\n \t}\n \n \theads = 0;\ndiff --git a/pack-check.c b/pack-check.c\nindex 372d6b2..a3262af 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"pack.h\"\n #include \"pack-revindex.h\"\n+#include \"progress.h\"\n \n struct idx_entry {\n \toff_t offset;\n@@ -43,7 +44,9 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,\n \n static int verify_packfile(struct packed_git *p,\n \t\t\t struct pack_window **w_curs,\n-\t\t\t verify_fn fn)\n+\t\t\t verify_fn fn,\n+\t\t\t struct progress *progress, uint32_t base_count)\n+\n {\n \toff_t index_size = p->index_size;\n \tconst unsigned char *index_base = p->index_data;\n@@ -136,8 +139,12 @@ static int verify_packfile(struct packed_git *p,\n \t\t\tif (eaten)\n \t\t\t\tdata = NULL;\n \t\t}\n+\t\tif (((base_count + i) & 1023) == 0)\n+\t\t\tdisplay_progress(progress, base_count + i);\n \t\tfree(data);\n+\n \t}\n+\tdisplay_progress(progress, base_count + i);\n \tfree(entries);\n \n \treturn err;\n@@ -166,7 +173,8 @@ int verify_pack_index(struct packed_git *p)\n \treturn err;\n }\n \n-int verify_pack(struct packed_git *p, verify_fn fn)\n+int verify_pack(struct packed_git *p, verify_fn fn,\n+\t\tstruct progress *progress, uint32_t base_count)\n {\n \tint err = 0;\n \tstruct pack_window *w_curs = NULL;\n@@ -175,7 +183,7 @@ int verify_pack(struct packed_git *p, verify_fn fn)\n \tif (!p->index_data)\n \t\treturn -1;\n \n-\terr |= verify_packfile(p, &w_curs, fn);\n+\terr |= verify_packfile(p, &w_curs, fn, progress, base_count);\n \tunuse_pack(&w_curs);\n \n \treturn err;\ndiff --git a/pack.h b/pack.h\nindex 70f3c29..324a1d7 100644\n--- a/pack.h\n+++ b/pack.h\n@@ -71,12 +71,13 @@ struct pack_idx_entry {\n };\n \n \n+struct progress;\n typedef int (*verify_fn)(const unsigned char*, enum object_type, unsigned long, void*, int*);\n \n extern const char *write_idx_file(const char *index_name, struct pack_idx_entry **objects, int nr_objects, const struct pack_idx_option *, unsigned char *sha1);\n extern int check_pack_crc(struct packed_git *p, struct pack_window **w_curs, off_t offset, off_t len, unsigned int nr);\n extern int verify_pack_index(struct packed_git *);\n-extern int verify_pack(struct packed_git *, verify_fn fn);\n+extern int verify_pack(struct packed_git *, verify_fn fn, struct progress *, uint32_t);\n extern void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t);\n extern char *index_pack_lockfile(int fd);\n extern int encode_in_pack_object_header(enum object_type, uintmax_t, unsigned char *);\n-- \n1.7.4.74.g639db\n"},{"id":"178849","messageId":"7v1utnd8yf.fsf@alter.siamese.dyndns.org","threadId":"28841","inReplyTo":"1320421670-518-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 0/4] fsck improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-04T18:43:52Z","receivedAt":"2011-11-04T18:43:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I think patches 1, 3 and 4 all are good ideas from a cursory look.\n\nI am not sure what purpose patch 2 serves, though. When we find a checksum\nmismatch for an object in a packstream due to a single-bit error, we would\nstill be able to salvage other objects in other parts of the pack as long\nas we have a good .idx file, and in such a case, wouldn't it be better if\nwe attempted to find as many corrupt objects that we know we cannot\nrecover as possible and tell the user about them, so that they can be\nskipped during the recovery process?\n"},{"id":"178853","messageId":"20111104201416.GA26591@sigill.intra.peff.net","threadId":"28841","inReplyTo":"1320421670-518-5-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH 4/4] fsck: print progress","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-11-04T20:14:16Z","receivedAt":"2011-11-04T20:14:16Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Nov 04, 2011 at 10:47:50PM +0700, Nguyen Thai Ngoc Duy wrote:\n\n> static int traverse_reachable(void)\n> {\n> +\tstruct progress *progress = NULL;\n> +\tunsigned int nr = 0;\n> \tint result = 0;\n> +\tif (show_progress)\n> +\t\tprogress = start_progress_delay(\"Checking connectivity\", 0, 0, 2);\n> \twhile (pending.nr) {\n> \t\tstruct object_array_entry *entry;\n> \t\tstruct object *obj;\n> @@ -146,7 +152,9 @@ static int traverse_reachable(void)\n> \t\tentry = pending.objects + --pending.nr;\n> \t\tobj = entry->item;\n> \t\tresult |= traverse_one_object(obj);\n> +\t\tdisplay_progress(progress, ++nr);\n> \t}\n> +\tstop_progress(&progress);\n> \treturn !!result;\n> }\n\nThanks, this is a good place to put a progress meter, too. If you're\nfeeling like pushing this topic further, \"git prune\" might be a good\nplace for a progress meter, too. It does a similar traversal[1] for\nreachability, and makes \"git gc\" appear to hang at the end (we have nice\nprogress meters for packing, but it takes something like 25 seconds to\nrun \"git prune\" at the end, during which we are silent).\n\n-Peff\n\n[1] I wonder why fsck doesn't use mark_reachable from reachable.c.\n"},{"id":"178867","messageId":"CACsJy8Ch0m5KQ--VWzsmyNq2R-uHsvSYofHP8L+qcvhRrqgoKQ@mail.gmail.com","threadId":"28841","inReplyTo":"7v1utnd8yf.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 0/4] fsck improvements","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-05T03:18:43Z","receivedAt":"2011-11-05T03:18:43Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/11/5 Junio C Hamano :\n> I am not sure what purpose patch 2 serves, though. When we find a checksum\n> mismatch for an object in a packstream due to a single-bit error, we would\n> still be able to salvage other objects in other parts of the pack as long\n> as we have a good .idx file, and in such a case, wouldn't it be better if\n> we attempted to find as many corrupt objects that we know we cannot\n> recover as possible and tell the user about them, so that they can be\n> skipped during the recovery process?\n\nIt's the inconsistency in that for(;;) loop. If we are going to\nsalvage as many objects as we could, should we do \"continue;\" instead\nof \"break;\" when unpack_entry() or check_sha1_signature() fails?\n-- \nDuy\n"},{"id":"178866","messageId":"CACsJy8AoEa3gR5dzTcscJRqjrfkv5_J+7Q2UVaYSjvSQyKJssA@mail.gmail.com","threadId":"28841","inReplyTo":"20111104201416.GA26591@sigill.intra.peff.net","subject":"Re: [PATCH 4/4] fsck: print progress","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-11-05T03:26:48Z","receivedAt":"2011-11-05T03:26:48Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2011/11/5 Jeff King :\n> Thanks, this is a good place to put a progress meter, too. If you're\n> feeling like pushing this topic further, \"git prune\" might be a good\n> place for a progress meter, too. It does a similar traversal[1] for\n> reachability, and makes \"git gc\" appear to hang at the end (we have nice\n> progress meters for packing, but it takes something like 25 seconds to\n> run \"git prune\" at the end, during which we are silent).\n>\n> -Peff\n>\n> [1] I wonder why fsck doesn't use mark_reachable from reachable.c.\n\nThanks. I'll have a look.\n-- \nDuy\n"},{"id":"178871","messageId":"7vehxn9m37.fsf@alter.siamese.dyndns.org","threadId":"28841","inReplyTo":"CACsJy8Ch0m5KQ--VWzsmyNq2R-uHsvSYofHP8L+qcvhRrqgoKQ@mail.gmail.com","subject":"Re: [PATCH 0/4] fsck improvements","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-11-05T05:26:04Z","receivedAt":"2011-11-05T05:26:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyen Thai Ngoc Duy writes:\n\n> 2011/11/5 Junio C Hamano :\n>> I am not sure what purpose patch 2 serves, though. When we find a checksum\n>> mismatch for an object in a packstream due to a single-bit error, we would\n>> still be able to salvage other objects in other parts of the pack as long\n>> as we have a good .idx file, and in such a case, wouldn't it be better if\n>> we attempted to find as many corrupt objects that we know we cannot\n>> recover as possible and tell the user about them, so that they can be\n>> skipped during the recovery process?\n>\n> It's the inconsistency in that for(;;) loop. If we are going to\n> salvage as many objects as we could, should we do \"continue;\" instead\n> of \"break;\" when unpack_entry() or check_sha1_signature() fails?\n\nI do not think making things worse for the sake of \"consistency\" is a good\nsell ;-). How hard would it be to make it also continue when these report\na corruption?\n"}]}