{"thread":{"id":"10571","subject":"[PATCH 0/3] yet more progress display stuff","startedAt":"2007-11-01T20:59:54Z","lastAt":"2007-11-02T04:59:50Z","messageCount":5,"participants":["Nicolas Pitre","Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"57861","messageId":"1193950797-29631-1-git-send-email-nico@cam.org","threadId":"10571","inReplyTo":null,"subject":"[PATCH 0/3] yet more progress display stuff","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-11-01T20:59:54Z","receivedAt":"2007-11-01T20:59:54Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"Here's 3 additional patches.  The first one revert one change already\nburried deep in the topic branch based on Shawn's concern about\nprune-packed progress display handling.\n\nThe second is a generic improvement to the throughput display.\n\nThe last is my reimplementation of total transfer byte count, replacing\nShawn's proposed patch which suffered from a few issues.\n\n\nNicolas\n"},{"id":"57862","messageId":"1193950797-29631-2-git-send-email-nico@cam.org","threadId":"10571","inReplyTo":"1193950797-29631-1-git-send-email-nico@cam.org","subject":"[PATCH 1/3] return the prune-packed progress display to the inner loop","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-11-01T20:59:55Z","receivedAt":"2007-11-01T20:59:55Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"This reverts commit 0e549137966feb016927a827fb6e359aec8264a3 so to return\nto the same state as commit b5d72f0a4cd3cce945ca0d37e4fa0ebbfcdcdb52.\n\nOn Wed, 31 Oct 2007, Shawn O. Pearce wrote:\n> During my testing with a 40,000 loose object case (yea, I fully\n> unpacked a git.git clone I had laying around) my system stalled\n> hard in the first object directory.  A *lot* longer than 1 second.\n> So I got no progress meter for a long time, and then a progress\n> meter appeared on the second directory.\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n builtin-prune-packed.c |    3 +--\n 1 files changed, 1 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin-prune-packed.c b/builtin-prune-packed.c\nindex f4287da..23faf31 100644\n--- a/builtin-prune-packed.c\n+++ b/builtin-prune-packed.c\n@@ -15,8 +15,6 @@ static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)\n \tstruct dirent *de;\n \tchar hex[40];\n \n-\tdisplay_progress(progress, i + 1);\n-\n \tsprintf(hex, \"%02x\", i);\n \twhile ((de = readdir(dir)) != NULL) {\n \t\tunsigned char sha1[20];\n@@ -32,6 +30,7 @@ static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)\n \t\t\tprintf(\"rm -f %s\\n\", pathname);\n \t\telse if (unlink(pathname) < 0)\n \t\t\terror(\"unable to unlink %s\", pathname);\n+\t\tdisplay_progress(progress, i + 1);\n \t}\n \tpathname[len] = 0;\n \trmdir(pathname);\n-- \n1.5.3.4.279.gb2d9d-dirty\n"},{"id":"57864","messageId":"1193950797-29631-3-git-send-email-nico@cam.org","threadId":"10571","inReplyTo":"1193950797-29631-2-git-send-email-nico@cam.org","subject":"[PATCH 2/3] make sure throughput display gets updated even if progress doesn't move","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-11-01T20:59:56Z","receivedAt":"2007-11-01T20:59:56Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"Currently the progress/throughput display update happens only through\ndisplay_progress().  If the progress based on object count remains\nunchanged because a large object is being received, the latest throughput\nwon't be displayed.  The display update should occur through\ndisplay_throughput() as well.\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n progress.c |    3 +++\n 1 files changed, 3 insertions(+), 0 deletions(-)\n\ndiff --git a/progress.c b/progress.c\nindex 34a5961..39d5d2c 100644\n--- a/progress.c\n+++ b/progress.c\n@@ -160,6 +160,9 @@ void display_throughput(struct progress *progress, unsigned long n)\n \t\ttp->last_misecs[tp->idx] = misecs;\n \t\ttp->idx = (tp->idx + 1) % TP_IDX_MAX;\n \t\ttp->count = 0;\n+\n+\t\tif (progress->last_value != -1 && progress_update)\n+\t\t\tdisplay(progress, progress->last_value, 0);\n \t}\n }\n \n-- \n1.5.3.4.279.gb2d9d-dirty\n"},{"id":"57863","messageId":"1193950797-29631-4-git-send-email-nico@cam.org","threadId":"10571","inReplyTo":"1193950797-29631-3-git-send-email-nico@cam.org","subject":"[PATCH 3/3] Show total transferred as part of throughput progress","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-11-01T20:59:57Z","receivedAt":"2007-11-01T20:59:57Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"Right now it is infeasible to offer to the user a reasonable concept\nof when a clone will be complete as we aren't able to come up with\nthe final pack size until after we have actually transferred the\nentire thing to the client.  However in many cases users can work\nwith a rough rule-of-thumb; for example it is somewhat well known\nthat git.git is about 16 MiB today and that linux-2.6.git is over\n120 MiB.\n\nWe now show the total amount of data we have transferred over\nthe network as part of the throughput meter, organizing it in\n\"human friendly\" terms like `ls -h` would do.  Users can glance at\nthis, see that the total transferred size is about 3 MiB, see the\nthroughput of X KiB/sec, and determine a reasonable figure of about\nwhen the clone will be complete, assuming they know the rough size\nof the source repository or are able to obtain it.\n\nThis is also a helpful indicator that there is progress being made\neven if we stall on a very large object.  The thoughput meter may\nremain relatively constant and the percentage complete and object\ncount won't be changing, but the total transferred will be increasing\nas additional data is received for this object.\n\n[from an initial proposal from Shawn O. Pearce]\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n progress.c |   29 ++++++++++++++++++++++++++---\n 1 files changed, 26 insertions(+), 3 deletions(-)\n\ndiff --git a/progress.c b/progress.c\nindex 39d5d2c..3f6a602 100644\n--- a/progress.c\n+++ b/progress.c\n@@ -15,13 +15,14 @@\n \n struct throughput {\n \tstruct timeval prev_tv;\n+\toff_t total;\n \tunsigned long count;\n \tunsigned long avg_bytes;\n \tunsigned long last_bytes[TP_IDX_MAX];\n \tunsigned int avg_misecs;\n \tunsigned int last_misecs[TP_IDX_MAX];\n \tunsigned int idx;\n-\tchar display[20];\n+\tchar display[32];\n };\n \n struct progress {\n@@ -128,6 +129,7 @@ void display_throughput(struct progress *progress, unsigned long n)\n \t\treturn;\n \t}\n \n+\ttp->total += n;\n \ttp->count += n;\n \n \t/*\n@@ -149,11 +151,32 @@ void display_throughput(struct progress *progress, unsigned long n)\n \tmisecs += (int)(tv.tv_usec - tp->prev_tv.tv_usec) / 977;\n \n \tif (misecs > 512) {\n+\t\tint l = sizeof(tp->display);\n \t\ttp->prev_tv = tv;\n \t\ttp->avg_bytes += tp->count;\n \t\ttp->avg_misecs += misecs;\n-\t\tsnprintf(tp->display, sizeof(tp->display),\n-\t\t\t \", %lu KiB/s\", tp->avg_bytes / tp->avg_misecs);\n+\n+\t\tif (tp->total > 1 << 30) {\n+\t\t\tl -= snprintf(tp->display, l, \", %u.%2.2u GiB\",\n+\t\t\t\t      (int)(tp->total >> 30),\n+\t\t\t\t      (int)(tp->total & ((1 << 30) - 1)) / 10737419);\n+\t\t} else if (tp->total > 1 << 20) {\n+\t\t\tl -= snprintf(tp->display, l, \", %u.%2.2u MiB\",\n+\t\t\t\t      (int)(tp->total >> 20),\n+\t\t\t\t      ((int)(tp->total & ((1 << 20) - 1))\n+\t\t\t\t       * 100) >> 20);\n+\t\t} else if (tp->total > 1 << 10) {\n+\t\t\tl -= snprintf(tp->display, l, \", %u.%2.2u KiB\",\n+\t\t\t\t      (int)(tp->total >> 10),\n+\t\t\t\t      ((int)(tp->total & ((1 << 10) - 1))\n+\t\t\t\t       * 100) >> 10);\n+\t\t} else {\n+\t\t\tl -= snprintf(tp->display, l, \", %u bytes\",\n+\t\t\t\t      (int)tp->total);\n+\t\t}\n+\t\tsnprintf(tp->display + sizeof(tp->display) - l, l,\n+\t\t\t \" | %lu KiB/s\", tp->avg_bytes / tp->avg_misecs);\n+\n \t\ttp->avg_bytes -= tp->last_bytes[tp->idx];\n \t\ttp->avg_misecs -= tp->last_misecs[tp->idx];\n \t\ttp->last_bytes[tp->idx] = tp->count;\n-- \n1.5.3.4.279.gb2d9d-dirty\n"},{"id":"57925","messageId":"20071102045950.GZ14735@spearce.org","threadId":"10571","inReplyTo":"1193950797-29631-4-git-send-email-nico@cam.org","subject":"Re: [PATCH 3/3] Show total transferred as part of throughput progress","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-11-02T04:59:50Z","receivedAt":"2007-11-02T04:59:50Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Nicolas Pitre <nico@cam.org> wrote:\n> Right now it is infeasible to offer to the user a reasonable concept\n> of when a clone will be complete as we aren't able to come up with\n> the final pack size until after we have actually transferred the\n> entire thing to the client.  However in many cases users can work\n> with a rough rule-of-thumb; for example it is somewhat well known\n> that git.git is about 16 MiB today and that linux-2.6.git is over\n> 120 MiB.\n...\n> [from an initial proposal from Shawn O. Pearce]\n\nThanks for rewriting this.  I agree, your replacement patch is\nmuch better looking than my proposal.  I also see Junio has already\napplied it to next.  Excellent.\n \n-- \nShawn.\n"}]}