Re: [PATCH] cg-pull: summarize the number of pulled objects

From

Jonas Fonseca <fonseca@diku.dk>

Date

May 30, 2005, 15:18 UTC

Message-ID

<20050530151817.GB28681@diku.dk>

In-Reply-To

<20050530082054.GB1036@pasky.ji.cz>

Petr Baudis <pasky@ucw.cz> wrote Mon, May 30, 2005:

> Dear diary, on Mon, May 30, 2005 at 03:56:50AM CEST, I got a letter
> where Jonas Fonseca <fonseca@diku.dk> told me that...
> > Show cg-pull progress by summarizing the very verbose output of the pull
> > backends into a continously updated line specifying the number of
> > objects which have already been pulled.
> > 		     
> > Signed-off-by: Jonas Fonseca <fonseca@diku.dk>
> > ---
> > 
> > Straight from the bloat department, perhaps, but it is nice to not have
> > the terminal backlog ruined and the object count is quite nice too. :)
> > 
> > Interesting, it counts 4950 objects when pulling over rsync and 4454
> > objects when pulling locally. Didn't test HTTP pulling other than to see
> > if the "got <sha>" lines was matched correctly.
> 
> It is nice, but actually losing information for me. It's ok for the HTTP
> and SSH "smart pulls", but if I pull over rsync, I have progress
> indication based on the first two digits - assuming roughly even
> distribution of hashes, I can see that when I cross 80/ I'm in the half
> etc. So what about showing some percentage or something in that case?

[ Ok, I've done the percentage thing. The number of bytes are also
  tracked. I also tried to return the correct exit code from the pull
  function using PIPESTATUS. ]

[PATCH] cg-pull: summarize the number of pulled objects

Signed-off-by: Jonas Fonseca <fonseca@diku.dk>

---

 cg-pull |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 51 insertions(+), 4 deletions(-)

diff --git a/cg-pull b/cg-pull
--- a/cg-pull
+++ b/cg-pull
@@ -29,6 +29,49 @@ if echo "$uri" | grep -q '#'; then
 	uri=$(echo $uri | cut -d '#' -f 1)
 fi
 
+pull_progress() {
+	percentage=""
+	objects=0
+	last_objects=0
+	size=0
+
+	while read line; do
+		object=
+
+		case "$line" in
+		link*|symlink*|\
+		"got "[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]*)
+			# Convert the sha to an object path
+			object=$(echo "$line" | sed 's,.* \([a-f0-9][a-f0-9]\),\1/,')
+			;;
+
+		[a-f0-9][a-f0-9]/[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]*)
+			object="$line"
+			# Estimate percentage done using the position of
+			# the object subdir. It might not get all the way
+			# up to 100% ...
+			position=$(echo "$line" | cut -d/ -f 1)
+			percentage=", $((0x$position * 100 / 0xff))% done"
+			;;
+
+		*)
+			if [ "$last_objects" != "$objects" ]; then
+				last_objects="$objects"
+				echo;
+			fi
+			echo "$line"
+			continue
+			;;
+		esac 
+
+		object="$_git_objects/$object"
+		size=$(($size + $(stat -c '%s' "$object" 2>/dev/null)))
+		objects=$(($objects + 1));
+
+		echo -ne "progress: $objects objects, $size bytes$percentage\r"
+	done;
+	[ "$last_objects" != "$objects" ] && echo
+}
 
 fetch_rsync () {
 	redir=
@@ -62,7 +105,8 @@ fetch_rsync () {
 }
 
 pull_rsync () {
-	fetch_rsync -s -u -d "$2/objects" "$_git_objects"
+	fetch_rsync -s -u -d "$2/objects" "$_git_objects" | pull_progress
+	return ${PIPESTATUS[0]}
 }
 
 
@@ -107,7 +151,8 @@ fetch_http () {
 }
 
 pull_http () {
-	git-http-pull -a -v "$(cat "$_git/refs/heads/$1")" "$2/"
+	(git-http-pull -a -v "$(cat "$_git/refs/heads/$1")" "$2/" 2>&1 /dev/null) | pull_progress
+	return ${PIPESTATUS[0]}
 }
 
 
@@ -142,7 +187,8 @@ fetch_ssh () {
 }
 
 pull_ssh () {
-	git-rpull -a -v "$(cat "$_git/refs/heads/$1")" "$2"
+	(git-rpull -a -v "$(cat "$_git/refs/heads/$1")" "$2" 2>&1 /dev/null) | pull_progress
+	return ${PIPESTATUS[0]}
 }
 
 
@@ -170,7 +216,8 @@ fetch_local () {
 }
 
 pull_local () {
-	git-local-pull -a -l -v "$(cat "$_git/refs/heads/$1")" "$2"
+	(git-local-pull -a -l -v "$(cat "$_git/refs/heads/$1")" "$2" 2>&1 /dev/null) | pull_progress
+	return ${PIPESTATUS[0]}
 }
 
 if echo "$uri" | grep -q "^http://"; then

-- 
Jonas Fonseca