{"thread":{"id":"5006","subject":"local clone performance","startedAt":"2006-07-26T23:38:00Z","lastAt":"2006-07-27T18:36:19Z","messageCount":3,"participants":["Jon Smirl","Petr Baudis"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"24178","messageId":"9e4733910607261637w75c99a2ehcb25b6c81449d298@mail.gmail.com","threadId":"5006","inReplyTo":null,"subject":"local clone performance","fromName":"Jon Smirl","fromEmail":"jonsmirl@gmail.com","sentAt":"2006-07-26T23:38:00Z","receivedAt":"2006-07-26T23:38:00Z","isPatch":false,"sender":{"key":"jonsmirl@gmail.com","avatar":"https://gravatar.com/avatar/cff3bf5bfdfa6708b905712ff91f0f9b8aaca161659f38c02b787920d5d28b7e?d=mp&s=160"},"body":"Comparing git-clone to cg-clone\n\n[jonsmirl@jonsmirl apps]$ time git-clone git foo\n......\nreal    0m5.755s\nuser    0m4.548s\nsys     0m0.512s\n\n[jonsmirl@jonsmirl apps]$ time cg-clone git foo\n....\nreal    0m18.970s\nuser    0m10.737s\nsys     0m1.392s\n\nWhy does it take cg clone so long to set up the missing tags?\ngit-clone is over 3x faster.\n\n-- \nJon Smirl\njonsmirl@gmail.com\n"},{"id":"24242","messageId":"20060727181554.GD13776@pasky.or.cz","threadId":"5006","inReplyTo":"9e4733910607261637w75c99a2ehcb25b6c81449d298@mail.gmail.com","subject":"Re: local clone performance","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-07-27T18:15:54Z","receivedAt":"2006-07-27T18:15:54Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jul 27, 2006 at 01:38:00AM CEST, I got a letter\nwhere Jon Smirl <jonsmirl@gmail.com> said that...\n> Comparing git-clone to cg-clone\n> \n> [jonsmirl@jonsmirl apps]$ time git-clone git foo\n> ......\n> real    0m5.755s\n> user    0m4.548s\n> sys     0m0.512s\n> \n> [jonsmirl@jonsmirl apps]$ time cg-clone git foo\n> ....\n> real    0m18.970s\n> user    0m10.737s\n> sys     0m1.392s\n> \n> Why does it take cg clone so long to set up the missing tags?\n> git-clone is over 3x faster.\n\n  How many tags do you have?\n\n  It's true that cg-clone does not scale very well with big amounts of\ntags, because it won't fetch tagged objects you wouldn't get otherwise\n(so if you tagged some random huge blob or a history line which is not\npart of any branch you have, cg-clone won't grab it all). This\nunfortunately costs us one special fetch invocation per tag; I will look\ninto hacking git-local-fetch to accept multiple commit ids at once, that\nshould speed it up considerably.\n\n  That said, with cg-clone -a I will be able to cut that difference\nentirely and blindly take everything from the other repository.\n\n  Could you please try the patch below and check if it has at least\nany measurable impact on the performance at all? Thanks.\n\ndiff --git a/cg-fetch b/cg-fetch\nindex a6e6959..23e0e77 100755\n--- a/cg-fetch\n+++ b/cg-fetch\n@@ -54,7 +54,6 @@ _git_wc_unneeded=1\n \n fetch_progress()\n {\n-\t[ $verbose -ge 2 ] && exec cat\n \tif [ -t 1 ]; then\n \t\texec \"${COGITO_LIB}\"cg-Xfetchprogress \"$_git_objects\"\n \telse\n@@ -166,7 +165,11 @@ fetch_http()\n {\n \twhead=\n \t[ \"$3\" ] && whead=\"-w $3\"\n-\t(git-http-fetch -a -v $whead $recovery \"$1\" \"$2/\" 2>&1 /dev/null) | fetch_progress\n+\tif [ $verbose -ge 2 ]; then\n+\t\tgit-http-fetch -a -v $whead $recovery \"$1\" \"$2/\"\n+\telse\n+\t\t(git-http-fetch -a -v $whead $recovery \"$1\" \"$2/\" 2>&1 /dev/null) | fetch_progress\n+\tfi\n \treturn ${PIPESTATUS[0]}\n }\n \n@@ -197,7 +200,11 @@ fetch_local()\n {\n \twhead=\n \t[ \"$3\" ] && whead=\"-w $3\"\n-\t(git-local-fetch -a -l -v $whead $recovery \"$1\" \"$2\" 2>&1 /dev/null) | fetch_progress\n+\tif [ $verbose -ge 2 ]; then\n+\t\tgit-local-fetch -a -l -v $whead $recovery \"$1\" \"$2\"\n+\telse\n+\t\t(git-local-fetch -a -l -v $whead $recovery \"$1\" \"$2\" 2>&1 /dev/null) | fetch_progress\n+\tfi\n \treturn ${PIPESTATUS[0]}\n }\n \n@@ -230,6 +237,7 @@ fetch_tags()\n \t\t\t# if so, fetch the tag -- which should be\n \t\t\t# a cheap operation -- to complete the chain.\n \t\t\techo -n \"Missing tag ${tagname#tags/}... \"\n+\t\t\tlocal verbose=2\n \t\t\tif $fetch \"$tagname\" \"$uri\" \"$tagname\" 2>/dev/null >&2; then\n \t\t\t\techo \"retrieved\"\n \t\t\telse\n\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nSnow falling on Perl. White noise covering line noise.\nHides all the bugs too. -- J. Putnam\n"},{"id":"24243","messageId":"9e4733910607271136n55545959y3dbff97b1e5efe31@mail.gmail.com","threadId":"5006","inReplyTo":"20060727181554.GD13776@pasky.or.cz","subject":"Re: local clone performance","fromName":"Jon Smirl","fromEmail":"jonsmirl@gmail.com","sentAt":"2006-07-27T18:36:19Z","receivedAt":"2006-07-27T18:36:19Z","isPatch":false,"sender":{"key":"jonsmirl@gmail.com","avatar":"https://gravatar.com/avatar/cff3bf5bfdfa6708b905712ff91f0f9b8aaca161659f38c02b787920d5d28b7e?d=mp&s=160"},"body":"On 7/27/06, Petr Baudis <pasky@suse.cz> wrote:\n> Dear diary, on Thu, Jul 27, 2006 at 01:38:00AM CEST, I got a letter\n> where Jon Smirl <jonsmirl@gmail.com> said that...\n> > Comparing git-clone to cg-clone\n> >\n> > [jonsmirl@jonsmirl apps]$ time git-clone git foo\n> > ......\n> > real    0m5.755s\n> > user    0m4.548s\n> > sys     0m0.512s\n> >\n> > [jonsmirl@jonsmirl apps]$ time cg-clone git foo\n> > ....\n> > real    0m18.970s\n> > user    0m10.737s\n> > sys     0m1.392s\n> >\n> > Why does it take cg clone so long to set up the missing tags?\n> > git-clone is over 3x faster.\n>\n>   How many tags do you have?\n\nI am cloning the git source tree. It looks to have about 40 tags.\n\n>   It's true that cg-clone does not scale very well with big amounts of\n> tags, because it won't fetch tagged objects you wouldn't get otherwise\n> (so if you tagged some random huge blob or a history line which is not\n> part of any branch you have, cg-clone won't grab it all). This\n> unfortunately costs us one special fetch invocation per tag; I will look\n> into hacking git-local-fetch to accept multiple commit ids at once, that\n> should speed it up considerably.\n>\n>   That said, with cg-clone -a I will be able to cut that difference\n> entirely and blindly take everything from the other repository.\n>\n>   Could you please try the patch below and check if it has at least\n> any measurable impact on the performance at all? Thanks.\n\nNew timings\nreal    0m11.259s\nuser    0m9.977s\nsys     0m1.280s\n\nThat's a 40% improvement.\n\n> diff --git a/cg-fetch b/cg-fetch\n> index a6e6959..23e0e77 100755\n> --- a/cg-fetch\n> +++ b/cg-fetch\n> @@ -54,7 +54,6 @@ _git_wc_unneeded=1\n>\n>  fetch_progress()\n>  {\n> -       [ $verbose -ge 2 ] && exec cat\n>         if [ -t 1 ]; then\n>                 exec \"${COGITO_LIB}\"cg-Xfetchprogress \"$_git_objects\"\n>         else\n> @@ -166,7 +165,11 @@ fetch_http()\n>  {\n>         whead=\n>         [ \"$3\" ] && whead=\"-w $3\"\n> -       (git-http-fetch -a -v $whead $recovery \"$1\" \"$2/\" 2>&1 /dev/null) | fetch_progress\n> +       if [ $verbose -ge 2 ]; then\n> +               git-http-fetch -a -v $whead $recovery \"$1\" \"$2/\"\n> +       else\n> +               (git-http-fetch -a -v $whead $recovery \"$1\" \"$2/\" 2>&1 /dev/null) | fetch_progress\n> +       fi\n>         return ${PIPESTATUS[0]}\n>  }\n>\n> @@ -197,7 +200,11 @@ fetch_local()\n>  {\n>         whead=\n>         [ \"$3\" ] && whead=\"-w $3\"\n> -       (git-local-fetch -a -l -v $whead $recovery \"$1\" \"$2\" 2>&1 /dev/null) | fetch_progress\n> +       if [ $verbose -ge 2 ]; then\n> +               git-local-fetch -a -l -v $whead $recovery \"$1\" \"$2\"\n> +       else\n> +               (git-local-fetch -a -l -v $whead $recovery \"$1\" \"$2\" 2>&1 /dev/null) | fetch_progress\n> +       fi\n>         return ${PIPESTATUS[0]}\n>  }\n>\n> @@ -230,6 +237,7 @@ fetch_tags()\n>                         # if so, fetch the tag -- which should be\n>                         # a cheap operation -- to complete the chain.\n>                         echo -n \"Missing tag ${tagname#tags/}... \"\n> +                       local verbose=2\n>                         if $fetch \"$tagname\" \"$uri\" \"$tagname\" 2>/dev/null >&2; then\n>                                 echo \"retrieved\"\n>                         else\n>\n>\n> --\n>                                 Petr \"Pasky\" Baudis\n> Stuff: http://pasky.or.cz/\n> Snow falling on Perl. White noise covering line noise.\n> Hides all the bugs too. -- J. Putnam\n>\n\n\n-- \nJon Smirl\njonsmirl@gmail.com\n"}]}