threads / rfc / 30456

RFC patchclone: inform the user we are checking out

Subject: [PATCH/RFC] clone: inform the user we are checking out

## tl;dr

7 messages between May 7, 2012 and May 7, 2012. Diffs are folded; open one to read it.

replies: 6people: 4as markdown or json

Erik Faye-Lund· May 7, 2012, 09:09 UTC · lore

When cloning a large repository over a local file-system, git can use hard-links to the old repository files, making the repository-cloning very fast. However, git also perform an implicit checkout of the files, which can be a lengthy operation.

Inform the user about this, and move the logic to notify the user that we are done until we actually are.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---

I just tried to clone a very large (proprietary) repository over a local filesystyem, and the output struck me as confusing:

---8<--- $ git clone some-repo.git some-other-repo Cloning into 'some-other-repo'... <happens instantly> done. <hangs for minutes> $ ---8<---

Now, seems to be because repo gets hard-linked, so the cloning part did not actually take much time. However, we perform a check-out at the end, and this does take time for my large repo. So the behavior probably makes sense from a git-internals point of view.

But from an end-user point of view, it's confusing. I asked git to clone, and it told me it finished, only to hang around for several minutes while, judging by the output, doing nothing.

So, perhaps it could make sense to do something along these lines?
This gives me this output instead:

---8<--- $ git clone some-repo.git some-other-repo Cloning into 'some-repo'... <happens instantly> Checking out HEAD... <hangs for minutes> done. $ ---8<---

...which seems much more informative to me.
 builtin/clone.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
Show changes to builtin/clone.c +7 −3
diff --git a/builtin/clone.c b/builtin/clone.c
index bbd5c96..3f863a1 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -368,9 +368,6 @@ static void clone_local(const char *src_repo, const char *dest_repo)
 		strbuf_release(&src);
 		strbuf_release(&dest);
 	}
-
-	if (0 <= option_verbosity)
-		printf(_("done.\n"));
 }
 
 static const char *junk_work_tree;
@@ -544,6 +541,9 @@ static int checkout(void)
 	if (option_no_checkout)
 		return 0;
 
+	if (0 <= option_verbosity)
+		printf(_("Checking out HEAD...\n"));
+
 	head = resolve_refdup("HEAD", sha1, 1, NULL);
 	if (!head) {
 		warning(_("remote HEAD refers to nonexistent ref, "
@@ -870,5 +870,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	strbuf_release(&key);
 	strbuf_release(&value);
 	junk_pid = 0;
+
+	if (is_local && 0 <= option_verbosity)
+		printf(_("done.\n"));
+
 	return err;
 }

-- 
1.7.10.1.457.g8275905
Johannes Sixt· May 7, 2012, 09:26 UTC · re: Erik Faye-Lund · lore

Re: [PATCH/RFC] clone: inform the user we are checking out

Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
> When cloning a large repository over a local file-system
...
> $ git clone some-repo.git some-other-repo
> Cloning into 'some-other-repo'...    <happens instantly>
> done.                                <hangs for minutes>
> $
...
> I asked git
> to clone, and it told me it finished, only to hang around for
> several minutes while, judging by the output, doing nothing.

We have a nice "Checking out files" progress indicator. I wonder why you do not see it.

At any rate, it's better to write "done" only after we're really done.
-- Hannes
Erik Faye-Lund· May 7, 2012, 09:35 UTC · re: Johannes Sixt · lore

Re: [PATCH/RFC] clone: inform the user we are checking out

On Mon, May 7, 2012 at 11:26 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
Show 15 quoted lines
> Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
>> When cloning a large repository over a local file-system
> ...
>> $ git clone some-repo.git some-other-repo
>> Cloning into 'some-other-repo'...    <happens instantly>
>> done.                                <hangs for minutes>
>> $
> ...
>> I asked git
>> to clone, and it told me it finished, only to hang around for
>> several minutes while, judging by the output, doing nothing.
>
> We have a nice "Checking out files" progress indicator. I wonder why you
> do not see it.
>

That's a very good point, I forgot about that one. I'll investigate that part. Just to be clear, I'm seeing this problem on version 1.7.10.1.457.g8275905 (junio's 'master' as of right now, AFAICT) on CentOS. I've built with: NO_OPENSSL = YesPlease NO_CURL = YesPlease

"git config -l" does not reveal that any progress related config-options are set.

> At any rate, it's better to write "done" only after we're really done.

Well, perhaps, but at the same time perhaps not: We only write it for local clones, because the remote-clones are quite chatty to begin with, and the remote-stuff seems to inform about it's actions by printing "<performing task x>...done!". Moving the final "done."-printing to the very end might be a step in making these outputs even less consistent. But it might not be important.

Erik Faye-Lund· May 7, 2012, 09:46 UTC · re: Erik Faye-Lund · lore

Re: [PATCH/RFC] clone: inform the user we are checking out

On Mon, May 7, 2012 at 11:35 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
Show 27 quoted lines
> On Mon, May 7, 2012 at 11:26 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>> Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
>>> When cloning a large repository over a local file-system
>> ...
>>> $ git clone some-repo.git some-other-repo
>>> Cloning into 'some-other-repo'...    <happens instantly>
>>> done.                                <hangs for minutes>
>>> $
>> ...
>>> I asked git
>>> to clone, and it told me it finished, only to hang around for
>>> several minutes while, judging by the output, doing nothing.
>>
>> We have a nice "Checking out files" progress indicator. I wonder why you
>> do not see it.
>>
>
> That's a very good point, I forgot about that one. I'll investigate
> that part. Just to be clear, I'm seeing this problem on version
> 1.7.10.1.457.g8275905 (junio's 'master' as of right now, AFAICT) on
> CentOS. I've built with:
> NO_OPENSSL = YesPlease
> NO_CURL = YesPlease
>
> "git config -l" does not reveal that any progress related
> config-options are set.
>

It seems progress is only enabled when cloning with the verbose-flag. This is because check_updates in unpack-trees.c only enabled progress if both o->update and o->verbose_update is set, and checkout in builtin/clone.c fills in opts.verbose_update with (option_verbosity > 0).

So that's that mystery. I still think the output without the verbose-flag is confusing as-is.

Erik Faye-Lund· May 7, 2012, 10:02 UTC · re: Erik Faye-Lund · lore

Re: [PATCH/RFC] clone: inform the user we are checking out

On Mon, May 7, 2012 at 11:46 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
Show 18 quoted lines
> On Mon, May 7, 2012 at 11:35 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> On Mon, May 7, 2012 at 11:26 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>>> Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
>>>> I asked git
>>>> to clone, and it told me it finished, only to hang around for
>>>> several minutes while, judging by the output, doing nothing.
>>>
>>> We have a nice "Checking out files" progress indicator. I wonder why you
>>> do not see it.
>
> It seems progress is only enabled when cloning with the verbose-flag.
> This is because check_updates in unpack-trees.c only enabled progress
> if both o->update and o->verbose_update is set, and checkout in
> builtin/clone.c fills in opts.verbose_update with (option_verbosity >
> 0).
>
> So that's that mystery. I still think the output without the
> verbose-flag is confusing as-is.

OK, some blaming shows that this changed in 5bd631b3 ("clone: support multiple levels of verbosity"), back in February 2010. Before this patch, one would have to specify the "quiet"-flag to clone to suppress progress-output, after the patch the default is progress being off.

This seems like the right thing to do if we want to resurrect the progress-output's default-on behavior: ---8<---

Show changes to builtin/clone.c +1 −1
diff --git a/builtin/clone.c b/builtin/clone.c
index 3f863a1..f48e603 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -569,7 +569,7 @@ static int checkout(void)
 	opts.update = 1;
 	opts.merge = 1;
 	opts.fn = oneway_merge;
-	opts.verbose_update = (option_verbosity > 0);
+	opts.verbose_update = (option_verbosity >= 0);
 	opts.src_index = &the_index;
 	opts.dst_index = &the_index;

---8<---
Jeff King· May 7, 2012, 13:54 UTC · re: Erik Faye-Lund · lore

Re: [PATCH/RFC] clone: inform the user we are checking out

On Mon, May 07, 2012 at 12:02:46PM +0200, Erik Faye-Lund wrote:
> OK, some blaming shows that this changed in 5bd631b3 ("clone: support
> multiple levels of verbosity"), back in February 2010. Before this
> patch, one would have to specify the "quiet"-flag to clone to suppress
> progress-output, after the patch the default is progress being off.

Yeah, I think this is a bug. We already show progress indicators from the transports without "-v", so there is no reason to require it for the checkout progress meter. And reading fbd631b3, it looks like the change is simply a mistake in the patch.

Show 17 quoted lines
> This seems like the right thing to do if we want to resurrect the
> progress-output's default-on behavior:
> ---8<---
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 3f863a1..f48e603 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -569,7 +569,7 @@ static int checkout(void)
>  	opts.update = 1;
>  	opts.merge = 1;
>  	opts.fn = oneway_merge;
> -	opts.verbose_update = (option_verbosity > 0);
> +	opts.verbose_update = (option_verbosity >= 0);
>  	opts.src_index = &the_index;
>  	opts.dst_index = &the_index;
> 
> ---8<---
Yes, that looks correct to me. Thanks for looking into this.
-Peff
Junio C Hamano· May 7, 2012, 19:03 UTC · re: Erik Faye-Lund · lore

Re: [PATCH/RFC] clone: inform the user we are checking out

Erik Faye-Lund <kusmabite@gmail.com> writes:
Show 22 quoted lines
> OK, some blaming shows that this changed in 5bd631b3 ("clone: support
> multiple levels of verbosity"), back in February 2010. Before this
> patch, one would have to specify the "quiet"-flag to clone to suppress
> progress-output, after the patch the default is progress being off.
>
> This seems like the right thing to do if we want to resurrect the
> progress-output's default-on behavior:
> ---8<---
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 3f863a1..f48e603 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -569,7 +569,7 @@ static int checkout(void)
>  	opts.update = 1;
>  	opts.merge = 1;
>  	opts.fn = oneway_merge;
> -	opts.verbose_update = (option_verbosity > 0);
> +	opts.verbose_update = (option_verbosity >= 0);
>  	opts.src_index = &the_index;
>  	opts.dst_index = &the_index;
>
> ---8<---

Sounds sensible, as the original said "verbosely update unless we were told to be quiet", so the normal case should be verbose_update which will give progress only if it takes too long.

Care to roll a signed-off patch?

← back to recent threads