threads / patch / 2464

patchShow URL in the "Getting <foo> list" http-fetch messages

Subject: [PATCH] Show URL in the "Getting <foo> list" http-fetch messages

## tl;dr

6 messages between Nov 12, 2005 and Nov 14, 2005. Diffs are folded; open one to read it.

replies: 5people: 3as markdown or json

Petr Baudis· Nov 12, 2005, 00:49 UTC · lore

Signed-off-by: Petr Baudis <pasky@suse.cz>
---

 http-fetch.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

Show changes to http-fetch.c +2 −2

diff --git a/http-fetch.c b/http-fetch.c
index e7655d1..e6e7df6 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -795,7 +795,7 @@ static int fetch_alternates(char *base)
 	buffer.buffer = data;
 
 	if (get_verbosely)
-		fprintf(stderr, "Getting alternates list\n");
+		fprintf(stderr, "Getting alternates list for %s\n", base);
 	
 	url = xmalloc(strlen(base) + 31);
 	sprintf(url, "%s/objects/info/http-alternates", base);
@@ -918,7 +918,7 @@ static int fetch_indices(struct alt_base
 	buffer.buffer = data;
 
 	if (get_verbosely)
-		fprintf(stderr, "Getting pack list\n");
+		fprintf(stderr, "Getting pack list for %s\n", repo->base);
 	
 	url = xmalloc(strlen(repo->base) + 21);
 	sprintf(url, "%s/objects/info/packs", repo->base);

Petr Baudis· Nov 12, 2005, 00:58 UTC · re: Petr Baudis · lore

Re: [PATCH] Show URL in the "Getting <foo> list" http-fetch messages

Dear diary, on Sat, Nov 12, 2005 at 01:49:59AM CET, I got a letter where Petr Baudis <pasky@suse.cz> said that...

Show 28 quoted lines

> Signed-off-by: Petr Baudis <pasky@suse.cz>
> ---
> 
>  http-fetch.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/http-fetch.c b/http-fetch.c
> index e7655d1..e6e7df6 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -795,7 +795,7 @@ static int fetch_alternates(char *base)
>  	buffer.buffer = data;
>  
>  	if (get_verbosely)
> -		fprintf(stderr, "Getting alternates list\n");
> +		fprintf(stderr, "Getting alternates list for %s\n", base);
>  	
>  	url = xmalloc(strlen(base) + 31);
>  	sprintf(url, "%s/objects/info/http-alternates", base);
> @@ -918,7 +918,7 @@ static int fetch_indices(struct alt_base
>  	buffer.buffer = data;
>  
>  	if (get_verbosely)
> -		fprintf(stderr, "Getting pack list\n");
> +		fprintf(stderr, "Getting pack list for %s\n", repo->base);
>  	
>  	url = xmalloc(strlen(repo->base) + 21);
>  	sprintf(url, "%s/objects/info/packs", repo->base);

By the way, this is only a pathetic remnant of my brave effort to get rid of seeing the "Getting alternates list" message on the same repository 20 times now since the parallel fetching got introduced. I know how to do silence the messages, but doing the _fetching_ itself seems as downright stupid thing in the first place.

Therefore, can I somehow stall fetching of an object? The idea is:

	I want to check the alternates list!
	I don't have it yet...
	...but a request for it is already queued. Ok, I'll wait.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Junio C Hamano· Nov 12, 2005, 01:17 UTC · re: Petr Baudis · lore

Re: [PATCH] Show URL in the "Getting <foo> list" http-fetch messages

Petr Baudis <pasky@suse.cz> writes:

Show 11 quoted lines

> By the way, this is only a pathetic remnant of my brave effort to get
> rid of seeing the "Getting alternates list" message on the same
> repository 20 times now since the parallel fetching got introduced.
> I know how to do silence the messages, but doing the _fetching_ itself
> seems as downright stupid thing in the first place.
>
> Therefore, can I somehow stall fetching of an object? The idea is:
>
> 	I want to check the alternates list!
> 	I don't have it yet...
> 	...but a request for it is already queued. Ok, I'll wait.

I agree, and probably this applies when index request is in transit as well.

Nick Hengeveld· Nov 12, 2005, 17:22 UTC · re: Junio C Hamano · lore

Re: [PATCH] Show URL in the "Getting <foo> list" http-fetch messages

On Fri, Nov 11, 2005 at 05:17:42PM -0800, Junio C Hamano wrote:

Show 8 quoted lines

> > Therefore, can I somehow stall fetching of an object? The idea is:
> >
> > 	I want to check the alternates list!
> > 	I don't have it yet...
> > 	...but a request for it is already queued. Ok, I'll wait.
> 
> I agree, and probably this applies when index request is in
> transit as well.

I just posted a patch to prevent another alternate request from starting if one is already in progress.

This should not be an issue with index requests because they are only initiated from fetch(). The previous patch to load alternates on demand added alternate handling to process_curl_messages() so that a 404 for an object can be handled immediately rather than waiting for the fetch() call for that object to notice.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

Nick Hengeveld· Nov 14, 2005, 19:02 UTC · re: Nick Hengeveld · lore

Re: [PATCH] Show URL in the "Getting <foo> list" http-fetch messages

On Sat, Nov 12, 2005 at 09:22:02AM -0800, Nick Hengeveld wrote:

Show 5 quoted lines

> This should not be an issue with index requests because they are only
> initiated from fetch().  The previous patch to load alternates on demand
> added alternate handling to process_curl_messages() so that a 404 for an
> object can be handled immediately rather than waiting for the fetch()
> call for that object to notice.

Seems like it might make sense to handle pack downloads immediately when an object is unavailable rather than waiting for the fetch() call. It could prevent attempts to download any other objects inside that pack, although queued requests that activate while a pack is downloading would have to wait to see whether the download is successful. If such an object also exists loose it would prevent a redundant download and if not it would at least prevent an unnecessary check and 404.

Thoughts?

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

Junio C Hamano· Nov 14, 2005, 20:11 UTC · re: Nick Hengeveld · lore

Re: [PATCH] Show URL in the "Getting <foo> list" http-fetch messages

Nick Hengeveld <nickh@reactrix.com> writes:

Show 13 quoted lines

> On Sat, Nov 12, 2005 at 09:22:02AM -0800, Nick Hengeveld wrote:
>
>> This should not be an issue with index requests because they are only
>> initiated from fetch().  The previous patch to load alternates on demand
>> added alternate handling to process_curl_messages() so that a 404 for an
>> object can be handled immediately rather than waiting for the fetch()
>> call for that object to notice.
>
> Seems like it might make sense to handle pack downloads immediately when
> an object is unavailable rather than waiting for the fetch() call.  It
> could prevent attempts to download any other objects inside that pack,
> although queued requests that activate while a pack is downloading
> would have to wait to see whether the download is successful.

I think that makes sense, and it probably is preferable to make them wait than blindly go ahead. Although we are issuing requests in parallel, these simultaneous requests are asking for related things (e.g. parent commit objects or the tree object of the commit we just saw; blob objects contained in the tree we just saw) and are more likely than not to be found in the pack being requested. Losing parallelism while a pack is in transit would not be too much of a problem.