Re: non-smooth progress indication for git fsck and git gc
- From
Junio C Hamano <gitster@pobox.com>
- Date
- Aug 16, 2018, 22:10 UTC
- Message-ID
- <xmqq36vedkud.fsf@gitster-ct.c.googlers.com>
- In-Reply-To
- <20180816200222.GA7422@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
Show 12 quoted lines
> On Thu, Aug 16, 2018 at 11:57:14AM -0400, Jeff King wrote: > >> The only way to solve that is to count bytes. We don't have a total byte >> count in most cases, and it wouldn't always make sense (e.g., the >> "Compressing objects" meter can show the same issue, but it's not really >> putting through bytes in a linear way). In some cases we do show >> transmitted size and throughput, but that's just for network operations. >> We could do the same for "gc" with the patch below. But usually >> throughput isn't all that interesting for a filesystem write, because >> bandwidth isn't the bottleneck. > > Just realized I forgot to include the patch. Here it is, for reference.
I've been wondering when you'd realize the omission ;-)
> Doing something similar for fsck would be quite a bit more invasive.
Yeah, on that codepath there is no streaming write passing through a single chokepoint you can count bytes X-<.
Show 62 quoted lines
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index 80c880e9ad..e1130b959d 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -837,7 +837,7 @@ static void write_pack_file(void)
> if (pack_to_stdout)
> f = hashfd_throughput(1, "<stdout>", progress_state);
> else
> - f = create_tmp_packfile(&pack_tmp_name);
> + f = create_tmp_packfile(&pack_tmp_name, progress_state);
>
> offset = write_pack_header(f, nr_remaining);
>
> diff --git a/bulk-checkin.c b/bulk-checkin.c
> index 9f3b644811..0df45b8f55 100644
> --- a/bulk-checkin.c
> +++ b/bulk-checkin.c
> @@ -178,7 +178,7 @@ static void prepare_to_stream(struct bulk_checkin_state *state,
> if (!(flags & HASH_WRITE_OBJECT) || state->f)
> return;
>
> - state->f = create_tmp_packfile(&state->pack_tmp_name);
> + state->f = create_tmp_packfile(&state->pack_tmp_name, NULL);
> reset_pack_idx_option(&state->pack_idx_opts);
>
> /* Pretend we are going to write only one object */
> diff --git a/pack-write.c b/pack-write.c
> index a9d46bc03f..b72480b440 100644
> --- a/pack-write.c
> +++ b/pack-write.c
> @@ -334,14 +334,15 @@ int encode_in_pack_object_header(unsigned char *hdr, int hdr_len,
> return n;
> }
>
> -struct hashfile *create_tmp_packfile(char **pack_tmp_name)
> +struct hashfile *create_tmp_packfile(char **pack_tmp_name,
> + struct progress *progress)
> {
> struct strbuf tmpname = STRBUF_INIT;
> int fd;
>
> fd = odb_mkstemp(&tmpname, "pack/tmp_pack_XXXXXX");
> *pack_tmp_name = strbuf_detach(&tmpname, NULL);
> - return hashfd(fd, *pack_tmp_name);
> + return hashfd_throughput(fd, *pack_tmp_name, progress);
> }
>
> void finish_tmp_packfile(struct strbuf *name_buffer,
> diff --git a/pack.h b/pack.h
> index 34a9d458b4..c87628b093 100644
> --- a/pack.h
> +++ b/pack.h
> @@ -98,7 +98,8 @@ extern int encode_in_pack_object_header(unsigned char *hdr, int hdr_len,
> #define PH_ERROR_PROTOCOL (-3)
> extern int read_pack_header(int fd, struct pack_header *);
>
> -extern struct hashfile *create_tmp_packfile(char **pack_tmp_name);
> +extern struct hashfile *create_tmp_packfile(char **pack_tmp_name,
> + struct progress *progress);
> extern void finish_tmp_packfile(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]);
>
> #endif