Re: Antw: Re: non-smooth progress indication for git fsck and git gc

From: Jeff King <peff@peff.net>
Date: Aug 21, 2018, 01:07 UTC
Message-ID: <20180821010712.GA32126@sigill.intra.peff.net>
In-Reply-To: <87woslpg9i.fsf@evledraar.gmail.com>

On Mon, Aug 20, 2018 at 10:57:13AM +0200, Ævar Arnfjörð Bjarmason wrote:

Show 6 quoted lines

> > That seems to apply. BTW: Is there a way go get some repository statistics
> > like a histogram of object sizes (or whatever that might be useful to help
> > making decisions)?
> 
> The git-sizer program is really helpful in this regard:
> https://github.com/github/git-sizer

Yeah, I'd very much agree that's the best tool for a general overview of the repository stats.

Ulrich, if you want to more analysis (like a histogram), probably something like:

  git cat-file --batch-all-objects --batch-check='%(objectsize)'

might be a good starting place to dump information (see the "BATCH OUTPUT" section of "git help cat-file" for more format items).

Show 11 quoted lines

> > If it's sorting, maybe add some code like (wild guess):
> >
> > if (objects_to_sort > magic_number)
> >    message("Sorting something...");
> 
> I think a good solution to these cases is to just introduce something to
> the progress.c mode where it learns how to display a counter where we
> don't know what the end-state will be. Something like your proposed
> magic_number can just be covered under the more general case where we
> don't show the progress bar unless it's been 1 second (which I believe
> is the default).

Yeah. We already have open-ended counters (e.g., "counting objects"), and delayed meters (we actually normalized the default to 2s recently).

I _think_ they should work together OK without further modification. Once upon a time the caller had to say "don't show if we're past N% after M seconds", but I think with the current code we'd just show it if we're not completely finished after 2 seconds.

So it really should just be a simple:

  progress = start_delayed_progress("Hashing packfile", 0);

That said, counting bytes would probably be ugly (just because the numbers get really big). We'd probably prefer to show a throughput or something. And as you noted, there's some refactoring to be done with pack-check for it to show multiple progress meters.

(I still think in the long run we would want to scrap that code, but that's a much bigger job; I'm fine if somebody wants to do incremental improvements in the meantime).

-Peff