From: Derrick Stolee <stolee@gmail.com>
Date: Wed, 15 Apr 2026 18:28:59 GMT
Subject: Re: [GSoC PATCH v2] backfill: add --[no-]progress option
Message-ID: <1988d824-0ff4-41a4-bc10-1b4e030878ef@gmail.com>
In-Reply-To: <wsbnw3am5fq6hpjwmmbguo2c3mnv4qkr3hh7apawch7smns6zx@rxegeuugbnhg>

On 4/13/2026 3:02 PM, Trieu Huynh wrote:
> On Sun, Apr 12, 2026 at 03:46:17PM -0400, Derrick Stolee wrote:
>> On 4/12/26 3:36 PM, Trieu Huynh wrote:
>>> From: Trieu Huynh <vikingtc4@gmail.com>
>>>
>>> 'git backfill' does not show an overall progress bar across
>>> batches, giving no cross-batch feedback during potentially
>>> long-running operations on large repositories.  By contrast,
>>> 'git fetch', 'git gc', and 'git index-pack' all support
>>> --[no-]progress.
>>>
>>> Add a --[no-]progress option that tracks the total number of
>>> missing blobs downloaded across all batches, defaulting to
>>> showing progress when stderr is a terminal (matching the
>>> behaviour of 'git fetch').
>>>
>>> Add tests to verify that:
>>>   - progress is shown by default on a TTY
>>>   - --progress forces output regardless of TTY
>>>   - --no-progress suppresses output
>>
>> I think the tests do show an improvement, but we're missing
>> the interaction with the underlying fetch's progress
>> indicators. I don't see any mention of how your backfill
>> progress indicators will work with or against the fetch's
>> progress from the remote and index-pack steps.
> Actually, I was missing adding it in the changelog, see below:
> As-is:
> remote: Enumerating objects: 7391, done.
> remote: Counting objects: 100% (293/293), done.
> remote: Compressing objects: 100% (162/162), done.
> remote: Total 7391 (delta 249), reused 131 (delta 131), pack-reused 7098 (from 1)
> Receiving objects: 100% (7391/7391), 4.09 MiB | 10.20 MiB/s, done.
> Resolving deltas: 100% (5617/5617), done.
> 
> To-be:
> remote: Enumerating objects: 7391, done.
> remote: Counting objects: 100% (293/293), done.
> remote: Compressing objects: 100% (162/162), done.
> remote: Total 7391 (delta 249), reused 131 (delta 131), pack-reused 7098 (from 1)
> Receiving objects: 100% (7391/7391), 4.09 MiB | 6.46 MiB/s, done.
> Resolving deltas: 100% (5618/5618), done.
> Downloading missing blobs: 157594, done.

These examples are nice, but only for one batch of objects.
You'll need to test with a smaller batch size or a larger
repo to get the output I'm looking for.

>> The complexity at the end is about what happens when the
>> user provides both --verbose and --progress, which is the
>> situation that this patch is currently in. How do the
>> progress indicators mingle with the verbose fetch output?
> IIUC, the fetch output for each batch completes before the progress
> bar updates, so they do not actually interleave. The
> "Downloading missing blobs" counter updates in place via carriage return
> during the run, display until it's done partially, and only prints the
> final "done." line at the end, for example:
> 
>   remote: Enumerating objects: 50106, done.
>   remote: Counting objects: 100% (780/780), done.
>   ...
>   Receiving objects: 100% (50106/50106), done.
>   remote: Enumerating objects: 50096, done.
>   ...
>   Receiving objects: 100% (50096/50096), done.
>   Downloading missing blobs: 157594, done.
> 
> So --verbose and --progress together produce readable output without
> any special handling needed.
> Does that direction sound reasonable to you?

I think I'd like to see the full output for multiple batches,
and then I can decide if the progress indicators make sense
together or if they look confusing.

Thanks,
-Stolee