Re: [GSoC PATCH v2] backfill: add --[no-]progress option

From: Trieu Huynh <vikingtc4@gmail.com>
Date: Apr 13, 2026, 19:02 UTC
Message-ID: <wsbnw3am5fq6hpjwmmbguo2c3mnv4qkr3hh7apawch7smns6zx@rxegeuugbnhg>
In-Reply-To: <d2cf741c-a381-42a6-9d26-e38481696adb@gmail.com>

On Sun, Apr 12, 2026 at 03:46:17PM -0400, Derrick Stolee wrote:

Show 24 quoted lines

> On 4/12/26 3:36 PM, Trieu Huynh wrote:
> > From: Trieu Huynh <vikingtc4@gmail.com>
> > 
> > 'git backfill' does not show an overall progress bar across
> > batches, giving no cross-batch feedback during potentially
> > long-running operations on large repositories.  By contrast,
> > 'git fetch', 'git gc', and 'git index-pack' all support
> > --[no-]progress.
> > 
> > Add a --[no-]progress option that tracks the total number of
> > missing blobs downloaded across all batches, defaulting to
> > showing progress when stderr is a terminal (matching the
> > behaviour of 'git fetch').
> > 
> > Add tests to verify that:
> >   - progress is shown by default on a TTY
> >   - --progress forces output regardless of TTY
> >   - --no-progress suppresses output
> 
> I think the tests do show an improvement, but we're missing
> the interaction with the underlying fetch's progress
> indicators. I don't see any mention of how your backfill
> progress indicators will work with or against the fetch's
> progress from the remote and index-pack steps.

Actually, I was missing adding it in the changelog, see below: As-is: remote: Enumerating objects: 7391, done. remote: Counting objects: 100% (293/293), done. remote: Compressing objects: 100% (162/162), done. remote: Total 7391 (delta 249), reused 131 (delta 131), pack-reused 7098 (from 1) Receiving objects: 100% (7391/7391), 4.09 MiB | 10.20 MiB/s, done. Resolving deltas: 100% (5617/5617), done.

To-be: remote: Enumerating objects: 7391, done. remote: Counting objects: 100% (293/293), done. remote: Compressing objects: 100% (162/162), done. remote: Total 7391 (delta 249), reused 131 (delta 131), pack-reused 7098 (from 1) Receiving objects: 100% (7391/7391), 4.09 MiB | 6.46 MiB/s, done. Resolving deltas: 100% (5618/5618), done. Downloading missing blobs: 157594, done.

Show 7 quoted lines

> 
> Further, if a user supplies 'git backfill --no-progress'
> then they are probably saying "I don't want any progress
> indicators" and that would signal also that the fetch should
> be quiet. This is perhaps the key detail that makes your
> current version unable to move forward. It creates an
> implication that it doesn't follow-through on.

Thank you for the point. You are right that the current patch does not address the interaction between the backfill progress bar and the underlying fetch's own output (remote counting/compressing objects, index-pack, etc.). Leaving both active at the same time would produce interleaved and confusing output, which is worse than no progress at all.

Show 15 quoted lines

> 
> One way to go about this is to hide the 'git fetch' output
> entirely by passing '--quiet' unconditionally from the
> backfill command. But this may also be too much for users
> who want to watch the download statistics from the remote.
> 
> Perhaps a way to have a robust set that allows all things
> to interact is to do the following:
> 
> 1. Add a --[no-]verbose option that is off by default. The
>    implementation sends the --quiet flag to 'git fetch' if
>    --verbose isn't provided from the user. This reduces the
>    noise for the default user.
> 
> 2. Add a --[no-]progress option as you've provided here.

Make sense to me, will change to implement that way in v3.

Show 5 quoted lines

> 
> The complexity at the end is about what happens when the
> user provides both --verbose and --progress, which is the
> situation that this patch is currently in. How do the
> progress indicators mingle with the verbose fetch output?

IIUC, the fetch output for each batch completes before the progress bar updates, so they do not actually interleave. The "Downloading missing blobs" counter updates in place via carriage return during the run, display until it's done partially, and only prints the final "done." line at the end, for example:

  remote: Enumerating objects: 50106, done.
  remote: Counting objects: 100% (780/780), done.
  ...
  Receiving objects: 100% (50106/50106), done.
  remote: Enumerating objects: 50096, done.
  ...
  Receiving objects: 100% (50096/50096), done.
  Downloading missing blobs: 157594, done.

So --verbose and --progress together produce readable output without any special handling needed. Does that direction sound reasonable to you?

> 
> Thanks,
> -Stolee
>