git/list[1] front-page[2] threads[3] people[4] search[5] about
wed 2026-10-07 17:33 UTC

Re: [GSoC PATCH v2] backfill: add --[no-]progress option

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 15, 2026, 18:28 UTC
Message-ID
<1988d824-0ff4-41a4-bc10-1b4e030878ef@gmail.com>
In-Reply-To
<wsbnw3am5fq6hpjwmmbguo2c3mnv4qkr3hh7apawch7smns6zx@rxegeuugbnhg>
On 4/13/2026 3:02 PM, Trieu Huynh wrote:
Show 42 quoted lines
> On Sun, Apr 12, 2026 at 03:46:17PM -0400, Derrick Stolee wrote:
>> On 4/12/26 3:36 PM, Trieu Huynh wrote:
>>> From: Trieu Huynh <vikingtc4@gmail.com>
>>>
>>> 'git backfill' does not show an overall progress bar across
>>> batches, giving no cross-batch feedback during potentially
>>> long-running operations on large repositories.  By contrast,
>>> 'git fetch', 'git gc', and 'git index-pack' all support
>>> --[no-]progress.
>>>
>>> Add a --[no-]progress option that tracks the total number of
>>> missing blobs downloaded across all batches, defaulting to
>>> showing progress when stderr is a terminal (matching the
>>> behaviour of 'git fetch').
>>>
>>> Add tests to verify that:
>>>   - progress is shown by default on a TTY
>>>   - --progress forces output regardless of TTY
>>>   - --no-progress suppresses output
>>
>> I think the tests do show an improvement, but we're missing
>> the interaction with the underlying fetch's progress
>> indicators. I don't see any mention of how your backfill
>> progress indicators will work with or against the fetch's
>> progress from the remote and index-pack steps.
> Actually, I was missing adding it in the changelog, see below:
> As-is:
> remote: Enumerating objects: 7391, done.
> remote: Counting objects: 100% (293/293), done.
> remote: Compressing objects: 100% (162/162), done.
> remote: Total 7391 (delta 249), reused 131 (delta 131), pack-reused 7098 (from 1)
> Receiving objects: 100% (7391/7391), 4.09 MiB | 10.20 MiB/s, done.
> Resolving deltas: 100% (5617/5617), done.
> 
> To-be:
> remote: Enumerating objects: 7391, done.
> remote: Counting objects: 100% (293/293), done.
> remote: Compressing objects: 100% (162/162), done.
> remote: Total 7391 (delta 249), reused 131 (delta 131), pack-reused 7098 (from 1)
> Receiving objects: 100% (7391/7391), 4.09 MiB | 6.46 MiB/s, done.
> Resolving deltas: 100% (5618/5618), done.
> Downloading missing blobs: 157594, done.

These examples are nice, but only for one batch of objects. You'll need to test with a smaller batch size or a larger repo to get the output I'm looking for.

Show 22 quoted lines
>> The complexity at the end is about what happens when the
>> user provides both --verbose and --progress, which is the
>> situation that this patch is currently in. How do the
>> progress indicators mingle with the verbose fetch output?
> IIUC, the fetch output for each batch completes before the progress
> bar updates, so they do not actually interleave. The
> "Downloading missing blobs" counter updates in place via carriage return
> during the run, display until it's done partially, and only prints the
> final "done." line at the end, for example:
> 
>   remote: Enumerating objects: 50106, done.
>   remote: Counting objects: 100% (780/780), done.
>   ...
>   Receiving objects: 100% (50106/50106), done.
>   remote: Enumerating objects: 50096, done.
>   ...
>   Receiving objects: 100% (50096/50096), done.
>   Downloading missing blobs: 157594, done.
> 
> So --verbose and --progress together produce readable output without
> any special handling needed.
> Does that direction sound reasonable to you?

I think I'd like to see the full output for multiple batches, and then I can decide if the progress indicators make sense together or if they look confusing.

Thanks, -Stolee

Previous: Tian Yuchen
Message 5 of 5 in “backfill: add --[no-]progress option”
  1. backfill: add --[no-]progress optionTrieu Huynh, Apr 12, 2026
  2. Derrick StoleeApr 12, 2026
  3. Trieu HuynhApr 13, 2026
  4. Tian YuchenApr 15, 2026
  5. Derrick StoleeApr 15, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.