threads / discuss / 62991

git client erroneously downloads files when --filter=blob:none and filtering unsupported

Subject: git client erroneously downloads files when --filter=blob:none and filtering unsupported

## tl;dr

3 messages between Feb 22, 2025 and Feb 28, 2025.

replies: 2people: 2as markdown or json

John Giorshev· Feb 22, 2025, 15:01 UTC · lore

The git client, in my opinion, erroneously downloads files when --filter=blob:none is specified and the server does not support filtering. I created a related question on this before coming here:

https://stackoverflow.com/q/79413099/15534181

Instead of giving a warning, it should instead error and exit. From a user perspective, when I request "don't pull down the file contents" and it does it anyway under some circumstances, this is unexpected. In my case it caused performance degradations on a repo scanner.

I propose something like this:
https://github.com/jagprog5/git/commit/c4bd8c9640c1491dc6e23acf31fa0230485b68b1

This is not backwards compatible. My question is, how best should this be handled? Is this breaking change ok? Or should there instead be a new CLI arg or config which enabled this new behaviour. Looking for advice, thanks.

Jeff King· Feb 25, 2025, 01:32 UTC · re: John Giorshev · lore

Re: git client erroneously downloads files when --filter=blob:none and filtering unsupported

On Sat, Feb 22, 2025 at 10:01:22AM -0500, John Giorshev wrote:
Show 19 quoted lines
> The git client, in my opinion, erroneously downloads files when
> --filter=blob:none is specified and the server does not support
> filtering. I created a related question on this before coming here:
> 
> https://stackoverflow.com/q/79413099/15534181
> 
> Instead of giving a warning, it should instead error and exit. From a
> user perspective, when I request "don't pull down the file contents"
> and it does it anyway under some circumstances, this is unexpected. In
> my case it caused performance degradations on a repo scanner.
> 
> I propose something like this:
> 
> https://github.com/jagprog5/git/commit/c4bd8c9640c1491dc6e23acf31fa0230485b68b1
> 
> This is not backwards compatible. My question is, how best should this
> be handled?  Is this breaking change ok? Or should there instead be a
> new CLI arg or config which enabled this new behaviour. Looking for
> advice, thanks.
I could see arguments going either way:
  - you asked for no blobs, but it's just an optimization, so we can
    complete the operation for a bit more expense. It should be a
    warning. That makes it safe just sprinkle "--filter=blob:none"
    wherever you like, and sometimes things get faster and sometimes
    not.
  - downloading the blobs is so expensive that it's better to fail than
    spend resources on something that will probably fail eventually
    anyway.

Which implies to me it should perhaps be configurable. And then that gives you a backwards-compatibility solution, too. Step 1 is to add the config. Step 2 may eventually be to flip the default, and there the config option gives people an escape hatch if they like the old behavior.

All that said, I wondered if there was another similar case: when the server supports filters but your particular filter is not allowed.

E.g., if the server has config like this:
  [uploadpackfilter "blob:none"]
  allow = false
But then interestingly, we already consider that a fatal error!
  $ git clone --filter=blob:none --no-local /path/to/repo
  fatal: filter 'blob:none' not supported
  fatal: remote error: filter 'blob:none' not supported

So I dunno. Maybe nobody actually cares about continuing with a warning for this case. OTOH, I doubt anybody would forbid _just_ blob:none; it's the cheapest filter to support. So it's likely that nobody has run into it. But the behavior would be the same with something like sparse:oid, which is likely to be forbidden because it's expensive. So:

  $ git clone --filter=sparse:oid=foo https://github.com/git/git
  Cloning into 'git'...
  fatal: remote error: filter 'sparse:oid' not supported

Which maybe implies that worrying about config or backwards compatibility is not worth it.

-Peff
John Giorshev· Feb 28, 2025, 20:27 UTC · re: Jeff King · lore

Re: git client erroneously downloads files when --filter=blob:none and filtering unsupported

Thanks for the reply, I appreciate it.

My understanding from this is that it might be ok to send the patch as is and not be backward compatible. Or, maybe we need feedback from others?

Created: https://github.com/gitgitgadget/git/pull/1869
On Mon, Feb 24, 2025 at 8:32 PM Jeff King <peff@peff.net> wrote:
Show 69 quoted lines
>
> On Sat, Feb 22, 2025 at 10:01:22AM -0500, John Giorshev wrote:
>
> > The git client, in my opinion, erroneously downloads files when
> > --filter=blob:none is specified and the server does not support
> > filtering. I created a related question on this before coming here:
> >
> > https://stackoverflow.com/q/79413099/15534181
> >
> > Instead of giving a warning, it should instead error and exit. From a
> > user perspective, when I request "don't pull down the file contents"
> > and it does it anyway under some circumstances, this is unexpected. In
> > my case it caused performance degradations on a repo scanner.
> >
> > I propose something like this:
> >
> > https://github.com/jagprog5/git/commit/c4bd8c9640c1491dc6e23acf31fa0230485b68b1
> >
> > This is not backwards compatible. My question is, how best should this
> > be handled?  Is this breaking change ok? Or should there instead be a
> > new CLI arg or config which enabled this new behaviour. Looking for
> > advice, thanks.
>
> I could see arguments going either way:
>
>   - you asked for no blobs, but it's just an optimization, so we can
>     complete the operation for a bit more expense. It should be a
>     warning. That makes it safe just sprinkle "--filter=blob:none"
>     wherever you like, and sometimes things get faster and sometimes
>     not.
>
>   - downloading the blobs is so expensive that it's better to fail than
>     spend resources on something that will probably fail eventually
>     anyway.
>
> Which implies to me it should perhaps be configurable. And then that
> gives you a backwards-compatibility solution, too. Step 1 is to add the
> config. Step 2 may eventually be to flip the default, and there the
> config option gives people an escape hatch if they like the old
> behavior.
>
> All that said, I wondered if there was another similar case: when the
> server supports filters but your particular filter is not allowed.
>
> E.g., if the server has config like this:
>
>   [uploadpackfilter "blob:none"]
>   allow = false
>
> But then interestingly, we already consider that a fatal error!
>
>   $ git clone --filter=blob:none --no-local /path/to/repo
>   fatal: filter 'blob:none' not supported
>   fatal: remote error: filter 'blob:none' not supported
>
> So I dunno. Maybe nobody actually cares about continuing with a warning
> for this case. OTOH, I doubt anybody would forbid _just_ blob:none; it's
> the cheapest filter to support. So it's likely that nobody has run into
> it. But the behavior would be the same with something like sparse:oid,
> which is likely to be forbidden because it's expensive. So:
>
>   $ git clone --filter=sparse:oid=foo https://github.com/git/git
>   Cloning into 'git'...
>   fatal: remote error: filter 'sparse:oid' not supported
>
> Which maybe implies that worrying about config or backwards
> compatibility is not worth it.
>
> -Peff

← back to recent threads