Re: [PATCH 7/9] ls-refs: ignore very long ref-prefix counts

From: Jeff King <peff@peff.net>
Date: Sep 14, 2021, 19:22 UTC
Message-ID: <YUD2ZEqKSU/pMzws@coredump.intra.peff.net>
In-Reply-To: <CAN0heSp9RdFwSo+x5buHSCAOa0Kay7Wcs1tNKTDXWmkCM8Latw@mail.gmail.com>

On Tue, Sep 14, 2021 at 09:06:55PM +0200, Martin Ågren wrote:

Show 16 quoted lines

> > But we can do better. Since supporting the ref-prefix capability is
> > optional anyway, the client has to further cull the response based on
> > their own patterns. So we can simply ignore the patterns once we cross a
> > certain threshold. Note that we have to ignore _all_ patterns, not just
> > the ones past our limit (since otherwise we'd send too little data).
> 
> This all makes sense to me. At some point, we should be able to go "I
> don't know what you're trying to do, but let me just ignore all this
> craziness and instead try to give you a useful result sooner rather than
> later".
> 
> I do wonder if we should document that the client can't trust us to
> actually do all this culling. In general, I find that it's a matter of
> hygiene for the client to do its own checks, but with this change they
> actually *need* to do them. (Unless they know our limit and that they're
> on the right side of it, but that kind of magic is even less hygienic.)

Perhaps we could say so more explicitly in the v2 protocol spec. I'll take a look.

Show 13 quoted lines

> > +               else if (skip_prefix(arg, "ref-prefix ", &out)) {
> > +                       if (too_many_prefixes) {
> > +                               /* ignore any further ones */
> > +                       } else if (data.prefixes.nr >= MAX_ALLOWED_PREFIXES) {
> > +                               strvec_clear(&data.prefixes);
> > +                               too_many_prefixes = 1;
> > +                       } else {
> > +                               strvec_push(&data.prefixes, out);
> > +                       }
> > +               }
> 
> Is it easier to reason about with something like this
> (whitespace-damaged) on top?

You're the second person to complain about this if-else chain. I'll take the hint. ;)

Show 28 quoted lines

> diff --git a/ls-refs.c b/ls-refs.c
> index 839fb0caa9..b3101ff361 100644
> --- a/ls-refs.c
> +++ b/ls-refs.c
> @@ -147,7 +147,6 @@ static int ls_refs_config(const char *var, const
> char *value, void *data)
>  int ls_refs(struct repository *r, struct packet_reader *request)
>  {
>         struct ls_refs_data data;
> -       int too_many_prefixes = 0;
> 
>         memset(&data, 0, sizeof(data));
>         strvec_init(&data.prefixes);
> @@ -164,14 +163,8 @@ int ls_refs(struct repository *r, struct
> packet_reader *request)
>                 else if (!strcmp("symrefs", arg))
>                         data.symrefs = 1;
>                 else if (skip_prefix(arg, "ref-prefix ", &out)) {
> -                       if (too_many_prefixes) {
> -                               /* ignore any further ones */
> -                       } else if (data.prefixes.nr >= MAX_ALLOWED_PREFIXES) {
> -                               strvec_clear(&data.prefixes);
> -                               too_many_prefixes = 1;
> -                       } else {
> +                       if (data.prefixes.nr <= MAX_ALLOWED_PREFIXES)
>                                 strvec_push(&data.prefixes, out);
> -                       }
>                 }

Hmm. At first I liked this, because it reduces the number of cases (and variables!). But there's something really subtle going on here. I thought at first it should be "<", but you are intentionally over-allocating by one entry to indicate the overflow. I.e., you've essentially stuffed the too_many_prefixes boolean into the count.

Show 8 quoted lines

> @@ -180,6 +173,9 @@ int ls_refs(struct repository *r, struct
> packet_reader *request)
>         if (request->status != PACKET_READ_FLUSH)
>                 die(_("expected flush after ls-refs arguments"));
> 
> +       if (data.prefixes.nr > MAX_ALLOWED_PREFIXES)
> +               strvec_clear(&data.prefixes);
> +

This is far from the parser, but I think that's OK. I'd probably couple it with a comment explaining why we need to clear rather than using what we got.

Show 8 quoted lines

> Maybe even name the macro TOO_MANY_PREFIXES (and bump it by one)
> to make the logic instead be
> 
>         if (data.prefixes.nr < TOO_MANY_PREFIXES)
>                 strvec_push(&data.prefixes, out);
>  ...
>         if (data.prefixes.nr >= TOO_MANY_PREFIXES)
>                 strvec_clear(&data.prefixes);

At first I thought this was just being cute, but it's an attempt to compensate for the off-by-one subtlety in the early check. I'll give it some thought. I kind of like it, but the fact that it took me a minute or three to be sure the code is correct makes me worried it's being too clever.

-Peff