Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

From: Junio C Hamano <gitster@pobox.com>
Date: Sep 4, 2013, 21:27 UTC
Message-ID: <xmqqd2ool1ks.fsf@gitster.dls.corp.google.com>
In-Reply-To: <52279107.2070205@kitware.com>

Brad King <brad.king@kitware.com> writes:

> Nothing else uses LF NUL.  I chose it as a starting point for
> this very discussion, which I asked about in $gmane/233653.

The primary reason why LF raised my eyebrow was because the reason why many subcommands use "-z" (and NUL) is often because the payload may have LF in a record and LF cannot be used as a record separator without escaping. And they use NUL knowing that the payload data in fields cannot contain a NUL. If we used LF as a signal to define the structure of the record, it pretty much defeats the whole point of defining "-z" format. The -m reason string will be made into a single liner deep in the codepath but it _can_ contain LF.

I would have been more receptive to, say, double-NUL as a record terminator, while using a NUL as a field terminator, or something, but then we would need to have a way to express an empty field.

Show 5 quoted lines

> In this particular use case we know the last field will never
> be LF but that may not be so for future cases.  There is no way
> to represent sentinel-terminated arbitrary variable-width records
> of NUL-terminated fields without some kind of escaping for the
> sentinel value, but the whole point of -z is to avoid escaping.

Indeed, but one escape hatch we have is that payload will not contain NUL anywhere, so whenever we see a NUL, we can trust that it defines the structure of the record, and is not a part of the payload.

Stepping back a bit, here are some observations on the arguments update-ref can take:

 * "-m <reason>" is a reason given for this entire update. As the
   point of this new feature is to give an all-or-none update to one
   or more refs, I think we should not have to accept more than one
   reason (more specifically, the -m option does _not_ belong to a
   specific record that describes what happens to a single ref).

 * "-d <ref> <oldvalue>" is a way to delete a ref. <oldvalue> may be
   missing.

 * "--no-deref <ref> <newvalue> <oldvalue>" and "<ref> <newvalue>
   <oldvalue>" are ways to update or create a ref. Again <oldvalue>
   may be missing.

So it looks to me that one possible format that is easy to generate by machine without ambiguity may be:

    * The first record could be

      m NUL <reason strong> NUL

      but it is optional. The reason string may contain LF but just
      like invocation from the command line, LF will eventually
      cleaned up into a SP.

    * Then a series of records of different kinds follow.

      - A delete record looks like this:

        d NUL <ref> NUL <oldvalue> NUL

        If you want to delete the ref without "oldvalue" protection,
        just say

        d NUL <ref> NUL NUL

      - A create/update record looks like one of these:

        u NUL <ref> NUL <newvalue> NUL <oldvalue> NUL
        n NUL <ref> NUL <newvalue> NUL <oldvalue> NUL

        Again, if you want to delete the ref without "oldvalue"
        protection, just say

        u NUL <ref> NUL <newvalue> NUL NUL
        n NUL <ref> NUL <newvalue> NUL NUL

     * EOF signals the end of the request.

I am not saying the above is the best format, but the point is that the mode of the operation defines the structure, so unlike parsing xml or json where you first parse the structure and then interpret what each element means, you can define a simple format where the kind of element comes upfront to allow the parser/interpreter know what is expected to follow it.