Re: git cat-file --batch surprising carriage return behavior

From: Jeff King <peff@peff.net>
Date: Oct 8, 2019, 20:00 UTC
Message-ID: <20191008200050.GA26453@sigill.intra.peff.net>
In-Reply-To: <20191008192257.GA16870@kitenet.net>

On Tue, Oct 08, 2019 at 03:22:57PM -0400, Joey Hess wrote:

Show 8 quoted lines

> I'm surprised to find that git cat-file --batch, on a Linux system,
> strips the \r from an input like "HEAD:foo\r\n"
> 
> It's obvious, of course, that it will remove the newline, and so this
> interface cannot be used to query about a filename that, for some
> horrible reason[1], contains a newline. But very surprising that it
> cannot be used for filename that contains a carriage return, at least
> on a non-Windows system.

This is likely due to b42ca3dd0f (cat-file: read batch stream with strbuf_getline(), 2015-10-28), and the matching c8aa9fdf5d (strbuf: make strbuf_getline_crlf() global, 2015-10-28).

I agree it's a bit surprising (though OTOH, I imagine the old behavior surprised some people in the opposite direction).

> The docs for cat-file --batch say the list of objects is separated by
> linefeeds. I don't know if updating the docs is the best fix.
> (I'd be happy to use a -z if it had one.)

Yeah, I agree that a -z option is the best path forward. For non-z input, I'm tempted to say we could unquote entries that start with a double-quote (the match to how we handle filenames in non-z diff output). That would mean breaking compatibility for refnames that start with a quote, though. If we just add a new "-z", that's less disruptive _and_ easier to use.

I suspect it's not entirely sufficient for clean input, though. You're not feeding filenames but rather full "object names". I wouldn't be surprised if we mis-parse "$rev:$path" when $path has "@{}" or similar in it.

So what you may actually want is some more robust input format that lets you specify the filename as an independent NUL-terminated entity.

-Peff