patch, 5 partsReplacing strbuf_getline_lf() by strbuf_getline() on trimmed input

This series deals with strbuf_getline_lf() in certain codepaths: Those, where the input that is read, is trimmed before doing anything that could possibly expect a CR character. As the codepath after trimming can not in any way expect a CR at the end of the line, because trim dropped it before the change, this can be safely done.

The series is an idea out of [1], where Junio proposed to replace the calls to strbuf_getline_lf() because it 'would [be] a good way to document them as dealing with "text"'. Apart from a general review, I would be happy about comments how I can improve this (and future) patches, as this is my first one. Especially I am not sure about how many arguments from the discussion on the mailing list should be put into the commits.

-Moritz

[1] http://thread.gmane.org/gmane.comp.version-control.git/284104

Moritz Neeb (5):
  bisect: read bisect paths with strbuf_getline()
  clean: read user input with strbuf_getline()
  notes: read copied notes with strbuf_getline()
  remote: read $GIT_DIR/branches/* with strbuf_getline()
  wt-status: read rebase todolist with strbuf_getline()

 bisect.c        | 2 +-
 builtin/clean.c | 6 +++---
 builtin/notes.c | 2 +-
 remote.c        | 2 +-
 wt-status.c     | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

    The lines read from BISECT_NAMES are trimmed with strbuf_trim()
    immediately. There is thus no logic expecting CR, so
    strbuf_getline_lf() can be replaced by its CRLF counterpart.

Signed-off-by: Moritz Neeb <lists@moritzneeb.de>
---
 bisect.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Moritz Neeb <lists@moritzneeb.de> writes:

We do not indent the whole log message.

You would also want to think about the necessity of strbuf_trim() here. Now strbuf_getline() would trim the trailing CR, would we still need to call strbuf_trim() here? The code will break if you just remove the call, but on the other hand, you will realize that the trimming done by calling it is excessive and unnecessary, once you inspect the code and learn who writes the file being read here and how.

On 02/01/2016 10:30 PM, Junio C Hamano wrote:

Thanks for spotting, I will change that in the next iteration.
 

I am not sure what you mean by excessive: How much can I assume that the input is like expected? The files we are talking about are supposed to be read and written by git only. But could be modified in theory with an editor, right? Then things could break, right? This question maybe holds true for the other patches as well, I still have to look into them.

Assuming that things are just like expected, what has to be done in read_bisect_paths before dequoting is to remove the leading space, which is added by 'git rev-parse --sq-quote "$@"' in git-bisect.sh.

I am just not sure, where to remove it:
* in sq_quote_argv? - see some analysis below.
* in read_bisect_paths via strbuf_ltrim instead of strbuf_trim,
  is that what you meant?
* in sq_dequote_argv_array? - see below as well.

I tried to understand, why the space is added by sq_quote_argv() in quotes.c in the first place. It is there since the beginning in 7cf6720. I have the feeling that the space is redundant for bisect, because the only other codepath where BISECT_NAMES is read is in bisect_visualize(), where the necessary space to separate the arguments is added before '$(...)':

eval '"$@"' --bisect -- $(cat "$GIT_DIR/BISECT_NAMES")

Other places where sq_quote_argv() and sq_dequote_to_argv_array() are called as well are builtin/am.c and trace.c.

In builtin/am.c when reading 'apply-opt', the data is trimmed in read_state_file, so the space in the beginning is superfluous.

Looking at trace.c it gets clear that this code is dependent on this space, otherwise the trace output will look like "trace: built-in: git'status'" which is obviously bogus. It would maybe make more sense to add this space directly there than to trim it in all the other places. I realized that this code came up in 7cf6720 as well which explains why things are like they are (and why I'm CC'ing Christian Couder).

Another option, at the moment my favourite because I think it's the least invasive, would be to "trim" this one space directly in sq_dequote_step(). See the patch below. This would not affect the tracing functionality, because there is no dequoting used.

-- >8 --
Subject: [PATCH] quote: remove leading space in sq_dequote_step

Because sq_quote_argv adds a leading space (which is expected in trace.c), sq_dequote_step should remove this space again, such that the operations of quoting and dequoting are inverse of each other.

Signed-off-by: Moritz Neeb <lists@moritzneeb.de>
---
quote.c | 2 ++
1 file changed, 2 insertions(+)

Moritz Neeb <lists@moritzneeb.de> writes:

These are all good questions you as a Git contributor to be asking yourself, and I really like the fact that you are thinking aloud here.

As to bisect pathspec, while I do not think it is sensible to change it in the middle of bisection, I do not think it would cause the bisect command to produce an incorrect result if you did so. You would be changing the definition of what is the "middle point" of the history on the fly by changing the pathspec, hence you may end up making the bisection suboptimal, but it shouldn't affect the correctness of the bisection.

So I tend to agree with you that it is a good idea to be prepared to parse what the end user may have futzed with the editor, as long as the user did not break the syntax of the quoted string.

Your answer to the question quoted at the top of this message could be a summary of the above reasoning, with a conclusion: "we would want to tolerate trailing (or leading) whitespaces the user might add with his editor; even though we use strbuf_getline() to remove the CR at the end of line, we still need to strbuf_trim() the line we read".

	Side note: of course, you can instead say "while it is
	technically possible, we have never advertised this file to
	be editable by user, or encouraged them to do so" and
	tighten the parsing to assume only what we write out,
	erroring out when we see any attempt to edit it.

Either way, the most important thing is to record the reasoning behind the design choice you made clearly in the proposed commit log message.

Thanks.

On 02/15/2016 06:05 AM, Junio C Hamano wrote:

Thanks for this feedback I will try to continue this thinking aloud whenever appropriate.

Sorry by the way for only answering every week - I currently only have time to work on git during the weekend. What is (assuming familiarity with the codebase) the expected "cooking"-time of a smaller patch? I'm sure the answer is "depends", but maybe you can give away some experiences.

On 02/15/2016 06:05 AM, Junio C Hamano wrote:

Yep I also think it would not affect the correctness. To reformulate your explanation: changing the pathspec does not change the good and bad "frontiers". And those should be still correct under the assumption you're still looking for the same bug/regression.

As to the question if it is sensible, I also don't really think it could be helpful. The only way I would expect it to be useful, it so "refine" the search by adding specific files to the filepath. But then it's possible to end up in a dead-end if the commits left are not touching the specified files with:

	No testable commit found.
	Maybe you started with bad path parameters?

I have also seen other messed up messages, like "-1 commits left" while playing around.

Now I am not convinced anymore, because I think if it potentially leads to troubles, it should not be made easier to get oneself into them.

I will include something like that in the patch v2 that is to follow soon.

    The input that is read is trimmed with strbuf_trim() immediately.
    There is thus no logic expecting CR, so strbuf_getline_lf() can be
    replaced by its CRLF counterpart.

Signed-off-by: Moritz Neeb <lists@moritzneeb.de>
---
 builtin/clean.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

The same comment (including "think if this trim is still necessary") as 1/5 applies here.

    The notes that are copied from stdin are trimmed with strbuf_rtrim()
    after splitting by ' '. There is thus no logic expecting CR, so
    strbuf_getline_lf() can be replaced by its CRLF counterpart.

Signed-off-by: Moritz Neeb <lists@moritzneeb.de>
---
 builtin/notes.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Same comment as 1/5 and 2/5 applies. You need to think if strbuf_rtim(split[1]) is necessary here.

    The line read from the branch file is directly trimmed after
    reading with strbuf_trim(). There is thus no logic expecting CR,
    so strbuf_getline_lf() can be replaced by its CRLF counterpart.

Signed-off-by: Moritz Neeb <lists@moritzneeb.de>
---
 remote.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

    In read_rebase_todolist() every line is, after reading, checked
    for a comment_line_char. After that it is trimmed via strbuf_trim().
    Assuming we do never expect a CR as comment_line_char,
    strbuf_getline_lf() can be safely replaced by its CRLF counterpart.

Signed-off-by: Moritz Neeb <lists@moritzneeb.de>
---

Notes:
    Looking at the code in read_rebase_todolist(), the
    lines are read, checked for a comment_line_char and then trimmed. I
    wonder why the input is not trimmed before checking for this character?
    Is it safe to replace strbuf_getline_lf() by strbuf_getline() anyway?
        The only case I can imagine that could lead to unexpected behaviour then
    would be when someone sets the comment_line_char to CR. How likely is that?
        Why is the trim _after_ checking for the comment char anyway? Should
    something like "   # foobar" not be considered as comment?
        I decided to roll out the patch because I considered it not as a risk to be
    taken seriously, that the comment line char will be '\r'.
        Meta-question: Should something of this discussion be put into the commit?

 wt-status.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Moritz Neeb <lists@moritzneeb.de> writes:

Yes to the meta question. Add something like this as the second paragraph of the log message, but do *not* change the patch text.

	The current code checks if the line begins with a comment
	line char (typically '#') before trimming, which is
	consistent with the way how commands like 'git commit'
	prepares commented lines in that this does not treat a line
	that begins with whitespaces and '#' as commented out.  We
	however made a mistake in the parser of "git rebase -i" long
	time ago to allow such a line to be treated as a comment,
	and made an exception with 1db168ee (rebase-i: loosen
	over-eager check_bad_cmd check, 2015-10-01).  It probably is
	a good idea to match that exception by swapping the order of
	comment check and trimming in this codepath as well.