Re: [PATCH v2 0/5] Fun with cpp word regex
- From
Johannes Sixt <j6t@kdbg.org>
- Date
- Oct 8, 2021, 22:11 UTC
- Message-ID
- <25363715-dc39-1f18-a937-f715b106f529@kdbg.org>
- In-Reply-To
- <87r1cvmg0c.fsf@evledraar.gmail.com>
Am 08.10.21 um 22:07 schrieb Ævar Arnfjörð Bjarmason:
Show 7 quoted lines
> * I wonder if it isn't time to split up "cpp" into a "c" driver, > e.g. git.git's .gitattributes has "cpp" for *.[ch] files, but as C++ > adds more syntax sugar. > > So e.g. if you use "<=>" after this series we'll tokenize it > differently in *.c files, but it's a C++-only operator, on the other > hand probably nobody cares that much...
Yes, it is that: <=> won't appear in a correct C file (outside of comments), so no-one will care. As far as tokenization is concerned, C is a subset of C++. I don't think we need to separate the drivers.
Show 7 quoted lines
> * I found myself back-porting some of your tests (manually mostly), > maybe you disagree, but in cases like 123'123, <=> etc. I'd find it > easier to follow if we first added the test data, and then the > changed behavior. > > Because after all, we're going to change how we highlight existing > data, so testing for that would be informative.
Good point. I'll work a bit more on that.
Show 10 quoted lines
> * This pre-dates your much improved tests, but these test files could > really use some test comments, as in: > > /* Now that we're going to understand the "'" character somehow, will any of this change? */ > /* We haven't written code like this since the 1960's ... */ > /* Run & free */ > > I.e. we don't just highlight code the compiler likes to eat, but also > comments. So particularly for smaller tokens that also occur in > natural language like "'" and "&" are we getting expected results?
Comments are free text. Anything can happen. There is no such thing as "correct tokenization" in comments. Not interested.
Thank you for the review. -- Hannes