Bug: git log showing nothing when using --since and --until flags with specific dates

Hi all,

Apologies if this has already been raised or PEBCAK, but I've noticed a bug where git log with certain date ranges breaks things. It appears to be any --since date with a --until date in the future between 2014-12-01 and 2014-12-09. Dates from 2014-12-10 appear to work, and so does the date 2015-12-01.

Tested with the following versions:

git version 2.2.0.rc1.18.gf6f61cb on Ubuntu

git version 2.0.1 on whatever the latest version of OS X is.

To reproduce, on a git repo with a recent enough change to view a checkin after October 1 2014, run 'git log --since=2014-10-01 --until=2014-12-02' - no errors or anything to indicate the command failed are shown, now run 'git log --since=2014-10-01 --until=2014-12-10'.

Btw, the mail daemon appears to reject emails with '550 5.7.1 Content-Policy reject msg: The message contains HTML subpart, therefore we consider it SPAM or Outlook Virus. TEXT/PLAIN is accepted.! BF:<U 0.499737>; S1752168AbaKMAGd' - makes reporting bugs a bit of a hassle...

Cheers,

Colin Smith

On Thu, Nov 13, 2014 at 11:27:06AM +1100, Colin Smith wrote:

Ugh. Approxidate strikes again:

  for i in 2014-11-01 2013-12-01 2014-12-01; do
    ./test-date approxidate $i
  done

produces:

  2014-11-01 -> 2014-11-01 09:35:19 +0000
  2013-12-01 -> 2013-12-01 09:35:19 +0000
  2014-12-01 -> 2014-01-12 09:35:19 +0000

The first two are right, but the fourth one is not. It's probably something simple and stupid.

-Peff

On Thu, Nov 13, 2014 at 04:36:06AM -0500, Jeff King wrote:

Less simple and stupid than I thought, but I think I have a fix. It is not about December specifically, but about the date being in the future. The first patch is a cleanup to help us more accurately test the bug; the interesting bits are in the second one.

  [1/2]: pass TIME_DATE_NOW to approxidate future-check
  [2/2]: approxidate: allow ISO-like dates far in the future

-Peff

The approxidate functions accept an extra "now" parameter to avoid calling time() themselves. We use this in our test suite to make sure we have a consistent time for computing relative dates. However, deep in the bowels of approxidate, we also call time() to check whether possible dates are far in the future. Let's make sure that the "now" override makes it to that spot, too, so we can consistently test that feature.

Signed-off-by: Jeff King <peff@peff.net>
---
This code path also gets called from the more-strict parse_date (which
does not know about the current time). This continues to call time()
dynamically, but it is not clear to me if that ever actually matters in
practice.

 date.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

When we are parsing approxidate strings and we find three numbers separate by one of ":/-.", we guess that it may be a date. We feed the numbers to match_multi_number, which checks whether it makes sense as a date in various orderings (e.g., dd/mm/yy or mm/dd/yy, etc).

One of the checks we do is to see whether it is a date more than 10 days in the future. This was added in 38035cf (date parsing: be friendlier to our European friends., 2006-04-05), and lets us guess that if it is currently April 2014, then "10/03/2014" is probably March 10th, not October 3rd.

This has a downside, though; if you want to be overly generous with your "--until" date specification, we may wrongly parse "2014-12-01" as "2014-01-12" (because the latter is an in-the-past date). If the year is a future year (i.e., both are future dates), it gets even weirder. Due to the vagaries of approxidate, months _after_ the current date (no matter the year) get flipped, but ones before do not.

This patch drops the "in the future" check for dates of this form, letting us treat them always as yyyy-mm-dd, even if they are in the future. This does not affect the normal dd/mm/yyyy versus mm/dd/yyyy lookup, because this code path only kicks in when the first number is greater than 70 (i.e., it must be a year, and cannot be either a date or a month).

The one possible casualty is that "yyyy-dd-mm" is less likely to be chosen over "yyyy-mm-dd". That's probably OK, though because:

  1. The difference happens only when the date is in the
     future. Already we prefer yyyy-mm-dd for dates in the
     past.

  2. It's unclear whether anybody even uses yyyy-dd-mm
     regularly. It does not appear in lists of common date
     formats in Wikipedia[1,2].

  3. Even if (2) is wrong, it is better to prefer ISO-like
     dates, as that is consistent with what we use elsewhere
     in git.

[1] http://en.wikipedia.org/wiki/Date_and_time_representation_by_country [2] http://en.wikipedia.org/wiki/Calendar_date

Reported-by: Colin Smith <colin.webdev@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
---
I did not single out "yyyy-mm-dd" here versus "yyyy/mm/dd"; the change
applies no matter which separator is used. If we wanted to be more
conservative, we could apply it only to the more ISO-looking form with
dashes, but I tend to think the reasoning makes sense no matter which
separate you use (i.e., I do not think any variant with year then day is
in common use).

 date.c          | 29 ++++++++++++++++-------------
 t/t0006-date.sh |  3 +++
 2 files changed, 19 insertions(+), 13 deletions(-)

Jeff King <peff@peff.net> writes:

Doesn't the new argument '0', which is "allow-future", look somewhat strange when we are already passing refuse_future?

On Thu, Nov 13, 2014 at 01:11:46PM -0800, Junio C Hamano wrote:

To be honest, I had trouble figuring out what the name "refuse_future" really meant. We do skip the future check, but it also means that is_date will munge the "struct tm" directly, even if we do not find a valid date. That worried me a bit.

But yeah, in theory, the callers I wanted to tweak can just pass in a NULL refuse_future.

-Peff

On Thu, Nov 13, 2014 at 04:36:47PM -0500, Jeff King wrote:

So here's what the patch looks like just using refuse_future.

It's definitely nicer to read, and it passes the tests. But I am still concerned there is some unknown case that is impacted by us half-filling out the tm_mon and tm_mday fields of the "struct tm" in the first half of is_date.

-- >8 --
Subject: approxidate: allow ISO-like dates far in the future

When we are parsing approxidate strings and we find three numbers separate by one of ":/-.", we guess that it may be a date. We feed the numbers to match_multi_number, which checks whether it makes sense as a date in various orderings (e.g., dd/mm/yy or mm/dd/yy, etc).

One of the checks we do is to see whether it is a date more than 10 days in the future. This was added in 38035cf (date parsing: be friendlier to our European friends., 2006-04-05), and lets us guess that if it is currently April 2014, then "10/03/2014" is probably March 10th, not October 3rd.

This has a downside, though; if you want to be overly generous with your "--until" date specification, we may wrongly parse "2014-12-01" as "2014-01-12" (because the latter is an in-the-past date). If the year is a future year (i.e., both are future dates), it gets even weirder. Due to the vagaries of approxidate, months _after_ the current date (no matter the year) get flipped, but ones before do not.

This patch drops the "in the future" check for dates of this form, letting us treat them always as yyyy-mm-dd, even if they are in the future. This does not affect the normal dd/mm/yyyy versus mm/dd/yyyy lookup, because this code path only kicks in when the first number is greater than 70 (i.e., it must be a year, and cannot be either a date or a month).

The one possible casualty is that "yyyy-dd-mm" is less likely to be chosen over "yyyy-mm-dd". That's probably OK, though because:

  1. The difference happens only when the date is in the
     future. Already we prefer yyyy-mm-dd for dates in the
     past.

  2. It's unclear whether anybody even uses yyyy-dd-mm
     regularly. It does not appear in lists of common date
     formats in Wikipedia[1,2].

  3. Even if (2) is wrong, it is better to prefer ISO-like
     dates, as that is consistent with what we use elsewhere
     in git.

[1] http://en.wikipedia.org/wiki/Date_and_time_representation_by_country [2] http://en.wikipedia.org/wiki/Calendar_date

Signed-off-by: Jeff King <peff@peff.net>
---
 date.c          | 4 ++--
 t/t0006-date.sh | 3 +++
 2 files changed, 5 insertions(+), 2 deletions(-)

Jeff King <peff@peff.net> writes:

Ah, OK. That worries me, too, now you mention it. I just didn't see it myself ;-)

On Fri, Nov 14, 2014 at 06:46:19AM +0100, Elia Pinto wrote:

It is not good to omit any explanation and just include a link, like:

  Fixes the bug reported in http://...

because people who are reading "git log" have to follow that link to even see what you are talking about (and the link might go away, or they might not have access at the time).

It is fine, and even desirable, to summarize the relevant content of a resource and provide a link for people who want to dig further. In this case, I am saying "Wikipedia claims that nobody uses this format" and backing it up with a link to indicate which pages I checked. You do not have to follow the link to know what I am saying, but if you want to dig deeper, you at least know where I left off my research.

Does that make sense?

-Peff

Jeff King <peff@peff.net> writes:

What you wrote matches the level of details I have been trying to stick to when writing references in my own log messages and when tweaking others' log messages.

Thanks.