Re: [PATCH] linear-assignment: fix potential out of bounds memory access

From: Thomas Gummerer <t.gummerer@gmail.com>
Date: Sep 12, 2018, 22:44 UTC
Message-ID: <20180912224410.GB1719@hank.intra.tgummerer.com>
In-Reply-To: <xmqqy3c6jx1m.fsf@gitster-ct.c.googlers.com>

On 09/12, Junio C Hamano wrote:

> Thomas Gummerer <t.gummerer@gmail.com> writes:

Show 25 quoted lines

> > --- >8 ---
> >
> > Subject: [PATCH] linear-assignment: fix potential out of bounds memory access
> >
> > Currently the 'compute_assignment()' function can may read memory out
> > of bounds, even if used correctly.  Namely this happens when we only
> > have one column.  In that case we try to calculate the initial
> > minimum cost using '!j1' as column in the reduction transfer code.
> > That in turn causes us to try and get the cost from column 1 in the
> > cost matrix, which does not exist, and thus results in an out of
> > bounds memory read.
> 
> This nicely explains what goes wrong.
> 
> > Instead of trying to intialize the minimum cost from another column,
> > just set it to INT_MAX.  This also matches what the example code in the
> > original paper for the algorithm [1] does (it initializes the value to
> > inf, for which INT_MAX is the closest match in C).
> 
> Yeah, if we really want to avoid INT_MAX we could use another "have
> we found any value yet?" boolean variable, but the caller in
> get_correspondences() does not even worry about integer overflows
> when stuffing diffsize to the cost[] array, and the other possible
> value that can go to cost[] array is COST_MAX that is mere 65k, so
> it would be OK to use INT_MAX as sentinel here, I guess.

Right, I'm not sure it would be worth introducing another boolean variable here. In the normal case we'll always enter the if condition inside the loop, and set a reasonable 'min' value.

That does not happen if we only have one column, and the 'min' will remain 'INT_MAX'. Now in that case it doesn't matter much, as having only one column means there's no possibility to assign anything, so the actual values shouldn't matter (at least that's my understanding of the algorithm so far).

Another improvement we may be able to make here is to not even try to compute the assignment if there's only one column for that reason, but I'm out of time today and the rest of my week looks a bit busy, so I probably won't get to do anything before the beginning of next week.

Show 50 quoted lines

> > Note that the test only fails under valgrind on Linux, but the same
> > command has been reported to segfault on Mac OS.
> >
> > Also start from 0 in the loop, which matches what the example code in
> > the original paper does as well.  Starting from 1 means we'd ignore
> > the first column during the reduction transfer phase.  Note that in
> > the original paper the loop does start from 1, but the implementation
> > is in Pascal, where arrays are 1 indexed.
> >
> > [1]: Jonker, R., & Volgenant, A. (1987). A shortest augmenting path
> >      algorithm for dense and sparse linear assignment
> >      problems. Computing, 38(4), 325–340.
> >
> > Reported-by: ryenus <ryenus@gmail.com>
> > Helped-by: Derrick Stolee <stolee@gmail.com>
> > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > ---
> >  linear-assignment.c   | 4 ++--
> >  t/t3206-range-diff.sh | 5 +++++
> >  2 files changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/linear-assignment.c b/linear-assignment.c
> > index 9b3e56e283..7700b80eeb 100644
> > --- a/linear-assignment.c
> > +++ b/linear-assignment.c
> > @@ -51,8 +51,8 @@ void compute_assignment(int column_count, int row_count, int *cost,
> >  		else if (j1 < -1)
> >  			row2column[i] = -2 - j1;
> >  		else {
> > -			int min = COST(!j1, i) - v[!j1];
> > -			for (j = 1; j < column_count; j++)
> > +			int min = INT_MAX;
> > +			for (j = 0; j < column_count; j++)
> >  				if (j != j1 && min > COST(j, i) - v[j])
> >  					min = COST(j, i) - v[j];
> >  			v[j1] -= min;
> > diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> > index 2237c7f4af..fb4c13a84a 100755
> > --- a/t/t3206-range-diff.sh
> > +++ b/t/t3206-range-diff.sh
> > @@ -142,4 +142,9 @@ test_expect_success 'changed message' '
> >  	test_cmp expected actual
> >  '
> >  
> > +test_expect_success 'no commits on one side' '
> > +	git commit --amend -m "new message" &&
> > +	git range-diff master HEAD@{1} HEAD
> > +'
> > +
> >  test_done