Re: [GSoC] Designing a faster index format

From: Jeff King <peff@peff.net>
Date: Apr 20, 2012, 20:02 UTC
Message-ID: <20120420200243.GA12578@sigill.intra.peff.net>
In-Reply-To: <877gxcoron.fsf@thomas.inf.ethz.ch>

On Thu, Apr 19, 2012 at 02:02:32PM +0200, Thomas Rast wrote:

Show 12 quoted lines

> > At the midterm, there will be a python prototype for the conversion of the old
> > index format to the new "future-proof" index format and a faster reader for the
> > new format. The native write part will be completed in the second part of the
> > Summer of Code.
> 
> I think this is the most important bit, and I'm curious if there are any
> objections/concerns on this.
> 
> It basically splits the project into
> 
> - first half: design, reader, raciness/locking issues
> - second half: writer, changes to take advantage of partial writing

I am OK with prototyping, and I'm OK with doing a throw-away prototype in a different language. But doing a throw-away prototype that lasts through the whole first half of the project makes me a little nervous. I'm worried that the conversion from a python prototype to actual git code in the second half is going to go badly and end up in either:

  1. A half-finished implementation, because the integration means we
     have to go back and re-visit assumptions made in the prototype that
     aren't valid in git itself.

  2. A conversion from python to C that is rushed and ends up doing
     things in a way that makes the conversion easier, but long term
     maintenance hard. Merge-recursive was originally written in python
     and converted to C, and it shows in the code. It's brittle, buggy,
     and hard to maintain (though that is just one data point; maybe it
     was just poorly written in the first place).

So I'd be more comfortable with the prototype being just for the design phase, and developing the real reader and writer simultaneously in C. Those implementations will also start as prototypes, but they can be iterated on, rather than being throw-aways.

-Peff