Re: Ad: fast-import problem importing dos format files under cygwin

From: Shawn O. Pearce <spearce@spearce.org>
Date: Dec 3, 2008, 16:04 UTC
Message-ID: <20081203160446.GY23984@spearce.org>
In-Reply-To: <51143.194.138.12.144.1228311791.squirrel@artax.karlin.mff.cuni.cz>

Jan Hudec <bulb@ucw.cz> wrote:

Show 19 quoted lines

> Dne 3 Prosinec 2008, 13:18, Johannes Sixt napsal(a):
> > Jan Hudec schrieb:
> >> On 3 December 2008, 08:51, Jan Hudec wrote:
> >>> Hello folks,
> >>>
> >>> I have been playing with fast-import in cygwin and I have problems
> >>> importing files with CR/LF line-endings. The size in data command is
> >>> calculated including the CRs and than the file is copied binary to the
> >>> fast-import input stream. However fast-import skips the CRs when
> >>> reading,
> >>> overreads by that number of bytes and fails when it tries to read the
> >>> next command from the middle.
> >
> > Do you happen to have core.autocrlf set in some way and could it make a
> > difference for fast-import? I have it unset.
> 
> I have it set to false explicitly in global config. Tried with not having
> it set at all and gives the same problem. Since the previous version of
> MSys Git worked for me, I suspect it's somehow cygwin-related.

Huh. So fast-import *never* does auto-CRLF conversion, even if the property is set. It just doesn't make those calls internally. It blindly copies data from the input stream into the pack. No exceptions.

fast-import under-reading near CRs and getting misaligned on its input indicates that the stdio library has given us a FILE* for stdin which is converting CRLF pairs into LFs, even within an fread() call.

My guess here is fast-import's stdin is set in text mode, but it really needs to be in binary mode. fast-import.c never attempts to correct that when it starts, so on DOS based systems we are probably totally screwed from the beginning...

-- 
Shawn.