It looks to me that CR/LF conversion does not happen during fast-import even if I have core.autocrlf set to 'input'. Is this a bug or is there a reason for that?
threads / discuss / 9161
fast-import and core.autocrlf option
Subject: fast-import and core.autocrlf option
Johannes SchindelinRe: fast-import and core.autocrlf option
Hi,
On Sun, 22 Jul 2007, Dmitry Kakurin wrote:
> It looks to me that CR/LF conversion does not happen during > fast-import even if I have core.autocrlf set to 'input'. > Is this a bug or is there a reason for that?
fast-import works on blobs only. So it is expected. Besides, it would slow down fast-import tremendously if you wanted to introduce that behaviour. Therefore I suggest just rolling your own dos2unix instead of trying to play cute games with fast-import.
Ciao, Dscho
Shawn O. PearceRe: fast-import and core.autocrlf option
Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 10 quoted lines
> On Sun, 22 Jul 2007, Dmitry Kakurin wrote: > > > It looks to me that CR/LF conversion does not happen during > > fast-import even if I have core.autocrlf set to 'input'. > > Is this a bug or is there a reason for that? > > fast-import works on blobs only. So it is expected. Besides, it would > slow down fast-import tremendously if you wanted to introduce that > behaviour. Therefore I suggest just rolling your own dos2unix instead of > trying to play cute games with fast-import.
Right, in fast-import we only process blobs as raw blobs. Its rare that we have a file path associated with the blob data at the time that we are actually processing the blob itself. E.g. applications can send us blobs up front, before they even start to send us commits and path information.
So if we were to offer the CRLF->LF conversion feature in fast-import it would need to be an option supplied at the time the 'data' command issued, rather than based upon the gitattributes system that is normally used for working tree operations.
It wouldn't be to difficult to add such an option to the data command, but I'm not going to be able to get around to it anytime soon. This might be an easy change for someone else to attempt if they were interested in some Git hacking. Should all be isolated to the cmd_data() routine in fast-import.c, and just needs to call crlf_to_git() in convert.c, which is uh, static. :-)
-- Shawn.
Junio C HamanoRe: fast-import and core.autocrlf option
"Shawn O. Pearce" <spearce@spearce.org> writes:
Show 17 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote: >> On Sun, 22 Jul 2007, Dmitry Kakurin wrote: >> >> > It looks to me that CR/LF conversion does not happen during >> > fast-import even if I have core.autocrlf set to 'input'. >> > Is this a bug or is there a reason for that? >> >> fast-import works on blobs only. So it is expected. Besides, it would >> slow down fast-import tremendously if you wanted to introduce that >> behaviour. Therefore I suggest just rolling your own dos2unix instead of >> trying to play cute games with fast-import. > > Right, in fast-import we only process blobs as raw blobs. > Its rare that we have a file path associated with the blob data > at the time that we are actually processing the blob itself. E.g. > applications can send us blobs up front, before they even start to > send us commits and path information.
Don't the front-ends usually have path information already when they feed you a blob data, especially most of them operate on per-file history? If that is the case,...
> So if we were to offer the CRLF->LF conversion feature in fast-import > it would need to be an option supplied at the time the 'data' > command issued, rather than based upon the gitattributes system > that is normally used for working tree operations.
... the "option" could be "this came from such and such path" instead of "this is DOS data, please apply crlf".
Shawn O. PearceRe: fast-import and core.autocrlf option
Junio C Hamano <gitster@pobox.com> wrote:
Show 10 quoted lines
> "Shawn O. Pearce" <spearce@spearce.org> writes: > > Right, in fast-import we only process blobs as raw blobs. > > Its rare that we have a file path associated with the blob data > > at the time that we are actually processing the blob itself. E.g. > > applications can send us blobs up front, before they even start to > > send us commits and path information. > > Don't the front-ends usually have path information already when > they feed you a blob data, especially most of them operate on > per-file history? If that is the case,...
Depends. A frontend can feed huge streams of blobs and use marks to make us remember their SHA-1s, then later feed us those marks and connect them to paths. Such a frontend may not have exact path information available when it feeds the blobs to us. Then again maybe it does. Depends on how the source information was organized.
Show 7 quoted lines
> > So if we were to offer the CRLF->LF conversion feature in fast-import > > it would need to be an option supplied at the time the 'data' > > command issued, rather than based upon the gitattributes system > > that is normally used for working tree operations. > > ... the "option" could be "this came from such and such path" > instead of "this is DOS data, please apply crlf".
How do we setup .gitattributes? Should it be read from a branch in memory, or the working directory?
If it should be read from a branch, which branch? What if the frontend does not want the .gitattributes file in the history of the import? Putting it into a branch just to cause CRLF->LF translation during import would require filter-branch afterwards to strip out the file.
I'd rather make things very explicit. If the frontend wants us to apply a filter to a data chunk before we work further on it then the frontend should give us the .gitattributes rules to apply as part of the data command.
-- Shawn.