Re: long fast-import errors out "failed to apply delta"

From: Dmitry Ivankov <divanorama@gmail.com>
Date: Jul 26, 2011, 21:09 UTC
Message-ID: <CA+gfSn8-+X9vrCYamN9Rg-PnGAQ3u5jH+S7LYxoZopMyEF9VPg@mail.gmail.com>
In-Reply-To: <20110726185504.GA2398@elie.dc0b.debconf.org>

On Wed, Jul 27, 2011 at 12:55 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:

Show 12 quoted lines

> Dmitry Ivankov wrote:
>> On Tue, Jul 26, 2011 at 10:58 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>
>>> Could you save the svn-fe output (e.g., by introducing "tee" in the
>>> middle of the "svn-fe | fast-import" pipeline) and put it up
>>> somewhere online?  This would also be a good starting point for coming
>>> up with a reduced testcase.
>>
>> It's 1.9G uncompressed, 0.7G lzo-compressed. Will setup a ftp or
>> torrent seed a bit later.
>
> Ah, never mind then. :)  Do you have a script to reproduce it?

I've uploaded the compressed svnadmin dump to dropbox - it's just 15M. So here is a Makefile to reproduce the bug. It needs svn-fe using deltas and fast-import knowing cat-blob-fd, cat-blob, ls. But once gcc.fi_stream is built, any more or less recent fast-import can be used.

bug: gcc.fi_stream gcc.fi_stream_more
        rm -rf tmp && \
        git init --bare tmp && \
        cat gcc.fi_stream gcc.fi_stream_more | \
        GIT_DIR=tmp git fast-import 2>&1 >/dev/null && \
        GIT_DIR=tmp git fsck
gcc.svndump.tar.bz2:
        wget http://dl.dropbox.com/u/36429197/gcc.svndump.tar.bz2
gcc.svndump: gcc.svndump.tar.bz2
        tar xjpf $<
        #command -v svnrdump && \
        #svnrdump dump -r0:15507 svn://gcc.gnu.org/svn/gcc >gcc.svndump
        #|| rm -f gcc.svndump
gcc.fi_stream: gcc.svndump
        command -v svn-fe && \
        rm -f backflow && \
        mkfifo backflow && \
        rm -rf tmp && \
        git init --bare tmp && \
        cat gcc.svndump | svn-fe 3<backflow | tee gcc.fi_stream | \
        GIT_DIR=tmp git fast-import --cat-blob-fd=3 3>backflow 2>&1 >/dev/null \
        || rm -f gcc.fi_stream
gcc.fi_stream_more:
        echo "commit refs/heads/master" >$@
        echo "mark :15508" >>$@
        echo "committer nobody
<nobody@69586fb2-bf4f-e74e-8260-21e4b08243f9> 874511505 +0000" >>$@
        echo "data 71" >>$@
        echo "This commit was manufactured by cvs2svn to create tag" >>$@
        echo "'egcs_ss_970917'." >>$@
        echo "#ls :14842 branches/gcc3/gcc/config" >>$@
        echo "M 040000 fbc83f80e9516c831918dff149058cba38a2e5f1
tags/egcs_ss_970917/gcc/config" >>$@
        echo "#ls :15459 trunk/gcc/config/alpha" >>$@
        echo "M 040000 9ffe84c346eec93b523d95ce642b54d54d23109c
tags/egcs_ss_970917/gcc/config/alpha" >>$@
        echo "#ls \"tags/egcs_ss_970917/gcc/config/alpha\"" >>$@
        echo "M 100644 inline
tags/egcs_ss_970917/gcc/config/alpha/vms-tramp.asm_" >>$@
        echo "data <<EOF" >>$@
        echo "123" >>$@
        echo "EOF" >>$@
        echo "progress Imported commit 15508" >>$@
.PHONY: bug