From: Richard Hartmann <richih.mailinglist@gmail.com>
Date: Sat, 17 Apr 2010 01:10:56 GMT
Subject: Re: Git import of the recent full enwiki dump
Message-ID: <y2h2d460de71004161810p2c331099q4b2d7dabd01e5f8@mail.gmail.com>
In-Reply-To: <m2ofabb9a1e1004161719h977b53b7oc2c452c2a2b0025e@mail.gmail.com>

On Sat, Apr 17, 2010 at 02:19, Sverre Rabbelier <srabbelier@gmail.com> wrote:

> Assuming you do the import incrementally
> using something like git-fast-import (feeding it with a custom
> exporter that uses the dump as it's input) you shouldn't even need an
> extraordinary machine to do it (although you'd need a lot of storage).

I am using a Python script [1] to import the XML dump.


> Speaking of which, it might make sense to separate the
> worktree by prefix, so articles starting with "aa" go under the "aa"
> directory, etc?

Very good idea. What command would I need to send to
git-fast-import to do that?


> Hope that helps, and if you do convert it (and it turns out to be
> usable, and you decide to keep it up to date somehow), put it up
> somewhere! :)

It did.
I will make it available if it turns out to be useful. Keeping it up to
date might be harder unless they keep on releasing new
(incremental) snapshots.


Thanks,
Richard


[1] http://github.com/scy/levitation/blob/master/import.py