Git tree for old kernels from before the current tree

Continuing on with kernel archeology for embedded systems, any interest in making a git tree with all of the kernel versions from the beginning up to the start of the current git tree? No history in the tree, just a simple way to quickly fetch and select a copy of all the various old releases. I know they are all available ftp, git tree would organize them all in one place and let you fetch them all at once.

If this is small enough you could add it to the current kernel tree. Git's super diffing performance might make this fairly small.

My dream system lets me checkout an ancient version, apply the diff from the vendor (or expand a tarball), use git status to see what files changed, and git diff to get detailed changes (all minus CVS keyword expansion noise). Major bonus points if it can detect detect a subsystem that has been backported.

On Jul 22 2007 16:49, Jon Smirl wrote:

Well, it would be cool if history was somehow available (I recognize this would be a lot of work). One currently has to go through the bk repo if something is to be searched for in that timeframe.

In fact, we could bisect on it. Though not sure how useful that is with really old versions :)

	Jan

On Sun, Jul 22, 2007 at 11:00:15PM +0200, Jan Engelhardt wrote:

Err, that's crap. Have you even looked at gitweb? There's at least:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git

This has trees all the way back to 2.5.0.

and Linus also has:

git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/old-2.6-bkcvs.git

for the BK history.

That's already more history than most people have any immediate use for, going beyond that starts getting in to general completeness before practical application territory.

Anyone still sending 2.4 patches with the intent of them being moved forward and applied to a current kernel needs to be killfiled.

On Jul 23 2007 06:13, Paul Mundt wrote:

Oh I did not know that. I don't glance too much around in git://git.kernel.org/ because there are sooo many :)

	Jan

On Jul 22 2007 23:46, Jan Engelhardt wrote:

Actually back to 2.4.0, including history. That I'd call mission done then :)

	Jan

Jan Engelhardt wrote:

Wouldn't be hard to make a git tree with all the patches all the way back to 0.01 even...

	-hpa

H. Peter Anvin wrote:

It'd be delightful from a completeness standpoint (and I do love completeness), but considering it already takes a good 20 minutes to clone the 2.6 tree over a respectable cable connection, I'd have to object on the grounds of size. Now, if it was kept off in its own tree for people who don't mind ravaging kernel.org resources to satisfy their own curiosity, that's fine too.

  -- m. tharp

On 7/22/07, Michael Tharp <gxti@partiallystapled.com> wrote:

git has an extremely effective diffing mechanism. You may surprised at how little it adds. For example, git compressed the 2.6GB mozilla cvs tree down to 400MB.

I used to clone trees all the time, but now I'm much better at using git and I haven't cloned a complete tree from kernel.org in a year. git remote is a cool feature.

On Sun, 22 Jul 2007, H. Peter Anvin wrote:

I actually tried to get something like this together back in the BK days and early in the SCO saga. It was pretty painful to try to find all the historic trees and patches - they're all in different format, and some of them are unreliable (ie CVS imports by people like Ted).

The good news is that git would be a lot more natural to the process of trying to create a history, because you could basically import random trees, and tag them as just independent trees, and then re-create the history after-the-fact by trying to stitch them all together. And if you find a new tree, you'd just re-stitch it - something that was very hard to do with BK (and BK generally wouldn't help you with keeping multiple independent trees around, and wouldn't generally accept the notion of re-doing the histories and keeping various versions of the histories around).

So I've been thinking about trying to re-create some really old history into git, but it's still a lot of work.. And obviously not very useful, just interesting from an archeological standpoint.

			Linus

On Mon, 23 Jul 2007, Linus Torvalds wrote:

I started this once.

I have (sort of) a GIT tree with all Linux revisions that I could find from v0.01 up to v1.0.9. But the most interesting information and also what is the most time consuming is the retrieval of announcement messages for those releases in old mailing list or newsgroup archives to serve as commit log data. It seems to be even arder to find for post v1.0 releases.

Nicolas

On Mon, 23 Jul 2007, Nicolas Pitre wrote:

Yes, I agree. Google finds some of them, but (a) I was never very good about announcements anyway and (b) there's nothing really good to search for, so it's very hit-and-miss.

Some of the really early release notes are easy to find, just because I made them available with the sources, but mostly I'd just have posten to the newsgroup/mailing lists.

If somebody creates a reasonably good tree (ie all the trees, easily diffable, tied together with at least *some* commit history and the most easily found release notes), I'm willing to try to spend some time just writing down recollections from looking at the diffs. Not very reliable, but better than nothing. This is the kind of thing that the "notes" thing could be useful for, since there are others around who might also be interested in adding their commentary.

		Linus

On Mon, 23 Jul 2007, Linus Torvalds wrote:

That's what I used when available, especially to properly time stamp those commits. Using the latest date on files included in the archive isn't always reliable.

OK so actually what I have is from v0.01 up to v1.0 creating 93 commits.

What is missing is:

 - v0.02 sources

 - v0.10 announcement

 - v0.96 sources

 - v0.99.12 announcement

 - sources for v0.99.13{abcdefghij} (got k, don't know where the serie 
   ends) as well as announcements for all of them

 - all announcements for v0.99.14{a-z} except for pl14r

 - announcements for pl15c to pl15j, 1.0-pre1, and ALPHA-1.0.

Otherwise the archive appears fairly complete with almost 3 years of Linux development history captured in a 3MB pack file.

Nicolas

On Mon, 23 Jul 2007, Nicolas Pitre wrote:

I think this really is gone. 0.03 was such an improvement on 0.02 that I think what happened was that I literally removed 0.02 (hey, it wasn't historically interesting at the time!). It's not the first time people have wondered about it.

0.03 was the first version where you could actually do things under Linux, and I think I could compile etc. I *think* it was released pretty close after 0.02, which made 0.02 appear even more flawed and a brown-paper-bag release.

Hmm. That one would be interesting, since the reason for the 0.03->0.10 jump was that I was getting so happy with how it was actually working for me (ie able to compile itself under itself). But I don't see it, and it's not on google in the comp.os.minix archives either, afaik.

Hmm. Odd. Might be another case of "0.96a was released as a brown-paper-bag fix for 0.96, and the latter hidden in shame".

Stupid bugs only appear endearing in retrospect.

Well, the 0.99.12 announcement is found by google.

	http://groups.google.com/group/comp.os.linux.announce/browse_thread/thread/8a19289f68a4af35/fe433c9df4b382a5?lnk=st&q=&rnum=1#fe433c9df4b382a5

In general, google groups (search by date and author, and make the group be something like *linux*) is good, I found the above on the first try.

Many of those might not have merited announcements. At some point I was just making tar-balls a few days apart, to let people track it.

Heh, nice. Those early versions are all smaller than the patch set we generate in a day these days :)

		Linus

On Mon, 23 Jul 2007, Linus Torvalds wrote:

Could that version be found anywhere?

Anyone out there keeping an archive of those old posts? Archival sites appear to have a black hole during that period.

Yeah... Don't know what happened. I must have found it then because I had a file to store it in my data directory, but somehow it was empty. Fixed now.

Nicolas

didn't Tigran at some point get a full history together?

On Mon, Jul 23, 2007 at 03:06:09PM -0400, Nicolas Pitre wrote:

Umm... IIRC, tar was unhappy with several tarballs on ftp.kernel.org. Do you have replacements (or instructions re combination of tar(1) options to make it eat them)?

Al Viro wrote:

There is some stuff at http://mirrors.kernel.org/oldlinux/

That's all I have, I'm afraid...

	-hpa

On Tue, 24 Jul 2007, Al Viro wrote:

I have sanitized .tgz files that I use to stuff a Git repo with. I recall that some of them were reconstructed through patching an earlier or later kernel version because the original ones were corrupted. Some patches were retrieved from other archival sites, etc. Then the result was cross checked with summary lists like this one:

http://www.uwsg.iu.edu/hypermail/linux/kernel/0110.1/1684.html

This was a while ago so I don't remember the exact steps, but that wasn't always trivial.

Nicolas

Nicolas Pitre wrote:

Then there is 0.95a, 0.95c and 0.95c+, which as far as I know only ever existed as 0.95 + patches posted to alt.os.linux.

	-hpa

On Mon, 23 Jul 2007, H. Peter Anvin wrote:

Indeed, already got those.

Nicolas

On Mon, 2007-07-23 20:10:23 -0400, Nicolas Pitre <nico@cam.org> wrote:

Some years ago, I also worked on old Linux releases. Have you found the tarballs in my directory at kernel.org?

MfG, JBG

On Mon, 20 Aug 2007, Jan-Benedict Glaw wrote:

No, I didn't know about them.

v1.0 though.

Nicolas

* Mon, 23 Jul 2007 11:02:39 -0700 (PDT)

Maybe this can be useful somehow: ftp://ftp.shout.net/pub/users/mec/kcs ____

On Mon, Jul 23, 2007 at 09:55:24AM -0700, Linus Torvalds wrote:

Um, *I* never had the bad taste to import Linux kernels into CVS.  :-)

I'm pretty sure we never had anything like that on tsx-11.mit.edu, either.

    	   	   	     	      	   - Ted

On Mon, 23 Jul 2007, Theodore Tso wrote:

Well, I remember having to clean up some of the source archives I've found which contained CVS directories. Can't tell which one anymore, but they were amongst the oldest ones.

Nicolas

On Mon, 23 Jul 2007, Theodore Tso wrote:

Ahh. I just checked. 

RCS.

There are old linux archive of yours that has some RCS files in it (0.10 and 0.12 at least)

		Linus

On Mon, Jul 23, 2007 at 12:44:57PM -0700, Linus Torvalds wrote:

Ah, yes, I used to use RCS in order to generate the patches that I sent to use. The RCS directories were short-term, and never lasted longer until the next kernel release. I'm kind of surprised that I actually had archives that had RCS files, since tsx-11 should have just had the original tar.gz files from ftp.cs.helsinki.edu. Maybe for some reason the original files got nuked and I replaced it with one of my saved files at one point. I dunno....

       	  	      	     	     - Ted

On 7/22/07, Paul Mundt <lethal@linux-sh.org> wrote:

These patches are coming from companies that aren't interested in participating in the GPL process but are being forced into releasing code because of the license. Some of them will go out of their way to make the changes difficult to read. All of the patches I am looking at come from embedded systems, many of these systems are still shipping 2.4 kernels.

Most of the patches contain junk, but there are occasional diamonds. One I'm looking at contains code for accessing encryption hardware. The goal is to look at the vendor diffs and see if I can spot anything useful. Spotting something useful can be hard if there are 100,000 lines of noise in the diffs, I'm also trying to spot missing drivers so that we can ask for more code.

On Sun, 22 Jul 2007, Jon Smirl wrote:

I just put the archive I've gathered so far in a Git tree on kernel.org:

    git://git.kernel.org/pub/scm/linux/kernel/git/nico/archive.git

It is also available through gitweb from:

    http://git.kernel.org/?p=linux/kernel/git/nico/archive.git

It currently stops at Linux v1.0. I would like to continue populating this archive up to the current "Linux kernel history" Git repository from Thomas Gleixner which currently doesn't go further back than v2.4.0. This is however extremely time consuming to track down old mailing list and newsgroup archives in order to find announcements to serve as commit log material. Therefore I'd be really glad if people could help me find those and send me copies of relevant announcements. The content of the current archive should give a good idea of what I'm looking for.

Also, the following items are currently missing and apparently vanished from the surface of the planet:

 - v0.02 sources

 - v0.03 sources and announcement

 - v0.10 announcement

 - v0.96 sources

 - sources for v0.99.13{abcdefghij} (got k, don't know where the serie
   ends) as well as announcements for all of them

 - all announcements for v0.99.14{a-z} except for pl14r

 - announcements for pl15c to pl15j, 1.0-pre1, and ALPHA-1.0.

If you happen to have any of those please drop me a note.

Otherwise the archive appears fairly complete with almost 3 years of Linux development history captured in a 3MB pack file.

Nicolas

On Sun, Aug 19, 2007 at 03:24:56PM -0400, Nicolas Pitre wrote:

 
I don't mean to nit-pick but, would it be possible to have the timestamps
for the commits closer to the release dates?

I must say, impressive work.

Thanks!

Josef 'Jeff' Sipek.

On Mon, 20 Aug 2007, Josef Sipek wrote:

Well, the author timestamp already is, based on the date of the announcement message used for the commit log, including the timezone. If you find errors in those please tell me.

The commit timestamp, however, is the time when _I_ created the repository, hence it should not be that interesting.

Thanks!

Nicolas

On Mon, Aug 20, 2007 at 01:40:52PM -0400, Nicolas Pitre wrote:

Ah ok. Too bad gitweb displays the commit times => throws off some of the sorting too.

/me prods gitweb people

Too bad that's the date you see all over the place :( 

Jeff.