Re-done kernel archive - real one?

Ok, nobody really objected to the notion of leaving the kernel history behind for now, and in fact most people seemed to basically agree. So with that decided, the old kernel testing tree was actually perfectly ok, except it had been build up with the old-style commit date handling, which made me not want to use it as a base for any real work.

So I re-created the dang thing (hey, it takes just a few minutes), and pushed it out, and there's now an archive on kernel.org in my public "personal" directory called "linux-2.6.git". I'll continue the tradition of naming git-archive directories as "*.git", since that really ends up being the ".git" directory for the checked-out thing.

I'm not going to announce it on linux-kernel yet, because I don't think it's useful to anybody but a git person anyway. Besides, I don't actually know how happy the kernel.org people are about this distribution method and whether it ends up being a horrible disaster for the mirroring setup.

Peter made some noises about /pub/scm, which makes sense, and would be a better place than my public tree. Apparently there are other places that are willing and able to host things too, so we'll see.

NOTE! The roughly 10x expansion of archive size goind from BK to git ends up in a similar 10x bandwidth expansion, in addition to just the overhead of reading tons of directory entries and comparing them (which is what both a wget and rsync thing ends up doing). I'm sure we can bring that down with smarter synchronization tools, but I also suspect that's some way away.

So is real common usage, though, so maybe it's not that bad at all. Who knows. We haven't hit a single real snag so far (except it took several days longer than I expected, but hey, I expect lots of things ;), and I'm sure real usage will show lots of them.

Similarly, we don't really have real merging, which makes tracking harder, but I suspect actually having a tree out there will make people more motivated and have more of a test-case. I'm feeling good enough about the plumbing that I think I solved the "hard" part of it, and now it's just the boring 95% left - scripting around it.

I think that with the new merge model, the easiest thing to do is to just download all new objects, and then download the HEAD file under a new name.

Ie we have two phases to the merge: first get the objects, with something like

	repo=kernel.org:/pub/kernel/people/torvalds/linux-2.6.git
	rsync --ignore-existing -acv $(repo)/ .git/

which will _not_ download the new HEAD file (since you already have one of your own), and then when you actually decide to merge you do

	rsync -acv $(repo)/HEAD .git/MERGE_WITH

and now you can look at your old HEAD, and the MERGE_WITH thing, look up the parents, and then do

	read-tree -m <parent-tree> <head-tree> <merge-with-tree>
	write-tree
	commit-tree <result-tree> -p <head-tree> -p <merge-with-tree>

(which should actually _work_, assuming that the merge had no file conflicts).

This seems to be a sane way to do merges, and if the scripting starts from there and then becomes smarter...

		Linus

On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:

We need to work out how we're going to manage to get our git changes to you. At the moment, I've very little idea how to do that. Ideas?

At the bottom is the script itself. There's probably some aspects of it which aren't nice, maybe Petr can advise on this (and maybe increase the functionality of the git shell script to fill in where necessary.)

However, I've made a start to generate the necessary emails. How about this format?

I'm not keen on the tree, parent, author and committer objects appearing in this - they appear to clutter it up. What're your thoughts?

I'd rather not have the FQDN of the machine where the commit happened appearing in the logs. (I've 'xxxx'd it out for the time being, because I'd rather not have yet more email-address-like objects get into spammers databases with which to hammer my 512kbps DSL line.)

Linus,

Please incorporate the latest ARM changes.

This will update the following files:

 arm/kernel/process.c                |   15 +++++++++++----
 arm/kernel/traps.c                  |    8 ++------
 arm/lib/changebit.S                 |   11 ++---------
 arm/lib/clearbit.S                  |   13 ++-----------
 arm/lib/setbit.S                    |   11 ++---------
 arm/lib/testchangebit.S             |   15 ++-------------
 arm/lib/testclearbit.S              |   15 ++-------------
 arm/lib/testsetbit.S                |   15 ++-------------
 arm/mach-footbridge/dc21285-timer.c |    4 ++--
 arm/mach-sa1100/h3600.c             |    2 +-
 asm-arm/ptrace.h                    |    5 +----
 asm-arm/system.h                    |    3 +++
 12 files changed, 32 insertions(+), 85 deletions(-)

through these ChangeSets:

	tree 7c4d75539c29ef7a9dde81acf84a072649f4f394
	parent d5922e9c35d21f0b6b82d1fd8b1444cfce57ca34
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113749462 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113749462 +0100
	
	[PATCH] ARM: bitops
	
	Convert ARM bitop assembly to a macro.  All bitops follow the same
	format, so it's silly duplicating the code when only one or two
	instructions are different.
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>
	tree fc10d3ffa6062cda10a10cb8262d8df238aea4fb
	parent 5d9a545981893629c8f95e2b8b50d15d18c6ddbc
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113749436 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113749436 +0100
	
	[PATCH] ARM: showregs
	
	Fix show_regs() to provide a backtrace.  Provide a new __show_regs()
	function which implements the common subset of show_regs() and die().
	Add prototypes to asm-arm/system.h
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>
	tree 5591fced9a2b5f84c6772dcbe2eb4b24e29161fc
	parent 488faba31f59c5960aabbb2a5877a0f2923937a3
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113748846 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113748846 +0100
	
	[PATCH] ARM: h3600_irda_set_speed arguments
	
	h3600_irda_set_speed() had the wrong type for the "speed" argument.
	Fix this.
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>
	tree 2493491da6e446e48d5443f0a549a10ed3d35b62
	parent e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113748615 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113748615 +0100
	
	[PATCH] ARM: footbridge rtc init
	
	The footbridge ISA RTC was being initialised before we had setup the
	kernel timer.  This caused a divide by zero error when the current
	time of day is set.  Resolve this by initialising the RTC after
	the kernel timer has been initialised.
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>

---

#!/bin/sh
prev=$(cat .git/heads/origin)
to=$(cat .git/HEAD)
who=Linus
what=ARM
 
cat << EOT
${who},
 
Please incorporate the latest ${what} changes.
 
This will update the following files:
 
EOT
 
git diff $prev $to | diffstat -p1
 
cat << EOT
 
through these ChangeSets:
 
EOT
 
this=$to
while [ "$this" != "$prev" ]; do
  cat-file commit $this | sed 's,.*,\t&,'
  this=$(cat-file commit $this | grep ^parent | cut -d ' ' -f 2)
done

Dear diary, on Sun, Apr 17, 2005 at 05:24:48PM CEST, I got a letter where Russell King <rmk@arm.linux.org.uk> told me that...

I think it would look nicer if you used git log format. Actually...

It is preferred to use the commit-id command instead; when git decides to totally dig over its metametadata storage, you won't need to care.

What about adding support to git log for not caring about side branches (if there are multiple parents, go only over the first one; you might add that as a switch to rev-tree, which would turn it to rev-line ;-)?

Then, you could also easily implement support for passing additional commit ID to git log, and it would then print out the range between these two.

That would do exactly what you want, and would be probably quite usable for other uses too.

On Sun, 17 Apr 2005, Russell King wrote:

To me, merging is my highest priority. I suspect that once I have a tree from you (or anybody else) that I actually _test_ merging with, I'll be motivated as hell to make sure that my plumbing actually works.

After all, it's not just you who want to have to avoid the pain of merging: it's definitely in my own best interests to make merging as easy as possible. You're _the_ most obvious initial candidate, because your merges almost never have any conflicts at all, even on a file level (much less within a file).

Indeed. I'd almost drop the whole header except for the "author" line. 

Oh, and you need a separator between commits, right now your "Signed-off-by:" line ends up butting up with the header of the next commit ;)

That's fine. Out short-logs have always tried to have just the real name in them, and I do want an email-like thing for tracking the developer, but yes, if you remove the email, that's fine. It should be easy enough to do with a simple

	sed 's/<.*>//'

or similar.

And if you replace "author" with "From:" and do the date conversion, it might look more natural.

		Linus

On Sun, Apr 17, 2005 at 09:36:09AM -0700, Linus Torvalds wrote:

Ok, I'll throw this tree onto master.kernel.org - how about master.kernel.org:/home/rmk/linux-2.6-rmk.git ? I think it's in the same format as your trees:

linux-2.6-rmk.git
|-- HEAD
`-- objects

where HEAD was copied from my .git/heads/master, and objects from .git/objects.

Done.

Done.

Done.

Also done. 8)

I still need to work out how to make my noddy script follow different branches which may be present though. However, for my common work flow, it fits what I require.

Ok, how about this format:

Linus,

Please incorporate the latest ARM changes.

This will update the following files:

 arch/arm/kernel/process.c                  |   15 +++++++++++----
 arch/arm/kernel/traps.c                    |    8 ++------
 arch/arm/lib/changebit.S                   |   11 ++---------
 arch/arm/lib/clearbit.S                    |   13 ++-----------
 arch/arm/lib/setbit.S                      |   11 ++---------
 arch/arm/lib/testchangebit.S               |   15 ++-------------
 arch/arm/lib/testclearbit.S                |   15 ++-------------
 arch/arm/lib/testsetbit.S                  |   15 ++-------------
 arch/arm/mach-footbridge/dc21285-timer.c   |    4 ++--
 arch/arm/mach-sa1100/h3600.c               |    2 +-
 include/asm-arm/arch-ebsa285/debug-macro.S |    7 +++++--
 include/asm-arm/arch-rpc/debug-macro.S     |    5 ++++-
 include/asm-arm/ptrace.h                   |    5 +----
 include/asm-arm/system.h                   |    3 +++
 14 files changed, 41 insertions(+), 88 deletions(-)

through these ChangeSets:

Author: Russell King: Sun Apr 17 16:28:31 BST 2005

	[PATCH] ARM: fix debug macros
	
	Fix debug EBSA285 and RiscPC debugging macros to detect whether the
	MMU is enabled.
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:51:02 BST 2005

	[PATCH] ARM: bitops
	
	Convert ARM bitop assembly to a macro.  All bitops follow the same
	format, so it's silly duplicating the code when only one or two
	instructions are different.
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:50:36 BST 2005

	[PATCH] ARM: showregs
	
	Fix show_regs() to provide a backtrace.  Provide a new __show_regs()
	function which implements the common subset of show_regs() and die().
	Add prototypes to asm-arm/system.h
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:40:46 BST 2005

	[PATCH] ARM: h3600_irda_set_speed arguments
	
	h3600_irda_set_speed() had the wrong type for the "speed" argument.
	Fix this.
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:36:55 BST 2005

	[PATCH] ARM: footbridge rtc init
	
	The footbridge ISA RTC was being initialised before we had setup the
	kernel timer.  This caused a divide by zero error when the current
	time of day is set.  Resolve this by initialising the RTC after
	the kernel timer has been initialised.
	
	Signed-off-by: Russell King

On Sun, 17 Apr 2005, Russell King wrote:

The way to handle that is that you need to 

 - remember (or re-fetch) what the latest HEAD was that you merged with in 
   my tree.

   if you didn't remember, you can just get all my objects and do a

	merge-head $(cat .git/HEAD) $linus-current-head

   or something (using the current git archive that has a "merge-head" 
   program. That gives you the most recent common head.

 - use "rev-tree" to show reachability

	rev-tree $my-current-head $last-merge-head
		| sort -n		# sort by date rather than sha1
		| cut -d' ' -f2		# get the sha1 + "flags" mask
		| grep :1		# show the ones that are only
					# reachable from $my-current-head

and you now have a nice list of sha1's ordered by date.

Or something. I didn't test the above. Testing is for users.

		Linus

On Sun, 17 Apr 2005, Linus Torvalds wrote:

Btw, you don't even have to really remember the head you merged with. If you have all my objects, you can just use the current head from whatever my repository is, and the "rev-tree" will show all your commit objects that aren't in my tree.

The only reason I said "remember the last head you merged" is that that way you don't even have to download the objects from my tree, and you can make the decision totally locally without looking at what I've done since.

Of course, at some point you want to get my updated objects _anyway_, but..

			Linus

(Dropped HPA from the CC line - I think he was only copied for the master.kernel.org issues.)

On Sun, Apr 17, 2005 at 12:33:22PM -0700, Linus Torvalds wrote:

My script currently sends between two commit-ids, so...

This will (and does) do exactly what I want. I'll also read into the above a request that you want it in forward date order. 8)

On Sun, 17 Apr 2005, Russell King wrote:

No, I actually don't _think_ I care. In many ways I'm more used to "reverse date order", because that's usually how you view a changelog (with a pager, and most recent changes at the top).

Which one makes sense when asking me to merge? I don't know, and I don't think it really even matters, but maybe we can add a "for now" to whatever decision you end up coming to?

		Linus

On Sun, Apr 17, 2005 at 01:08:51PM -0700, Linus Torvalds wrote:

It's trivial to change - it's either sort -n or sort -nr !

I'll pick the reverse format so that we generate the changelog in the order which people have come to expect from our previous set of tools. Consistency with existing practises is a good thing at this point.

On Sun, 17 Apr 2005, Russell King wrote:

Yes.

Btw, it might make sense to do the "grep" before the sort, since the sort doesn't scale as nicely with lots of output. On the other hand, then the grep pattern needs to be a bit more complex (so that it doesn't pick up on things that have _parents_ with ":1"). I think the rev-tree output is obvious enough that it's pretty trivial to grep for..

		Linus

On Sun, Apr 17, 2005 at 01:26:27PM -0700, Linus Torvalds wrote:

sed works as well - in fact, we can use it to both select lines based upon the :1 _and_ cut out the bulk of the stuff we're not interested in. So:

rev-tree $to $prev | \
  sort -nr | cut -d' ' -f2 | grep :1 | cut -d: -f1

becomes:

rev-tree $to $prev | \
  sed -n 's,\([[:digit:]]\+\)[[:space:]]\+\([[:xdigit:]]\+\):1.*,\1 \2,p' | \
  sort -nr | cut -d' ' -f2

(appologies to those who don't like regexps 8))

One thing to note about that sed expression though - using \+ is a GNU extension - do we care about that? The portable way is to use \{1,\} but that'd clutter it some more.

Ok, since the last one was soo successful, and I'm up for more punishment, here's another attempt. The diffstat is rather interesting in this one, claiming no changes. It should look like this:

 arch/arm/lib/bitops.h |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+)

However, it seems that git diff can't handle new files appearing yet.

The other interesting thing to note is that patches are generated for '-p0' rather than '-p1' application, which is contary to our historical requirements. This is going to confuse people - can we make it generate -p1 patches please?

Linus - assuming I un-messed-up my tree properly (it appears to be correct and fsck-cache $(commit-id) is happy) please merge this. Thanks.

Linus,

Please incorporate the latest ARM changes, which can be found at:

	master.kernel.org:/home/rmk/linux-2.6-rmk.git

This will update the following files:

 0 files changed

through these ChangeSets:

From: Russell King: Mon Apr 18 22:50:01 BST 2005
	
	[PATCH] ARM: Add missing new file for bitops patch
	
	Signed-off-by: Russell King

Dear diary, on Tue, Apr 19, 2005 at 12:16:52AM CEST, I got a letter where Russell King <rmk@arm.linux.org.uk> told me that...

Patches welcome. :^)

Actually it shouldn't be hard. Just for through add-queue/rm-queue and diff it with /dev/null.

This should be fixed with latest Junio's show-diff changes. I might change it to be identical with gitdiff-do output later though, using tree ID and "uncommitted" as the directories instead of 'a' and 'b'. It will stay -p1 now, though.

On Mon, 18 Apr 2005, Russell King wrote:

It should definitely be able to do that. 

Do a "git log | less" to look up the trees involved, and do a "git diff <parent-tree> <child-tree>" to see the output. If you don't see your new file, then either you have an old "git diff" that doesn't like the new tools (and you need to add a "-z" flag to diff-tree), or you didn't check in the new file successfully ;)

You can also always do "tree-diff -r old-tree new-tree" which will show you the tree-level changes. That's the low-level plumbing stuff: it doesn't show you the actual file contents, just how the tree changed.

That should already be the case now after the latest diffs from Junio.

Looks ok, which seems to mean that your scripts are buggered since they didn't pick up the new file.

Merge pushed out.

			Linus

Dear diary, on Tue, Apr 19, 2005 at 12:16:52AM CEST, I got a letter where Russell King <rmk@arm.linux.org.uk> told me that...

Fixed. :-)

On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:

BTW, there appears to be "errors" in the history committed thus far. I'm not sure where this came from though. Some of them could be UTF8 vs ASCII issues, but there's a number which seem to have extra random crap in them ("^M)" and lots of blank lines).

One thing which definitely needs to be considered is - what character encoding are the comments to be stored as? Currently, it's whatever the committers character encoding is, which will be completely random. For instance, dwmw2 will definitely be using UTF, whereas I'll definitely be using ISO-8859-1 (UTF is far too much of a pain in the ass to use, unless _all_ your systems are running UTF, which mine don't yet.)

ID: 75f86bac962b7609b0f3c21d25e10647ff8ed280
[PATCH] intel8x0: AC'97 audio patch for Intel ESB2
         
        This patch adds the Intel ESB2 DID's to the intel8x0.c file for AC'97 audio
        support.
         
        Signed-off-by: <A0>Jason Gaston <Jason.d.gaston@intel.com>
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

ID: baab5c52135dfa2a02c5dcc8422fc5d048acc682
[PATCH] ppc32: fix compilation error in arch/ppc/kernel/time.c
         
        make defconfig give the following error on ppc (gcc-4):
         
        arch/ppc/kernel/time.c:92: error: static declaration of <E2><80><98>time_offset<E2><80><99>
        follows non-static declaration
        include/linux/timex.h:236: error: previous declaration of <E2><80><98>time_offset<E2><80><99>
        was here
         
        The following patch solves it (time_offset is declared in timer.c).
         
        Signed-Off-By: Benoit Boissinot <benoit.boissinot@ens-lyon.org>
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

ID: fa357627062fa6f4727638bc4f302b23f5a3acb8
[PATCH] vmscan: pageout(): remove unneeded test
         
        ^M)
         
         
         
        We only call pageout() for dirty pages, so this test is redundant.
         
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

ID: e5fc9026c6a372a67e652c6ba1648e5f8e543600
[PATCH] arm: fix SIGBUS handling
         
        ^M)
         
         
        From: Russell King <rmk+lkml@arm.linux.org.uk>
         
        ARM wasn't raising a SIGBUS with a siginfo structure.  Fix
        __do_user_fault() to allow us to use it for SIGBUS conditions, and arrange
        for the sigbus path to use this.
         
        We need to prevent the siginfo code being called if we do not have a user
        space context to call it, so consolidate the "user_mode()" tests.
         
        Thanks to Ian Campbell who spotted this oversight.
         
        Signed-off-by: Russell King <rmk@arm.linux.org.uk>
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

On Sun, 17 Apr 2005, Russell King wrote:

Ah, yes. That is actually from the original emails from Andrew. I do not know why, but I see them there. It's his script that does something strange.

(Andrew: in case you care, the first one is

	[patch 003/198] arm: fix SIGBUS handling

which has the email looking like

	...
	From: akpm@osdl.org
	Date: Tue, 12 Apr 2005 03:30:35 -0700
	Status: 
	X-Status: 
	X-Keywords:                   
	
	^M)
	
	
	From: Russell King <rmk+lkml@arm.linux.org.uk>
	
	ARM wasn't raising a SIGBUS with a siginfo structure.  Fix
	__do_user_fault() to allow us to use it for SIGBUS conditions, and arrange
	for the sigbus path to use this.
	...

To git, it's just a byte stream, and you can have binary comments if you want to. I personally would prefer to move towards UTF eventually, but I really don't think it matters a whole lot as long as 99.9% of everything we'd see there is still 7-bit ascii.

That <A0> is also there in Andrew's original email. It's "space with the high bit set", and I have no idea why.

		Linus

On Sun, 17 Apr 2005, Russell King wrote:

...

Linus Torvalds replied:

I would _heartily_ recommend moving towards UTF-8 as the internal charset for all comments. Alternatives are possible (e.g., recording the charset in the header), but they're incredibly messy. Even if you don't normally work in UTF-8, it's pretty easy to set most editors up to read & write UTF-8. Having the data stored as a constant charset eliminates a raft of error-prone code.

--- David A. Wheeler

Dear diary, on Sun, Apr 17, 2005 at 08:13:59PM CEST, I got a letter where "David A. Wheeler" <dwheeler@dwheeler.com> told me that...

I tend to agree here. My toilet stuff is what can handle various locale-based conversions at the commit-tree / cat-file tree sides etc, but UTF-8 should be certainly the way to go internally.

Not that the plumbing should actually _care_ at all; anyone who uses it should take the care, so this is more of a "social" thing.

On Sun, Apr 17, 2005 at 02:13:59PM -0400, David A. Wheeler wrote:

Except, I believe, MicroEMACS, which both Linus and myself use. As far as I know, there aren't any patches to make it UTF-8 compliant.

The alternative is, I suppose, iconv. However, iconv in _my_ glibc seems buggy (segfaults) and my efforts for building glibc 2.3.2 for ARM have failed. Effectively that means iconv is inaccessible to me.

...

I replied:

Petr said:
 >Not that the plumbing should actually _care_ at all; anyone who uses it
 >should take the care, so this is more of a "social" thing.

The _plumbing_ shouldn't care, but the stuff above needs to know how to interpret the stuff that the plumbing produces.

Russell King said:

Since plain ASCII is a subset of UTF-8, as long as MicroEMACS users only create ASCII comments, then the comments you create in MicroEMACS will still be UTF-8. No big deal.

For reading comments, if the text is almost entirely plain ASCII, you could just ignore the problem and have the occasional character scramble. If you need more, you'll need a tool that's more internationalized or a working iconv, but if that's important you'd be motivated.

Again, I'm looking for more generalized solutions, where non-English comments are more common than in Linux kernel code.

--- David A. Wheeler

On Sun, 2005-04-17 at 19:20 +0100, Russell King wrote:

OT, and probably not much help, but glibc-2.3.5 is out ...

Linus Torvalds wrote:

I would suggest something like /pub/scm/kernel/git/<people> on kernel.org. This is easy to do, and being outside the "mirrored worldwide" set shouldn't cause anyone any issues.

/pub/linux/... is mirrored worldwide and that might cause some consternation.

If this is OK with everyone I'll try to set them up (I'm at LCA at the moment, so I might have limited connectivity at any one particular moment in time.)

	-hpa

Linus Torvalds wrote:

Could you place a tarball there for people like me who are no "real" kernel hackers and don't have a kernel.org account? Or is there an "anonymous" account that I'm just to ignorant to know of?

I'm just somebody who is very interested in the new things happening here, but I guess there will be some month to go and a lot of things to learn until you see the first kernel patch that is signed off by me. :-)

By the way: Does the git repository include all 60.000 changes of just the latest version of the 17.000 file in the kernel?

Best regards Jochen Römling

On Sun, 17 Apr 2005 23:50:40 +0200 Jochen Roemling wrote:

| Linus Torvalds wrote:
| 
| >Ie we have two phases to the merge: first get the objects, with something
| >like
| >
| >	repo=kernel.org:/pub/kernel/people/torvalds/linux-2.6.git
| >	rsync --ignore-existing -acv $(repo)/ .git/
| >  
| >
| Could you place a tarball there for people like me who are no "real" 
| kernel hackers and don't have a kernel.org account? Or is there an 
| "anonymous" account that I'm just to ignorant to know of?

You don't need a kernel.org account to rsync it... this works too:

rsync -avz -e ssh --progress --ignore-existing  rsync://rsync.kernel.org/pub/linux/kernel/people/torvalds/linux-2.6.git/ .git/

| I'm just somebody who is very interested in the new things happening 
| here, but I guess there will be some month to go and a lot of things to 
| learn until you see the first kernel patch that is signed off by me. :-)
| 
| By the way: Does the git repository include all 60.000 changes of just 
| the latest version of the 17.000 file in the kernel?

It's currently just 2.6.12-rc2 (17,000 files) plus changes...

Two people have announced full history gits if you are interested
in that...  Check the git email archives for mail from Ingo Molnar:
Subject: full kernel history, in patchset format

and Thomas Gleixner <tglx@linutronix.de>:
Subject: BK -> git export done

--- ~Randy

Dear diary, on Mon, Apr 18, 2005 at 12:09:21AM CEST, I got a letter where "Randy.Dunlap" <rddunlap@osdl.org> told me that...

Or

	mkdir linux-2.6.git
	cd linux-2.6.git
	RSYNC_FLAGS="--progress --stats" git init rsync://rsync.kernel.org/pub/linux/kernel/people/torvalds/linux-2.6.git/

which also does the initial checkout for you. ;-) (Requires latest git-pasky, though.) Then you can bring the latest and greatest to your tree by a mere git pull.

On Sat, 2005-04-16 at 16:01 -0700, Linus Torvalds wrote:

Do you want the commits list running for it yet? Do you want the changesets which are already in it re-mailed without a 'TESTING' tag?

On Mon, 18 Apr 2005, David Woodhouse wrote:

I really don't know. I'm actually very happy where this thing is right now, and completing that first merge successfully was a big milestone to me personally. That said, actually _using_ this thing is not for the faint-of-heart, and while I think "git" already is showing itself to be useful, I'm very very biased.

In other words, I really wonder what an outsider that doesn't have the same kind of mental bias thinks of the current git tree. Is it useful, or is it still just a toy for Linus to test out his crazy SCM-wannabe.

Can people usefully track my current kernel git repository, or do you have to be crazy to do so? That's really the question. You be the judge. Me, I'm just giddy from a merge that was clearly done using interfaces that aren't actually really usable for anybody but me, and barely me at that ;)

		Linus

Btw, I also do want this to show up in the BK trees for people who use BitKeeper - the same way we always supported tar-ball + patch users before. So I'll have to try to come up with some sane way to do that too. Any ideas? The first series of 198 patches is obvious enough and can be just done that way direcly, but the merge..

On Sun, Apr 17, 2005 at 03:17:50PM -0700, Linus Torvalds wrote:

I guess I'll have the pleasure to find that out when I update my tree with your latest changes... which I think is a project for tomorrow.

On Sun, Apr 17, 2005 at 11:19:59PM +0100, Russell King wrote:

I pulled it tonight into a pristine tree (which of course worked.)

In doing so, I noticed that I'd messed up one of the commits - there's a missing new file. Grr. I'll put that down to being a newbie git.

On Sun, 17 Apr 2005, Russell King wrote:

Goodie.

Actually, you should put that down to horribly bad interface tools. With BK, we had these nice tools that pointed out that there were files that you might want to commit (ie "bk citool"), and made this very obvious.

Tools absolutely matter. And it will take time for us to build up that kind of helper infrastructure. So being newbie might be part of it, but it's the smaller part, I say. Rough interfaces is a big issue.

		Linus

On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:

Note the "pristine".  Now comes the real test...

Ok, I just tried pulling your tree into the tree you pulled from, and got this:

Tree change: e7905b2f22eb5d5308c9122b9c06c2d02473dd4f ee423ea56280512778a5961ee58a785a73acb7d1
...
*100644->100644 blob    46f0a3caae02b4bb8f903d7ac86456aa0c37954b->ba4afd7956173b6f89eb6b0b9ad23b392d5c0aee      arch/arm/kernel/process.c
*100644->100644 blob    4a36fa7192e11df36f5e0928b064239dabe1e305->ec0bc8f315ab5d78a4220e176e7aee76d52d1c74      arch/arm/kernel/traps.c
*100644->100644 blob    311d19ee00208faf02359f9e7c5394577a40f253->bf923a953703c6ca0c88eac3b2850cf07b838996      arch/arm/lib/changebit.S
*100644->100644 blob    c07afa31695654e6489ec59c3f837183b325e9da->41f89b3a393d5af939f04f63c5bf4991b2bf6599      arch/arm/lib/clearbit.S
...
Tracked branch, applying changes...
Merging e7905b2f22eb5d5308c9122b9c06c2d02473dd4f -> ee423ea56280512778a5961ee58a785a73acb7d1
        to df4449813c900973841d0fa5a9e9bc7186956e1e...
COPYING: needs update
CREDITS: needs update
Documentation/00-INDEX: needs update
Documentation/BK-usage/00-INDEX: needs update
...
patching file arch/arm/kernel/process.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file arch/arm/kernel/process.c.rejpatching file arch/arm/kernel/traps.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file arch/arm/kernel/traps.c.rej
patching file arch/arm/lib/changebit.S
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/changebit.S.rej
patching file arch/arm/lib/clearbit.S
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/clearbit.S.rej

so obviously git pull isn't able to indentify what's already in the local repository.

Interestingly, the files listed above as having rejects are excluded from the list of "needs update". And I don't know why git is staying that these files need updating, because they haven't changed since they were initially checked out.

This was with some random version of git-pasky-0.04. Unfortunately, this version doesn't have the sha1 ID appended, so I couldn't say definitively that it's the latest and greatest. It might be a day old.

On Mon, 2005-04-18 at 10:23 +0100, Russell King wrote:

gitmerge.sh does not yet have support for the new merge stuff as far as I know, and if it does, then its a very recent version (ie, one that have the sha1 ID appended).

Dear diary, on Mon, Apr 18, 2005 at 11:23:32AM CEST, I got a letter where Russell King <rmk@arm.linux.org.uk> told me that...

For unknown reason, patch sucks at this, even if I pass it

-N  or  --forward
  Ignore patches that seem to be reversed or already applied.  See also -R.

:-(

git merge does not use the in-GIT merging capabilities yet.

Could you do

	merge-base $(tree-id ee423ea56280512778a5961ee58a785a73acb7d1) \
		$(tree-id df4449813c900973841d0fa5a9e9bc7186956e1e)

and check if it returns e7905b2f22eb5d5308c9122b9c06c2d02473dd4f, please? I guess it won't. So you could try to wipe the -b "$orig_head" from gitmerge.sh invocation in gitpull.sh.

Because it is checking out only the files which changed (pass -a to git merge to check out everything; I will probably make git pull take a -m argument which will take merge arguments ;). And then it does update-cache --refresh which complains about missing files. :/

On Mon, 18 Apr 2005, Russell King wrote:

No, that can't work. The pesky tools are helpful, but they really don't do merges worth cr*p right now, excuse my french.

The _real_ way to pull is to do the (horribly complex) thing I described by the merge, but noticing that one of the commits you are merging is a proper subset of the other one, and just updating the head instead of actually doing a real merge (ie skipping the "read-tree -m" and "write-tree" phases).

I'm afraid that until Pasky's tools script this properly, a "pull" really ends up being something like this (which _can_ be scripted, never fear):

NOTE NOTE NOTE! This is untested! I'm writing this within the email editor, so do _not_ do this on a tree that you care about.

	#!/bin/sh
	#
	# use "$1" or something in a real script, this 
	# just hard-codes it.
	#
	merge_repo=master.kernel.org:/pub/linux/kernel/people/torvalds/linux-2.6.git

	echo "Getting object database"
	rsync -avz --ignore-existing $merge_repo/ .git/

	echo "Getting remote head"
	rsync -avz $merge_repo/HEAD .git/MERGE_HEAD

	head=$(cat .git/HEAD)
	merge_head=$(cat .git/MERGE-HEAD)
	common=$(merge-base $head $merge_head)
	if [ -z "$common" ]; then
		echo "Unable to find common commit between" $merge_head $head
		exit 1
	fi

	# Get the trees associated with those commits
	common_tree=tree=$(cat-file commit $common | sed 's/tree //;q')
	head_tree=tree=$(cat-file commit $head | sed 's/tree //;q')
	merge_tree=tree=$(cat-file commit $merge | sed 's/tree //;q')

	if [ "$common" == "$merge_head" ]; then
		echo "Already up-to-date. Yeeah!"
		exit 0
	fi
	if [ "$common" == "$head" ]; then
		echo "Updating from $head to $merge_head."
		echo "Destroying all noncommitted data!"
		echo "Kill me within 3 seconds.."
		sleep 3
		read-tree $merge_tree && checkout-cache -f -a
		echo $merge_head > .git/HEAD
		exit 0
	fi
	echo "Trying to merge $merge_head into $head"
	read-tree -m $common_tree $head_tree $merge_tree
	result_tree=$(write-tree) || exit 1
	result_commit=$(echo "Merge $merge_repo" | commit-tree $result_tree -p $head -p $merge_head)
	echo "Committed merge $result_commit"
	echo $result_commit > .git/HEAD
	read-tree $result_tree && checkout-cache -f -a

The above looks like it might work, but I also warn you: it's not only untested, but it's pretty fragile in that if something breaks, you are probably left with a mess. I _tried_ to do the right thing, but... So it obviously will need testing, tweaking and just general tender loving care.

And if the merge isn't clean, it will exit early thanks to the

	write-tree || exit 1

and now you have to resolve the merge yourself. There are tools to help you do so automatically, but that's really a separate script.

You shouldn't hit the "merge" case at all right now, you should hit the "Updating from $head to $merge_head" thing.

If Pesky wants to take the above script, test it, and see if it works, that would be good. It's definitely a much better "pull" than trying to apply the patches forward..

		Linus

On Mon, 18 Apr 2005, Linus Torvalds wrote:

Ok, one out of three isn't too bad, is it? Pesky/Pasky, so close yet so far. Sorry,

		Linus

Dear diary, on Mon, Apr 18, 2005 at 07:05:19PM CEST, I got a letter where Linus Torvalds <torvalds@osdl.org> told me that...

No problem. :-) Or you can just call me Petr if you want. ;-)

On Mon, Apr 18, 2005 at 08:23:16AM -0700, Linus Torvalds wrote:

It did the right thing for me at least!

This wants to be:

	common_tree=$(cat-file commit $common | sed 's/tree //;q')
	head_tree=$(cat-file commit $head | sed 's/tree //;q')
	merge_tree=$(cat-file commit $merge_head | sed 's/tree //;q')

Don't we want to do an update-cache --refresh here?

Maybe Petr can improve the error handling, and incorporate it (or at least some of it) into git-pasky

Exactly what happened.  Thanks.

On Mon, 18 Apr 2005, Russell King wrote:

Yup, I already fixed that in the current git version.

Yes. Thanks, added.

		Linus

Dear diary, on Mon, Apr 18, 2005 at 11:53:57PM CEST, I got a letter where Russell King <rmk@arm.linux.org.uk> told me that...

This does not need to touch git pull at all now; all the relevant logic can change in git merge (and git commit), and I'm hacking that now. It should be rather easy, I think.

I think I won't make git merge commit automatically - I think the user should get a chance to do a git diff on what is getting merged to check if everything is all right.

What is actually a little annoying is having to cd ,,merge and then back, though. I don't know, but the current pull-merge script does not bother with the temporary merge directory neither, even though Linus wanted it. Linus, do you still do? ;-)

On Tue, Apr 19, 2005 at 12:48:52AM +0200, Petr Baudis wrote:

In the case I highlighted, we don't want to end up having to require user intervention. This is a common case here, and was one which was entirely scripted with BK.

Essentially, with BK, at 7am localtime each morning, I'd:

- update my baseline linux 2.6 tree
- for each working tree which may be pulled from
  - if the baseline is a superset
    - update working tree from baseline

The net result is that my workflow consisted entirely of:

1. commit whatever into working tree
2. test
3. send linus a pull request
4. repeat next day

The tree resynchronisation happened completely and entirely in the background with no user intervention required at all.

With your suggested requirement for user intervention whenever there's a merge, it means that this just isn't possible - you could automate the pulls, but you need to ensure that you'd visited each and every unmerged tree before the next day, or you don't script it at all and do the whole thing manually.

Hey, I'm lazy, and that means that just won't get done, and my trees will end up being horrendously out of date all the time. But isn't this precisely what we have computers and scripts for?

Dear diary, on Tue, Apr 19, 2005 at 12:59:52AM CEST, I got a letter where Russell King <rmk@arm.linux.org.uk> told me that...

Well, you can script that

	cd ,,merge && echo 'Merge with Linus' | git ci && cd ..

too. ;-)

But it seems like a good idea to just proceed with commit in case of no conflicts (and possibly have a switch which will tell git merge to just merge, not commit).

And in the case of conflicts...?

On Tue, Apr 19, 2005 at 01:09:42AM +0200, Petr Baudis wrote:

If the baseline is a superset of the working tree, there will never be any conflicts. Note that as I said above, this is a condition on doing the pull in the first place.

How we determine that with git is another matter though. 8)

On Tue, 19 Apr 2005, Petr Baudis wrote:

No, now that the merge is done entirely in the index file, I don't care any more. The index file _is_ the temporary directory as far as I'm concerned.

		Linus

On Mon, Apr 18, 2005 at 10:23:32AM +0100, Russell King wrote:

Since this happened, I've been working out what state my tree is in, and I restored it back to a state where I had one dangling commit head, which was _my_ head.

I then checked whether my objects matched the objects which I uploaded to master.kernel.org, and discovered I'd removed some extra ones. With them restored, I have an additional dangling commit.

Now, I'm pretty sure that I had an up to date tree when I did the original commits, so I'm a little confused.

What I'm seeing is:

b4a9a5114b3c6da131a832a8e2cd1941161eb348
+- e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
   +- dc90c0db0dd5214aca5304fd17ccd741031e5493 <-- extra dangling head
   +- 488faba31f59c5960aabbb2a5877a0f2923937a3
      +- 5d9a545981893629c8f95e2b8b50d15d18c6ddbc
         +- d5922e9c35d21f0b6b82d1fd8b1444cfce57ca34
            +- ff219d69be01af1fd04ada305b5fe7cd4c563cc6
               +- df4449813c900973841d0fa5a9e9bc7186956e1e <-- my head

It's very much like I somehow committed against the _parent_ of the head, rather than the head itself.

However, I've lost the state that this tree was in when I did the initial commit, so who knows why this happened... I think it's something to keep an eye out for though.

On Mon, 18 Apr 2005, Russell King wrote:

For the future, if your tree gets messed up to the point where you say "screw it" and just want to go back in time, you can do this (it's equivalent to "undo" in BK speak):

	git log | less -S

	.. find which HEAD it was that you trusted..

In this case your HEAD before I merged with it was this one:

	df4449813c900973841d0fa5a9e9bc7186956e1e

So to get back to that one, you can do

	echo df4449813c900973841d0fa5a9e9bc7186956e1e > .git/HEAD

and now

	cat-file commit $(cat .git/HEAD) | head -1

gives you

	tree a43c4447b2edc9fb01a6369f10c1165de4494c88

so you can restore your checked-out state with

	read-tree a43c4447b2edc9fb01a6369f10c1165de4494c88
	checkout-cache -f -a
	update-cache --refresh

and your tree should be valid again.

Now, to remove any bogus objects, you can then run my "git-prune-script" (look at it carefully first to make sure you realize what you are doing).

NOTE NOTE NOTE! This will _revert_ everything you had done after the "trusted" point. So you may not actually want to do this. Instead:

That's very common if you just forget to update your new ".git/HEAD" when you do a commit.

Again, it's the tools that make it a bit too easy to mess up. The "commit-tree" thing is supposed to really only be used from scripts (which would do something like

	result=$(commit-tree ...) && echo $result > .git/HEAD

but when doing things by hand, if you forget to update your HEAD, your next commit will be done against the wrong head, and you get dangling commits.

The good news is that this is not that hard to fix up. The _trees_ are all correct, and the objects are all correct, so what you can do is just generate a few new (proper) commit objects, with the right parents. Then you can do the "git-prune-script" thing that will throw away the old broken commits, since they won't be reachable from your new commits (even though their _trees_ will be there and be the same).

So in this case:

	b4a9a5114b3c6da131a832a8e2cd1941161eb348
	+- e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
	   +- dc90c0db0dd5214aca5304fd17ccd741031e5493 <-- extra dangling head
	   +- 488faba31f59c5960aabbb2a5877a0f2923937a3

you can do

	cat-file commit dc90c0db0dd5214aca5304fd17ccd741031e5493

to remind you what your old tree and commit message was, and then just re-commit that tree with the same message but with the proper parent:

	commit-tree xxxx -p 488faba31f59c5960aabbb2a5877a0f2923937a3

and then you need to do the same thing for the other commits (which will now need to be re-based to have the new commit-chain as their parents).

Then, when you fixed up the final one, remember to update .git/HEAD with its commit ID, and now the prune-thing will get rid of the old dangling commits that you just created new duplicates of.

		Linus

Dear diary, on Mon, Apr 18, 2005 at 01:24:24AM CEST, I got a letter where Linus Torvalds <torvalds@osdl.org> told me that...

I just committed some simple git status, which is equivalent to svn status or cvs update (except it does no update). So it shows all the files not tracked by git with a question mark in front of them.

This will need some ignore rules, though (currently it just ignores *.o
and the tags file). Now it turns out that it is rather unfortunate that
git ignores hidden files, since this would be a perfect object for that
- I think it is useful to have the ignore list tracked by git. I think I
will just name it git-ignores to be found in the working directory for
now.

On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:

Speaking of tools, you had a "dotest" program to apply patches in email form to a bk tree. And from what I can gather, you've changed that to handle git archives, right? Any pointers to where I can find this so I could try to build up some git trees for you to merge with? I think I can even make a tree with a merge issue if you want to test that out :)

thanks,

greg k-h

On Mon, 18 Apr 2005 08:04:57 -0700 Greg KH wrote:

| On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
| > 
| > Tools absolutely matter. And it will take time for us to build up that 
| > kind of helper infrastructure. So being newbie might be part of it, but 
| > it's the smaller part, I say. Rough interfaces is a big issue.
| 
| Speaking of tools, you had a "dotest" program to apply patches in email
| form to a bk tree.  And from what I can gather, you've changed that to
| handle git archives, right?  Any pointers to where I can find this so I
| could try to build up some git trees for you to merge with?  I think I
| can even make a tree with a merge issue if you want to test that out :)

it's at http://www.kernel.org/pub/linux/kernel/people/torvalds/git-tools.git/

--- ~Randy

On Mon, 18 Apr 2005, Greg KH wrote:

Yup.

It's a git archive at 

	kernel.org:/pub/linux/kernel/people/torvalds/git-tools.git

and it seems to work. It's what I've used for all the kernel patches (except for the merge), and it's what I use for the git stuff that shows up as authored by others.

		Linus

On Mon, Apr 18, 2005 at 08:42:14AM -0700, Linus Torvalds wrote:

Hm, have you pushed all of the recent changes public?  I get the
following when trying to apply a patch:
$ ../git-tools/dotest < ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch 
 mailsplit <mbox> <directory>
$ 

I set my PATH to point to the git-tools/ directory: $ which mailsplit /home/greg/linux/git/git-tools/mailsplit

Any thoughts?

thanks,

greg k-h

On Mon, Apr 18, 2005 at 03:05:41PM -0700, Greg KH wrote:

Doh, you want it as a command line argument now.  Stupid me...

Anyway, I try it this way and get:

$ dotest ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch             

Applying USB: visor Tapwave Zodiac support patch

fatal: preparing to update file 'drivers/usb/serial/visor.c' not uptodate in cache

What did I forget to do?

thanks,

greg k-h

On Mon, 18 Apr 2005, Greg KH wrote:

You should update to the newest version anyway..

The most common reason is that the scripts _really_ want the index to match your current tree exactly. Run "update-cache --refresh". And if you have any uncommitted information, make sure to commit it first.

(Not _strictly_ true - you can leave edited files in your directory, and just hope the patch never touches them. The thing you should _not_ do is to do an "update-cache xxxx.c" to commit any changes to the 'index', because then the patch applicator will actually commit that one too).

		Linus

On Mon, Apr 18, 2005 at 04:16:45PM -0700, Linus Torvalds wrote:

Ah, that was the step I was missing, thanks, it's working now.

greg k-h

On Mon, 18 Apr 2005, Greg KH wrote:

Oops. Obviously not. Will fix.

		Linus

I have set up /pub/scm/linux/kernel/git on kernel.org. Everyone who had directories in /pub/linux/kernel/people now have directories in /pub/scm/linux/kernel/git. For non-kernel trees it would probably be better to have different trees, however; I also would like to request that git itself is moved to /pub/software/scm/git; I have created that directory and made it owned by Linus.

	-hpa

On Sun, 17 Apr 2005 15:17:50 -0700 (PDT) Linus Torvalds wrote:

| 
| 
| On Mon, 18 Apr 2005, David Woodhouse wrote:
| > 
| > Do you want the commits list running for it yet? Do you want the
| > changesets which are already in it re-mailed without a 'TESTING' tag?
| 
| I really don't know. I'm actually very happy where this thing is right
| now, and completing that first merge successfully was a big milestone to
| me personally. That said, actually _using_ this thing is not for the
| faint-of-heart, and while I think "git" already is showing itself to be
| useful, I'm very very biased.
| 
| In other words, I really wonder what an outsider that doesn't have the
| same kind of mental bias thinks of the current git tree. Is it useful, or
| is it still just a toy for Linus to test out his crazy SCM-wannabe.
| 
| Can people usefully track my current kernel git repository, or do you have
| to be crazy to do so? That's really the question. You be the judge. Me,
| I'm just giddy from a merge that was clearly done using interfaces that
| aren't actually really usable for anybody but me, and barely me at that ;)

TBD...

| 		Linus
| 
| Btw, I also do want this to show up in the BK trees for people who use 
| BitKeeper - the same way we always supported tar-ball + patch users 
| before. So I'll have to try to come up with some sane way to do that too. 
| Any ideas? The first series of 198 patches is obvious enough and can be 
| just done that way direcly, but the merge..

David did the commits-mailing-list script and I'm working on a commits web-page like what was formerly seen at: http://www.kernel.org/pub/linux/kernel/v2.6/testing/cset/ (with daily tarball)

based on some older scripts from David, however I'm wondering if a variant of the gitlog.sh script wouldn't be a better starting point for it.

--- ~Randy

On Sun, 2005-04-17 at 15:22 -0700, randy_dunlap wrote:

My commits-list script is in fact based on gitlog.sh. You'll probably find useful things to crib from in both that and the original bkexport.sh script.

The commits script also wants updating to print the date properly now that we've changed how it's stored -- I'll try to find some time this week to update it and set it running on master.kernel.org again, but it may end up waiting till after LCA.

On Mon, 18 Apr 2005 09:21:45 +1000 David Woodhouse wrote:

| On Sun, 2005-04-17 at 15:22 -0700, randy_dunlap wrote:
| > David did the commits-mailing-list script and I'm working on a
| > commits web-page like what was formerly seen at:
| > http://www.kernel.org/pub/linux/kernel/v2.6/testing/cset/
| > (with daily tarball)
| > 
| > based on some older scripts from David, however I'm wondering if
| > a variant of the gitlog.sh script wouldn't be a better starting
| > point for it.
| 
| My commits-list script is in fact based on gitlog.sh. You'll probably
| find useful things to crib from in both that and the original
| bkexport.sh script.

Yep, already had all of that in progress.

| The commits script also wants updating to print the date properly now
| that we've changed how it's stored -- I'll try to find some time this
| week to update it and set it running on master.kernel.org again, but it
| may end up waiting till after LCA.

Here's a working gitgenwebpage.sh script, although it may be possible to do it better/faster, and merge it with the commits mail script (as you mentioned).

--- ~Randy