Using public-inbox+lei+Emacs+mu+mu4e (was: Large delays in mailing list delivery?)

On Mon, Dec 06 2021, Eric Wong wrote:

Thanks. I finally got around to setting this up.

The above instructions didn't quite work for me, but here's what I did (in the form of a script lifted from a screen(1) config I've got). Indented with un-indented comments. The "stuff" is screen's way of "run this command" (or well, input these characters):

I had a ~/g/git-ml clone already, but this makes one:
    
    stuff "cd ~/g/git-ml || git clone https://public-inbox.org/git ~/g/git-ml^M"

The initial index:

    ## This will create a .git/publici-inbox in ~/g/git-ml. Takes a while
    ## the first time.
    stuff "time public-inbox-index -v \$PWD^M"

I fiddled with this for a bit because it refused to work, turns out it was missing the .git at the end, i.e. it expected a bare repo[1]:

    ## When we add the lei external it *must* have the ".git" part,
    ## because it'll try to find the "public-inbox" folder at wherever we
    ## point it.
    stuff "test -d ~/.config/lei || lei add-external ~/g/git-ml/.git^M"

A bit of a UX wart not to be able to specify no --limit, or maybe I'm missing a way:

    ## The one-off massive import of the Git ML. TODO: No way to specify
    ## an infinite limit? Not --no-limit or --limit=0.
    stuff "test -d ~/Maildir/lei-q-git-ml || time lei q --limit=999999999 -v -o ~/Maildir/lei-q-git-ml l:git.vger.kernel.org^M"

The initial indexing:

    ## After the one-off import this will take forever *the first time*
    ## (or around 20m), but subsequent invocations will be fast:
    stuff "time lei up ~/Maildir/lei-q-git-ml^M"

Runs an ad-hoc script to keep it up-to-date, which is quoted below:

    ## Run it in a loop
    stuff "public-inbox-lei-pull-index^M"

That script (which I whipped up just now. Is there a better/more
standard way? to keep a public-inbox+lei pair up-to-date with
sleep/backoff etc?
	
	#!/bin/sh
	set -xe
	
	repo=~/g/git-ml
	while true
	do
		oid=$(git -C $repo rev-parse HEAD)
		git -C $repo pull
		noid=$(git -C $repo rev-parse HEAD)
		if test "$oid" = "$noid"
		then
			echo Nothing to update
			sleep 60
			continue
		fi
		(
			cd $repo &&
			public-inbox-index -v "$PWD"
		)
		lei up ~/Maildir/lei-q-git-ml
		sleep 1
	done

I use Emacs+mu4e for my E-Mail. And since I index ~/Maildir having these files dropped in there will be added to its index. Then I just changed my saved search to also look through that maildir (I guess the entire first condition could be dropped, but whatever):

    "(maildir:/personal-gmail/* OR maildir:/lei-q-git-ml/*) AND list:git.vger.kernel.org OR recip:git@vger.kernel.org OR recip:git-packagers@googlegroups.com"

Because "mu" is generally good about de-duplicating stuff I've now got an inbox with mixed messages I can sync from GMail (including my "Sent" folder), and I get up-to-the-minute ML traffic now (it's bee 10hrs-4day delayed for 3-4 months at least).

So new messages are generally from the "lei" directory, but when I send one it'll be dropped in the personal-gmail.

I still need to check if it's doing the wrong thing with e.g. "read" flags if I read a mail synced via lei that later arrives in GMail. But I mostly don't use "read" statuses anyway...

1.  Maybe this "I only tested if it complied" patch would make sense to catch that?