Feature request - Subtree checkouts

Heya,

I'm trying to dig at various issues that are potential holdups for migrating the Gentoo CVS tree into Git.

Since shallow checkouts are now available, there's just one more thing that's missing: subtree checkouts. Not to be confused with sub-projects.

If the master tree has this as some example contents: /foo /abc/... /bar/example /bar/baz/some-content

We need to be able to check out arbitrary subtrees. So I might want to check out everything (as the CVS administrator), while one of the more focused developers just wants to check out /bar/baz/.

P.S. Does this list do some weird spam-blocking? I've tried 3 times now to use git-send-email to send an unrelated minor patch set (--subject-prefix for git-format-patch), and it has never shown up on the list :-(.

"Robin H. Johnson" <robbat2@gentoo.org> wrote:

Yes. It has cut down on our spam, but it has also caused some things to be blocked, almost without good reason. ;-)

I send a fair number of patches to this list (at times anyway) and am also unable to use git-send-email. If I have the email also CC back to me it does make it through a number of SMTP server hops, including my own spam filters, but it never makes it through the Git mailing list. So I dump the patches to an mbox with --stdout, open them up in mutt and resend them there. I've bound shift-G to resend-message for that purpose...

"Shawn O. Pearce" <spearce@spearce.org> writes:

Well, we need to do something about this. I haven't seen Robin's patch neither on the list nor in my mailbox (if they were CC'ed to me).

One thing that people need to be careful about is which SMTP server they use. I had an impression (I do not use send-email myself) that it defaulted to local MTA, so the mail trail would look like your local MTA receives from the MUA (which is send-email), which forwards it to whereever destination (or intermediaries). On the other hand, I suspect many people use their ISP's SMTP server when using their usual MUA, so the mail trail would look different. I do not know what filtering vger does, but if it is filtering based on the MTA address (dul.dnsbl.sorbs filtering comes to mind), that would make a difference.

On Tue, 10 Apr 2007, Junio C Hamano wrote:

ABSOLUTELY!

There are a ton of spam blockers that simply *refuse* to accept email from people who just randomly send to port 25.

For example, I will personally never see email that comes directly to my email address though an open mail relay *or* from something that appears to be just a random botnet PC (I forget the exact rule, since I'm hapily ignorant of MIS, but I think it boils down to requiring a good reverse DNS lookup).

That's getting much more common. Most spam is done through botnets, and they still try to do the direct-to-port-25 thing, exactly because if you go through a *real* SMTP host, your ISP will generally shut you down pretty quickly if you're spamming.

So special spammers with their own machines will work with ISP's that don't care (they make money off the spammers), but botnets depend on cracked Windows machines, and those often have ISP's that *do* care, because they get complaints and it costs them money if they don't.

A lot of people configure their MUA to send specially, and never even configure their MTA at all apart from whatever default configuration it has.

For example, I may have sendmail installed on my machine, but its only purpose in life is to do *local* email delivery (using fetchmail -> sendmail). It wouldn't know how to send an email outside of the machine.

Instead, my MUA is configured to do this:

	# List of SMTP servers for sending mail. If blank: Unix Pine uses sendmail.
	smtp-server=localhost:10025

where localhost port 10025 is just a ssh tunnel to inside the osdl network. If I tried to send email any other way, people would often not accept it, because various SMTP servers will simply *refuse* to forward emails that claim to be coming from a point that isn't somethign that they actually *receive* email for.

And I don't think my setup is at all unusual. I may be unusual in my choice of MUA, but most MUA's will have configuration for where to send the mail, and I think very few people actually configure sendmail to do it right.

So no, I don't think people should assume that "sendmail" magically knows how to send emails other than to the local machine (ie use it to send problem reports to "root", or let fetchmail deliver to it for local emails, but that's about it)

			Linus

On Tue, 10 Apr 2007, Linus Torvalds wrote:

Depending on your definition of "good".

I run my mail server off my DSL line. I prefer having control over my mail server instead of being chained to what my ISP provides. (The problems of having been a sysadmin for way too many years.) I don't have control over the reverse ip address, but I do over my DNS resolution. (Well, most of it. A couple domains are sitting on really old dns servers from years past.)

Which makes Greylisting a useful tool. However, some people define a "real SMTP host" as being the one your ISP provides and no other. No matter how good your OS or how stringent your rulesets for sending mail are.

And the various RBL lists will just nuke everyone in the same class c. (I have seen it happen for people at colos. Someone on the class c get compromised, sends spam and the whole address space gets blocked. Sometimes I think the anti-spam methods are as obnoxious as the spammers. Almost.)

But the people who use git are probably the exceptions. (At least I hope so...)

A number of programs assume that the local MTA can send mail. (The bug handling software for Gnome is an example of that.)

This sounds more like a distro problem though. The MTA configuration should be a proper part of an install, not something that you have to uncover and piece together later.

Of course I also believe that Sendmail configuration was designed by an organic chemistry major as part of an "experiment" in ergot derivitives.

On Tue, 10 Apr 2007, alan wrote:

Well, the most common case (and the thing I *think* our spam software does here) is to just confirm that the reverse DNS lookup (that you want to do *anyway* for the "Received" headers for the email) will resolve back to the same IP (aka "FCrDNS").

It's also possible to just not accept mail if the reverse lookup indicates that the sending IP address is a dynamic address, which you can sometimes see from the hostname. I would suggest you *not* name your hosts to contain a lot of numbers and the string "dhcp", for example ;)

I'll take strict anti-spam methods any day. I get about 10 pieces of spam a day, that I can handle easily without worrying about it. I shudder to even just think about what it used to be like before aggressive spam filtering.

So I'm personally *solidly* in the camp that says "if you want to send me email, it's worth making a conscious effort to not look like spam".

		Linus

On Tue, 10 Apr 2007, Linus Torvalds wrote:

I actually have a fixed IP address.

Greylisting dropped my spam level by at least 90%. RBLs have, for the most part, had far too many false positives to be useful. (If it was just me, it would not be so bad, but my wife gets mail on this server as well. She is not so forgiving.)

I try pretty hard. However, some anti-spam methods share some of the same methods with dowsing and other witchcraft. What "looks like spam" seems pretty subjective at times.

I do have a reason for being a bit negative about it. I once ran a very large development list back in the last century. A piece of spam got past my spam filters and onto the list. Some idiot got offended and reported me as a spammer (because my ip was the first thing he saw) and reported me to my isp and to my isp's upstream feed. Took me a couple days to get everyone calmed down. Excessive anti-spammers still get on my nerves after that incident.

Linus Torvalds <torvalds@linux-foundation.org> writes:

That would be a very bad idea I think. Doing that would lose quite a lot of small companies and individuals such as me that run a mail server but are unable to get the ISP to change the reverse DNS. For example I do have a fixed IP, but have an reverse DNS pointer which looks like 1-2-3-4-5a.foo.bar.bostream.se.

Forcing everybody to send mail through their ISP (and I'm not even sure if my ADSL subscription includes such a service) would be a big loss. First of all its a philosophical thing, I think that it's very important that small shops or individuals should be able to control the services they need, the internet is supposed to be peer to peer. Second because the ISP's mess up a lot more often than I do, for example Telia, one of the largest ISPs in Sweden have been having massive mail server problems during the last week which I'm happily unaffected by.

    /Christer

On Tue, 11 Apr 2007, Christer Weinigel wrote:

I am in the same situation. I also have three domains. Which one do I pick? I can't afford to get an individual ip address for each. Virtual servers on a single ip also will have similar problems.

My ISP only reciently started to think about using greylisting. They route all their mail through a filter service that I do not trust. (Too many false positives. I also expect that all this talk about "forking children" would get me on some list somewhere.)

I have more experience running mail servers than the people at my ISP. (Not their fault. They are young.)

On Tue, 10 Apr 2007, alan wrote:

this isn't a problem. as long as you can lookup 1-2-3-4-5a.foo.bar.bostream.se. and get your IP address you pass this test.

David Lang

On Tue, Apr 10, 2007 at 02:33:54PM -0700, alan wrote:

greylisting unfortunately requires "some maintenance" to keep it going well, and it also breaks some mail appliances and probably some MTA's that are completely compliant in retrying to send mails.

another tactic which is probably just as good if not better than greylisting is "nolisting", that is to have your primary mx point to a non-existant machine with a real ip-address and dns entries (or even a machine with a firewall that runs iptables that blackholes or does funky stuff to anything coming in on port 25 and then just drops the connections).

if the remote end is a compliant MTA it will failover to the secondary mx which is a real machine that receives mail. but it probably suffers from the same problem of mail appliances not being completely compliant to the specs on how MTA's should work.

nolisting offers almost as good effects as greylisting without the hassle of maintaining a list.

Jimmy.

On Tue, Apr 10, 2007 at 01:28:17PM -0700, Junio C Hamano wrote:

They definitely were CC'd to you Junio.

No, it's not SMTP path differences.

I _know_ that my mail path is identical for git-send-email as well as my MUA, because I had problems with GIT and whitespace in email addresses initially ;-).

Since I have access to all the mail servers in my path (home -> AUTH SMTP @ work -> wherever), I dug at the logs, and found that vger did indeed accept my email to the list, but the messages never turned up on the list.

The only weirdness I saw in that, is that the envelope sender did not appear to be set correct from git-send-email. Give me a moment to explore down that path.

On Tue, Apr 10, 2007 at 02:10:41PM -0700, Robin H. Johnson wrote:

Ok, see my patch series for envelope sender, that is the source of the problem.

Junio C Hamano <junkio@cox.net> wrote:

I'm pretty sure the last time I tried git-send-email I had the MTA path exactly the same. My mutt sends to `localhost`, which forwards over an SSL channel to my colo'd spearce.org mail server, and that relays to the final destination. Hence spearce.org mail always originates from spearce.org.

Now I ran git-send-email on a different system, but had it connect over SMTP to port 25 of the same system mutt runs on, so the initial Received line was different, but othewrise the mail path was the same.

I also have a few other addresses that I can send to that will go out the spearce.org colo'd box to another SMTP system, then bounce back, and the round-trip was the same for anything from mutt and from git-send-email.

I marked the whole mess up to some mail header difference that git-send-email wasn't sending the same way mutt was, and that vger cared about. I didn't see the difference easily. I just gave up. ;-)

Hi,

On Tue, 10 Apr 2007, Robin H. Johnson wrote:

It should not be _that_ difficult. If I'm correct, it should touch exactly the same code as the commit v1.3.0~263^2~6 ("Assume unchanged" git).

ATM I have no time to do anything about it, so feel free to give it a try.

Ciao, Dscho