So a couple of weeks or months ago, I noticed something odd with the mailing lists I run. People on Yahoo and AOL claimed that they were missing messages, and Gmail was stuffing mailing list messages from people on Yahoo or AOL into the Spam folder, even though I’d received literally hundreds of messages from those people on those mailing lists in the past.
After investigating, it turns out that both Yahoo and AOL had turned on an anti-spam feature called “DMARC”. Basically what it meant if a message came with a From line saying it was from either of those, but not coming from an approved mail sender, they were asking the rest of the net to treat it as spam. Gmail honored the DMARC request by putting it in the Spam folder, but Yahoo and AOL and some other ISPs were just bouncing the messages or throwing them away.
This DMARC was obviously a huge problem for mailing lists, because what they do is they accept an email from a person, and then send out the message to all the members of the mailing list, and most of them use the person’s email address in the From line of the mailing list message. This breaks under DMARC, because if my mailing list server recieved an email from joe.blow@yahoo and sends out a message to the mailing list members with a From: joe.blow@yahoo, then all those mail servers that implement DMARC are going to see that I’m not designated by yahoo as a valid sender of yahoo email, and they’re going to drop it.
The developers of the Mailman mailing list software were quick to offer some solutions. First they issued 2.1.16, which had a quick and dirty work-around, and then they rolled out 2.1.18, which had what I think is a much better solution. But my problem is that my mailing list server is pure Debian Stable, and I want to only install packages, not get into the hassle of installing things from source and then having to monitor if things are updated. So I waited for 2.1.18 to get backported to Debian Stable (which uses 2.1.15). I put in request tickets to get it backported. They never did. Instead, they made it a package in Debian Testing, which is less stable.
So I did some googling and discovered something called “apt pinning” that would allow me to install some Debian Testing packages on my Debian Stable system. I tried it, and it wanted to drag in a new version of python, which wanted to drag in a new version of libc, and so on. That’s just stupid – the minimum required python for 2.1.18 is exactly the same as the minimum required python for 2.1.15. Whoever set up the .deb was a little over zealous in the requirements section.
I did not particularly want to drag in unstable versions of the very core libraries of a Linux system for no reason, so my next possibility was to install it from source. That was more complicated than it should have been, but relatively painless. First I tried following the instructions that Bill Bradford pointed me at. Unfortunately, immediately it told me that “Distutils is not available or is incomplete for /usr/bin/python” and “be sure to install the -devel package”. Well, unfortunately there isn’t a “python-devel” package. I looked at the script that configure was using to determine what it was looking for, and the problem was a missing Python.h in /usr/include/python2.7/. A bit of searching, and I discovered that this was installed by a package called “python2.7-dev” – so close, but so far from the “python-devel” I had been searching for. After that, I discovered I had to install the “make” program (like I said, this was a pure server system and I hadn’t been building software on it before) and I did my “make install”. Mail seemed to flow, but I couldn’t access the web interface. Bill suggested running the “check_perms -f”, which found and fixed 26 permissions problems, but still things weren’t working. I compared the perms on a few directories between this installation and my last backup, and discovered that neither the installation program nor check_perms had noticed that the cgi-scripts in the /usr/lib/cgi-bin/mailman directory were setgid “root” instead of setgid “list”. I fixed that, and everything started to work.
Now I wanted to test whether the new “dmarc_moderation_action” setting that 2.1.18 provided would actually fix the problem. So I changed the setting on one of my mailing lists, and emailed a guy on yahoo who was on the mailing list to see if he could test it for me. Unfortunately he wasn’t around, so the next morning I bit the bullet and created a yahoo mail account and added it to that list. I tried a post by this user to the list, and it did the right thing (changed the From address to the list address, but used the Yahoo’s person name part in the person name part), and testing that gmail didn’t stuff it in the Spam folder. I made sure it doesn’t do that with non-DMARC addresses like gmail. And then I made that setting change to all my lists.
Finger crossed, and hope that there aren’t too many more updates I have to apply before a 2.1.18 or later Mailman shows up in Debian Stable.