Hotmail & Yahoo Mail Using Secret Domain Blacklist 345
On December 7th I sent out a normal batch of emails to the Circumventor mailing list, where I send out new proxy sites for getting around Internet filters. I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)
The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing, but major email providers keep finding new ways to block the emails as "spam," which sometimes provide interesting insights into how the filters work behind the scenes.
After the last mailing, for example, all of my newly registered domains got disabled by the registrar because two of the domains had been incorrectly blacklisted by the Spamhaus Domain Block List. It took two days to discover the problem and then several hours to trace the problem to Spamhaus, although once I found Spamhaus's automated form I was able to get the domains un-blacklisted immediately. So the registrar re-enabled the domains a few hours later, although the traffic to the domains never returned to its previous levels. Spamhaus, meanwhile, continues to claim the DBL is a "zero false-positive" list, and has yet to acknowledge the error or contact me to help get to the bottom of how it happened. Well, they know how to reach me.
At least this time around, my domains didn't get disabled. Instead, the messages rolled out for a few hours with no problem (replies from users indicated that at least some hotmail.com and yahoo.com users were receiving them), until bounces abruptly started coming in from hotmail.com and yahoo.com addresses saying:
----- Transcript of session follows -----
... while talking to mta5.am0.yahoodns.net.:
>>> DATA
<<< 550 Message Contains SPAM Content
554 5.0.0 Service unavailable
After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo), when the dust finally settled, I tried reproducing the error by sending test messages from my server's IP address to a test Hotmail account. It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body. (This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself.)
But interestingly, Yahoo Mail started bouncing my messages at about the same time — out of the seven domain names, the same four domain names were being bounced by Yahoo Mail as by Hotmail, also with the error "550 Message Contains SPAM Content." That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names that cause a message to be blocked as spam. (As it happens, the other three domains were also being bounced by Yahoo Mail with the error "Message Contains SUSPECT Content" — as opposed to "SPAM Content" — while those three domains were not blocked by Hotmail at all. That of course is aggravating, but the real clue lies in the fact that both Yahoo Mail and Hotmail were giving "SPAM Content" errors to the exact same subset of domains.)
I don't want to publish the list of all seven domain names here, so as not to make it too easy for censorware companies to block them all, but one of the four blacklisted domains was 'golflanding.com.' (All of the new domains I register are nonsensical two-word combinations, since those are the only .com domains that are likely to be (1) still available and (2) easy to remember.) As soon as it seemed like Hotmail and Yahoo Mail were working off of a common blacklist, I checked to see if Spamhaus had screwed up again and listed our domains, but none of the seven domains were on Spamhaus's lists.
I looked up golflanding.com on the blacklistalert.org service, which checks against all major spam blacklists, but no hits were listed there either (except for on some defunct services which haven't been updated in years).
So if Hotmail and Yahoo Mail are both using the domain blacklist, perhaps it's a list compiled by one company and then licensed to the other, or perhaps it's a third-party list not widely known to the public. (Hotmail uses their own SmartScreen filter, but I've found nothing online about Yahoo using it as well.) It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold.
Neither the Hotmail postmaster site nor the Yahoo postmaster site mention anything about a list of domain names that could cause a message to be blocked for mentioning the domains in the message body. Yahoo Mail does provide a support form for newsletter publishers to send inquiries about why their mail is being blocked; I submitted that on Saturday and started a thread with email "support," although so far their response has just been to copy and paste articles from the Postmaster site, with tips like "Send email only to those that want it." Each time, I reply saying, No, this is not the problem, the problem is that the domains in the messages are getting incorrectly blacklisted, and each time, support cheerfully sends me another article. If I'm not literally talking to a bot, I might as well be.
I opened a similar ticket with Hotmail, and they sent me a form letter saying that the emails were being blocked because of SmartScreen, and that as a matter of policy, they would refuse to fix any errors being made by the SmartScreen filter. Waiting to see if I get a reply from a human next.
So why should you care? Well, for one thing, if you care about users in China and Iran being able to receive proxies to get around their Internet blockers, right now Hotmail and Yahoo are thwarting these proxies more effectively than those countries' own censors are. Yes, these are real people who really do write back to me after a mailing goes out, telling me about how they were able to use the proxies to receive banned political information, and sometimes how long the proxy lasted before the censors blocked it. This week, they had to do without.
But more importantly, this is an example of a general problem: That there are certain types of issues, like blocking of legitimate mail by spam filters, where the "free market" does not deliver the best experience to consumers, and the costs get passed on to everybody. Sometimes the problems could be solved with some effort, but the effort does not get made, because people believe that the free market will solve the problem, or that it already has.
In theory, if consumers have enough information about different companies and their services, the companies can compete to provide the best product to users. The problem is that if one type of information is systematically hidden from users — in this case, the fact that their mail provider is blocking mails from reaching them — then the "theory" falls apart. Since spam getting into your inbox is a visible problem, but missed email messages are an invisible problem, Hotmail's incentive is not to give the user the best experience, but rather to err on the side of blocking legitimate messages — even if the user might prefer to get slightly more spam, than to miss one important email that they were waiting for.
This means we're not just talking about a few messages getting caught in filters, which could happen even in an efficient marketplace. We're talking about a permanent equilibrium where the user gets a sub-par experience by default — a trade-off that causes them to miss more messages than they want to — and senders have to pay the cost of overcoming the marketplace inefficiencies. (Which means if the sender is a business you buy from or a charity you support, the costs get passed on to you.)
Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes. If a company or organization uses an email list hosting company like AWeber or Constant Contact to email their users, they pay a fee of about $1 per month for every 100 users on their list (which would run me about $4,000 per month). That fee doesn't go towards bandwidth — even a 1-million-subscriber list, emailed once a month, would use less than 3 GB per month of bandwidth, which is what GeoCities was was giving away for free 10 years ago. What you're paying for is the fact that AWeber and Constant Contact have friends in the right places at Hotmail, Yahoo, and Gmail, so if your mails are getting blocked, they know the people to call to fix the problem. If you run your own list instead of paying a hosting fee to AWeber or Constant Contact, you'll end up paying other costs indirectly, through loss of income when your messages don't reach recipients, or in time and money spent trying to fix the issue. (I have to take this option anyway, since I send different URLs to different random subsets of my list, which is not supported by AWeber or Constant Contact.)
On the other hand, if the market actually "worked" — if email providers did reliably deliver non-spam messages to their users — a company or charity could run their own list for virtually zero cost, and would be able to keep all of that money. (I incur no up-front fees for running my own list; all of the costs are the time spent trying to get Yahoo, Gmail, and Hotmail to stop blocking it.) So every time you donate to a charity or buy from an online retailer, a little bit of that money goes towards the cost of that organization having to fight past marketplace failures in order to get their email to you.
I don't think there's an easy algorithmic solution, like crowdsourcing Facebook complaints or using random-sample voting on Digg. Generally, I just think we need more awareness of the fact that, under certain conditions (including those surrounding email deliverability), the "free market" is virtually guaranteed to arrive at a non-optimal solution. One manifestation of that awareness would be if Hotmail, Yahoo Mail, and Gmail created public points of contact where legitimate email publishers could find out why their emails were blocked, and had real humans responding to the messages and fixing the problems. By default, the imperfect information in the marketplace leads toward an equilibrium that errs on the side of blocking too much legitimate email, so anything that pushes the equilibrium back towards more legitimate messages getting delivered will improve the experience for users and lower costs for senders.
Besides, there's a more basic ethical issue here. If you're Hotmail and you tell your users that you're providing them with "email accounts," then those users expect those accounts to work — including having the ability to receive mails from mailing lists that they've signed up for. Helping legitimate emails get through to users is not just a matter of addressing a marketplace inefficiency, it's a matter of honesty.
Larry Lessig's book "Code is Law" describes how default choices built into the architecture of the Internet and other environments — the "code" — can steer our behavior in ways that we might not choose otherwise. I'm making essentially the same point in saying that some problems are not fixed by market forces, because people are not aware of the problem at all. I think the evidence and the reasoning are straightforward in this case, but it's hard to convince people who have adopted it as an axiom that whatever the free market arrives at, must be the solution. My favorite single sentence in Lessig's book was, "Put your Ayn Rand away." I could imagine the years of pushing against dogmatic fanaticism that led him to write that sentence, and I knew how he felt.
Summary (Score:5, Insightful)
Re: (Score:3)
Is there a summary of the summary available?
We call them "titles"
Here's one example: Hotmail & Yahoo Mail Using Secret Domain Blacklist
Except the summary is probably wrong. (Score:3)
I wouldn't be suprised if it's just Bayes. The majority of messages with links leading to those registrars' domains were categorized by human readers as spam, so automated bayesian analysis picked it up.
As long as you have Internet governance that is primarily concerned with eliminating certain forms of political speech (Great FireWall of [insert name of nation here]) rather than ensuring a free market and fair trade, you're going to have this problem. The same low-rent registrars are going to be used fo
Re: (Score:3)
Simple summary (Score:5, Informative)
He's saying that Hotmail, Yahoo, and GMail are running a cartel of free online webmail services.
He's trying to get opt-in email to accounts on these systems, and it's not going through. He has evidence indicating these services operate a common hidden blacklist service keeping those emails from getting to the accounts. He cannot reach people within these organizations to open up emails coming from his domains, as he does not have an inside contact to "assist" him with this problem. This leads him to speculate that Hotmail, Yahoo, and GMail are operating like a cartel, where only "approved" email list hosting service companies with inside contacts are able to do business with these services.
Better?
Re:Simple summary (Score:5, Interesting)
Re:Simple summary (Score:5, Insightful)
> I gave up using my own server to send email a couple of years ago for precisely these reasons
In fact, that's probably what the cartel wants, ultimately.
Re: (Score:3, Insightful)
Or, you could just keep using your server as before. People who use providers which block your server could wise up and use something else, rather than let Google harvest all their email for marketing purposes while sometimes letting them see an email they want to see.
When you switch to Google, you become part of the problem.
Re: (Score:3)
This is weird... I don't think Google was mentioned in the summary at all.
But regardless, they're not operating with a list of approved senders. I build my own systems and send mail through them all the time. Sometimes just regular mail service, some for mass emailing (legally and legitimately). You'll have to take my word that I don't have super-secret inside contacts at Google, Yahoo and Microsoft to make sure this works.
Now if you meant to say they have anti-spam filters that occasionally throw false-pos
Re: (Score:2)
Re: (Score:3)
It's ok to blacklist email received from open proxies. It's not ok to block legitimate email for just *mentioning* them.
There are evil forces out there (Score:3)
Re: (Score:3)
No, but I can summarize the summary of the summary: People are a problem.
Re: (Score:3)
Just FYI, I seen this guy bitching about it MONTHS ago. Apparently he still hasn't made a lot of headway. However, if you operate like a spammer (sending the same email to multitudes of folks, while relaying information about open proxy servers as information), then you will be treated like a spammer
Re:Summary (Score:4, Insightful)
Why? By definition he is NOT a spammer since his messages are neither unsolicited nor commercial. It should be fairly easy for the responsible parties to verify he following best practices and whitelist him but apparently that's too much work for the postmasters at the big 3 webmail providers. Basically the postmasters at yahoo, gmail, and hotmail aren't doing their jobs. I know if our email admin was so bad at rectifying false positives he wouldn't be here for long but because of the scale of these organizations that pressure isn't happening.
Re: (Score:2)
Re:Summary (Score:5, Informative)
Why? Listservs are older than SMTP and have always been one of the use cases for electronic communications. Plus it's not like those providers are blocking all listservs, just those that don't pay their friends stupid high monthly fees for the privileged of emailing their users.
Re:Summary (Score:5, Informative)
Let's use your physical mail analogy, under your idea charitable organizations would not be allowed to mail people who have signed up as supporters unless they went through a commercial mass mailing company paying a huge fee per piece mailed. While that's kind of the status quo for poorly run charities with a high overhead cost none of the charities I choose to support are so stupid, why you would want to reduce the amount of money reaching deserving causes and feed the commercial mass mailers I have no clue.
Comment removed (Score:4, Insightful)
Re: (Score:3)
yeah: guy discovers cloudmark domain blacklist is used by two cloudmark customers. At least, that's my opinion. this information isn't new, this list has been around for years, and you don't get on it easily. It takes multiple reports from multiple accounts before they add you.
Re: (Score:2)
Is there a summary of the summary available?
It's a pain in the butt to get yourself removed from yahoo's email blacklist.
The end.
Re:Summary (Score:5, Insightful)
According to TFA his list is opt-in only, so unless he's lying about that he doesn't appear to be a spammer.
I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.
I can assure you I have never sent a single spam email in my life.
This is the whole point of TFA though, there's no incentive for companies running mail services to ensure that legitimate mail gets delivered. It's simply cheaper to not bother with false positives at all because the cost of non-delivery is placed squarely on the shoulders of the sender.
This is why Spamhaus could easily force me to switch ISPs, it doesn't cost them anything to put my IP range on a shitlist, but it cost me money and effort to migrate my service.
Re:Summary (Score:5, Interesting)
I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.
I've had problems like that with them as well. The thing is, Google et al. do provide very good spam filters. Out of the thousand or so spam messages that hit my mailbox every month, only about 5 make it through. A 99.95% success rate is nothing to sneeze at, so credit where credit is due. But the problem here is still architectural -- very few people respond to spam so the odds are very high that responses are to legitimate e-mail. Higher, I would think, than the 99.95% rate above. Multiple responses to the same address should override any spam-rating system they have automatically, and if not, there should at least be a 'white list' option for users to bypass the filter in the event of a failure such as this.
Neither option exists, and there is no remediation pathway available. The author (correctly) concludes this is deliberate and not merely a process oversight. Such is the nature of operations where the profit margins are so tiny that any support would obliterate it. Google only provides gmail so it can mine keywords and phrases from your e-mails to build a marketing profile and then target advertisements at you. Despite the very low rate of success here, it still beats the cost of the hardware maintenance and bandwidth when aggregated over a few hundred million regular users. But the only support incentive here is customer retention, and the support provided is very minimal and highly automated (as the author has discovered). This guy isn't a google customer -- he's trying to contact google customers, which places him in the "liability" column, not the "asset" column. Unless this guy can show that hundreds of thousands of Google customers are impacted and the impact is severe enough for them to switch, or consider switching, to another provider, there is no incentive for Google to even read his complaint, no matter how justified or rational, or easy to fix.
That's the free market problem he's run into: He thinks he's a customer, but he isn't. He's a service. And one that costs google more to support than any potential revenue that may be generated. The business decision here is clear, if not very friendly.
Re: (Score:2)
I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.
"Wasn't well liked" == "complaints had been received that they allowed their customers to send spam."
I agree with spamhaus. This puts pressure on ISPs to police their customers, or else their decent customers will leave. And everyone can choose whether they want to use providers that allow all contact through, or providers that filter out contact from ISPs that don't police their customers.
there's no incentive for companies running mail services to ensure that legitimate mail gets delivered
Well, there's some incentive in that if their customers truly want the mail and aren't receiving it, they'll have to
Re: (Score:3)
I'm not on his list so I don't know how it actually operates, but there's a lot of variation in these things. I've opted in to lists that don't provide a real opt-out, or in other ways don't comply with the old can-spam guidelines. I usually see this from foreign companies, which will require me to log in to an account on their website to get off the mailing list. Those get the spam button, every time, no questions asked.
Spamhaus does BL domains for no apparent reason, though. We're talking about properly c
Re:Summary (Score:5, Interesting)
1. Email blacklists are a terible idea, and I really sympathise with this guy's plight. I've been at the nasty side of a Spamhaus issue with my own mail server and I can tell you, those guys are nothing but a bunch of digital thugs who have managed to get themselves a nice big stick that they use to hit people randomly with. My server, being private, had just about every conceivable spam prevention mechanism turned on. SSL only connections, authorised SMTP-submission sending only, properly set up SPF records, PTR records correctly registered against the IP to allow reverse lookup. It got registered with Spamhaus and it took me a LONG time to get them to play ball. I'm still listed with a few older BL's but oh well.
2. If someone in a country wishes to circumvent government censors, why on Earth would they use a proxy? Why would they not just use Tor, which can't be blocked or filtered in that manner? If the government is doing deep packet inspection and will infer illegality from mere encrypted traffic, surely transferring illegal content in the clear is worse? Furthermore, setting up Tor is not materially more difficult than setting up a proxy. Not trolling, genuinely interested to know why one would choose the proxy path over Tor.
WTF upvotes for baseless aspersions (Score:5, Insightful)
This man is running a list (among many other activities) supporting individuals' rights to information freedom under repressive governments and you're implying he's either incompetent or, worse, underhanded?
This is inane.
And how much effort is required to fucking test this?
What causes rudy_wayne and those who upvoted his post to like the idea that Bennett Haselton is spamming and lying about it? And is their credulity what keeps them from performing such an easy test? Whatever the cause of the inanity, how can we discourage this problem in the future?
Re: Summary (Score:5, Informative)
As a long-time subscriber to his list (at least 6 years), no, he's absolutely not. He provides a fantastic service and does a damn good job of ensuring only those who want the messages are receiving them. And I get less than one message per month from that list. If he's a spammer, so is literally every single person or organization that has ever sent me an email.
Re:Summary (Score:5, Informative)
That's "hear hear", as in "hear him, hear him!" (which is where that phrase is rooted.)
yeah, spam blacklists are a poor solution (Score:2, Insightful)
I could maybe see their necessity 10 or 15 years ago, but statistical classification techniques are good enough these days that a blunt tool like a domain blacklist doesn't really make much sense. Heck, Paul Graham was arguing that seven years ago [paulgraham.com], and it hasn't gotten less true.
Server load (Score:2)
Now, I agree that blacklists are bad, but we do need some system that doesn't require large amounts of CPU time or other resources. Hashcash is interesting here, in that the CPU time is mostly spent by clients; one might be able to slow spam down enough to let a combination o
Re: (Score:2)
Blacklists are nice because they reduce server loads. Sure, running a statistical classifier for one user is not so hard, but if you have to process hundreds of millions of messages per day, that is a lot of CPU time spent on spam.
The CPU time spent on running something like SpamAssassin is insignificant compared to the bandwidth, disk writes, etc., caused by spam. Keeping the incoming e-mail in a RAM disk until you have truly accepted it for delivery (which isn't dangerous even if the server crashes hard) is the #1 thing that speeds up e-mail intake. At that point, scanning takes almost no time.
As you mention, though, greylisting does the best job of keeping your overall load down, since you don't even need to use network bandwidt
Re: (Score:2)
but we do need some system that doesn't require large amounts of CPU time or other resources.
Why? CPU time is dirt cheap if you can concentrate your task. The bandwidth (a much scarcer resource) is already being spent, and better decisions will just tend to reduce your costs there. To me this smacks of laziness, not efficiency.
Re: (Score:2)
The spammers have found various ways around these. Often they throw a bunch of the "high target" key words (e.g. viagra, cialis, penis enlargement) in as images, or they'll use computer generated text that looks somewhat real enough to even fool some human readers in order to throw off those filters. This works because the more words you have, the less likely the small terms will be snagged.
Re: (Score:2)
I wonder how many job opportunities I've missed or friends I've drifted apart from because of email dropped by statistical classification techniques. That's why everybody uses Facebook to keep in touch now.
Friends? An AC on Slashdot?
Jobs? An AC on Slashdot?
Not to worry.
Re: (Score:2, Insightful)
Re: (Score:2, Insightful)
Re: (Score:3)
Also, what about the fact that, at that level of users (100k+ish) Facebook *won't* post your update to each of your facebook friends? They just silently drop messages.
I don't know - just a thought.
Spam is like cancer (Score:2, Insightful)
The only treatment is a deadly poison that you hope kills off the bad parts before the good suffers too much.
Distribute the load (Score:3)
Re: (Score:2)
There are several distributed reputation filter systems but they are all commercial AFAIK.
Re: (Score:2)
Hashcash distributes the load somewhat, in that it forces spammers to use more resources to send out their message and can slow them down somewhat.
Unfortunately, until you get to a significant number of bits, hashcash doesn't take all that long to compute, and you can pre-compute them.
I use 23-bit hashcash on all my outgoing e-mails, but if the address has been sent to before, there is likely a pre-computed 25-bit hashcash waiting. I use idle server time to pre-compute for any address that has been sent to from my servers. Since the hashcash expires in 25 days, I don't have to do this very often unless the recipient is a frequent one. Then, to keep
"Free market" scare quotes (Score:2, Insightful)
What's with the gratuitous complaints about the "free market" not giving some mythical "optimal solution" that lets you send your "100% guaranteed opt-in" spam without interference? I call bullshit. If Hotmail isn't accepting your "really honest it's not spam" mailing list stuff, maybe you should try contacting them about it. The "free market" doesn't magically solve problems without people doing what it takes to address the problems.
Re: (Score:3)
The problem with most be free email providers IS contacting them. You're not paying them, so they don't give a shit. Hell Google is hard enough to get a hold of when you are paying them.
The second problem is spammers lie about everything. This has turned server operators on to the line of thought that 'everyone is a liar'. If you weren't a spammer you wouldn't have been blocked in the first place. Needless to say this causes a number of race conditions.
And yes, I do run outbound and inbound SMTP services fo
Re:"Free market" scare quotes (Score:4, Interesting)
Maybe Hotmail blew him off because he acts just like any other spammer. Changing domains and using remailer proxies isn't exactly the behavior of the usual legitimate bulk emailer. And yes, I do subscribe to a few of those, and I use ATT's Yahoo email account and I get my subscribed stuff just fine.
Re: (Score:2)
Re: (Score:2, Insightful)
That's just silly. If you can't be arsed to do something about your "honest it's not spam" emails getting blocked, you don't have any business complaining about the people who do the blocking. Stop complaining about "the free market" as if you'd prefer an unfree one.
Question that was never answered last time... (Score:5, Interesting)
Are the proxy servers you are sending out on these lists capable of relaying mail onwards on port 25? If so this is probably a significant factor in these blacklistings. If you block outbound connections to port 25 when you set up these proxies, you'll probably find your blacklist problems are significantly reduced.
gold standard for responsible mailing (Score:5, Informative)
Re:gold standard for responsible mailing (Score:5, Informative)
Here's the latest email I got from Mr Haselton (with the email addresses changed though).
It's apparently very easy to subscribe. (Though it's not one click as you do need to enter your email address if you use the webpage option.) Is that good enough for you?
Re: gold standard for responsible mailing (Score:2)
I'm on this particular mailing list, so I can confirm that he makes unsubscribing quite easier. Easier than any other list I've ever been on in fact. Every email has the following text as the first paragraph:
[You are receiving this because you subscribed to the Circumventor distribution list.
To unsubscribe from this list, click here:
http://www.peacefire.org/circumventor/cv-unsub.html [peacefire.org]
or reply with the word "unsubscribe" in the subject.]
Re: (Score:3)
Is this a repeat? (Score:2)
I could swear this same guy was complaining about problems with his "I swear it's not spam" mailing list several months ago.
Re: (Score:2)
Sounds like the same guy. At least the exact same scenario...
Re: (Score:2)
Re:Is this a repeat? (Score:4, Informative)
Yep, 2 months ago [slashdot.org]
Re: (Score:2)
Re: Is this a repeat? (Score:3)
Hence the words "frequent contributor" at the top.
I've been using his service for at least six years. It's as far from spam as you can get. Certainly far less spammy than the emails from newegg or Amazon (which is among the worst!) or any of the others that have no problem at all getting through spam filters. Multiple ways to unsubscribe right at the top of every message, verified opt-in, low volume, no embedded tracking features (all plain text), and legitimate content.
So what the hell else do you want? Sh
Independent verification of verified/double opt-in (Score:2, Interesting)
I used to work security at a major hosting provider. If we got complaints about your mailing list, the first thing we'd do is ask you about how you got your list, to see if it complied with our requirement for verified opt-in lists only. We'd also sign up ourselves or check logs and code, because customers always lie (except when they don't).
Right now, I'd apply the same standard of skepticism. I understand that revealing such things would make your proported aim of censorship circumvention hard, but I'd st
Re: Independent verification of verified/double op (Score:3)
Been on the list since late 2005 and I never delete an email, so I can confirm.
You subscribe at his website and you get a confirmation request email. You confirm, and it sends another message confirming that you've been added. The content is legitimate, the volume is fairly low, every email gives two unsubscribe methods in the first paragraph of the message (click a link or reply with unsubscribe) and all messages are plain text.
Not a hard problem to solve for PGP. (Score:2)
I don't understnd the animosity here (Score:3, Insightful)
Early on (before I quit reading) the OP said:
It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body.
It seems to be treating his email as spam even when he sends one email to a single address.That isn't spam.
His emails simplify the blacklister's job (Score:2)
Ironic. Almost all blacklist providers keep proxy sites on their default "bad sites" list. Were I running URLBlacklist or similar, I would simply sign up for his email service and make a point of adding every web domain spotted in his emails. Almost an instant kill for the blacklist provider; by the time email recipients can act on the information, it's already been blacklisted.
2 days (Score:2)
How in the world did it take you two days to figure out Spamhaus was blocking your stuff?
Save yourself some time down the road and just go to mxtoolbox.com. Enter the domain name it and can check all kinds of things for you. If a list is blocking it, you can get details as to why. In the past I've seen various reasons, but most are pretty detailed and provide quick access to the forms you need to get removed.
As for your idea of a secret shared blacklist between hotmail and yahoo, it sounds more like it's
Here's the real issue (Score:2)
That said, the real issue is the censorship of people's messages without their knowledge or consent. Granted, nobody wants to have to filter through millions of V1@gr@ ads just to read their mail, but on the same note, nobody wants someone else going through their mail and arbitrarily deciding what will and will not be delivered. I understand the purpose of the spam filter, and am glad it's there - but a sec
Perform listwashing, just like spammers do (Score:4, Informative)
Ironically enough, you can isolate the "moles" by listwashing [wikipedia.org], just like spammers do for spam traps.
You've already started the process: you know that three sevenths of your subscriber base is probably safe. In your next run, make sure each of the remaining four groups is subdivided again. Each time you find a group that isn't a mole, you've reduced the potential mole list. Eventually, you'll have just a few accounts and you can silently drop them from your service (or confront them, your call).
There was also an earlier comment on spammer abuse of your proxies [slashdot.org] that I'd like to expand upon. While it asks you about proxying port 25, there's also the potential for abuse with respect to port 80/443: 419er [wikipedia.org]s are increasing their use of proxies to hide their identity from free webmail providers so they can get free passes on sending spam. If you're better at cracking down on them (by e.g. blocking access to yahoo and hotmail on your proxies), you'll probably have better luck overall.
Maybe you can combine the above two ideas: groups of subscribers known to contribute to getting blocked will get domains whose proxies can't use freemail.
Re: (Score:2)
if you care about users in China and Iran
You had me up till there. At that point I realized you're an asshole and stopped reading.
Re: (Score:3)
I hate to use the if you were legit then you wouldn't need a proxy argument. However If he was using email the way most services want you to use it, he wouldn't have a problem.
Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.
Re: (Score:3)
I have to use a mail proxy, not because I spam (we send about 20 emails a month) but because verizon blocks port 25 outbound, and won't let me get a static IP at home for my mail server.
I pay 20/year for my mail proxy, gives me 200/mo that we never hit.
Re: (Score:2)
Why not use Verizon's mail server?
Most likely, because it sucks great big donkey balls. Now, that said I don't use version so I don't know for sure. What I do know from working for one of the top 10 ISPs (size wise) in the country is, most big ISP mail servers suck. Send any attachments of any size and they're apt to be blocked, get stuck in the queue, or just go in to the blackhole. Other issues are that the ISP might flag or block as your messages as spam because you want to send 200 messages on friday. And you have to put up with their
Re: (Score:2)
Because they won't let me relay my own domain through it, it sucks big fat donkey balls and it's subject to far tighter restrictions than what I use to send outbound.
I run all my outbound mail through a good spam filter (that forces all outbound to be scanned, regardless if it makes it through the mail server) and have a fairly open file size limitation (20MB, compared to last I tested Verizons 5 MB)
My outbound proxy has no size limitations, my outbound proxy handles all blacklist issues, and my outbound pr
Re: Dude (Score:2)
[Quote]Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.[/quote]
[Citation needed]
Email is Electronic Mail. You have large mailing lists like these with physical mail; you'd have to be an idiot to have thought something similar wouldn't be developed with email.
Re: (Score:2)
I hate to use the if you were legit then you wouldn't need a proxy argument. However If he was using email the way most services want you to use it, he wouldn't have a problem.
Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.
Then how do you send a message to a large group of subscribers (let's ignore the spam angle for now and say these people want the updates) notifying them of site updates, special offers, alerts, or whatnot. I don't think it's enough to say "well they should just go to the site and check it when they want to." First, I don't want to call up every web site I might have signed up with every day. I just don't want to go through that hassle. I would end up not doing it. Email is perfect for me, I can scan it qui
Re:Dude (Score:5, Insightful)
After the last article I signed up for the service of getting emailed the proxy sites. Guess what, I've had no problem. I've not recieved any spam to the email address I used. I've only received emails that I specifically requested.
So, ah.
Dude, you're a fucking idiot. Hotmail and Yahoo are not doing anyone good... Get lost!
If someone is running an incredibly popular opt-in email list, that doesn't automatically make them a spammer. In fact, because it's all opt-in it makes them the opposite. It's solicited, not unsolicited. Mr Haselton is one of the good guys, and you are a moron if you can't see that.
Re: (Score:2)
Bennett Haselton is no spammer. He's been involved in anti-censorship for nearly 20 years; he began in high school by investigating the block lists operated by the filtering software installed in many schools and libraries.
Not a spammer.
wg
Re:Dude (Score:4, Informative)
He has even sued spammers [wikipedia.org].
Re: (Score:2)
Its not Spam if you opt in. Spam is unsolicited. For this you have to request. Now is it possible the guy is bull shitting that part sure, however if we accept that the articles are bull why bother to read them?
Re: Dude (Score:3)
I've been on his list for around six years, and as far as I can tell, everything he says in the article is 100% accurate.
Also worth noting that he submits articles about these things to Slashdot quite regularly. I recall one a few months back where he was first considering this exact experiment. I'd go find it, but I'm posting from my phone.
Re: (Score:3, Informative)
Do you people not understand the concept of an email newsletter? For instance, I am subscribed to NASA Tech Briefs 's email newsletter, which purports to have an audience of over 77,000. Being a newsletter, of course those emails all have "the same web address in them" -- they're the same bloody content. This has been going on for decades (they've been a big thing since home users who never heard of usenet started getting internet access...), and as long as it ONLY GOES TO PEOPLE WHO VOLUNTARILY SUBSCRIBED,
Re: Dude (Score:5, Informative)
FWIW, I'm on that list. And if I was using hotmail or Yahoo I would be PISSED about missing those messages. Been on it since highschool where I used them to bypass the school's web filters (occasionally teachers would even promote these sites because we literally couldn't do our work without them); today I still use them for testing and occasionally at work if, for example, I need a document from scribd (why that is blocked I'll never understand...)
Re:You are a spammer (Score:5, Informative)
You missed the point. (Score:3)
The issue is that no one on the list of recipients got the chance to refuse the message.
How can you be certain he is not part of an internet forum dedicated to anonymity? What if he were sending an email with updates on domains that are security risks to a long list of subscribers to his IPsec newsletter?
There is a very long list of possibilities for what he could have been doing that was perfectly legitimate. Basically, USPS, UPS, FedEx, DHL, $common-carrier should not read your text-only message to dete
Re:You are a spammer (Score:5, Interesting)
His behaviors are _similar_ to those of a spammer in number only. Having visited his site: http://www.peacefire.org/ [peacefire.org] it seems that he gets his email list from people subscribing to it on his site. If I understand it correctly, people who sign up for this list are looking for regular updates to proxies so that they can avoid censorship. As proxies are discovered by governments or certain companies , they are blacklisted, and new proxies must be created and sent out to the interested masses:
Now it could be that there is a better way of doing this, but it seems to me that no matter how this game is played, constant updates to users should be the norm...
Now that I think of it, perhaps a Firefox extension could do the trick. Signed extensions can be updated automatically. The extension could have obfuscated URLs that are decrypted with something like this: https://addons.mozilla.org/en-US/firefox/addon/domcrypt/ [mozilla.org] and then wired in to automatically select an available proxy from the current batch. Not perfect by any stretch of the imagination, but it solves the "spam" problem. Also, it maybe easier for users and harder for censors? Crap... now I'm not going to get any work done...
Re: (Score:3)
Now it could be that there is a better way of doing this, but it seems to me that no matter how this game is played, constant updates to users should be the norm...
Now that I think of it, perhaps a Firefox extension could do the trick. Signed extensions can be updated automatically. The extension could have obfuscated URLs that are decrypted with something like this: https://addons.mozilla.org/en-US/firefox/addon/domcrypt/ [mozilla.org] and then wired in to automatically select an available proxy from the current batch. Not perfect by any stretch of the imagination, but it solves the "spam" problem. Also, it maybe easier for users and harder for censors? Crap... now I'm not going to get any work done...
There are multiple benefits of email delivery that aren't present in the Firefox Addon model:
If I were the OP, I'd cons
Re: (Score:2)
Want to know who sends mass email in batches like that?
Apple, Microsoft, NewEgg, Amazon, Zappos (an amazon company), Woot (another Amazon company), ZD Net, and so on.
Not every large volume emailer is a spammer.
Re:5 second summary (Score:5, Informative)
You assume that this is case, yet the poster provides a link to management data which at least appears to show that your assumption is incorrect. Did you read the post where it mentions that "[it] showed a 'complaint rate' of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list)," or are you simply going to deny any version of reality that doesn't align with your assumptions.
Apparently, deny any version of reality that doesn't align with your assumptions.
BAD 'EXPERT'!
If I sign up to a mailing list, I expect to receive the output of that mailing list until I unsubscribe. I certainly don't want the mailing list silently dropping me, and I'm not very interested in the ISP offloading its mailing list problem onto me by making me affirmatively renew my subscription. Especially when you offer no evidence that 'addresses that signed up a long time ago' make up a disproportionate fraction of the alleged 0.1% spam report rate.
Pushing the problem onto the 400,000+ individual users instead of dealiing with it at the ISP level is exactly the sort of free market failure tha the poster complains of.
Again, deny any version of reality that doesn't align with your assumptions. He isn't being blocked by SpamHaus. He's being blocked by Hotmail and Yahoo. Just admit that you haven't actually read the post, that you're spouting off about your own personal bugbear, and that your advice has almost no bearing on the actual problem. It'll make you feel better, honest.
Re: (Score:3)
I assume this is the case because, like I said, having actually worked on a large spam filter I've seen this kind of story many times before. These people are always amazed to discover that people are pressing report spam on their wonderful bulk mail. Yet the fact remained that people were doing exactly that. They didn't want the mail.
Look at it this way. This guys
Re: (Score:3, Interesting)
Why does he need to send 400,000+ emails in the first place? If it's just a list of proxy domains, why not just have an RSS feed that people can subscribe to? No emails needed.
Re:5 second summary (Score:5, Insightful)
Because then someone from the censorship companies or the censorship departments could easily get all the latest domains and block them automatically. By creating multiple domains and emailing them to a section of his subscriber list, he makes it that much harder to block all of them.
Re: (Score:2)
Because the RSS feed's server will likely get blocked, but the emails are less likely.
Re: (Score:2)
Um... wow, it's sorta sad that I have to explain this.
Imagine you're the Chinese Minister of Censorship, or the flunky that manages the Great Firewall. You learn about a website with an RSS feed with a continually updated list of anti-censorship proxies. What do you do?
(On the other hand, you haven't blocked Hotmail or Yahoo! or other email providers, because, well, riots are bad for business.)
Re: (Score:2)
Because then the blocking companies would just subscribe to the RSS and the proxies would be blocked as soon as they were posted.
Re: (Score:2)
+1. TD;DR the article, but the parts I did made this whole story reek of "your unsubscription method isn't braindead obvious enough to end-users, so they're unsubscribing by hitting the Spam button until your emails go away for good."
Re: (Score:3)
At the top of the emails:
Seems pretty easy to me...
Re: (Score:3)
Wrong.
If it requires anymore than clicking a link in the email, its failed. Going to a page, doing more crap, blah blah blah, I just hit 'spam' and move on, so does everyone else. If I don't want it, its spam, period. You as the sender need to make it so A) I want it and B) I don't get bored/annoyed trying to get rid of it after I'm done wanting it.
He also hasn't bothered to setup feedback loops with Yahoo and Hotmail, which would solve his problem and show that he had a clue.
He's also sending a list of
Re: (Score:3)
They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it.
That's one possibility, and may even be likely considering his subject material. In this example he says he sent a total of 7 new proxy domains to 420,000 addresses, but only sent 1 domain to each person. So each domain got sent to a random 60,000 people, his reasoning being so that a censor could not subscribe and get a list of all new proxies, they would only get one (per address, at least).
But, instead of them getting those emails and blocking the proxies, it may be more effective for the censors to al
Re: 5 second summary (Score:2)
RTFS. Hotmail confirmed that the portion of users marking it as spam was extremely small.
Furthermore, do you realize how many users will click the 'spam' button when they fully know it's something they subscribed to simply because they can't be bothered to take half a second to click the prominent unsubscribe link or send a reply? These people are trashing spam filters. And I know they're out there, because I got it all the time in college. Ran a student club with a mailing list of around 400 users (out of
Re: (Score:2)
Re: (Score:2)
Screw that (Score:3)
And should the HFH and ACLU and all the other newsletters I subscribe to be blocked as spam as well? They send far more than 400k emails a month. Email is more convenient than RSS or worse Twitter, and is newsletters are a perfectly legitimate use of the medium.