Are Filters the Way to Stop Spam?

Yes, Says Paul Graham

There are two senses of stopping spam--stopping it from filling up our in-boxes and stopping spammers from sending it. Of course, if you solve the first problem, you also solve the second. Spammers send spam to make money. If no one sees the spam, they'll be wasting their time sending it and soon will stop.

Two of the most promising solutions to the spam problem are to filter it and to outlaw it. It's too early to say for sure which will win, but so far, filtering works and laws don't.

Building Better Filters

A year ago, few people thought filtering was a practical solution. Earlier filters, which identified mail as spam based on whether it contained specific words, were not very effective. If you made them tight enough to catch most spam, you got too many false positives--e-mails mistakenly identified as spam.

The new generation of statistical (also known as Bayesian) filters are much better. Mine lets through 2.5 spams per 1,000, with about 0.5 per 1,000 false positives. Moreover, the false positives that statistical filters yield tend to be mail that resembles spam: newsletters and advertising, not personal mail.

The argument against filters is that we still have to pay the cost of transmitting the e-mail. But this cost would go away if filters were widespread because response rates would be so low that it wouldn't pay to spam. And filters are becoming widespread because it is in the interest of the big online services to implement them. It decreases their infrastructure cost if they're known to be spam-proof, and, as MSN's full-page ads testify, effective spam protection is a big marketing advantage.

Creating Laws

There are two problems with trying to outlaw spam--the legitimate direct marketing lobby and the difficulty of enforcement. Direct marketers want to ensure that spam laws still permit them to contact their customers. The resulting loopholes are so big that spammers get through, too. Because the company they bought your e-mail address from is an "affiliate," they consider you their customer, too. Perhaps a law could be written that is tight enough to prevent this, but I doubt it.

There are several grades of spammers, from companies that call themselves "opt-in" mailers to the guys who hijack mail servers to send pornography. A tightly written law might shut down the "opt-in" spammers, but without effective enforcement the pornography spammers will just ignore it.

Enforcement is a hard problem. Spammers route a lot of their spam through servers offshore. What happens when they move their companies offshore, too? Are we going to be able to extradite people for spamming?

I'm not against trying to outlaw spam. I just don't think new laws will work any better than the current laws. Filtering works now.

Graham has written two books on Lisp and was a founder of the start-up that became Yahoo Store. Recently he has worked on spam filters and a new language called Arc. For more on filtering, see PaulGraham.com.

No, Says Jason Catlett

Filtering is no more a solution to the spam problem than it is to water pollution. The right thing to do is to restrain the producers of pollution, rather than routinely burden someone downstream with the task of cleaning up an unfairly imposed mess. The cleanup task is necessarily an imperfect and expensive business.

There is no federal law against spamming, but Congress might act this session. A good law would ban bulk unsolicited commercial e-mail and let individuals who are spammed sue the spammer, just as they currently can sue junk faxers. A bad law would let spamming continue provided the spam is labeled with "ADV" or some such indicator of an unsolicited advertisement in the subject line.

The idea, appealing on the surface, is to make spam easy to filter. But this ignores several facts: Not all spammers will label, not everyone has filters, and, even if they did, much of the unfair burden on the Internet infrastructure would remain, as ISP servers forward spam to networks only to have it deleted at some later point.

Spam has grown in recent years from less than 10 percent of all e-mail to about 40 percent. If it continues at this rate, the resources required simply to delete most of the junk before it is forwarded will run into billions of dollars. This is an unfair tax on consumers and organizations.

Imperfect Process

Filtering by ISPs and corporate networks is commonplace, but filtering is and will always be an imperfect process. Filters inevitably make two types of errors: false negatives, in which they let a piece of spam go through; and false positives, in which they throw out something that the recipient actually wants. ISPs are forced to be conservative in their filtering to avoid false positives, which can be costly for businesses that rely on e-mail as an interface to customers for sales and service.

Although filtering technology has become extremely sophisticated, spammers play the cat-and-mouse game with great agility and currently hold the advantage of larger numbers and little economic or legal incentive to stop. When appropriate legislation increases those incentives, filtering at the network level still will be necessary, but it should not be a major systemic cost. If the public resigns itself to tolerating spam and accepting the burden of filtering it (imperfectly), users' disenchantment with e-mail likely will reach a tipping point where many abandon e-mail.

Believing that filters can prevail against unrestrained spammers is betting that we can win an expensive arms race that we already appear to be losing. It also is buying in to a very expensive maintenance overhead and ongoing collateral damage to our daily correspondence. To put our faith solely into technology here would be as foolish as the residents of Venice ignoring the rising tide around them and relying on the water pumps in their basement.

Catlett is president and founder of Junkbusters, a privacy advocacy firm in Green Brook, N.J. He can be reached at Junkbusters.com.