Am I a member?
Browse the member listing...

How One Mid-Size Firm Slew the Spam Monster

Spam, in my opinion the biggest annoyance since the demise of WordPerfect 5.1, simply won’t go away, despite efforts of even the most aggressive spam blockers.  The meteoric rise in spam volume last year brought the issue to a head at our firm — and here’s the story of how we fought it and how we triumphed.

Where We Started
When I assumed the role of CTO at Stoll, Keenon & Park early last year, my first task was to survey our firm’s technology landscape.  With 200+ users in four locations, we have been dedicated Novell/GroupWise users all the way back to the days of WordPerfect Office; and when I came onboard, we were running GroupWise 6.5.  Spam was certainly on our radar, but not dead center, as the exponential surge in junk e-mail volume had not yet occurred. 

Testing the Waters with Gwava

Most of the firm’s high-level Netware work was outsourced.  Our contractor was recommending an NLM-based product called Gwava and reporting successful deployment.  We loaded the trial version and took it for a spin.  The impact was immediate and dramatic.  Offers for drugs, mortgage deals and various body-part enhancers got caught in the trap, and users breathed a sigh of relief.  We had an IT staff of exactly one at the time (that’s right, our ratio was 200:1!).  He was reviewing the spam trap on a daily basis looking for, and releasing to the user, legitimate e-mail messages incorrectly filtered as spam.  Initially, he spent about an hour a day on the task of rooting out false positives, and for a while, all was well.

Positively Daunting
But then we started finding false positives on a regular basis.  And then we started observing that as the volume of spam skyrocketed, the Gwava heuristic filters were becoming less and less accurate.  This was especially true as the spammers got smarter and started embedding graphics in lieu of text in their messages.  Plus, we still had the same volume of e-mail messages making their way to our server, and we still had problems with people being careless with attachments.  By year’s end our IT person was spending two to three hours a day reviewing the spam trap.  This onerous job was becoming punitive and out of control.  But by spring we were devoting four hours a day to monitoring the spam trap.  We got hammered by a couple of nasty virus assaults along the way and some pesky malware.  

The situation steadily grew worse.  False positives continued to increase.  And lawyers opted out, fearing loss of legitimate e-mail. (Gwava could send a notification to the sender of any e-mail message trapped, but some attorneys felt this was in poor taste.) 

Time to Fight!
Finally, we’d had more than enough!  We had originally upgraded to GroupWise 6.5 on the promise of improved junk e-mail handling but found it sorely lacking.  Now it was time to aggressively search for other options.

Postini to the Rescue
We checked out a number of products that promised to tame the beast and help us regain control over our e-mail system.  And then a very favorable Web review of Postini was forwarded to me by the firm administrator.  I immediately called them and signed up for a trial run.  We selected a small test group, moved our MX record and, in a matter of hours, had Postini fully functional.  Our test lasted less than a week.  The results were so profoundly positive that on April 1 (no fooling!) we moved the entire office to Postini.  Adding 200 new users took only 15 minutes.  All I had to do was build a user list like the following and submit it as a script at the Postini site:

ADDUSER username1@skp.com
ADDUSER username2@skp.com
ADDUSER username3@skp.com

New users were automatically sent an
e-mail notification with instructions and a link to their personal account on Postini’s server.  That’s where they went to check their quarantined e-mail, build their blacklists and release false positives.

Postini also quarantines messages with virus infections, cleans them and allows the user to view them.

Since going this route, we have experienced the following:

  • Zero virus infestations.  Zip.  Nada.  Yes, really.  Postini uses McAffee technology to isolate any message traffic with a virus payload.  Our server uses Symantec technology as a second line of defense.

    We enjoy an astounding 77 percent average trap rate for spam.  Nearly eight out of 10 messages directed to our domain are junk e-mail.

    A side benefit is that 77 percent of e-mail directed to our mail server never gets there.  So we’ve reclaimed bandwidth and disk space.
    False positives are practically non-existent, and attorney confidence levels are extremely high.

    Administrative time for the IT staff has been reduced to zero.  They can now focus on client service, as they should.

    Across the board buy-in from users — no more opt-outs.  We made participation in Postini mandatory but gave our users control over their quarantine.  Nobody has complained.

Lessons Learned
Our original mistake had been trying to manage spam internally — we simply didn’t have the IT manpower to deal with this onerous task.  We had also grossly underestimated the exponential growth of spam content in 2003.  In addition, some worried that our client
e-mail was being filtered by a third party — in violation of client confidentiality.  Fortunately, the accuracy of Postini’s filters eliminated that concern in short order.

Early on, any missing e-mail messages were immediately attributed to our spam-blocking efforts.  We learned that attorneys and staff alike had a false sense of immediacy and perhaps complacency that when they hit “Send,” a message was assumed to reach its destination.  We spent considerable time and energy educating users about other reasons for failed delivery, such as receiving systems being down, message quotas exceeded, recipient spam defenses blocking our messages, delays caused by router outages in distant cities, inadvertent blacklisting and on and on.  We reinforced in our users the need to back up transmission of time-sensitive
e-mail and documents with a phone call to the recipient to ensure receipt. 

Gwava did not give us the option of pushing false positive management out to the end-user, so we had no choice but to manually review the spam trap on a daily basis.  In hindsight, that was a mistake.  Human eyes reviewing thousands of messages for false positives makes for tired eyes — it’s risky business.  After seeing hundreds of credit card offers, the term VISA becomes emblazoned in the reviewer’s mind as spam.  But when you have an immigration practice group, “Visa” takes on a different meaning.  Legitimate
e-mail can get lost.

With Gwava, infected e-mail attachments still found their way into our e-mail system, despite our use of Norton’s corporate anti-virus product.  Like the child who touches the hot stove after being warned, we had users who disregarded our warnings about suspect e-mail attachments. 

The Postini approach completely changed the model in these ways:

1. Suspect e-mail got quarantined at their server, not ours.  If a virus was found, it was removed.
2. Postini continually updates their virus-filtering schema and works closely with McAfee to post up-to-the-minute virus traps.  Every user can review infected suspect e-mail, but that e-mail stays on Postini’s server, not ours.
3. Our second level scan at our server, using a different technology gives us redundancy.  No virus problems have been reported since April 1, 2004.

Four Happy Conclusions

1. Keeping suspect e-mail traffic off your server is absolutely a better approach than a system of internal scans once the e-mail reaches your server.
2. Giving users responsibility for monitoring their own spam quarantines builds confidence in the system and takes an enormous burden off of a limited and overworked IT staff.
3. Spam, a productivity killer that exposed our users to salacious and often graphically sexual content against their wishes, has been reduced to a nuisance level.
4. Spam is not going away in the near-term, but we can keep it off our networks and out of our faces with the right combination of tools.

One Caveat
Postini is priced on an annual per user basis, but there is a minimum buy.  It may therefore not be affordable for small firms.  However, with competition heating up in this space, that pricing model could change.

Spam?  What spam?  Our firm has been liberated from the evil clutches of this monster.  Yours could be, too.

About our author . . .

Paul Mansfield is the CTO for Stoll, Keenon & Park, LLP in Lexington, Kentucky.  Paul served for 10 years as a member of the State Bar of New Mexico Technology Utilization Committee and is a current member of the Economics of Law Section of the American Bar Association.  He can be reached at 505.898.0710 or mansfield@skp.com.

From: 
Email:  
To: 
Email:  
Subject: 
Message: