AT&T Worldnet and Brightmail spam filter

This web page is a summary of my observations of the efficacy of the spam filter (Brightmail) made available to me as an AT&T Worldnet customer.

These observations are on mail sent to my email address that was established seven years ago. I have made no effort to obfuscate this address. It is present in many usenet newsgroups and on many web pages, thus it is "ripe" for spammers to "harvest" as a valid address.

I turned on the Brightmail spam filter on September 28, 2002. I opted to have the filter put messages identified as spam into a "screened mail" folder rather than delete the messages outright. I review the screened messages each day. On the rare occasion that a message is wrongly identified as spam (a "false positive") I move it into my inbox and then download it into my mail client (Netscape 7.1). After reviewing the screened mail folder each day, I empty the folder.

When I get a spam message into my mail client that was missed by Brightmail, I forward it, complete with all headers, to missed-spam@att.net. The exception is those messages containing a virus; I don't send them to missed-spam.

I didn't keep track of the number of messages caught by brightmail in 2002. However, I forwarded 916 messages to missed-spam in 2002. (In 2004 I opted to stop sending messages to missed-spam - see correspondence below.)

Starting in 2003, I recorded each day's total of spam messages caught by Brightmail and placed into my screened mail folder. I recorded spam missed by Brightmail; these are the messages I forwarded to missed-spam. I have also recorded false positives, and, recently, the number of the "swen" virus messages that were placed into my screened mail folder by Brightmail. I count as "swen" those messages that are 143K in size in the screened mail folder; this conclusion is based on my experience with this virus and may not be entirely accurate.

I do not use the Brightmail virus filter; I suspect that the results would be somewhat different if I was using both the spam filter and the virus filter. At this time I prefer to handle virus messages with my mail client and McAfee VirusScan.

The number of spam messages has increased from 43 per day in January 2003 to 186 per day during the last full week of November 2003. Even though Brightmail has improved from catching 77% in January to 82% at the end of November, the sheer number of spam messages gives a false appearance that Brightmail isn't working as well as it did earlier in the year.

I feel Brightmail is doing a superb job; it isn't perfect, but the number of false positives are nearly zero.

I hope this information is useful to others.

This web page is copyrighted. Link to this page if you like, but if you wish to publish or use this information in any other way, please contact me for permission to do so.

Jay O'Brien
Rio Linda, CA
November 29, 2003

Month	Brightmail Screened Mail	% spam caught by Brightmail	Brightmail missed spam	Total spam	spam per day	false positives	screened swen virus
1/03	1028	76.8%	311	1339	43	0
2/03	1017	69.4%	449	1466	52	0
3/03	1616	77.1%	481	2097	68	1
4/03	2022	78.1%	568	2590	86	0
5/03	2783	82.0%	610	3393	110	0
6/03	2997	85.2%	519	3516	117	7
7/03	3282	85.3%	565	3847	124	0
8/03	2890	86.7%	443	3333	108	1
9/03	3414	90.1%	376	3790	126	2	33
10/03	3748	88.6%	482	4230	137	3	350
11/03	4161	83.2%	840	5001	167	0	129
12/03	3706	78.3%	1028	4734	153	0	36
Total 2003	32664	83.0%	6672	39336	108	14	548
1/04	3366	82.4%	718	4084	132	1	74
2/04	3831	92.1%	328	4159	143	1	124
3/04	3795	90.7%	387	4182	135	0	28
4/04	4136	88.4%	542	4678	156	2	15
5/04	3818	89.2%	464	4282	138	0	1
6/04	3676	89/.9%	411	4087	136	0
7/04	3397	87.8%	474	3871	125	0
8/04	3575	83.7%	697	4272	138	0

Week starting	Brightmail Screened Mail	% spam caught by Brightmail	Brightmail missed spam	Total spam	spam per day	false positives	total email received
6/1/03	656	86.9%	99	755	108
6/8/03	706	88.4%	93	799	114
6/15/03	695	82.1%	152	847	121	7
6/22/03	744	83.5%	147	891	127
6/29/03	631	85.2%	110	741	106
7/6/03	559	82.0%	123	682	97
7/13/03	816	84.6%	149	965	138
7/20/03	837	86.4%	132	969	138
7/27/03	846	86.2%	135	981	140
8/3/03	763	85.1%	134	897	128
8/10/03	510	82.5%	108	618	88	1
8/17/03	601	89.0%	74	675	96
8/24/03	707	91.2%	68	775	111
8/31/03	636	91.0%	63	699	100
9/7/03	806	89.4%	96	902	129
9/14/03	671	91.2%	65	736	105	1
9/21/03	975	90.3%	105	1080	154
9/28/03	969	90.8%	98	1067	152	1
10/5/03	950	91.4%	89	1039	148	2
10/12/03	713	89.7%	82	795	114
10/19/03	872	88.9%	109	981	140	1
10/26/03	823	83.7%	160	983	140
11/2/03	888	85.0%	157	1045	149
11/9/03	969	83.2%	195	1164	166
11/16/03	963	82.9%	199	1162	166
11/23/03	1061	81.3%	244	1305	186
11/30/03	898	78.2%	250	1148	164
12/7/03	817	75.4%	266	1083	155
12/14/03	833	78.5%	228	1061	152
12/21/03	770	78.6%	210	980	140		2121
12/28/03	785	79.0%	209	994	142		2606
1/4/04	655	77.1%	194	849	121		2451
1/11/04	744	75.7%	239	983	140	1	2452
1/18/04	846	88.7%	108	954	136		2494
1/25/04	849	91.3%	81	930	133		2590
2/1/04	938	87.3%	137	1075	154		2458
2/8/04	927	92.2%	78	1005	144		2270
2/15/04	937	94.4%	55	992	142	1	2304
2/22/04	917	94.5%	53	970	139		2411
2/29/04	994	94.7%	56	1050	150		2583
3/7/04	646	91.5%	60	706	101		2032
3/14/04	884	88.8%	111	995	142		2320
3/21/04	853	89.9%	96	949	136		2052
3/28/04	929	86.7%	143	1072	153		2655
4/4/04	979	86.8%	149	1128	161		2459
4/11/04	949	88.8%	120	1069	153		2301
4/18/04	1037	89.3%	124	1161	166		2659
4/25/04	869	91.1%	85	954	136	2	2296
5/2/04	818	87.8%	114	932	133		2214
5/9/04	842	88.7%	107	949	136		2479
5/16/04	862	90.0%	96	958	137		2463
5/23/04	973	90.1%	107	1080	154		2566
5/30/04	937	87.9%	129	1066	152		2373
6/6/04	888	90.9%	89	977	140		2518
6/13/04	840	89.8%	95	935	134		2408
6/20/04	808	91.5%	75	883	126		2409
6/27/04	785	91.4%	74	859	123		2592
7/4/04	710	86.3%	113	823	118		2455
7/11/04	732	88.5%	95	827	118		2730
7/18/04	846	89.5%	99	945	135		3262
7/25/04	753	84.7%	136	889	127		3560
8/1/04	823	86.1%	133	956	137		3274
8/8/04	837	82.5%	177	1014	145		3102
8/15/04	782	86.1%	126	908	130		3043
8/22/04	808	80.9%	191	999	143		3402
8/29/04	759	81.5%	172	931	133		2722
9/5/04	714	78.7%	193	907	130		1938
9/12/04	750	82.1%	164	914	131		1983
9/19/04	794	89.4%	94	888	127		2178
69 weeks	56484	86.5%	8817	65301	135	17	note 1

Follow-up information and correspondence: