[Tim]
> Another area for potentially fruitful study:  it's clear that the
> highest-value indicators usually appear "early" in msgs, and for spam
> there's an actual reason for that:  advertising has to strive
> to get your attention early.  So, for example, if we only bothered to
> tokenize the first 90% of a msg, would results get worse?
[Neil Schemenauer]
> Spammers could exploit this including a large MIME part at the beginning
> of the message.  In pratice that would probably work fine.
Note that timtest.py's current tokenizer only looks at decoded text/* MIME
sections (or raw message text if no MIME exists); spammers could put
megabytes of other crap before that and it wouldn't even be looked at
(except that the email package has to parse non-text/* parts well enough to
skip over them, and tokens for the most interesting parts of Content-{Type,
Disposition, Transfer-Encoding} decorations are generated for all MIME
sections).
Schemes that remain ignorant of MIME are vulnerable to spammers putting
arbitrary amounts of "nice text" in the preamble area (after the headers and
before the first MIME section), which most mail readers don't display, but
which appear first in the file so are latched on to by Graham's scoring
scheme.
But I don't worry about clever spammers -- I've seen no evidence that they
exist <0.5 wink>.  Even if they do, the Open Source zoo is such that no
particular scheme will gain dominance, and there's no percentage for
spammers in trying to fool just one scheme.  Even if they did, for the kind
of scheme we're using here they can't *know* what "nice text" is, not unless
they pay a lot of attention to the spam targets and highly tailor their
messages to each different one.  At that point they'd be doing targeted
marketing, and the cost of the game to them would increase enormously.
if-you're-out-to-make-a-quick-buck-you-don't-waste-a-second-on-hard-
targets-ly y'rs  - tim

from localhost (jalapeno [127.0.0.1])
	by jmason.org (Postfix) with ESMTP id A535616F03
	for <jm@localhost>; Fri,  4 Oct 2002 11:00:40 +0100 (IST)
from jalapeno [127.0.0.1]
	by localhost with IMAP (fetchmail-5.9.0)
	for jm@localhost (single-drop); Fri, 04 Oct 2002 11:00:40 +0100 (IST)
from egwn.net (ns2.egwn.net [193.172.5.4]) by
    dogma.slashnull.org (8.11.6/8.11.6) with ESMTP id g941oUK29944 for
    <jm-rpm@jmason.org>; Fri, 4 Oct 2002 02:50:30 +0100
from smtprelay9.dc2.adelphia.net (smtprelay9.dc2.adelphia.net
    [64.8.50.53]) by egwn.net (8.11.6/8.11.6/EGWN) with ESMTP id g941lXf28542
    for <rpm-list@freshrpms.net>; Fri, 4 Oct 2002 03:47:34 +0200
from [192.168.1.2] ([24.50.195.5]) by
    smtprelay9.dc2.adelphia.net (Netscape Messaging Server 4.15) with ESMTP id
    H3FOZ301.09V for <rpm-list@freshrpms.net>; Thu, 3 Oct 2002 21:47:27 -0400
From rpm-list-admin@freshrpms.net  Fri Oct  4 11:00:41 2002