> Well, I've finally got around to pulling down the SF code. Starting
> with it, and absolutely zero local modifications, I see the following:
How many runs is this summarizing?  For each, how many ham&spam were in the
training set?  How many in the prediction sets?  What were the error rates
The effect of set sizes on accuracy rates isn't known.  I've informally
reported some results from just a few controlled experiments on that.
Jeremy reported improved accuracy by doubling the training set size, but
that wasn't a controlled experiment (things besides just training set size
> My next (current) task is to complete the corpus I've got - it's currently
> got ~ 9000 ham, 7800 spam, and about 9200 currently unsorted. I'm tossing
> up using either hammie or spamassassin to do the initial sort  -
> I've used various forms of 'grep' for keywords and a little gui thing to
> pop a message up and let me say 'spam/ham', but that's just getting too,
Yup, tagging data is mondo tedious, and mistakes hurt.
I expect hammie will do a much better job on this already than hand
grepping.  Be sure to stare at the false positives and get the spam out of
> I can't make it available en masse, but I will look at finding some of
> the more 'interesting' uglies. One thing I've seen (consider this
> 'anecdotal' for now) is that the 'skip' tokens end up in a _lot_ of the
With probabilities favoring ham or spam?  A skip token is produced in lieu
of "word" more than 12 chars long and without any high-bit characters.  It's
possible that they helped me because raw HTML produces lots of these.
However, if you're running current CVS, Tokenizer/retain_pure_html_tags
defaults to False now, so HTML decorations should vanish before body

from localhost (localhost [127.0.0.1])
	by phobos.labs.netnoteinc.com (Postfix) with ESMTP id CEE9544134
	for <jm@localhost>; Tue, 13 Aug 2002 05:22:13 -0400 (EDT)
from phobos [127.0.0.1]
	by localhost with IMAP (fetchmail-5.9.0)
	for jm@localhost (single-drop); Tue, 13 Aug 2002 10:22:13 +0100 (IST)
from lugh.tuatha.org (root@lugh.tuatha.org [194.125.145.45]) by
    dogma.slashnull.org (8.11.6/8.11.6) with ESMTP id g7D91Nb24248 for
    <jm-ilug@jmason.org>; Tue, 13 Aug 2002 10:01:23 +0100
from lugh (root@localhost [127.0.0.1]) by lugh.tuatha.org
    (8.9.3/8.9.3) with ESMTP id KAA02477; Tue, 13 Aug 2002 10:00:49 +0100
by w2k-server.bcs.ie with Internet Mail Service (5.5.2653.19) id
    <QMRGFKKV>; Tue, 13 Aug 2002 10:03:16 +0100
From ilug-admin@linux.ie  Tue Aug 13 10:30:01 2002