[Tim]
> One effect of getting rid of MINCOUNT is that it latches on more
> strongly to rare clues now, and those can be unique to the corpus
> trained on (e.g., one trained ham says "gryndlplyx!", and a followup
> new ham quotes it).
This may be a systematic bias in the testing procedure:  in real life, msgs
come ordered in time.  Say there's a thread that spans N messages on c.l.py.
In our testing setup, we'll train on a random sampling throughout its whole
lifetime, and test likewise.  New ham "in the middle" of this thread gets
benefit from that we trained on msgs that appeared both before and *after*
it in real life.  It's quite plausible that the f-p rate would rise without
this effect; in real life, at any given time some number of ham threads will
just be starting their lives, and if they're at all unusual the trained data
will know little to nothing about them.

from localhost (jalapeno [127.0.0.1])
	by example.com (Postfix) with ESMTP id CCC0116F17
	for <zzzz@localhost>; Tue,  8 Oct 2002 10:55:22 +0100 (IST)
from dogma.slashnull.org (localhost [127.0.0.1]) by
    dogma.slashnull.org (8.11.6/8.11.6) with ESMTP id g98804K06008 for
    <zzzz@example.com>; Tue, 8 Oct 2002 09:00:04 +0100
From rssfeeds@example.com  Tue Oct  8 10:55:27 2002

URL: http://diveintomark.org/archives/2002/10/08.html#teach_a_man_to_fish
Date: 2002-10-08T00:22:08-05:00
_Kevin Hemenway_: Finding More Channels[1]. &#8220;In simple terms, there are
thousands of web sites that are actively providing their news and headlines in
a format AmphetaDesk can understand [RSS]. And while AmphetaDesk knows about a
good number of these sites, it'd be impossible to hunt down each and every
single possibility. So, this page is here to teach you how to fish.&#8221;
[1] http://www.disobey.com/amphetadesk/finding_more.html

[Guido]
> There seem to be two "drivers" for the classifier now: Neale Pickett's
> hammie.py, and the original GBayes.py.  According to the README.txt,
> GBayes.py hasn't been kept up to date.
It seemed that way to me when I ripped the classifier out of it -- I don't
think anyone has touched it after.
> Is there anything in there that isn't covered by hammie.py?
Someone else will have to answer that (I don't use GBayes or hammie, at
least not yet).
> About the only useful feature of GBayes.py that hammie.py doesn't (yet)
> copy is -u, which calculates spamness for an entire mailbox.  This
> feature can easily be copied into hammie.py.
That's been done now, right?
> (GBayes.py also has a large collection of tokenizers; but timtoken.py
> rules, so I'm not sure how interesting that is now.)
Those tokenizers are truly trivial to rewrite from scratch if they're
interesting.  The tiny spam/ham collections in GBayes are also worthless
now.  The "self test" feature didn't do anything except print its results;
Tester.py since became doctest'ed and verifies that some basic machinery
actually delivers what it's supposed to deliver.
> Therefore I propose to nuke GBayes.py, after adding a -u feature.
+1 here.
> Anyone against?