Junk Filter Question

In article ,
Howard Brazee wrote:

> On Fri, 8 Feb 2008 19:25:26 +0000, peter.DeleteThis@cara.demon.co.uk (Peter
> Ceresole) wrote:
>
> >> I don't want to program it - I just want to give it guidance.
> >
> >You do that by marking mails as 'not junk', or leaving them in the junk.
>
> I guess I'm not very good at communicating. It does *not* know what
> criteria I use in determining junk. It has false positives and misses
> negatives.
>
> It could be improved by knowing what I want.

And you tell it what you want by correcting its behavior. If you find a
false positive, correct it. If you find a negative, correct it. Over
time, you should see a gradual increase in accuracy.

--
Note: Please send all responses to the relevant news group. If you
must contact me through e-mail, let me know when you send email to
this address so that your email doesn't get eaten by my SPAM filter.

JR
 >> Stay informed about: Junk Filter Question 

Howard Brazee wrote:

> I guess I'm not very good at communicating.

No, you made it perfectly clear. Provably I was the one who put it
badly...

> It does *not* know what
> criteria I use in determining junk.

But by marking mails 'junk' or 'not junk', you are indicating your
criteria, and the program gradually learns them. It then applies those
criteria- *your* criteria- to new incoming mail. That's how heuristics
work.

> It could be improved by knowing what I want.

Indeed. Which is precisely what you are indicating when you go through
the junk mailbox and sort the mail within it according to your criteria.
--
Peter
 >> Stay informed about: Junk Filter Question 

On Fri, 8 Feb 2008 20:26:21 +0000, peter.RemoveThis@cara.demon.co.uk (Peter
Ceresole) wrote:

>> It could be improved by knowing what I want.
>
>Indeed. Which is precisely what you are indicating when you go through
>the junk mailbox and sort the mail within it according to your criteria.

It could be improved by knowing why I classify one e-mail as junk and
another that would look to you to be very similar - not junk.
 >> Stay informed about: Junk Filter Question 

In article ,
Howard Brazee wrote:

> On Fri, 8 Feb 2008 20:26:21 +0000, peter.RemoveThis@cara.demon.co.uk (Peter
> Ceresole) wrote:
>
> >> It could be improved by knowing what I want.
> >
> >Indeed. Which is precisely what you are indicating when you go through
> >the junk mailbox and sort the mail within it according to your criteria.
>
> It could be improved by knowing why I classify one e-mail as junk and
> another that would look to you to be very similar - not junk.

I doubt that.

--
Note: Please send all responses to the relevant news group. If you
must contact me through e-mail, let me know when you send email to
this address so that your email doesn't get eaten by my SPAM filter.

JR
 >> Stay informed about: Junk Filter Question 

Howard Brazee wrote:

> It could be improved by knowing why I classify one e-mail as junk and
> another that would look to you to be very similar - not junk.

Possible, but very unlikely. Heuristic algoriths, if properly written
and regularly fed with info, are almost certainly cleverer than we are.
--
Peter
 >> Stay informed about: Junk Filter Question 

On Fri, 8 Feb 2008 20:53:37 +0000, peter DeleteThis @cara.demon.co.uk (Peter
Ceresole) wrote:

>> It could be improved by knowing why I classify one e-mail as junk and
>> another that would look to you to be very similar - not junk.
>
>Possible, but very unlikely. Heuristic algoriths, if properly written
>and regularly fed with info, are almost certainly cleverer than we are.

I don't disagree. But I wasn't proposing a choice between the two, I
was proposing an integration of two heads.
 >> Stay informed about: Junk Filter Question 

Howard Brazee wrote:

> I don't disagree. But I wasn't proposing a choice between the two, I
> was proposing an integration of two heads.

In that case, you've already got it. Nothing stops you writing your own
filters which would work in conjunction with Eudora's junk filter. Or am
I wrong about that..?
--
Peter
 >> Stay informed about: Junk Filter Question 

This message is not archived
 >> Stay informed about: Junk Filter Question 

In article ,
Dave Balderstone wrote:

> The junk mail filter needs to be the first one. Anything matching a
> rule run before the spam filter, where that rule includes a "stop
> processing" flag, will by definition NOT be processed by the spam
> filter.
>
> It ain't rocket science.

Since the spam filters are run first and it's trapping wanted mail,
tweaking the spam filters or using "training" are other options. That's
already been covered by other replies.

If spam filtering came second why would stop processing be a problem?
It's wanted mail - having those desired messages processed further by
the spam filters is not necessary. Telling the application to "stop
processing" these wanted messages and to move on to filtering other
messages would be appropriate and helpful.

I wasn't aware that spam filters ran first. Now I am. Thanks.

--
Sharon F
 >> Stay informed about: Junk Filter Question 

In article ,
peter.TakeThisOut@cara.demon.co.uk (Peter Ceresole) wrote:

> Howard Brazee wrote:
>
> > But many spam filters only get told "yes this is spam", or "no this
> > isn't spam". I want them to know why.
>
> The whole point of heuristic learning (which I believe the spam filters
> use) is that they derive their own rules from the examples you give
> them. These rules can be very complicated. This avoids human
> programming, which is the source of so many errors.

That's not really how most training-based spam filters work. Most are
based on a system known as "Naive Bayesian" classification. That
approach isn't based on complex rules, but rather on a lot of very
simple rules with scores derived statistically. The words in the message
are tallied to a big list of word frequencies in spam and non-spam
messages, and that list of relative frequencies is used as a lot of very
simple scoring rules, with each rule testing the presence of a word and
adding a score based on the frequency of the word in prior spam and
non-spam.

--
Now where did I hide that website...
 >> Stay informed about: Junk Filter Question 

Bill Cole wrote:

> > The whole point of heuristic learning (which I believe the spam filters
> > use) is that they derive their own rules from the examples you give
> > them. These rules can be very complicated. This avoids human
> > programming, which is the source of so many errors.
>
> That's not really how most training-based spam filters work. Most are
> based on a system known as "Naive Bayesian" classification. That
> approach isn't based on complex rules, but rather on a lot of very
> simple rules with scores derived statistically. The words in the message
> are tallied to a big list of word frequencies in spam and non-spam
> messages, and that list of relative frequencies is used as a lot of very
> simple scoring rules, with each rule testing the presence of a word and
> adding a score based on the frequency of the word in prior spam and
> non-spam.

I expressed it badly; the point is that this process, involving many
simple criteria, is in itself a complicated one.
--
Peter
 >> Stay informed about: Junk Filter Question 

In article ,
peter RemoveThis @cara.demon.co.uk (Peter Ceresole) wrote:

> Bill Cole wrote:
>
> > > The whole point of heuristic learning (which I believe the spam filters
> > > use) is that they derive their own rules from the examples you give
> > > them. These rules can be very complicated. This avoids human
> > > programming, which is the source of so many errors.
> >
> > That's not really how most training-based spam filters work. Most are
> > based on a system known as "Naive Bayesian" classification. That
> > approach isn't based on complex rules, but rather on a lot of very
> > simple rules with scores derived statistically. The words in the message
> > are tallied to a big list of word frequencies in spam and non-spam
> > messages, and that list of relative frequencies is used as a lot of very
> > simple scoring rules, with each rule testing the presence of a word and
> > adding a score based on the frequency of the word in prior spam and
> > non-spam.
>
> I expressed it badly; the point is that this process, involving many
> simple criteria, is in itself a complicated one.

Well, I guess so...

But only if you consider probabilistic analysis complicated.

<g,d,&r>

--
Now where did I hide that website...
 >> Stay informed about: Junk Filter Question