[spambayes-dev] Mine_received_headers
Tony Meyer
ta-meyer at ihug.co.nz
Wed Jan 7 19:10:25 EST 2004
While I was testing, I figured I should do this one, too. This gives me
terrible results! I haven't tested with the old ('reverse') method, so I
don't know if that would have made a difference or not.
Without bigrams:
-> <stat> tested 357 hams & 395 spams against 3311 hams & 3704 spams
[all other stat lines in the message snipped - they're all basically this]
false positive percentages
0.000 0.000 tied
0.000 0.000 tied
0.000 0.000 tied
0.246 0.246 tied
0.000 0.000 tied
0.000 0.000 tied
0.557 0.557 tied
0.559 0.279 won -50.09%
0.287 0.287 tied
0.000 0.000 tied
won 1 times
tied 9 times
lost 0 times
total unique fp went from 6 to 5 won -16.67%
mean fp % went from 0.164881884948 to 0.136948924055 won -16.94%
false negative percentages
0.253 0.506 lost +100.00%
0.781 1.042 lost +33.42%
0.462 1.155 lost +150.00%
0.756 0.756 tied
0.243 0.243 tied
0.247 0.494 lost +100.00%
0.240 1.202 lost +400.83%
0.494 1.235 lost +150.00%
0.973 1.217 lost +25.08%
0.454 0.680 lost +49.78%
won 0 times
tied 2 times
lost 8 times
total unique fn went from 20 to 35 lost +75.00%
mean fn % went from 0.490257037938 to 0.852825140424 lost +73.95%
ham mean ham sdev
1.18 0.93 -21.19% 7.76 7.22 -6.96%
0.99 0.91 -8.08% 6.64 6.88 +3.61%
0.84 0.61 -27.38% 6.14 5.32 -13.36%
1.99 2.05 +3.02% 9.46 9.97 +5.39%
0.49 0.44 -10.20% 3.59 3.43 -4.46%
0.85 0.89 +4.71% 5.45 6.08 +11.56%
1.16 1.28 +10.34% 9.30 9.84 +5.81%
1.20 0.99 -17.50% 8.13 7.13 -12.30%
1.55 1.72 +10.97% 8.05 9.18 +14.04%
0.47 0.43 -8.51% 3.22 3.53 +9.63%
ham mean and sdev for all runs
1.08 1.03 -4.63% 7.13 7.26 +1.82%
spam mean spam sdev
98.75 98.79 +0.04% 8.72 8.79 +0.80%
97.67 96.87 -0.82% 11.26 13.66 +21.31%
98.08 96.97 -1.13% 10.12 13.46 +33.00%
98.16 97.75 -0.42% 10.19 11.56 +13.44%
98.35 98.28 -0.07% 8.77 9.05 +3.19%
98.45 98.10 -0.36% 8.97 10.13 +12.93%
98.35 97.70 -0.66% 9.73 12.49 +28.37%
98.25 97.91 -0.35% 9.16 11.41 +24.56%
97.93 97.17 -0.78% 11.99 13.97 +16.51%
98.92 98.66 -0.26% 7.62 9.34 +22.57%
spam mean and sdev for all runs
98.30 97.82 -0.49% 9.72 11.55 +18.83%
ham/spam mean difference: 97.22 96.79 -0.43
With bigrams:
false positive percentages
0.000 0.000 tied
0.000 0.000 tied
0.000 0.000 tied
0.000 0.246 lost +(was 0)
0.000 0.000 tied
0.000 0.000 tied
0.279 0.279 tied
0.000 0.000 tied
0.000 0.287 lost +(was 0)
0.000 0.000 tied
won 0 times
tied 8 times
lost 2 times
total unique fp went from 1 to 3 lost +200.00%
mean fp % went from 0.0278551532033 to 0.0811608099572 lost +191.37%
false negative percentages
0.253 0.506 lost +100.00%
1.042 1.302 lost +24.95%
0.693 1.155 lost +66.67%
0.252 0.504 lost +100.00%
0.728 0.243 won -66.62%
0.000 0.247 lost +(was 0)
0.481 0.962 lost +100.00%
0.494 1.235 lost +150.00%
0.730 1.217 lost +66.71%
0.227 0.227 tied
won 1 times
tied 1 times
lost 8 times
total unique fn went from 20 to 31 lost +55.00%
mean fn % went from 0.489899714703 to 0.759596596728 lost +55.05%
ham mean ham sdev
0.95 0.81 -14.74% 6.64 6.30 -5.12%
0.83 0.75 -9.64% 5.53 5.65 +2.17%
0.49 0.36 -26.53% 4.08 3.74 -8.33%
1.53 1.62 +5.88% 8.16 8.47 +3.80%
0.30 0.29 -3.33% 3.25 3.01 -7.38%
0.70 0.64 -8.57% 5.27 5.09 -3.42%
0.85 0.84 -1.18% 7.11 7.25 +1.97%
0.93 0.68 -26.88% 7.23 5.83 -19.36%
0.90 1.04 +15.56% 6.47 7.26 +12.21%
0.41 0.38 -7.32% 4.07 4.00 -1.72%
ham mean and sdev for all runs
0.80 0.75 -6.25% 6.01 5.93 -1.33%
spam mean spam sdev
98.71 98.67 -0.04% 7.83 8.75 +11.75%
97.38 96.86 -0.53% 12.55 14.27 +13.71%
97.78 96.99 -0.81% 11.09 13.46 +21.37%
97.89 97.61 -0.29% 10.49 11.33 +8.01%
97.90 97.78 -0.12% 10.03 10.13 +1.00%
98.32 97.83 -0.50% 8.63 10.20 +18.19%
98.19 97.89 -0.31% 10.21 11.44 +12.05%
97.68 97.61 -0.07% 10.99 11.94 +8.64%
97.86 97.13 -0.75% 11.56 13.79 +19.29%
98.73 98.71 -0.02% 7.57 8.14 +7.53%
spam mean and sdev for all runs
98.05 97.71 -0.35% 10.20 11.51 +12.84%
ham/spam mean difference: 97.25 96.96 -0.29
Table:
filename: bases mine_recs basebis mine_rec_bis
ham:spam: 3668:4099 3668:4099 3668:4099 3668:4099
fp total: 6 5 1 3
fp %: 0.16 0.14 0.03 0.08
fn total: 20 35 20 31
fn %: 0.49 0.85 0.49 0.76
unsure t: 178 198 207 205
unsure %: 2.29 2.55 2.67 2.64
real cost: $115.60 $124.60 $71.40 $102.00
best cost: $93.00 $119.80 $65.60 $78.20
h mean: 1.08 1.03 0.80 0.75
h sdev: 7.13 7.26 6.01 5.93
s mean: 98.30 97.82 98.05 97.71
s sdev: 9.72 11.55 10.20 11.51
mean diff: 97.22 96.79 97.25 96.96
k: 5.77 5.15 6.00 5.56
=Tony Meyer
More information about the spambayes-dev
mailing list