[Spambayes] Frequency distribution for wordinfo counts?

Tony Meyer tameyer at ihug.co.nz
Sun Feb 15 18:57:49 EST 2004


[Thanks for the code Tim!]

Here's mine today; I'm using bigrams and 'url slurping' with a prefix.

nham 99 nspam 220
ham counts

value  #times   cumm      %   cumm%
     0  61854  61854  48.58  48.58
     1  54466 116320  42.78  91.36
     2   6172 122492   4.85  96.20
     3   1702 124194   1.34  97.54
     4   1143 125337   0.90  98.44
     5    485 125822   0.38  98.82
     6    310 126132   0.24  99.06
     7    239 126371   0.19  99.25
     8    185 126556   0.15  99.40
     9    127 126683   0.10  99.50
    10    111 126794   0.09  99.58
    11     77 126871   0.06  99.64
    12     59 126930   0.05  99.69
    13     48 126978   0.04  99.73
    14     40 127018   0.03  99.76
    15     35 127053   0.03  99.79
    16     26 127079   0.02  99.81
    17     14 127093   0.01  99.82
    18     28 127121   0.02  99.84
    19     23 127144   0.02  99.86
    20     17 127161   0.01  99.87
    21     13 127174   0.01  99.88
    22     13 127187   0.01  99.89
    23      7 127194   0.01  99.90
    24     11 127205   0.01  99.91
    25      8 127213   0.01  99.91
    26     11 127224   0.01  99.92
    27      9 127233   0.01  99.93
    28      9 127242   0.01  99.93
    29      7 127249   0.01  99.94
    30      6 127255   0.00  99.95
    31      6 127261   0.00  99.95
    32      5 127266   0.00  99.95
    33      4 127270   0.00  99.96
    34      3 127273   0.00  99.96
    35      2 127275   0.00  99.96
    36      2 127277   0.00  99.96
    37      2 127279   0.00  99.96
    38      5 127284   0.00  99.97
    40      1 127285   0.00  99.97
    42      1 127286   0.00  99.97
    43      4 127290   0.00  99.97
    44      1 127291   0.00  99.97
    45      2 127293   0.00  99.97
    46      1 127294   0.00  99.98
    48      1 127295   0.00  99.98
    50      1 127296   0.00  99.98
    51      1 127297   0.00  99.98
    52      1 127298   0.00  99.98
    55      1 127299   0.00  99.98
    60      2 127301   0.00  99.98
    61      1 127302   0.00  99.98
    63      1 127303   0.00  99.98
    64      4 127307   0.00  99.99
    65      1 127308   0.00  99.99
    66      1 127309   0.00  99.99
    71      2 127311   0.00  99.99
    74      1 127312   0.00  99.99
    76      1 127313   0.00  99.99
    77      1 127314   0.00  99.99
    78      1 127315   0.00  99.99
    87      1 127316   0.00  99.99
    89      3 127319   0.00 100.00
    97      3 127322   0.00 100.00
    99      3 127325   0.00 100.00

spam counts
value  #times   cumm      %   cumm%
     0  54754  54754  43.00  43.00
     1  61662 116416  48.43  91.43
     2   5367 121783   4.22  95.65
     3   2012 123795   1.58  97.23
     4    950 124745   0.75  97.97
     5    635 125380   0.50  98.47
     6    323 125703   0.25  98.73
     7    199 125902   0.16  98.88
     8    160 126062   0.13  99.01
     9    109 126171   0.09  99.09
    10     81 126252   0.06  99.16
    11    282 126534   0.22  99.38
    12    107 126641   0.08  99.46
    13     71 126712   0.06  99.52
    14     51 126763   0.04  99.56
    15     49 126812   0.04  99.60
    16     26 126838   0.02  99.62
    17     34 126872   0.03  99.64
    18     37 126909   0.03  99.67
    19     26 126935   0.02  99.69
    20     30 126965   0.02  99.72
    21     22 126987   0.02  99.73
    22     20 127007   0.02  99.75
    23     17 127024   0.01  99.76
    24     17 127041   0.01  99.78
    25     13 127054   0.01  99.79
    26     15 127069   0.01  99.80
    27     14 127083   0.01  99.81
    28     14 127097   0.01  99.82
    29     14 127111   0.01  99.83
    30     10 127121   0.01  99.84
    31      8 127129   0.01  99.85
    32      8 127137   0.01  99.85
    33      5 127142   0.00  99.86
    34     10 127152   0.01  99.86
    35      6 127158   0.00  99.87
    36     13 127171   0.01  99.88
    37      4 127175   0.00  99.88
    38      8 127183   0.01  99.89
    39      7 127190   0.01  99.89
    40      6 127196   0.00  99.90
    41      3 127199   0.00  99.90
    42     12 127211   0.01  99.91
    43      1 127212   0.00  99.91
    44      7 127219   0.01  99.92
    45      2 127221   0.00  99.92
    46      7 127228   0.01  99.92
    48      3 127231   0.00  99.93
    49      4 127235   0.00  99.93
    50      1 127236   0.00  99.93
    51      2 127238   0.00  99.93
    52      4 127242   0.00  99.93
    53      2 127244   0.00  99.94
    54      3 127247   0.00  99.94
    55      1 127248   0.00  99.94
    56      3 127251   0.00  99.94
    57      3 127254   0.00  99.94
    58      1 127255   0.00  99.95
    59      2 127257   0.00  99.95
    60      1 127258   0.00  99.95
    62      2 127260   0.00  99.95
    67      2 127262   0.00  99.95
    68      2 127264   0.00  99.95
    70      1 127265   0.00  99.95
    72      1 127266   0.00  99.95
    73      2 127268   0.00  99.96
    76      1 127269   0.00  99.96
    77      1 127270   0.00  99.96
    78      2 127272   0.00  99.96
    81      4 127276   0.00  99.96
    88      2 127278   0.00  99.96
    89      4 127282   0.00  99.97
    90      1 127283   0.00  99.97
    91      1 127284   0.00  99.97
    92      1 127285   0.00  99.97
    93      1 127286   0.00  99.97
    96      1 127287   0.00  99.97
   100      2 127289   0.00  99.97
   101      2 127291   0.00  99.97
   102      1 127292   0.00  99.97
   104      1 127293   0.00  99.97
   112      1 127294   0.00  99.98
   113      1 127295   0.00  99.98
   114      1 127296   0.00  99.98
   115      1 127297   0.00  99.98
   118      1 127298   0.00  99.98
   122      1 127299   0.00  99.98
   125      1 127300   0.00  99.98
   129      1 127301   0.00  99.98
   132      1 127302   0.00  99.98
   146      1 127303   0.00  99.98
   152      1 127304   0.00  99.98
   156      1 127305   0.00  99.98
   161      1 127306   0.00  99.99
   168      2 127308   0.00  99.99
   169      2 127310   0.00  99.99
   175      1 127311   0.00  99.99
   177      1 127312   0.00  99.99
   178      1 127313   0.00  99.99
   181      1 127314   0.00  99.99
   192      1 127315   0.00  99.99
   201      1 127316   0.00  99.99
   203      1 127317   0.00  99.99
   204      1 127318   0.00  99.99
   207      2 127320   0.00 100.00
   215      2 127322   0.00 100.00
   217      1 127323   0.00 100.00
   218      1 127324   0.00 100.00
   220      1 127325   0.00 100.00

Database is Berkeley, disk size 10,338,304 bytes.  Like Tim, the plain-text
db export file is a third that size (3,462,853 bytes).

=Tony Meyer




More information about the Spambayes mailing list