Help-Site Computer Manuals
Software
Hardware
Programming
Networking
  Algorithms & Data Structures   Programming Languages   Revision Control
  Protocols
  Cameras   Computers   Displays   Keyboards & Mice   Motherboards   Networking   Printers & Scanners   Storage
  Windows   Linux & Unix   Mac

SyslogScan::Daemon::SpamDetector::BlockList
maintain an IP-based blocklist of spam sources

SyslogScan::Daemon::SpamDetector::BlockList - maintain an IP-based blocklist of spam sources


NAME


 SyslogScan::Daemon::SpamDetector::BlockList - maintain an IP-based blocklist of spam sources


SYNOPSIS


 plugin SyslogScan::Daemon::SpamDetector as sd_

 sd_plugin SyslogScan::Daemon::SpamDetector::BlockList

        debug           3

        track_dbi_dsn   'DBI:mysql:database=mailstats;host=localhost'

        track_dbi_user  'username'

        track_dbi_pass  'passwd'

        block_dbi_dsn   'DBI:mysql:database=iimaildb;host=blockhost'

        block_dbi_user  'username'

        block_dbi_pass  'passwd'

        minblock        0.0416666666666667      # minimum block length (days)

        maxblock        1.0     # maximum block length (days)

        blockmult       1.2     # amount to grow/shrink the block

        shrinktime      0.25    # grow the block length if spammed again in less than this (days)

        growtime        0.04    # amount of times w/o a spam to shrink block length (days)

        firstblock      10      # number of spams required to trigger first blocking

        cleanup         86400   # clean up tables, remove blocks (seconds)

        blockmemory     8       # keep history of previous blocks (days)

        spammemory      8       # keep history of spams sent (days)

        recipweight     .33333333333333  # how much does an extra recipient count towards being enough spam to block

        blockcommand    ''

        repeatblocks    50      # times blocked to get worse treatment

        repeatoffender  250     # count of spams to get worse treatment

        rpmult          4       # how much more/less time repeat offenders get to grow shrink their block time

        fastspammer     20      # number of spams sent while blocked to get penealty

        fastspammermult 1.5     # blocktime multipe for fastspammers


DESCRIPTION

Track and react to the spam noticed by the SyslogScan::Daemon::SpamDetector manpage.

Track the spam in an SQL database. Build a blocklist.

Block hosts for a bit. If they keep spamming, block them for longer periods.

We reccomend that this module not be combined with the SyslogScan::Daemon::SpamDetector::SpamAssassin manpage becuase it will end up blocking forwarded mail.


CONFIGURATION PARAMETERS

The following configuration parameters are supported:

debug
Debugging 0 (off) to 5 (verbose).

minblock
The minimum amount of time to block an IP address in days. (Default: about an hour)

maxblock
The maximum amount of time to block an IP address in days. (Default: 1)

blockmult
If a site is being re-blocked within the growtime window, multiply the block time by this amount. If a site is being re-blocked outside of the shrinktime window then divide the block time by this amount. (Default: 1.2)

growtime
If a site sends more spam and thus needs re-blocking, punish with a longer block if it's needing re-blocking before this amount of time has passed. In days. (Default: about one hour)

shrinktime
If a site sends more spam and thus needs re-blocking but it has been longer than this amount of time since it was last blocked, then give it a shorter block than it had before. In days. (Default: .25)

firstblock
Don't block a site until this many spam have been recevied. (Default: 10)

block_dbi_dsn =item block_dbi_user =item block_dbi_pass
Use this database for the IP block list. The table is small.

track_dbi_dsn =item track_dbi_user =item track_dbi_pass
Use this database for tracking incoming spam. The table is large.

cleanup
Perform cleanup of old on the database this often. Seconds. (Default: 43200 - 12 hours)

blockmemory
Remember inactive blocks for this long. Days. (Default: 4).

spammemory
Remember incoming spam for this long. Days. (Default: 10).

blockcommand
Run this command when a block is created. (No default)

recipweight
When the same message is sent to multiple recipients, count the subsequent deliveries as this much of a spam. (Default: .333333333333)

repeatblocks
When an IP address has been been blocked this man times treat it specially: instead of multiplying block times by blockmult, multiply them by rpmult.

repeatoffender
When an IP address has sent this many spams, treat it specially: instead of multiplying block times by blockmult, multiply them by rpmult.

rpmult
For espeically bad sources, instead of multiplying block times by blockmult, multiply them by rpmult. (Default: 4)

fastspammer
If this many spams are sent while being blocked (presumably because the block doesn't cover the entire network), then multiply the block time by fastspammermult.

fastspammermult
Block multiplier penealty for fast spammers. (Default: 1.5)

cleanupfirst
Clean out old entries from the database tables at startup? (Default: 1)


TABLE CREATION

SQL tables are not automatically created. Here are examples for Mysql5:


 CREATE TABLE spam_received (

        ip              VARCHAR(16),

        spamtime        DATETIME,

        messageid       TEXT,

        recipients      INT,

        host            TEXT,

        score           TEXT,

        matched         TEXT,

        INDEX           ip_index(ip)

 ) TYPE=MyISAM;

 CREATE TABLE block_history (

        ip              VARCHAR(16),

        blockstart      DATETIME,

        blockend        DATETIME,

        blockcount      INT,

        spamcount       INT,

        messageid       TEXT,

        countdays       INT,

        PRIMARY KEY     (ip),

        INDEX           done(blockend)

 ) TYPE=MyISAM;

Replace ``Los Angeles'' with a city in your time zone.


 drop view blocking;

 CREATE VIEW blocking AS

 SELECT ip, 

        CONCAT("450 blocked for ", 

        SUBTIME(TIMEDIFF(blockend,blockstart),

                SEC_TO_TIME(SECOND(TIMEDIFF(blockend,blockstart)))),

        " until ", ADDTIME(blockend, timediff(now(), utc_timestamp())),

        "-Los Angeles time due to ", spamcount, " spams in ", countdays, 

        " days, for example <", messageid, ">") AS message,

        SUBTIME(TIMEDIFF(blockend,blockstart),

                SEC_TO_TIME(SECOND(TIMEDIFF(blockend,blockstart)))) AS blocktime,

        case when blockend > utc_timestamp() then 1 else 0 end AS active,

        ADDTIME(blockend, timediff(now(), utc_timestamp())) as blockend_local,

        spamcount

 FROM   block_history;


USEFUL QUERIES

To watch what's going on with the block list, the following queries are helpful.


 select ip, COUNT(*), SUM(recipients) from spam_received 

 group by ip order by COUNT(*) desc limit 25;

 select COUNT(*), SUM(recipients) from spam_received;

 select active, count(*), AVG(blocktime), AVG(spamcount), MAX(spamcount)

 from blocking group by active;

 select ip, message from blocking where active = 1 order by spamcount;

 select spamcount, COUNT(*), AVG(blocktime), AVG(spamcount)

 from blocking where active = 1 group by spamcount order by spamcount;

 select blocktime, count(*), MIN(spamcount), AVG(spamcount), MAX(spamcount) 

 from blocking where active = 1 group by blocktime order by blocktime;

 select ip, blocktime, spamcount, active

 from blocking 

 order by spamcount desc

 limit 25;

 select count(*), SUM(active) from blocking where spamcount > 250;

 select truncate(spamcount,-1), COUNT(*), AVG(blocktime), AVG(spamcount)

 from blocking where active = 1 group by truncate(spamcount,-1) order by truncate(spamcount,-1);


POSTFIX CONFIG

To configure Postfix to block using this block list, include a line like:


 smtpd_client_restrictions = check_client_access mysql:/etc/postfix/mysql_spam_senders.cf

Then include a query in the referenced file:


 hosts = localhost maildb

 dbname = spamdb

 user = dbuser

 password = dbpass

 query = SELECT message FROM blocking WHERE ip = '%s' AND active = 1

If your postfix doesn't have SQL support or you're using a less capable mailer, you can dump the block list into a file:


 mysql -N -B --host=mx --database=spamdb --user=dbuser \

 -e 'select ip, message from blocking where active = 1' \

 > file


DNS BLOCKLIST

One way to turn this blocklist into a DNS-based blocklist that can be used like other blocklists is to use rbldnsd(8).

Set up a the SyslogScan::Daemon::Periodic manpage job to rebuild the blocklist:


 plugin SyslogScan::Daemon::Periodic 

        debug   1

        period  120

        command '/usr/local/bin/update_blocklist'

The update_blocklist script I use is:


 #!/bin/sh -e

 SUB=1171308796

 SECONDS=`date +%s`

 SEQN=`expr $SECONDS - $SUB`

 DIR="/var/lib/rbldns"

 ZONE="blocked"

 sed "s/SERIAL/$SEQN/" < $DIR/$ZONE.head > $DIR/$ZONE.new

 mysql -N -B --host=MYDBHOST --database=MYDB --user=MYDBUSER \

   -e 'select   ip, concat(":1:",message) \

       from     blocking where active = 1' \

   | sed 's/:450 /:/' \

   >> $DIR/$ZONE.new

 mv $DIR/$ZONE.new $DIR/$ZONE

The /var/lib/rbldns/blocked.head file I use:


 $SOA 1800 ns.idiom.com. muir.idiom.com. SERIAL 300 60 3600 300

 $NS 1800 MYHOSTNAME

 $TTL 120

I start rbldnsd(8) with:


 /usr/sbin/rbldnsd -p /var/run/rbldnsd.pid \

  -r/var/lib/rbldns -4 -bMYIPADDRESS/53 \

  spammers.MYDOMAIN:ip4set:blocked

Then, to use it with postfix(1), I set up a rbl_reply_maps so that I can give a temporary failure.


 smtpd_client_restrictions =

  permit_mynetworks,

  reject_unauth_pipelining,

  reject_rbl_client cbl.abuseat.org,

  reject_rbl_client spammers.MYDOMAIN

 rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps

My rbl_rply_maps file:


 cbl.abuseat.org   $rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason}

 spammers.MYDOMAIN 450 4.7.1 Service unavailable; $rbl_class [$rbl_what] $rbl_reason


SEE ALSO

the SyslogScan::Daemon::SpamDetector manpage


THANK THE AUTHOR

If you need high-speed internet services (T1, T3, OC3 etc), please send me your request-for-quote. I have access to very good pricing: you'll save money and get a great service.


LICENSE

Copyright(C) 2006 David Muir Sharnoff <muir@idiom.com>. This module may be used and distributed on the same terms as Perl itself.

Programminig
Wy
Wy
yW
Wy
Programming
Wy
Wy
Wy
Wy