This presentation was given at Python North West. It explains a complete Kamaelia application for greylisting which was written specifically to eliminate my personal spam problem. It walks through the code as well (though that's best looked at with the code side by side!)
Scale your database traffic with Read & Write split using MySQL Router
Kamaelia Grey
1. Relevant disclosure? - Kamaelia came out of my day work @ BBC Research - Kamaelia grey came out of my personal frustration, and has nothing to do with the BBC :-) Michael Sparks , [email_address] Kamaelia Grey
2. Drastic Spam reduction through Greylisting Greylisting made easy through natural concurrency Natural concurrency made easy through Kamaelia ... since that's what Kamaelia is for Kamaelia Grey
26. Logging? Debug log: /var/log/greylist.log 250 OK 250 ACCEPTED 451 4.7.1 Please try again later 220 mail.cerenity.org ESMTP Kamaelia-SMTP 1.0 Tue Nov 6 17:26:01 2007 500 Command Not Recognised 250 mail.cerenity.org Hello lse 89.252.24.7 250 OK Note – this is what's being sent over SMTP
27. But also... Debug log: /var/log/greylist-debug.log *debug* THREADS['Kamaelia.Chassis.Pipeline.Pipeline_7', 'Kamaelia.Internet.Selector.Selector_11', '__main__.GreylistServer_8', '__main__.PeriodicWakeup_5', '__main__.TCPS_10', '__main__.WakeableIntrospector_6'] Note – this says the current internal components running
28. Internals Debug log: /var/log/greylist-debug.log ['Kamaelia.Chassis.Pipeline.Pipeline_7', 'Kamaelia.Internet.Selector.Selector_11', '__main__.GreylistServer_8', '__main__.PeriodicWakeup_5', '__main__.TCPS_10', '__main__.WakeableIntrospector_6'] Note – this says the current internal components running
29. Internals Debug log: /var/log/greylist-debug.log Kamaelia.Internet.Selector.Selector_11 – wakes system on network events __main__.TCPS_10 – sits inside and handles listening for '__main__.GreylistServer_8 – A configured network server Kamaelia.Chassis.Pipeline.Pipeline_7 – For debugging '__main__.PeriodicWakeup_5 – For debugging '__main__.WakeableIntrospector_6 – For debugging Note – this says the current internal components running
36. Internals The following pages are intended as a walk through of the key highlights of the code-base. It skips the body of the code since that's best looked at by scrolling through code and chatting rather than dumping into slides. In a presentation its easy to say this, and then after going through slides giving road pointers go through the code. Presentation files are a little more limited. The code referred to is here: http://tinyurl.com/2sbjxl
39. Internals class ConcreteMailHandler(MailHandler): Inboxes = { "inbox" : "Data from the client connecting to the server comes in here", "control" : "Shutdown & control messages regarding client side socket handling", "tcp_inbox" : "This is where we get respones from the real SMTP server", "tcp_control" : "This is where we get shutdown information from the real SMTP server", } Outboxes = { "outbox" : "Data sent here goes back the the client connecting to the server", "signal" : "Shutdown & control messages regarding client side socket handling", "tcp_outbox" : "Data sent here is sent to the real SMTP server", "tcp_signal" : "We send messages here to shutdown the connection to the real SMTP connection", } peer = "*** UNDEFINED ***" peerport = "*** UNDEFINED ***" local = "*** UNDEFINED ***" localport = "*** UNDEFINED ***" servername = "Testing.server.local" serverid = "MPS SMTP 1.0" smtp_ip = "192.168.2.9" smtp_port = 25
45. And that... Saves me having to wade through now literally tens of thousands of spams over the past two months: # grep DEFERRED /var/log/greylist.log |wc -l 73798
46. Time to write? 2 days start to finish including protocol Bug fixes ~1-2 days tops