Entrepreneurial Geekiness

It is Christmas and I exercise my right to wave my hands in the air, tell a story and make a bold prediction. You may exercise your right to comment and tell me just what you think of my idea.

During this year, especially whilst growing ShowMeDo, we’ve had to contend with a rise in spam. We get spam on our site (each video has a comments section, like comments on a blog), spam on our blog and spam in our forums. Dealing with spam eats hours, and when you only have hours to grow the site each evening this quickly gets boring.

Sadly, I see this only getting worse as time goes on, at least until technology catches up.

This year we’ve witnessed the rise of a new type of image spam – frequently used for delivering pump n’dump messages (which fool enough people to net the spammers a healthy profit). These images fool OCR spam-filtering tools, and this is likely to only get worse.

The two types of spam that frequently get through to my inbox (and my personal inbox via Yahoo) include clever use of text-phrases in text-only mails, and this new type of image-based spam (with random visuals in the background which fool the OCR software).

Humans are good are understanding both types of spam, but computers are bad so the filtering begins to break down. Spamming is easy, computers are relatively badly defended, so delivering torrents of spam is cheap…yet the rewards are high for the spammer. Economics drives the entire process, and more spam is the likely result.

Add to this new initiatives like the One Laptop Per Child project, which aims to deliver very cheap laptops to the third world. One obvious form of employment for unscrupulous individuals will be to write text and image based spam which gets past our current spam-filtering technology. Given that the OLPC is network-enabled (via built-in WiFi), it’ll be relatively easy and cheap to hook up cheap-human-brains to the spam-generation system.

How can we stop this new breed of better-crafted spam? We can’t employ people to act as human filters, that’ll get too expensive too quickly. Instead we’ll need to improve current A.I. techniques in the field of Image Processing and Natural Language Processing.

So, here’s my Christmas prediction. By next Christmas we’ll see some big advances in the application of Artificial Intelligence to the problem of spam-detection, in addition to the statistical and rule-based methods currently used. This will have been driven by improved spamming techniques, often centred around the use of cheap human labour to supplement the current algorithm-based approaches.

I’d be curious to know what you think…