Ian Ozsvald picture

This is Ian Ozsvald's blog (@IanOzsvald), I'm an entrepreneurial geek, a Data Science/ML/NLP/AI consultant, author of O'Reilly's High Performance Python book, co-organiser of PyDataLondon, a Pythonista, co-founder of ShowMeDo and also a Londoner. Here's a little more about me.

High Performance Python book with O'Reilly

View Ian Ozsvald's profile on LinkedIn

ModelInsight Data Science Consultancy London Protecting your bits. Open Rights Group


22 December 2006 - 14:28Spam giving rise to new-breed A.I.?

It is Christmas and I exercise my right to wave my hands in the air, tell a story and make a bold prediction. You may exercise your right to comment and tell me just what you think of my idea.

During this year, especially whilst growing ShowMeDo, we’ve had to contend with a rise in spam. We get spam on our site (each video has a comments section, like comments on a blog), spam on our blog and spam in our forums. Dealing with spam eats hours, and when you only have hours to grow the site each evening this quickly gets boring.

Sadly, I see this only getting worse as time goes on, at least until technology catches up.

This year we’ve witnessed the rise of a new type of image spam – frequently used for delivering pump n’dump messages (which fool enough people to net the spammers a healthy profit). These images fool OCR spam-filtering tools, and this is likely to only get worse.

The two types of spam that frequently get through to my inbox (and my personal inbox via Yahoo) include clever use of text-phrases in text-only mails, and this new type of image-based spam (with random visuals in the background which fool the OCR software).

Humans are good are understanding both types of spam, but computers are bad so the filtering begins to break down. Spamming is easy, computers are relatively badly defended, so delivering torrents of spam is cheap…yet the rewards are high for the spammer. Economics drives the entire process, and more spam is the likely result.

Add to this new initiatives like the One Laptop Per Child project, which aims to deliver very cheap laptops to the third world. One obvious form of employment for unscrupulous individuals will be to write text and image based spam which gets past our current spam-filtering technology. Given that the OLPC is network-enabled (via built-in WiFi), it’ll be relatively easy and cheap to hook up cheap-human-brains to the spam-generation system.

How can we stop this new breed of better-crafted spam? We can’t employ people to act as human filters, that’ll get too expensive too quickly. Instead we’ll need to improve current A.I. techniques in the field of Image Processing and Natural Language Processing.

So, here’s my Christmas prediction. By next Christmas we’ll see some big advances in the application of Artificial Intelligence to the problem of spam-detection, in addition to the statistical and rule-based methods currently used. This will have been driven by improved spamming techniques, often centred around the use of cheap human labour to supplement the current algorithm-based approaches.

I’d be curious to know what you think…

6 Comments | Tags: ArtificialIntelligence, Life, ShowMeDo

22 December 2006 - 14:10Christmas wishes

Right, no posts for absolutely ages…that’s because our ShowMeDo.com is growing really rather nicely. So well in fact that as of January, backed by a bank-loan, I’ll be working 50% time on growing the site, attracting more authors and adding in a few sensible revenue ideas to make the company grow in a sustainable way.

Back when we started in December 2005, we weren’t sure how things would go. Who’d have thought we would have close to 130,000 tutorial downloads within 12 months, with 108 videos and over 20 authors contributing to the site – all whilst growing the site in evenings and weekends?

There is lots to do and plenty more fun to be had. Kyran and I have had a blast growing things this year, now we have to get serious about taking it all to the next level. Expect even less personal posting on this blog, and far more over at the ShowMeDo blog!

Christmas wishes to everyone 🙂

No Comments | Tags: Life