Two years back I posted an entry listing the science companies I knew around Brighton who are involved in high-tech software (i.e. not science companies who make physical products). The list has changed a bit with some nice additions so I’ve updated it below.  If you know of one that I’m missing do send me an update. I’m interested because I’m an A.I. researcher for industry by trade.

• PANalytical at SInC (one of my current employers for interesting A.I. work – I work on CUDA for parallelisation and pattern recognition and optimisation for solution finding, Prof. Paul Fewster is the head of the R&D team)
• Qtara (a new employer of mine creating a cutting-edge Intelligent Virtual Human)
• BrandWatch in the BrightonMediaCentre (a social metrics company using natural language processing)
• SecondLife in the North Laines (this office is a big part of their European presence)
• Ambiental at SInC (great flood-risk simulations and modelling, I help them with speeding up and improving the science behind their flood models, Justin Butler is the founder)
• Proneta at SInC (very small company, John Hother sometimes has A.I. related questions)
• Observatory Sciences at SInC (Philip Taylor is the main chap here, they use EPICS and LabView)
• Ricardo in Shoreham (a big engineering consultancy)
• Elektro Magnetix at SInC
• NeuroRobotics at SInC
• MindLab at SInC (they do non-invasive brain monitoring)
• Animazoo in Shoreham (they build motion-capture suits for dancers and actors)
• BotBuilder in Brighton (a robot focused design and build company)

Another nice addition to Brighton is the BrightonHackerSpace, a collective of like-minded souls who build new electronic devices and pull things apart to understand how they work. This HackerSpace has spawned BotBuilder (above) and I’m looking forward to seeing a few more created.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

We’re plotting our 14th £5 App meet.  This, our second Christmas Special, will have a gamesy happy crimbo feel.  Picture (from a few months back) kudos to Josh:

Date: Wednesday 10th December, sign-up on Upcoming please.  Location The Werks (Hove nr Palmeira Sq) – note their piggy bank for a second projector at the end of this post please.

Our very own Aleks Krotoski will lead the evening with the launch of the Guardian’s new on-line text adventure SpaceShip!

Following Aleks’ main talk we’ll have a set of shorter 10 minute demos:

1. Lightsaber mobile phone duelling by Marko (Lastminute.com Labs)
2. Fighting Mini-sumo robots by Emily
3. In-development 3D iPhone game + backstory by Dominic Mason
4. Flash 3D Snow and Xmas games by Seb (PluginMedia)
5. Eye-controlled Pong by Ben Rubinstein (CogApp)

You’ll meet lots of local developers, freelancers and business founders including people of Farm Brighton, Girl Geeks, Sussex Innovation Centre, Inuda and The Skiff, The Werks, EuroGamer, BrandWatch, ClearLeft and Madgex.

The Ribots are sponsoring us with Festive Alcohol and (fingers crossed) Xmas Cakery.

Related – Seb’s Big Screen Bonanza Flash night is the day before ours, check it out for 200+ seatage Flash-demo crazyness.

Note – The Werks are looking for donations towards a second projector, do the right thing and support ‘em here:

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

Dr. Michael Thomas approached me with an interesting A.I. job to see if we could speed up his neural network code from a 10 year old research platform called PlaNet. Using new Sun boxes they weren’t getting the speed-ups they expected, old libs or other monkey business were suspected.

As a first investigation I took Neil Schemenauer’s bpnn.py (a 200 line back-prop artificial neural network library with doc and comparison). The intention was to see how much faster the code might run using psyco and shedskin.

The results were really quite surprising, notes and src follow.

Addition – Leonardo Maffi has written a companion piece showing that his ShedSkin output is 1.5 to 7* slower than hand-coded C.  He also shows solutions using the D language and runtimes for Python 2.6 (I use Python 2.5 below).  He notes:

“I have translated the Python code to D (using my D libraries) in just few minutes, something like 15-20 minutes, and the translation was mostly painless and sometimes almost mechanical. I have translated the D code to C in many hours. Translating Python => C may require something like 20-30 times the time you need to translate Python => D + my libs. And this despite I have used a rigorous enough method to perform the translation, and despite at the end I am not sure the C code is bug-free. This is an enormous difference.”

End addition.

Addition – Robert Bradshaw has created a Cython version with src, see comments. End addition.

The run-time in minutes for the my harder test case are below.  Note that these are averages of 4 runs each:

1. Vanilla Python 153 minutes
2. Python + Psyco 1.6.0.final.0 57 minutes (2.6* faster)
3. Shedskin 0.0.29 0.78 minutes [47 seconds] (196* faster)

The test machines uses Python 2.5.2 on Ubuntu 8.04. The box is an Intel Core Duo 2.4GHz running a single process.

The ‘hard’ problem trains the ANN using 508 patterns with 57 input neurons, 50 hidden and 62 output neurons over 1000 iterations. If you know ANNs then the configuration (0.1 learning rate, 0 momentum) might seem unusual, be assured that this is correct for my researcher’s problem.

There is a shorter version of this problem using just 2 patterns, this is useful if you want to replicate these results but don’t want to wait 3 hours on your first run.

My run times for the shorter problem are (again averaged using 4 runs):

1. Vanilla Python 42 seconds
2. Python + Psyco 14 seconds
3. Shedskin 0.2 seconds (210* faster)
   

Shedskin has an issue with numerical stability – it seems that internally some truncation occurs with floating point math. Whilst the results for vanilla Python and Python+Psyco were identical, the results with Shedskin were similar but with fractional divergences in each result.

Whilst these divergences caused some very different results in the final weights for the ANN, my researcher confirms that all the results look equivalent.

Mark Dufour (Shedskin’s author) confirms that Python’s C double is used the same in Shedskin but notes that rounding (or a bug) may be the culprit. Shedskin is a young project, Mark will welcome extra eyes if you want to look into this.

Running the code with Shedskin was fairly easy. On Ubuntu I had to install libgc-dev and libpcre3-dev (detailed in the Shedskin docs) and g++, afterwards shedskin was ready. From download to first run was 15 minutes.

On my first attempt to compile bpnn.py with Shedskin I received an error as the ‘raise’ keyword isn’t yet supported. I replaced the ‘raise’ calls with ‘assert False’ for sanity, afterwards compilation was fine.

Edit – Mark notes that the basic form of ‘raise’ is supported but the version used in bpnn.py isn’t yet supported.  Something like ‘raise ValueError(’some msg’)’ works fine.

Mark notes that Shedskin currently works well up to 500 lines (maybe up to 1000), since bpnn.py is only 200 lines compilation is quick.

Note that if you can’t use Psyco because you aren’t on x86, Shedskin might be useful to you since it’ll work anywhere that Python and g++ compile.

Running this yourself

If you want to recreate my results, download bpnn_shedskin_src_20081117.zip. You’ll see bpnn_shedskin.py, this is the main code. bpnn_shedskin.py includes either ‘examples_short.py’ or ‘examples_full.py’, short is the easier 2 pattern problem and full has 508 patterns.

Note that these patterns are stored as lists of tuples (Shedskin doesn’t support the csv module so I hardcoded the input patterns to speed development), the full version is over 500 lines of Python and this slows Shedskin’s compilation somewhat.

By default the imports for Psyco are commented out and the short problem is configured. At the command line you’ll get an output like this:

python bpnn_shedskin.py
Using 2 examples
ANN uses 57 input, 50 hidden, 62 output, 1000 iterations, 0.100000 learning rate, 0.000000 momentum
error 65.454309      2008-11-17 15:22:58.318593
error 45.176110      2008-11-17 15:22:59.060787
error 44.616933      2008-11-17 15:23:00.246280
error 44.026883      2008-11-17 15:23:01.743821
error 44.049276      2008-11-17 15:23:02.815876
error 44.905183      2008-11-17 15:23:03.860352
error 44.674506      2008-11-17 15:23:05.270307
error 43.365627      2008-11-17 15:23:06.757126
error 43.299160      2008-11-17 15:23:08.244466
error 42.540076      2008-11-17 15:23:09.732035
Elapsed: 0:00:41.472192

If you uncomment the two Psyco lines your code will run about 2.6* faster.

Using Shedskin

To use shedskin, first run the Python through shedskin and then ‘make’ the result. The compiled binary will run much faster than the vanilla Python code, the result below shows the short problem taking 0.19 seconds compared to 41 seconds above.

shedskin bpnn_shedskin.py
*** SHED SKIN Python-to-C++ Compiler 0.0.29 ***
Copyright 2005-2008 Mark Dufour; License GNU GPL version 3 (See LICENSE)
[iterative type analysis..]
***
iterations: 3 templates: 519
[generating c++ code..]
*WARNING* bpnn_shedskin.py:178: function (class NN, 'weights') not called!
*WARNING* bpnn_shedskin.py:156: function (class NN, 'test') not called!

make
g++  -O2 -pipe -Wno-deprecated  -I. -I/usr/lib/shedskin/lib /usr/lib/shedskin/lib/string.cpp /usr/lib/shedskin/lib/random.cpp /usr/lib/shedskin/lib/datetime.cpp examples_short.cpp bpnn_shedskin.cpp /usr/lib/shedskin/lib/builtin.cpp /usr/lib/shedskin/lib/time.cpp /usr/lib/shedskin/lib/math.cpp -lgc  -o bpnn_shedskin

./bpnn_shedskin
Using 2 examples
ANN uses 57 input, 50 hidden, 62 output, 1000 iterations, 0.100000 learning rate, 0.000000 momentum
error 65.454309      2008-11-17 16:11:08.452087
error 44.970416      2008-11-17 16:11:08.476869
error 46.444249      2008-11-17 16:11:08.506324
error 44.209054      2008-11-17 16:11:08.519375
error 44.058518      2008-11-17 16:11:08.532430
error 45.655892      2008-11-17 16:11:08.545741
error 44.518816      2008-11-17 16:11:08.558520
error 43.643572      2008-11-17 16:11:08.571705
error 44.800429      2008-11-17 16:11:08.584241
error 43.710905      2008-11-17 16:11:08.597465
Elapsed: 0:00:00.198747

Why is the math different?

An open question remains as to why the evolution of the floating point arithmetic is different between Python and Shedskin. If anyone is interested in delving in to this, I’d be very interested in hearing from you.

Extension modules

Mark notes that the extension module support is perhaps a more useful way to use Shedskin for this sort of problem.

A single module can be compiled (e.g. ’shedskin -e module.py’) and with Python you just import it (e.g. ‘import module’) and use it…with a big speed-up.

This ties the code to your installed libs – not so great for easy distribution but great for lone researchers needing a speed boost.

Shedskin 0.1 in the works

Mark’s plan is to get 0.1 released over the coming months. One aim is to get the extension module to a similar level of functionality as SWIG and improve the core library support so that Shedskin comes with (some more) Batteries Included.

Mark is open to receiving code (up to 1000 lines) that doesn’t compile.  The project would always happily accept new contributors.

See the Shedskin homepage, blog and group.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

John and I are holding our first Brighton Python meet this Wednesday 20th at The Hampton Arms. Paul Silver’s The Farm is running on the same night – we’ll be sitting on a nearby table.

We’ll have a copy of Learning Python on the table, I’ll have my laptop with ShowMeDo’s TurboGears code and John should have his laptop with the Django-based FivePoundApp.com code.

We can talk about A.I. and C-integration stuff (IPython, scipy, matplotlib, Numpy, ctypes) too, along with IDEs, resources and anything else you need to know. You can be experienced or ‘just interested’ – all are very welcome.

Our Five Pound App Day [Upcoming] is listed on the Digital Festival site now, running on Saturday 10th November from 10am-6pm. Please sign-up on Upcoming so we can plan our numbers.

The theme is ‘moving start-ups a step forward’, we’ll have four sessions during the day on:

1. The Perils of Bootstrapping (by me + other founders)
2. Developing your £5 app (John Montgomery and others)
3. Paul Silver discussing SEO
4. A guide to successful Copy-writing (Ellen)

Each session lasts about 1.5 hours, there will be a short talk followed by an interactive session. Preferably several people will have flagged the issues they’d like to discuss, e.g. someone’s site copy-writing which needs improving or they’d like to learn about improving their search ranking results.

The first talk will focus on the ups and downs of boot-strapping (drawing on examples from ShowMeDo and others) with thoughts on why it may (or may not) be for you.

John will lead the second talking about the how of developing an application – looking at various technology areas and pointing out things which will save a new boot-strapped effort a lot of wasted time.

The third and fourth talks will focus on existing websites and how+why they can be improved.

We’ll be looking for volunteers to put up their site/business for use in the discussions, along (obviously) with questions during the sessions.

The day will be ad-hoc (i.e. you can come and go), you’ll need to provide your own drinks+food. We will provide office-space for the talks and work and wi-fi.

The event is kindly sponsored by Alan Newman, founder of Sensible Development.

I’m pleased to see that the London Python meetups are running again. I’m also annoyed that I didn’t realise that one was running last night, but ho hum!

There’s a write-up by Fuzzyman (Mr. voidspace) and Tim Golden and it sounds like the free beer, pizza and great talks went down a storm.

Next time I hear about an event I’ll ask around and see who fancies a group venture to London.

ClearLeft’s Jeremy Keith will be talking on how he built a large web community at next Tuesday’s £5 App. Sign-up on Upcoming please so I know numbers, or Unattend if you can’t make it (so I buy the right amount of beer).

The Session (http://www.thesession.org/) is a community website
dedicated to Irish traditional music that’s been running for about 8
years now. You’ll get a peek behind the curtain at the technology
running the site but the really interesting aspect is seeing how the
community has grown over the years and how that affects the structure
of the site.

Last month’s event was by Martin talking on his experiences as a Mac Indie Shareware Developer and Martin is likely to be involved with our resident artist ShardCore (now exhibiting at the Tin Drum Kemptown) for our Christmas Special – tentatively titled ‘The $50 Million Apps’.

Write-ups of all the previous meets are linked on my blog and via the official £5 App Site alongside the official (and low frequency) £5 App Blog.

Note also that we have a £5 App Day coming during November (replacing the £5 App Meet just for November) where we want to help a whole heap of companies take a step forwards. Expect sessions, panels and talks. Tentative details at Upcoming, Sunday Nov 10th.

A few days back Mihai commented an interest in the Artificial Intelligence work that I’ve undertaken in the past. I figure that a short run-down of the kinds of problems I’ve tackled might be interesting.

Since 2004 I have run my own A.I. research consultancy – I’m blogging about the experience of becoming a freelance programmer and researcher at the moment (part 2, part 3, several more to come).

Currently I’m working with PANalytical in the UK R&D lab (under Professor. Paul Fewster) to apply evolutionary search algorithms and statistical analysis to multi-dimensional search problems along with an old colleague from MASA (John Anderson).

We’re looking at improving their highly-regarded Epitaxy and Reflectivity X-Ray analysis tools so that they can solve more complex problems more quickly and reliably.

Techniques include evolutionary algorithms, pattern matching, statistical signal processing and a lot of lateral thinking. I won’t say any more as the details are confidential and my work is on-going.

I enjoyed some ad-hoc work at Ambiental previously on their flood-modelling software. The interesting side of things is thinking about what you can when you can accurately model floods – can you predict the best place for flood defences? Can you apply the same techniques to crowds or gas dispersal (e.g. bombs)?

During 2003 and 2004 I worked at Algorithmix on Natural Language Processing problems under Nick Jakobi (now at Google). Up until Corpora acquired Algorithmix I worked on cutting-edge approaches to sentiment analysis and for new-news ‘burst’ reporting.

During that time I also did my own work looking at the use of Bayesian Algorithms (which were becoming rather hot for personal spam filtering) for network-based spam filtering. I worked on the assumption that ISPs saw lots of the same spam so training a filter would be much more efficient at the ISP than on the end-user’s machine.

Algorithmix was spun out of the French MASA Group where I was Senior Programmer for 5 years. I worked on the logistics optimisation side of the business (competing with iLog) into what is now the Blue Kaizen division.

The general work was to use evolutionary search algorithms on heavily-constrained logistics problems to e.g.

• Route postmen efficiently in vans to collect mail
• Route petrol tankers to deliver fuel to many cities on complex road networks with varying traffic levels
• Route waste-collection trucks which handle different types of waste to the appropriate management facility whilst respecting French hours-worked rules and road systems.

These problems were reasonably representative of the hardest logistics problems that high-end desktop computers could solve at the time, given the constraints of the problems.

I’ve always had an interest in electronic circuit design and in my early days at MASA I did some of my own research into floor planning, routing and device placement. Each of these are hard problems which will only get harder as e.g. our CPUs become more complex.

Another area of research at MASA was in the world of financial trading. I was involved in a long project on straight-forward stock market prediction (and no, it wasn’t successful and don’t get me started).

Later, separate from MASA, I was involved in a short piece of work looking at baskets of tradable financial instruments for statistical arbitrage which was fun.

Is AI alive and well? Yes, of course it is. It isn’t necessarily GOFAI and robots don’t clean our houses but there’s a heck of a lot that AI offers us.

One of the big reasons that I like AI is that it can be used to relieve humans of a lot of the tedium of analysing large amounts of data:

• Spam classifiers will have to get more-AI-ish to deal with the visual and language elements that spammers keep bringing to the party
• Logistics optimisation will get more complex as we have more constraints, more things to do and less time for planning
• Circuit designs continue to follow Moore’s law and get more complex at a frightening rate.
• As physics analysis machines become more complex the wealth of data becomes un-navigable unless you have the appropriate analysis tools.

What to get involved? If you don’t have a background in the area then find a subject that interests you, do some reading and choose a flexible dynamic language so that you can iterate quickly (I favour Python for all my AI work with number crunching in C++).

[I shall quickly plug our ShowMeDo's Python tutorial videos, there is nothing directly for A.I. in the list but there are videos for programming, physics, graphics and useful utilities that are associated with the domain.]

Search Amazon for terms like ‘artificial intelligence’, ‘evolutionary algorithms’ and ‘natural language processing’. I like ‘New ideas in Optimisation‘ by Corne, Dorigo and Glover (click the link and click the author’s names to see the AI books they published themselves).

You’ll find plenty of resources on the web and feel free to leave a comment if you’d like a bit of guidance on how to get started.

Duncan is a good man. His .htaccess page made life easy. Gracias!

A quick tip for any installers of PhpWiki on GoDaddy – it doesn’t seem to work if you use dba or dbm database types, but the flatfile type is fine.