EuroSciPy Parallel Python tutorial now online

I taught Parallel Python at EuroSciPy 2012 last week in Brussels, I’ve uploaded all the necessary stuff. In the talk we covered:

  • multiprocessing (built in)
  • parallelpython (an easy shift from multiprocessing to do mult-machine and -core processing)
  • gearman (cross-platform job server for heterogeneous job processing)
  • picloud.com (was python only, now any infrastructure cloud-based processing using EC2)
  • ipython cluster (for easy parallelisation with ipython)

Here’s my github repo with the powerpoint and python source, the README outlines the install requirements. I used python 2.7 on Ubuntu but it should run on any platform. This is related to my PyCon 2012 High Performance Python tutorial. Here’s my class of 70, taught by a bleary-eyed me as I’d just flown from San Francisco (9 hours time difference ugh!):

I’m thinking of writing an updated High Performance+Parallel Python guide (probably as a self-published book), if you’re interested in hearing about it please join the High Performance Python Mailing List (I’ve only got a list right now). I’ll make an announce once I know more.

We had I think 190 folk at the tutorials and 170 for the conference over the weekend (along with a discussion about whether weekend conferences hurt attendance…). 25% of the attendees were small companies (my tweet) and I was jolly pleased to attend my first sprint, which I promptly ignored, such that I could hack a change to Fabian’s new memory_profiler such that you don’t have to use a decorator to choose the function to profile (which, um, I must submit back in the next few days).

I also gave a 3 minute lightning talk on my experiences building (and ending) StrongSteam via StartupChile. The point I forgot to make (dang just 3 minutes!) is that whilst StartupChile will be on various government radars around the world, I doubt it’d be on related radars for governments looking to build a similar programme that includes open research (hopefully also coupled with entrepreneurship). As and when a government/institution suggests inviting researchers to their country to collaborate on large, ground breaking research problems requiring much collaboration, perhaps with business opportunities, I hope they look to StartupChile as a template for successfully inviting 1000s of folk under a simple grant systems to a foreign country.

As noted in my slides I’m also looking for new work opportunities around Big Data/Natural Language Processing/High Performance Computing – take a look at my work site and please drop me an email.

As usual we had a big social, organised by Ludovic Gasc of AFPyro. Here are my mussels nestled in their bed with snails wrapped in warm cheese blankets (fine food!):

I also got introduced to the fine Free Beer, modelled by Didrik of Enthought:


Ian is a Chief Interim Data Scientist via his Mor Consulting. Sign-up for Data Science tutorials in London and to hear about his data science thoughts and jobs. He lives in London, is walked by his high energy Springer Spaniel and is a consumer of fine coffees.

4 Comments

  • Ludovic Gasc
    Hi, Thanks for the tutorial. Nevertheless, for your information, Nicolas Pettiaux helps me to organize the AFPyro (social event), but it's me the organizer ;) Best regards.
  • My mistake! I've swapped you for Nicholas. Ian.
  • Paul
    Hi Ian, thanks for the great tutorials - I really enjoyed them. Is there a video of this tutorial available? Best, Paul
  • Sorry, they're not out yet, I'm still waiting for the EuroSciPy videos to come online too!