I’m at IUI 2010, this is a mostly academic conference focused on using new techniques to make intelligent user interfaces.  I’ll update this entry as the conference proceeds.

Day 1 (Sunday) – Workshops

I’m in the Eye Gaze for Intelligent Human Machine Interaction workshop, there’s a full breakdown of this session’s talks here. The talks focus on the use of eye-gaze tracking tools to let humans interact with computers in an intuitive and easy fashion.

Two talks have really caught my eye. Manuel Möller has presented “The Text 2.0 Framework – Writing Web-Based Gaze-Controlled Realtime Applications Quickly and Easily” (via here). Text20.net is the background site, they’re offering a browser plug-in (Safari at present, Chrome/Firefox to come) that augments your browsing experience if you’ve got a head tracker. They’ve added some new mark-up tags like:

• OnGazeOver – like OnMouseOver but fires if your gaze goes over the element (e.g. to make an image change or high-light)
• OnPerusal – if you quickly scan a piece of text then this would fire
• OnRead – only fires if your start to properly read the text

They propose using a site like DBPedia to augment your browsing experience – perhaps bringing in additional text if your gaze rests on a block of text, bringing in alternative images if you look at an image or translating text that you re-read if it knows you’re a foreign-language user.

The above is only useful if you have a gaze-sensing device and these are a bit pricey (think: $10,000-$20,000). However…

Shortly before Wen-Hung Liao presented “Robust Pupil Detection for Gaze-based User Interface” (via here) where he described a $60 device (the $60 refers to the cost of a standard 640×480 30fps webcam) that gives reasonable eye-gaze tracking on a desktop computer. Pretty much he’s describing a way to replace $20,000 work of high-end eye-gaze tracking tools with the webcam in your laptop.

The resolution achieved is around 40×40 – pretty low but enough to support a lightly modified web browser that allows eye-gaze control. The modification is a zoom whenever the user’s gaze rests on an area – that section zooms so you can more accurately select a link.

Here’s a demo showing “eye typing” (see some more under VIPLpin):

There is a downside – natural light washes out too much detail (and casts shadows and reflections) so the camera needs a simple modification. By popping out the normal lens and using an IR lens the camera senses light in the infra-red range – for this algorithm the input is far cleaner. It is quite conceivable that we’ll have a second (IR style) webcam in our laptops and this second device could give us simple gaze control on our machines. This algorithm runs comfortably on a dual-core machine at 30fps (previous generation algorithms are laggy as they’re too CPU-intensive).

What happens if we combine this $60 device (free for me – I have a good webcam in my MacBook that could be modified…) with the Text 2.0 plug-in? I can probably navigate web pages when reading wikipedia purely using gaze. If the gaze is getting to the bottom of the screen then it could auto-scroll and I’d certainly like annotations from sites like wikipedia augmenting my research experience.

The workshop is over and we’ve ended up having a further chat about Pico projectors costing $350USD (apparently a bit dangerous – they’re laser-based and can burn the retina) and augmenting reality with said devices as you wander around (imagine strapping one to your chest).

In the poster session that followed Stylianos Asteriadis showed a head pose detector that works using a desktop webcam using a published algorithm – this could be used in gaming and for hands-free control. It detects the attitude of the head on 3 axis by investigating a bounding box around the head and the location of features like eyes and the mouth.

Some interesting people met so far – Chuck Rich (cool robots), Isamu Nakao (Sony R&D), Wen-Hung Liao (National Chengchi Uni). Tweets are under #iui2010.

Day 2 (first day of conference talks)

The first talk of the day was Cortically Coupled Computer Vision by Paul Sajda. The intent was to speed up search for a target image from a large database using fast brain recognition techniques. The user has a target image in mind, they throw 10s of images at a user showing each for 100ms. By recording brain activity using non-invasive techniques like EEG and a custom labeling approach the they were able to significantly improve precision and recall in search problems.

This was followed by the 1-minute madness session where 20 or so speakers introduced the posters that would be shown at the banquet the next night. Two that caught my eye were Henry Lieberman’s Why UI (he’s one of the creators of ConceptNet) and another chap’s $3 Gesture Recognizer (based on Android and Wii devices):

Amy Harrison gave an interesting talk on Automatically Identifying Targets Users Interact With During Real World Tasks. Given my background with screencasting and interest in scripted (automatic) screencasting, the ideas around taking screenshots and identifying screen targets (like buttons, scroll bars etc) to extract additional information was very interesting. Her techniques using CRUMBs identify 89% of user interface features vs 74% for the Microsoft accessibility interface.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

I’ve just started to play with pyCUDA which lets you run parallel math operations on a CUDA-compliant NVidia graphics card through Python.

CUDA stands for Compute Unified Device Architecture – it is an architecture that lets us program the Graphics Processing Unit (GPU) on a high powered graphics card to do scientific or graphical math calculations rather than the usual texture processing for games.  In essence it is a mini supercomputer that is specialised just for fast math operations – if you can figure out how to use it.

The goal is to off-load the CPU-intensive calculations for two of my clients (a physics company and a flood modelling company) to achieve 10* to 100* speed-ups using commodity graphics cards.

pyCUDA makes it easy to interactively program a CUDA device rather than hitting C++ code with the slow write/compile/debug loop.  Recent MacBooks (mine was bought in January 2009) have NVidia cards with CUDA-compatible devices built-in (mine is a 9400M).  For my desktop computer I have a 9800 GT (costing £100).

It turns out that this is bleeding-edge stuff – getting pyCUDA compiled on my MacBook and Win XP machine took some time (forum posts for Mac and Windows issues) thankfully the group is helpful and the wiki has an installation section for Windows, Mac and Linux and some reasonable documentation.

Right now I’ve got as far as running some of the demo code on my MacBook (showing a 5* speed-up over the CPU) and my desktop (showing a 30* speed-up over the CPU).  I’ll report more as I progress.

Update – pyCUDA works inside IPython too, lovely.

Update – I don’t have OpenGL working for gl_interop.py but as noted here you need “CUDA_ENABLE_GL = True” in siteconf.py and you need PyOpenGL installed.  When rebuilding my MSVC threw a hissy fit, it isn’t essential to my work so I’m skipping this demo.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

On January 27th here in Brighton I’m co-running a SkillSwap evening, I’ll spend 45 minutes teaching screencasting (based on a Mac) and Andy White will spend 45 minutes teaching podcasting.  We’ll cover planning, recording, editing, distributing and mics between us.

We’re both aiming the talks at freelancers (so they can communicate better with clients) and small companies (for training, marketing and demos).  We’re also the authors of The Screencasting Handbook and Podcasting Unleashed.

I’ll cover at least these topics:

• Free and commercial tools on a Mac (and Windows/Linux if requested)
• Recording your first screencast with Jing and hosting it on the Web
• Planning your screencast so it meets the needs of your audience
• The differences between a sales/marketing screencast and a tutorial
• Using ScreenFlow to record, edit and produce a screencast and then upload it to YouTube
• Hosting your own screencast and other distribution options

If you bring a laptop then I can get you started with the free Jing so you can walk away with a recording and hosting solution for Mac and Windows.

If you’re in Brighton then the event is free, see details in Upcoming and sign-up on EventBrite. SkillSwap has been running for years – cheers to Nat and James for finding a spot for us.

Madgex will be sponsoring beer and nibbles, the atmosphere will be relaxed and friendly.  Nat is recording the audio for a podcast and I intend to record a video of the evening for distribution via Vimeo (but of course that won’t be the same as being there and being able to ask questions!).

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

I wanted to play with text to speech, I’ve been looking for a cross-platform open-source solution that sounds reasonable.  I’m really impressed with the festival project, the web demo lets you enter your own text.

Festival is cross-platform but compiling it on a Mac takes a touch of effort (it looks like it is easier on Linux and Win).

This article shows you how to use it and how to web-enable it with some php.  For the simplest demo I used ‘bin/text2wave input.txt -o output.wav’ with input.txt containing a sentence.

To get started, get the latest code.  I have v1.96beta.  You may also want the official festlang-talk list and possibly this more complete archive.

Compiling speech_tools-1.2.96-beta.tar.gz

It ought to have been as simple as ‘make clean; make’ but there’s a few changes to make first.  First we need this fix or we get a compile error in macosxaudio in kAudioUnitProperty_SetInputCallback:

If you add
#include <AudioUnit/AUNTComponent.h>
after the include block on lines 45-48 in audio/macosxaudio.cc the
problem should be solved.

By the way, remember to change the byte order if you have an intel
mac, i.e. on line 131:
     waveformat.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger
		 | kLinearPCMFormatFlagIsPacked;
	// For Intel	| kLinearPCMFormatFlagIsPacked;
     // For PowerPC    | kLinearPCMFormatFlagIsPacked |
kLinearPCMFormatFlagIsBigEndian;

The following was a trickier error to solve:

g++ -c -fno-implicit-templates -O3 -Wall -I../include sigpr_frame.cc
sigpr_frame.cc: In function
‘void lpc2cep(const EST_FVector&, EST_FVector&)’:
sigpr_frame.cc:318: error: ‘__isnan’ was not declared in this scope
make[1]: *** [sigpr_frame.o] Error 1
make: *** [sigpr] Error 2

The fix was known but the relevant archive was missing, some googling for ‘__isnan mac‘ results in this cached 2006 page:

--- ../test/speech_tools/include/EST_math.h     2006-08-03  
08:49:35.000000000 -0500
+++ include/EST_math.h  2006-08-17 17:53:33.000000000 -0500
@@ -43,7 +43,7 @@
#if defined(__APPLE__)
/* Not sure why I need this here, but I do */
-extern "C" int isnan(double);
+extern "C" int isnan(float);
#endif
/* this isn't included from c, but just to be safe... */
@@ -101,7 +101,6 @@
/* Apple OSX */
#if defined(__APPLE__)
#define isnanf(X) isnan(X)
-#define isnan(X) __isnan(X)
#endif
/* FreeBSD *and other 4.4 based systems require anything, isnanf is  
defined */

Compiling festival-1.96-beta.tar.gz

Once speech-tools is compiled, getting ‘festival-1.96-beta.tar.gz’ compiled is as easy as ‘make clean;make’.

Python’s MacSpeechX

I also had a play with the macspeechx module which ties Python to the Mac’s voice-synthesiser.  See list_voice_name() in macspeechX.py for an example of how it all works.

It works to power the speech synthesiser but it doesn’t appear to let you record the speech to a file (unlike festival above).

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

I’ve just launched my second Twitter bot – @ConceptNetDaily takes a random concept from the A.I. site ConceptNet and posts it to Twitter with a link back to the site. A tweet looks like:

“When humans own horses, humans groom and ride horses.” http://tinyurl.com/ydvf7vg

The TinyURL expands out to an address like: http://openmind.media.mit.edu/en/assertion/143313/

The aim of the site is to build a large repository of common-sense knowledge, exactly the kind of knowledge that humans take for granted and never write down as statements for a computer to understand.  Currently it tracks over 1,026,553 statements.

Using the link you can vote on the concept.  Vote up if the concept is solid (i.e. something a human would say is ‘right’) or down if it is wrong, silly or erroneous.  The site supports OpenID which makes starting a touch easier.

My goal with this bot is to remind people every day to vote on the concepts and to add new knowledge.  If a concept has many votes then we can have faith that it is ‘common-sense knowledge’.  If a concept is voted down enough then we can have faith that it is ‘unhelpful or wrong’.

You’ll find a searchable list of Concepts and some random examples on the English homepage.  For good examples see all the information that ConceptNet knows about humans, chess and girls.

Details:

I’ve written the bot in Python using PyYAML, Python-tinyurl and Python-twitter.  It runs every day via a cron job.  It works by guessing a random id for a raw_assertion and checking to see if a concept lives at the URL.  See this XML example for id 143313, I extract the .yaml version via PyYAML but the .xml version renders nicely in your browser if you want a peek.

ConceptNet’s web API is well documented.  ConceptNet itself is written in Python using Django but I’m not using the downloaded version here, just the web API.

My first Twitter bot – @BrightonJobDoom:

Just in case you live here in Brighton you might want to track @BrightonJobDoom to see how healthy (or…not) the job market is in the UK during this rather wobbly recession   I wrote this bot for our £5 App’s 5k coding competition.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

A month back I tried to build an Ubuntu-based Eucalyptus cloud/cluster environment for a client for a parallel processing research project.  The project was thwarted by an overly aggressive corporate firewall and my lack of understanding of low-level network config-fu.

I’ve revisited the project using the same machines but with an external public internet connection (no firewall – yay!).

Grub2

On the node machine I still needed to dual-boot to Windows.  Unfortunately whilst reboots to Linux are fine, if Windows is booted it ‘does something’ to the MBR and the machine is unbootable.  I delved into the boot-loader and had to learn some Grub2-fu.

Grub2 was introduced in Ubuntu 9.10, it replaces Grub which in turn replaced boot managers like lilo.  The wiki page is pretty good for recovering a boot-loader using an Ubuntu LiveCD but it didn’t work quite to plan.

The step for ’sudo chroot /mnt’ fails as bash or sh can’t be run from within /mnt (which at this point is looking at the originally installed hd).  There is something odd going on with the LiveCD, much googling didn’t seem to reveal the answer.

To run grub-install on the hd, rather than via the CD (because chroot fails) I used ’sudo grub-install –root-directory=/mnt /dev/sda’, it reports that ‘(hd0) /dev/sda’ is installed.

Sidenote – on later attempts somehow a reference to (fd0) got involved and this broke the boot process.  I edited /mnt/boot/grub/device.map to remove the fd0 reference, leaving the hd0 reference.  I ran grub-install again and all was fine.  Now the machine can boot again.

Mounting a USB memory stick

Whilst a 8Gb memory stick was recognised, it didn’t get mounted.  I had to edit /etc/fstab and add:

/dev/sdf1 /mnt/stick auto umask=0,user,iocharset=iso8859-1,sync,codepage=850,noauto,exec,users 0 0

After this I used ’sudo mkdir /mnt/stick’, ’sudo mount /dev/sdf1′ and it mounted just fine.

Installing Eucalyptus

The install process this time around was much the same as before, except this time without the firewall it all ‘just worked’.  Seeing the fnords part 1 took me through the basic install.

I got the feeling from later steps that the cloud controller needs a static IP so I switched the cluster controller from DHCP to a static IP and rebooted.

The discover nodes process (’sudo euca_conf –no-rsync –discover-nodes’) for euca_conf also required that I’d setup ssh keys on the Node, step 6 in the NodeInstall doc has the instruction.  Typo note – if you spell ‘eucalyptus’ wrong you’ll go round in circles trying to figure out why the password won’t work!

Sometimes I couldn’t get ‘euca-describe-availability-zones verbose’ to work, it’d just report ‘No route to host’.  It seems that a reboot of the CC and Node are required, plus a minute or so of patience after boot for Apache to sort itself out, before this problems just goes away.

Using the Ubuntu Store

Having installed the CC and registered a Node, next I ran the web interface via ‘https://10.0.0.4:8443′.  Note ‘https’.  If you visit the website too soon after a reboot (i.e. <1 minute) then the webapp won’t respond or maybe it won’t recognise the admin user.  Having logged in, the first login forces a password change.

Next check the ‘Configuration’ tab and verify the IP addresses.  For reasons beyond my understanding our switch rebooted during my first attempt to setup the cluster and it switched from the ‘192.168.x.x’ address range to ‘10.x.x.x’ – this royally barfed my configuration.  I chose to re-install the CC from scratch (I was plagued by ‘no route to host’ problems no matter how much tweaking I tried).

Next visit the ‘Store’ tab and download an image, I’m using ‘Ubuntu 9.10 Karmic Koala (i386)’.  Today this works – I’ve spent 2.5 days building and re-building the cluster to get it to this point.  Often the Store would download an image and then report ‘no route to host’.  This process is pretty darned frustrating and seems to lack useful error messages.

But ultimately – no cigar

Rather frustratingly I can’t get my Node to run an image.  I can see that the Node exists though ‘euca-describe-availability-zones verbose’ shows that a Node exists but doesn’t list its IP address which is odd, the online docs say it should be shown.

If I run an image then it enters the ‘pending’ state and then the ‘terminating’ state.  Digging around in Google shows that other people currently have the same problem, it might be related to the lack of Hypervisor instructions on my Node machine (though they’re not supposed to be required…).  Possibly also the current build in unstable, there’s a lot of bug-fixing going on.

Debug notes

Eucalyptus has a trouble-shooting guide, this blog series is very useful.

Conclusion

Eucalyptus should give you an EC2-like cloud that runs on your own machines, using an EC2-compatible API so you could move to the cloud when you want to scale up or are less concerned about the privacy of your data.  Currently I can’t get it to work but others do have it working – it seems to depend upon your hardware.  It also lacks clear error messages so debugging is hard – I resorted to clean installs on three occasions.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

I’m using WingIDE on my MacBook and I couldn’t get copy/paste to work between WingIDE (running in X11) and native apps.  This meant copying URLs and code snippets was impossible…hugely frustrating!

There is a simple fix, as outlined here just run the Property List Editor, open the specifed .plist, tick the 5 checkboxes, save, restart all of X11 and then the clipboard is shared between X11 and native apps.  Phew.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

On Wednesday night we ran our music-themed £5 App Xmas Special (fivepoundapp.com).

It was fab!  John and I had a fab time organising things and watching the night run so down-to-earthly – it seems that many others did too.  I particularly like:

“I bloomin’ love £5 app! The event that’s happy to be itself, and is more rewarding for all as a result. Here with @ribot & @lastminute teams” – ribotminimus

“Home from #fivepoundapp, letting the awesomeness sink in.” – j4mie

Get to the end of the very last video and you’ll hear a special £5 App rendition of Jingle Bells.

Note: I want photos!  Email me links to flickr’d images please.  I also want your blog write-ups, mail them to me or comment down below.

Particular thanks for our sponsors Alan Newman (Sensible Development) and Paul Silver (PaulSilver.co.uk) along with John (psychicorigami) and my ProCasts for putting up cash to fund a few hours of free beer.  Also super-huge thanks to the Ribots for supplying piles of mince pies (yummy!) and John for baking a batch of crunchy cookies.

The event was organised through Philip and Declan of PlayGroup, they use Hector’s House for arts and science gigs (thanks BuildBrighton for the connection!).  Cheers chaps, it was exactly the space we needed!

“Seb’s Slightly Failed Music Career”

Seb spoke on the highs and lows of forming a band, showed previously-unseen footage and generally gave the lowdown on how it all works. Rick-rolling was included.  Seb has his own write-up.

Sadly Seb’s hard-drive died after the talk taking all his transcoded footage but on the flip-side Seb inspired Simon to share footage from his old cover band.

Here’s the 60 minute video of Seb’s talk:

£5 App #20 “Seb’s Slightly Failed Music Career” for the 2009 Xmas Special from IanProCastsCoUk on Vimeo.

We were absolutely honoured that Seb and Jenny unveiled their new Xmas song tonight, see it here and share it around:

“Toby Cole – Zero to Theremin in 20 days” (with demo)

Toby Cole shows the ThereThing constructed through BuildBrighton and unveiled at a live gig the previous month.

Paul Silver took a video of the ThereThing in action:

Sadly the ThereThing is slightly out of shot during the video of the talk but you can hear Toby and see the screen just fine (and the ThereThing link shows it in detail).

£5 App #20 “Toby Cole – Zero to Theremin in 20 days” for the 2009 Xmas Special from Ian Ozsvald on Vimeo.

“Jim – Mrmr/LiveAPI guitar-mounted iPhone ableton live interface”

Jim Purbrick showed Mrmr, the LiveAPI guitar mounted iPhone Ableton live interface.  Jim’s also the head of Second Life (UK) and is known for building robots.

£5 App #20 “Jim Purbrick – Mrmr/LiveAPI guitar mounted ableton live interface” for the 2009 Xmas Special from Ian Ozsvald on Vimeo.

“Lastminute.com Lab’s with Bottle-Rock-It” (with an additional proper demo video)

Richard, Sam and Mathias (LastMinute.com Labs) came down from London (thanks guys!) to demo Bottle-Rock-It, a group iPhone musical instrument.

The background talk gives loads of detail, sadly the demo went a bit sideways so we sang Jingle Bells as a loud (and slightly tipsy) group instead.

Check this BBC News story to see Bottle Rock It in action.

£5 App #20 “Lastminute.com’s Bottle-Rock-It” for the 2009 Xmas Special from Ian Ozsvald on Vimeo.

100 Robots (band)

After the talks finished Jim Purbrick and Max went on to play live n’loud as 100 Robots.

2010 and beyond…

If you want to keep in touch with future £5 App events then join the £5 App Google Group – it is very low volume and is mostly there just for the announces.

We’ll probably run some more competitions next year, the 5k competition went very well and John wants to do more around that idea and I want to play with some open-source A.I. kits.  Details to follow.

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

This is a quick review of four chatbot that are easily found on the web.  I’ll take a look at the granddaddy ELIZA (wikip), A.L.I.C.E. (wikip) which uses AIML, Fake Kirk (with speech synthesis and a face) and O2’s Ask Lucy.

The goal of these chats was to see how each of the bots broke down and to learn about how the different technologies worked.  ELIZA has a small, hard-coded rule set.  A.L.I.C.E. is rule driven but with a big AIML rule set.  Fake Kirk uses statistical training on Star Trek scripts.  I don’t know what O2 use (but it doesn’t feel very sophisticated at all!).

ELIZA

A.L.I.C.E.

Fake Kirk

I’m also adding this 10 min video on Fake Kirk by someone from Pandorabots, it might be the author of Fake Kirk:

O2’s Ask Lucy

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.

Here’s a simple idea to help visitors to a new area.  Maybe it’s been done before and someone can leave a comment about it?

The problem – when you visit a place you don’t know you have no idea what you need to see, where to get a map, which pubs and cafes are nice, where the worthy landmarks are etc.

Possible solution – visit a site that gives you 1-2 pages of printable (or iPhoneable) data culled from WikiPedia, OpenStreetMap/GMaps, OpenPlaques, Flickr, Twitter and more.  The pages would give you a summary of what’s there to see, some history, maps and also some recent information (probably via Twitter).

The printable option would be useful, iPhone coverage still isn’t great in the UK in the smaller and more interesting towns.

Personally I’d use this – we go walking to places that we don’t know every weekend and some background, a map and some topical info (e.g. are there any fairs or events happening today?) would be super useful.  I’d guess that this would be useful for anyone visiting an area, even just for parents coming to visit for the weekend.

Does anything like this already exist?

---

Ian produces professional screencasts (ProCasts), writes The Screencasting Handbook, programs Python, researches Artificial Intelligence (Mor Consulting) and is also a sea-side dweller and consumer of fine coffees.