Volvelles multi-touch hardware is now working

May 19, 2008

What you see in the video is the hardware for the Digital Volvelles project. The app that is being used is just a Flash application that is bundled with the touchlib project: 



Impressions ‘New’ Table

April 27, 2008

How to connect a Wii Remote to a Mac in Bootcamp

April 24, 2008

I tested the procedure to connect a Wii Remote (Wiimote) via Bluetooth with both a Mac Mini as well as a MacBook in Bootcamp with Windows XP. However, I cannot guarantee that it works with all Macs in Bootcamp. Definitely a problem is, that I’ve got only a German XP for testing purposes. Thus I can’t guarantee that I found the right translations for the names of buttons and labels in the XP Bluetooth wizard.

  1. In “Systems Preferences” go to “Bluetooth devices”
  2. If there already is a Wiimote listed, press “Remove” (yes, it’s tedious but I could not figure out another way.)
  3. Click “Add Device”
  4. Select “Device is ready to be found”
  5. On your Wiimote click both 1 & 2
  6. Immediately click “Search devices” in Bluetooth devices window.
  7. After Bluetooth devices have been found select “Nintendo RVL…” and click Next.
  8. Choose “No key necessary” and click “next”. Attention: Wiimote must still be blinking after this step. If you’ve done everything right a bubble should appear over Windows’ task bar which tells you “New HID device found”.
  9. Start Wii yourself’s demo.exe. A Dos shell window opens up with the message “Searching for Wiimote”
  10. Press 1 & 2 buttons of Wiimote if not blinking. Attention: demo.exe should now tell you “Connected!” behind “Searching for Wiimote”.
  11. Close demo.exe.
  12. Open up the application in which you want to use your Wiimote e.g. Johnny Chung Lee’s Wii Whiteboard. Attention: I was not able to use the exe that was provided on the download page of the project’s Sourceforge page. I had to open up the project in Visual Studio (I used the VS Express 2008 version that is freely available on Microsoft’s homepage) and press the play button in the menu bar (next to “debug”). Then everything worked just fine.
Update: Just figured out that I’ve forgot to upload Demo.exe. This is a Dos application, which shows the basic functionalities of the WiiLib project when a Wii remote is connected to the computer. Anyway you can download the source yourself from their page and compile it with Visual Studio (a VS project file is included) but somehow the compiled Demo.exe failed to work on some computers. So if you just take my version of Demo.exe you will be faster and the possibilty that you will be successful is higher. Demo.exe
I hope your virus scanner will try to kill if you’re about to open a downloaded, zipped exe from the Internet 😉

For some videos of our Flash multi-touch applications also visit:

Multi-touch Table Progress

April 7, 2008

I’ve been able to get hold of an old frame made of perforated metal plates. I reassembled the parts in a way that I’ve got a frame for my multi-touch display. Currently it has the table form factor but it can easily be readjusted to be a console or even a panel.

The multi-touch display itself is made of an acrylic glass plate in which infrared light is being emitted from the sides by 24 infrared leds. Thus the light is being endlessly reflected internally in the plate. If a finger is touching the plate the optical effect of light frustration will take place that effects light to be emitted vertically on the frustration place (the point where your finger tip is touching the display). Onto the back of the display where a back projection foil is mounted an image is projected by a video beamer (a Panasonic PT-AX200E). The display is being scanned by a webcam with a removed infrared light filter (a Philips SPC900NC with this lens). A Mac Mini is producing the image that is projected by the beamer onto the display and is doing touch calculations with the help of Touchlib.

Here are some pictures from the current state of it:

Multi-touch Table First Pictures

April 7, 2008

Here are some pictures of my first multi-touch table attempts:

Multi-touch applications with Flash

March 14, 2008

I’ve written in a previous post that there is the idea of a standardised technique to connect multi-touch hardware to multi-touch software via TUIO. As we want to concentrate on Flash as visualisation and interaction layer for our digital volvelles I will present a Flash AS3 API for multi-touch with TUIO in this and further posts.

The touchlib project from the NUI Group is a collection of software tools for the realisation of a multi-touch table. It provides tools like digital image processing for the tracking of fingers on a multi-touch screen, a TUIO server and so on. It additionally provides a Flash AS3 library in order to process TUIO messages inside of Flash and to provide standard interface elements e.g. sliders.

All Flash applications that are being built with touchlib’s AS3 library can not only be used with a TUIO enabled multi-touch device but also with a mouse. This offers advantages in the development process as you do not (necessarily) need a multi-touch device in order to use your application. This will be very useful if more than one person are involved in the development process (as in our case a designer, a Flash developer and a hardware developer are involved) because everyone can get a feeling for the interaction with the application without the need for a colocated multi-touch device.

The touchlib project including the mentioned AS3 library can be downloaded here. The AS3 library with demo applications can be found in the following folder within the zip file: touchlib -> AS3 -> src.

For some videos of our Flash multi-touch applications also see:

Links for more information at the NUI Group regarding the following topics:

First test with digital Wii Whiteboard

March 6, 2008

What do I do in the video?

  1. Calibrate the Wii Whiteboard application
  2. Use the IR Pen with MS Paint. In the second image I drew with Paint I wrote “Hello Mel” but the bad auto white balance function of my bad Nokia 6610 Navigator cam (which cam sucks extraordinarily!!!) prevented readability.
  3. Use IR Pen with MS Explorer (NOT Internet Explorer)

I compiled Johnny Chung Lee’s Wii Whiteboard Application and used it with an infrared (IR) light pen that I built on my own (see figure 1).

I projected a videobeamer’s image onto white cardboard (the ones which are used for passepartouts) that stood on an easel. Before that I put grey cardboard onto the easel but the grey cardboard produced bad reflection of infrared light which caused bad tracking results by the Wiimote.

My IR pen and Wiimote with its customised stand

Figure 1: My IR pen (with old switch) and Wiimote with its “customised” stand

Anyway, this is just the kick-off for further actions with the Wiimote and IR pens but the setup for this was really hard to establish.
Finally: Guess to which music I was listening to!