Digital Volvelles at the Mall Galleries

May 28, 2008

Finally our little baby will be ready for some live testing. Our Multitouch-Table with the wheels-interface will be presented in the context of the Final Degree Show of the Central Saint Martins College of Art and Design. If you are around London from the 13th of June to the 19th of June feel free to stop by and have a look and feel!

Entrance to the Mall Galleries:

Things that happen every day right outside the Mall Galleries:

Location: Mall Galleries, The Mall, London SW1Y 5BD

Opening Times: Friday 13 June to Thursday 19 June – 10am to 5pm

Official URL: www.macd2008.com


Volvelles multi-touch hardware is now working

May 19, 2008

What you see in the video is the hardware for the Digital Volvelles project. The app that is being used is just a Flash application that is bundled with the touchlib project: 

 


Multi-Touch Wheel Prototype Trial

April 1, 2008

Me, Johannes, and Niqui spent a few very productive days in Brighton to put up and test the multi-touch function of a wheel prototype. This video shows the working trial set up with a beamer, a laptop, a WII-Remote Control and funky looking selfmade infrared-pens.

Wii-Remote Control from Star Wars

Johannes and Wii-Remote


Multi-touch applications with Flash

March 14, 2008

I’ve written in a previous post that there is the idea of a standardised technique to connect multi-touch hardware to multi-touch software via TUIO. As we want to concentrate on Flash as visualisation and interaction layer for our digital volvelles I will present a Flash AS3 API for multi-touch with TUIO in this and further posts.

The touchlib project from the NUI Group is a collection of software tools for the realisation of a multi-touch table. It provides tools like digital image processing for the tracking of fingers on a multi-touch screen, a TUIO server and so on. It additionally provides a Flash AS3 library in order to process TUIO messages inside of Flash and to provide standard interface elements e.g. sliders.

All Flash applications that are being built with touchlib’s AS3 library can not only be used with a TUIO enabled multi-touch device but also with a mouse. This offers advantages in the development process as you do not (necessarily) need a multi-touch device in order to use your application. This will be very useful if more than one person are involved in the development process (as in our case a designer, a Flash developer and a hardware developer are involved) because everyone can get a feeling for the interaction with the application without the need for a colocated multi-touch device.

The touchlib project including the mentioned AS3 library can be downloaded here. The AS3 library with demo applications can be found in the following folder within the zip file: touchlib -> AS3 -> src.

For some videos of our Flash multi-touch applications also see:

Links for more information at the NUI Group regarding the following topics:


Using TUIO to connect multi-touch hardware to multi-touch applications

February 27, 2008

To be able to develop multi-touch applications you must be in possession of multi-touch hardware. However, there hardly doesn’t exist any off-the-shelve multi-touch hardware with which you could start developing multi-touch applications right away. The only product in my mind that enables you to develop customised multi-touch applications for it would be the Lemur from Jazzman. But 1000$ for a Lemur mixer is not too cheap and development possibilities are constricted.

Lemur multi-touch mixer

Figure 1: Lemur from JazzMutant

The result is that you have to develop your own multi-touch hardware. Lucky enough there are those pals from the Nuigroup that support you with manuals, with a software called Touchlib and with billboards where you can ask (stupid) questions and they stay patient with you.

A major downside of developing your own hardware is that there are hardly any standards, which would describe a way to connect your hardware to your computer (drivers or similar stuff) or which would describe how you can connect your applications to your hardware (usually called an application programming interface -> API). The task to connect a multi-touch table to a computer can be solved by using the already mentioned software Touchlib. In terms of connecting your multi-touch hardware to your software there is at least the idea of a standardised way: TUIO (A Protocol for Table-Top Tangible User Interfaces) invented by reactable software.

TUIO describes a way to convey touch information from your hardware over the network to your application. In figure 2 you can see how TUIO can be used e.g. with Flash:

  • The multi-touch sensing hardware is connected to computer A. Computer A runs e.g. Touchlib and extracts touch information from the data yielded by the sensing hardware. Afterwards Touchlib wraps this information in TUIO data and sends this data via network to computer B.
  • Computer B runs a Flash application. It receives the TUIO data from computer A (via AS3 sockets) and includes this data in the Flash application.
  • The multi-touch screen displays the Flash application from computer A.

TUIO principle
Figure 2: How TUIO integrates within multi-touch applications

Why would you want to use something like TUIO? Why does not use the computer to which the multi-touch hardware is connected to run the (Flash) application? Possible answers:

  • You want to use your applications with different multi-touch hardwares. With TUIO you could easily exchange the underlying hardware if both hardwares sent their touch information via TUIO. In our case we could develop a volvelle flash application and we could use it with both an FTIR multi-touch table OR with a Wiimote based IR pen solution. Just connect the volvelle application to the TUIO enabled hardware of your choice and you’re done.
  • Calculation of touch information can be a demanding task: Imagine two or more persons touching an FTIR table with both of their hands. The computer to which the FTIR table is physically connected has to calculate the touch information via image processing with the Touchlib (touch information gets extracted from a video camera image). Even with the newest hardware every bit of processing power could be needed for this task. On the other hand using touch information of e.g. 20 fingers in your Flash application is also a demanding task for which you also may want to use all availabe processing power of another computer.
  • But you still can run both applications on one computer!
  • Exchangable software: Imagine 100 development teams developing multi-touch software. Wouldn’t it be nice to exchange software with them? Or to create a repository of multi-touch enabled software? To be able to include your own application in a show-case of a major visualisation agency? At least I do think so!
  • There should be tons of arguments more but I leave it with those indicated above.

How can you use TUIO in your own applications?

This one might be the core bit of information that is interesting for you. How can you implement the network protocol that TUIO describes in your own application? The answer is you don’t need to. The guys from Reactable provide all necessary core applications for TUIO on their homepage. If you want to use TUIO within your Flash application simply use this core template. To develop and test your TUIO applications without available multi-touch hardware the TUIO simulator should be a great help for you. However, the TUIO simulator demands a Java Runtime (1.6 or higher) on your computer.

Update: TUIO is based on the Open Sound Control (OSC) protocol. As explained on Seth Sandler’s blog Flash needs an additional component to talk with OSC components i.e. TUIO enabled hardware:

“You also need FLOSC (flash open sound control) which allows Flash to connect to programs that send OSC messages like the ones above. You can get FLOSC here. You will need java installed in order to run FLOSC.”

Furthermore, Seth Sandler’s downloads page gives a nice overview on how to set up TUIO with Flash.

There is an excellent Getting Started with TUIO and Flash post on the nuigroup forum.

For some videos of our Flash multi-touch applications also visit:


Jon Ruppel’s Options

February 22, 2008

… for showing content (also with circles). Visit the website: http://www.oneover.com/ and click on OPTIONS -> play….

Screenshot of his site