OpenAL is software that enables game developers to create an immersive sound experience. In OpenAL sources can be placed on 3D locations in a virtual world. Moreover, a so-called Listener can be placed in this same 3D world. This listener, in most cases, the game player, can move around. The OpenAL software then automatically makes sure that the listener hears all the sources in the right proportions. The sources nearby should be louder than the sources far away if they emit sound with the same strength. This is called sound attenuation. OpenAL also implements the Doppler effect and certain other effects. In the Delta3D simulator, upon which the Symbricator3D simulator is built, OpenAL is used too as underlying sound library.

The big problem is that OpenAL does not allow for multiple listeners. You will find many people asking for this functionality. First of all, it is necessary if the game actors that are not players need to hear something. If you are playing a game with intelligent computer-driven opponents, they might hear you coming closer! This is implemented in no game that I know off. It is quite complicated to implement artificial intelligence for such game actors, and probably game developers yet do not feel ready for it. Secondly, and most important in the Replicator project, it is important to have multiple listeners in a simulator with multiple robots. To have only one listener in the environment, means that you can only emulate one microphone. With a robot swarm or a robot organism this is of course not enough. There is no open-source software available that implements that however! Also Player/Stage/Gazebo or Webots do not implement sound for multiple robots. This is because all that robot simulation software is built on top of OpenAL which does not support it.

So, what to do? First I emailed Chris Robinson who created the openal-soft implementation. In the meanwhile I started to implemented openal-sim. First of all I played around with OpenAL in combination with PulseAudio. It is possible to have multiple listeners showing up in the PulseAudio console "pavucontrol". However, it is not possible to redirect individual capture streams to recording streams. So, what that means is that I can only mix together what multiple listeners hear in one big mix. But help is near! In the OpenAL implementation there is a so-called "Wave File Writer" device. This allows for writing to a file on the disk. Rewriting the openal-soft implementation so that multiple wave files can be written in parallel and we are halfway! Then it is only necessary to read the data back. For that I choose to use the already existing RingBuffers. This is a well-known programming solution that stores data in a circular construction in which when the user comes to the end of the RingBuffer the write pointer is automatically set to the beginning again. This all nicely encapsulated with proper mutexes, so that it is impossible to read when someone is just writing. Anyway, this also allows for retrieving the sound mix concocted by OpenAL with attenuation, Doppler and eventual other effects to be retrieved again by microphones in the simulator! Then Chris emails back that it is a nice problem and perhaps something he might implement in the future. Not needed anymore!

The result can be checked out from the Almende SVN Server. And a file to test it at dtJack. This is not the end of the story, the Delta3D simulator also needs to be adapted to use openal-sim. You will find this at dtAudio. All this is implemented within a week, so don't expect maturity!

With this software set up it is finally possible to have multiple robots in a simulator each hearing each other. Something never shown before! Now it becomes possible that robots are gonna behave like a bunch of birds, tjilping/twittering different tunes! Each of them attracted by sounds in its own way!