Draw and hear your own soundscapes!

Java applet for online spectrographic sound synthesis and analysis. Includes examples in the area of speech and music synthesis/analysis. More examples of its use can be found on the The vOICe Sonification Applet page. This fully interactive page allows you to draw your own 64 × 64, 16 grey-tone image and immediately hear the corresponding 64-voice polyphonic soundscape being synthesized on the fly! See and hear how The vOICe mapping works for your input. The 64-channel sound synthesis here maps the image into an exponentially distributed [500 Hz, 4 kHz] frequency interval for a 1.05 second soundscape.

Furthermore, you can view sound waves, sonify existing images, train for audiovisual synesthesia, perform on-line composing, make soundscape animations and create spectrograms.

The vOICe mapping: vertical positions of points in a soundscape are represented by pitch, while horizontal positions are represented by time-after-click. Brightness is represented by loudness. In this manner, pixels become... voicels!

JavOICe
» Wait until the applet is running before trying any examples below! «

Click to sonify the glade

The vOICe Java applet/application written by Peter B.L. Meijer

Click somewhere within the image: The example image will disappear and your first point will appear, together with the soundscape of this single point. You can at any time pick one of the 16 grey tones from the palette on the right to draw whatever you like, and even add shading. The vOICe applet will incrementally update the soundscape for the changes you make. The right mouse button acts like an (Undo) button, removing point by point what you have drawn with the left mouse button. With the (Cursor) checkbox you can activate a sound cursor, indicating the position of the graphical cursor in your sound design.

The (Wave) button is a toggle: it alternatingly switches between the soundscape drawing display mode and the sound wave display mode. Within the plot area of the wave display, you can use the left and right mouse button to zoom in and out, respectively. The new time interval will be centered around the time point you clicked. Using the left and right mouse button just beneath the wave plots allows you to pan left and right, respectively. For mono sounds, the waveform in the left channel (green curve) will be identical to the waveform in the right channel (red curve). The subsection of the time domain waveform that you zoomed into will be indicated by blue grid lines when returning to the soundscape drawing display mode.

The (Mute) button is also a toggle: it alternatingly switches the audio output ON or OFF without affecting the soundscape generation or the contents of the soundscape: you can continue drawing your soundscape and turn the audio ON again whenever you wish. Part of the text in the applet becomes greyish when the audio is OFF to indicate the audio state.

Pressing the (Clear) button resets the soundscape (only the reference click will remain) and wipes out the drawing area of this auditory display.

The (Reset) button acts like the (Clear) button, except that it allows you to reinitialize the soundscape and drawing area with non-default parameter settings. Within the menu that appears after pressing the (Reset) button, the (Default) button restores all the default applet parameter settings, while the (Apply) button applies any of the parameter settings you may have changed. You can specify some key parameters like the resolution (number of rows and columns) of the auditory display, the lowest and highest frequency used in the sonification, the image-to-sound conversion time, the type of frequency distribution (pitch scale) and the time window type (voicel envelope).

Animation: When the applet is set up for multiple soundscape frames (NFRAME>1), the left and right arrow keys (left/right cursor keys) can be used to cycle through each of the image frames, such that you can draw each of them in turn, while the (Animate) checkbox starts and stops the image/soundscape animation. Note that this will cycle through snapshots of the image frames as stored when they were last drawn. Therefore, they need not reflect the actual current state of the applet, unless you first manually cycle through each frame with the left or right arrow keys to redraw them before starting the animation through the checkbox. By following this same procedure in the wave display mode, you can also animate the wave display as a sequence of frames to view frame-by-frame changes in the acoustic waveform.
Some parameter settings are not yet functional: this is indicated by ``N.A.'' (Not Available).

From image to sound. With The vOICe applet, it is also possible to import and sonify a .gif (GIF) or .jpg (JPEG) image file (256 colors maximum) directly Note: Java applets unfortunately impose security restrictions to this image import facility!  Use The vOICe application via the URL, by typing in the proper http: address where the image is stored, and then pressing the (Return) key or the (Load from) button. However, due to Java security restrictions, the applet may normally only load files from the site where the applet itself was loaded from! That is, this site. If you wish to sonify your own local images, you need to install the applet on your own local machine first. See also the text box on the right.

With the (OIC) checkbox you can specify that auditory image enhancement should be tried while loading an image. This activates an experimental algorithm that is likely to change in future versions. It detects edges and aims to boost perceptually relevant features at the expense of the less relevant ones.

With the (NEG) checkbox you can specify that the negative of an image should be loaded. Sonifying (auralizing) an entire image at once is a major computational task, so this may take a while! You can track the progress in the status bar. Also note that an imported image is mapped into a square area, which will therefore distort the aspect ratio for non-square images. However, this does not matter for the soundscape representation, because there simply is no particular ``correct'' aspect ratio: it all depends on our preferences for frequency range and conversion time.

Visit The vOICe Sonification Applet Page for free images to use with the applet.

For those interested in voice applications, George Dillon from the University of Washington has created a collection of phonetic samples putting the vOICe sonification within the realm of speech synthesis and analysis. Click Here to visit the site. The applet is free for non-commercial use, but you must agree to the License Conditions. (Applet in Zip file also available from this page).

Usage: Download javoice.zip and unzip it within your HTML directory.  An example html file called javoice.html is provided with the .class files required for use.

Notes:
You will want to view the source for this page, or visit The vOICe Sonification Applet page in order to get the JavaScript code that this applet/application uses.