Serveur © IRCAM - CENTRE POMPIDOU 1996-2005.
Tous droits réservés pour tous pays. All rights reserved.

Xspect: a New Motif Signal Visualisation, Analysis and Editing Program

Xavier Rodet, Dominique François, Guillaume Levy

Proceedings of the International Computer Music Conference, Hong Kong (1996)
Copyright © Ircam - Centre Georges-Pompidou 1996

Abstract

Xspect is a visualisation and analysis tool aimed at helping in the processing and synthesis of sound signals or generally for any musical and scientific work on sound signals. It is written in Xwindow/Motif to be portable on Unix workstations and be usable across a network. It has already been in use at Ircam for several months on SGI and DEC-Alpha platforms, and is available for external users. Xspect accepts several sound-file formats and offers many facilities in order to precisely adjust displays, analysis, zooming, panning, etc. in any view independently. Xspect can also behave as a visualisation server.

1. Introduction

In computer music, as in the speech field, sound signals are examined, listened to, measured, analysed, processed and synthesized by musicians and scientist. For this type of work, a signal visualisation and analysis program (SVAP) is needed. Many SVAPs have been developed for speech, such as SPIRE [Roads 83] or waves+ (a trademark of Entropic), and for music signals, such as MacMix [Freed 86], Xge [Feiten 94] or Mixview [Cook 90]. For our work at Ircam, we wanted a SVAP with the following properties: portability on different Unix platforms, use of standard graphical libraries, client-server architecture on ethernet, suitability to compare signals, support for arbitrary long signals and different file formats, user adaptability and extensibility, and a reasonable price for a multi-user and multi-machine licence. Having not found a program which could fulfill our prerequisites, we where lead to write a new SVAP, named Xspect.

2. Presentation

Xspect is an interactive graphical program written in ANSI-C, using the standard Xwindow/Motif graphic libraries, Ircam's UDI [Depalle 90] signal processing library and the SVP extended phase vocoder [Depalle 91]. Xspect is developed and tested under DEC Alpha OSF V.3.2 and SGI IRIX 5.3 and could easily be ported on other Unix platforms. At Ircam, using Unix workstation consoles, X-Terminals, MacIntoshes and PC's where an X server is installed, and Audio Servers running on the machines, it is possible to have the following distributed facilities :

run Xspect on any Alpha or SGI processor,
visualize and interact on any console, X-terminal, MacIntosh or PC,
study sound files of any format lying on any disk on any Unix workstation ,
play selected segments of sounds on any sound output on Alpha, SGI and X-Terminals.

3. Main Features

Xspect allows users to play, display, compare and measure signals and break-point functions, to perform different analysis on signals, resulting in new signals which can be displayed, compared and measured in the same way as signals from files.

The largest graphical object of Xspect is the Window (Fig. 1), an X-window with a menu bar and push-buttons. Several Windows can be opened in order to show different graphs or images. The interaction with the program is done through mouse clicks, push-buttons and menu items. Key strokes known as "accelerators" facilitate the use of menu items.

Each Window can be arbitrarily cut horizontally and/or vertically into any number of Frames of any size (Fig. 1). This allows for easy comparison of different signals, or of different channels of a signal, by putting them in frames of same size placed, for instance, one under the other. It also allows, for example, to place a spectrum in a frame to the right of the signal which it comes from. In one given Frame, several graphics can be superposed as if they were traced on transparencies (Fig. 2). Each such graphic is displayed in a View. This means that any number of Views can be created and superposed in a Frame. Each view is identified by one of the text-fields (named Identifier Boxes) piled in the right up corner of the pane. If these Identifier Boxes hinder the image, they can be shrunk or removed. Xspect offers many facilities in order to precisely adjust graphs, analysis, zooming, panning, etc., in any detail in any View independently by use of the so-called Contexts (See section 7, "Contexts").

The signals which are displayed in Views come from files or from operations (e.g. FFT) on other signal segments. When you want to examine a file, Xspect places it in a so called Buffer but uses memory mapping to allow for any file size, only limited by disk space. Xspect manipulates files and buffers "à la" Emacs (in the limits of possible analogies). Xspect maintains a list of Buffers, which can be selected with the mouse, to be examined in Views.

Different types of Marks can be placed and moved on a signal displayed in a view (Fig. 2). A Mark can receive a Label to be displayed at its extremity. One can recall a Mark by its label, in order to display the page around the location of the Mark. Several Marks can appear in a view. One at most is active for interaction at a given moment. Inactive Marks appear in a special rendering such as dashed lines. The horizontal or vertical values pointed at by a Mark can also be displayed. A single Mark is named a Cursor and can point at a certain time in a sound or at a certain frequency in a spectrum and appear as a vertical or horizontal line in a view. A Selection is a couple of Marks on the horizontal or vertical axis, which defines the begin and end values of a Segment of signal. Several selections can be placed in a given view. One at most is active for interaction. A Grid is a set of Marks regularly placed on the horizontal axis and used to point at harmonic partials or at periodic signals. With the mouse, it is possible to stretch a Grid (in order to change the spacing, i.e. the fundamental frequency or the period), or to displace it. The Marks displayed in a view can be saved in an ASCII Mark-file, for example to help for segmenting and processing of a sound. Inversely, a Mark-file can be restored in a view.

4. Analysis

In Xspect, analysis are performed on Segments of signal and produce result signals. FFTs produce spectrum signals. Fundamental frequency (F0) estimation, produces an F0 value It also displays, superimposed on a spectrum, a Grid of Marks at harmonic values of the fundamental frequency. Analysis conditions and parameter settings are chosen in the Context panels (See section 7, "Contexts"). For example, FFTs can be done with various windows such as Hamming, Hanning or Blackman, the magnitude of the FFT can be displayed on a linear or on a logarithmic scale, F0 estimation can be done on a low pass spectrum, etc.. Sliding window analysis are performed by calls to SVP, Ircam's phase vocoder. Extensions are possible through Unix system calls of other programs which take their data on the standard input and write their results on the standard output.

5. Synchronisation of Actions

Very often one has to compare signals and spectra, such as an original and a synthetic signal. Therefore, it is helpful that the Segment of a first signal in a View could automatically be given the same begin and end times than the Segment appearing in a second View whenever this begin or this end is modified. Another typical case is the examination of several channels of a multi-channel signal in different Views. Or you may want to change a selection in a first View containing the energy of a signal and have a second View be Synchronized to display the corresponding portion of the signal just below, aside or in another window.

This Synchronisation mechanism is generalised and allows the user to attach any synchronisable action in a View, to any synchronisable action in another View. Examples of synchronisable items are limits on the horizontal or vertical axis, Cursor or Selection modification and analysis execution. The user can set Synchronisations of Actions between Views by opening Synchronisation panels attached to Views and choosing the actions. Infinite loops which could be created by Synchronisations are detected and avoided.

6. File formats

AIFF format files as well as non-compressed AIFC are recognised and supported. Ircam's sound files (.sf files) also are recognised and supported by Xspect, including short integer (16 bits) and floating point representations from Big and Little Endian Unix machines. For other file formats, Xspect has a raw file facility in which the user can specify short/float representation, Big/Little Endian representation, header size, channel number and sampling rate. These values can be given in a file read at start or interactively during runtime.

7. Contexts

A Context is a structure that contains parameters for a class of features and actions such as Display, Buffer, Analysis, Options, etc.. Each View has a Display Context, a Buffer Context, an Analysis Context, a Scale Context, etc.. When the user opens a context, it appears as a control panel with number boxes for each parameter to allow him for seeing and changing parameter values. For example, the FFT Analysis Context panel lets the user choose the analysis window among Hamming, Hanning, Blackman, etc., as well as a linear magnitude, power or logarithmic sale for the spectrum and several other settings. The Scale Context panel lets the user choose the size of each margin around graphic displays in a View and other settings concerning graphic displays. The Option Context panel lets the user set values for the numerous options available (See section 9, "Options").

8. Other features

Xspect can also behave as a visualisation server. It uses a TCP/IP protocol defined in Ircam's real-time system FTS. Clients, such as FTS, can send to Xspect the data to be displayed. Data received by Xspect is placed in Buffers and shown in Views and can be treated exactly like other signals.

Xspect can also load, edit, manipulate and display Breakpoint functions in a View, to be used, for example as an envelope or for a time-varying parameter given to an analysis. Breakpoint functions can represent other quantities, such as fundamental frequency, that can then be edited by hand.

9. Options

Xspect offers a large number of Options that can be set according to user's preference. This is done in a resource file automatically loaded at the start, or on the command line or interactively at runtime. First, all the Xwindow/Motif resources of the widgets which Xspect is built with can be set in the resource file. This includes colors, sizes, fonts, language, etc.. Secondly, there are other settings more specific to Xspect, such as the way several signals are placed in Windows, Frames and Views, or the margins of the display in a View, or the duration of signal initially displayed when a signal is attributed to a View (if the signal is very long, you may not want all of it to appear in the View), etc.. In the same set of options, warnings, which are helpful for the beginner, can be suppressed for the experienced user, one can change the way zooms are applied, keeping the same center, left or right, etc.. This second set of options can be changed interactively in the Option Context panel.

10. Future developments

Many other features are foreseen. Analysis data is often stored in parameter files. A parameter file resulting from additive analysis contains frequency, amplitude and phase of partials at successive time instants. It would be helpful to display this data in a View of Xspect, or similar data such as filtering data, formant data, and many others from analysis and for synthesis programs. There is also a need for editing facilities for these data, with powerful editing features such as using a sort of a "soft cursor", i.e. a sort of a virtual rubber ball for pushing sinusoidal partials or spectral envelopes in one direction or another. Even more ambitious developments would use all the new graphic tools which have been encouraged by A. Freed in [Freed 95]. This could be done by using GLX, a library of Open-GL extensions of X. The so-called X-Visuals can be given to Open-GL calls for powerful 2-D and 3-D drawings, rendering and manipulations.

References

[Roads 83] A Report on SPIRE: An Interactive Audio Processing Environment, C. Roads, Comp. Music Journal, Vol. 7, No. 2, Summer 1983.

[Freed 86] MacMix: Mixing Music with a Mouse, A. Freed, Proc. ICMC 1986, The Hague, Oct. 1986.

[Freed 95] Improving Graphical User Interfaces for Computer Music Applications, A. Freed, CMJ Vol. 19, No. 4, Winter 1995, pp. 4-8.

[Cook 90] Mixview: A Portable, Graphic-Based Soundfile Editor and Processor, D. Cook, Proc. ICMC, Glasgow, U.K., Sept. 1990, pp. 262-265.

[Feiten 94] A Modular Construction Set for Time-Domain Editors, B. Feiten & M. Spitzer, Proc. ICMC, Aarhus, Denmark, Sep. 1994, pp. 284-285.

[Depalle 90] UDI: A Unified DSP Interface for sound signal analysis and synthesis, P. Depalle & X. Rodet, Proc. ICMC, Glasgow, U.K., Sept. 1990.

[Depalle 91] SVP: A Modular System for AnalysisProcessing and Synthesis of Sound Signals, P. Depalle & al., Proc. ICMC, Montreal, Oct. 91.

Fig. 1: Two Windows opened by Xspect and several Frames in a Window

Fig. 2: Several Views in a Frame