Chapter 20

A plugin is an external library that is dynamically loaded when an object that need it is created. The system looks for plugins in the following locations:

Additionaly, a user path can be set, where the system will look for plugins in first position. See section 8.5.4 p.83.

The plugins are shared libraries which extension is platform dependent. The plugin name should not include the extension. The expected extensions are the following: .dylib on MacOS and Linux, .dll on Windows.

20.1 FAUST plugins

FAUST [Functional Audio Stream]1 is a functional programming language specifically designed for real-time signal processing and synthesis. A FAUST/INScore architecture allows to embed FAUST processors in INScore, for the purpose of signals computation. A FAUST plugin is viewed as a parallel signal and thus it is created in the signal address space. Similarly to signals, it is associated to an OSC address in the form /ITL/scene/signal/name where name is a user defined name.

20.1.1 Set Message

There are two ways to create a FAUST Processor :

20.1.2 Specific messages

A FAUST processor is characterized by the numbers of input and output channels and by a set of parameters. Each parameter carries a name defined by the FAUST processor. The set of messages supported by a FAUST processor is the set of signals messages extended with the parameters names and with specific query messages.

faustmessage PICT

Querying a FAUST processor input and output count:

/ITL/scene/signal/myFaust get in out;

  gives as output:

/ITL/scene/signal/myFaust in 2;
/ITL/scene/signal/myFaust out 4;

Modifying the value of a FAUST processor parameter named volume:

/ITL/scene/signal/myFaust volume 0.8

20.1.3 Feeding and composing FAUST processors

A FAUST processor accepts float values as input, which are taken as interleaved data and distributed to the input channels.

From composition viewpoint, a FAUST processor is a parallel signal which dimension is the number of output channels. Thus, a FAUST processor can be used like any parallel signal. However, the signal identifier defined in 14.1.2 is extended to support adressing single components of parallel signal as follows:

signal PICT

where n selects the signal #n of a parallel signal. Note that indexes start at 0.

Creating 3 parallel signals using the 3 output channels of a FAUST processor named myFaust:

/ITL/scene/signal/y1 set ’myFaust/0’ 0.01 0. 1. 1. 1. ;
/ITL/scene/signal/y2 set ’myFaust/1’ 0.01 0.5 1. 1. 1. ;
/ITL/scene/signal/y3 set ’myFaust/2’ 0.01 -0.5 1. 1. 1. ;

20.2 Gesture Follower

INScore supports gesture following using the technology developed by the IRCAM IMTR team. These features are available as a plugin that is included in the INScore distribution (version 1.03 or greater) or available from the IRCAM.

20.2.1 Basic principle

Gesture following is provided as a mean to interact with a score. From input viewpoint, the gesture follower is similar to signals (see section 14.1.1 p.116): it accepts data stream as input both in learning and following modes. It implements a specific set of events related to gesture following and can generate message streams parametrized with the gesture follower current state.

A gesture follower is setup to handle a given count of gestures, which are actually denoted by streams of float vectors. We’ll refer to the size of the float vector as the gesture dimension. For example, the dimension of a gesture captured from x, y and z accelerometers is 3.

A gesture follower operates in two distinct phases: a learning phase where it actually stores the gestures data, and a following phase where it tries to match incoming data to the stored gestures data. When not learning nor following, we’ll talk of an idle phase.

In the following phase, the system maintains a list of likelihood for the learned gestures, a list of positions in the gestures and a list of speeds representing how fast the gestures are made. Of course, the higher the likelihood, the more these data are meaningfull. It’s the user responsability to decide on the meaningfull likelihood threshold value. Interaction events are triggered only in the following phase and for meaningfull likelihoods.

20.2.2 Messages

A gesture follower is created in a scene using the imtrgf type. It has a graphic appearance that may be used for debug purpose but it is hidden by default.

gesturefollower PICT

The parameters are:

A gesture follower is created with a fixed count of gestures that can be learned and decoded. These gestures are named gestures and can be addressed at /ITL/scene/myfollower/gesturename where the part in italic are user defined names and where myfollower is a gesture follower.

gesturefollower PICT

Creating a gesture follower for 3 dimensional data and a typical learning sequence:

/ITL/scene/gf set imtrgf 3 1000 gestureA gestureB gestureC gestureD ;
/ITL/scene/gf learn gestureA ;
/ITL/scene/gf 0.1 0.5 -0.2 ... 0.7; ! the data size must be a multiple of 3
/ITL/scene/gf stop;

20.2.3 Gestures management

Messages can also be sent to gestures i.e. to addresses in the form /ITL/scene/myfollower/gesturename where myfollower is a gesture follower.

A gesture could be in two states:

gesture PICT

Gestures supports also specific queries :

gestureget PICT

20.2.4 Events and interaction

Events are defined at gesture level and events management messages should be addressed to gestures.

gestureevents PICT

A message associated to a gesture supports the following specific variables:

gesturevariable PICT

These variables support the scaling feature associated to position variables and described in section 16.5.1 p.167.

Variables described in section 16.5 p.167 may also be used but they are meaningless and contains default values.

20.2.5 Gesture Follower Appearance

A gesture follower object has a graphic appearance and supports all the standard objects properties, including mapping and synchronization. This graphic appearance is provided mainly for debug purpose and by default, the object is hidden. Figure 20.1 shows the gesture follower appearance in its different phases:


Figure 20.1: The gesture follower appearance in its different phases.

20.3 Httpd server plugin

INScore can embed Http server to expose real time screenshot image of a scene to the web. This feature is based on libmicrohttpd 2 and is available as a plugin that is included in the INScore distribution (version 1.11 or greater). The Url to get the image is the base url of the server.

20.3.1 Set Message

The http server object is created in a scene like other objects and served image of his scene.

httpdserver PICT


/ITL/scene/server set httpd 8000;

If the http port is already used, the server cannot start.

20.3.2 Specific messages

The http server status can be delivered with a specific message.

httpdmessage PICT

A string corresponding to the server status ("started" or "stopped") is return.


/ITL/scene/server get status;