Finnput

Finnput is a tool to let Finn control a computer with primitive sounds. My 2-year-old daughter Thura made a short video tutorial.

Prerequisites

Before downloading Finnput some external software components need to be installed first:

Finnput has been tested on a Windows XP machine but should also work on any other platform that supports the used software components.

Download, installation and start

Download the latest version from here and unzip all files into one single folder. Start the program by double-clicking the Finn-Gui.py file.

Operation

After starting Finn-Gui.py the default sound method uses Finn's specific voice characteristics. In the current version the following sound triggers can be used:

Activate means that the application is started that is associated with the file type behind the selected picture. If, for example, the selected item is a movie, the movie is started with the default movie player associated with the particular file type (e.g. RealPlayer for avi/flv/mov file extensions). Every possible file type can be used as long as the file type is associated with some program (you can even load web site pages or start .exe programs). This can be tested with File Explorer: just double click the file (or shortcut) and see if a program starts. Otherwise use File Explorer to associate the file type with a specific program.

The only special file extension is '.xml'. This extension is used to store which pictures are visible and what files are behind those pictures. These files are called 'visual settings'  and can be loaded via the 'File->Load visual settings...' menu item (format is described in section ' visual settings format').  If a picture is selected with a '.xml file' behind it,  the file is loaded in the same way as via the 'File->Load visual settings...' menu item. In this way it is possible to create hierarchical menu structures that can be fully controlled with sound input.

I've used an open source tool Csound for sound processing. Different configurations for sound processing can be loaded from within the program via the 'File->Load sound processing settings...'. These configurations have a separate window with input sliders for controlling sound parameters. There's currently no way to save parameter values for later use other than manually edit the '.csd' file.

The defaults configuration for sound processing is stored in the file 'finn-pitch.csd'. I've tuned the filter parameters in the 'finn-pitch.csd' based on the recordings you've sent. These parameter settings might be depending on the type of microphone, soundcard and soundcard settings.  If you load 'finn-pitch-recorded.csd' it will use the file 'FinnAhhAndOpen.wav' instead of the microphone input. With this file you should see advance, activate and end events. With the 'finn-pitch.csd' and your own microphone it might be necessary to adapt one or more of the three filter parameters in the Csound window. If the right parameters are found you can change them permanently in the '.csd' file.

I've added a record function. You can activate this function via the 'Tools->Record' menu or the 'Record' button in the status bar. When activated 2 files are created in the  same folder as where Finn-Gui.py is located. The names of the files start with the date and time when the recording start. The one with the '.wav' extension contains the recorded audio and the one with the '.log' contains some internal diagnostics of the program. If you sent me these files after a 'training session' with Finn, I might be able to improve the program.

Visual settings format

Visual settings files are .xml files with the following structure:

<finnput_config folder="">
	<items>
		<item picture="" entersound=".wav" actionfile=""/>
		...
		<item picture="" entersound=".wav" actionfile=""/>
	</items>
</finnput_config>

folder contains the path to a folder where all files belonging to this configuration need to be stored. If the folder doesn't contain a drive letter the path is relative to the folder where Finn-Gui.py is located. folder="." means that all pictures, sounds and actionfiles need to be stored in the same folder where Finn-Gui.py is located.

Each <item> contains one visual menu entry containing:

 <picture> : filename of a jpeg picture file in the folder specified by <folder>. The picture file is used for building the visual menu. The current version does not scale a picture to the actual screen size.

<entersound> : filename of a wav sound file in the folder specified by <folder>. The sound file is played when the corresponding menu item becomes active (receive focus).

<actionfile> : filename of a file in the folder specified by <folder>. The file is 'opened' when the corresponding menu item is activated. If the filename has an .xml extension it is assumed to be located in the same folder as where Finn-Gui.py is located. Such a file is loaded as visual settings file causing a rebuilding of the visual menu.

When Finn-Gui.py is started it loads the visual settings file 'FinnDefaultConfiguration.xml' from the same folder as where Finn-Gui.py is located. Please study this file in order to make you own visual settings files.

Some practical hints