Summary of the impact

Research by the University of Huddersfield has made a significant contribution through the development of state-of-the-art, modular, open-source software used in the creation and enhancement of electronic music. The HISSTools Impulse Response Toolbox allows users to deploy custom convolution-related solutions specific to their needs rather than having to rely on fixed and therefore inherently limited options, as was commonly the case previously. Its deliberately musician-centric approach has been acknowledged via international commercial adoption, including integration into a world-leading product with a user base of 1.7m and a crucial role in the design of concert halls by a global firm of engineering consultants.

Underpinning research

Convolution has become a key element of digital music-making in the sampling age. One significant appeal of the technology is that it allows the reverberant behaviour unique to a certain acoustic space — for example, a room, a recording booth or a concert hall — to be captured and recreated. It can also be used to apply filters and in other more creative applications for sound design. The original convolution reverbs were hardware boxes that, because of their expense, were confined almost exclusively to studios. The technology is now commonly available as affordable computer software, but technical limitations have constrained utility.

Work in this field by the University of Huddersfield's Department of Music can be traced back to 2008, when Dr. Pierre Alexandre Tremblay (Senior Lecturer, 2005-2010; Reader in Composition, 2010-2013; Professor, 2013-present) began the research project Thinking Inside the Box (TITB), which aimed to improve compositional judgment in mixed music by allowing a composer to emulate the sound of a concert hall in a studio. Tremblay took a number of impulse responses for various combinations of loudspeaker types and speaker/listening positions to generate highly realistic emulations of a concert space using convolution. The project investigated how this process might allow composers to better judge their work in relation to the final concert presentation and to actively reflect on suitable loudspeaker choices and placement. This is further explained in reference output [1] and is part of the compositional process for output [5].

It was the exploration of the complementary approach to TITB that provided the initial impetus for the HISSTools Impulse Response Toolbox (HISS = Huddersfield Immersive Sound System).

In 2011 Dr. Alex Harker (Research Fellow, 2011-2012; Lecturer, 2012-present) joined Huddersfield to work on the HISSTools project and released a package of 82 externals for the digital music software system MaxMSP. These addressed a range of creative and technical problems in a modular fashion. Several of Harker's externals were based on innovative spectral processing techniques, including convolution. Rather than limiting the use of convolution techniques to emulating reverb, Harker's approach allows a more open set of applications.

With the HISSTools, Tremblay and Harker set about achieving results closer to the sound of a studio within a concert hall environment. This involved measuring and inverting the room/loudspeaker impulse response to correct the frequency response to more closely match the studio listening environment. A similar approach was required to improve the sound quality of the close-proximity microphone positions most suitable for live use. Rather than a purely technical improvement, one of the key aims of the research methodology was to have a demonstrative impact on the effective translation of musical ideas from the studio to the concert hall.

Providing a single tool for these kinds of inversion/correction problems was not felt to be the most beneficial output format for the project, as the sub-problems inherent in the task are relevant to a range of potential applications of much larger scope, both creative and pragmatic. The notion of a modular and reusable `toolbox' was therefore conceived, opening up the toolbox to a much wider set of creative and technical applications.

The resulting software offers well-encapsulated objects (or modules), each dealing with a specific convolution-related task or problem as explained and disseminated in output [3]. Many are relevant to speaker/mic/room correction, while others have more general applications. The toolbox brings together a core set of pre-existing algorithms in a single package that has been practically evaluated for musical use. It is the modularity and flexibility of the toolbox that has allowed for its deployment in a range of musical applications and allows users to tailor solutions to context-specific needs.

The research process also brought about innovative approaches through the novel combination of algorithms. For example, the standard method of retrieving an impulse response from a recorded swept sine signal (typically the ESS or exponentially swept sine) involves convolving the result with an analytic inverse sine sweep, in order to achieve the necessary deconvolution. Due to the bandlimited nature of the sweep signals, the result of this process is implicitly filtered by a linear phase bandpass filter, with equal temporal distribution of filter ring (both pre and post ringing). In the HIRT the deconvolution is carried out in the frequency domain using a large FFT, allowing a refactorisation of the equation that gives control over the phase of the implicit filter. Thus, by using a minimum (or almost minimum) phase filter, the sharpness of measured impulse responses can be maintained, and important frequency domain information retained when the beginning of the impulse is trimmed. Tremblay used the software as a testing ground for his own compositions throughout the course of his research (see reference output [6]).

References to the research

Outputs — publications:

Outputs — compositions:

Grants

AHRC: Thinking Inside the Box: A New Integrated Approach to Mixed Music Composition and Performance, October 2008 to April 2009 - £15,932;
University Research Fund Grant: HISSTools development, September 2011 to July 2012 - £68,000;

SRIF: Thinking Inside the Box: A New Integrated Approach to Mixed Music Composition and Performance, October 2008 to April 2009 - £132,000.

Details of the impact

The HISSTools Impulse Response Toolbox has helped make high-level technology available to mid-level practitioners and has become an integral component of the work of major international companies, organisations and independent research centres involved in the creation and reproduction of state-of-the-art sound. The flexibility and module design of the Toolbox has been demonstrated by its uptake in a range of commercial and artistic contexts. These range from music production for new musical compositions at SudWestDeutscherRundfunk to the architectural consultants ARUP.

In 2012 Berlin-based software company Ableton, a world leader in its field, approached Harker to develop a new convolution reverb device for the Max for Live environment to run within the firm's flagship Live 9 software, which has an international user base of approximately 1.7m. This collaboration was initiated on the basis of an early version of the HISSTools Impulse Response Toolbox that was presented at both the Cycling '74 Expo 2011 (a non-academic conference run by Cycling '74, the San Francisco company behind MaxMSP) and an event staged by NK (an artist-run, independent, non-profit organisation) in Berlin in early 2012.

The resultant commercial devices — two reverb devices and an impulse measurement device — were made available as part of the downloadable Max For Live Essentials pack, which can be used by any owner of Max for Live (now part of the `Suite' version of Live 9). The devices are dependent on the HISSTools Impulse Response Toolbox for core functionality and, due to the co-development of these projects, some features beneficial to both the reverb devices and the extended applications of the software.

The devices offer an unprecedented level of accessibility for the end user to modify and adapt them to their needs. Rather than presenting the user with a closed-source and fixed software package they are open-source and fully customisable devices. The reverb devices also offer specific novel features that depend on the flexibility and comprehensive feature set of the HISSTools Impulse Response Toolbox. These include the ability to use different sampled spaces for the early and late parts of the reverb, offering an enhanced level of flexibility in the design of virtual spaces, and real-time control over virtual positioning within the space. The partitioning of impulse responses into early and late parts within the reverb devices is also based on analysis, rather than determined by fixed time periods as is currently implemented by many commercially available plug-ins. This more accurately represents the duration and hence sound of each part.

In March 2013, shortly after Live 9 Suite's release, Ableton produced a video promoting the convolution reverb facility, highlighting the ability to "use the sound of real spaces and tweak them as you like". This was been viewed over 50,000 times on YouTube. Another video on the company's YouTube channel shows producer and DJ Robert Lippok discussing Live 9's new features and praising the convolution reverb, observing that users had been waiting for just such a facility. In addition, a number of third-party online tutorial videos have received a total of more than 7,500 views, while Sound on Sound magazine described the convolution reverb as "the most impressive new Max for Live device".

In March 2013 a collaborative project was started with boutique guitar amp manufacturer MATAMP, of Meltham, near Huddersfield. This project focused on developing custom measuring software based on the HISSTools Impulse Response Toolbox. This allows a faster and more accurate method of evaluating amplifier and equalisation circuit design than was possible using the company's previous toolset. Earlier techniques relied on taking a set of measurements at fixed frequencies for a given system, requiring a lengthy set of manual observations to gain only a partial impression of the frequency response of a circuit. Using HISSTools allowed an almost immediate measurement that could monitor a system in close to real-time, making it possible to store and catalogue accurate sets of measurements for design and product specification purposes.

The SudWestDeutscherRundfunk studio is using the HISSTools IR Toolbox for a number of productions including major commissions by Dai Fujikura and Daniel Peter Biro. The SWR is renowned for its work with innovative tools, having developed unique resources for composers such as Luigi Nono stretching back over forty years. The Artistic Director of the studio Detlef Heusinger explains that Harker and Tremblay's "work on convolution and especially the non commercial availability of the flexible "HISSTools" opened up a new step in the musical application of convolution...The modular construction with the inherent flexibility and the multichannel options are features making the "HISSTooIs" superior to any other commercially available convolution software." [10]

ARUP (New York), a global firm of consulting engineers, designers and planning managers, have been using the HISSTools toolbox in the context of their architectural acoustics lab. The real time convolution objects have been used for a real-time auralization system that they have been developing for simulating live room responses for musicians and speakers so that clients can hear what a concert hall (or other such space) will sound like prior to construction. More recently they have been looking into using these objects in the context of room equalization. For Senior Consultant Terence Caulkins and colleagues at ARUP the HIRT "has now become an integral part of our workflow for acoustic auralization. Recent developments of this toolbox have started to make their way into how we conduct the calibration and fine tuning of our SoundLab installations." [9]

The Norwegian Centre for Technology in Music and the Arts (NoTAM) recently chose the HISSTools Impulse Response Toolbox for a major project to capture the impulse responses of 60 venues in Oslo to create a "sonic map" of the city and an archive of high-quality impulse responses for use in musical and post-production contexts. NoTAM evaluated a number of potential approaches before selecting HISSTools on the strength of its modularity and its suitability for easily capturing and post-processing multi-channel impulse resposnses. NoTAM's work is recognised as of the highest quality, as illustrated, for example, by the award of a Spellemann Prize — Norway's Grammy equivalent — to Håkon Thelin's album Light, which was recorded, mixed and mastered by NoTAM sound engineer Cato Langnes. The artistic director of NoTAM Notto Theile writes: "Not only does the tool kit offer the possibility of capturing a large amount of channels; it can be done quickly and the results could be checked while in the venue. This is extremely important in order to attain an efficient workflow when embarking on capturing lRs from the large amount of spaces we are planning to. On top of that, the quality of the convolution reverb achieved with HISSToois exceeds that of Altiverb." [8]

Sources to corroborate the impact

1. Software Toolbox for MaxMSP environment (and source code): Harker, A, and Tremblay, PA (2012): The HISSTools Impulse Response Toolbox
   http://eprints.hud.ac.uk/id/eprint/14897
2. HISSTools Impulse Response Toolbox online tutorial video
   http://vimeo.com/55440630
3. Ableton Live 9 online advert
   http://www.youtube.com/watch?v=2xbf7KbqDEM
4. Producer and DJ Robert Lippok discusses convolution reverb's contribution to Live 9
   http://www.youtube.com/watch?v=ukQXMfJ-tRc (2.10 onwards)
5. Matt Jackson, Product Designer, Ableton AG
6. Detlef Heusinger, Artistic Director, Südwestrundfunk
7. Jeff Lewis, Owner, MATAMP
8. Notto Thelle, Director, NoTAM
9. Terence Caulkins, Senior Consultant, ARUP