Abstract or Description

A methodology for deploying interactive machine learning and audio tools written in C++ across a wide variety of platforms, including web browsers, is described. The work flow involves development of the code base in C++, making use of all the facilities available to C++ programmers, then transpiling to asm.js bytecode, using Emscripten to allow use of the libraries in web browsers. Audio capabilities are provided via the C++ Maximilian library that is transpiled and connected to the Web Audio API, via the ScriptProcessorNode. Machine learning is provided via the RapidLib library which implements neural networks, k-NN and Dynamic Time Warping for regression and classification tasks. An online, browser-based IDE is the final part of the system, making the toolkit available for education and rapid prototyping purposes, without requiring software other than a web browser. Two example use cases are described: rapid prototyping of novel, electronic instruments and education. Finally, an evaluation of the performance of the libraries is presented, showing that they perform acceptably well in the web browser, compared to the native counterparts but there is room for improvement here. The system is being used by thousands of students in our on-campus and online courses.

Item Type:

Conference or Workshop Item
(Paper)

Additional Information:

This work was partially funded under the HEFCE Catalyst Programme, project code PK31.The research leading to these results has also received funding from the European Research Council under the European Union's Horizon2020 programme, H2020-ICT-2014-1 Project ID 644862