The SALAMI project will generate five high-impact research products:
A web-accessible collection of structural analysis files for ~350,000 pieces of music audio. Music scholars, MIR researchers, students, and the general public alike should find the results of our structural analyses useful and inspirational for their research and personal explorations since nothing of this magnitude has ever been done before. The analytical results will be available via user-friendly facilities (and programmatically via NEMA’s REST APIs, Webservices, and as ‘linked data’) that will allow searching and browsing by structural type, genre, period, composer, etc. The SALAMI system will also afford user-specified “subcollection” building for future, secondary analyses of the data. Because each musical piece will be analyzed using a variety of tools and algorithms, there will be multiple (many-to-one) structural “views” of the music, based upon, for example, a work’s harmonic, rhythmic, timbral, and/or temporal facets. Each of the views will be accompanied by provenance information that describes how the analysis came into being so that new audio data can be analyzed in a consistent manner. Our underlying data store will be based upon the Resource Description Framework (RDF), a modularised Music Ontology, and linked data models (see Sec. G) to ensure maximum interoperability with other Webservices and resources both within and beyond NEMA.
Interoperable file formats and ontology for music structure that can be used by a wide variety of visualization software and other applications. SALAMI will develop a standardized ontology for music structure from which will be derived the specific data models used in a variety of available open-source music visualization software packages (e.g., Sonic Visualiser [1], Audacity [2], Variations Timeliner [3], etc.), building on work already underway in the community. Thus, music scholars and the general public will be able to interact with structural data along with its source audio in hitherto unrealized ways. For example, users could go directly to specific sections of the music (e.g., chorus, recapitulation, or coda) that they are interested in and listen to them. Similarly, audio engineers could quickly move around while editing recordings, without manually labelling sections or relying on timing information. Because our work will include the creation of SALAMI VAMP “plug-ins” [4] that will allow Webservice communications between SALAMI’s vast store of structural analysis files and the visualization software, scholars creating thematic indices or incipits for large collections of music can use the SALAMI data and its plugins to simplify their tasks.
Open source structural analysis software/services well trained on a large ground-truth dataset. The SALAMI analytic algorithms will be trained and tested against the structural ground-truth dataset to be created at McGill’s Schulich School of Music. This will help ensure the validity of the resulting structural analysis files. The analytic software being developed will be made available for interactive use as a part of the larger NEMA Webservice infrastructure. However, because the SALAMI software code will be released under an open-source regime and also made available via the NEMA code repository, other researchers can take the code and develop their own stand-alone applications as they see fit. The SALAMI partners have a track record in developing and sustaining community software.
An open source ground-truth structural dataset consisting of thousands of pieces, spanning a variety of musics, and verified by highly trained musicians. Because of the cost involved in creating high-quality ground-truth datasets, such datasets are extremely valuable to the research community for training and evaluating machine-learning algorithms. The ground-truth dataset that we will be creating at McGill to build and test our SALAMI algorithms will be made available for use by other MIR and CM researchers so they can create or improve their own structural analysis software. The CM researchers will also be able to add new ground truth using the tools and ontologies that we will develop.
Sample exemplar experiments to demonstrate the potential use of the analytic data. For the SALAMI project to be truly useful, it must be more than the creation and collection of a vast store of individual structural analysis files. SALAMI must demonstrate to the world some of the interesting analyses that can be performed so as to inspire others to explore this unique resource. To this end, we will conduct several exemplar experiments/explorations that ask questions about large subcollections of the SALAMI collection. We could, for example, perform some large-scale clustering analyses to explore for geographic or time period tendencies in the data. We could also undertake some supervised learning experiments to see whether structural information is useful in various classification tasks (e.g., genre, mood, composer, etc.). It is our intention to encapsulate our exemplar experiments as Meandre data flows (see Secs. E and F) and present them as NEMA Webservices so others might replicate or modify our examples. We also plan to disseminate the actual findings of our experiments in such venues as ISMIR (International Conferences on Music Information Retrieval), Digital Humanities, Computing in Musicology, etc. to build up interest in the possibilities afforded by the SALAMI data and services.
See http://www.sonicvisualiser.org/. Sonic Visualiser is being developed and supported by NEMA partner (see Sec. E), Centre for Digital Music, Queen Mary, University of London. See Appendix A: Fig. 2 for a screenshot.
See http://audacity.sourceforge.net/. See Appendix A: Fig. 3 for a screenshot.
See http://variations.sourceforge.net/vat/. Prof. Eric Isaacson, a Variations2 Project Co-PI and a developer of the Variations Timeliner, sits on the SALAMI Advisory Board.