If the SALAMI project were a song, you could say we've reached the "intro" part! Last Thursday, May 6th, we held an audition/training session with several students to explain the proposed annotation procedure and to collect feedback. In analyzing pieces to produce ground truth, one always seeks out opportunities to get a second opinion; it was a special occasion, then, to have nine people to share their points of view!
Agreeing on an annotation procedure is a key component in our project right now: the guidelines we set will soon direct people's independent work as they begin to annotate roughly 3000 songs. Data collection of this sort has not been done before on this scale, nor with this variety of music.
You can access a PDF of the initial annotator's guide we produced. (Note that although the guide contains no citations, the procedure as described owes a debt to a recent article by Geoffroy Peeters and Emmanuel Deruty.) Also available is a PDF of the training session wrap-up document (which refers to a homework assignment), but the links to the music examples have been removed.