As the annotation portion of the SALAMI project recently passed its halfway point, now may be a good time to look at some figures to take stock of how far along we are.

So far, we have two annotations for each of 267 songs, accounting for roughly half of the 981 annotations that have been submitted so far. We are currently aiming to produce two annotations for each of 2000 songs, which means we still have almost three quarters of our work ahead of us. However, our annotators have boosted their output in the last couple weeks and they appear to be willing to at least keep up the current pace through the end of August.

Our 8 annotators have so far worked for 543 hours (roughly 40 of these having been devoted to training and meetings in the earlier weeks). Of course, each has been working a different number of hours: the total number of annotations produced by each annotator currently ranges from 34 to 201, with the average annotator having produced around 120.

As for how long each annotation has been taking, I calculated the average annotation time as a function of when the annotation was submitted to obtain the following, promising graph, which shows that after a brief learning curve, the average annotation time has been tapering off.


Image icon annotation_time_figure.jpg

Add a comment

Next Post Previous Post