pith. sign in

arxiv: 2605.22255 · v2 · pith:RCYBR5VMnew · submitted 2026-05-21 · 💻 cs.CV · cs.IR

Direct content-based retrieval from music scores images

Noelia Luna-Barahona , Antonio R\'ios-Vila , F\'elix Fuentes-Hurtado , David Rizo , Jorge Calvo-Zaragoza This is my paper

Pith reviewed 2026-06-30 17:20 UTC · model grok-4.3

classification 💻 cs.CV cs.IR
keywords content-based retrievalmusic scoresoptical music recognitiontransformerdomain shiftinformation retrievalscore images
0
0 comments X

The pith

OMR-based methods retrieve music scores more accurately in the same domain while transcription-free models handle changes in score style better.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper investigates content-based retrieval directly from images of music scores rather than using metadata like titles. It introduces a systematic way to create query sets from existing annotated music corpora and compares several retrieval strategies. These include pipelines that first use optical music recognition to transcribe the scores, a transformer model that matches queries to images without transcription, and a large language model prompted with text. Tests on four corpora that differ in size, image quality, and how the music is typeset show that transcription-based methods work better when the query and the collection come from similar sources, but methods that skip transcription are more robust when the sources differ.

Core claim

OMR-based pipelines achieve higher in-domain retrieval, whereas transcription-free models handle domain variability more effectively across four corpora with diverse characteristics in dataset size, image quality, and typesetting mechanisms.

What carries the argument

The systematic method to build query datasets from any annotated corpus, used to evaluate OMR-based, transcription-free Transformer, and text-prompted LLM approaches for content-based search in music score images.

Load-bearing premise

The four corpora together with the query datasets constructed from them capture the important variations found in real-world music score image searches.

What would settle it

Evaluating the same methods on a fifth corpus with a new combination of image quality and typesetting style that produces opposite performance rankings between OMR and transcription-free approaches.

read the original abstract

The digitization of musical scores plays a crucial role in their preservation and accessibility, yet information retrieval still depends mainly on metadata searches, such as by title or composer. Content based search in music score images remains underexplored compared to text documents, despite its potential value for musicians, musicologists, and educators. This work contributes to the field by first studying which characteristics of a score are most relevant for search and by defining a systematic method to build query datasets from any annotated corpus. We also consider diverse methods for content-based search on music score images, ranging from transcription-based approaches relying on Optical Music Recognition (OMR), to a transcription-free Transformer model trained to recognize queries directly from score images, and a text-prompted Large Language Model. Our experiments evaluate these models on four corpora exhibiting diverse characteristics in terms of dataset size, image quality, and typesetting mechanisms. Overall, each method excels under different conditions: OMR-based pipelines achieve higher in-domain retrieval, whereas transcription-free models handle domain variability more effectively.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript studies content-based search in music score images, proposing a systematic method to build query datasets from annotated corpora. It compares transcription-based approaches using Optical Music Recognition (OMR), a transcription-free Transformer model trained directly on score images, and a text-prompted Large Language Model. Experiments on four corpora with diverse characteristics lead to the conclusion that OMR-based pipelines achieve higher in-domain retrieval performance, while transcription-free models are more effective at handling domain variability.

Significance. If the results hold with supporting quantitative evidence, the work would be significant for music information retrieval by providing a comparative analysis of transcription-based versus transcription-free paradigms and by introducing a reusable method for query dataset construction from annotated corpora.

major comments (2)
  1. [Abstract] The abstract states high-level outcomes regarding performance differences between OMR-based and transcription-free methods but supplies no metrics, error bars, dataset sizes, or experimental details. This prevents verification that the reported performance differences support the central claim about in-domain vs. domain variability handling.
  2. [Abstract and experimental evaluation of corpora] The claim that transcription-free models handle domain variability more effectively rests on the four corpora exhibiting sufficient, real-world-relevant variability. The abstract describes them as exhibiting 'diverse characteristics in terms of dataset size, image quality, and typesetting mechanisms' but provides no quantitative measures of shift (e.g., image statistics, OMR error divergence, or typesetting differences), leaving open whether cross-corpus results test variability handling or compare four particular collections.
minor comments (1)
  1. [Abstract] The abstract could be strengthened by including at least one key quantitative result (e.g., a retrieval metric on one corpus) to give readers an immediate sense of effect size.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment point by point below, indicating the revisions we will make to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] The abstract states high-level outcomes regarding performance differences between OMR-based and transcription-free methods but supplies no metrics, error bars, dataset sizes, or experimental details. This prevents verification that the reported performance differences support the central claim about in-domain vs. domain variability handling.

    Authors: We agree that the abstract would benefit from greater specificity. In the revised version we will incorporate key quantitative results, including mean average precision values for the main in-domain and cross-corpus experiments, the sizes of the four corpora, and a brief reference to the reported variability (error bars) where they appear in the full evaluation. revision: yes

  2. Referee: [Abstract and experimental evaluation of corpora] The claim that transcription-free models handle domain variability more effectively rests on the four corpora exhibiting sufficient, real-world-relevant variability. The abstract describes them as exhibiting 'diverse characteristics in terms of dataset size, image quality, and typesetting mechanisms' but provides no quantitative measures of shift (e.g., image statistics, OMR error divergence, or typesetting differences), leaving open whether cross-corpus results test variability handling or compare four particular collections.

    Authors: We accept that quantitative characterization of the domain shifts would make the argument more robust. We will add a new subsection (or expand the experimental setup) that reports measurable differences across the corpora, such as image-resolution and contrast statistics, average OMR symbol-error rates when transcription is performed, and basic typesetting-variability indicators. These additions will clarify that the cross-corpus protocol evaluates genuine variability rather than merely four fixed collections. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical evaluation on external corpora

full rationale

The paper reports an empirical comparison of retrieval methods (OMR pipelines, transcription-free Transformer, text-prompted LLM) across four distinct corpora. No equations, derivations, parameter fits, or self-citation chains appear in the provided abstract or description. Performance claims rest on direct experimental measurements rather than any reduction of outputs to inputs by construction. The central result (differential behavior under in-domain vs. cross-domain conditions) is falsifiable against the stated corpora and does not rely on self-definitional steps or load-bearing self-citations.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities are described in the abstract.

pith-pipeline@v0.9.1-grok · 5722 in / 918 out tokens · 23634 ms · 2026-06-30T17:20:43.932623+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    In: International Conference on Theory and Practice of Digital Libraries, pp

    Kurth, F., Damm, D., Fremerey, C., M¨ uller, M., Clausen, M.: A framework for manag- ing multimodal digitized music collections. In: International Conference on Theory and Practice of Digital Libraries, pp. 334–345 (2008). Springer

    2008

  2. [2]

    ACM Computing Surveys (CSUR)53(4), 1–35 (2020)

    Calvo-Zaragoza, J., Jr, J.H., Pacha, A.: Understanding optical music recognition. ACM Computing Surveys (CSUR)53(4), 1–35 (2020)

    2020

  3. [3]

    IEEE transactions on knowledge and data engineering11(1), 81–93 (1999)

    Yoshitaka, A., Ichikawa, T.: A survey on content-based retrieval for multimedia databases. IEEE transactions on knowledge and data engineering11(1), 81–93 (1999)

    1999

  4. [4]

    In: Proceed- ings of the 2016 ACM on International Con- ference on Multimedia Retrieval, pp

    Schedl, M.: The lfm-1b dataset for music retrieval and recommendation. In: Proceed- ings of the 2016 ACM on International Con- ference on Multimedia Retrieval, pp. 103–110 (2016)

    2016

  5. [5]

    In: 2017 21st International Conference Informa- tion Visualisation (IV), pp

    De Prisco, R., Esposito, A., Lettieri, N., Malandrino, D., Pirozzi, D., Zaccagnino, G., Zaccagnino, R.: Music plagiarism at a glance: metrics of similarity and visualizations. In: 2017 21st International Conference Informa- tion Visualisation (IV), pp. 410–415 (2017). IEEE

    2017

  6. [6]

    In: Proceedings of the 10th International Conference on Digi- tal Libraries for Musicology

    Crawford, T., Lewis, D., Porter, A.: Exploring early vocal music and its lute arrangements: Using f-tempo as a musi- cological tool. In: Proceedings of the 10th International Conference on Digi- tal Libraries for Musicology. DLfM ’23, pp. 77–81. Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3625135.3625142 . https:...

    work page doi:10.1145/3625135.3625142 2023

  7. [7]

    Expert Systems with applications238, 121776 (2024)

    Ramoneda, P., Jeong, D., Eremenko, V., Tamer, N.C., Miron, M., Serra, X.: Combin- ing piano performance dimensions for score difficulty classification. Expert Systems with applications238, 121776 (2024)

    2024

  8. [8]

    Communications of the ACM28(3), 289–299 (1985)

    Blair, D.C., Maron, M.E.: An evaluation of retrieval effectiveness for a full-text document-retrieval system. Communications of the ACM28(3), 289–299 (1985)

    1985

  9. [9]

    Big Data and Cognitive Computing6(1), 23 (2022)

    Zhu, T., Fournier-S’niehotta, R., Rigaux, P., Travers, N.: A framework for content-based search in large music collections. Big Data and Cognitive Computing6(1), 23 (2022)

    2022

  10. [10]

    Bainbridge, D., Cunningham, S., Downie, J.: How people describe their music information needs: A grounded theory analysis of music queries (2003)

    2003

  11. [11]

    Thomas, B., Garcia-Gathright, J., Thom, J.: Just give me what i want: How people use and evaluate music search

    Hosey, C., Vujovi´ c, L., St. Thomas, B., Garcia-Gathright, J., Thom, J.: Just give me what i want: How people use and evaluate music search. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ’19, pp. 1–12. Association for 14 Computing Machinery, New York, NY, USA (2019)

    2019

  12. [12]

    New Journal of Physics11(9), 093017 (2009)

    Serra, J., Serra, X., Andrzejak, R.G.: Cross recurrence quantification for cover song iden- tification. New Journal of Physics11(9), 093017 (2009)

    2009

  13. [13]

    In: M¨ uller, M., Goto, M., Schedl, M

    Grosche, P., M¨ uller, M., Serr` a, J.: Audio Content-Based Music Retrieval. In: M¨ uller, M., Goto, M., Schedl, M. (eds.) Multimodal Music Processing. Dagstuhl Follow-Ups, vol. 3, pp. 157–174. Schloss Dagstuhl – Leibniz- Zentrum f¨ ur Informatik, Dagstuhl, Germany (2012)

    2012

  14. [14]

    Inter- national Journal of Multimedia Information Retrieval2(1), 45–58 (2013)

    Salamon, J., Serra, J., G´ omez, E.: Tonal rep- resentations for music retrieval: from version identification to query-by-humming. Inter- national Journal of Multimedia Information Retrieval2(1), 45–58 (2013)

    2013

  15. [15]

    In: Antonacopou- los, A., Hinze, A., Piwowarski, B., Cous- taty, M., Di Nunzio, G.M., Gelati, F., Van- derschantz, N

    Aly, A., Pivert, O., Thion, V.: Database approaches to the modelling and querying of musical scores: A survey. In: Antonacopou- los, A., Hinze, A., Piwowarski, B., Cous- taty, M., Di Nunzio, G.M., Gelati, F., Van- derschantz, N. (eds.) Linking Theory and Practice of Digital Libraries, pp. 292–309. Springer, Cham (2024)

    2024

  16. [16]

    In: Proc

    Typke, R., Wiering, F., Veltkamp, R.C.,et al.: A survey of music information retrieval systems. In: Proc. 6th International Con- ference on Music Information Retrieval, pp. 153–160 (2005). Citeseer

    2005

  17. [17]

    The virtual score: representation, retrieval, restoration12(113-124), 160 (2001)

    Good, M.: Musicxml for notation and analysis. The virtual score: representation, retrieval, restoration12(113-124), 160 (2001)

    2001

  18. [18]

    abcnotation

    Walshaw, C.: The abc music standard 2.1 (dec 2011). abcnotation. com wiki12, 2011 (2011)

    2011

  19. [19]

    In: Proceedings of the First Interna- tional Conference on Musical Applications Using XML, vol

    Roland, P.: The music encoding initiative (mei). In: Proceedings of the First Interna- tional Conference on Musical Applications Using XML, vol. 1060, pp. 55–59 (2002). Milan, Italy

    2002

  20. [20]

    Huron, D.: Humdrum and Kern: selective feature encoding, pp. 375–401. MIT Press, Cambridge, MA, USA (1997)

    1997

  21. [21]

    International Journal of Multimedia Information Retrieval1(3), 173– 190 (2012)

    Rebelo, A., Fujinaga, I., Paszkiewicz, F., Marcal, A.R., Guedes, C., Cardoso, J.S.: Optical music recognition: state-of-the-art and open issues. International Journal of Multimedia Information Retrieval1(3), 173– 190 (2012)

    2012

  22. [22]

    Applied Sciences8(4), 606 (2018)

    Calvo-Zaragoza, J., Rizo, D.: End-to-end neural optical music recognition of mono- phonic scores. Applied Sciences8(4), 606 (2018)

    2018

  23. [23]

    In: Document Analysis and Recog- nition - ICDAR 2024, pp

    R´ ıos-Vila, A., Calvo-Zaragoza, J., Paquet, T.: Sheet music transformer: End-to-end optical music recognition beyond monophonic tran- scription. In: Document Analysis and Recog- nition - ICDAR 2024, pp. 20–37. Springer, Cham (2024)

    2024

  24. [24]

    In: ISMIR, pp

    Luna-Barahona, N.N., Rosello, A., Alfaro- Contreras, M., Rizo, D., Calvo-Zaragoza, J.: Unsupervised synthetic-to-real adaptation for optical music recognition. In: ISMIR, pp. 462–469 (2024)

    2024

  25. [25]

    In: Proceed- ings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, Septem- ber 23-27, 2018, pp

    Crawford, T., Badkobeh, G., Lewis, D.: Searching page-images of early music scanned with omr: A scalable solution using minimal absent words. In: Proceed- ings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, Septem- ber 23-27, 2018, pp. 233–239 (2018). http://ismir2018.ircam.fr/doc/pdfs/210%5fPaper.pdf

    2018

  26. [26]

    Faculty of Grad- uate Studies, University of Western Ontario, London, Ont

    Downie, J.S.: Evaluating a Simple Approach to Music Information Retrieval: Conceiving Melodic N-grams as Text. Faculty of Grad- uate Studies, University of Western Ontario, London, Ont. (1999)

    1999

  27. [27]

    In: ICMC (1998)

    Lemstr¨ om, K., Laine, P.: Musical information retrieval using musical parameters. In: ICMC (1998)

    1998

  28. [28]

    In: 15 ISMIR (2004)

    Sapp, C.S., Liu, Y.-W., Selfridge-Field, E.: Search effectiveness measures for symbolic music queries in very large databases. In: 15 ISMIR (2004). Citeseer

    2004

  29. [29]

    Musicae Scientiae 13(1 suppl), 201–234 (2009)

    Typke, R., Walczak-Typke, A.C.: Some vantage indexing approaches for rhyth- mic and melodic search. Musicae Scientiae 13(1 suppl), 201–234 (2009)

    2009

  30. [30]

    In: Exploring Music Con- tents: 7th International Symposium, CMMR 2010, M´ alaga, Spain, June 21-24, 2010

    Urbano, J., Llor´ ens, J., Morato, J., S´ anchez- Cuadrado, S.: Melodic similarity through shape similarity. In: Exploring Music Con- tents: 7th International Symposium, CMMR 2010, M´ alaga, Spain, June 21-24, 2010. Revised Papers 7, pp. 338–355 (2011). Springer

    2010

  31. [31]

    In: Calvo-Zaragoza, J., jr., J.H., Pacha, A

    Achankunju, S.P.: Music search engine from noisy omr data. In: Calvo-Zaragoza, J., jr., J.H., Pacha, A. (eds.) 1st International Work- shop on Reading Music Systems, Paris, France, pp. 23–24 (2018)

    2018

  32. [32]

    rism online

    McKay, C.: R´ epertoire international des sources musicales (rism) digital center team. rism online. Renaissance and Reformation 47(1), 184–191 (2024) https://doi.org/10. 33137/rr.v47i1.43449

    2024

  33. [33]

    Data & Knowledge Engineer- ing153, 102340 (2024)

    Rigaux, P., Thion, V.: Topological querying of music scores. Data & Knowledge Engineer- ing153, 102340 (2024)

    2024

  34. [34]

    4 (2024)

    Calvo-Zaragoza, J., Fuentes-Martınez, E., Luna-Barahona, N., Rıos-Vila, A.: Can multi- modal large language models read music score images? In: 6 Th International Workshop on Reading Music Systems, p. 4 (2024)

    2024

  35. [35]

    Open Mind8, 1058–1083 (2024)

    Cuskley, C., Woods, R., Flaherty, M.: The limitations of large language models for understanding human language and cogni- tion. Open Mind8, 1058–1083 (2024)

    2024

  36. [36]

    In: Findings of the Associa- tion for Computational Linguistics: NAACL 2025, pp

    Tang, M., Li, J., Yang, L., Zhang, Z., Tian, J., Li, Z., Zhang, L., Wang, P.: Nota: Multimodal music notation understanding for visual large language model. In: Findings of the Associa- tion for Computational Linguistics: NAACL 2025, pp. 7160–7173 (2025)

    2025

  37. [37]

    arXiv preprint arXiv:2506.23009 (2025)

    Chen, J., Ma, W., Liu, P., Wang, W., Song, T., Li, M., Wang, C., Qin, J., Zhang, R., Chen, C.: Musixqa: Advancing visual music understanding in multimodal large language models. arXiv preprint arXiv:2506.23009 (2025)

    work page arXiv 2025

  38. [38]

    In: ISMIR, pp

    Hsiao, Y.-W., Hung, T.-Y., Chen, T.-P., Su, L.: Bps-motif: A dataset for repeated pattern discovery of polyphonic symbolic music. In: ISMIR, pp. 281–288 (2023)

    2023

  39. [39]

    (No Title) (2004)

    Hewlett, W.B., Selfridge-Field, E.: Music query: methods, models, and user studies. (No Title) (2004)

    2004

  40. [40]

    Online open textbook

    Gotham, M., Gullings, K., Hamm, C., Hughes, B., Jarvis, B., Lavengood, M., Peter- son, J.: Open Music Theory. Online open textbook. https://openmusictheory.com/

  41. [41]

    Schachter, C., Aldwell, E.: Harmony & voice leading (2011)

    2011

  42. [42]

    Phan, A., Thomae, M.E., De Luca, E., Orio, F.: Plainchant analyser for mei neumes: A tool for understanding chant transmission

  43. [43]

    Ros-F´ abregas, E.: Codified spanish music heritage through verovio: The online plat- forms fondo de m´ usica tradicional imf-csic and books of hispanic polyphony imf-csic. (2021)

    2021

  44. [44]

    In: Proceed- ings of the 8th International Conference on Digital Libraries for Musicology

    Madue˜ no, A., R´ ıos-Vila, A., Rizo, D.: Autom- atized incipit encoding at the Andalusian Music Documentation Center. In: Proceed- ings of the 8th International Conference on Digital Libraries for Musicology. Association for Computing Machinery, Online (2021)

    2021

  45. [45]

    arXiv preprint arXiv:2405.12105 (2024)

    R´ ıos-Vila, A., Calvo-Zaragoza, J., Rizo, D., Paquet, T.: End-to-end full-page optical music recognition for pianoform sheet music. arXiv preprint arXiv:2405.12105 (2024)

    work page arXiv 2024

  46. [46]

    In: Proceedings of the 23rd Inter- national Conference on Machine Learning, pp

    Graves, A., Fern´ andez, S., Gomez, F., Schmidhuber, J.: Connectionist tempo- ral classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd Inter- national Conference on Machine Learning, pp. 369–376 (2006) 16

    2006

  47. [47]

    In: International Confer- ence on Document Analysis and Recognition, pp

    Rosell´ o, A., Fuentes-Mart´ ınez, E., Alfaro- Contreras, M., Rizo, D., Calvo-Zaragoza, J.: Source-free domain adaptation for optical music recognition. In: International Confer- ence on Document Analysis and Recognition, pp. 3–19 (2024). Springer

    2024

  48. [48]

    PaliGemma 2: A Family of Versatile VLMs for Transfer

    Steiner, A., Pinto, A.S., Tschannen, M., Key- sers, D., Wang, X., Bitton, Y., Gritsenko, A., Minderer, M., Sherbondy, A., Long, S., et al.: Paligemma 2: A family of versatile vlms for transfer. arXiv preprint arXiv:2412.03555 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  49. [49]

    ICLR1(2), 3 (2022) 17

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.,et al.: Lora: Low-rank adaptation of large language models. ICLR1(2), 3 (2022) 17

    2022