HARP: Agentic Hybrid Retrieval and Analysis for Long-Form Audio

HARP (Hybrid Audio Retrieval Pipeline) is an agentic framework and benchmark for systematically studying retrieval and evidence representations in long-audio analysis. This repo holds the companion artifacts: the query sets for the music and health tasks, and the CSER checkpoint. The code is on GitHub. No corpus audio is included — you download the source datasets and rebuild the audio locally.

huggingface-cli download cjli/harp --local-dir $HARP_OUT_ROOT
export HARP_CSER_CKPT=$HARP_OUT_ROOT/checkpoints/cser/epoch_200.pt

Directory names match what the code expects, so the download is a drop-in.

songeval/
  queries/track_NN.jsonl     500 comparison queries
  languages2.jsonl           language-ID results — skips the slow Whisper pass
  tracks_metadata.jsonl      the 50 medleys: song order and boundaries
  clip_index.json            5 s reference snippets
medmosaic/
  queries.jsonl              212 reported/exhibited queries
checkpoints/
  cser/epoch_200.pt          BiLSTM head over WavLM-Large, predicts V/A/D per second

Rebuild the audio the queries refer to with HARP's audio stage:

dataprep/run_music.sh  audio --src /raw/songeval  --out $HARP_OUT_ROOT/songeval
dataprep/run_health.sh audio --src /raw/medmosaic --out $HARP_OUT_ROOT/medmosaic

Licensing

Queries and checkpoint are MIT. The source corpora are not redistributed here:

Task	Corpus	Licence
music	ASLP-lab/SongEval	CC BY-NC-SA 4.0
health	MedMosaic	CC BY 4.0
emotion	MSP-Podcast + MSP-Conversation	academic licence, signed per person

songeval/ describes and references SongEval songs, so respect its non-commercial, share-alike terms. The CSER checkpoint was trained on MSP-Conversation; see emo-reasoning.

Emotion queries are not included. The MSP licence forbids redistribution and the queries quote transcript directly. Once you hold MSP, regenerate the full set — seeded and greedy-decoded, so it reproduces the benchmark:

dataprep/run_emotion.sh all --src /path/to/msp --out $HARP_OUT_ROOT/longemo_dataset

Citation

@article{li2026harp,
  title   = {{HARP}: Agentic Hybrid Retrieval and Analysis for Long-Form Audio},
  author  = {Li, Chin-Jou and Someki, Masao and Jin, Woojeong and
             Siriwardena, Yashish M. and Laud, Tanmay and Puri, Shanil and
             Watanabe, Shinji},
  journal = {arXiv preprint arXiv:2609.14116},
  year    = {2026}
}