STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals

Statute retrieval aims to find relevant statutory articles for specificqueries. This process is the basis of a wide range of legal applications suchas legal advice, automated judicial decisions, legal document drafting, etc.Existing statute retrieval benchmarks focus on formal and professional queriesfrom sources like bar exams and legal case documents, thereby neglectingnon-professional queries from the general public, which often lack preciselegal terminology and references. To address this gap, we introduce the STAtuteRetrieval Dataset (STARD), a Chinese dataset comprising 1,543 query casescollected from real-world legal consultations and 55,348 candidate statutoryarticles. Unlike existing statute retrieval datasets, which primarily focus onprofessional legal queries, STARD captures the complexity and diversity of realqueries from the general public. Through a comprehensive evaluation of variousretrieval baselines, we reveal that existing retrieval approaches all fallshort of these real queries issued by non-professional users. The best methodonly achieves a Recall@100 of 0.907, suggesting the necessity for furtherexploration and additional research in this area. All the codes and datasets are available at:https://github.com/oneal2000/STARD/tree/main