The Research#

For those interested in some more technical details, (parts of) the work discussed comes from empirical studies with 25 domain experts across five stakeholder groups, resulting in publications at RecSys 2025:

RecSys 2025:

• “Recommender Systems for Digital Humanities and Archives: Multistakeholder Evaluation, Scholarly Information Needs, and Multimodal Similarity” (DOI | SSRN)
• “A Multistakeholder Approach to Value-Driven Co-Design of Recommender System Evaluation Metrics in Digital Archives” (DOI)

RecSys 2024 workshops:

• “Value Identification in Multistakeholder Recommender Systems” at NORMalize 2024 (CEUR)
• “Challenges in Implementing a Recommender System for Historical Research” at AltRecSys 2024 (arXiv)

These formalize the research funnel framework (discovery, interaction, integration, impact) and propose metrics like Research Path Quality, Collection Representation, and Contextual Appropriateness.

What’s Next#

The podcast touched on translating stakeholder values into concrete metrics—collection representation, serendipity, scholarly impact. The conceptual frameworks are out there, but operationalizing them mathematically is another story: formal definitions, worked examples, explicit trade-off mechanisms between competing priorities. We’re working on that!

Also worth exploring: why (and if) collaborative filtering fails with sparse cultural heritage data, session-based approaches for scholarly patterns, multimodal embeddings, and evaluation across different timeframes. Expect more soon.

Thanks again to Kyle Polich and the Data Skeptic team. The participatory approach to value-based evaluation should transfer beyond cultural heritage—useful anywhere you need to balance competing stakeholder priorities explicitly.

For updates, follow DiDip or connect via the sidebar.