You've perfectly described the classic gap that diary studies are designed to fill. They're specifically for uncovering those patterns, routines, and emotional shifts that happen between discrete research sessions. Instead of bringing users to a lab, you give them a simple way to document their experiences in their own environment over a period of days or weeks. It reveals the "why" behind the analytics and the context that surveys miss. If you're looking for a practical, start-to-finish guide on how to plan, run, and analyze one—including the different types, how to structure entries, and how to keep participants engaged—this deep dive on what is a diary study is the most comprehensive resource I've found. It turns a complex method into an actionable plan.