BACKGROUND

Working in partnership with one other researcher, we designed and executed all stages of the 2-month research project for a recurring client during the busy, holiday season. This client typically comes to us to test their digital kids products remotely, but this time around, they wanted to understand an experience that at its core happens in-person and off-screen: doing chores.

In developing a new feature for their AI assistant, our client wanted to understand if kids would actually like using it, if it would be truly successful in its goal of facilitating chores, and how it could be better developed to fit into families’ existing chores systems. On top of this, the kids product team and the parent product team had never worked together before, so we facilitated their collaboration, bridging gaps of understanding and highlighting ways in which their products needed to cooperate in order to create a seamless family experience.

To do this, we designed our research sessions to include usability tests for a low-fidelity prototype for kids and a high-fidelity prototype for parents as well as interviews with both parent and kid to gather the more generative data.

LIMITATIONS

Though I am well-versed in the intricacies of running research with kids, this project had a few unique constraints that made it more challenging fun than normal:  

• Chores are an innately at-home experience, but for legal reasons, our client would not approve running the sessions in-home.

• Post-pandemic downtown San Francisco, where our research lab is, had become an increasingly difficult place to convince participants, let alone families, to go to for research sessions.

• We had a very limited budget as these projects were typically scoped to be run remotely.

• Our highly engaged stakeholders flew in to participate in the research, meaning we’d need to involve them even more in data collection, staying mindful that too many strangers in the room can quickly overwhelm kid participants.

• Both kids and parents are likely to talk about chores in a way that is socially desirable, so how do we get realistic answers and behaviors to inform building a product that is truly useful for families? 

So what did this mean for us?

Long story short, we created a pop-up kids bedroom in the conference room of a co-working space that we had to break down and rebuild daily. The constraints of the space’s size and availability meant that we had to develop a way for the designers to run the Wizard of Oz prototype and observe the session from outside of the testing room. This mobile, tech set-up and fake bedroom facilitated observers both remote and in-person to see all parts of the research close-up, without creating too unnatural of an experience for participants.

PROCESS

After a handful of alignment meetings with the product teams responsible for the kid and parent products, we dove into the research design. We first collaborated with a trusted participant recruiter to find the right participants. Then we developed a discussion guide filled with questions and tasks that would get us closer to answering our research objectives. My partner and I switched off who would moderate and who would note-take each session. Though it can be hectic, moderation is my favorite part, especially when kids are our participants!

With a sample size of eight child/parent pairs, we designed the study to account for the long list of things you must consider when doing research with kids, to name a few:

• Kids’ comfort level completing the session with their parent in another room

• Kids’ attention and alertness since sessions had to run after school and into the early evening

• How to ensure the kid gave us their honest opinion about chores, a topic at risk for response bias and trying to appease their parents

• Shy kids relying on their parents to respond for them —> worksheet to look like the parent is distracted and they’re not being watched

• Kids might have a hard time articulating their thoughts —> emojis and thumbs up scales to encourage them to elaborate

Out of the norm for this client, the designers from the kid product team wanted to be highly involved in the research process and traveled to attend sessions. This was both a blessing and a curse we had to adapt to.

We debriefed with the designers between sessions, sharing observations and developing insights. Our clients’ presence meant we were always ON: entertaining, caffeinating, answering their questions.

By having those stakeholders in-person with us all week, we ultimately got so much more context from them about the product’s history, roadmaps, internal politics, etc. that as external consultants is usually hard to come by. This led to us developing deeper, more informed insights for the final report and recommendations.

Our client’s rules around kids data limited the analysis tools we could use, so we could not use some of the more time-saving  tools like Dovetail. Instead we did a lot of old-school, thematic analysis, combing through the video clips and relying on a solid note-taking structure to point us in the right direction to find the data we were looking for.

This project culminated with a written report and final presentation to the designers, product managers, and internal researchers from the two product teams. We tied our findings directly to the research questions, sharing recommendations for changes as well as suggested areas of focus for future product iterations. As is the case for many of our clients, some stakeholders put more weight in quantitative data than in qualitative data. We carefully considered how to report any quant data to make sure no one drew inappropriate conclusions from it, emphasizing how qualitative research tells you the “why” and instead suggesting follow-up methods to understand scale.

IMPACT

Overall, the client left knowing which features to prioritize in their product roadmap long-term, as well as which design changes would be necessary to make the experience more usable for kids and parents more immediately.

There was a lot to juggle during this project, so I’m particularly proud the designers said it was a valuable experience, hoping to have the opportunity to work with us again.