Reflecting on the Card Sort Focus Group Interview Process

For the card sort focus group assignment, the first challenge was writing the script for it. As the person in charge of the script, I consulted several sources and received feedback from the team. It was particularly helpful hearing inputs from Jianping, who recruited four of our five participants and checked with them to see if any of them would mind being photographed or video-taped. Since we didn’t hear from all our participants before the card sort focus group activities took place, I prepared two versions of participants consent form in case any participant would mind our photographing or videotaping the process.

Putting together the script and consent form with no example or model to hang on to took time and thoughts, but it’s a good learning experience, as it required thinking through what we tried to find out and how we were planning to approach it using the structure provided in the assignment. It’s exactly what Eric talked about in the Experience Design class where he presented a schema that ties the questions (what & why), lens selection, tool selection, and data analysis elements of the design research process. These elements have to fit in a coherent way and as design researchers we need to constantly reflect on whether we’re losing sight of certain elements in the process.

The second challenge I faced was being the facilitator of the card sort and focus group interview activities. Even though I prepared a pretty detailed script that laid out the specific activities for participants to do and starting questions to ask them during the focus group interview, I was aware that my role as the facilitator would also include addressing whatever contingencies that might arise. I was also aware that a good interview would almost never be one that just reading from a list of pre-scripted questions.

One of the contingencies I had to deal with was that there was some confusion about how the cards should be prepared and what should go on them. We literally finished the preparation of the cards for the card sort activity the last second. Lessen learned: always check if everything is in place with enough time to fix potential problems.

I was a participant of a card sort before, but I never facilitated one prior to this experience. Even though I gave our participants instructions on how to sort the cards, initially them seemed confused. Fortunately, thanks to good division of labor (the rest of the team was either photographing, videotaping, or note-taking) and observation from the team, I was able to clarify and clear the problem on the spot.

I had conducted many interviews before, and I believe interview is a technique that gets better with practice. Still, every interview is different with different participants and background or domain knowledge involved. I’ve learned that listening very carefully during the interview process and asking good follow-up questions are essential parts of good interviews.

As Werner Herzog argues, "Facts do not constitute truth … construct a reality that illuminates the truth." (Theodore 2014) People have their own vantage points based on past experiences and their inner emotional world. Therefore, even though our participants played the same game and engaged in the same card sort activity together, they had different viewpoints. It was my job as the facilitator to probe with good questions.

The questions not only had to dig into what we tried to find out and why we conducted these activities. They also had to be considerate from participants’ psychological viewpoints. For example, as Gregory (2015) suggests, “why” questions are generally bad, and so is making assumptions about participants or using language that the participants might not know. How to probe without making participants feel uncomfortable requires sensitivity. Additionally, techniques such as validation, “tentafiers” (“Do you mind if I ask you . . . ”), strength identification, and empathetic responses can be useful in interviews (Grogory, ibid). The challenge is how to use these techniques skillfully so that the interview flows like a good conversation, as opposed to a cross-examination or robotic Q&As.

The efforts I took in preparing for potential breakdowns paid off. Even though we didn’t encounter any serious breakdowns, having thought it through primed my mindset for dealing with any potential glitches. For example, a few participants were not as vocal than the others. I was able to notice that and solicit their opinions on the spot. Fortunately, they responded pretty well and articulated their thoughts.

The other challenge I see in this card sort focus group exercise is connecting our activities in a meaningful way. Initially I was having a hard time seeing why we were doing card sort. Seeing how our participants talked to each other enthusiastically during the focus group interview convinced me that the eight minutes we had had them play our selected game “dumb ways to die” served as an excellent conversation piece for the discussion, just like usability test can be a good way to start an interview as Galt (2015) suggests. The card sort activity was arguably as relevant as the game play, but to some extent it refreshed participants’ memory about what they saw in the game.

In the end, it is up to us how we mix and match methods. We can use various methods or techniques to triangulate data; we can deploy one method as a starting point for another method; we can use techniques that draw insights that complement each other; etc. The crucial thing is that it is done in a methodologically sound way that serves a clear purpose, and I believe reflecting on our experience of using these methods is critical for learning to use the right methods in the right contexts for the right purpose.


References

Malcolm Galt. 2015. Conduct Usability Testing To Create Killer Online Marketing Campaigns. Retrieved from http://blog.uxeria.co.za/conduct-usability-testing-to-create-killer-online-marketing-campaigns/, Feb. 27, 2015.

Alice Gregory. 2015. R U There? Retrieved from http://www.newyorker.com/magazine/2015/02/09/r-u, Feb. 27. 2015.

Marie-Françoise Theodore. 2014. 12 Things I Learned at Werner Herzog's Rogue Film School. Retrieved from http://www.indiewire.com/article/12-things-i-learned-at-werner-herzogs-rogue-film-school-20140924 on Feb. 27, 2015.

Note: This post was originally composed for a weekly blog post assignment for the course “Interaction Design Methods” at the School of Informatics and Computing at Indiana University Bloomington.