Interview Connoisseurship

Before our team conducted a contextual inquiry on the use of the exercise wristband Nike+ SportWatch GPS by our participant (pseudonym “Sam”), we worked on the study procedure and expected outcome. From there we prepared a list of question to ask Sam before and after we observed him run with the wristband.

Most of what we observed and learned from Sam is within what we expected based on the research we did and information the website of the wristband. However, something he said did stand out. I didn’t expect to hear it.

Toward the end of our post-observation interview, I asked Sam if using his exercise band had changed his habit or running routine in any way. He paused briefly before starting to tell a short story. He said that it used to hurt to run because he had some flat foot issues. He then talked about how “the watch” - as he calls his exercise wristband - quickly became the companion he needed to motivate him to run. Compared to similar products Sam has put his hands on, the exercise wristband he showed us is favored by him. He demonstrated the messages he sees on the interface of this exercise band, and how the conversational tone on the interface works for him. He said the wristband would show a message “Are we going to run?” When he answers “Okay,” the band would respond “That sounds great!” Sam said “It got me into running,” “I really really like this watch. It’s really useful.” and “The interface is the best. It’s almost like it has a personality.”

Coming into the contextual inquiry, I didn’t expect this emotional attachment Sam elaborated. It was only through probing based on what he had displayed and expressed we were able to uncover this dimension of his engagement with the wristband beyond the behavioral and functional aspects of this product.

Nike may or may not have categorized this level of user attachment as an “intended use” of this product. Its website does not mention any of it, so it is fair to say that Nike does not sell this product by appealing to potential buyers beyond its functional value.

I really enjoy the process of eliciting the deep personal meanings unique to our participant. Being able to hear from Sam the companionship his exercise band means to him and how important it was as a motivator for his running routine made all the time and efforts put into the preparation of the contextual inquiry worthwhile.

As laid out in the syllabus of our Experience design course, one of the course objectives is to “develop a critical and creative practical sensibility with regard to ‘experience’ in relation to interactivity” (Stolterman 2015: 1). I believe this sensitivity includes being able to elicit and discern the unique meanings users develop over things.

However, researching to uncover the embedded meanings someone has over something is challenging. It requires synthesizing and interpreting what the participant says on the fly, and framing follow-up questions accordingly. Distinguished journalists such as Bill Moyers, Margaret Warner, Terry Gross , and Charlie Rose have demonstrated this skill superbly time and again.
Even with their career success, I do not believe these accomplished journalists would conduct an interview without doing their homework. Even though they do not seem to read their questions from their notes, I am sure they always prepare a list of questions or at least an outline for any interview, in addition to researching their interview participants’ background and setting a goal for each interview. To undiscerning eyes, their interviews seem effortless as they flow like excellent conversations. However, I have no doubt that their not having to read questions from their notes shows they internalize the background information and what they would like to ask, which is essential to be able to probe deeply into their subjects’ inner worlds.

Professor Eric Stolterman elaborated this Tuesday (March 10) in class on Elliot W. Eisner’s notion of connoisseurship – “the ability to make fine-grained discrimination among complex and subtle qualities.” I believe interview as a technique also involves connoisseurship. Being able to do it well demands an appreciation that there is more to the interview technique than “just going out there and asking questions.” Like appreciation for good wine, cuisine, architecture, and other designs and developing differentiating tastes for them, it also requires lots of practice.

Additionally, learning from good interviews such as those conducted by reputable journalists and being able to recognize good interviews are conducive to improving our interview skills over time, as everything else concerning connoisseurship. Moreover, I believe critiquing our own interviews is also essential, because seeing ourselves can be extremely effective as a heuristic experience. Many people feel surprised when they see themselves in video recordings, because it is inherently difficult to see ourselves in action. That’s why I think getting the participant’s approval to videotape interviews (which need not show the participant’s face) is important, at least in this stage of our career.

When I am in charge of preparing for the participant consent form, I try to prepare different versions for the contingencies where participants may or may not agree to be video-recorded. I also try to work with the person(s) who recruit our participants to know before our study whether the participants would agree to let us videotape the study. The goal is to maximize the chance of getting their consent to let us videotape the process so that we can review how we did in the interview and improve based on the experience. The recording also has the benefit of allowing us to conduct a more thorough analysis of what the participants displayed or expressed.

In all, learning to draw unexpected threads of meanings from our interview participants is not trivial in both the skills it involves and the practice it takes. More importantly, it is not trivial in terms of the profound understanding we can acquire about our participants or their experience. Personally, I feel it is really worth it.

Reference

Eric Stolterman. 2015. Syllabus for I544 Experience Design. School of Informatics and Computing, Indiana University Bloomington (unpublished).

Video "Under the Dome"

Chai Jing's "Under the Dome" is inspiring. She paces superbly in her narration, which is in the TED talk style, but much longer - 104 minutes. There are no fancy visual effects, but the audience obviously resonated with the storyline and arguments Chai presented. The entire presentation flows beautifully, so even though it's very substantive and contains lots of statistics, it's engaging.

She uses a series of very clear and sensible questions to guide her investigation and walk the viewers through her narration. She most likely did not know exactly what she would find, so her investigation plan - including where to go, whom to interview, etc. - must have adjusted to the circumstances. This process is similar to what good design research requires. 

Her interview skill is very impressive, too. She obviously did her homework, which made her probing questions really sharp and to the point. Her demeanor effectively mitigates the critical nature of her questions.

The video of her talk became so popular that within a few weeks more than 20 million people had watched it before the Chinese government censors banned it from public access.

Until this video was released on Feb. 28, the middle class in China generally seemed to have bought into the Chinese authority's economic development argument - the country is making progress steadily in political reform, sustainability, and other quality-of-life measures while making sure first and upmost living standards are improving. Chai's video effectively challenges this myth, even though she didn't say it explicitly in the video. After listening to Chai, it's not hard to see the Chinese middle class ask questions such as: If the air is so toxic, does it matter how much the GDP growth rate is? Why hasn't the government taken more action to protect the health of the general public? Why have the environmental protection authorities and the laws been idle instead of doing their job?

This video may have created another watershed - Many Chinese hitherto have believed corruption is an individual problem isolated case by case. Banning the video may have made people realize that it's more than systemic than they previously thought, otherwise what does the government have to fear about the video?

It remains to be seen the full impact of this video.

Image Source: https://www.youtube.com/watch?v=T6X2uwlQGQM (1:00:22)

On Subjectivity

Boehner et al. (2007) claim that designers have a subjective role in creating probes and that they "reveal themselves through the design proposals or through speculative design inspired by the probe results". Is this subjectivity problematic?

Whether we are researchers or designers, we inevitably wear certain lenses in filtering information and focusing on specific aspects of “facts.” As Werner Herzog says, "Facts do not constitute truth." (Theodore 2014). As much as we may try to be objective, our perspectives are colored by past experiences and the resulting inner emotional world, sometimes without us even realizing it.

As designers, we are taught to reflect in action and reflect on action (Schön 1987). Reflecting on how our viewpoint affects what we design is a part of it. It’s not that we necessarily have an agenda in mind and deliberately try to design based on it. The subjectivity can be on autopilot, especially if we have little self-awareness about what we bring to the table. That’s part of the reason why reflection is so important for designers.

Perspectives differ. Perspectives can vary widely from one persona to another. Therefore, the same study purpose can result in drastically different interviews or even methods. For example, when we conduct an interview, whom we decide the interview means whom we are giving a voice; what questions we ask in the interview determines what our subject can articulate. All these decisions depend on our judgments, which are influenced by our perspectives.

Additionally, like Boehner et al (ibid.) argue, cultural probes can also have very different configurations from one researcher to another. Even with the same configuration, the interpretation of the result often is wide open, too.

Even quantitative analysis involves perspectives. For example, when we remove outliers from our dataset, we choose to ignore data points that do not conform to the majority pattern. This practice marginalizes what is not “normal.” There are ethical and possibly other issues in this practice, and the consequences can be profound and far-reaching.

Is it possible to be objective? Should we try to be objective? In my view, we can strive to be objective, but we need to be aware of the danger in claiming to be objective by keeping a critical eye on such claims. Even claiming to be objective represents a certain perspective. I think the most important thing is to hold ourselves accountable for the judgments we make in the form of rationale, and not to be afraid of making judgments, which according to Nelson and Stolterman (2014) is an essential part of what designers do.

Perhaps when we are not ashamed of saying we have our perspectives, we can articulate our rationale better, and we can better appreciate the connection between people’s viewpoints and their past experiences.

References

Kirsten Boehner, Janet Vertesi, Phoebe Sengers, and Paul Dourish. 2007. How HCI Interprets the Probes. Paper presented at CHI 2007, April 28-May 3, 2007 in San Jose, CA, USA.

Harold G. Nelson and Erik Stolterman. 2014. The Design Way: Intentional Change in an Unpreductable World. 2nd edition. Cambridge, MA: The MIT Press.

Donald Schön. 1987. Educating the Reflective Practitioner. San Francisco, CA: Jossey-Bass.

Marie-Françoise Theodore. 2014. 12 Things I Learned at Werner Herzog's Rogue Film School. Retrieved from http://www.indiewire.com/article/12-things-i-learned-at-werner-herzogs-rogue-film-school-20140924 on March 4, 2015.

Note: This post was originally composed for a weekly blog post assignment for the course “Interaction Design Methods” at the School of Informatics and Computing at Indiana University Bloomington.

Reflecting on the Card Sort Focus Group Interview Process

For the card sort focus group assignment, the first challenge was writing the script for it. As the person in charge of the script, I consulted several sources and received feedback from the team. It was particularly helpful hearing inputs from Jianping, who recruited four of our five participants and checked with them to see if any of them would mind being photographed or video-taped. Since we didn’t hear from all our participants before the card sort focus group activities took place, I prepared two versions of participants consent form in case any participant would mind our photographing or videotaping the process.

Putting together the script and consent form with no example or model to hang on to took time and thoughts, but it’s a good learning experience, as it required thinking through what we tried to find out and how we were planning to approach it using the structure provided in the assignment. It’s exactly what Eric talked about in the Experience Design class where he presented a schema that ties the questions (what & why), lens selection, tool selection, and data analysis elements of the design research process. These elements have to fit in a coherent way and as design researchers we need to constantly reflect on whether we’re losing sight of certain elements in the process.

The second challenge I faced was being the facilitator of the card sort and focus group interview activities. Even though I prepared a pretty detailed script that laid out the specific activities for participants to do and starting questions to ask them during the focus group interview, I was aware that my role as the facilitator would also include addressing whatever contingencies that might arise. I was also aware that a good interview would almost never be one that just reading from a list of pre-scripted questions.

One of the contingencies I had to deal with was that there was some confusion about how the cards should be prepared and what should go on them. We literally finished the preparation of the cards for the card sort activity the last second. Lessen learned: always check if everything is in place with enough time to fix potential problems.

I was a participant of a card sort before, but I never facilitated one prior to this experience. Even though I gave our participants instructions on how to sort the cards, initially them seemed confused. Fortunately, thanks to good division of labor (the rest of the team was either photographing, videotaping, or note-taking) and observation from the team, I was able to clarify and clear the problem on the spot.

I had conducted many interviews before, and I believe interview is a technique that gets better with practice. Still, every interview is different with different participants and background or domain knowledge involved. I’ve learned that listening very carefully during the interview process and asking good follow-up questions are essential parts of good interviews.

As Werner Herzog argues, "Facts do not constitute truth … construct a reality that illuminates the truth." (Theodore 2014) People have their own vantage points based on past experiences and their inner emotional world. Therefore, even though our participants played the same game and engaged in the same card sort activity together, they had different viewpoints. It was my job as the facilitator to probe with good questions.

The questions not only had to dig into what we tried to find out and why we conducted these activities. They also had to be considerate from participants’ psychological viewpoints. For example, as Gregory (2015) suggests, “why” questions are generally bad, and so is making assumptions about participants or using language that the participants might not know. How to probe without making participants feel uncomfortable requires sensitivity. Additionally, techniques such as validation, “tentafiers” (“Do you mind if I ask you . . . ”), strength identification, and empathetic responses can be useful in interviews (Grogory, ibid). The challenge is how to use these techniques skillfully so that the interview flows like a good conversation, as opposed to a cross-examination or robotic Q&As.

The efforts I took in preparing for potential breakdowns paid off. Even though we didn’t encounter any serious breakdowns, having thought it through primed my mindset for dealing with any potential glitches. For example, a few participants were not as vocal than the others. I was able to notice that and solicit their opinions on the spot. Fortunately, they responded pretty well and articulated their thoughts.

The other challenge I see in this card sort focus group exercise is connecting our activities in a meaningful way. Initially I was having a hard time seeing why we were doing card sort. Seeing how our participants talked to each other enthusiastically during the focus group interview convinced me that the eight minutes we had had them play our selected game “dumb ways to die” served as an excellent conversation piece for the discussion, just like usability test can be a good way to start an interview as Galt (2015) suggests. The card sort activity was arguably as relevant as the game play, but to some extent it refreshed participants’ memory about what they saw in the game.

In the end, it is up to us how we mix and match methods. We can use various methods or techniques to triangulate data; we can deploy one method as a starting point for another method; we can use techniques that draw insights that complement each other; etc. The crucial thing is that it is done in a methodologically sound way that serves a clear purpose, and I believe reflecting on our experience of using these methods is critical for learning to use the right methods in the right contexts for the right purpose.


References

Malcolm Galt. 2015. Conduct Usability Testing To Create Killer Online Marketing Campaigns. Retrieved from http://blog.uxeria.co.za/conduct-usability-testing-to-create-killer-online-marketing-campaigns/, Feb. 27, 2015.

Alice Gregory. 2015. R U There? Retrieved from http://www.newyorker.com/magazine/2015/02/09/r-u, Feb. 27. 2015.

Marie-Françoise Theodore. 2014. 12 Things I Learned at Werner Herzog's Rogue Film School. Retrieved from http://www.indiewire.com/article/12-things-i-learned-at-werner-herzogs-rogue-film-school-20140924 on Feb. 27, 2015.

Note: This post was originally composed for a weekly blog post assignment for the course “Interaction Design Methods” at the School of Informatics and Computing at Indiana University Bloomington.