User Feedback Plan and Results

Overview

In order to establish the effectiveness of ‘Who’s That?’ in context, two tests will be conducted on an interactive prototype of the system. The first, an active test, will examine the user’s comfort with and and understanding of the interactions facilitated by the system. This test will also look at the ability of those interactions to help the user accomplish social goals. The second will be a passive test that evaluates the system’s ability to grab user attention and initiate interaction.

Test procedure

2 Roles: Interviewer and technician

Setup:

Setup computer with slide-show in media gateway (or where ever). Technician to make sure system is running smoothly, and take notes.
Interviewer will meet participant in NQ lobby with a bluetooth badge in hand.

Procedure:

Interviewer will meet user, do introduction.

Explain system/premise–What the system does, how RAs would have students create profiles.
Explain what QR codes are.

Walk over to test-area, where technician will have system running and be ready to record initial reactions as they enter the area. Interviewer will explain that we would like to get their initial impressions.

**As you enter area, turn on Bluetooth badge**
**As you enter area, and when user are watching to profiles during the test, technician will stealthily change the profile**
Have them look at system, give them time to interact with it if they’d like. (Don’t lead them on about the QR codes yet. See what they try to do. Do they touch it, wave at it?)
Ask them to think out initial impressions. (What do they think of the tag clouds?)
[TEST] Ask them to read a profile, and when they feel like they are bored/done reading, to say “Done”. Time them. Which ones do they find interesting and do those take longer? (Don’t have to do this all at once…)

Ask them to try interacting with a profile of their choice via QR code.

Ask them to think out loud. Why this one?
What is their first reaction?
Do they try to use the touchscreen buttons?

Ask them to walk through the interaction they chose on phone.

Are they getting the QR code/how to scan?
Have them think through the interaction on the phone.

Ask them to try a second interaction.

Repeat.

Ask them to try the third.

Repeat.

Post-test Questions:

Creating a profile

What are your general impressions of the process of creating a profile?
(If they chose to include an interaction) Why did you chose to include that interaction?
What did you like/enjoy about creating your profile?
What did you dislike about creating your profile?
Did you have any difficulty creating your profile?
What were your expectations after creating a profile?
Do you have any questions about creating a profile?
Based on the initial information you were given, would you have been motivated to create a profile so you could participate in ”Who’s That”?

Viewing other profiles

What are your general impressions of viewing profiles on the monitor?
Were their certain aspects of the profile that made you want to interact or not interact with the monitor? Please explain why.

Interactions

Was it comfortable interacting with the system? Why?
Did you feel you had enough time to interact with the profile?
Did you encounter any difficulties interacting with the profile?
What did you like or dislike about interacting with the profile?
How do you feel about using a QR code to interact with the profile?

Sharing information

How would you feel about the bluetooth detecting your presence in front of the monitor?
Do you see a need for such a system? Why?
What are the potential benefits of such a system?
What are the potential
Were your expectations of the system met? Please explain.
Did anything surprise you?
Did anything disappoint you?
Did anything delight you?

Results

Below are potential solutions to issues from user feedback that we were not able to address within the confines of the course project:

Import Data from Existing Social Networks: Users indicated that they would like the option to import information from existing social networks instead of manually entering information. In fact, one user indicated that he manually copied and pasted information from his Facebook page to fill out his profile. Allowing users to import their data would make the process of creating a profile less burdensome. Since many users carefully cultivate the information on their social networking profiles, importing this information would also give the application access to more granular information about the individual, increasing the likelihood of matching residents on unique interests (i.e. matching residents by their favorite bands as opposed to ‘music’).
Visual Cues for the Timing of Profile Rotations: When users held their phones up to scan the QR Code, they did not notice when profile (and hence the QR Code) had changed. The transition between profiles needs to be more obvious so that the user realizes the profile/QR Code has changed. It would also be useful to indicate of how much time is left before the next profile appears so that the user can pause the profile through the touchscreen control so they have more time to pull out their phone and capture the QR Code.
Make Messaging Easy and Informal: If messaging is cumbersome, residents will not be motivated to communicate with each other through our application. Currently, our concept is designed to allow users to receive messages through the “Who’s That?” web interface. Incorporating a messaging system within the “Who’s That?” mobile application would allow residents to receive notifications and view incoming messages without having to log in. Also, if we could use presence detection as identification into our application, users could send and receive messages through the touchscreen interface on the monitor.
Show Relevant Course Tags: User enjoyed learning about both similarities and differences they had with the profile they were viewing. Unfortunately, the cryptic nature of the course tags (i.e. EECS498) make them meaningless to people who were not in the same major or course. Since one of the goal’s of the application is to serve as a conversation starter between residents, course tags that are not relevant or of interest to the viewer should not be displayed.
Dynamic Profile Photos: The profile photo is one of the first things to catch the viewer’s eye. By allowing users to upload multiple photos and tag them appropriately, the application could select the profile photo that would be likely to appeal to the viewer, as well as reinforce the theme of the tags being displayed in the profile.
Keep Content Fresh: As voiced in our initial interviews with North Quad residents, users indicated that they would lose interest in the monitors if the content did not seem to change. One user suggested only having the profiles display at the beginning of each semester when the course tags would be updated. Profile content could also be tied to dorm initiatives in which residents would be encouraged to update their profiles to participate in a dorm-wide theme/challenge.

Comments (0)

Leave a comment

Who's That?