Smart Home & Assistive Robot

GRA Project

  • Background
                • The Basics
                              • Challenges
                                            • Discover
                                                          • Define
                                                                        • Design
                                                                                      • Test
                                                                                                    • Next Step


                                                                                                                  The project focused on exploring possibilities of using assistive robots at smart home environment, with the intent to use them for home-based assistive robotics and healthcare applications.
                                                                                                                  Initial user research were required to understand human-robot interaction preferences in smart home environment.

                                                                                                                  The Basics




                                                                                                                  Qualtrics, Miro, Spreadsheet

                                                                                                                  My role

                                                                                                                  UX Researcher


                                                                                                                  Avery Ao, Wendy Gui

                                                                                                                  Research Goals


                                                                                                                  To understand the user viewpoints of robotic agents in a home environment.  


                                                                                                                  To understand how users feel assistive robotic agents would be integrated with smart home, and what tasks they would best serve.


                                                                                                                  To explore human natural language command for specifying daily activities and initiating requests.


                                                                                                                  To collect common natural language utterances and slots that users would use to command assistive robots in smart home environments.

                                                                                                                  Research Phases

                                                                                                                  There are 2 phases for the initial user research:

                                                                                                                  Phase 1: Robot Task @ Smart Homes

                                                                                                                  Recruitment & Screening

                                                                                                                  Target users were recruited through a research institute at Georgia Tech named TechSage. Emails, text messages and phone calls were made to reach out to potential participants.
                                                                                                                  To screen qualified participants from the pool, a screener survey was distributed to interested participants both online (filled by participants) and through phone calls (read by me and filled based on participants' answers).

                                                                                                                  📝 Check out the Screener Survey→

                                                                                                                  Focus Group/ Think Tank

                                                                                                                  In order to understand people's perspective of having assistive robotic agent at homes, and how would robots be integrated with smart home devices, focus groups were conducted to collect user opinions through group discussion and a brainstorming activity.
                                                                                                                  🧩 Springboard brainstorming method was used.
                                                                                                                  Participants were given If I (or the family member) _____, I wish the robot____ statement to help brainstorm robot tasks at smart homes.
                                                                                                                  To accommodate Covid health guidelines and concerns, focus groups were conducted both in-person and online through conference meetings.
                                                                                                                  🔎 3 sessions x 4-5 people/session were conducted.
                                                                                                                  Activity outline:
                                                                                                                  1. Ice-breaker and warm up discussion
                                                                                                                  2. Brainstorm #1: general home scenario
                                                                                                                  3. Brainstorm #2: assistive population home scenario
                                                                                                                  4. Summarize and end study
                                                                                                                  7 different scenarios were used each focus group session to collect as many different tasks as possible.
                                                                                                                  Focus Group Activity

                                                                                                                  Data Analysis

                                                                                                                  210 tasks were collected and coded based on task categories, including 214 IW statements and tasks mentioned in warm up discussions. For human-task codings, tasks were coded into 3 levels:
                                                                                                                  1. Primary task category:
                                                                                                                    Chores, Cognitive assistance, Emotional support,Entertainment, Health monitoring, Security and safety
                                                                                                                  2. Secondary task action:
                                                                                                                    Information management, fetch object, emergency notification, task reminder, etc.
                                                                                                                  3. Tertiary task description:
                                                                                                                    clean and wipe, detect supply shortage and auto refill, turn off devices, etc.

                                                                                                                  📝 Check out the Focus Group Outline→

                                                                                                                  Phase 2: Human Commands For Robots

                                                                                                                  After coding tasks based on both human preferences and robot capabilities, key tasks were selected out to be used to collect users' verbal command utterances for robot natural language processing.
                                                                                                                  Participants were recruited from phase 1's participants pool.

                                                                                                                  One-On-One Interview

                                                                                                                  Unlike traditional interviews, the purpose of the session was to collect people's verbal wordings and sentences for target daily tasks to help train the robot to translate natural language queries into known system specifications.
                                                                                                                  🤔 We wanted to understand how people request robot to complete certain tasks while smart home devices send out status alert or ask for actions.
                                                                                                                  To collect verbal commands, we aimed to collect keywords (slots) and sentences structures (utterance) that people use to send request through voice UI.
                                                                                                                  🔎 12 participants were interviewed.
                                                                                                                  5 scenarios were given to help participants visualize robot-human interaction cases, and each participant was prompt to verbalize how they would request the task, through which device, and expectations for possible failures.
                                                                                                                  Sample Scenario Prompts

                                                                                                                  Data Analysis

                                                                                                                  Utterances of each scenario were collected and organized based on action types and tasks.

                                                                                                                  📝 Check out Interview Outline→

                                                                                                                  Take Away

                                                                                                                  Accessible Study Setup

                                                                                                                  For elderly and people with physical disability, recruitment through phone call is necessary;
                                                                                                                  Accommodations for accessible online meeting should be available (caption, screen reader, etc.)

                                                                                                                  In-person study

                                                                                                                  Send map link of study location to participants to make sure they can access the location easily.

                                                                                                                  Cross-functional Communication

                                                                                                                  Follow up and prompt on scheduled tasks is important to make sure project meets the timeline.