Color & Math Game (Ionic)

1. Background

Mild Cognitive Impairment (MCI) is a condition that causes noticeable problems with memory, thinking, and reasoning, but not severe enough to significantly interfere with daily life or independent function. People with MCI often experience difficulties such as forgetting appointments, losing track of conversations, or struggling with decision-making. While these symptoms are less severe than those seen in dementia, they can still reduce quality of life, create frustration, and increase reliance on family members or caregivers. Importantly, individuals with MCI remain aware of many of their challenges, which can also contribute to feelings of anxiety, stress, and social withdrawal.

As the global population ages, the number of people affected by MCI is expected to grow. This has created an urgent need for supportive tools and interventions that can help people maintain independence, reduce caregiver burden, and improve overall well-being. One promising approach is the use of socially assistive robots that provide not only practical reminders and guidance but also emotional support and companionship.

During my internship with the PHAST Lab at SFSU, our lab is developing Carmen, a robot designed specifically to assist people with MCI in their daily lives. I helped to build the robot hardware and translated a set of cognitive games from an iOS app into a responsive web app. I used Ionic (Angular, HTML, CSS, TypeScript) to make the games more accessible across devices. Furthermore, we want to have the web app send and receive messages from the robot. For example: If the users input the correct answer, the robot will make some movement or show some colors that tell them they did it correctly.

Our final goal is to make a Carmen robot that aims to support memory through reminders for medications and appointments, guide users through everyday tasks, and provide conversational interaction to reduce loneliness. By combining assistive functions with social engagement, Carmen has the potential to enhance the autonomy and confidence of people living with MCI, while offering caregivers reassurance that their loved ones are supported.

2. What I Built

• Math Game — I created a numeric input game with a large on-screen keypad and clear prompts.
• Color Memory Game — I developed a sequence-recall game where users remember and select colors in the correct order.
• Toggle Feature — I added a switcher so users can move easily between games.

3. Why It Matters

• Encourages engagement through play while supporting cognitive training goals.
• Strengthens memory, attention, and visual sequencing—aligned with home-based interventions.
• Accessibility-first UI with large, clear components and responsive layouts.

4. My Contributions & Challenges

After I finished building the game, I pushed my code to GitHub so I could pull it onto Carmen, our robot platform, for testing. This testing phase helped me adjust the scaling of the text prompts, number pad, and response pop-up. I intentionally made these UI elements larger to accommodate users with cognitive impairments, ensuring that everything is easy to read and interact with.

When I did this work, there were a lot of problems setting up the environment and managing GitHub. My mentor, Mathew, and another teammate named Daniel worked together with me to solve those issues.

One of the challenges I encountered was configuring the correct scaling and resolution when deploying to Carmen. The Raspberry Pi didn’t automatically detect the default resolution of our tablet display. To fix this, we had to manually set the HDMI output through the Pi terminal. This step was important; without it, applications would overflow the screen and become unreadable.

Another issue occurred when updating our code on GitHub. I didn’t realize at the time that I needed to pull the latest changes before pushing my updates. As a result, I encountered a merge conflict while backing up and pushing the code, which ultimately required us to reinstall everything from scratch.

During this internship, I learned how to collaborate and communicate effectively with my teammates, mentor, and professor. It also helped me improve my self-study skills, problem-solving abilities, and gain more hands-on experience with coding.

5. Wrap Up

After two months of development, our team brought all of our work together and completed the final integration of the project. Each teammate contributed a unique skillset. One teammate focused on the 3D CAD design, creating the structural components needed for the robot and its setup. Another teammate, whose major is psychology, reviewed numerous research papers on how humanoid robots interact with people in physical environments. Their research helped guide our design choices so the robot could engage users in a more natural and effective way. Another teammate handled the Raspberry Pi setup, including studying how to connect a portable AC power system so the robot could move between locations without being tied to a single outlet. They also worked on installing the Dynamixel packages on the Pi to control the motors.

My main responsibility was building the cognitive games, but I also assisted with the Raspberry Pi setup and installing the Dynamixel package when we encountered issues. Throughout the internship, we held weekly meetings to update each other on progress, solve technical and robot-related challenges, and stay aligned as a team. At the end of the internship, we created a full PowerPoint presentation and shared our project with different labs to showcase what we accomplished.

6. Next Steps

• Currently, the web app runs on the Raspberry Pi but there is no connection between the web app and the robot software. I have built cognitive games, and the next step is to enable the web app to send and receive messages from the robot. For example: the robot could respond dynamically to a user’s answer in the math or color game.


• Add more levels and variety to the color game.


• Continue improving cross-device compatibility and performance.


• Work on deeper integration with robotic therapy (CARMEN) workflows.

7. Presentation Experience

Presenting our final project was one of the most challenging and memorable parts of the internship. In the week leading up to the presentation, I practiced my section repeatedly because each team member was responsible for presenting the part they contributed to. I wanted to challenge myself by presenting without reading directly from the slides, so I rehearsed until I felt confident explaining the cognitive games from memory. Even with all that preparation, I still felt extremely nervous knowing I would be speaking in front of many people, including professors, graduate students, and researchers.

During my actual presentation, something unexpected happened: the demo for my cognitive games failed to load due to a technical issue. The screen froze, and none of the interactions I planned to show appeared. For a moment, I completely froze too—I didn’t know what to say because my talk depended heavily on showing the demo. But after a few seconds, I tried my best to calm myself down. I took a breath and started explaining what the demo would have shown, describing the game flow and features from memory. It wasn’t smooth, and I could feel my nerves the whole time, but I pushed through and finished my part.

Even though my presentation didn’t go the way I hoped, I’m proud that I didn’t give up or walk off the stage. Speaking in front of a large audience without relying on notes was a personal challenge I set for myself, and I learned how important it is to stay composed when things go wrong. It was a valuable experience that will help me become more confident, more fluent, and better at capturing an audience’s attention in the future.

In the end, our team received certificates for completing the internship, and we took a group photo together—a positive and encouraging way to wrap up the entire program.

Demo