Tools for Abey

PRoject Description

Tools for Abey is a user-centered design project completed in collaboration with an Occupational Therapist, Kaitlin Olivieri, and the intended user, Abey, a 12 year old boy with Cerebral Palsy who is unable to use a mouse, keyboard, or voice control to access a computer. The goal was to design a system which would allow Abey to use his computer to communicate, participate in class, complete homework assignments, browse the Internet, write emails/texts, play games, compose music, and make movies. The system combines Switch Control, a powerful accessibility feature built into the Mac OS X operating system, and a dual switch array to allow Abey to access a Mac desktop computer or laptop by using upper body momentum to hit switches. We designed and built custom panels within the Switch Control panel editor to streamline common tasks and actions for different apps, specific to Abey’s needs.

USER CONSULTATION

To understand both Abey’s needs and strengths, Kaitlin and I conducted interviews with Abey and his mom, Michelle. Talking to Abey and his family was crucial through every step of the design and development process.

When we met Abey, he was communicating using Tell Us Abey, a 6 switch array and program designed and written by his parents, Michelle and Josh. Tell Us Abey has six color coded switches that correspond to key strokes (1-6 on the keyboard). The system allows Abey to type out letters on a white screen to communicate with his teachers, classmates, friends, and family. Abey also uses Tell Us Abey to write papers, answer worksheets, and to do algebra and other math computations using the grid math program. The switches are accessed by Abey being placed in standing, supported at his pelvis by a person, and using trunk momentum to hit the switches.

While the system effectively helps Abey to communicate, he does not have a way to access the Internet, save and print documents, or access other applications on the computer such as word processing programs, movie/music editing programs, email, or games. He is unable to perform a Google search, write a text message or email, play chess, or write music unless he dictates to others through typing. Additionally, everything is programmed manually for the program that runs locally on his computer, making it difficult for the system to adapt and evolve to fit Abey's needs as he becomes older.

The initial meeting with Abey and Michelle established two needs which guided our research and design/development process: Abey’s access to various computer programs  and the ability to access the computer independently.

Secondary Research

In order to familiarize ourselves with the current landscape, the first step was to read as much literature as possible about computer access for for individuals with severe motor and physical impairments. We researched different switch and scanning systems, and various ways to access switches.

After completing a literature review and conducting a landscape review, we determined that using Mac OS X Switch Control options on a Mac laptop with customized panels and two directional scanning would best fit Abey's needs, as it would allow him to:

• Access any software or apps on a Mac
• Type in any program, software, or app using the computer keyboard
• Control functions on the computer with the cross directional scanner

Two directional scanning is based on a vertical and horizontal lines being selected by the user to select a particular point on the computer screen. It allows a user increased precision in their selection of items, as well as the ability to drag items on a screen. A dual switch system can be used to control the scan with one switch and select with a second switch.

The second need for Abey and Michelle is the ability to access a switch system independently. Gesture-based navigation, eye-gaze, and brain-computer interface are all potential options for Abey to navigate the switch system independently.

DESIGNING/PROTOTYPING

Before we could begin designing custom panels in Switch Control to fit Abey's needs, we needed to introduce the cross scanner to Abey, as this was a completely new skill for him. The initial goal was to help Abey learn that he could select anything on the computer screen. We decided to use the chess application on a Macbook Pro as a way to introduce the skills to Abey, since he loves chess and also because the grid and large game pieces made it easy to model how to use the scanner to select something on the screen.

Chess was a way to start small and scaffold the learning of the new skills required to use Mac Switch Control, which are listed in the task analysis below:

1. Use the dual switch system to control the scan with one switch and select with a second switch
2. Access software on the computer (open the chess game)
3. Open the "move and click" cross scanner
4. Use the timing and precision of the cross scanner to select an item
5. Move the cursor to a different area of the screen

User testing & Insights

After setting up the dual switch system to interact with the Mac laptop using Switch Control, we conducted a user trial with Abey using the chess game to familiarize him with opening up applications on the laptop with dual switch system, as well as to use the cross scanner to play chess.

The first area which was problematic was the timing of the autoscan for selecting the pointer function “move and click” with one switch. Due to the location (1st item on menu) and the auto-scan, Abey was having difficulty timing his movements to hit the switch. Luckily, everything is customizable in Switch Control. We were able to increase the time spent on each option to 10 seconds, which improved Abey's accuracy.

The second issue was the amount of loops the cross scanner completes before it turns off if not activated. We originally had the settings set to complete one loop, and then moved it to 2 loops. In our follow up with Abey, he asked for 4 loops (demonstrated in the video below).  The loops increased Abey’s ability to time his movements when activating the switch, so that he had more time to select, and didn't have to keep starting the scan over again. 

The third area of difficulty for Abey we came across during user testing was the number of options on the default pointer panel, which we were using with Abey to play Chess. There are many options on the panel for mouse functions that he did not need to play chess such as move, left click, double click, scroll up, and scroll down. If he missed his switch hit for “move and click,” the first option, he would have to scan through each option.  Therefore, we designed a custom panel for the chess game which had two buttons (“move and click” and “open chess”). This custom panel was much more efficient for Abey when he tried to play the game the next time. 

One other feature we explored was changing the transparency of the panel. We found that the visibility was not appropriate for Abey and he could see it better when it was darker.

All of these insights led to the conclusion that each custom panel will have to be trialled with Abey in order to make sure it is the most efficient use of buttons for how he interacts with the program. Timing, size, color, and number of options are all incredibly important when considering usability, and are specific to each user. These were realizations we were able to make only after watching Abey interact with the prototype. All of these customizations can be made within the Panel Editor, located in the Switch Control options under Accessibility, without having to do any programming. This is perhaps one of the biggest benefits of Switch Control, as the system Abey is currently using can only be customized by updating the code. With Switch Control, anyone, including Abey's teachers or OTs, would have the ability to make small customizations to meet Abey's needs, without having any programming knowledge.

The initial user trial overall was a success, as Abey was able to pick up on the new skill, after a few trials and customizations. He also expressed to us that he was excited to try an Internet search the next time. Abey and Michelle both felt that Switch Control could be a useful for gaining increased access to various functions on the computer (music programs, Internet searches, research papers).

DESIGNING CUSTOM PANELS

After the initial user testing session with Abey, we designed and built custom panels within the Switch Control panel editor to streamline common tasks and actions for different apps, specific to Abey’s needs. The video below shows a Google search being performed using the dual switch system and two-directional scanning, and a custom keyboard designed for Abey (we redesigned the default keyboard to look more like the keyboard Abey is accustomed to using with Tell Us Abey).

The timing and location of each letter/function on the keyboard still need to be tested with Abey. As we learned, the customization can only be perfected through user testing.

One unique benefit of using the custom panels we discovered is the ability to record shortcuts for buttons. For example, if we were building a custom panel for a word processing program, we could record the shortcut command+p for a button, and call it the print button. This would eliminate several steps for Abey, as he would not have to use the cross scanner to go to the menu and select print from the menu, but could rather use a single button press to print.

We also built a custom panel for a music composing program, Sibelius, which Abey had expressed interest in using independently to write music. Each button has a recorded shortcut to streamline common tasks and actions.  

In order to increase efficiency for opening email, we created a custom panel with buttons which were set to store specific text phrases such as Abey's email address and password, so that he could easily fill in the username and password field without having to type out his entire email address and password using the keyboard. 

CONCLUSIONS

The results of the user testing were promising, as Abey was able to pick up the new skill of using the two-directional scanning and successfully select and move items on the screen. The next steps we discussed with Abey and his family were:

• Increase practice with the cross scanner during occupational therapy sessions
• Practice using the custom keyboard in conjunction with the two-directional scanning
• Test timing and different needs for custom panels
• Further work with Abey accessing the switches independently

At the conclusion of the project, we were quite excited to find out that Abey's family would likely be purchasing either a Mac or an iPad so that Abey could start using Switch Control in the future to access the Internet and other software/apps.