iOS 18 App Intents
Overview
As a way to stay informed on emerging technologies and how we could utilize them in our work, a team consisting of me and 1 Senior UX Designer—with the support of a UX Design Manager—researched the upcoming iOS 18 features that were announced at Apple’s Worldwide Developers Conference 2024 (WWDC24). With this research, we created a prototype that showcases the new capabilities of Siri 2.0 and how it works with Apple Intelligence to complete complex tasks for the user.
Challenge
The CVS Health app was in the midst of an overhaul, with an entirely new look, a new design system, and enhanced user experiences. The healthcare industry tends to lag behind other sectors when it comes to technology adoption, so even when the redesign is complete, it is crucial to stay updated with the latest and greatest. How might we leverage emerging and modern technologies in order to support experiences within the app as it continues to evolve in the future?
Goals
Propose utilizing a modern technology and demonstrate how the new feature can support the CVS SuperApp.
Role
UX Designer
UX Researcher
While researching emerging technologies, we were inspired by the new iOS features announced at Apple’s Worldwide Developers Conference 2024. In particular, we were most interested in the updates to Siri, App Intents, and the implementation of Apple Intelligence. At this point in our project, iOS 18 was still months away from release, and the beta version was still lacking key features. As a result, much of our background research went into what information was available to the public about iOS 18, and the current iteration of App Intents in iOS 17.
We also consulted with engineers in order to understand what is and isn’t feasible with App Intents. At the time, knowledge about the capabilities of iOS 18 App Intents were limited, so we learned what we could about iOS 17 App Intents.
There was no documentation on iOS 18’s design system, so we tried to follow the new design conventions as closely as we could based on screens from iOS 18 revealed thus far in trailers, presentations, etc.
Sophia frequently uses the CVS app to schedule vaccination appointments for her family, manage her mother’s medication refills, and save coupons. She has mastered using the app efficiently to make the most of her time. Despite this, having a busy lifestyle with two young children means her hands are always full, both figuratively and literally.
With the collaboration of the CVS Health app and native iOS 18 App Intents, Sophia is able to multitask interactions with the app using only her voice. The following deliverables show an example use case of how App Intents and Apple Intelligence can work together to help Sophia complete tasks within the CVS app without ever opening it.
Sophia is feeling unwell, and suspects she has a sinus infection. She decides to schedule a MinuteClinic appointment to speak with a physician about her symptoms. While preparing breakfast for her children, she verbally asks Siri to help her schedule an appointment.
With iOS 18, Siri has updated natural language capabilities. Because of this, Siri is able to understand Sophia's intentions, even if she stumbles over her words or changes her mind within the same sentence. Siri is also able to process that this appointment will be for Sophia, and not for anyone else in her CVS family account.
In the backend, App Intents will trigger depending on what the user says. Apps can create custom Intents, where they select words they want to associate with certain tasks. For example, the CVS app can set the phrase "schedule an appointment" to be associated with the CVS Scheduling flow. Since Sophia said the magic words, Siri pulls up the appropriate App Intent.
When creating a custom Intent, apps will specify what information they need from the user to complete a request. Siri will use this information to define the conversational flow when interacting with the user in order to receive this data. If the resolution requires the user to disambiguate between two or more choices, SiriKit will prompt the user to select one of the choices (either verbally or through tapping the options).
In this case, the CVS app has identified multiple available time slots for Sophia's preferred physician, and needs Sophia to pick the one that works best for her.
Siri + Apple Intelligence in iOS 18 is also able to pull information from other apps within a device in order to complete tasks, when necessary. After Sophia picks the virtual visit option, Siri is able to complete the rest of the process using information Sophia provided in previous CVS visits. Filling out this information is typically a 6-step process that is vastly simplified using iOS 18 Siri, Apple Intelligence, and App Intent's capabilities.
Sophia is able to successfully book a MinuteClinic appointment for her sinus infection through App Intents, all while multitasking her many responsibilities. She will receive a live activity from CVS when it's almost time for her appointment.
Data is stored locally in the user’s device. This can mitigate legal issues with PHI/PPI.
Simplify healthcare for seniors and people who struggle with technology by reducing number of steps.
There is a competitive advantage. If we utilize and create Intents before any of our competitors, Siri will use our app to complete certain tasks (scheduling an appointment, making a purchase, etc.) by default.
- Engineers were excited about idea and built a working demo (based on iOS 17 limitations).
- Executives were excited about idea; the prototype was shared and passed around throughout Design and Digital
- Create an Android version with Google Gemini
- Run a workshop to brainstorm more ways to utilize App Intents