Apple Watch debuts 'Audio Intelligence' in beta: Live Rewind and Siri Recaps to review conversations

Apple has begun rolling out in beta, initially to developers and as part of iOS 27.3, its "Audio Intelligence" AI technology for the Apple Watch. It was announced in September during the "Surprise and Shine" hardware event. According to TechCrunch, it will later reach the public beta for broader testing before becoming available to all users.

The package includes four optional features for Apple Watch Series 12 owners: detecting important sounds, identifying songs, catching up on conversations and taking notes. According to the article, the features rely on the Apple Watch Series 12 and Apple Watch Ultra 4.

Sound detection is designed for people who are deaf or hard of hearing, and recognizes alerts such as sirens, alarms or doorbells. Song identification uses Shazam to recognize background music and display the title and artist.

The most sensitive features are those that listen to human conversations: Live Rewind and Siri Recaps. The article recalls the backlash against AI recording devices such as Meta's glasses and other wearables, and explains that Apple is trying to deliver usefulness while respecting the privacy of people nearby.

Live Rewind lets users "rewind" and see as text what someone has just said. A double press of the Digital Crown brings up a snippet of the last 15 seconds on the watch face. It is useful, for example, for verbal directions or a list of specials at a restaurant. Users can ignore it, in which case snippets are automatically deleted after 30 seconds, or save it to the standalone Siri app included with iOS 27, handy for remembering a name or taking notes during a quick presentation. There is also an "Ask Siri" option to ask about a technical term or an unfamiliar dish.

To let others know something has been captured, pressing the crown plays an alert sound and shows an animation on the watch. The gesture, the sound and the visual cue thus signal that what was said has been recorded.

Siri Recaps goes further: it generates high-level notes of conversations. Apple says it does not identify speakers, does not keep the original audio and does not provide verbatim transcripts. It summarizes the conversation with a short sentence and several key points, such as important dates or deadlines, prices or payment plans and other factual details. Apple pitches it for work meetings, parent-teacher meetings or personal training sessions. The idea is to mimic the notes the user would take themselves, without emotion or tone, and it is useful when one's hands are busy or one wants to focus on the conversation.

Siri Recaps does not record while listening to music with headphones or when noise cancellation is in use. At the end of the day, and also when unlocking the watch in the morning, users are prompted to review saved recaps and choose which to keep. They can go to the Notes app or, if more personal, be exported to Journal. If nothing is done, they are deleted automatically after seven days. It is turned on manually or on a schedule, for example only during work hours or never at night, and can be turned off at any time from Control Center. Apple says the system omits sensitive information, such as government IDs, financial and authentication data, certain personal identifiers and language promoting self-harm.

As for the architecture, it relies on Apple hardware, software and services: the S11 chip, the built-in microphone, Apple Intelligence models running on the iPhone and Private Cloud Compute. The always-on processor includes a secure enclave with a dedicated audio buffer that processes raw audio without it being accessible to anyone, not even Apple. Only the paired iPhone can access it, via a secure pairing mechanism.

The flow is as follows. Lightweight models on the S11 detect whether there is a conversation involving the wearer, and it is passed to the iPhone's secure enclave. There, on-device processing converts the audio into condensed text, stripped of filler words. That text is encrypted and sent to Private Cloud Compute, which produces the recap and returns it to the phone. The recap and intermediate summaries are deleted from PCC immediately afterward. Recaps are accessible only to the user, not to Apple, and are encrypted when synced across Apple devices. For Live Rewind, the watch also transfers audio to the iPhone to create the 15-second snippets. Apple says none of these features should affect the battery life of the watch or the phone.

It is worth noting that the article was published with a subsequent update to clarify the software's rollout status. The privacy guarantees described are Apple's claims, and their real-world effectiveness will have to be verified during testing.