Apple launched its iPhone 16 series on Monday (September 9, 2024), overhauling the phones with software capabilities that capitalise on advances Gen AI. Called Apple Intelligence, the feature, for now, offers AI-powered writing tools, advanced Siri and photo-editing capabilities. These features will be rolled out to iPhone 15 Pro and Pro Max, and all iPhone 16 models. With the release of the iOS 18.1 beta update in October, Apple Intelligence features will be available to users. 

Text features

With the roll out, users will find a summarisation feature in email, and two-line summaries in alerts so they can decide whether to click the notification or not. They can also prioritise notifications to make the most important ones are on top. The AI will also proof-read texts and suggest alternatives based on the tone and wording. 

Apple Intelligence will be able to proofread your text, rewrite different versions of a piece of content adjusting the tone and wording and summarise selected text with a tap. There’s also a feature called Smart Reply that’s been demonstrated quite a bit already where the bot gives users contextual suggestions on drafting mails.

Some early testers did complain about the feature’s unreliability as they often found it to misunderstand a news headline or a video clip and summarised it incorrectly or interchanged the names. These “hallucinations” are a commonplace in large language models. 

Audio features

With the iOS 18 update, users can record phone calls and get a transcription of the conversation without having to access any third-party app. Once the feature is activated, all parties will be informed that the call is being recorded. Users can also record audio within the Notes app so the call can be transcribed. 

Image features

Apple Intelligence has found applications in the Photos app that were found to be generally more useful by reviewers, like the search option where users can type in a description in natural language to find specific images from a time or a place. This even works for time stamps within videos. The Clean Up feature helps users remove unwanted objects from images. Users can also make a movie using media from their Photos library by writing out a prompt that describes the narrative. 

A new Siri

The decade-old smart assistant has been spruced up to be sharper and more personalised. During the Glowtime demo, Apple showed how Siri 2.0 will now appear in rainbow colours on the sides and edges of the screen of an iPhone, iPad or CarPlay and on a Mac. The voice assistant will be accessible even with written prompts and is expected to be generally smarter at understanding the way in which humans speak. For example, up until now Siri couldn’t really comprehend a pause in a conversation or how follow-up questions worked. Apple has also claimed that more complex questions will be sent to the Private Cloud Compute server which acts as a computational extension to your device but promises to not retain any data. 

Upcoming features

However, Apple has forewarned that it could take a few years for them to fully realise their vision of what the revamped Siri is supposed to be, which is to have a complete contextual understanding of what’s happening on screen. Siri will also one day be able to complete in-app requests like making photo edits in an image editing app using pictures in your Photos app. 

Besides these, Apple plans to release other features like Genmoji and Image Playground available with the iOS 18.2 update which will mostly be out in December. Users can produce entirely original images with the Image Playground feature using a descriptive prompt, ideas and even based on a person from their Photos library. The style can be customised and changes can be made to match whatever is on the users’ Freeform board. 

On the other hand, Genmoji can be used to make custom emojis based on a description or even an image of a real person from your library. 

Apple also aims to connect OpenAI’s chatbot ChatGPT with Siri so more advanced questions can be processed by ChatGPT. 

Another new feature that generated considerable buzz was Visual Intelligence which will be a rolled out “later this year,” the company said. Seemingly a take on Google Lens, the feature allows users to search and learn about it by just taking a photo. Users can activate it using a Camera Control side button and then take a picture of say a store in real life to get information around it or take a photo of a film poster and add it as an event to your calendar.