|
Title: Everything You Need to Know About Apple Intelligence Post by: HCK on October 29, 2024, 04:05:05 pm Everything You Need to Know About Apple Intelligence
Apple Intelligence is what Apple is calling its artificial intelligence feature set, previewed at WWDC in June 2024. The first Apple Intelligence features launched with iOS 18.1, iPadOS 18.1, and macOS Sequoia (https://www.macrumors.com/roundup/macos-sequoia/) 15.1, but this is only an initial taste of what Apple has planned. (https://images.macrumors.com/article-new/2024/10/apple-intelligence-black-everything-you-need-to-know.jpg) There are a selection of Apple Intelligence features that are available now, along with additional capabilities that will be rolling out across the rest of 2024 and 2025. <h2 id="apple_intelligence_features_available_now">Apple Intelligence Features Available Now</h2> <h3 id="writing_tools">Writing Tools</h3> <img class="aligncenter size-full wp-image-960221" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-writing-tools.jpg" alt="" width="2000" height="1125" /> <ul> <li>Proofread text checks for spelling and grammar errors, including word choice and sentence structure. You can accept all suggestions with a tap or go through them one by one with explanations.</li> <li>Rewrite cleans up what you've written and shifts the tone without impacting your content. Options include Friendly, Professional, and Concise.</li> <li>You can select text and get a summary of it with Apple Intelligence. You can choose to create a paragraph, pull out key points, make a list, or create a table. Summaries are available in Mail, Messages, and more.</li> </ul> You can select any text on your iPhone (https://www.macrumors.com/guide/iphone/), iPad (https://www.macrumors.com/roundup/ipad/), or Mac and use Apple Intelligence to access Writing Tools for summaries and other features. <h3 id="siri">Siri</h3> <img class="aligncenter size-full wp-image-960222" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-siri.jpg" alt="" width="2000" height="1125" /> <ul> <li>There's a new glow around the edges of the display when Siri (https://www.macrumors.com/guide/siri/) is activated, applicable to ‌iPhone‌, ‌iPad‌, and CarPlay (https://www.macrumors.com/roundup/carplay/). On Mac, the ‌Siri‌ window can be placed anywhere. The glow animates responsively to the sound of your voice so you can tell when ‌Siri‌ is listening without interrupting other things you're doing.</li> <li>A double tap at the bottom of the display brings up the Type to ‌Siri‌ interface so you can type requests instead of speaking them. On Mac, you need to press the Command key twice to bring up Type to ‌Siri‌. Type to ‌Siri‌ includes suggested requests so you can get your questions answered faster.</li> <li>‌Siri‌ can maintain context between requests so you can ask a question and then reference it in a second reply. If you ask about the temperature in Raleigh, for example, and then follow up with "what's the humidity?" ‌Siri‌ should know you mean in Raleigh.</li> <li>If you stumble over your words when speaking to ‌Siri‌, or change what you're saying mid-sentence, ‌Siri‌ will follow along.</li> <li>‌Siri‌ has Apple's product knowledge and support base for answering questions about your device's features and settings, and can even find settings when you don't know the exact name by using natural language search.</li> </ul> <h3 id="mail">Mail</h3> <img class="aligncenter size-full wp-image-960207" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-smart-replies.jpg" alt="" width="2000" height="1125" /> <ul> <li>There is a summarize button for summarizing any of your incoming emails, plus you will see a brief summary of an email in your inbox list rather than the first few lines of the email.</li> <li>Mail surfaces time sensitive messages first when applicable, putting them at the top of your inbox so you see what's important right away.</li> <li>Smart Reply provides quick-tap responses to emails that you've been sent, with contextual options based on what's in the email.</li> <li>Multiple notifications from Mail will be summarized on your Lock Screen so you can see what's in an email without opening the app.</li> </ul> <h3 id="messages">Messages</h3> <img class="aligncenter size-full wp-image-960223" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-messages-smart-reply.jpg" alt="" width="2000" height="1125" /> <ul> <li>Messages has Smart Reply options for incoming texts, which analyze the content of messages to offer suggestions of what you might want to say.</li> <li>Multiple Messages notifications are summarized on your Lock Screen.</li> <li>You can use all of the Writing Tools features in the Messages app for proofreading and refining what you're planning to send.</li> </ul> <h3 id="photos">Photos</h3> <img class="aligncenter size-full wp-image-960208" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-memory-maker.jpg" alt="" width="2000" height="1125" /> <ul> <li>You can create a Memory Movie with just a description, such as "My cat in 2024," or "Orlando in the summer." The feature automatically picks relevant photos and chooses songs, but you can tweak through the Memory Mixes feature or choose a mood to guide the direction of the audio. You can also add in specific scenes and images you want to see throughout the memory when you're creating the prompt.</li> <li>Natural language search is available in Photos (https://www.macrumors.com/guide/photos/), so you can just describe what you're looking for, such as "Eric rollerskating while wearing green."</li> <li>Search can also find specific moments in video clips.</li> <li>Search offers up smart complete suggestions for narrowing down what you might want to find.</li> </ul> <h4>Clean Up</h4> The ‌Photos‌ app also includes "Clean Up," a feature that lets you remove unwanted objects from your photos. The Clean Up tool in the ‌Photos‌ app is able to automatically detect objects in an image that might not be wanted, but you can also tap, circle, or brush over an unwanted object to remove it. Zooming in on an image can help with using a finger as a brush to remove smaller blemishes and issues with an image, and it is intelligent enough not to remove part of a person even if a person or main subject is selected. <img class="aligncenter size-full wp-image-962933" src="https://images.macrumors.com/article-new/2024/08/iOS-18-Photos-Clean-Up-Feature-1.jpg" alt="" width="2500" height="1406" /> Clean Up works on all images in the ‌Photos‌ library, including older images and images captured by other devices like a point and shoot camera or a DSLR. <h3 id="transcription_summaries">Transcription Summaries</h3> In Notes and other apps, you can record audio and get a transcript along with a summary of your transcript, which is useful for recording lectures and other audio. Transcription isn't an Apple Intelligence feature, but summaries are. <h3 id="focus_modes">Focus Modes</h3> There is a dedicated Reduce Interruptions Focus Mode that only shows you important notifications that need attention while filtering out everything else. <img class="aligncenter size-full wp-image-960224" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-focus-mode-changes.jpg" alt="" width="2000" height="1125" /> When customizing an existing Focus mode or creating a new one, there is a new toggle to turn on Intelligent Breakthrough and Silencing. This option allows important notifications to interrupt you, while muting notifications that are not important, which is the same thing that the Reduce Notifications Focus does. You can override the notification settings for specific apps and people, which is how it worked prior to iOS 18.1. <img class="aligncenter size-full wp-image-960225" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-focus-reduce-interruptions.jpg" alt="" width="2000" height="1125" /> <h3 id="notification_summaries">Notification Summaries</h3> Your incoming notifications are summarized so you can see what's new for each app at a glance. <img class="aligncenter size-full wp-image-968372" src="https://images.macrumors.com/article-new/2024/09/ios-18-apple-intelligence-notification-summary.jpg" alt="notifications" width="2000" height="1125" /> <h3 id="phone">Phone</h3> Summaries of transcriptions generated from your phone calls are supported in iOS 18.1. The iOS 18.1 update adds the option to record a phone call and get a transcription, which is not an Apple Intelligence feature. What does require Apple Intelligence, though, is getting a summary from that transcription without reading through the entire thing. Note that you can start a recording by tapping on the record button in the upper left corner of the display when on a phone call. All participants are notified that the call is being recorded before the recording starts. <img class="aligncenter size-full wp-image-960199" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-call-recording-1.jpg" alt="" width="5416" height="3047" /> Recorded phone calls are stored in the Notes app, where you can tap in to view a transcript and get a summary generated from that transcript. <h3 id="safari">Safari</h3> When reading an article in Reader Mode, there is an option to have Apple Intelligence summarize the article for you. <img class="aligncenter size-full wp-image-960226" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-safari-summarize.jpg" alt="" width="2000" height="1125" /> <h2 id="apple_intelligence_features_coming_in_ios_18.2">Apple Intelligence Features Coming in iOS 18.2</h2> The next set of Apple Intelligence features will come in iOS 18.2, iPadOS 18.2, and ‌macOS Sequoia‌ 15.2, and this is when Apple plans to release Image Playground (https://www.macrumors.com/guide/image-playground/), Genmoji, and ChatGPT ‌Siri‌ integration. These features are being beta tested right now. Apple has a waitlist for ‌Image Playground‌, Genmoji, and Image Wand, and these features will be rolling out to developers over the coming weeks so that Apple can collect feedback and make improvements to the image generation capabilities before a public launch later this year. <h3 id="image_playground">Image Playground</h3> ‌Image Playground‌ can be used to generate images in cartoon-like styles using a text-based prompt. Apple has built-in suggestions and concepts that you can choose from, like costumes, locations, items, and themes. (https://images.macrumors.com/article-new/2024/10/ios-18-2-image-playground-crocodile.jpg) While ‌Image Playground‌ is a standalone app, it is also integrated into the Messages and Notes app. In these apps, Apple can use context from what you've typed for image generation suggestions. For inspiration, you can upload or take a photo, and then get a cartoon AI version of it, and you can also use images of friends and family members pulled from the People album in the ‌Photos‌ app. You can start with a base suggestion or photo and then continue to add to it until you get what you want. You can remove suggestions at any time, and save your favorite creations for use in other apps. Anytime you create something with ‌Image Playground‌, you'll get multiple options so you can choose the best one. There are only Animation and Illustration styles for ‌Image Playground‌, so there is no option for creating photorealistic images. <h3 id="image_wand">Image Wand</h3> Image Wand is basically the same thing as ‌Image Playground‌, but in the Notes app. When you have notes that you've taken, you can circle an empty spot or some text and Image Wand will add a contextually relevant image. (https://images.macrumors.com/article-new/2024/08/apple-intelligence-image-wand.jpg) So if you have notes on photosynthesis, you can add in an image of a plant under the sun. Image Wand isn't able to generate complex images, so if you want a picture of the internal structure of mitochondria, you're out of luck, but it can make a stylized image featuring the organelle. On an ‌iPad‌, you can draw a rough sketch of what you want to add to your notes with an Apple Pencil (https://www.macrumors.com/guide/apple-pencil/), and then use Image Wand to generate something more polished. <h3 id="genmoji">Genmoji</h3> Genmoji are custom emoji characters that you can create using a text-based description. If there's an emoji you can't find but need, like a duck eating a sandwich or an alligator skateboarding, Genmoji can make it for you. (https://images.macrumors.com/article-new/2024/08/ios-18-genmoji.jpg) Genmoji aren't too far off from ‌Image Playground‌ images, but the generation system tends to want to add a person for a lot of requests. You can choose yourself or a friend or family member, or just use a generic emoji character as your base. In Messages and other apps, Genmoji behave like emoji, but they're not going to display properly for anyone running an operating system earlier than iOS 18.1, or on an Android device. In iOS 18.1 and iOS 18.2, they work like emoji, but Genmoji are displayed as a blank box and an accompanying full-size image on other versions of iOS. If you have iOS 18.1 and iOS 18.2 and someone sends a Genmoji, you can long press on it and tap the "Emoji Details" button to see what prompt was used to create it, plus you can add it to your own Genmoji/sticker collection to use. To create a Genmoji character, tap into the Emoji keyboard and tap on the Emoji with a "+" button next to the search bar. From there, you can type in your idea. <h3 id="siri_chatgpt_integration">Siri ChatGPT Integration</h3> ‌Siri‌ ChatGPT integration lets ‌Siri‌ hand requests over to OpenAI's ChatGPT. ChatGPT is off by default, but you can turn it on in the Apple Intelligence and ‌Siri‌ section of the Settings app. (https://images.macrumors.com/article-new/2024/08/iOS-18-ChatGPT.jpg) If ChatGPT integration is enabled, ‌Siri‌ will consult ChatGPT for complex requests. Complex requests might include creating an image, generating text from scratch, making recipe ideas based on what's in your refrigerator, describing what's in a photo, and more. ‌Siri‌ will analyze each request to see if it's something that needs to be answered by ChatGPT, but you can also automatically invoke ChatGPT for a request by using a request like "Ask ChatGPT to give me a chocolate chip cookie recipe." ‌Siri‌ will ask your permission before querying ChatGPT, but there is an option to turn off that extra permission step by toggling off "Confirm ChatGPT Requests" in the ChatGPT section of Settings. You don't need an account to use ChatGPT, and it is free, but if you have a paid account, you can sign in. If you're not signed in, OpenAI does not store any of your ChatGPT requests, nor is your information used for training ChatGPT. If you sign in, ChatGPT can save a copy of your queries. Apple does not store ChatGPT queries. ChatGPT can be used with ‌Siri‌, but it is also integrated into Writing Tools and Visual Intelligence. With Writing Tools, ChatGPT can generate text, and with Visual Intelligence, ChatGPT can answer questions about what the Camera sees. <h3 id="visual_intelligence">Visual Intelligence</h3> Visual Intelligence is an iPhone 16 (https://www.macrumors.com/roundup/iphone-16/) feature that uses the Camera Control button. If you long press it, you can get into Visual Intelligence mode, where the Camera app can be used to identify what's around you. (https://images.macrumors.com/article-new/2024/10/Apple-Visual-Intelligence.jpg) If you point the camera at a store, for example, you can see ratings, hours, and other information. If you take a photo of an object, you can get more information about the object from ChatGPT, or use it with Google Search to find similar images. The Google Search feature is a good way to search for products that you want to find. Other Visual Intelligence features include reading text out loud, detecting phone numbers and addresses to add them to the Contacts app, copying text, and summarizing long passages of text. <h3 id="writing_tools">Writing Tools</h3> Writing Tools is in iOS 18.1, but in iOS 18.2, you can make more open-ended changes to what you've written. You can come up with your own tone changes, so if you want something to be more flowery or elaborate, Writing Tools can make it happen. You can also ask for your text to be converted into a different format, like a poem. Writing Tools also has ChatGPT integration in iOS 18.2 so if you want to generate text from scratch, you can do so with ChatGPT. <h3 id="additional_languages">Additional Languages</h3> In iOS 18.2, Apple Intelligence supports localized English in Australia, Canada, New Zealand, South Africa, Ireland, and the UK in addition to U.S. English. <h2 id="apple_intelligence_features_coming_later">Apple Intelligence Features Coming Later</h2> There are additional Apple Intelligence features that will be coming in updates to iOS 18 (https://www.macrumors.com/roundup/ios-18/) in 2025. <h3 id="priority_notifications">Priority Notifications</h3> Priority notifications will show up at the top of your notification stack, so you can get to what's most important first. <h3 id="siri">Siri</h3> Some initial ‌Siri‌ updates are available in iOS 18.1, such as ‌Siri‌'s new glow that encompasses the edges of the display, and ChatGPT integration is coming in iOS 18.2, but we'll need to wait for iOS 18.3 and iOS 18.4 for additional ‌Siri‌ capabilities. Apple is working on onscreen awareness, personal context, and the ability to take more actions in and across apps. Onscreen awareness will let ‌Siri‌ take actions when you ask something about what's on your display. If you're looking at a photo and want to message it to your friend Eric, you'll be able to tell ‌Siri‌ to "Send this picture to Eric," and ‌Siri‌ will understand and do it. Personal context will let ‌Siri‌ do more with your personal data like emails and messages. This is an on-device feature, and it will let ‌Siri‌ learn more about you, who you're communicating with, and how you use your device. Personal context will let you do things like ask ‌Siri‌ to find a specific message, or remind you when you took a photo that you're looking for. The ‌Siri‌ option to take more actions in and across apps will drastically improve what ‌Siri‌ is capable of. You'll be able to move files from one app to another and control app functions with ‌Siri‌ that you never could before. It'll work in third-party apps as well as Apple's own apps. <h3 id="macos_features">macOS Features</h3> Memory Maker is in iOS 18.1 and iPadOS 18.1, but not ‌macOS Sequoia‌ 15.1. Genmoji is in iOS 18.2 and iPadOS 18.2, but not ‌macOS Sequoia‌ 15.2. Apple plans to release Memory Maker and Genmoji for ‌macOS Sequoia‌ at a later time. <h2 id="when_to_expect_more_apple_intelligence_features">When to Expect More Apple Intelligence Features</h2> More Apple Intelligence features will come in iOS 18.2, iOS 18.3, and iOS 18.4. We'll get iOS 18.2 in December, likely around the middle of the month. Apple is already testing iOS 18.2 with developers. In late January or so, we'll get iOS 18.3, which could potentially have some new ‌Siri‌ features. iOS 18.4, which isn't expected until around March 2025, will have the bulk of the ‌Siri‌ Apple Intelligence features. We're also expecting to see Apple roll out support for additional languages in 2025. <h2 id="apple_intelligence_device_requirements">Apple Intelligence Device Requirements</h2> Apple Intelligence requires a device with one of Apple's newest chips and 8GB RAM. Eligible models are listed below. <ul> <li>All ‌iPhone 16‌ models</li> <li>iPhone 15 (https://www.macrumors.com/roundup/iphone-15/) Pro and ‌iPhone 15‌ Pro Max</li> <li>All Apple silicon iPads</li> <li>A17 Pro iPad mini (https://www.macrumors.com/roundup/ipad-mini/)</li> <li>All Apple silicon Macs</li> </ul> <h2 id="apple_intelligence_waitlist">Apple Intelligence Waitlist</h2> When you first install iOS 18.1, iPadOS 18.1, and ‌macOS Sequoia‌ 15.1, you have to go to the Settings app and join the Apple Intelligence waitlist. Apple uses a waitlist mechanic to ensure that behind the scenes downloads go smoothly and that the system isn't overloaded. While on the waitlist, Apple devices download necessary files for on-device processing, and the waitlist should only take a few hours at most. The waitlist is on a per-account basis, so you only need to sign up for it on one device to have access on multiple devices. ‌Image Playground‌, Genmoji, and Image Wand in iOS 18.2 require a second waitlist, but that's only relevant if you're running the iOS 18.2 developer beta. If you are, note that Apple is rolling out access to these features over the coming weeks. <h2 id="apple_intelligence_settings">Apple Intelligence Settings</h2> In the Privacy and Security section of the Settings app you can access an Apple Intelligence Report that lets you export your Apple Intelligence data as part of Apple's promise for transparency around Apple Intelligence. Biometric authentication is required to access and export Apple Intelligence data. <img class="aligncenter size-full wp-image-960227" src="https://images.macrumors.com/article-new/2024/07/ios-18-1-apple-intelligence-report.jpg" alt="" width="2000" height="1125" /> You can also disable Apple Intelligence by toggling off the setting under the Apple Intelligence and ‌Siri‌ section in the Settings app. <h2 id="apple_intelligence_availability">Apple Intelligence Availability</h2> Apple Intelligence is only available in U.S. English at this time, and it is not available to in the European Union (‌iPhone‌ and ‌iPad‌) or China. Device region and language need to be set to the United States. In iOS 18.2, localized English in Australia, Canada, New Zealand, South Africa, and the UK is supported in addition to U.S. English. Apple plans to add support for additional languages in 2025, like Chinese, English (India), English (Singapore), French, German, Italian, Japanese, Korean, Portuguese, Spanish, and Vietnamese. Apple Intelligence will launch on the ‌iPhone‌ and the ‌iPad‌ in the European Union in April 2025. <div class="linkback">Related Roundups: iOS 18 (https://www.macrumors.com/roundup/ios-18/), iPadOS 18 (https://www.macrumors.com/roundup/ipados-18/), macOS Sequoia (https://www.macrumors.com/roundup/macos-sequoia/)</div><div class="linkback">Related Forums: iOS 18 (https://forums.macrumors.com/forums/ios-18.244), iPadOS 18 (https://forums.macrumors.com/forums/ipados-18.245), macOS Sequoia (https://forums.macrumors.com/forums/macos-sequoia.246)</div> This article, "Everything You Need to Know About Apple Intelligence (https://www.macrumors.com/guide/apple-intelligence/)" first appeared on MacRumors.com (https://www.macrumors.com) Discuss this article (https://forums.macrumors.com/threads/everything-you-need-to-know-about-apple-intelligence.2441182/) in our forums Source: Everything You Need to Know About Apple Intelligence (https://www.macrumors.com/guide/apple-intelligence/) |