**The Power of Live Text and Apple's Ongoing Efforts to Enhance iOS**

Apple's on-device intelligence team has been making significant strides in recent years, and this year has seen some particularly exciting developments. One feature that has caught my attention is Live Text, a system-wide text recognition technology that has become one of my favorite new features. While Google has had its own versions of similar technology for quite some time, I have to say that Apple's implementation is truly impressive.

Live Text is not something that I expected or anticipated, but it's now become one of the most useful features in iOS. It uses machine learning models and the Apple Neural Engine Core to scan and index text in real-time, without the need for background scanning or indexing. This means that when you open a photo with text on it, whether it's typed or handwritten, Live Text will make it live. You can even take a photo of something with text and have it automatically recognize and transcribe it. The machine learning models behind Live Text are incredibly powerful, and I've found myself using it all the time.

But that's not all - Live Text also has a particular set of skills when it comes to identifying objects in photos. It can look up images and identify them using Siri's knowledge system. This is like something out of science fiction, and I'm still getting used to it. Even simple tasks like recognizing numbers or sealing phone numbers are now easier than ever before. And with the integration of Live Text into Spotlight search, you can access this feature right from the lock screen.

**The Rise of Siri: Improved Speech Recognition and Offline Query Processing**

But what really caught my attention was the improvements made to Siri's speech recognition capabilities. Apple has been working hard to improve Siri's sequential inference, allowing it to better keep context during conversations and hold onto user data. This means that Siri no longer needs to round-trip to Apple servers for basic queries, making the whole system feel much more responsive.

I've seen this firsthand with the Mac moving from Intel to M1, and it's clear that Apple is doing something similar here. By integrating these improvements into the core of Siri, Apple is finally closing the gap between its AI capabilities and those of its competitors. And let's not forget about the addition of a feature I've been lobbying for since Remember Me was introduced: the ability to share files, photos, web pages, and more with just a few taps.

**Brilliant.org: A Resource for Learning Algorithms and Neural Networks**

As I continued exploring iOS 16, I found myself at a website that has become an indispensable resource in my own journey of learning algorithms and neural networks. Brilliant.org is an amazing platform that offers courses on everything from character recognition like Live Text to search like Spotlight, all the way to advanced topics like math, science, logic, and more.

What I love about Brilliant.org is that it's built on a process of learning while doing and solving real challenges in real-time with no memorization required. The instant feedback system coaches you bit by bit as you improve your understanding of these foundational concepts behind some of the most important new careers and technologies out there. With Brilliant.org, you can rapidly improve and learn the skills necessary to tackle even the most complex topics.

**The Future of iOS and Apple's Plans for AI**

As I looked into the future of iOS and Apple's plans for AI, it was clear that this is a recurring theme throughout this preview. The combination of Live Text, Spotlight search, Siri improvements, and Brilliant.org has laid the foundation for an incredibly powerful ecosystem that will continue to shape the way we interact with our devices.

While there are still many features to be explored, I'm excited to see where Apple's AI capabilities take us next. From the on-device computer vision system to the continued development of AR OOS, it's clear that Apple is committed to pushing the boundaries of what's possible with technology. And as we move forward into a future filled with endless possibilities and challenges, one thing is certain: Apple will be there at the forefront, leading the charge.

**Conclusion**

As I concluded my exploration of iOS 16, I couldn't help but feel a sense of excitement for the future of Apple's operating system. With Live Text, Siri improvements, Spotlight search, and Brilliant.org, the foundation has been laid for an ecosystem that will continue to shape the way we interact with our devices. Whether you're a seasoned developer or just starting out on your journey to learn about algorithms and neural networks, there's never been a better time to get involved.

By exploring these features and resources, you'll be well on your way to unlocking the full potential of your device. And who knows? You might just find yourself on your way to becoming a brilliant developer or AI expert. The possibilities are endless, and I can only imagine what the future holds for Apple's ongoing efforts to enhance iOS.

"WEBVTTKind: captionsLanguage: en- SharePlay,ScreenShare, Shared with You,FaceTime for Everyone, Focus for Anyone,Status, Notification Summaries,More Better Maps, Notes and Weather,Much Better Safari, IDs, Keys,On-Device Siri, Live Text,and Wide Translate, Lens, I mean Look Up,Brighter Spotlight, more MusicalMemories, Health Sharing,Mobility Caring, PrivacyReports, Mail Protection,Private Relay, Legacy,Recovery, and so much more.It's the iOS 15 public beta.It supports every iPhone,going back to the 2015 6S and 2016 SE.Yes, seriously, you can grab it now,but just remember beta means beta.So always practice safe updateand backup, then buckle up.Sponsored by Brilliant.I'm breaking down all themost important featuresfrom all of Apple's public betas,and doing a slew oftechnical deep dives as well.Click the subscribe button and bellso you won't miss any of them.Zoom won the pandemic, wasn't even close.It got verb, but thanks to FaceTime'sease of use integration and absenceof continual security malfeasanceit also saw a ton of Applesto Apples usage as well.And that's what's beingbuilt on in iOS 15.Spatial audio which builds a3D soundstage all around youis being used to virtuallyisolate and place everyonein a group call relative towhere they appear on the callto help them soundclearer and more natural.And at this point, justspatial audio all the things.There's a new grid view because everyonefloating and throbbing onscreen is fun for a demoor a few minutes, but just waytoo much all day every day.The grid is currentlylimited to six people though,while group calls can still go over 30.You can turn on voice isolation,which is like the oppositeof active noise cancellation,helping the other person on the other endhear what you're sayingwithout all the din of thebackground, all the clutter,or you can flip to wide spectrum,which is like the oppositeof transparency modeto amplify every littlesound in your environmentso the person you're talkingto can hear what you hear.Like to better include themin a happy birthday partyor a musical moment.If you want to hide any personal itemsor just the mess behind you,you can turn on portrait mode.The controls in general canbe a little tricky to findor can change state or place unexpectedlyso I hope Apple just keepsiterating on that partbefore release.And all of those things are availableto third-party apps as well.So Zoom or anyone else coulduse the built-in portrait modeif they wanted to, at leastfrom release point on.Apple's really on completeoverdrive this yearwhen it comes to third-party integrationsfor a bunch of the new features,which you know what's goingon in the U.S. and EU or not,you just really love to see it.You can also create FaceTimelinks now and either share themor attach them to calendar events.And in addition to Apple devices,Android and Windows userscan open them up on theweb to join in the call,no account or login needed.Anyone using the link needsto be approved by someone,anyone already on the call.And if you accidentally social the linkand someone unintended joins,anyone already on the callhas 30 seconds to just yotethem right back out again.Then there's Shareplay whichturns your FaceTime callinto a group music or video experience.And yes, you'll need some family,friends or colleagues oniOS 15 to test that with,but everything is justautomatically in sync for everyone.All controls work foreveryone. If anyone talks,the volume will adjust soyou can promptly shush themand point them towards the built-inmessaging functionality insteadas the laws of nature and decency demand.And they can thank youlater with the Memoji stylesand poses, because we all know it'sall about the Emoji and Memoji.For quick songs, Tik Tok, stuff like thatI think it will be greatright on the iPhone.For longer form shows movieslike TV or Disney plus,especially Twitch streams,really long Twitch streams,you can start them on your iPhone,but watch them on your Apple TV,which I think will be far better,just far more comfortable andkind of really sets the stageand the tech for shared VRand AR experiences one day,which has me all shades of hype.Even though Apple ismaking Shareplay availableto all developers, all apps,there's been nothing fromYouTube about it yet,which is vexing because itwould be legit great with this.Also no Netflix becausethey seem to always want togo it alone and never surrenderany interface to Apple ever.And no Spotify, which fora company that complainsso continuously aboutthe lack of system accessseems to also never take that accesswhen, and as it's offered,which is just so victim-y.You can also use,Shareplay to screen sharebut not just play, butpretty much anythingand everything on your iPhone display.And while you're still onFaceTime, still talking,still seeing each other.It's been availableon the Mac since the elder days of iChatbut an iOS is just one of thebiggest remaining check boxesever, finally getting that box checked.And it's going to saveso many support tripsacross town for so many people. So many.Notifications are like the essential truthof the Vampire Lestat that anyone makingan informed choice willjust always, alwayswant notifications. And anyonedeluged by notificationswill do almost anythingto be rid of them forever.And Apple's trying tohelp with that tension.That problem again, again,by expanding Do Not Disturbinto something more nuancedcalled Focus. With Focus, youcan select a preset like OGDo Not Disturb, Sleep, whichhas been around for a minutefirst in Clock then in Health,but is now newly integratedhere, Personal or Work,or you can roll your own.At that point, it becomeslike a force shield,just bouncing off all theinterruptions all around youexcept for the people andapps you specifically chooseto allow through.And whether to cure ouranxiety or just our FOMO,you can choose whether or not to lettime sensitive notificationsto come throughno matter what as well.And Siri can even announce those for youif you have like a workout focusand are off on a run with your AirPods.So what's considered truly time-sensitive?Apps get to decide that for themselves,they get to opt into the integration.But if they abuse it, you canopt them right back out againand hard.And you can also go in andset different home screensto show or hide based on the Focus.So the apps that are most available to youare the ones most appropriatefor that specific focus.And once you invoke a focuson one of your devices,like your iPhone, it gets invokedacross all of your deviceslike Watch, iPad, Mac,and anyone on the other side will seethat you're in D and D modeand can wait and messageyou until you come out againso they really do not disturb you.Or if it's urgent, they can chooseto punch through yourshield anyway, which againshould cut down on anxiety and FOMOabout missing something really urgent.And if anyone abuses that,it'll just give you the joyof muting that conversationor an excuse to block themfull on if you have to.And I realize for some peopleFocus may sound like a lot.Like I already got 12 jobsand my phone shouldn't beanother one of them a lot,which is why I got a damniPhone to begin with a lot.But Apple provides those sample personaland work focuses and recommendationsand steps you throughthe people in the apps.So it's super easy,not even an inconvenience to get started.And you can always juststick to old school D and Dand no one will ever judge you for itexcept maybe Thomas Frank,because Thomas Frank.And since Apple has givennotifications in general,a fresh coat of icon andprofile pick inspired paint,and will bundle up appnotifications into a summary for youwhile keeping peoplenotifications front and center,it's more glanceable and manageableand even more human now either way.Maps is getting Apple'sversion of Google Earth,at least the big spinny globe part.More interestingly though,a new, more detailedand more stylized cityand driving experienceand a more immersive augmented realitywalking directionsfeature that lets you scanto place yourself for really accuratestep-by-step directions.And I hate, I just hate to be the boywho cried future here again,because it's becoming such a cliche,but this kind of tech is yetanother example of Apple doingwhat needs to be put into place todayin order for us to have amuch more exciting tomorrow.And I just can't waitfor that day to get here.Same with IDs in Wallet. And I know,I know it's going to feellike forever for every state,province, region, to addlicenses, known travelersand other documents to Apple's system,but at least it's starting nowand ID uses the samesecurity as Apple Pay.So you need face ID,touch ID or your passcodeto authenticate your ID.And if your phone was previously lockedand someone takes it whenyou're presenting your ID,they can't unlock it any furtherthan the Wallet interfacewithout your face ID,touch ID, or passcode.And if you get particularly bad vibes,you can always squeeze thepower and volume buttonsto just crash the west wing.And what I mean by that isthrow away the hardware, keys,disable biometrics, andlock everything else downuntil you reenter yourpasscode or password,which will require you to have or regainphysical control of your phone again.And when you fold in home,office, and hotel keys,we're just one stepcloser to living our bestphysical card free life. AndI can't wait for that either.I'm going to cover Safariproperly in the Mac OS previewbecause long video already long,but unique to the iPhone implementationis the new bottom loaded tab bar,which in theory is greatfor one-handed ease of use.So you can just do yourcoffee talk and walkswhile browsing way better,but as currently implementedit just jumps from bottom to top too muchand has way more cognitive loadand interface spelunking than it ought to.And I'm really hoping Applespends more time with itprior to release tojust zero regression itlike every version of Safarihas been mandated to dosince the beginning of timeor Alexander or whatever.It should be the samerules as Apple Watch;every interaction just asfast, WebKit fast as possible.Apple pre-announced thenew accessibility featuresthis year so I've alreadygot a video up on those.Also some of the new privacy featureslike a full-on private relay deep divewhich comes with iCloud Plus,the new name for the extrastorage tiers only now with more featuresthan just extra storage. Soif you have that or Apple One,you have private relayand also hide my email,which lets you use rando signin with Apple style addressesinstead of your own personal one,but also custom domainsfor your iCloud emailthat you can create andshare with your whole family,even home kits secure video storagethat doesn't countagainst your iCloud limit.And that I think is just Appletrying to increase the valueof the Plus services bundlesthat they keep offering.Also because for most regularhuman type, non-info sec,high value target type people,having our photos stolenis far less of a concernthan losing access tothose photos forever.So Apple is adding bothaccount recovery contacts anddigital legacy contacts.It works by you picking oneor more people you trustto help you get back into your accountif you're ever locked out.They get a code andthey can give it to youand then you can use thatto regain your accessand you can choose the samepeople or different people,friends or family, toget access to your dataon the event of your death.And Apple is by no meansthe first to do this,but it's really, reallyimportant that they're doing itbecause even though you're gone,that can be the absoluteworst time for themtrying to deal with allthis stuff you left behind.With Find My, you can nowsee family and friendsin real time so it's less frustratingwhen you're trying to trackeach other down at Disneyand you can also getseparation alerts for devices,including AirTags. Soyou leave no bag behind.Plus you can locate devicesthat have been powered downfor up to 24 hours, evenones that have been erased,but are still activation locked.And they'll say their activationlocked right on the screenand trackable in case somebrainiac thinks they can tryand sell them for a quick buck.Either way, any way,always remember thatyou are not the Batman,so never not ever putyourself in a positionwhere you might lose your lifejust trying to get back your stuff.Apple's on-device intelligenceteam has just beenHulk-smashing it over the last few yearsand this year inparticular with Live Text,it's not something Iexpected or anticipatedbut it's become one ofmy favorite new features.And yes, sure Google has hadLenses since at least 1812,but I like having Live Text and Translatejust system-wide on iOSand I all caps love the waythat Apple is implementing itbecause just like the name on the tin saysit's really truly live.And that means it's notscanning and indexing textsin the background. It's notround tripping to any servers.The machine learning modelsand apple neural engine coresare just tearing through itall on-device and in real timeand it is something. Opena photo with text on it,whether it's typed orhandwritten, it'll make it live.Open the camera, don't eventake a photo, same thing.Go to a webpage with a pictureon it. You get the idea.Live Text has a particularset of machine-learned skills.And if you come across an image,it will look for the text.It will find it and it will kill it.Making that text selectable, copyable,even actionable for you.Like sealing, click it.See a number, call it.I sometimes can't eventell anymore what's Safariand what's a screenshotthat I've stuck in photos.It is completely trippy.The on-device computer visionsystem can also identifyobjects in photos now as welland look them up usingSiri's knowledge system.It's like the first primitivedrag from the primordial AIand AR OOS stage of afull-on Jarvis HUD system.And yes, I realized thisis a recurring themein this preview but anerd cannon will dream.All of this is now alsointegrated into spotlight searchas it should be. Andyou can access that now,right from the lock screen ifthat's how you want to roll.Even Siri, even Siri isliving its best life nowwith improved sequentialinference to better keep contextduring conversation andhold on to your bits.On-device speech recognitionand offline query processing.And that means Siridoesn't have to round tripto Apple servers for the basics anymorewhich not only removesa particularly annoyingprevious point of failure,but makes the whole systemfeel so much more responsive.Like when the Mac moved fromIntel to M1 more responsive.Apple's also adding afeature that I've spent manya WWC lobbying for inperson ever since they addedRemember This a few years back.Basically, whatever youcould use NS activityto remember for you previously,you can now use to sharephotos, webpages, just hey, share this,name your target and it's shared.And I know that might soundlike just a minor convenience,but it's one of thosefirst few critical stepsthat Apple has to take,nevermind to make Siri dominatelike Kramer in karate,but just to make Siri competitive againin the assistance space.And to help along the wayto get out and push so tospeak brilliant.org/ReneRitchieis the best place to start.It's this awesome websiteand app that'll teach you thefundamentals of algorithmsand neural networks.Everything from characterrecognition like live text tosearch like spotlight,but also math, scienceand computer science,logic and deduction,physics, quantum mechanics,game theory, even cryptocurrency and morebecause it's built on this whole processof learning while doingand solving real challengesin real time with nomemorizing long, messy formulasor fact sheets, no tests or grades.Just instant feedback thatcoaches you bit by bitso you can rapidly improve and learnthe foundational concepts behindall the most important newcareers and technologiesliterally before you even realize it.So if you want to go from just using iOSto maybe one day working on it,you can get your startwith Brilliant today.Just go to brilliant.org/ReneRichieor click the link in the description,pick a course and get startednow. brilliant.org/ReneRitchieand clicking on that linkreally helps out this channel.Hit the playlist or YouTube'srecommendation abovefor videos on all the other public betas.More features I didn't have room to coverin this video and a bunchof deep dives to come.Just hit that playlist andI'll see you in the next video.\n"