Apple's iPhone is gearing up to join the AI-infused smartphone trend that has been dominating the Android market. A recent research paper by Cornell University researchers collaborating with Apple sheds light on the tech giant's plans to enhance its AI capabilities.
The paper titled 'Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs' introduces a multimodal large language model designed to comprehend mobile user interfaces, such as those found on iPhones. This model can identify icons, text, widgets, and describe on-screen elements, offering a more interactive user experience.
The upgraded Ferret-UI, which builds upon the existing Ferret model released in October 2023, is expected to bring several benefits to iPhone users. It will enable enhanced accessibility features, streamline complex tasks, and improve user interactions with the device.
Furthermore, developers can leverage Ferret-UI as a testing tool to simulate various user scenarios, enhancing app functionality and user experience.
Recent discoveries in backend code hint at new server-side tools for individual iPhones, including 'Safari Browsing Assistant' and 'Encrypted Visual Search.' While these features align with the AI advancements discussed in the research paper, their implementation and usage remain speculative at this stage.
Apple's upcoming Worldwide Developer Conference in June is anticipated to provide more insights into the company's AI strategy as it prepares for the launch of the iPhone 16 series later this year.
As Apple continues to innovate in the AI space, the tech industry eagerly awaits further developments that could reshape the smartphone landscape.
.jpg?w=600)
