The new feature would reportedly take the form of computer vision. Apple’s work-in-progress camera app will be able to identify objects in-frame and recognize faces, according to the report. And it will be made available to developers in the form of a software development kit.
The most recent news we’ve heard about Apple’s augmented reality plans suggest that Apple may have as many as 1,000 engineers working on an AR-related product in Israel — a product that will make its way to the next iPhone, at least according to analyst Steven Milunovich and his team at UBS. The report, from Business Insider, also highlights that Apple has recently made a number of AR-related purchases, including PrimeSense, a 3D sensing company in Tel Aviv, and RealFace, a facial recognition company also based in Tel Aviv.
The Cupertino, California-based company is far from the first to enter the AR arena. Google Goggles, Google’s eponymous object recognition app, recognizes landmarks, barcodes, books, and works of art, and parses text of labels and signage using optical character recognition. And Amazon’s Flow app can decode QR codes, scan business cards, and recognize tens of millions of books, DVDs, and packaged products.
Apple’s system sounds nearly as ambitious: an app that can identify objects that users point the iPhone’s camera at in real time. It will rely on machine learning, a type of artificial intelligence that “learns” and improves over time, and a database of 3D objects that Apple will either license or build itself.
Beyond those basics, the project’s implications aren’t clear. Google’s Project Tango, an augmented reality platform, leverages sensors to measure the depth of surroundings, but the iPhone lacks the hardware necessary to perform that sort of tracking. Apple Insider speculates that Apple’s brand of machine-learning-powered object tracking could be used for spatial mapping, and that facial recognition, meanwhile, could be used to apply Snapchat-style filters to people’s faces.
Spearheading the project is a team comprised of employees from recent Apple acquisitions. The iPhone maker purchased PrimeSense, the Israeli company behind the motion-tracking hardware in Microsoft’s Kinect sensor, in 2013. It bought Metaio in February 2014, FaceShift in 2015, and Flyby Media in January 2016, all of which specialize in virtual reality and AR technologies. And it hired a senior optics manufacturing engineer specializing in heads-up displays, camera systems, and image sensors in September.
The project builds on another reportedly in development at Apple’s Cupertino headquarters: AR glasses. According to Business Insider, Apple is developing a compact pair of AR eyewear that connect wirelessly with an iPhone and show images and other information in the wearer’s field of vision.
If anything is for certain, it’s that an updated camera app will debut far ahead of a headset. Bloomberg reports that Apple has begun talking with suppliers about the glasses and has ordered “small quantities” of displays for testing. The publication pegs 2018 as the product’s earliest possible release window.
Until then, Snapchat’s Spectacles will have to do.
Updated on 03-02-2017 by Christian de Looper: Added new Business Insider report saying that Apple had as many as 1,000 engineers working on AR.
