Recently, Apple’s research team has developed a groundbreaking technology that can determine 3D depth from a 2D image in just a few seconds. This useful technology has potential applications across various industries.
On October 4, VentureBeat reported that Apple published a paper on the Depth Pro, a monocular depth estimation model that infers depth from a single image, on the arXiv platform.
Monocular depth estimation has long been considered a challenging task, as accurate depth measurement typically requires multiple images or metadata like focal length. However, Depth Pro can generate a 'depth map' that estimates both relative and absolute depth from natural state images without any metadata. A depth map is an image that displays the relative distances of 2D image pixels, with closer distances represented by bright pixels and further distances by darker pixels.
This indicates that Depth Pro can provide actual measurements of the real world. This technology is essential for applications like augmented reality (AR) and autonomous driving, where accurately placing virtual objects within physical spaces is crucial.
Moreover, Depth Pro can make precise predictions without training on specific domain datasets. It can generate measured depth maps in a zero-shot learning environment, accurately reproducing the shapes of objects, scene layouts, and absolute sizes. This capability reduces the time and cost required for model training.
The model can generate high-resolution depth maps at 2.25 million pixels in just 0.3 seconds using a standard GPU. It can even detect fine details, such as hair or plants, which are often difficult to capture using other methods.
The research team explained, "Through an efficient multi-scale vision transformer for dense predictions, we can simultaneously handle the overall context of the image and fine details."
Depth Estimation Model Benchmark Tests
In benchmark tests comparing depth estimation models, Depth Pro ranked highest with an average score of 2.5. It outperformed models like Depth Anything v2 and Metric3D in terms of accuracy.
The researchers believe this versatility can be applied across various industries. For instance, when consumers point their smartphone cameras at a room, they can preview how furniture will fit in their home. In the automotive sector, real-time high-resolution depth maps generated from a single camera can help autonomous vehicles better recognize their surrounding environments.
Notably, Depth Pro has addressed one of the most challenging issues in depth estimation: the 'flying pixels' problem. These pixels appear to be floating due to depth mapping errors.
Additionally, it claims to be several times better than other systems in 'boundary tracking,' which sharply distinguishes objects from their edges. This capability is a critical element for applications requiring precise object segmentation, such as image matting that combines foreground and background images or medical imaging.
Currently, Apple has made the model weights and code for Depth Pro available as open source on GitHub. Users can also experience Depth Pro through a live demo provided on Hugging Face.
3 Line Summary for You
Apple has developed Depth Pro, a technology that determines 3D depth from 2D images quickly. This advancement has potential applications in augmented reality and autonomous driving. The model is now open source and can be experienced through a live demo.
Subscribe!! Likes, comments, and clicking on ads really help me a lot.
Starting Google Play App Distribution! "Tester Share" for Recruiting 20 Testers for a Closed Test.
Tester Share [테스터쉐어] - Google Play 앱
Tester Share로 Google Play 앱 등록을 단순화하세요.
play.google.com
