صراعات الذكاء الاصطناعي في التعرف على حجم المعالم

نماذج لغة الرؤية تفهم المعالم، لكنها لا تزال غير قادرة على رؤية الصورة الكاملة…

من بين مهارات البقاء الأولى التي نتطورها هي القدرة على التمييز بين الأشياء الصغيرة أو البعيدة. يمكننا مسح القمر بالعقل دون أن نعتقد أنه بحجم العشر سنتات، لأننا قدinternalized فهمًا للقياس النسبي.

هذه مهارة غير عادية الصعوبة لأنظمة الرؤية الحاسوبية، لأن معظمها يعتمد على التعليقات السابقة، والتي لا تساعد على “فهم” القياس بنفس الطريقة التي يفعلها البشر. بالإضافة إلى ذلك، ما وراء حد معين وواضح، كل شيء في المسافة يقع خارج قدرة الرؤية العمياء على حلها – السيارة في الطرف الآخر من الفناء؛ الناطحة السحاب في المسافة وراء ذلك؛ والقمر الهلال الصاعد فوقها… كلها كائنات “ثنائية الأبعاد”، بالنسبة لمعظم أنظمة التعلم الآلي القائمة على الرؤية.

بالطبع، عندما يظهر مثال معين لشيء “بعيد” لكن تم تفسيره بشكل خاطئ في بيانات التدريب، يمكن لأنظمة التي شهدت هذه البيانات أن تكون صعبة الخداع:

ChatGPT-5.5 ليس منبهر بهذا المثال السياحي الكلاسيكي.

كلما كان الفضاء اللاتيني للمodel أقل احتواءً على معلومات محددة ومتكررة، زادت حاجته إلى التعميم وinternalization مفاهيم القياس التي ندركها في سن صغير. بدون ذلك، حتى الأمثلة الشهيرة يمكن أن تسبب سوء تقدير للقياس:

في هذا المثال التخيلي، الذي تم تعديله من الورقة الجديدة التي نناقشها اليوم، يظهر POV للكاميرا قوس النصر في الخلفية – لكن النظام لا يعرف حجمه، ويفعل تخمينًا خاطئًا. مصدر

الخطر، مع كائنات محددة وذات سمات مميزة مثل برج إيفل، هو أن النظام سوف يلجأ إلى طريقة مختصرة لتخمين الحجم التي تكون صحيحة للنموذج الأصلي، ولكنها ليست صحيحة للعديد من التماثيل للمعلم الباريسي التي هي على مسافة بعيدة من الرؤية العمياء، ولكنها ليست كبيرة جدًا.

لذلك من المهم أن تتقدم أنظمة الرؤية بموهبة جاهزة، وليس مجرد مجموعة من “كودات الخداع”.

توسيع النطاق

بهذا المعنى، تقدم التعاون الجديد بين الولايات المتحدة والصين قاعدة بيانات علاجية، جنبًا إلى جانب طريقة تقدير، ت解决 هذه القضية:

المنهج الجديد يعدل نظامًا سابقة من خلال مواد تدريب محسنة – بيانات متنوعة بدرجة كافية لتوفير فهم أعمق لقضايا العمق.

تم إطلاقها معًا مع موقع ويب مرافق، وتتميز مبادرة MetricScenes بـ بيانات و إصدارات كود.

يصرح الورقة*:

‘[وجدنا أن الأساليب الحالية في الوقت الراهن تفشل بشكل متكرر في تقدير مقياس المشهد الصحيح، مما يؤدي إلى ظاهرة انهيار المقياس المستمرة في السيناريوهات “في البرية”.’

‘[الصور أعلاه] تظهر مثالًا حيث توجد مراجع دلالية واضحة (أشخاص) ، ومع ذلك ، فإن النماذج مثل MoGe-2 تظهر عدم مطابقة مقياس كبيرة عبر مدى المسافات: مقياس المتري المتوقع للكائنات القريبة هو معقول – في هذه الحالة ، يبلغ ارتفاع السياح ارتفاعًا معقولًا – ومع ذلك ، فإن مقياس الكائنات البعيدة يتم تحديده بشكل كبير – هنا ، يتم تقدير قوس النصر في الخلفية بمقياس متري يبلغ 18.8 مترًا فقط ، وهو أكثر من ضعف الحجم الصغير من العرض الحقيقي (44.8 مترًا).

‘MoGe-2 قد اقترح معلمًا مصغرًا، على الرغم من الإشارات المتضاربة.’

قوة الثلاثة

تم تجميع مجموعة المؤلفين الجديدة عن طريق الجمع بين ثلاث مجموعات بيانات موجودة: MegaScenes و AerialMegaDepth و Stereo4D:

صور مثال من MegaScenes، التي تشكل جزءًا من التجميع الجديد. مصدر

المشكلة في مجموعات البيانات التي تساهم في MetricScenes، عند أخذها بمفردها، هي أن كل منها ينطبق على مجالات محدودة، مثل لقطات POV للسيارات أو المشاهد الداخلية، عندما يكون مجالًا مشتركًا مطلوبًا ل解决 المشكلة وbringing أنظمة الرؤية أقرب إلى فهم بشري لمفهوم القياس.

كل صورة مصحوبة بصور RGB، وعمق جزئي مُشتق من بناء الهيكل من الحركة (SfM) و استريو متعدد المناظير (MVS) أو معايير هندسية أخرى، جنبًا إلى جنب مع خريطة عمق مكتملة تم إنشاؤها من خلال عملية استكمال بويسون ثنائية المراحل، وبيانات كاميرا مرتبطة.

تم تحسين إطار عمل MoGe-2 على مجموعة البيانات الجديدة ‘يخفف بشكل كبير‘ من انهيار المقياس الذي يشيرون إليه، ويُreported تحقيق نتائج متفوقة في المشاهد المفتوحة، وأداء على مستوى الدولة في المعايير المرتبطة:

The new paper is titled Honey, I Shrunk the Arc de Triomphe!, and comes from four researchers across Cornell University and Shanghai Jiao Tong University.

المنهج

MetricScenes draws in part on the aforementioned AerialMegaDepth and MegaScenes – two collections of Internet photographs spanning historical archives, tourist images, and professional photography. Though MegaScenes offers large-scale Structure from Motion (SfM) reconstructions, these scenes lack any inherent real-world scale. To address this, geotagged imagery from online mapping services was used to align the reconstructions with known physical locations and dimensions.

Conversely, AerialMegaDepth already incorporates geotagged Google Earth views, providing metric-scale landmark reconstructions.

Potential reconstruction errors caused by visually similar but geographically-distant structures were addressed using MASt3R-SfM and the Doppelgangers++ classifier. After Multi-View Stereo (MVS) reconstruction, unstable depth estimates and depth-bleeding artifacts were filtered using a combination of stability checks and predictions from MoGe-2:

AerialMegaDepth derives real-world scale by combining Internet photographs with geotagged Google Earth views, while MegaScenes scenes are aligned to physical dimensions using geo-referenced street-level imagery. After Multi-View Stereo (MVS) reconstruction, unstable depth estimates and depth-bleeding artifacts are filtered out, producing cleaner metric-scale depth maps suitable for training. Yellow boxes highlight transient objects removed during processing, while red boxes indicate corrected depth-bleeding regions.

Metric scale was then recovered through geo-referenced imagery. AerialMegaDepth already derives scale from Google Earth renderings captured from known locations, while MegaScenes was aligned to real-world dimensions using geotagged street-level imagery obtained from mapping services.

These images were matched to existing reconstructions with MASt3R, refined with the Doppelganger classifier, aligned with COLMAP, and scaled through RANSAC-based estimation using Earth-Centered, Earth-Fixed (ECEF) coordinates. Scenes with unreliable scale estimates, or poor registration quality, were discarded.

رؤية ستيريو

The MetricScenes collection also draws on the aforementioned Stereo4D dataset, which features thousands of real-world stereoscopic video sequences captured with VR180 cameras, offering a temporal dimension to the captures:

The Stereo4D dataset was built from stereoscopic Internet videos, combining camera poses, depth estimates, and motion trajectories to recover dynamic 3D scenes at scale. The resulting dataset contains hundreds of thousands of video clips represented as point clouds with long-range motion tracks, providing a large source of real-world 3D geometry and motion for training vision models. Source

Because the physical distance between the two camera lenses varies across different devices, only videos with documented camera configurations were used, allowing scene depth to be recovered at an accurate real-world scale.

Stereo4D originally relied on the optical-flow system SEA-RAFT to estimate scene geometry, but the authors found that imperfect camera calibration could distort reconstructed scenes, causing structures that should be parallel to converge unnaturally. Therefore, to improve accuracy, they replaced this approach with a multi-view reconstruction pipeline that jointly estimates camera poses and depth from multiple frames.

After comparing π³, DepthAnything V3, and MapAnything, π³ was selected for its geometric robustness and ability to preserve fine details:

Metric-scale depth recovery from Stereo4D. Standard stereo-matching methods can produce distorted geometry when camera calibration is imperfect, while π³ generates more consistent scene reconstructions, and preserves fine detail. The recovered geometry is then aligned to the known physical baseline of the stereo camera, yielding accurately-scaled metric-depth maps.

Because π³ reconstructs scenes at an arbitrary scale, the resulting depth maps were aligned to real-world dimensions using the known physical baseline of each stereo camera rig. Additional filtering removed low-quality frames, depth inconsistencies, calibration errors, and unreliable scale estimates.

Additionally, a two-stage depth-completion process was used, combining foreground predictions from MoGe-2 with background geometry from Multi-View Stereo (MVS), producing cleaner metric-scale training data with more consistent scale and sharper object boundaries:

Two-stage depth completion. Using only background anchors can preserve scene structure while distorting overall scale, whereas combining foreground and background constraints in a single pass introduces scale drift and boundary artifacts. The two-stage approach maintains consistent metric scale across both near and distant objects while preserving clean object boundaries.

The authors observed that Internet photo collections often lack reliable foreground depth, while stereo imagery frequently misses distant background regions. Though MoGe-2 can infer dense geometry across an entire scene, its estimates tend towards the same scale-collapse problem that the project seeks to address. Therefore the two-stage depth-completion pipeline was designed to combine the strengths of MoGe-2 and Multi-View Stereo (MVS).

Background geometry was recovered using MVS-derived metric anchors, creating a base depth map with reliable large-scale structure. In a second stage, foreground estimates from MoGe-2 were reintroduced through an edge-aware completion process designed to preserve object boundaries while preventing scale drift and depth-bleeding artifacts.

The depth maps produced by this approach, the paper contends, were both visually complete and more consistent in real-world scale:

Two-stage depth completion pipeline. In the first stage, Multi-View Stereo (MVS) anchors are used to recover background geometry at a reliable metric scale. In the second stage, foreground estimates from MoGe-2 are reintroduced through an edge-aware composition process, producing a final depth map designed to preserve both large-scale accuracy and sharp local detail.

بيانات واختبارات

The final MetricScenes collection comprises 47,579 exclusively real-world images covering 134 scenes from AerialMegaDepth; 29,583 images from 356 scenes from MegaScenes; and 22,549 frames taken from 1,725 videos from Stereo4D.

The collection, from which 10 scenes per source were held back as validation, covers exterior and interior contexts, as well as ground-level and aerial views, and urban as well as natural landscapes – a collated and cohesive context not available in any of the individual contributing collections.

For an initial qualitative test, the authors fine-tuned the MoGe-2 ViT-Large-Normal model on the new MetricScenes dataset for 10,000 iterations at a batch size of 32 – effectively around three epochs. Cropping and general data augmentation approaches were taken from the original MoGe-2 tests, and training occurred at a learning rate of 1×10^-6 (backbone) and 1×10^-5 (all other parameters). For the qualitative test, depth reconstructions were undertaken by the fine-tuned WildMoGe model, pitted against base MoGe-2; DepthAnything V3; Metric3Dv2; UniDepth v2 ; and DepthPro:

Comparison of metric-scale landmark reconstruction. Ground-truth measurements from Google Maps are shown in the left column. Across unseen real-world landmarks, WildMoGe produces scale estimates more closely matching known dimensions, while MoGe-2, DepthAnything V3, and Metric3D V2 frequently underestimate the size of distant structures. UniDepth V2 often yields more plausible scales, but remains inconsistent, whereas DepthPro occasionally produces severe scale errors.

Of this result, the paper states:

‘[WildMoGe]  consistently recovers more accurate absolute scales across diverse landmarks, closely matching ground-truth dimensions (e.g., 31.4m vs. 32.4m for the Philadelphia Museum of Art, 46.7m vs 46.5m for Piazza della Signorina). MoGe-2, DepthAnything v3 and Metric3D v25 exhibit scale-collapse behavior, consistently underestimating the size of far-field structures.

‘UniDepth v2 produces more realistic scales but still deviates from ground truth, and DepthPro often fails to recover absolute scale, producing results that are orders of magnitude smaller than reality. Note that these scenes are absent from the training set.

‘This performance demonstrates that WildMoGe can generalize to unseen content, as opposed to simply memorizing training scenes.’

To ensure the gains found were not limited to landmarks and large outdoor scenes, the authors also evaluated WildMoGe on ordinary indoor and street-level images, where it produced scale estimates broadly consistent with MoGe-2, while achieving better accuracy on an ETH3D  courtyard scene:

Comparison on standard scenes. Across ordinary indoor and street-level environments, WildMoGe produces scale estimates broadly consistent with MoGe-2, while achieving greater accuracy on the ETH3D courtyard benchmark, recovering object dimensions that more closely match ground-truth measurements.

To assess whether MetricScenes actually improved metric-scale reasoning, evaluation was performed both on a dedicated MetricScenes test set and on NYUv2; KITTI; ETH3D; iBims-1; GSO; Sintel; DDAD; DIODE; Spring; and HAMMER.

The authors note that obtaining dense ground-truth measurements for unconstrained Internet imagery remains difficult, meaning the MetricScenes labels are not perfect. Standard benchmarks were therefore included to verify that any gains did not come at the expense of general geometric performance.

Comparisons were made against MoGe-2; UniDepth V2; DepthPro; MASt3R; Depth Anything V2; Depth Anything V3; ZoeDepth; and Metric3D V2:

Quantitative evaluation of relative and metric geometry. On the MetricScenes test set, WildMoGe outperformed MoGe-2 across every reported metric while remaining broadly competitive with ZoeDepth, Metric3D V2, Depth Anything V2, Depth Anything V3, MASt3R, UniDepth V2, and DepthPro on established benchmarks, indicating that improved metric-scale estimation was achieved without sacrificing general geometric reconstruction quality.

WildMoGe substantially improved metric-scale prediction on MetricScenes, outperforming MoGe-2 across every reported metric and achieving stronger metric-geometry and metric-depth scores than MoGe-2, Depth Anything V3, Metric3D V2, UniDepth V2, and DepthPro.

Performance on NYUv2, KITTI, ETH3D, iBims-1, GSO, Sintel, DDAD, DIODE, Spring, and HAMMER remained broadly comparable to MoGe-2. The authors attribute these gains to MetricScenes’ metric supervision, which apparently helps to reduce scale collapse while preserving general scene reconstruction performance.

الختام

The MetricScenes solution to ‘scale collapse’ comes across as somewhat of a Heath-Robinson affair, in the paper – a hail-Mary melding and distillation of multiple datasets, each of which has some valuable viewpoint to contribute. It seems a little like trying to determine the shape of an elephant by touch.

Perhaps the most valuable service the paper offers is in calling greater attention to the issue, which seems to require some kind of novel or adapted universal standard. However, since such an innovation would interrupt the reproducibility and consistency of current methodologies, it would have to be very convincing.

* My conversion of the authors’ inline citations to hyperlinks.

First published Thursday 11th June, 2026