Projects

Robots that understand their surroundings, measure what is hidden, and interact with the physical world.

01 · RGB-D perception and geometry

Lunar Rock Reconstruction

This pipeline turns registered RGB and inverse depth into one metric mesh for each visible rock. YOLO26x-seg supplies instance masks; each mask is eroded by three pixels and back-projected through the camera intrinsics into the shared Unreal world frame.

After ShadowCorr associates observations of the same rock, the merged cloud is downsampled and filtered. Hidden geometry is completed with a 2× covariance ellipsoid centered at the lower third along its local vertical axis, trimmed by the estimated ground boundary, and converted to a validated alpha-shape mesh.

Detect. Four-channel YOLO26x-seg predicts one mask per visible rock.

Project. Registered depth turns mask pixels into metric surface points.

Complete. Local-axis ellipsoid filling supplies the unobserved lower geometry.

Measure. Closed meshes yield volume and uniform-density geometric centroid.

Paper DOI ↗ GitHub · Source code ↗

16.13%

Median volume MAPE

38.70 mm

Median centroid error

4.01%

Centroid error / true longest dimension

1,940

Matched four-camera rocks

Four registered views · drag to orbit

Lower-third ellipsoid completion · 2× scale

02 · Cross-view correspondence

ShadowCorr

A rock that is partly occluded can appear as several disconnected segments in different camera views. ShadowCorr projects each segment behind its observed surface into a shared voxel grid, learns embeddings from the resulting volumetric consensus, and assigns the original segments to physical rocks.

The Mask75 evaluation assigns every method over the same original segment support and reports end-to-end runtime for a consistent comparison.

Voxelize. Partial segments cast confidence-weighted evidence into a common 3D grid.

Encode. A sparse network separates different rocks while keeping same-rock voxels compact.

Read out. Voxel components vote assignments back onto the original observation segments.

GitHub · V2 ↗ Dataset DOI ↗

Partial segments → shared voxel evidence → rock identity

96.09%

Segment ARI

99.65%

Point-weighted ARI

98.84%

Average purity

183.8 ms

Mean runtime / scene

Method	Segment ARI	Point-wtd. ARI	Average purity	Runtime
ShadowCorr	96.09%	99.65%	98.84%	183.8 ms
PTv3-PointVote	89.33%	95.72%	96.05%	189.7 ms
Open3DIS-MatchedMasks	71.20%	89.32%	89.77%	197.8 ms

03 · Vibration sensing

Burial-Depth Estimation

Vibration reveals buried geometry beyond the exposed surface. This first-of-its-kind method excites a partially embedded rock with an instrumented hammer and measures its response with a scanning laser Doppler vibrometer.

Measurements from two orthogonal strike directions become eleven interpretable geometry and vibration features: exposed height, cross-section width, resonance frequency, peak mobility, damping ratio, and spatial vibration-decay slope encoded as directional means and differences. A compact multilayer perceptron predicts absolute burial depth without measured rock mass, assumed density, or soil parameters.

Excite. Apply repeatable impulses from two orthogonal directions.

Measure. Record multi-point frequency-response functions without attaching a sensor to the rock.

Predict. Evaluate the eleven-feature model with nested leave-one-rock-out selection.

GitHub · Code + data ↗ Paper accepted

Scanning laser Doppler vibrometry burial-depth experiment

25.1%

Nested-LORO MAPE

0.94 cm

Root mean squared error

102

Paired samples

Independent test blocks

Predicted burial depth compared with true burial depth

04 · Robotic additive manufacturing

In-Field Train-Track Repair

Developed with the FASER Lab and MELD Manufacturing, this full-scale system combines an Additive Friction Stir Deposition print head, a six-degree-of-freedom Stewart platform, and a servo-driven rail mover to repair worn rail in place.

My work covered the 8-ft rail-feeding mechanism and support frame, structural and finite-element analysis under process loads, workspace and reachability analysis for the hexapod, and coordinated control software for the feeder, platform, and print head.

Feed. Translate and clamp full-length rail beneath the manufacturing head.

Position. Use the Stewart platform to orient the spindle across the rail profile.

Validate. Check structural deflection, stability, reachability, and load capacity before printing.

Coordinate. Synchronize rail motion, hexapod pose, and deposition through the control stack.

Full-scale MELD print head, Stewart platform, and rail mover

12,000 lb

Vertical process load

3,000 lb

Lateral process load

8 ft

Servo-driven rail travel

6 DOF

Stewart-platform motion

RGB-D perception · Synthetic lunar scenes

Lunar Rock Reconstruction

I built an RGB-D pipeline that turns partial views of lunar rocks into estimates of their volume and centroid. It combines rock detection, cross-view association, and geometric completion to give robots a fuller picture of the rocks around them.

01 · Synthetic inputs

The same scene in color and depth

I generated Unreal Engine scenes with NASA-derived rock models and partial burial, producing aligned RGB and metric-depth images for the visual pipeline.

Synthetic lunar rock field rendered in Unreal Engine — RGB

Metric depth of the same lunar rock field — Metric depth

02 · Rock detection

Separate each rock from its surroundings

YOLO26x-seg combines RGB with inverse depth to predict an individual mask for each visible rock.

Ground-truth instance masks, with separate rocks shown in different colors — Ground truth

YOLO26x-seg predicted instance masks for the same scene — YOLO26 prediction

Foreground agreement: correct pixels in green and missed or extra pixels in red — Pixel agreement

99.39% mask mAP50 · Synthetic test set

Correct foregroundMissed / extra pixels

03 · Multi-view reconstruction

Bring the camera views together

Masked depth becomes 3D surface points. Known camera poses align the views, and ShadowCorr associates observations of the same rock into one point cloud.

Four-view point-cloud stitching · Drag to orbit · Expand to zoom

04 · Geometry completion

From a partial surface to a complete mesh

I combine the measured surface with ellipsoid-based support below the estimated ground plane, then form a closed mesh to calculate volume and geometric centroid.

Observed points → Estimated support → Closed mesh

05 · Results

Measuring reconstructed rock geometry

Four-camera results across 100 synthetic scenes, measured on 1,940 matched, valid reconstructions.

16.13%Median absolute volume error

38.70 mmMedian centroid error

4.01%Median centroid error / rock length

Median errors across the evaluated camera configurations.
Camera views	Volume error	Centroid error
Two	19.97%	63.07 mm
Three	18.69%	48.40 mm
Four	16.13%	38.70 mm

Cross-view correspondence

ShadowCorr

I developed ShadowCorr to match fragments of the same rock across camera views, even when the visible surfaces do not overlap. It turns possible hidden volumes into shared 3D evidence, then learns which fragments belong together.

01 · The idea

Different fragments. One rock.

Projecting behind each observed surface reveals where fragments may belong to the same object. Overlap between these candidate volumes provides evidence that surface matching alone can miss.

Partial surfaces → Overlapping volumes → Rock identity

02 · Hidden-volume projection

Give each fragment a possible hidden extent

An ellipsoid fitted to the visible points sets the projection depth. The projected volume follows the segment’s silhouette and tapers near its edges to limit spill into neighboring rocks.

Ellipsoid depth prior

Silhouette-aware projection

03 · Shared evidence

Bring every observation into one 3D grid

A shared voxel grid combines projected volumes across the scene. Confidence, support from different views, and visible empty space help distinguish consistent evidence from accidental overlap.

Scene-wide volumetric evidence · Drag to orbit · Expand to zoom

04 · Learned correspondence

Turn shared evidence into rock identities

My co-occurrence graph encoder captures which fragments support the same voxels. A Point Transformer V3 U-Net learns an embedding that groups voxels by rock, and a deterministic decoder assigns each original fragment an identity.

ShadowCorr network: sparse voxel evidence and co-occurrence encoder feed a PTv3 U-Net, followed by an embedding and margin decoder — Co-occurrence encoder → PTv3 U-Net → Rock identities

05 · Incomplete observations

Keep the identity when visible support shrinks

Mask75 retains 75% of each segment’s visible support. This controlled test creates gaps between observations while preserving the true rock identities.

Mask100 Original observationsMask75 75% visible support

The same scene, with reduced visible support

06 · Visual results

See which fragments stay together

Scene 516 under Mask75. Colors identify the predicted rock groups; the correctness maps reveal fragments assigned incorrectly or split into extra groups.

Ground truth

Reference identities

CorrectIncorrect / extra splitCorrectness maps below

Open3DIS-MatchedMasks

70.43% segment ARI

Open3DIS correctness map, with correct fragments in green and incorrect or extra splits in red — Correctness

PTv3-PointVote

88.63% segment ARI

PTv3 correctness map, with correct fragments in green and incorrect or extra splits in red — Correctness

ShadowCorr

100.00% segment ARI

ShadowCorr correctness map showing all fragments in Scene 516 correctly grouped in green — Correctness

07 · Results

Stronger correspondence across the test scenes

Mask75 results across 200 synthetic test scenes using two camera views, with all methods evaluated on the same original segments.

96.09%Segment ARI

99.65%Point-weighted ARI

98.84%Average purity

ARI measures agreement with the true rock groups; 100% is perfect. Higher is better for all three metrics.
Method	Segment ARI	Point-weighted ARI	Average purity
ShadowCorr	96.09%	99.65%	98.84%
PTv3-PointVote	89.33%	95.72%	96.05%
Open3DIS-MatchedMasks	71.20%	89.32%	89.77%

Vibration sensing · IEEE Sensors Journal (accepted)

Burial-Depth Estimation by Vibration

I developed a method that estimates hidden burial depth from a rock’s vibration response and exposed geometry. The laboratory estimator combines measurements from two strike directions without requiring measured mass, density, or soil parameters as inputs.

01 · The experiment

Measure how a buried rock responds

An impulse hammer excites the specimen, and a scanning laser Doppler vibrometer measures its surface velocity without attaching a sensor. I tested 18 concrete blocks in sand at known burial depths, collecting 102 paired samples from two strike directions.

Laser vibrometer beside a concrete block partly buried in sand — Laser vibration measurement and the buried specimen

18 physical specimens102 paired samples0.99–8.40 cm burial range

02 · Measured features

Combine vibration response with exposed geometry

Frequency, peak mobility, damping, and the change in vibration with scan height describe the response in each direction. Their directional means and differences, together with the exposed dimensions, form 11 inputs to a compact neural network.

Frequency-response features

Scan positions and exposed geometry

03 · Results

Estimate depth on held-out specimens

Nested leave-one-rock-out testing holds out each physical block in turn and selects the model using only the remaining blocks. The headline errors average 20 training seeds.

25.1%Mean absolute percentage error

0.938 cmRoot mean square error

95.1%Predictions within 2 cm

Predicted burial depth plotted against measured depth — Predicted versus measured burial depth

Prediction errors plotted over the measured burial-depth range — Error across the tested depth range

The prediction scatter shows one representative seed. The diagonal marks perfect prediction; dashed lines mark ±25% relative error. Relative error is highest at shallow burial.

04 · The value of vibration

Improve on geometry alone

Adding vibration measurements roughly halves both relative and absolute error compared with exposed geometry alone under the same nested evaluation.

Nested evaluation on held-out blocks. Lower error is better.
Inputs	MAPE	RMSE
Geometry only	50.56%	1.846 cm
Geometry + vibration, without spatial slope β	25.75%	0.990 cm
Full 11-feature model	25.10%	0.938 cm

Publications

01

Y. Ruan and E. Komendera, "Lunar Rock Reconstruction with RGB-D Images," 2025 IEEE SENSORS, pp. 1–4, 2025. DOI ↗

02

Y. Ruan and E. Komendera, "ShadowCorr: Cross-View Correspondence of 3D Segments via Volumetric Consensus," under review at IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026. GitHub ↗

03

Y. Ruan and E. Komendera, "Burial Depth Estimation for Partially Embedded Rocks Using Scanning Laser Doppler Vibrometry and Neural Networks," accepted for publication in IEEE Sensors Journal, 2026. GitHub ↗

Projects

Lunar Rock Reconstruction

ShadowCorr

Burial-Depth Estimation by Vibration

In-Field Train-Track Repair

Assembler Truss Gripper

Lunar Rock Reconstruction

ShadowCorr

Burial-Depth Estimation

In-Field Train-Track Repair

Publications