Hi there!

A real-time capable pixel-aligned Gaussian splatting-based method for realistic novel view synthesis of humans from sparse input images, enhancing flexibility by reducing reliance on priors and preprocessing.

• Proposed a simpler and more flexible method than the existing methods:

  Method	Priors	Preprocessing	Limitation	#Params
  GHG (ECCV24)	SMPL-X Template	SMPL-X Template extraction, Texture-map rearrangement, Scaffold creation	Fixed angle between cameras, Relies on good SMPL-X estimation	17.8M
  GPS-Gaussian (CVPR24)	Binocular-View	Image Rectification	Max. 72-degree baseline	5.5M
  Ours	Monocular-View	None of the above	None of the above	3.5M

  • Priors:
    GHG (ECCV24): SMPL-X Template,
    GPS-Gaussian (CVPR24): Binocular-View,
    Ours: Monocular-View
  • Preprocessing:
    GHG (ECCV24): SMPL-X Template extraction, Texture-map rearrangement, Scaffold creation,
    GPS-Gaussian (CVPR24): Image Rectification,
    Ours: None of the above
  • Limitation:
    GHG (ECCV24): Fixed angle between cameras, Relies on good SMPL-X estimation,
    GPS-Gaussian (CVPR24): Max. 72-degree baseline,
    Ours: None of the above
  • Number of Parameters:
    GHG (ECCV24): 17.8M,
    GPS-Gaussian (CVPR24): 5.5M,
    Ours: 3.5M
• Developed a reliable pixel-aligned Gaussian Splatting method for generalizable 360-degree novel view synthesis of human heads from minimal input data and demonstrated capacity on the NPHM dataset.
• Demonstrated in-domain and cross-domain generalization capability on sparse-view full-body human datasets (THuman 2.0, CustomHumans).
• Conducted experimental analysis of various extensions to the proposed method, including:
  • In-Domain and Cross-Domain Generalization
  • Sub-Resolution Rendering
  • Combining Multi-View Information
  • Refinement with a Neural Network
  • Refinement with non-rigid 3DGS optimization
  • Pruning of 3D Gaussians
  • Camera Pose Encoding with Plücker Coordinates
  • Using Template Priors As Supervision (SMPL-X)
  • Transfer Learning with a Frozen Human-Centric Encoder (Sapiens)
  • LPIPS as a Loss Function

Pytorch Slurm OpenCV Open3D NeRFStudio Neural Rendering 3D Reconstruction Gausian Splatting Transfer Learning

Focused on improving semantic segmentation for autonomous driving in challenging weather conditions, leveraging self-training, semi-supervised learning, and data augmentation from generative models like GANs and diffusion models.

• Article: W-ControlUDA: Weather-Controllable Diffusion-assisted Unsupervised Domain Adaptation for Semantic Segmentation
• Journal: IEEE Robotics and Automation Letters
• Publication Date: 24 February 2025
• Volume: 10, Issue: 5
• On Page(s): 4204-4211
• Print ISSN: 2377-3766
• Online ISSN: 2377-3766
• DOI: 10.1109/LRA.2025.3544925

Semantic Segmentation Distillation Autonomous Driving Generative Models GAN Diffusion Pytorch Lightning MMSegmentation MMCV MMEngine

Introduced transformers to different components of a 3D Dense Captioning pipeline to boost performance.

• As team, we proposed two architectures based on transformers to achieve state of the art performance in 3D Dense Captioning Task.
• I mainly focused on 3DETR-S2C, which achieved SOTA results on 3 of the 5 benchmarking criteria.
• Briefly, I integrated a transformer backbone to extract features and detect 3D bounding boxes from 3D point clouds, which are then processed further with a Graph Neural Network (GNN) based relational module and a recurrent neural network based NLP module to describe relations between 3D objects of an indoor scene.
• Furthermore, I have done end-to-end training and optimization of the neural network on remote Google Cloud Platform servers.
• I also supported my teammate on implementing S2C-MMT, which achieved SOTA results on 2 of the 5 quantitative criteria. Built upon a PointNet++ backbone, S2C-MMT partially or fully replaces the relational module and NLP module with the Meshed Memory Transformer (MMT).

Pytorch 3D NLP GNN RNN GCP Transformers

Trained an agent to navigate in dynamic environments using reinforcement learning and curriculum learning.

• Identified and proposed a promising robotics research project concerning social navigation - autonomous navigation of a robot in a dynamic environment filled with pedestrians.
• Extended and improved the world simulation by adding static obstacles and limited field of view constraints.
• As a team, developed a new RL agent with an extended version of the baseline.
• Intertwined training with world simulation extension via curriculum learning by optimizing the RL agent training with gradually increasing simulation difficulty.

Pytorch OpenAI Gym Reinforcement Learning Robotics GCP

A supervised monocular depth prediction model leveraging polarimetric characteristics of light. During the model development, particular emphasis was put on improvement of the depth estimation for photometrically challenging objects.

• As a team of four, we proposed a monocular depth estimation model turning a video stream from a single polarimetric camera into dense depth prediction.
• We leveraged properties of polarized light to improve depth estimation reliability on photometrically challenging non-Lambertian materials.
• We demonstrated qualitative and quantitative improvements on transparent objects such as glass and on shiny surfaces such as metal.
• I focused on optimizing the neural network architecture by introducing modality specific encoders, channel-wise and spatial attention modules, and cross-component skip connections to push performance higher.

Python Numpy Kornia CNN's Attention U-Net

Implemented localization, sensor fusion, navigation, control, and path planning of an autonomous drone.

• In a team of two, programmed AR Drone 2 to autonomously carry a packet between random start and end points within an environment with known 3D occupancy grid.
• Implemented manual steering of the drone using keyboard input with SDL2 library.
• Implemented and tuned a PID controller for feedback control from visual intertial state estimation signals.
• Implemented collision-avoidance capability, enabling safe navigation between user selected checkpoints when a log-odds occupancy map is given.
• Extended occlusion-aware navigation with A* trajectory search, yielding a reliably motion plan between start and end points.
• Further programmed and streamlined the autonomous packet delivery task and successfully completed the navigation challenge.

C++ CMake ROS Gazebo OpenCV SDL2

Programming two Franka Panda robot arms to achieve collaboration in CoppeliaSim (V-REP), while avoiding self-collision and obstacles.

• As a team of 5, we took the robothon challenge of the Fundamentals of Human Centered Robotics course.
• Carried the team by taking the lead on 2/3 of the tasks.
• Programmed the robot for collision-checking from specific joint configurations.
• Programmed end-effector trajectory planning and shared object manipulation.

Go Lua V-REP Physics Simulation

Main goal here was to test the capabilities of Coding IDE's and CLI's. Used Windsurf and Claude Code for this project. Vibe coded a multi-modal data collection pipeline.

• Created a robust Python package using MediaPipe for detecting and analyzing nodding behavior through 3D face and pose landmark tracking.
• Implemented advanced visualization with Rerun, enabling real-time tracking of head pitch angles and nod detection.
• Developed a modular, extensible system with comprehensive command-line interface and JSON export functionality.
• Ensured high code quality through pre-commit hooks, unit testing, and rigorous static type checking.

As a hobby project, 3D printed, built, calibrated SO-100 robot arms. Tweaked LeRobot source code for dual follower teleoperation.

• Experienced the entire learning journey from the beginning by 3D printing the robot components rather than ordering a kit.
• The two follower and one leader SO-100 robot arms are still fully operational.
• Looking forward to connect with hobbyists and enthusiasts to share knowledge and collaborate!