News

  • Our lab’s first spinoff startup Building Diagnostic Robotics (BDR) was founded in 2023!
  • Our lab is featured on IEEE Spectrum together with other three NYU robotics teams!
  • [11/2024] Congratulations to Yiming for passing his PhD defense! Watch the defense talk.
  • [10/2024] Two papers (3DGM, MSG) accepted by NeurIPS 2024.
  • [10/2024] Xinhao presented 3 papers (SSCBench, Occ4cast, Roofus) from our lab at IROS 2024 Abu Dhabi.
  • [09/2024] Dr. Feng receives 2024 Tandon Faculty Award and now holds the title of Institute Associate Professor.
  • [06/2024] Dr. Feng has received his tenure and promotion approval from NYU. Congratulations!
  • [04/2024] Congratulations to our junior members for being accepted to PhD programs: Jiamu Bai -> Penn State PhD Fall'2023, Nobel Dang -> Clemson PhD Fall'2023, Nuo Chen -> UM Twin City PhD Fall'2024, Zonglin Lyu -> Utah PhD Fall'2024, Sizhe Wei -> GaTech PhD Fall'2024, Xinran Tang -> UCF PhD Fall'2024, Zhitao Li -> Lehigh PhD Fall'2024.
  • [02/2024] Two papers accepted by CVPR 2024.
  • [02/2024] Congratulations to Wenyu for successfully defending his PhD thesis! His defense talk is here.
  • [01/2024] Three papers accepted by ICRA 2024.
  • [12/2023] Congratulations to Yiming for winning the NVIDIA Scholarship! Read the news.
  • [05/2023] One paper accepted by RSS 2023.
  • [04/2023] Congratulations to Yuchen and Yaxuan for being accepted to PhD programs respectively at Purdue CORAL Lab and Stevens Institute of Technology.
  • [02/2023] Two papers are accepted by CVPR 2023.
  • [02/2023] Dr. Feng received the NSF Career Award under the Foundational Research in Robotics (FRR) program.
  • [02/2023] Dr. Feng will serve as the area chair for ICCV 2023.
  • [02/2023] We will organize the first coperception workshop at ICRA 2023.
  • [02/2023] Congratulations to Yuxiang for being accepted to PhD program at CUHK MMLab, to Guanbo for being accepted to PhD program at Rensselaer polytech, to Xinzhou for joining Horizon Robotics, to Sriharsha for interning in Tesla, to Rushi for interning in DEKA Research and Development, and to Fady for interning in Bell labs!
  • [01/2023] One paper is accepted by ICLR 2023.
  • [01/2023] Three papers are accepted by ICRA 2023.

Current Team

Senior Members

Chen Feng

Principle Investigator (PI) of the AI4CE group. One of Chen’s most ambitious academic dream is to enable robots like Wall-E and Baymax which could help humanity.

Chen Feng

Associate Professor at NYU

Jing Zhang

Jing Zhang is a postdoctoral fellow. She received her B.E. in Wuhan University. Her research interest lies in 3D computer vision and the application of artificial intelligence in archaeology.

Jing Zhang

Postdoctoral Fellow

Xuchu Xu

Xuchu is a PhD student. His motto is “Stay hungry, Stay foolish” and wants to be a man from the future.

Xuchu Xu

Ph.D. Candidate in Mechanical Engineering

Yiming Li

Yiming is a PhD student. He received his B.E. in Tongji University. His research interest is Robot Perception, Navigation and Machine Learning.

Yiming Li

Ph.D. Candidate in Mechanical Engineering

Hanwen Zhao

Hanwen is a PhD student. He received his B.S. from University of Colorado Boulder and M.S. from Columbia University. His research interests are Robotics and Artificial Intelligence.

Hanwen Zhao

Ph.D. Candidate in Mechanical Engineering

Chao Chen

Chao is a PHD student. He is interested in computer vision and robotics. His interest shaped him the sense of engineering to adapt rapidly to the study of computer science.

Chao Chen

Ph.D. Candidate in Computer Science

Anbang Yang

Anbang Yang is a PhD candidate in the AI4CE lab at New York University, focusing on developing visual navigation systems to assist visually impaired individuals. He holds a Master’s degree in Mechanical Engineering from New York University and a Bachelor’s degree from Dalian University of Technology. Anbang has experience as a software engineer, developing navigation systems and constructing 3D maps for indoor navigation. His research interests include visual localization, 3D reconstruction, and spatial reasoning.

Anbang Yang

Ph.D. Candidate in Mechanical Engineering

Ruixuan Zhang

Ruixuan is a Ph.D. student in Transportation at the Tandon School of Engineering, co-advised by Prof. Kaan Ozbay. His research focuses on machine learning and intelligent transportation systems, aiming to provide multi-modal (vision and language) solutions for safety and decision-making.

Ruixuan Zhang

Ph.D. Candidate in Transportation Engineering

Bilal Sher

Bilal has an MS in Civil Engineering from NYU, and a BSc. from the University of Calgary. He is passionate about using technologies such as computer vision and deep learning to solve civil engineering problems.

Bilal Sher

Founder and CEO of Building Diagnostic Robotics, Inc.

Diwei Sheng

Diwei is an Ph.D. student in computer science. His research interests are artificial intelligence and computer vision. He is doing research on visual place recognition.

Diwei Sheng

Ph.D. Student in Computer Science

Irving Fang

Irving is a Ph.D. student from the CSE department. He did his undergraduate at UC Berkeley majoring in Data Science and Math. His research interests lie at the intersection of robotics, vision, and machine learning.

Irving Fang

Ph.D. Candidate in CSE

Juexiao Zhang

Juexiao Zhang is a Ph.D. student at the Courant Institute majoring in Computer science. He obtained his BS in EE from Tsinghua University. He is interested in machine learning, especially building AI systems that can efficiently learn representations from multimodal signals.

Juexiao Zhang

Ph.D. Candidate in Computer Science

Sihang Li

Sihang Li is a Ph.D student in M.E in Tandon School of Engineering. He received his B.E. in Tongji University. His research interests are Robot Perception, autonomous driving and Machine Learning.

Sihang Li

Ph.D. candidate in Mechanical Engineering

Xinhao Liu

Xinhao is a Ph.D student in Computer Science. He received his B.Sc. degree from NYU and NYU Shanghai. His research interest is 3D perception and human-like navigation.

Xinhao Liu

Ph.D. Student in Computer Science

Anuj Doshi

My name is Anuj Doshi. I am passionate about working on computer vision problems. My interests are in microscopic object detection, multi-modal AI, and collaborative robotics. I earned my master’s in robotics and mechatronics from NYU and currently lead AI projects at Nanotronics Imaging.

Anuj Doshi

Ph.D. Student in Mechanical Engineering

Mingrui Teng

Mingrui Teng is a visiting Ph.D. student in Civil Engineering, currently pursuing his doctoral degree at Tsinghua University. His research focuses on co-designing and constructing novel masonry structures using robotic arms, employing low-cost hardware setups and address randomness encountered during construction.

Mingrui Teng

Visiting Ph.D. Student

Yuzhong Chen

Yuzhong Chen is a Ph.D student in Computer Science in Tandon School of Engineering. He received his B.Eng. from CUHK(SZ) and M.S. from NYU. He is interested in robot perception and mobile manipulation.

Yuzhong Chen

Ph.D. Student in Computer Science

Junior Members

Antonia Ang

Antonia is pursuing a B.S. in Electrical and Computer Engineering at NYU Tandon. Her interests lie in the intersections of robot vision and machine learning

Antonia Ang

B.S. in ECE

Antonio Carlone

I received a B.Sc. in Civil Engineering from the TU Hamburg, Germany and am currently pursuing a M.Sc. in Civil Engineering at the TU Berlin, Germany. My Master’s thesis in the field of Computing in Civil Engineering is a cooperation between TU Berlin and NYU.

Antonio Carlone

Visiting Research Student

Kacy Murphy

Kacy Murphy is an undergraduate student studying Mechanical Engineering at NYU Tandon School of Engineering. She is additionally pursuing a minor in Robotics and is expanding her interest in the subject as the Engineering Intern for Building Diagnostic Robotics as a member of the NSF Fellowship.

Kacy Murphy

B.S. in Mechanical Engineering

Kairui Shi

Never Give Up

Kairui Shi

Visiting research student

Siqi (Kevin) Tan

Siqi is an MSCS student at NYU with research interests in computer vision and robotics, currently focusing on visual localization, 3D reconstruction, and robotic perception.

Siqi (Kevin) Tan

M.S. in Computer Science

Yipeng Wang

Research interests lie in robot learning in manipulation & vision, and control theories.

Yipeng Wang

B.S. in Electrical & Computer Engineering

James Jin

James majors in computer science and wishes to turn his research interests in computer vision into a PhD career. He believes in the value of serving the academics and enriching its open exchange of knowledge.

James Jin

B.S. in Computer Science

Moonjun Gong

Moonjun Gong is a visiting student. He is passionate about Computer Vision and its applications in robotics.

Moonjun Gong

Visiting research student

Akasha Tigalappanavara

Hello, I am Akasha. I am currently pursuing an M.S. in Computer Engineering, focusing on Deep Learning Algorithms and their applications in Computer Vision and Natural Language Processing.

Akasha Tigalappanavara

M.S. in Computer Engineering

Beichen Wang

Beichen Wang obtained a B.Eng in Mechanical Engineering from SJTU and is currently pursuing an MSCS at NYU Courant. Beichen has worked on projects related to intelligent manufacturing, NLP, and robotics, and has a strong interest in analysis within theoretical mathematics. He aspires to combine his interdisciplinary background to become a full-stack robotic scientist.

Beichen Wang

M.S. in Computer Science

Celestino Martinez

I am currently pursuing a bachelors degree in Computer Engineering with a minor in Robotics. My interests include machine learning, deep learning, and robotics.

Celestino Martinez

B.S. in Computer Engineering

John Abanes

I am a visiting undergraduate from NYU Abu Dhabi. My interests include robot manipulation and navigation. I aspire to be proficient in the entire design process, from mechanical design to implementing control systems and machine learning. At AI4CE, I designed, wired, and programmed mobile data collection systems for the Curb2Door project.

John Abanes

B.S. in Mechanical Engineering at NYUAD

Niranjan Sujay

As a Robotics Engineering graduate from NYU’s Tandon School of Engineering, I have dedicated my academic journey to mastering areas such as Machine Learning, Robotic Perception, and Visual Odometry. My passion for these subjects has not only been theoretical but also practical, as demonstrated in my work at the AI4CE Lab. In the MappingNYC Project at the AI4CE Lab, I was able to apply my knowledge and skills in a real-world context. This project involved creating a detailed map of New York City using advanced perception techniques. This experience allowed me to delve deeper into the practical aspects of robotic perception and significantly enhance my understanding and skills in these areas. With a strong academic background and practical experience, I am excited about the future. I am ready to apply my expertise in robotics and AI, solve complex problems, and contribute to advancements in the field. I am confident in my ability to make a significant impact in the world of robotics and AI. I look forward to the exciting opportunities that lie ahead.

Niranjan Sujay

M.S. in Robotics

Pengfei Zheng

B.S. in Data Science from the University of California, Santa Barbara, and an M.S. in Computer Engineering from NYU Tandon. I have strong research interest in machine learning and computer vision.

Pengfei Zheng

M.S. in Computer Engineering

Shunyu Yao

I am Shunyu Yao, a master’s student in Computer Science with research interests in Point Cloud Registration and Path Planning. An AI researcher with an exceptional background in software engineering.

Shunyu Yao

M.S. in Computer Science

Shuwen Dong

Shuwen Dong is a graduate student in NYU CE. He’s interested in generative artificial intelligence.

Shuwen Dong

M.S. in Computer Engineering

Xueying Tang

Xueying is a senior undergraduate student in Computer Science at CAS. Her research interest is the application of Machine Learning and Computer Vision.

Xueying Tang

B.S. in Computer Science

Jonah Kamat

Jonah is a computer engineering graduate and researcher in the AI4CE lab. His interests lie in computer vision, natural language processing, and 3D reconstruction.

Jonah Kamat

B.S. Computer Engineering

Wenkai Sun

Wenkai is MSCS student at Courant. His research interest is machine learning.

Wenkai Sun

M.S. in Computer Science

Zehong Wang

M.S. student, New York University

Zehong Wang

M.S. in Computer Science

Aarav Pandya

Hi, I am Aarav, pursuing MS in computer engineering at NYU. I am interested in computer vision and reinforcement learning, with my current research at AI4CE focusing on deepmapping and point cloud registration. I am passionate about applying AI in automation, especially robotics and software assistants. I also like to study ethics and implications of AI. When I am not working on AI, I like to play soccer and chess.

Aarav Pandya

M.S. in Computer Engineering

Mohit Mehta

Mohit is an MS student in Computer Science at NYU Tandon. His research interests are 3D computer vision, Autonomous Vehicles, NeRF, and Reinforcement Learning

Mohit Mehta

M.S. in Computer Science

Shuhang Ge

I’m a master student at NYU Tandon in the ECE department. My research interest lies on 3D computer vision, and human–computer interaction

Shuhang Ge

M.S. in Electrical and Computer Engineering

Taarun Srinivas

Hello! I am Taarun Srinivas and currently pursuing masters in Mechatronics and Robotics at NYU Tandon. Deep Learning, Computer Vision, and robotic applications that involve AI techniques are my areas of focus. I love exploration and one of my visions is to explore the unexplored avenues of deep learning and contribute more to the research community.

Taarun Srinivas

M.S. in Mechatronics and Robotics Engineering

Zijun Wang

Zijun’s research and career interests lie in the fields of autonomous vehicle, machine learning and computer vision.

Zijun Wang

B.S. in Mathematics and Computer Science Double Major

Andre(Binglin) Wang

I am a junior major in mathematics and computer science joint major. I am interested in SLAM, robotics, computer vision, and backend programming. I like weight lifting.

Andre(Binglin) Wang

B.A. joint major in CS/Mathematics

Ashay Ajbani

I am currently pursuing my Masters in Mechatronics and Robotics at NYU Tandon. My passion lies in Robot Perception, specifically in the fields of Computer Vision and Deep Learning. In my free time, I enjoy playing cricket.

Ashay Ajbani

M.S. in Mechatronics and Robotics

Haorui Song

M.S. in Computer Science

Idan Lau

I am currently pursuing a BS in Computer Science and a Minor in Mathematics at NYU Tandon School Of Engineering. My interests include Backend Programming, Data Science/Machine Learning and Computer Networking

Idan Lau

B.S. in Computer Science

Keifer Lee

Keifer is a master student at NYU Tandon in the ECE department. He received his bachelors degree in Electrical and Electronics Engineering at Taylor’s University. His research interests lie in the intersection of Machine Learning, Computer Vision and Meta-Learning.

Keifer Lee

M.S. in Computer Engineering

Kevin Lee

Kevin Lee graduated with a Master of Science (M.Sc.) in Computer Science in 2024 from New York University, following his Bachelor of Science (B.Sc.) also in Computer Science (2023). While pursuing his studies, he actively contributed to research at Building Diagnostic Robotics, a New York University robotics startup. There, he focused on integrating deep learning with robotics to develop robust autonomous diagnostics systems. His research interests lie in deep learning, robotics, and signal processing.

Kevin Lee

M.S. in Computer Science

Nayan Sarode

I am Nayan Sarode, an aspiring engineer specializing in Mechatronics, Robotics, and Automation Engineering.Currently pursuing my Master’s degree at New York University (NYU), I’m set to graduate in 2024. My area of interest lies in Autonomous vehicle field. With a strong foundation in Mechanical Engineering from Dr. Babasaheb Ambedkar Technological University, India, I possess a diverse skill set including proficiency in software tools like SolidWorks, MATLAB, ROS, and programming in Python and C++. Projects like creating an autonomous warehouse inventory robot and designing an autonomous floor cleaning robot showcase my hands-on expertise. I’ve also conducted research on electric mobility challenges in India and even filed a patent for energy-generating shock absorber shoes. With a passion for innovation and a commitment to excellence, I am eager to contribute my skills to dynamic projects and teams in the field of robotics and automation.

Nayan Sarode

M.S. in Mechatronics and Robotics

Pratham Mehta

Pratham Mehta is Masters student in Computer Engineering. His research work in past included about Currency Detection for Impaired Users, Abstractive text summarization uses the Pegasus model,and Sanitizing of Differential Privatized Data Models. He is currently researching in Ai4CE lab. His research interests are in Computer Vision, Natural Language Processing, Artificial Intelligence and Cloud Computing.

Pratham Mehta

B.S. in Computer Science

Shubham Gupta

I am a Master’s in Computer Engineering student at NYU Tandon. I work with Dr. Chen Feng on Neural Radiance Fields and efficient VPR for mobile computing. I also work with Dr. Laure Zanna at M2LInES as a research assistant. I lead the Computer Vision team at NYU Robomasters.

Shubham Gupta

M.S. in Computer Engineering

Somik Dhar

Somik is a MS student from the ECE department. His research interests lie at the intersection of robotics, vision, and controls.

Somik Dhar

M.S. in Electrical and Computer Engineering

Tim Guo

Tim (Qinsong) Guo an undergraduate student majoring in mechanical engineering. He likes playing the piano, running, and gaming with friends. In AI4CE, he is a member of the DeepSoRo group and NYU Selfdrive, focusing on soft robotics, machine learning, and hardware controls.

Tim Guo

B.S. in Mechanical Engineering

Wei-Heng Lin

I graduated from NYU Tandon with M.S. in Computer Science in 2024 and am currently working at BDR to develop ML solutions for building anomaly detection. My interest lies in solving real-life problems using Computer Vision and Reinforcement Learning.

Wei-Heng Lin

M.S. in Computer Science

Xiangzhi Tong

Xiangzhi Tong is a graduate student majoring in ECE at Cornell. His research lies in the intersection of Computer Vision and Robotics.

Xiangzhi Tong

Visiting research student

Yang Zou

Hi my name is Yang Zou! I am a undergraduate student at NYU tandon. I am really interested in deep mapping.

Yang Zou

B.S. in Computer Science

Yiping (Dennis) Du

I’m an undergraduate student majoring in Electrical Engineering. I serve as the lab manager at AI4CE and am the student leader of the NYU Self-Drive VIP (Vertically Integrated Projects).

Yiping (Dennis) Du

B.S. Electrical Engineering

Recent Publications

*
CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos
Memorize What Matters: Emergent Scene Decomposition from Multitraverse
Multiview Scene Graph
FusionSense: Bridging Common Sense, Vision, and Touch for Robust Sparse-View Reconstruction
VLM See, Robot Do: Human Demo Video to Robot Action Plan via Vision Language Model
EgoPAT3Dv2: Predicting 3D Action Target from 2D Egocentric Vision for Human-Robot Interaction
Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset
ActFormer: Scalable Collaborative Perception via Active Queries
LUWA Dataset: Learning Lithic Use-Wear Analysis on Microscopic Images
NYC-Indoor-VPR: A Long-Term Indoor Visual Place Recognition Dataset with Semi-Automatic Annotation
Tell Me Where You Are: Multimodal LLMs Meet Place Recognition
LiDAR-based 4D Occupancy Completion and Forecasting
Among Us: Adversarially Robust Collaborative Perception by Consensus
SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving
Collaborative Multi-Object Tracking with Conformal Uncertainty Propagation
Metric-Free Exploration for Topological Mapping by Task and Motion Imitation in Feature Space
Toward Zero-Shot Sim-to-Real Transfer Learning for Pneumatic Soft Robot 3D Proprioceptive Sensing
DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization
VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion
Learning Simultaneous Navigation and Construction in Grid Worlds
Uncertainty Quantification of Collaborative Detection for Self-Driving
Multi-Robot Scene Completion: Towards Task-Agnostic Collaborative Perception
Self-Supervised Visual Place Recognition by Mining Temporal and Feature Neighborhoods
V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving
Self-supervised Spatial Reasoning on Multi-View Line Drawings
Egocentric Prediction of Action Target in 3D
A Deep Reinforcement Learning Environment for Particle Robot Navigation and Object Manipulation
Deep Weakly Supervised Positioning for Indoor Mobile Robots
Learning Distilled Collaboration Graph for Multi-Agent Perception
NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences
Mobile 3D Printing Robot Simulation with Viscoelastic Fluids
Mobile projective augmented reality for collaborative robots in construction
Projector-Guided Non-Holonomic Mobile 3D Printing
Simultaneous Navigation and Construction Benchmarking Environments
Fooling LiDAR Perception via Adversarial Trajectory Perturbation
3D Point Cloud Processing and Learning for Autonomous Driving: Impacting Map Creation, Localization, and Perception
A scene-adaptive descriptor for visual SLAM-based locating applications in built environments
LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood
Real-time Soft Body 3D Proprioception via Deep Vision-based Sensing
Regularizing Neural Networks via Minimizing Hyperspherical Energy
SPARE3D: A Dataset for SPAtial REasoning on Three-View Line Drawings
Deep unsupervised learning of 3D point clouds via graph topology inference and filtering
An assistive low-vision platform that augments spatial cognition through proprioceptive guidance: Point-to-Tell-and-Touch
An Occupancy Grid Mapping enhanced visual SLAM for real-time locating applications in indoor GPS-denied environments
DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds
VLASE: Vehicle Localization by Aggregating Semantic Edges
Simultaneous Edge Alignment and Learning
Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
FoldingNet: Point cloud auto-encoder via deep grid deformation
Fast Resampling of Three-Dimensional Point Clouds via Graphs
FasTFit: A fast T-spline fitting algorithm
Direct Multichannel Tracking
CASENet: Deep Category-Aware Semantic Edge Detection
Deep Active Learning for Civil Infrastructure Defect Detection and Classification
Camera marker networks for articulated machine pose estimation
Fast plane extraction in organized point clouds using agglomerative hierarchical clustering
Point-plane SLAM for hand-held 3D sensors

Datasets

MARS
In collaboration with the self-driving company May Mobility, we present the MARS dataset which unifies scenarios that enable multiagent, multitraversal, and multimodal autonomous vehicle research.
View Details
SSC Bench
Semantic scene completion (SSC) is crucial for holistic 3D scene understanding by jointly estimating semantics and geometry from sparse observations. We introduce SSCBench, a comprehensive benchmark that integrates scenes from widely-used automotive datasets (e.g., KITTI-360, nuScenes, and Waymo).
View Details
NYU-VPR
Composed of images recorded in Manhattan, New York from April 2016 to March 2017. The images were recorded by cameras installed on the front, back, and side parts of taxis with auto-exposure.
View Details
V2X-Sim
A Virtual Collaborative Perception Dataset and Benchmark for Autonomous Driving
View Details
SNAC
Simultaneous Navigation and Construction Benchmarking Environments
View Details
RealCity3D
A Large-scale Georeferenced 3D Shape Dataset of Real-world Cities
View Details
SPARE3D
Contains three types of 2D-3D reasoning tasks on view consistency, camera pose, and shape generation, with increasing difficulty
View Details
EgoPAT3D
An Egocentric Action Target Prediction Dataset
View Details

Teaching

We offer the following courses at NYU across multiple departments:
 
 
 
 
 
ROB-GY 6203 Robot Perception
Every Fall (since 2018) Graduate
 
 
 
 
 
ROB-UY 3203 Robot Vision
Every Spring (since 2020) Undergraduate
 
 
 
 
 
NYU Self-Drive (VIP)
Every Semester (since 2019)
 
 
 
 
 
Machine Learning for MAE/Robotics
2020 Summer Graduate
 
 
 
 
 
Emerging Information and Automation Technologies
2019 Spring Graduate

Funding Sources

We are very grateful for the funding supports from the following agencies, organizations, and companies:

Contact

  • cfeng at nyu dot edu
  • 4th floor, RH409, 6 MetroTech Center, New York University, NY 11201
  • Email to book an appointment