Background: This is the final project for my Master's degree of Robotics in Northwestern University. Date: 2019/12/15.
Abstract: The project name is called Simon Says: After I say a number, the Turtlebot Robot detects this number on the whiteboard in front of it, and then moves there and crashes into the number.
Demo:
In 2019/06, I took the above demo video. The project's repo is here. However, the code is in a mess and is hardware dependent, which I do not recommend reading.
In 2019/10~2019/12, I divided this big project into 4 separate ROS packages and refactored the code. Each package uses ROS topics/services as interface, and is self-contained. See below. (But I haven't combined them to create another video demo.)
Method: MFCCs features + LSTM.
Abstract: (1) Press key to record audio; (2) Speak a word to microphone; (3) Finally, see the classification result on GUI and ROS topic.
Repo: https://github.com/felixchenfy/ros_speech_commands_classification
Method: Data augmentation + YOLOv3.
Abstract: Run 3 scripts to (1) Synthesize images (by putting few template images onto backgrounds), (2) Train YOLOv3, and (3) Detect objects for: one image, images, video, webcam, or ROS topic.
Repo: https://github.com/felixchenfy/ros_yolo_as_template_matching
Method: Depth Image + RANSAC
Abstract: A python node to detect planes from depth image by using RANSAC algorithm.
Repo: https://github.com/felixchenfy/ros_detect_planes_from_depth_img
Method: "Move to Pose" algorithm
Abstract: ROS services for controlling Turtlebot3 to target pose.
Repo: https://github.com/felixchenfy/ros_turtlebot_control
I have developed several other small packages that might be helpful when dealing with image data:
- ros_images_publisher: A python script to publish color or depth images from a folder to ROS topic.
- ros_pub_and_sub_rgbd_and_cloud: Python nodes to publish/subscribe RGB-D images and their point clouds (or any of them) to/from ROS topics.
- ros_record_rgbd_images: Press key to record color/depth images from ROS topics or Realsense.
- record_images_from_usbcam: Run one script and press 's'/'d' to save your laptop's camera images to disk.
- open3d_ros_pointcloud_conversion 2 Python API functions for point cloud conversion between Open3D and ROS.
The idea of this Simon Says project was proposed by Prof. Ying Wu. Many thanks to Prof. Wu for providing me the chance of joining his lab, working on this interesting project, and sharing papers in the group meeting.
Also, I'd like to express great thanks to Matthew, who is the Vice Director of MS Robotics program, for the guidance and help to my project through out the whole program.