From Introduction to Robotics (CS460)
Jump to: navigation, search

Object Seeking: Lets Find the Object and Gets me out

The objective of this project is to design and implement a robot controller for the Turtlebot robot to recoginze the set of objects using kinect senos while trying to escape from enclosure using its bumper sensor. This project will help the students to apply the knowledge about OpenCV and ROS.

Learning outcomes

At the completion of this project, the student will be able to:

  • Using Kinect sensor to get images.
  • Understanding basic concepts of OpenCV.
  • Using template matching to find the objects in area.
  • Use ROS topics to make local decision in the robot
  • Control the behaviour of the robot using ROS
  • Manipulate ROS topics and nodes to control the robot

Submission and deadline

The deadline for project submission is November 13, 2013.
Details will be given later on what and how to submit.

ROS version

This project is designed to be developed using ROS Groovy version with roscpp package (using C++). You must have ROS Groovy installed and properly configured in your computer machine. Your machine must run Linux Ubuntu 12.04 or Ubuntu 12.10.
For any query, ask your instructors.


Before starting this project, you are assumed to have completed the Beginner Level ROS tutorials. You also need to see the package "Vision" [1] and also cover the tutorials for Vision_OpenCV [2]. In particular, you should have basic knowledge about OpenCV.
In addition, you are assumed to know how to start Camera node from kinect Sensor[3].

Problem Statement

In this project, you are requested to implement the program object_seeking.cpp that makes the robot to get the images from Kinect Camera after getting the image the image has to be matched with the stored images. If the object are found the robot makes the hurray sound and then try to perform random moves in a closed environment area until it goes out of the enclosure. The robot should start moving at a certain direction until it his an obstacle, then decide to turn left or right to choose another direction.


Create a workpackage

In the sandbox folder of your ROS workspace already created, you need to create a new ROS package called enclosure_escape. The command is

$ roscd
$ cd sandbox 
$ roscreate-pkg object_seeking roscpp kobuki_msgs geometry_msgs cv_bridge std_msgs image_transport

If you forgot the details on how to create a ROS package see Creating ROS Package Tutorial.
Note that we included three ROS dependencies in this package namely roscpp, for C++ ROS code, kobuki_msgs for reading sensor data from the kobuki based (in particular the bumper sensor through the message kobuki_msgs/BumperEvent), and the geometry_msgs required for sending linear and angular velocities through geometry_msgs/Twist message structure.

Then, in the src folder of the object_seeking package, you create a new CPP file called object_seeking.cpp.

Develop your program

To help you starting out, we provide you the following code below. Copy and paste the code your object_seeking.cpp file.

#include <ros/ros.h>
#include <iostream>

#include <image_transport/image_transport.h>
#include <cv_bridge/cv_bridge.h>
#include <sensor_msgs/image_encodings.h>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/contrib/contrib.hpp>

using namespace cv;
using namespace std;

int main(int argc, char** argv) 
cv::namedWindow("Find Object"); 

  ros::init(argc, argv, "Template_Mat_converter");
  ros::NodeHandle nh_;
  cv_bridge::CvImagePtr cv_ptr0;
  cv_bridge::CvImagePtr cv_ptr1;

   cv::imshow("Find Object", ref);

   return 0;

check that the code is compiles with no problems. Debug first if any compilation problem arise.

$ rosmake object_seeking

Hints for solving the problem

To be provided ...