Movies of Object Tracking software

These movies are for Object Tracking Software.

1. Simple Camshift




2. Simple Camshift (Pocky, Camera:fixed)




3. Hybrid method with Gaussian (Pocky, Camera:fixed)





4. Hybrid method with Median (Pocky, Camera:fixed)




5. Hybrid method with Median (Hough Transform)

Object Tracking -Shelf Detection & Camshift -

I incorporated Hough Transform component and Object Tracking component (Camshift). What I made is a hybrid object tracking system of shelf-detection and Camshift.
Shelf-detection component sends Camshift component camera displacement data observed, and Camshift component tracks the item, using the data for setting an initial region to search the item.
And then, shelf-detection component start the observation of the shelf near the item location tracked by Camshift component.

Here is a screen shot of the software. In this case, "Pocky"(Japanese famous chocolate cookie) is the target.


Unfortunately, the result was a disappointing one. The shelf-detection isn't accurate enough to track the object effectively, while Camshift component is working well.

I wonder if I should give up using Hough Transform for shelf-detection...

Moving Average of "middle point"s & Filters

I modified the program using 5-point moving average of "middle point"s, instead of "middle point" itself, for observation of displacement and direction of the camera's motion.

It works to stabilize the detection of "middle point". But It might not be enough.
Now I use a Gaussian filter to suppress noises as pre-processing. Gaussian filters also make edges muddy. So I tried to find another filter to improve the situation.

[Median Filter]
This filter is effective for eliminating pepper & salt noises and remains edges sharp. It's great simply for edge detection. However, in this case, it might increase the number of edge points for Hough-Transform processing, and slow down the process.

[Bilateral Filter]
This is a kind of non-linear filter. Pixels having similar values in a certain region are unified to have the same value. If it's applied again and again, it will change real images into illustration like image. This filter is likely to work effectively to detect the edge of shelves. The problem is it needs much power to process. I'm afraid that it's not possible to use in real-time, though it might be useful to detect the color of the edge of shelves in initial process.

I should keep trying to find better filters, while I'm working on incorporating Hough-Transform component into object tracking algorithm.

Stability of Line detections (Week2)

Is it holiday today? I don't get used to American national holiday system...

I showed the line detection component with Hough Transforms last time. It could detect lines corresponding to edges of shelves. But I found it's sometimes unstable.

These are examples of detected lines. You can see the length and the location of the red line is changing (jumping), and so the location of the green circle - "middle point" does. It's certainly owing to noises on captured images.

It might be a problem to use the location of "middle point". I tried to adjust parameters of Hough transform and Gaussian filter for more stable detection, but it still remains unsolved.

I had a discussion about this problem with the advisior Serge Belongie. He was still positive with this result. He told me it was not so bad with such a simple technology, and advised me to move on developing object tracking component.

Detection of Lines from Captured Image (Week 1)

By the end of week 1, I built a Hough Transform component to detect lines from captured image by web camera (Microsoft-VX 6000).

It could detect lines corresponding to edges of shelves, under certain conditions. Now I don't have any data-sets available, such as still images and movies at a grocery store. So I tested it by using a desk in my laboratory.

This is an example of captured image with two boxes of RITZ.


And here is the result of Hough Transform. This software detects lines whose slopes are under 30 degrees to the horizontal line, so that it could detect those corresponding to edges of shelves.

The red line and the blue line are corresponding to edges of the desk. And a green circle is a "middle point" of these lines. In this case, the circle represents the location of edges well.

Hough Transform

Hough Transform is a conventional way to detect lines from images. Let me show you an example. Here is a crazy collection of "TIDE" (from rakuten.co.jp).


And the result of Hough Transform on this image is below.


White lines are showing edge lines of the image, and red lines are detected by Hough Transform. Red lines correspond to edges of shelves well.
It might be useful to detect edges of shelves and get information on the location of visually-impaired against shelves.

Interface of RSG @ GroZi

I designed and implemented Remote Sighted Guide Interface last summer (Please see the picture below).
The GroZi is a solution for visually impaired to enjoy an individual shopping for groceries without the aid of another person. And the RSG interface is software for guides who support visually-impaired shopping from a remote area.

However, there are no functions such as object recognition and object tracking in current RSG interface. In this CSE-190a course, I am going to study the effectiveness of object tracking for helping RSG.