Multimedia Content Management

Introduction:

The goal of this assignment is to develop a visual feature extraction tool to

create visual feature descriptors based on color and use these feature descriptors to conduct content-based image retrieval.

Content-based multimedia management is based on effective descriptions of the perceived media content. Color is an important visual feature for both human vision and computer processing. In this assignment, you will build a tool to extract color histograms in the RGB and HSV color spaces. To evaluate your color feature descriptors, you will perform image retrieval experiments to evaluate their effectiveness.

Task 1

You will create a class called ColorFeatureExtractionTool and implement the following public methods.

1. ColorFeatureExtractionTool::GetHistogramRGB(...)

This method will calculate the histograms of an image in RGB space.

2. ColorFeatureExtractionTool::GetHistogramHSV(...)

This method will calculate the histograms of an image in HSV space.

Two ways can be used to generate a histogram. One is to treat three channels jointly. For example, if 6 bins are used for each channel, the total number of bins will be 6 x 6 x 6 = 216. The other is to treat the three channels individually. If 6 bins are used for each channel, the total number of bins will be 6 + 6 + 6 = 18.

You may create other public or private methods where appropriate.

Task 2

Create a program or more programs to evaluate the color features. You

should carry out experiments and report the results of ranking and

similarity/distance in the following table for each experiment.

The following experiments are to be reported. All experiments use the image “rose0.jpg” as a query and rank the rest of the images using a selected color feature and similarity/distance measure.

Experiments:

Note: For each of the following experiment, you need to

• test both “jointly” and “individually” ways to create the color histograms.
• chose the appropriate number of bins for each channel based on your investigation.

1. 1. Color histogram in the RGB color space and cosine similarity;
2. 2. Color histogram in the RGB color space and L1 distance;
3. 3. Color histogram in the RGB color space and L2 distance;

4. 4. Color histogram in the HSV color space and cosine similarity;
5. 5. Color histogram in the HSV color space and L1 distance;
6. 6. Color histogram in the HSV color space and L2 distance;

Task 3

Create a color image database that shall satisfy the following requirements:

1. 1. The number of categories shall be 10. The categories can be “trees”, “sea”, “Sydney opera house”, “rose”, et You are free to decide the categories that you are going to use;
2. 2. The number of images in each category shall be 2
3. 3. You can use existing image databases in the community of computer vision or multimedi Also, you are free to collect the images from

the Internet such as Google (by taking care of the copyright issue).

4. 4. Randomly select 10 images from each category and use each of them as the query in turn. Evaluate the Precision value for the top 5, 10, 15,

20, 30, 40 and 50 retrieved images, and plot the average Precision curve for these 100 queries.

5. 5. The Precision for top k retrieved images is defined as the percentage of true relevant images among the top k retrieved images;
6. 6. For each query, the “true relevant images” are defined as the images

that are in the same category as this query.

Report

Write a report in PDF format that includes the following parts:

1. 1. Explanation of the color spaces, the process to extract the color histograms, and the process to create the image database in one to two pag
2. 2. Description of data structures, main components and work flow of your programs in one to two pag
3. 3. Report of all experimental results in Task 2 and the analysis and

discussion.

4. 4. Report of all experimental results in Task 3 and the analysis and discussion.

Important Note:

• You must include the command-line to generate the results for the experiments in Task 2.
• You do NOT need to submit the image database created in Task 3 but need to include at least one example image for each category in the report.
• You may be required to demonstrate all tasks.