Compact Composite Descriptors

Compact Composite Descriptors (CCDs) is set of low lever features that can be used to describe various types of multimedia information. The set contains descriptors for 3 different types of images. A group of 2 descriptors (CEDD and FCTH) combines color and texture information in order to describe the visual content of  the real-world color images. At the same time, another descriptor (BTDH) combines brightness and texture characteristics in order to describe the visual content of grayscale images (as well as radiology medical images). The recently proposed SpCD combines color and spatial distribution characteristics to describe artificially generated images (computer graphics, color sketches etc.). From this page, one can download open source software (C#, Java and MATLAB) as well as CUDA based implementations of the CCDs.

 compact-composite-descriptorsCompact Composite Descriptors for Content Based Image Retrieval. Basics, Concepts, Tools

ISBN-10: 363937391X, August 24, 2011, Paperback: 216 pages

Available at Amazon

Authors: Savvas A. Chatzichristofis and Yiannis S. Boutalis

 

Book Description: This book covers the state of the art in image indexing and retrieval techniques paying particular attention in recent trends and applications. It presents the basic notions and tools of content-based image description and retrieval, covering all significant aspects of image preprocessing, features extraction, similarity matching and evaluation methods. Particular emphasis is given in recent computational intelligence techniques for producing compact content based descriptors comprising color, texture and spatial distribution information. Early and late fusion techniques are also used for improving retrieval results from large probably distributed inhomogenous databases. The book reports on the basic international standards and provides an updated presentation of the current retrieval systems. Numerous utilities and techniques are implemented in software, which is provided as a supplementary material under an open-source license agreement. The book is particularly useful for postgraduate students and researchers in the field of image retrieval, who want to easily elaborate and test state of the art techniques and possibly incorporate it in their development.

[Download the Preface]

Color and Edge Directivity Descriptor (CEDD) and Fuzzy Color and Texture Histogram (FCTH)

 (NEW) Local Color And Texture descriptor (LoCATe) (The first local CCD)

Brightness and Texture Directionality Histogram (BTDH)

– Spatial Color Distribution Descriptor (SpCD)

Web Based Demo 


1. Compact Composite Descriptors for Natural Color Images: Color and Edge Directivity Descriptor (CEDD) and Fuzzy Color and Texture Histogram (FCTH)

The structure of these descriptors consists of n texture areas. In particular, each texture area is separated into 24 sub regions, with each sub region describing a color. CEDD and FCTH use the same color information, as it results from 2 fuzzy systems that map the colors of the image in a 24-color custom palette.
To extract texture information, CEDD uses a fuzzy version of the five digital filters proposed by the MPEG-7 EHD, forming 6 texture areas. In contrast, FCTH uses the high frequency bands of the Haar wavelet Transform in a fuzzy system, to form 8 texture areas. When an image block (rectangular part of the image) interacts with the system that extracts CEDD and FCTH, this section of the image simultaneously goes across 2 units. The first unit, the color unit, classifies the image block into one of the 24 shades used by the system. Let the classification be in the color m,m E [0,23]. The second unit, the texture unit, classifies this section of the image in the texture area a,a E[0,n]. The image block is classified in the bin a X 24 +m. The process is repeated for all the image blocks of the image. On the completion of the process, the histogram is normalized within the interval [0:1] and quantized for binary representation in a three bits per bin quantization. The most important characteristic of CEDD and FCTH is the very good results that they bring up in various known benchmarking image databases [Download the retrieval results in several databases].

Another important characteristic of CEDD and FCTH is the small size needed for indexing images. The CEDD length is 54 bytes per image while FCTH length is 72 bytes per image.

[*] Important notice: There is no significant difference between these two versions of the descriptors (experiments conducted on UCID and Wang database). Ver 1.01 fixes minor bugs and slightly improves the efficiency of the descriptor. If you choose to update the descriptor in your implementation, please note that you have to re-index your data.

[**] For academic purposes only

1.1 C# Instructions:

In order to use them, add the dll as references in your c# project and then call the classes as:

(HighlightSearching in more than 1.000.000 images with the CEDD feature

1.2 MATLAB Instructions:

The source code is quite simple and easy to be handled  by all users. There is a main function that has the task of extracting the CEDD descriptor from a given image.

  • Download the MATLAB implementation of CEDD [Ver 1.0] [FAPO] [updated:1/27/2014]

function [ DescriptorVector ] = CEDD(ImageRGB)

1.3 Joint Composite Descriptor:

Observing the CEDD and FCTH results in various queries, it is easy to ascertain that in some of the queries, better retrieval results are achieved by using CEDD, while in others by using FCTH.
Given that the CEDD descriptor as well as the FCTH descriptor for each image is available in the index file that the retrieval system uses, it should be asked if the 2 descriptors could be combined to achieve better retrieval results. Based on the fact that the color information given by the 2 descriptors comes from the same fuzzy system, we can assume that joining the descriptors will rely on the combining of texture areas carried by each descriptor.
Therefore, uniting the CEDD and FCTH will lead to a new descriptor that will be made up of a combination of CEDD and FCTH texture areas. The combined descriptor is called Joint Composite Descriptor (JCD).

  • Download JCD as C# Source Code [Ver 1.0]  [GNU GPL]

In order to use the JCD Source Code, add the CEDD and FCTH dlls, as references in your c# project and then call the classes as:

1.4 How to Compare the descriptors:

Finally, in order to calculate the distance between two descriptors use the following method (Tanimoto Coefficient)

tanimoto

Download an example application, in which we describe the retrieval procedure. Source Code included (C# Visual Studio 2008 Solution) [GNU GPL]

1.5 Java Version of the Descriptors:

A JAVA implementation of the descriptors (CEDD, FCTH and JCD) is available through the LIRE library.  Lire is an open source project hosted at Google Code

1.6 CUDA™ based implementation of CEDD (NEW):

In order to execute this application, the latest version on the nVIDIA CUDA driver must be installed. A message stating “There is no device supporting CUDA” will appear during execution if CUDA’s GPU is not enabled or the respective driver is not up-to-date.  The CUDA™ drivers can be found here.

cuda

Make sure to place the “cudart32_42_9.dll” file together with the application in the same directory and execute the latter using the Windows Command Prompt. Executing the application without any arguments will extract the CEDD descriptors for all images (bmp format) that are contained in the same directory as the application. Including the folder’s full path as an argument in the Command Prompt, allows the user to extract the descriptors of images stored in directories other than the one the application is in. Please note that the application only works with bmp formatted images.

  • Download CUDA based implementation [Ver 0.7  (beta)] [GNU GPL]
  • Download the source code of the CUDA™ based implementation of CEDD [CUDA_Only] [Hybrid] [GNU GPL]

Preliminary results

cuda

Average Indexing Time for 1000 Images

More details, comparisons and other staff will be added soon. Please note that CUDA based implementation produce a lossless representation of the descriptor.

Credits must be given to Loukas Bampis 😉

1.7 More details and References:

Details regarding these descriptors can be found at the following papers: (in other words, if you use these descriptors in your scientific work, we kindly ask you to cite one or more of the following papers 😉 )

1. S. A. Chatzichristofis and Y. S. Boutalis, “FCTH: FUZZY COLOR AND TEXTURE HISTOGRAM- A LOW LEVEL FEATURE FOR ACCURATE IMAGE RETRIEVAL”, «9th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS)”, IEEE Computer Society, pp.191-196 , May 7 to 9, 2008, Klagenfurt, Austria.

2. S. A. Chatzichristofis and Y. S. Boutalis, “CEDD: COLOR AND EDGE DIRECTIVITY DESCRIPTOR – A COMPACT DESCRIPTOR FOR IMAGE INDEXING AND RETRIEVAL.”, «6th International Conference in advanced research on Computer Vision Systems (ICVS)», Lecture Notes in Computer Science (LNCS), pp.312-322, May 12 to 15, 2008, Santorini, Greece.

3. S. A. Chatzichristofis, Y. S. Boutalis and M. Lux, “SELECTION OF THE PROPER COMPACT COMPOSIDE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL.”, «The Sixth IASTED International Conference on Signal Processing, Pattern Recognition and Applications (SPPRA)», ACTA PRESS, pp.134-140, February 17 to 19, 2009, Innsbruck, Austria.

4. S. A. Chatzichristofis, K. Zagoris, Y. S. Boutalis and N. Papamarkos, “ACCURATE IMAGE RETRIEVAL BASED ON COMPACT COMPOSITE DESCRIPTORS AND RELEVANCE FEEDBACK INFORMATION”, «International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI)», Volume 24, Number 2 / February, 2010, pp. 207-244, World Scientific.


2. CCD for Medical Radiology Images: Brightness and Texture Directionality Histogram (BTDH)

Fuzzy rule based scalable composite descriptor (BTDH) is a compact composite descriptor that can be used for the indexing and retrieval of radiology medical images. This descriptor uses brightness and texture characteristics as well as the spatial distribution of these characteristics in one compact 1D vector. The most important characteristic of the proposed descriptor is that its size adapts according to the storage capabilities of the application that is using it. This characteristic renders the descriptor appropriate for use in large medical (or gray scale) image databases.
To extract the proposed descriptor, a two unit fuzzy system is used. To extract the brightness information, a fuzzy unit classifies the brightness values of the image’s pixels into L_{Bright} clusters. The cluster centers are calculated using the Gustafson Kessel Fuzzy Classifier.
The texture information embodied in the proposed descriptor comes from the Directionality histogram. This feature is part of the well known Tamura texture features.
Fractal Scanning method through the Hilbert Curve or the Z-Grid method is used to capture the spatial distribution of brightness and texture information.
Seven approaches used in our experiments in order to evaluate the performance of this descriptor:

The approach displaying the best correlation between size and result is approach E5. [Download the retrieval results in several databases].

  • Download BTDH as .NET DLL [ Ver 1.01 (Updated Aprils 1st 2013)] [GNU GPL]
  • Download BTDH as c# Source Code [ Ver 1.01 (Updated Aprils 1st 2013)] [GNU GPL]
  • Download an example application, in which we describe the retrieval procedure [Ver 1.00] [Ver 1.01] [GNU GPL]

2.1 Instructions:

In order to employ BTDH in your solution, add the dll as reference in your c# project and then call the class as:

btdh

Details regarding BTDH descriptor can be found at the following paper:

2.2 More details and References:

1. S. A. Chatzichristofis and Y. S. Boutalis, “CONTENT BASED RADIOLOGY IMAGE RETRIEVAL USING A FUZZY RULE BASED SCALABLE COMPOSITE DESCRIPTOR” «Multimedia Tools and Applications», Volume 46, Numbers 2-3 / January, 2010, pp. 493-519, Springer.



3. CCD for Artificially Generated Images: Spatial Color Distribution Descriptor (SpCD)

The descriptor is not available at this time.

Details regarding SpCD descriptor can be found at the following paper:

3.1 More details and References:

1. S. A. Chatzichristofis, Y. S. Boutalis and M. Lux, “COMBINING COLOR AND SPATIAL COLOR DISTRIBUTION INFORMATION IN A FUZZY RULE BASED COMPACT COMPOSITE DESCRIPTOR.”, «Communications in Computer and Information Science, Springer-Verlag, CCIS 129», J. Filipe, A. Fred, and B. Sharp (Eds.), pp. 49-60, 2011.