Presentation

[elevenaugust]

Back to 2007..... The California drone story began this year and, while others researchers used 3D techniques to show how (especially the 'Raj' drone) had possibly faked shadows, I thought that another good technique could be to try to discriminate in an irrefutable way Computer Generated Images from real Photographies.

This technique, theoretically, could serves the whole Ufology as well, that's why I thought that take again my study (began almost two years ago on OMF) and try to finish it could be very important.

Everyone who have any Java knowledge is welcome to help me.

In order to try to accomplish this, I began to search if any technical paper was already made about this. With a simple Google search, an interesting paper popped up:

Detecting Differences between CGI and Photographs

This paper was presented by their three authors, Jie Wu, Markad V. Kamath and Skip Poehlman (from the Department of Computing and Software and Department of Medicine, MacMaster University, Hamilton, Ontario, Canada) at the 24th IASTED International Conference of Signal Processing, Pattern Recognition and Applications, Innsbruck Austria, in February 15/17 2006.

Here what the paper says (my preliminary comments are in blue)

1- ABSTRACT:
With the development of computer graphic rendering software and the appearance of more and more photorealistic pictures, the need for automatically distinguish Computer Generated Images from real photographs has become of particular interest to criminal and forensic science investigators.
Previous studies have been based on wavelet statistics, while in our study we examined several visual features derived from colour, edge, saturation and texture feature extrated with the Gabor filter.
Based on the feature extraction, we examined these commonly-used classifiers: non-linear SVM, Weighted k-nearest neighbors and Fuzzy k-nearest neighbors with 1.044 Computer Generated Images and 1.114 photographs  (It's their control image basis) downloaded from open sources.
Finally, we report on the comparative analysis of the results of these automatic classifications: Gabor filter based feature shows very promising results (99% for photo and 91.5% for CGI) while visual features show some abilities to perform differentiation.

[elevenaugust]

1.  Introduction

One of the goals in realistic rendering is to generate images that are indistinguishable from photographs.
When images that are computer generated cannot be distinguished from real photographs, issues such as
freedom of expression may surface. For example, the United States Congress passed The Child Pornography
Prevention Act which prohibited any image that appears to or conveys the impression of someone under 18
engaged in sexually explicit conduct in 1996.
This law made it illegal to transmit computer generated pictures that only appear to show minors involved in sexual
activity. In 2002, however, the United States Supreme Court struck down this law in their 6-3 ruling in Ashcroft
v.Free Speech Coalition - the court said language in the 1996 child pornography law was unconstitutionally vague
and far-reaching. This ruling makes it considerably more difficult for law enforcement agencies to prosecute child
pornography crimes, since one may claim that a certain image is computer generated.

Thus the need for automatically distinguishing Computer Generated Images and real photographs is of
particular interest to fields such as Forensic Science and Law Enforcement. Also, research seems to be in its
infancy in this area.

We use the term Computer Generated Images (CGI) to identify computer generated images which appear to be
photographs. It is different from the definition of other authors where Computer Graphics include various kinds of
graphics generated by computer, like icons, buttons, etc.
But in our study, CGI only means photo-realistic images, which are very difficult to distinguish from photographs,
(while UFO photographies were until now mostly fuzzy night lights, blurry tiny object, etc.... The drones [alleged] pictures appears to be a clear and neat-looking kind of UFO, that could be the first fruits of it)
excluding those relatively easily distinguished graphics (icons,buttons,etc).

So the problem of classification can be defined as follows: given an image, classify it into either a
photograph or a CGI. We train the system with a set of labeled images.Following training, any input image will
be assigned to either one of the two classes.

The contributions of this paper are as follows:
1)  Four image visual features derived from the images which differentiate photos and CGIs are proposed
and evaluated.
2)  A weighted distance based K-NN algorithm is used on classification and shows better performance.
3)  Gabor Wavelet transform is used to extract texture features and these features are demonstrated as being
useful in the discrimination of photos and CGIs.

The rest of the paper is organized as follows: Section 2 briefly outlines the previous work in related image
classification area. Section 3 introduces the datasets. Section4 examines visual features used in our study and
their evaluation as well as classification. Section 5 discusses Gabor wavelet texture features and the
classification. Finally in section 6 conclusions are drawn from the work.

[elevenaugust]

2. Literature Survey

To our knowledge, there is only one study which addresses the problem of photograph-CGI discrimination: [1] by Lyu and Farid. (Hanid Farid produces a lot of great papers about Forensic analysis and techniques for detecting forgeries...) They described a statistical model for photographic images that is built upon a wavelet-like decomposition. The model consists of first- and higher-order statistics that capture regularities that are inherent to photographic images. In our paper, we extracted Gabor wavelet based texture features to find the irregularities embedded in photos and CGIs. Besides these, visual features are evaluated for their ability to discriminate between photos and CGIs.

There are other papers with a focus on improving the photorealism of computer generated images: [13], [14] and [16], [17].

Some publications centered on other fields of image classification: techniques to differentiate between city and landscape images: [3] and [4], in-door and out-door [5], photograph vs graphics [7] and photographs vs painting [6] differentiation.

Differentiating city from landscape images and in-door from out-door images are more concerned about semantic issues, while our differentiation is not concerned with the internal semantics. Differences between photographs and CGI are subtler than the differences between photographs vs graphics and photographs vs paintings. Radmacher [2] and Mayer [8] have discussed these issues from a human perception point of view. Differentiating photos from CGI through texture features is shown to be useful in our studies.

(References will be added at the end of the paper presentation, and before my study)

[elevenaugust]

3.  Data Sets

In this study 1,044 CGI and 1,114 photographs were used. Among these CGIs and photographs, 200 of the CGIs and 200 of the photos were used as test sets and the rest were used as training sets. The definition of CGI in our context has been given in section 1.
These pictures were downloaded from open sources. Approximately 80% of the photographs were downloaded from Charles W. Cushman Photograph Collection of Indiana University, and the rest were from [20]. (I used for my own study the exact same sources with the same ratio) Their pixel dimensions range from 375×500 to 528×1024.
The photorealistic images were downloaded from [21] and [22]. (I have had some problems to find 1.044 CGI using these only two sources, that's why I haven't finished to make up the whole data set....[635 CGI at the date of today]) Their dimensions ranged from 270×315 to 1600×1200 pixels. Both sources contain a variety of content such as human subjects, landscapes, indoor scenes, etc.
The image set contained only colour pictures because the present work focuses on features obtained from colour images.


4. Visual Features

We seek the determination of a good feature set which is able to discriminate photos from CGI. But it is quite a challenge to choose from a number of visual features. The first step in our study was to select four visual features and evaluate them.

4.1 Feature Selection
What are the key visual features that can readily distinguish photographs and CGIs? Rademacher [2] proposed that Surface Smoothness and Shadow Softness are two important factors when humans distinguish photos and CGIs with their eyes. Cutzuetal [6] proposed a set of features that are good for distinguishing photos from paintings. Based upon our visual inspection of some photographs and photorealistic images, we found that several features described by Cutzuetal [6] can be used in our algorithms to distinguish photographs and CGIs.

   •  Number  of  unique  colours:
Computer Generated Images tend to have fewer unique colours than photographs. In a previous study, Cutzu et al [6] found that paintings appear to have more colours than photographs. But our observation about CGI  is quite the opposite. In our study we observed that the average number of colours of photographs is shown to be about 25% more than that of CGI. Although CGI software provides a large colour palette, as does a real palette, drawing a picture on a computer is still different from drawing a painting on canvas. For example, if we draw a line with a brush on canvas, the colour will start fading from the beginning of the line to the end. But if we do it using computer software, the line will remain the same colour as the one you choose, unless we do something to change it afterwards. (In other words, there are more chances to find for the same numbers of colour pixels the exact same colour in CGI than in photographies... Unless you use a special brush in CGI for the whole creation) But photographs may contain all natural colours, so actually CGIs do not have as many unique colours as photos do. The richness of the colour palette of images, U, as described in [6], can be calculated by normalizing the total number of unique colours by the total number of pixels. Given an image with a <R,G,B> triplet for each pixel, we say that two pixels have the same colour  if and only if they have the same R, G and B components. To reduce the impact of noise, a colour triplet was counted only if it appears in more than 10 pixels.

[elevenaugust]

Now, here's my Java code for this first visual feature:

Code: [Select]

[b][i]import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import javax.imageio.ImageIO;

public class ColorCounter {

    public static void main(String[] args) {
       
        // lecture de l'image
        BufferedImage image = null;
        try {
            image = ImageIO.read(new File("26.jpg"));
        } catch (IOException e) {
            e.printStackTrace();
            return;
        }
       
        // Map (rgb)=>count
        Map<Integer, Integer> rgbmapcount = new HashMap<Integer, Integer>();
       
        // parcours des pixels de l'image, et creation/mise a jour de la Map
        for(int y=0;y<image.getHeight();y++) {
            for(int x=0;x<image.getWidth();x++) {
                // valeur [a]rgb, codé sur un entier (4 octets)
                int rgb = image.getRGB(x,y);
               
                // recherche dans la map
                if (!rgbmapcount.containsKey(rgb)) {
                    rgbmapcount.put(rgb, 1); // n'existe pas => creation
                } else {
                    int previouscount = rgbmapcount.get(rgb);
                    rgbmapcount.put(rgb, previouscount+1); // n'existe => mise a jour
                }
            }
        }
       
        // Combien de couleurs apparaissent plus de 10 fois ?
        int distinctcolor=0;
        int pixelcount=0;
        for(int rgb:rgbmapcount.keySet()) {
            int rgbcount = rgbmapcount.get(rgb);
            if (rgbcount<10) continue;
            distinctcolor++;
            pixelcount+=rgbcount;
        }
        System.out.println("nombre de couleurs distinctes: "+distinctcolor);
        System.out.println("nombre de pixels: "+pixelcount);
       
        // Richesse de la palette
        double U = (double)distinctcolor / pixelcount;
        System.out.println("Richesse de la palette: "+U);
    }
   
}[/i][/b]
The implementation is in French, that's why some parts of it are written in French.
If someone is interested for any translation, just let me know and I'll do it.

• Spatial  variation  of  colour:
  

We  used  the observation  that  colour  changes  occur  in  a  lesser extent  from  pixel  to  pixel  in  CGIs  than  in photographs.  The  reason  is  possibly  because photographs  contain  real  scenes  and  objects  with many natural colour changes, while CGIs do not have such  a  luxury  because  of  inherent  limitations  of  drawing  with  a  limited  colour  palette.  In  order  to quantify the spatial variation of colour, for each pixel of images, we determined the orientation of the plane that best fitted a 5*5 neighborhood centered on the pixel of interest.
After normalizing triplets <R,G,B> to

Then a variation α can be obtained by:

where the Euclidean Distance of each pixel in the 5*5 neighborhood  to  the  centre  pixel  is  summed  up  and normalized by the total number 25.

.....to be continued....