Publication Date

8-2017

Date of Final Oral Examination (Defense)

6-19-2017

Type of Culminating Activity

Thesis

Degree Title

Master of Science in Computer Science

Department

Computer Science

Supervisory Committee Chair

Edoardo Serra, Ph.D.

Supervisory Committee Member

Tim Andersen, Ph.D.

Supervisory Committee Member

Francesca Spezzano, Ph.D.

Abstract

Unknown image type identification is the problem of identifying unknown types of images from the set of already provided images that are considered to be known, where the known and unknown sets represent different content types. Solving this problem has a lot of security applications such as suspicious object detection during baggage scanning at airport customs, border protection via remote sensing, cancer detection, weather and disaster monitoring, etc. In this thesis, we focus on identification of unknown landscape images. This application has a huge relevance to the context of a smart nation where it can be applied to major national security tasks such as monitoring the borders or the detection of unknown and potentially dangerous landscapes in critical locations.

We propose effective semi-supervised novelty detection approaches for the unknown image type identification problem using Convolutional Neural Network (CNN) Transfer Learning. Recently, the CNN Transfer Learning approach has been very successful in various visual recognition tasks especially in cases where large training data is not available. Our main idea is to use pre-trained CNNs (i.e. already trained on large datasets like ImageNet [10]) that are then used to train new models specifically applicable to the landscape image dataset. Features extracted from these domain-specific trained CNN are then used with standard semi-supervised novelty detection algorithms like Gaussian Mixture Model, Isolation Forest, One-class Support Vector Machines (SVM) and Bayesian Gaussian Mixture Models to identify the unknown landscape images.

We provide two fine-tuning approaches: supervised and unsupervised. Supervised fine-tuning approach simply uses the the class categories (landscape classes, e.g. airport, stadium, etc.) of the known images dataset. The unsupervised fine tuning approach on the other hand learns the class categories from the known images using the unsupervised clustering-based algorithm. We conducted extensive experiments that prove the effectiveness of our approaches. Our best values of AUROC and average precision scores for the identification problem are 0.96 and 0.94, respectively. In particular, we statistically prove that both fine-tuning methods significantly increase the performance of the identification with respect to the non fine-tuned CNN, and unsupervised and supervised fine tuning approaches are comparable.

DOI

https://doi.org/10.18122/B2C70F

Download

Share

COinS