Convolutional neural networks (CNNs)

This is student work which has not yet been approved as correct by the instructor

Case study notes

Introduction

A convolutional neural network is a class of feedforward (which means that the connections between the units do not form a cycle and every unit in a layer is connected with all the units in the previous layer, as shown in the picture) artificial neural network that is used to analyze images.

How does it work or a deeper look

One of the biggest advantages of CNNs over other image classification algorithms is that, like all Aritficial Neural Networks, they are relatively independent from pre-processing, or prior knowledge provided by humans. Instead of using hand-engineered algorithms, they learn on their own how to most accurately and effectively evaluate their input through processing huge amounts of labelled information from large image databases. CNNs fundamentally work on the same set of rules as other feedforward neural networks. They operate in two phases: the learning phase and the classification phase.

In the learning phase, they take take input (in case of image processing - a set of numbers describing each pixel, e.g. numbers from 0-255 for greyscale) together with its correct classification (e.g. a given image description has to fit one category, e.g. “cat pictures” or “dog pictures”). The set of values describing a given image is presented at the inputs. It will then be transformed depending on the weights of each connection as it reaches the output layer. Each unit from the output layer belongs to a different category and so now the output of the network is compared to the ideal output that would have been, has the pattern been correctly classified (the largest value in the correct category unit, very small values for other categories). On the basis of this comparison and through the use of advanced algorithms, such as backpropagation, all the connection weights are slightly modified to ensure that the next time the same information is presented at the inputs, the correct output unit has a higher value and the incorrect units have lower values than before. Ultimately the goal is that the network can correctly classify all the information-category pairs and learn to generalize and correctly classify new, unknown inputs.

In the classification phase the connection weights are no longer changeable. The information given at the inputs will be transformed based on the fixed weights until it reaches the output layer. Now the classification is simple - the output unit with the highest value is selected as the best fitting category. Obviously, the classification phase is much faster and more simple than the learning phase.

Examples

Ever since Alex Krizhevsky used CNNs to win the ImageNet competition in 2012 (basically, the annual Olympics of computer vision) dropping the classification error record from 26% to 15%, a host of companies have been using deep learning at the core of their services. Facebook uses neural nets for their automatic tagging algorithms, Google for their photo search, Amazon for their product recommendations, Pinterest for their home feed personalization, and Instagram for their search infrastructure.

One of the new applications of CNNs at the frontier of new technologies is to use them in self driving cars to process and analyse the environment around them.

Pictures, diagrams

Pictures and diagrams go a LONG way to helping someone understand a topic. Especially if your topic is a little abstract or complex. Using a picture or diagram is a two part process:

1. upload a file
2. use the file on a wiki page

Devote time and attention to gaining knowledge of (an academic subject), especially by means of books

a group or system of interconnected people or things.

Break down in order to bring out the essential elements or structure. To identify parts and relationships, and to interpret information to reach conclusions.

a group or system of interconnected people or things.

Assess the implications and limitations; make judgments about the ideas, works, solutions or methods in relation to selected criteria.

a group or system of interconnected people or things.

Arrange or order by class or category.

Break down in order to bring out the essential elements or structure. To identify parts and relationships, and to interpret information to reach conclusions.