Image Classifier Using MLP/CNN

Build a classifier on CIFAR10 dataset using

  • 3-layer MLP
  • 3-layer +1 CNN

Compare performance

Using 3-Layer MLP

In [1]:
import tensorflow as tf
import numpy as np

from keras.datasets import cifar10
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.core import Dense, Flatten, Activation
from keras.layers import Conv2D, MaxPooling2D
from keras.callbacks import ModelCheckpoint
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.utils import to_categorical, plot_model
In [2]:
# load dataset
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
num_labels = len(np.unique(y_train))
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

image_size = x_train.shape[1]
input_size = image_size * image_size * 3

# resize and normalize
x_train = np.reshape(x_train, [-1, input_size]) 
x_train = x_train.astype('float32') / 255
x_test = np.reshape(x_test, [-1, input_size])
x_test = x_test.astype('float32') / 255
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
170500096/170498071 [==============================] - 4s 0us/step
170508288/170498071 [==============================] - 4s 0us/step
In [3]:
# hyper-parameters
batch_size = 256
n_units = 256
epochs = 30
In [4]:
# model is a 3-layer MLP
model = Sequential(name='3-layer-MLP')
model.add(Dense(n_units, input_dim=input_size, activation='relu', name='input_layer'))
model.add(Dense(n_units, activation='relu', name='hidden_layer'))
model.add(Dense(num_labels, activation='softmax', name='output_layer'))
model.summary()
Model: "3-layer-MLP"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_layer (Dense)          (None, 256)               786688    
_________________________________________________________________
hidden_layer (Dense)         (None, 256)               65792     
_________________________________________________________________
output_layer (Dense)         (None, 10)                2570      
=================================================================
Total params: 855,050
Trainable params: 855,050
Non-trainable params: 0
_________________________________________________________________
In [5]:
model.compile(loss="mean_squared_error", optimizer="sgd", metrics=["mean_squared_error"])

history = model.fit(
    x_train,
    y_train,
    validation_data=(x_test, y_test),
    epochs=epochs,
    batch_size=batch_size,
)

loss, acc = model.evaluate(x_test, y_test, batch_size=batch_size, verbose=False)

print("\nTest accuracy: %.1f%%" % (100.0 * acc))
Epoch 1/30
196/196 [==============================] - 15s 6ms/step - loss: 0.0911 - mean_squared_error: 0.0911 - val_loss: 0.0896 - val_mean_squared_error: 0.0896
Epoch 2/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0895 - mean_squared_error: 0.0895 - val_loss: 0.0890 - val_mean_squared_error: 0.0890
Epoch 3/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0890 - mean_squared_error: 0.0890 - val_loss: 0.0887 - val_mean_squared_error: 0.0887
Epoch 4/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0887 - mean_squared_error: 0.0887 - val_loss: 0.0883 - val_mean_squared_error: 0.0883
Epoch 5/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0883 - mean_squared_error: 0.0883 - val_loss: 0.0880 - val_mean_squared_error: 0.0880
Epoch 6/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0881 - mean_squared_error: 0.0881 - val_loss: 0.0877 - val_mean_squared_error: 0.0877
Epoch 7/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0877 - mean_squared_error: 0.0877 - val_loss: 0.0874 - val_mean_squared_error: 0.0874
Epoch 8/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0875 - mean_squared_error: 0.0875 - val_loss: 0.0872 - val_mean_squared_error: 0.0872
Epoch 9/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0872 - mean_squared_error: 0.0872 - val_loss: 0.0869 - val_mean_squared_error: 0.0869
Epoch 10/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0870 - mean_squared_error: 0.0870 - val_loss: 0.0867 - val_mean_squared_error: 0.0867
Epoch 11/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0867 - mean_squared_error: 0.0867 - val_loss: 0.0864 - val_mean_squared_error: 0.0864
Epoch 12/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0865 - mean_squared_error: 0.0865 - val_loss: 0.0862 - val_mean_squared_error: 0.0862
Epoch 13/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0863 - mean_squared_error: 0.0863 - val_loss: 0.0859 - val_mean_squared_error: 0.0859
Epoch 14/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0860 - mean_squared_error: 0.0860 - val_loss: 0.0857 - val_mean_squared_error: 0.0857
Epoch 15/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0858 - mean_squared_error: 0.0858 - val_loss: 0.0855 - val_mean_squared_error: 0.0855
Epoch 16/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0855 - mean_squared_error: 0.0855 - val_loss: 0.0852 - val_mean_squared_error: 0.0852
Epoch 17/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0852 - mean_squared_error: 0.0852 - val_loss: 0.0850 - val_mean_squared_error: 0.0850
Epoch 18/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0851 - mean_squared_error: 0.0851 - val_loss: 0.0848 - val_mean_squared_error: 0.0848
Epoch 19/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0849 - mean_squared_error: 0.0849 - val_loss: 0.0846 - val_mean_squared_error: 0.0846
Epoch 20/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0847 - mean_squared_error: 0.0847 - val_loss: 0.0844 - val_mean_squared_error: 0.0844
Epoch 21/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0845 - mean_squared_error: 0.0845 - val_loss: 0.0842 - val_mean_squared_error: 0.0842
Epoch 22/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0842 - mean_squared_error: 0.0842 - val_loss: 0.0840 - val_mean_squared_error: 0.0840
Epoch 23/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0841 - mean_squared_error: 0.0841 - val_loss: 0.0838 - val_mean_squared_error: 0.0838
Epoch 24/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0840 - mean_squared_error: 0.0840 - val_loss: 0.0837 - val_mean_squared_error: 0.0837
Epoch 25/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0837 - mean_squared_error: 0.0837 - val_loss: 0.0835 - val_mean_squared_error: 0.0835
Epoch 26/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0836 - mean_squared_error: 0.0836 - val_loss: 0.0833 - val_mean_squared_error: 0.0833
Epoch 27/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0834 - mean_squared_error: 0.0834 - val_loss: 0.0832 - val_mean_squared_error: 0.0832
Epoch 28/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0832 - mean_squared_error: 0.0832 - val_loss: 0.0830 - val_mean_squared_error: 0.0830
Epoch 29/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0832 - mean_squared_error: 0.0832 - val_loss: 0.0828 - val_mean_squared_error: 0.0828
Epoch 30/30
196/196 [==============================] - 1s 4ms/step - loss: 0.0829 - mean_squared_error: 0.0829 - val_loss: 0.0827 - val_mean_squared_error: 0.0827

Test accuracy: 8.3%

Using 3-Layer CNN

In [6]:
# load dataset
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
In [7]:
# input image dimensions
image_size = x_train.shape[1]
# resize and normalize
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255

# network parameters
input_shape = (image_size, image_size, 3)
batch_size = 256
kernel_size = 3
pool_size = 3
filters = 32
In [8]:
model = Sequential()
model.add(Conv2D(filters=filters,
                 kernel_size=kernel_size, 
                 activation='relu', 
                 kernel_initializer='he_uniform', 
                 padding='same', 
                 input_shape=input_shape,
                 name='input_layer'))

model.add(MaxPooling2D(pool_size,name='hidden_layer'))
model.add(Flatten(name='hidden_layer2'))
model.add(Dense(10, activation='softmax',name='output_layer'))

model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_layer (Conv2D)         (None, 32, 32, 32)        896       
_________________________________________________________________
hidden_layer (MaxPooling2D)  (None, 10, 10, 32)        0         
_________________________________________________________________
hidden_layer2 (Flatten)      (None, 3200)              0         
_________________________________________________________________
output_layer (Dense)         (None, 10)                32010     
=================================================================
Total params: 32,906
Trainable params: 32,906
Non-trainable params: 0
_________________________________________________________________
In [9]:
model.compile(loss="categorical_crossentropy", optimizer="sgd", metrics=["accuracy"])
# train the network
model.fit(x_train, y_train, epochs=epochs, batch_size=batch_size)

loss, acc = model.evaluate(x_test, y_test, batch_size=batch_size)
print("\nTest accuracy: %.1f%%" % (100.0 * acc))
Epoch 1/30
196/196 [==============================] - 31s 7ms/step - loss: 2.2855 - accuracy: 0.1836
Epoch 2/30
196/196 [==============================] - 1s 7ms/step - loss: 1.9397 - accuracy: 0.3161
Epoch 3/30
196/196 [==============================] - 1s 7ms/step - loss: 1.8230 - accuracy: 0.3659
Epoch 4/30
196/196 [==============================] - 1s 7ms/step - loss: 1.7521 - accuracy: 0.3915
Epoch 5/30
196/196 [==============================] - 1s 7ms/step - loss: 1.7004 - accuracy: 0.4155
Epoch 6/30
196/196 [==============================] - 1s 7ms/step - loss: 1.6645 - accuracy: 0.4246
Epoch 7/30
196/196 [==============================] - 1s 7ms/step - loss: 1.6193 - accuracy: 0.4424
Epoch 8/30
196/196 [==============================] - 1s 7ms/step - loss: 1.5992 - accuracy: 0.4481
Epoch 9/30
196/196 [==============================] - 1s 7ms/step - loss: 1.5670 - accuracy: 0.4608
Epoch 10/30
196/196 [==============================] - 1s 7ms/step - loss: 1.5419 - accuracy: 0.4699
Epoch 11/30
196/196 [==============================] - 1s 7ms/step - loss: 1.5288 - accuracy: 0.4759
Epoch 12/30
196/196 [==============================] - 1s 7ms/step - loss: 1.5045 - accuracy: 0.4830
Epoch 13/30
196/196 [==============================] - 1s 7ms/step - loss: 1.4870 - accuracy: 0.4859
Epoch 14/30
196/196 [==============================] - 1s 7ms/step - loss: 1.4717 - accuracy: 0.4923
Epoch 15/30
196/196 [==============================] - 1s 7ms/step - loss: 1.4469 - accuracy: 0.5022
Epoch 16/30
196/196 [==============================] - 1s 7ms/step - loss: 1.4431 - accuracy: 0.5033
Epoch 17/30
196/196 [==============================] - 1s 7ms/step - loss: 1.4229 - accuracy: 0.5123
Epoch 18/30
196/196 [==============================] - 1s 7ms/step - loss: 1.4066 - accuracy: 0.5170
Epoch 19/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3892 - accuracy: 0.5226
Epoch 20/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3766 - accuracy: 0.5273
Epoch 21/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3665 - accuracy: 0.5300
Epoch 22/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3458 - accuracy: 0.5408
Epoch 23/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3360 - accuracy: 0.5412
Epoch 24/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3252 - accuracy: 0.5426
Epoch 25/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3151 - accuracy: 0.5469
Epoch 26/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3080 - accuracy: 0.5485
Epoch 27/30
196/196 [==============================] - 1s 7ms/step - loss: 1.3007 - accuracy: 0.5516
Epoch 28/30
196/196 [==============================] - 1s 7ms/step - loss: 1.2964 - accuracy: 0.5553
Epoch 29/30
196/196 [==============================] - 1s 7ms/step - loss: 1.2860 - accuracy: 0.5595
Epoch 30/30
196/196 [==============================] - 1s 7ms/step - loss: 1.2757 - accuracy: 0.5616
40/40 [==============================] - 0s 4ms/step - loss: 1.2965 - accuracy: 0.5587

Test accuracy: 55.9%

Performance

The MLP model was outperformed by the CNN model using the same epochs and layers. The MLP's accuracy of ~48% was lower compared to the CNN accuracy of ~55%. The MLP also used more parameters at 855,050 compared to the CNN using only 32,906. Thus, the CNN model was more parameter efficient and has higher accuracy. Using the CNN model is more suitable for this dataset.

In [ ]: