Assignment 2 (15% of total course weight) – Computer vision with CNNs

You can use Spyder (installed through Anaconda from week 1 Thursday’s lecture) or other IDE.

Files to Edit and Submit: You will need to edit and submit (digitRecognizer.py) file which will consist of your CNN model for MNIST dataset. You can copy and paste all the necessary pieces of code that we discuss here in the Assignment details section. Once you have completed the implementation of your model in this file, you should save your model with the name “digitRecognizer.h5” and upload it in Zipped folder in your assignment. To load your model, you can do the import “from keras.models import load_model” library from Keras. Once completed your digitRecognizer.py, you will need to implement the code in evaluation.py file to load your model (i.e., digitRecognizer.h5) via load_model, predict the classes for the test cases via the function already implemented in evaluation.py and print your predictions. For the development of your CNN model, please refer to the flowchart given in Assignment details section and the code we covered in Week 13 for CNNs implemented for cat/dog classification. You will need to test your model with evaluation.py to copy screenshots in your report.

Academic Dishonesty: Your code will be checked against other submissions in the class for logical redundancy. If you copy someone else’s code and submit it with minor changes, they will be detected, so please do not try that, and submit your own work only. In case of cheating, the University’s academic policies on cheating and dishonesty will strictly apply which may result from the deduction in your grade to expulsion.

Getting Help: If you are having difficulty in implementing your CNN model, contact course staff for help and refer to the content recorded and uploaded in Week 13 part 2 (Thursday) on Computer Vision using CNNs). Office hours and Slack are there for your support. If you are not able to attend office hours, then please inform your instructor to arrange for additional time. The intent is to make this assignment rewarding and instructional, not frustrating and demoralizing. You can either complete this assignment on your own or discuss the problem and collaborate with another member of the class (or different section). Please clearly acknowledge and mention your group member in your assignment report submission who you will collaborate with in this assignment. Your report and program (digitRecognizer.py, evaluation.py files and digitRecognizer.h5 model) will be separately submitted by yourself on Blackboard irrespective of your collaboration with your group member. Group discussions are encouraged but copying of programs is NOT allowed. Programming based on your own skills is encouraged.

1

Assignment Details

1. Modified National Institute of Standards and Technology (MNIST) dataset

The MNIST dataset was compiled and processed by Yann LeCun, Corinna Cortes and Christopher Burges to develop and evaluate new machine learning models on handwritten digit classification problem. The dataset was constructed from scanned document dataset available from the NIST, therefore naming it to Modified NIST (MNIST). As discussed in the class, the images of digits were taken from a variety of scanned documents, and then normalized in size and centered for model development. Image data preprocessing already makes the dataset in nearly ready form requiring little to no cleaning/augmentation task unlike we experienced in cat/dog classification task in Week 13. Each image in the dataset is a 28 × 28 pixel square, with a total of 784 pixels. When you load the dataset via Keras library in your code, you will realize that the dataset is already divided into train and test splits. There is a total of 70,000 images where 60,000 images are used to train a model and a separate set of 10,000 images are used to test it.

To date, various models including the CNNs have been developed with near state-of-the-art accuracy of above 99%. You have also seen a live demonstration of handwritten digit recognition via the web in the class. Your task will be to develop your own similar CNN for handwritten digit recognition via Keras library in Python. With a fairly larger CNN (see section 2 for implementation), it is expected that you will achieve an accuracy of nearly or even above 97% with <3% classification error. To learn how to load and assign train/test portions of the data, please visit this web page:

2. Convolutional Neural Network (CNN) for MNIST dataset – 15%

Please see below flowchart to implement your CNN model. For more details, please see the empty digitRecognizer.py file with comments to implement your code, and review code example covered in Week 13 on Computer Vision using Convolutional Neural Networks (CNN) with the example of cat/dog classification. Please note that you might need to change your CNN settings (i.e., number of Conv filters in each layer, number of units in each Dense layer or even adding/removing Conv/Dense layers) to achieve the accuracy of 97% or above.

Note: For model evaluation, you can use built-in Keras model evaluation function by writing code like model.evaluate(X_test, y_test, verbose=0). For this work, please write the description of your work in the report (2-page maximum) that may include your modifications in the CNN structure, or the description of your CNN even if you did not make any modification in the structure but still were able to get the accuracy of >97%. Use screenshots wherever needed.

Evaluation: You will copy the code in your evaluation.py file to only load your saved model from digitRecognizer.py file, use your saved model and make prediction on a few test images given in the assignment folder.

3. Assignment Report – 15%

Your report will be based on your explanation on MNIST dataset which will include your explanation of your CNN model implementation. You can even propose your own test images of custom drawn images (from 0 to 9) in a software of your own choice (e.g., digits drawn in MS paint with black background and white digit pixels). Please use screenshots (or manually edited versions) of your model training process to support your explanation. Your model training process should print 10 epochs with model metrics.

Assignment 2 (15%)

CSE 5120 (Section ##) – Introduction to Artificial Intelligence – Spring 2021 

Submitted to  

Department of Computer Science and Engineering California State University, San Bernardino, California 

by

Student name (CSUSB ID)

(Your collaborator in this homework (if any))

Date: Month Day, Year

Email:

· Your email

· Your collaborator’s email (if you collaborated with any)

Assignment Report

Brief description of your work here acknowledging your collaboration with your class fellow (or a friend from other CSE 5120 section), and the capacity at which he/she collaborated with you, followed by the algorithms you implemented.

1. digitRecognizer.py for MNIST dataset

Your brief explanation of the dataset, your code solution, and any documentation with screenshots of your code Evaluation (results from digitRecognizer.py)

2. Evaluation (evaluation.py) for your model performance evaluation

You can also provide brief description of your code written in evaluation.py to load your saved model that can be readily used on test dataset for the staff.

MNIST DatasetConvolution2D + ReLU(30filters, size = 5î5)MaxPooling2D(Pool size = 2, Strides = 2)Convolution2D + ReLU(15 filters, size = 3î3)MaxPooling2D(Pool size = 2, Strides = 2)Dropout(Probability 20%)FlattenDense (128 units + ReLU)Dense(50units)+ReLUDense (10 units) + softmaxDo prediction hereYou will need to reshape your train/test images to 28î28 pixels and then normalize them (e.g., Xtrain=Xtrain/255)You will need to reshape new unseen image for prediction to 28î28 pixels and then normalize it (e.g., newImage=newImage/255)