Lab and Exercises on Neural Networks

Work through the following sections, in our adapted version of the Deep Learning lab from the book.

• Single Layer Network on Hitters Data
• Multilayer Network on the MNIST Digit Data
• Convolutional Neural Networks
• Using Pretrained CNN Models

• dl-requirements.txt
• dl_imports.py
• hitters.py
• mnist.py
• cifar.py
• pretrained_cnn.py

1. From Section 10.10 of ISLP, exercises 7 & 8

2. Experiment with The TensorFlow Playground and write a report on your findings.

   There are many ways in which you can vary the configuration, such as

   • Dataset
   • Problem type
   • Features
   • Number of layers
   • Number of neurons in each layer
   • Activation function
   • Regularization and regularization rate
   • Learning rate
   • Train-test split ratio
   • Added noise

   Vary these individually and in combination with each other. Which settings and combinations of settings are more effective? Which are less so? Which settings influence the run time in terms of epochs and in terms of wall clock time? Which settings are more suitable for one problem type and/or dataset than another? Which feature sets are more suitable for one problem type and/or dataset than another?

   After describing some experiments, synthesize your findings. In other words, don't just write up a "washing list" of findings, but order your thoughts to show the understanding you have gained from your experiments.

   (Previously students have written between 500-1500 words on this assignment, including screenshots and/or graphs where appropriate. Be sure to cite any sources you may use.)