CS 5970 Programming Assignment 1 — Backprop

Due Monday, September 21, 2016

NOTE: This assignment is due at the beginning of the class
period. This means that if you are even a minute late, you lose 20%. If
you are worried about potentially being late, turn in your assignments
ahead of time. Do this by submitting them through D2L before the
time they are due. Do not send assignments to me through email or leave
them in my departmental mail box.

1. Motivation

One of the most widely known, best understood, and most widely used
types of artificial neural network (ANN) is a feedforward neural network
(FFNN) trained using supervised learning via error backpropagation
(backprop). Therefore, knowing how to design and train backprop nets for
various problems are fundamental skills for ANN practitioners and
understanding the functioning of these ANNs is a fundamental skill for ANN
researchers. Moreover, backprop nets are quite often used for pattern
classification and function approximation. As you have already worked to
develop your understanding of using FFNNs for classification problems in
Homeworks 1 and 2, you will here focus on function approximation.

2. Goals

The goals of this assignment are:

to give you baseline experience with implementing an ANN,

to give you experience with backprop learning systems for FFNNs in
particular,

to familiarize you with some of the issues facing ANN
implementors,

to give you experience with the supervised learning process.

3. Assignment Overview

You will design, program, and run a simple backprop net to learn the
following simple functions:

f(x) = 10×sin(x)&plus;x2; −5 ≤ x ≤ &plus;5

f(x,y) = 10×sin(x)&plus;y2; −5 ≤ x ≤ &plus;5, −10 ≤ y ≤ &plus;10

You will train, validate, and test your ANN on these functions both with
and without noise. The noise for the first function will have a mean of
zero and a variance of 5. The noise for the second function will have a
mean of zero and a variance of 50. You will also turn in written material
regarding the design and implementation choices you made regarding the FFNN
and an analysis of the results you will collected from your runs.

4. Assignment Details

Carry out the following steps. Underlined steps require a
written response, those in code require you to write
software, and those in italics require you to collect data.
Written responses, code, and data will be
turned in for grading

Consider the choices one needs to make regarding the design
and implementation of any ANN learning system.

List the choices that need to be made when designing an
ANN learning system that have already been made for you
in this assignment. You should be able to list at least
four.

For each of these choices, list which option I chose for you
in making this assignment.

List the choices you need to make regarding the design of your
ANN. (Note, these do not include purely implementation choices
such as programming language.) You should be able to list at least
four.

Choose an option that seems reasonable to you for each of these
design choices and explain why it seems reasonable to you. If you
do not have a good reason for your chosen option, say so. Note that you
may want to choose different options for your networks for each of the
functions you are to approximate.

Implement your ANN.

Now that you have implemented your ANN, you are likely to have
recognized more choices that you needed to make along the way.
List the choices you needed to make regarding the design of your
ANN. Including the choices you listed previously, your list
should now contain at least eight choices.

For each of
these design choices, list the option you chose, and explain why it
seems reasonable to you. If you do not have a good reason for
your chosen option, say so.

Answer the following questions about data collection, reporting,
and conclusions so that you are ensured of collecting the appropriate
data. Attempt to justify your answers to these questions. As you
do so, keep in mind that what you want is a data set that allows you to
understand the workings of your ANN as a function approximation tool and
you want to report the minimum amount that allows your reader to
thoroughly understand what you have learned. Note that you may lack a
justification for your answers to some of these questions at this time.
That is acceptable since this is your first programming assignment in
this course. However, you should keep all of these questions in your
mind as the course progresses and be able to give good, justified answers
to similar questions for later assignments.

How many data samples will you use to train your ANN? To validate
it? To test it?

How many times will you run (train, validate, and test) your ANN in
order to be able to give an accurate measure of its performance on
these functions? Once? Ten times? Twenty times? 100 times?

If you run your ANN more than once, what will you change
from run to run? What will you keep the same?

What data will you collect?

What data will you report? Everything? Best performance? Worst
performance? Averages? If averages, what will be averaged together?
Average error at a given time step during a single run? At the end of
a run? Averages over multiple runs?

How will you report this data? Text? Numbers? Graphs? What
form will these take?

What conclusions will you be able to draw from your results?

Run your ANN on f(x) without noise.

Report your results and conclusions regarding the application
of your ANN to approximating this function.

Run your ANN on f(x) with noise.

Report your results and conclusions regarding the application
of your ANN to approximating this function.

Run your ANN on f(x,y) without noise.

Report your results and conclusions regarding the application of
your ANN to approximating this function.

Run your ANN on f(x,y) with noise.

Report your results and conclusions regarding the application of
your ANN to approximating this function.

Compare the performance of your ANN on these two functions with
and without noise and report your conclusions.

5. What to Turn In

Write Up

You will turn in an electronic copy of your write up. Your write-up
should be a coherent document that covers all of the underlined
steps from the assignment above. Note that selected data in a digested
form (such as tables or graphs) should be included in your writeup;
however, your raw data should not be included here.

Code

You will turn in an electronic copy of your code. You will turn in
the source code you have written for this ANN. Your source code should
be well structured and well commented. It should conform to good coding
standards (e.g., no memory leaks).

Data

You will turn in an electronic copy of your data. This
may be in a single file or multiple files. You will also need to include
a brief writeup on how the data is organized.

6. Notes on this Assignment

You may write your program from scratch or may start from programs
for which the source code is freely available (such as on the web or
from friends or student organizations). If you do not start from
scratch, you must give a complete and accurate accounting of
where all of your code came from and indicate which parts are original
or changed, and which you got from which other source. Similarly, for
the written components of this assignment you may follow the format or
content of other written works but you must give a complete and
accurate accounting of who deserves credit for all parts of your
documents. Failure to give credit where credit is due is academic fraud
and will be dealt with accordingly. Please see the University’s web pages on academic
integrity.