COMP9444 and COMP9844 Simple Recurrent Networks Assignment

Aim | Specification | Configuration Schema | Resources

Assessment | What to Submit | How to Submit | Extra question for COMP9844 students | Due Date

Aim

The aim of this assignment is for you to:

• devise a suitable data encoding/decoding scheme for a neural network
• design a recurrent neural network and training regime
• use a neural network application (tlearn) to implement a recurrent neural network
• analyse training and testing results.

Specification

You will use the xtlearn NN simulator to configure and train a simple recurrent network. The network will be trained to predict the value of Woolworths stock market shares.

The file orig_data.txt (graph) contains the 36 months' worth of data - one row for each month. The first column is the value of the stock on that month. The second column is the value of the "Retail" index on that month. Both columns have been normalised such that on month 1 the value is $100. You will note that as time progresses (down the list), the value of the stock and the index fluctuates.

The problem for the neural network is, "Given the current month's stock value and retail index value, what is the stock value for the next month?" Using our data file this can be obtained by looking at the next row. In other words, the target values for the network are those given in file orig_teach.txt. You will note that we only have 35 months worth of target values as we do not know the value of the stock or index on month 37. This implies we can only train the network on the first 35 months worth of data.

Now, the numbers in the two data files are not suitable for training a neural network in their current form. Your first task is to design a suitable encoding scheme and translate the two files accordingly.

As an example translation, you could encode the number using binary notation. In that case the translated data could be as in these data and target files. Note that this is an example only. These files may contain mistakes and you may consider this encoding scheme to be unsuitable! You will be marked on your justification of the choice of representation that you make - there is no one right answer.

For this assignment you will be using a Elman (= simple recurrent) network. Consequently, the network will have:

• an input layer that will have a number of units depending upon your encoding scheme
• an output layer that will also depend upon your encoding scheme
• a hidden layer of some number of units, which you will decide on
• the state, or "copy back" units, with one copy back unit for each hidden layer unit.

Once you have designed your network you will need to use tlearn to create it. The following assumes that you have access to tlearn (see Resources below). Note that different implementations of tlearn have different menu structures so you will have to explore to become familiar with it.

Using the tlearn menu, create a new project and call it "wow". This will create a file called wow in your current directory: The .cf, .data, and .teach files are created and edited with a regular text editor. The project file wow is affected by the changes that you make in the options dialogues in xtlearn. The wow.cf file will be used to define the neural network. The file wow.data will contain the encoded input data and wow.teach will contain the encoded target patterns.

The specification in the wow.cf file is a little complicated due to the flexibility provided by tlearn. To simplify this process, a schematic configuration is provided below.

The first two lines of the wow.data and wow.teach files need to specify the format of the data. This can be done using the following:

distributed
35

This says that the input is a set of 35 vectors. The remaining 35 lines should be your encoded data - one line per month.

Ensure the project files are saved then use the tlearn menu "Training Options..." to access the training options dialogue. Select a suitable training rate and momentum and ensure "Train Sequentially" is selected. The "Training Sweeps" parameter sets the number of sweeps to perform for training, where processing one input pattern is a sweep (i.e. 35 sweeps to a epoch).

Use the tlearn menu to train the network. This will generate an error log file and one or more weights files. You can also watch the progress of the training by showing various windows. More details on error logging in the xtlearn tutorial. You have have to hand in an error log as part of the assignment submission: please don't submit a huge error log - just a reasonable sample of error - maybe 100-200 lines long. To do this, you need to adjust the logging interval - if you train your simulation for 35000 sweeps (say), then to get 200 lines of error log, you would need to have set the logging interval to 35000/200 = 175. It is a good idea to set the logging interval to a multiple of the number of training patterns. The marker has better things to do than scroll through 150,000 lines of error log, and may not feel friendly if you make him do so.

Note: xtlearn appends error values to an existing .err file. Thus you should remove/delete your .err file before your final run, so that the .err file that is created in your final run consists only of the error values from that final run. It's smart to check how many lines are in the file before you submit it, e.g. by doing wc -l *.err (in a Unix/Linux environment), or similar. In fact, you could actually look at what's in the .err file before you submit it.

When you are happy with the training, you can use the network to generate stock market predictions. To make predictions from the wow.data file select the menu option, "Verify network has learned". You can use the "Testing Options" dialogue to send the output to a file. The output must be translated using your decoding scheme to obtain the predicted stock values. How good are the predictions? Why do you think your network has obtained these results? What value does your network predict for the unknown 37th month? What trade-offs are being made by your design decisions?

It is likely that your results inspire changes in either the encoding scheme, network design or training regime. You should document the different approaches you take and include them in your assignment report. Evidence of intelligent experimentation will be rewarded.

Configuration Schema

The following is a schematic configuration file to implement an Elman network using tlearn. To form a complete network specification, replace the variables with actual numbers. (Note that the hyphens in the schema are not minus signs and need to be in the configuration file!)