COMP9444 Neural Networks and Deep Learning
Term 3, 2019

Exercises 5: Language Processing

1. Consider the sentence

   "two flowers grew tall on two tall towers"

   1. Write the co-occurrence matrix X for this sentence, using a 4-word context window (i.e. two context words on either side of the central word)
   2. Try to find a software package (e.g. Octave, Matlab) with a built-in command for computing the singular value decomposition. Use it to compute the singular value decompositon of this matrix X = USV^T
   3. Extract a word representation from the first two columns of U and plot the words on a 2-dimensional graph.