#### COMP9444 Neural Networks and Deep Learning

### Quiz 6 (Word Vectors)

This is an optional quiz to test your understanding of
Word Vectors from Week 5.

- What are the potential benefits of continuous word representations
compared to synonyms or taxonomies?

- What is meant by the Singular Value Decomposition of a matrix X?
What are the special properties of the component matrices? What is the time
complexity for computing it?

- What cost function is used to train the word2vec skip-gram model?
(remember to define any symbols you use)

- Explain why full softmax may not be computationally feasible
for word-based language processing tasks.

- Write the formula for Hierarchical Softmax and explain
the meaning of all the symbols.

- Write the formula for Negative Sampling and explain
the meaning of all the symbols.

- From what probability distribution are the negative examples normally drawn?