COMP9444 Neural Networks and Deep Learning
Quiz 6 (Word Vectors)
This is an optional quiz to test your understanding of
Word Vectors from Week 5.
- What are the potential benefits of continuous word representations
compared to synonyms or taxonomies?
- What is meant by the Singular Value Decomposition of a matrix X?
What are the special properties of the component matrices? What is the time
complexity for computing it?
- What cost function is used to train the word2vec skip-gram model?
(remember to define any symbols you use)
- Explain why full softmax may not be computationally feasible
for word-based language processing tasks.
- Write the formula for Hierarchical Softmax and explain
the meaning of all the symbols.
- Write the formula for Negative Sampling and explain
the meaning of all the symbols.
- From what probability distribution are the negative examples normally drawn?