Tensor Product Networks

COMP9444 Neural Networks

Tensor Product Networks

At http://www.cse.unsw.edu.au/~billw/cs9444/star.c you will find C code for the mother:baby::mare:what? proportional problem done with tensor product networks, as described in the lecture notes. This includes, obviously, code to do tensor product network learning and retrieval operations: teachTensor, validityScore, findPredicates, and findArg2s. You might find it useful to look at this code, if you are unsure about the algorithmic description in the lectures.

A solution is not appropriate for this exercise.
Using the vectors representing rabbit, mouse, carrot, and cat found in the section of the lecture notes headed "Learning with Hadamard Representations", compute the following outer products
1. carrot^T × mouse
  Answer: From the lecture notes, carrot is represented by ½×(1, 1, –1, –1), and mouse is represented by ½×(1, –1, 1, –1). So the outer product is 0.25 ×
  
  1 –1 1 –1
  
  1 –1 1 –1
  
  –1 1 –1 1
  
  –1 1 –1 1
2. rabbit^T × cat (think baby rabbit and biggish cat).
  Answer: From the lecture notes, rabbit is represented by ½×(1, 1, 1, 1), and cat is represented by ½×(1, –1, –1, 1). So the outer product is 0.25 ×
  
  1 –1 –1 1
  
  1 –1 –1 1
  
  1 –1 –1 1
  
  1 –1 –1 1
3. Add these two outer products together.
  Answer: ½ ×
  
  1 –1 0 0
  
  1 –1 0 0
  
  0 0 –1 1
  
  0 0 –1 1
4. What does the resulting tensor represent?
  Answer: It represents the relation {(carrot, mouse), (rabbit, cat)}. Since we were thinking of this as an is eaten by relation, you could say that it represents the relation in which mouse eats carrot and cat eats rabbit.
5. What is the rank of this tensor?
  Answer: 2
6. How many binding units does it have?
  Answer: 16 = 4 × 4

1	–1	1	–1
1	–1	1	–1
–1	1	–1	1
–1	1	–1	1

1	–1	–1	1
1	–1	–1	1
1	–1	–1	1
1	–1	–1	1

1	–1	0	0
1	–1	0	0
0	0	–1	1
0	0	–1	1