# Learning Group - 19 Jun., 2017

Chaoran Huang, UNSW Sydney.

# Tensor Decomposition

Presenter: Chaoran Huang
chaoran.huang@unsw.edu.au

## What is a tensor?

An $n$th-rank tensor in $m$-dimensional space is a mathematical object
that has $n$ indices and $m^n$ components and obeys certain transformation
rules.

Each index of a tensor ranges over the number of dimensions of space.

## Basic Operations

##### For $\mathbf{A} \in \mathbb{R}^{I\times{J}}$ and $\mathbf{B} \in \mathbb{R}^{K\times{L}}$

Kronecker Product

##### $\mathbf{A}\otimes\mathbf{B} = \begin{bmatrix} \mathbf{a}_{11}\mathbf{B} & \mathbf{a}_{12}\mathbf{B} & \cdots & \mathbf{a}_{1J}\mathbf{B}\\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{a}_{I1}\mathbf{B} & \mathbf{a}_{I2}\mathbf{B} & \cdots & \mathbf{a}_{IJ}\mathbf{B} \end{bmatrix}$

Khatri-Rao Product(a.k.a matching columnwise Kronecker Product)

## Basic Operations

##### Aussuming $I = K$, $J = L$ $\mathbf{A}\circledast\mathbf{B} = \begin{bmatrix} \mathbf{a}_{11}\mathbf{b}_{11} & \mathbf{a}_{12}\mathbf{b}_{12} & \cdots & \mathbf{a}_{1J}\mathbf{b}_{1J}\\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{a}_{I1}\mathbf{b}_{I1} & \mathbf{a}_{I2}\mathbf{b}_{I2} & \cdots & \mathbf{a}_{IJ}\mathbf{b}_{IJ}\\ \end{bmatrix}$

Interesting Property

## Matricization

Fibers and Slices

## CP Decomposition

Review of Matrix Factoriztion

## CP Decomposition

Different names of CANDECOMP/ PARAFAC Decomposition

h

## CP Decomposition

Why the sudden fascination with tensors?

• Offer more natural representations of data
•   - Videos;
- Complex subject-items relationships;
- ...

• Boosting in computing power recently
•   - Google Tensor Processing Unit (TPU)/ 28-40 W @ 45 TFlops
- NVIDIA® Tesla® V100/ 300W @ 120 TFlops

h

## CP Decomposition

The modal unfoldings

h

## CP Decomposition

The modal unfoldings(continue)

h

## CP Decomposition

Computing the CP Decomposition(Alternating Least Square)

## One Application Case - C.Huang's Tensor

Tensor Factorization based Expert Recommendation

## Weekly Report

• Draft of the tensor paper
• Prepared text corpus
•   - 98 GiB plain text...

• Training language model
•   - almost finished
- est. 3200 CPU hrs/40 threads