• objects   (e.g. Kate, Arnie, soccer ball, penalty area, ...)

• relationships between objects   (e.g. Arnie and door)

• objects move   (so relationships change over time)

Specific goal of this work:

Develop a model for describing spatiotemporal relationships among moving objects in video, animation, etc. that is useful as a basis for querying.

Needs to be: comprehensive, concise, efficient

Does not need to be: 100% accurate

We assume: objects have been identified and segmented in each frame