COMP9414/9814 Artificial Intelligence

Feedback Quiz on Machine Learning - Solutions

It is best not read the answers until you've tried to answer the questions yourself.

1. What do dendrites, axon tree, and synapses, in a biological neuron, correspond to in the artificial neuron model described in lectures?

   Answer:

   dendrite	input to artifical neuron
   axon tree	output from artifical neuron
   synapses	weights on connections between artificial neurons

2. Why is a non-linearity used in the artificial neuron model described in lectures? What are the important features of a suitable non-linearity?

   Answer: Without a non-linearity, the system would only ever be able to compute linear functions. In addition, in some cases activations could become extremely large as learning progressed.
   Let's call the non-linearity function σ(x).
   Important features:
   lim_x→∞σ(x) = 1
   lim_x→–∞σ(x) = –1 (or maybe 0)
   σ(x) should be differentiable.
   

3. What is the weight change equation used by the error backpropagation learning algorithm, and what do the symbols in this equation signify?

   Answer: Δw_ji = ηδ_jy_i
   where
   w_ji is the weight from node i to node j,
   η is the learning rate parameter,
   δ_j is the local gradient, and
   y_i is the input signal to node j.

4. What happens in the forward pass in error backpropagation learning? What happens in the backward pass in error backpropagation learning?

   Answer: In the forward pass, the node activations for the current inputs are computed, and hence the errors, i.e. the differences between the target outputs and the actual outputs. In the backward pass, these error signals are used to assign credit and/or blame and so to modify the weights so that the error will be less next time around.

5. What does over-fitting mean in the context of error backpropagation learning?

   Answer: Over-fitting refers to the situation where a learning system has learned the idiosyncrasies of a particular set of training data too well, so that it is unable to satisfactorily compute output for unseen data.

6. What is a decision tree?

   Answer: A decision tree is a tree each branch node of which corresponds to a question to be asked about a case being classified, and each possible answer corresponds to a subtree of that node. The leaf nodes correspond to completed classifications - in other words, they are labelled with the classification.

7. Give the formula for the entropy measure used in lectures.

   Answer: –Σp_ilog_ep_i

8. Briefly, in words, describe how ID3 chooses the best attribute to split on.

   Answer: It computes the information gain for splitting on each available attribute, and then it splits on the attribute giving the greatest information gain.

9. Give two reasons why there might be "noise" in training data.

   Answer: Error in measurement of the attributes of an instance, and error in data entry.

10. What is the formula for the Laplace error estimate?

    Answer: E(S) = (N – n + k – 1) / (N + k)
    where
    S is the set of examples in a node,
    k is the number of classes,
    N is the number of examples in S,
    C is the majority class in S, and
    n out of N examples in S belong to C.

Bill Wilson's contact info

UNSW's CRICOS Provider No. is 00098G