A Hidden Markov Model has several components. It has a set of states, a set of output symbols, a set of transitions which have associated with them a probability and an output symbol, and a starting state. When a transition is taken, it produces an output symbol. The complicating factor is that the output symbol given is not necessarily unique to that transition, and thus it is difficult to determine which transition was the one actually taken -- and this is why they are termed ``hidden''.
The way that HMM's are used is the opposite of producing output symbols. People who use HMM's extract meaning by answering the question: ``Right. I have a set of output symbols. What was the sequence of states and transitions that resulted in those output symbols?'' The output symbols might be something like words, and from them we might be trying to extract some meaning, such as the type of word (noun or verb).
Note that this is not deterministic, and since the model is hidden, there may in fact be more than one sequence of transitions that resulted in that sequence of symbols. Thus the most likely sequence of transitions is searched for, using an algorithm known as the Vitterbi algorithm.
To build an HMM that can actually be taught, the person typically decides on a set of states that he or she feels are important, and a set of all possible transitions (although it is possible to create a complete set of transitions and let the algorithms decide on the ones with transition probability of 0). The transitions are assigned random probabilities.
Then, using algorithms such as the Baum-Welch algorithm, it is possible to train the HMM by adjusting the weights of the transitions to better model the relationship of the actual training samples.
Although the HMM technique is different to neural nets, it has many of the same problems, including deciding what the states are. Again, the decision of what the states and transitions are is a black art.