Introduction to Natural Language Processing

Reference: Allen, chapter 2

Aim:
To review the grammar of English, introducing some terms for describing different types of English phrases, and the concept of a grammar rule. We also have a quick look at how the different levels of linguistic knowledge interact.
Keywords: abstract noun, active voice, ADJ, adjective, adjective phrase, ADJP, ADV, adverb, adverbial phrase, ADVP, agreement, apposition, article, aspect, AUX, auxiliary verb, BELIEVE, bitransitive, bound morpheme, cardinal, case, common noun, concrete noun, CONJ, conjunction, count noun, declarative, demonstrative, descriptive grammar, determiner, ellipsis, embedded sentence, features in NLP, first person, free morpheme, FUT, future perfect, gender, grammar, imperative, indicative, infinitive, inflection, intensifier, INTERJ, interjection, intransitive, lexeme, mass noun, morpheme, morphology, N, nominal, noun, noun modifier, noun phrase, NP, number (grammatical), object, ordinal, participle, particle, passive voice, PAST, past perfect, person, phone, phoneme, phonetics, phonology, phrasal verb, phrase, pluperfect, plural noun, possessive, PP, PP attachment, pragmatics, predicate, PREP, preposition, prepositional phrase, PRES, prescriptive grammar, present perfect, progressive, proper noun, proposition, qualifier, quantifier, quantifying determiner, relative clause, S, second person, sentence, simple future, simple past, simple present, singular noun, speech act, string, subject, subjunctive, syntax, tense, third person, transitive, V, verb, verb complement, verb group, verb phrase, VP, wh-question, word, y/n question


NLP Intro Plan

Plan:
  • overview of linguistics. Our focus: lexicon, morphology, syntax, semantics, reference We will only look at the simple basics of English syntax.
  • parts of speech and refinements (e.g. mass and count nouns)
  • phrase types (e.g. noun phrase and verb phrase)
  • inflection: -ing, -ed, -est
  • grammar rules: NP → DET ADJ N
  • tense, aspect, active/passive, transitivity
  • complement structure for verbs and adjectives
* note: in past versions of these notes, ART (article) has been used instead of DET (determiner). If any instances of ART have been missed in updating the notes, treat them as though they were DET for the purposes of this course.

Topics in NLP

 Introduction
  • applications
  • linguistics overview
  • ambiguity
  • example systems
 Syntax
  • linguistic knowledge
  • grammars and parsing
 Semantics
  • logical form
  • semantic interpretation
 Pragmatics
  • reference
  • discourse processing

Typical Applications


Related Disciplines

Linguistics - study of language and of languages
Psycholinguistics - language and the mind, models of human language processing
Neurolinguistics - neural-level models of language processing
Logic - an unambiguous formal language useful for representing (unambiguous) meanings

Map of the Discipline of Linguistics

SR = Speech Recognition; NLP = Natural Language Processing

Map of the discipline of linguistics

Australian-English Phonemes


Major Issue - Ambiguity

Structural or Syntactic Ambiguity
picture of sentence *man on hill saw boy with telescope*

The man saw the boy on the hill with the telescope.


Word-Sense Ambiguity

dog:

  1. domesticated carnivore Canis familiaris
  2. animals related to 1.
  3. the male of such an animal (as opposed to bitch)
  4. various unrelated but vaguely similar animals (e.g. prairie dog)
  5. a despicable fellow
  6. a fellow in general (you old dog, you)
  7. a mechanical device for gripping or holding (dog the hatches)
  8. an andiron
and more.


Referential Ambiguity

Steven hated Paul. He hit him.

he = Steven, or he = Paul ?


Eliza/Doctor (1966)

Men are all alike.
In what way?

They're always bugging us about something.
Can you think of a specific example?

Well, my boyfriend made me come here.
Your boyfriend made you come here?

He says I'm depressed much of the time.
I am sorry to hear you are depressed.

It's true - I am unhappy.
Do you think coming here will help you not to be unhappy?

I need some help.

...

Earlier you said you were unhappy.


Eliza 2

Eliza ...

My favourite exchange from Eliza:

...
My sister is going out with a fish.
Are any other members of your family going out with a fish?

Syntax

Reference:Allen, Chapter 2
Aim: To review linguistic knowledge, introducing some terms for describing different types of English phrases, and the concept of a grammar rule.
Plan:
  • words; types of words
  • phrase types
  • grammar rules: NP → DET ADJ NOUN
  • inflection: -ing, -ed, -est
  • tense, aspect, active/passive, transitivity
  • complement structure for verbs and adjectives

Words and Word Types


Subclassification

* NB: this is bad karma, and the staff of COMP9414 will not be held responsible if you take such a course of action


Phrase Types


Grammar


Sentence Forms

declarative (indicative)John is listening
yes/no question (interrogative)Is John listening?
wh-question (interrogative)When is John listening?
imperativeListen, John!
subjunctiveIf John were listening, he might hear something to his advantage

The subjunctive mood often describes a counter-factual situation - that is, it describes a situation that is not a fact - in our example of the subjunctive form, John is not listening.


Noun Phrases (NPs)


Grammar Rules


Grammar Rules 2


Verb Groups (VG)


Inflections, Tense

The head verb is (may be) inflected:

INFPRES
PART
PRES
TENSE
PAST
TENSE
PAST
PART
 
eateatingeatsateeatenIRREGULAR (STRONG) much change
setsettingsetssetsetIRREGULAR (STRONG) little change
--cancould-IRREGULAR (MISSING FORMS)
bebeingam/is/arewas/werebeenIRREGULAR (VERB "to be")
killkillingkillskilledkilledREGULAR (WEAK)

INF = INFINITIVE
PRES = PRESENT
PART = PARTICIPLE

Auxiliaries, Modals, Tense, Aspect


Passive Voice


Particles, Negation, Adverbial Phrases


Agreement


Embedded Sentences (S)

John's giving up the game was cowardly

The man who gave Paul the money was crazy

The money that was given to Paul was lost or
The money given to Paul was lost

Complements

Benedict believes he is the Pope

Margaret wants to own a fire-engine red Porsche

Blake promised that he would never steal a bear again


Adjectives (ADJ) & Adjective Phrases (ADJP)


Conjunctions (CONJ)


Lexical and Phrasal Categories


Summary: Outline of English Syntax
While reviewing English syntax, we have introduced a number of terms and symbols for describing types of words and phrases in English, including the lexical categories N, V, ADJ, ADV, CONJ, INTERJ, and PREP, and phrasal categories NP, VP, PP, ADVP, ADJP, VG, and S. In passing, we also introduced the concept of grammar rules such as
NP → DET NOUN

CRICOS Provider Code No. 00098G
Last updated:
Copyright © Bill Wilson, 2007, except where another source is acknowledged.