Natural Language and Artificial Intelligence

We will give a first introduction to computer programming (in Scheme) with a focus on how computers understand (and misunderstand!) natural languages like English. The study of language offers a unique window into the mind, and we will explore how artificial intelligence relates to natural intelligence. Students will write programs to explore the meanings of individual words, phrases, and sentences. Students will learn how search engines work, how computers can translate between multiple languages, and identify authors. Students will encounter very strange English sentences, and learn what makes them confusing or humorous.

This survey class requires no previous programming experience, but strong algebra skills are required, and exposure to probability is a plus. We will use programming as our tool, and the focus will be on the study of human language. There is no required text. Students must have access to a computer on which they can install PLT Scheme, and you will need email and Web access. There will be homework of one to two hours per week.

The course is extremely ambitious, so it's for the most excited among you. It will be difficult for you if you miss a class.

Syllabus:

July 12 -- First Steps

We will introduce the concepts of syntax, semantics, and meaning, in both human languages and computer languages. We will take our first steps in writing a Scheme program, and start to think deeply about English.

• Computing: expressions, naming, environment, procedures, conditionals, and predicates

• Linguistics: nouns, context, verbs, conditionals, and adjectives

• Lab exercise: get set up with subversion and PLT; write "hello world" and a little more.

July 19 -- Collections

Computing is about processes, how to do things, and many processes involve collections of items rather than just one item. We will learn how to apply processes to collections.

• Computing: recursion and iteration, lists and dictionaries, map-reduce

• Linguistics: singulars, plurals, and coordination

• Lab exercise: We will write a small search engine for twitter.

July 26 -- Probability

The probability of an event is the frequency of that event divided by the total number of all relevant events; When two events do not depend on each other, then the probability of them both happening is the product of their individual probabilities. Different sounds have different probabilities in different languages -- can we tell just by listening what language we're listening to?

• Math: probability, conditional probability

• Linguistics: phonetics

• Lab exercise: Language identification based on frequencies of single letters

Aug 2 -- Translation

We will talk about the meanings of words, about words with multiple meanings, and how translating to another language can help us separate meanings from one another.

• Math: logarithms, mutual information

• Computing: floating point representation

• Linguistics: selectional restrictions and word senses

• Lab Exercise: We will improve an automatic Spanish-English translator.

Aug 9 -- Language Models

When we hear or read sentences, we don't get them all at once, we get them a word at a time, a sound at a time, a letter at a time. We can finish each other's sentences more often than not. Can a computer capture some of the regularity of language without really understanding any of it?

• Computing: nested data structures

• Linguistics: center embedding, garden path sentences, semantic roles

• Lab exercise: We will write a program to generate poetry.

Aug 16 -- Development

Small children, toddlers, at first say just one word, then two together, then short sentences, and then more and more complicated sentences. How old are they when they get to each stage? Do they use some kinds of words or structures before others?

• Computing: visualization

• Linguistics: syntax, x-bar theory

• Lab exercise: We will explore children's speech at various ages, and make graphs that show our findings.

Aug 23 -- Meanings as Programs

How can we translate between human languages and computer languages? What might the dictionary look like? We will look at one possible view in depth -- we will show a meaning for numbers, and for simple mathematical expressions (expressed in English), and show that a computer can meaningfully answer questions.

• Computing: procedures as return values, currying

• Linguistics: semantics

• Lab exercise: write meanings for simple time expressions that let you search Twitter within a particular time range.

Aug 30 -- Grounding

In order for computers to really understand anything of our language, they must share some of the same kinds of experiences. Computers are learning to see, to hear, to move around, and interact with the world around them, so there is hope. They do need us to teach them what things mean, though. What's so hard about this?

• Computing: automatic evaluation

• Linguistics: prepositions, especially spatial prepositions

• Lab exercise: annotate time and spatial expression examples