There some great resources on the web to get us started with microcontroller audio.
This series of articles on Arduino audio (part 1, part 2, DAC options) gives a fantastic introduction to the theory side. There's tons of great stuff in these pages that I won't duplicate here.
Another project feeds audio samples into the microcontroller from a PC serial port. The microcontroller is like a simple PC sound card, it's not capable of independent operation. This project, and the great video, inspired me to design this digital audio recorder.
TI and ATMEL have application notes detailing designs for simple digital audio recorders. TI's design[pdf] records 12 seconds of low quality audio to the flash program memory of an MSP430. It uses a specialized chip to create the audio signal, complicating the design. A great TI app note on speech compression[pdf] also has some interesting support circuitry for digital audio. The ATMEL digital sound recorder[pdf] uses a small external memory chip to store a few seconds of audio.
Pulse-width modulated audio synthesis
The cheapest, easiest way to generate audio on a modern mirocontroller is to use a hardware pulse-width modulator (PWM). A PWM is a circuit that generates a repeating time period (called the period), and turns on a switch during a percentage of that time period (called the duty cycle). This happens so fast that only the average value of the on and off periods is measurable. Different audio tones can be generated by varying the duty cycle.