Mac Mini lacking 5.1 audio output

[adam h]

Hello!
I'm having a problem; I've tested DVDs through the OS X DVD player, and the 5.1 comes through fine, so I'm sure everything's set up properly regarding cables.

In XBMC I've set the audio hardware settings to digital output ... anything obvious I could be missing? I'm kind of a n00b here.

Thanks in advance!

EDIT:
Movies compressed with latest version of Handbrake, with the default appleTV codec, which I double checked was AAC + AC3.

[saltyzoo]

Make sure you have the AC3 track selected. I believe Handbrake 0.9.2 put the AAC track first which means you have to switch to track 2 to get AC3.