I am not a big fan of IIS but if you want to serve Silverlight apps with IIS , you must add the appropriate MIME Types in server 2003 and vista,
* .xap application/x-silverlight-app
* .xaml application/xaml+xml
* .xbap application/x-ms-xbap
The Dude at this link created a VBS script to make the process easier.