This is from vague memory as I don't have all the code to hand, but you can create a custom Message (inherit for the class "Message") type to return in your REST services and override certain methods to create custom responses.
protected override void OnWriteMessage(XmlDictionaryWriter writer)
{
...
}
protected override void OnWriteStartEnvelope(XmlDictionaryWriter writer)
{
...
}
protected override void OnWriteStartBody(XmlDictionaryWriter writer)
{
...
}
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
...
}
Not a complete answer, but might push you down the right path.