Hilarious fail - it seems that +Microsoft Azure validates the CNAME for a domain case-sensitively...
The error page that tells me they found a CNAME of myproject.azurewebsites.net but I need to change that into a CNAME to MyProject.azurewebsites.net. Oh dear.