My experience is when you choose a magento hosting provider, they must have:
- Magento hosting package configuration: what they invest to configure server for magento only to get the best performance
- Magento support: of course very neccessary since almost customers will need help
- Security: they must take care the server security for us
- Speed and location: they should have a better caching system more than file caching system (slowest caching) and their server near your customers as much as possible.
You can also check this one Mageperfect server hosting.