If you can manage the server yourself, you can’t get any cheaper than Digital Ocean or Linode. Configure nginx instead of Apache, setup a caching plugin like W3TC or WP-Rocket, and run it through Cloudflare and you’d be surprised how much a little 512mb instance can handle.
Then there are the “managed WordPress hosts” like WPEngine, SiteGround, Pagely, Pressable, and GoDaddy. [Before someone punches me in the face for recommending GoDaddy, this is specifically their managed WordPress hosting (they bought MediaTemple some time ago) and not their shared hosting.] They are a little pricier, but fully managed so there isn’t much to worry about. The pricing model for overages on some (all?) is based on “hits” so read the fine print.
A nice middle ground is Cloudways which charges a fee for managing your instances on Amazon AWS, Google Compute Engine, or Digital Ocean. They will one click setup your instance with security, LEMP + varnish + memcached, and WordPress (or Drupal, Laravel, etc).