Forge is quite capable for most tasks - we use it a lot at UserScape. For $10-15 a month you can't beat it - it's really an insanely low price.
As a bonus, the script it generates if you provision a custom VPS gives you an idea of how it sets everything up (there isn't much magic going on, which is nice too - KISS).
For non-forge servers, I create a set of Ansible roles to setup a server. Ansible seems to be the nicest configuration management tool for anyone who's familiar with running commands on a server over SSH (since it itself is ssh-based, with a lot of programming in place to check the state of things to only make the changes it needs to).
We use this extensively to automate HelpSpot trials.
I definitely, hardily, ravenously, recommend NOT setting up your own mail server. It's difficult, finding resource online are often confusing, old, or incomplete. Setting it up in a way that will improve your deliverability is a constant on-going battle (and that's just sending mail, there's also receiving spam). Making this harder, since all server providers have a pool of IP addresses the re-use, you never really know if an IP address has been used maliciously before.
Paying for an email service is absolutely worth it. Anything else is a complete waste of precious time.