I can hardly believe it, but it's been almost three years since I wrote the original VPS! Getting Drupal up and running on a linode article. These days, I'm still with http://linode.com, where a lot of my clients end up hosting their web app sites; and I'm still installing everything by hand, which is still the best way in all but a few cases, since it gives me what I came to a VPS for in the first place: control over what's going on, and avoiding locking myself into immediately obsolete schemes unless I choose to do so. Of course, there are some excellent install scripts, which Linode calls Stack Scripts, with many of them dedicated to getting Drupal up and running in a variety of excellent manners, even including Pantheon Mercury and Aegir turnkey "single click" scripts. Certainly a great way to go in certain alternatives.
[Be sure to see the more recent update to this article,
VPS! Getting Drupal up and running on a linode (revisited)]
Well, after realizing the limitations of shared hosting for Drupal development, I decided to go with the big boys and use a dedicated server or VPS solution, at least for development. So I can make a multisite install for the docs and I can make subdomains for each development site.
While I was waiting for my account to be ready (after all today is Thanksgiving: but it still took no more than 40 minutes), I mosied over to ZoneEdit (see excellent tutorial reference below) and signed up for DNS service and domain management (free for the first 5 domains that you add to your account).