At first, I didn't understand what exactly EC2 was. The Amazon promotional material could be a little more clear in this area. It's a virtual machine (what I refer to as a slice, Amazon calls it an instance) that is billed out by the hour (real world time, not CPU-used time). It has an impressive set of client command-line tools that let you control your slices from any Windows or Linux box on the Internet. Everything uses public key cryptography.
To get started, Amazon has an online forum that offers free, pre-configured instances. They have all of the Linux flavors, Solaris, OpenBSD, etc. I found a slimmed down version of Gentoo enabled with PHP and Apache2 and was up and running in under 5 minutes.
Another cool concept that Amazon has is the ability to purchase IP addresses and them route them to any machine on the EC2 network. Amazon calls this concept "elastic IP addresses". All of your instances run with private IP addresses on the Amazon network. For them to be accessible to the outside world, you use the client tools to connect your public IP address with the private one. One of the obvious advantages of doing this is that it makes it extremely easy to re-route your traffic to a different instance any time you want with zero DNS downtime.
Getting back to instances, they are ephemeral in the sense that when you shut them down, they are gone forever. I figured this out the hard way when I configured an instance to serve up a custom web page, shut it down, and then started it back up again and discovered that my changes were gone. If you want to preserve any changes you made to the pre-configured image, you need to store them on Amazon S3. Luckily, Amazon provides tools to make this very easy to do.
In addition to getting up and running very quickly with minimal capital, AWS provides essentially unlimited scaling. Read this blog post about Animoto's launch of their Facebook app for a case study. SmugMug is another good case study.
One interesting facet of Amazon's web services offerings is the ecosystem that's starting to build up around them. Look at RightScale as one example of a company offering value-added services to AWS.
All in all, a very impressive offering: compute, storage, queue, database, and (with third-party help) scaling. Everything you need to launch whatever you want to.
But you do pay for everything: Instance time, data transferred in/out of AWS, storage. I estimate the minimum monthly cost of a basic instance at around $100. Still dirt cheap compared to buying your own equipment for colo, but not as cheap as Slicehost or some of the other VPS vendors (around $20 including quite a bit of bandwidth and a fair amount of storage).
Next up on my to-investigate list: Google App Engine.