Amazon's Secret Plan to Take Over the Internet

Ok. So here is the question. Where can you buy books, music, movies and the infrastructure to build the next MySpace, Flickr or YouTube?

Amazon.com has the worlds biggest retail site. That takes quite a bit of infrastructure to run a site like that…right? But an infrastructure like that is also a valuable commodity if you can monetize it…Amazon seems to be trying to do just that. They want to provide the infrastructure that will power the next wave of web sites. The amazing part of the Amazon Web Services offering is that there are no setup charges or other up-front costs required to setup a world-class scalable web application. If you can imagine it, you can build it…on the Amazon framework.

So here is what they are actually offering and what it means.

Amazon’s Simple Storage Service (S3)

This is the gem of Amazon’s web service offerings. Unlimited (theoretically) storage on tap. While there were some hickups in the beginning and there are some limitations (5GB maximum object size), S3 provides scalable, highly available, secure persistent storage. I looked at this when it was first offered and while it was a beautiful service, it was hard to roll into certain types of products since control over the “secret” key needed to store objects was critical to security. It required certain back-flips to use a client installed package that could store objects to a central storage facility and pushing the data through a 3rd party web-host effectively limited the benefits of the famous S3 scalability.

Amazon’s Elastic Computing Cloud (EC2)

This is so cool it’s hard to fully describe. Think virtualization on steroids. Unlimited computing power on tap. No setup fee and a fee of 10 cents per processor hour and you’ve got something very interesting. It is a very technical service requiring quite a bit of knowledge and ability to setup but this just creates another market for technical shops to be able to offer support services for EC2 and the other services.

Amazon’s Simple DB (no abbreviation here)

Ok. By this time, if you have the next killer application in mind and all you need is some VC money to make it happen, you are starting to get excited. You are starting to see a strategy that will allow you to boot-strap your business and your application without shelling out the big bucks for the rack servers and the data center. Amazon’s SimpleDB…one of the key elements that can pull it all together is exactly what it’s name says it is. A simple database. But…a simple database that scales to extremely large numbers of connections, large numbers of tables and massive record counts. If you don’t understand what that means for your application just think of it like this. You could build…well, the next Amazon.com on this platform.

Let’s see…store the content on S3, put the data in SimpleDB and analyze the data for recommendations and other batch jobs using EC2. Wait a minute…this is really getting good.

Now, remember that I mentioned there was a problem with S3 with the secret key?

The problem was partly a security problem and partly a billing problem. S3 is billed on transfer and storage space. Not very much, but as you pack on the gigabytes, the pennies add up. So you either have to over bill to give you some room for customers who use more resources and make your profit on the backs of the customers who don’t abuse the service, or you have to come up with some way of tracking usage. This is a tough thing to do and still retain scalability. Also, there is still the security issue. You have to really work to protect the secret key used to store your data. But…Amazon, either realized this and rushed filled the gap or always understood this and just didn’t release the solution until recently.

Amazon’s DevPay (ok..so no catchy abbreviations but who cares…I don’t)

DevPay is a commerce layer on top of S3 and EC2. Takes care of the security problem. Takes care of the billing problem. The only minor drawback is that the customer is a shared customer with Amazon even though you provide the application. For most applications this won’t be a problem and if you built your app on S3, EC2 and SimpleDB anyway, well…you are already in bed with Amazon so this is not a tough pill to swallow.

The Other Stuff

Ok…there are other web services as well. There is SQS (Simple Queue Service), the Mechanical Turk (this one is really strange…you pay pennies for human power to perform repeatable, tedious tasks), the Flexible Payment Service (another commerce model that allows you to charge for services and goods using Amazon’s commerce backend) and of course, Amazon’s affiliate web services.

The thing about all this is that its not obvious. These web services have been rolled out quietly and the average person has no clue what this actually means. But there are quite a few businesses that are not only successfully using these services to launch their businesses, they are doing it with much larger profit margins and significantly less capital drain as a result.

They keep up like this and one day, there will be a handful of major players (especially if Microsoft succeeds with their hostile takeover of Yahoo) and a large number of medium and small web application/service providers with a large number of them using Amazon’s platform.
Ok…now for my psychic act.

If Amazon does this well and proves the market (I have to admit, they seem to be doing it), then Google will wake up and do the same thing and become the second major virtual application platform. Microsoft will realize that they missed the boat and quickly build their own boat on a .NET based platform and jump in.

The result will be good for all of us. New more powerful, more scalable applications running on AmazonSpace, GoogleSpace or .NETSpace.

What do you guys think?