How does my DC and services I run there translate to the cloud?
A lot of companies are running their entire IT infrastructure still in physical datacenters and while in some scenarios there are a valid reasons to stay on earth, missing knowledge and uncertainties are often a blocker for an informed decision. A problem we as consultants often see when we talk to customer is that they aren’t aware of the full potential and different services the cloud can offer to them. The cloud is not just another platform to migrate your virtual machines from your VMWare cluster to, it’s a platform which enables you to simplify, speed up and transform your whole infrastructure.
The physical datacenter
Let’s take a look at a standard DC setup like you can see it often in the wild. We will begin with the general infrastructure,: we have a frontend, and a backend - each of them holding common infrastructure pieces like:
- Networking / Network Security: Router, Switches, Firewalls, Loadbalancer, etc.
- Compute: Physical Servers, VMWare Clusters, Kubenetes, etc.
- Storage: Server, SAN, NAS, etc.
- Databases: Postgres, MySql, etc.
We already have a lot of complex infrastructure components in this simple DC and if we want all of them to be fully redundant, HA and highly secure, we already have a lot of complexity to deal with. In fact you probably need a specialised person for each of those components and you probably want two for each component because if one is eaten by a shark, you have zero of them.
And this is just the infrastructure. You also have software running inside your VMare clusters. You run a lot of services on your virtual machines as well:
- Queues/Streaming: RabbitMQ, Kafka etc.
- Your actual money generating software etc.
And some of those components might need specialised people as well (two, remember the sharks). This is a lot of stuff you have to take into consideration in a standard dc, including a lot of people who can’t go to the beach because of the frequent shark attacks. How does all the complexity we have in our physical datacenter translate to a cloud environment in AWS?
The AWS Cloud
Now what happens then if we take all the componenents above and translate/transform them into a cloud environment. The first thing we have to clarify before answering this question is that a ‘lift and shift’ scenario is not the answer, it is acceptable for a transition period but it won’t make you happy in the long run for a lot of reasons especially for the operational costs.
So we are in the cloud now, what changes in my infrastructure. Let’s start with the fundament of every datecenter.
Networking and Security
In the cloud there is no physical room in which we put our switches and cable them so how do we do this? AWS is providing us with a construct called “Virtual Private Cloud” or for short VPC. This VPC is our new layer 2/3 construct in which we define Subnets, Routing, NATing as well as other objects relevant to Layer 2/3.
The VPC is also the place in which we define security and loadbalancing and instead of having highly complex appliances. The cloud provides us with virtual constructs like security groups (firewall on EC2/VM level) and loadbalancer objects, which can easily route huge amount of traffic to your endpoints. Load balancers in particular provide us with some great benefits like a huge amount of available metrics, automatic TLS certs and routing on hostname level.
And the icing on the cake here is that all of this constructs like security groups, loadbalacers and everything else provided by the cloud are automatically updated and security patched without your concern and without any downtime for your infrastructure. No longer will your teams come in on a sunday morning to patch critical infrastructure components in time slots with low traffic.
Computing and Storage
We already see that AWS provides us with a lot of help on the network layers, which simplifies a lot of our infrastructure compared to the physical DC. But what does it provide on the upper layers?
The most obvious feature here is of course EC2 instances (VMs) which you can deploy inside your VPC in a blink of an eye. The real power here is not only the the fast deployment, it’s the ability to scale the number of machines behind a load balancer according to metrics.
In a physcial DC you have a load balancer and a static number of VMs as endpoints. In the cloud you have a load balancer with the capability to automatically deploy as many EC2 instances as needed to manage incoming connections. This way you can automatically scale horizontally your application to your workload demands.
Another thing the cloud provides us with is a simplification of the storage area. Storage is a complicated topic, you need block storage (network storage or just low cost long term storage). You can implement this in a myriad of ways: directly attached, iSCSI, fibre channel or whatever. And it gets more complex and complicated if it’s done HA and secure.
In the cloud you get EBS volumes (direct attached storage) which you can connect to your EC2 instances, you have S3 as an endlessly scaling object storage, you have EFS as AWS implementation of NFS which is insanely fast and infinitely scalable as well. Again all of this storage systems are patched, maintained and secured by AWS so you don’t have to worry about them, you just have to use them.
Databases
At this point we have network, compute and storage resources – so we are missing only the last one of the big four: databases.
Databases are usually the most crucial part of any infrastructure as they hold your important application data. Databases use everything we have seen so far: they need network, storage and compute resources to do their work. If one of those doesn’t work your database won’t either. (Not) surprisingly AWS provides us with a managed solution for this as well.
Amazon Relational Database Services or in short RDS provides us a fully managed database solution for our needs. They provide this service for multiple common databases like PostgreSQL, MySQL, MariaDB and some others. After clicking a button (or better: Infrastructure as Code!) and some minutes of waiting, AWS has setup a fully functional database for us, reachable via a DNS name and takes control of the following things for us:
- Automatic backups at your discretion: daily, monthly or whatever you need
- Automatic export of db logs
- High Availability
- Version upgrades with little to zero downtime in a timeslot of your choice
You have still to define how much computational power and storage you need, but you don’t have to provide those things explicitly, as AWS will deploy those automatically for you. Now we have the biggest building blocks of our Infrastructure in place, but of course it doesn’t end here: There are tons of other things you need and most of them are also provided by AWS but we will just take a look at one of the two most common.
Streaming / Queues
Streaming systems like Kafka and queuing system like RabbitMQ are fairly common services these days, running in any kind of application setups. They usually run on dedicated machines in HA and are a way of decoupling your software components. They are a crucial part of your infrastructure, so (as basically evertyhing we talk about) you want them secure and HA and as always, this is not a trivial task.
AWS provides us with managed services for Active and RabbitMQ, as well their own service called Simple Queue Service (SQS, actually the first AWS service ever). As you can already guess at this point it is highly scalable, secure and there is nothing for you to deploy manually, so this area of infrastructure can be managed by AWS for you too.
Summary
We’ve seen that AWS is not just a place to move your VMs to, it’s a whole new way to see the datacenter. Instead of maintaining all of the infrastructure yourself, AWS does it for you and you can focus yourself on more important tasks. So is AWS or cloud in general the optimal solution for everyone? No, although considering the rising number of use cases it becomes steadily more relevant.
Most companies would certainly profit from the benefits the cloud provides. The ability to create managed, secure infrastructure in a fast way is a game changer for fast developing systems. Of course the complexity of a full blown datacenter does not simply disappear, it shifts partly into the cloud, but it’s greatly reduced. And if you need help for that journey, we are here.