Kubernetes becomes the universal orchestrator and it’s good news

With Rancher announcing their next major version will be designed on top of Kubernetes, I think it is fair to say, Kubernetes has become the ubiquitous orchestration API. Most major PaaS players have announced they will at least support it as first-class citizen or move entirely to its API. Indeed, Mesosphere announced they would integrate with Kubernetes to orchestrate resources. Weave announced features targetting Kubernetes users in the weave cloud product a few months ago. VMWare integrates Kubernetes in its Photon controllerOpenShift was built on top of Kubernetes early on. More broadly, Pivotal also recently announced its PKS solution, adding Kubernetes to its product offering, in sync with the Google Container Engine API.

Docker has not thrown the towel yet of course and it’s good to hear they deliver new features continuously. However, the fact most players gradually adopt the Kubernetes API isolates them a bit. Whether they’ll decide to integrate with the Kubernetes API down the line remains to be seen.

Is it good news to have a single dominating player though? I prefer avoiding monopoly situations leading to lock-in. But I don’t think that way towards Kubernetes for two reasons. First, Kubernetes has had a consistent history of openness. Second, the project is part of the Cloud Native Computing Foundation, which I believe is strong on an open ecosystem. I doubt the CNCF would tolerate a rogue project.

The fight for containers orchestration is mostly over, competition will continue but I think the container ecosystem is now moving on to building on top of the Kubernetes API. This means we should start seeing a fight at a higher level, the one about running your microservices for instance.

One aspect we should all look forward to is that the Kubernetes is engaged in breaking down its API monolith and that is for the best .

Where does this leave you as a user? Well, I’d say, do not worry too much about managing your orchestrator anymore, Kubernetes as a service is here now, you can safely fully focus on your application! That’s a good news.

Chaos is the new order

Let it be said: «Chaos is the new order».

I have been working with microservices for the past few years and it’s been quite a ride. By breaking down the monolith into more focused independent entities, questions have presented new challenges. Obviously, from a software architecture perspective, but also operational and organizational.

Indeed, in a monolith, developers make a lot of the calls that can impact even operations but leave the ops side with little knobs to tweak. One of the first consequence of the breakdown is that operations should be much more involved in the design choices of the application. A sane DevOps culture might even emerge.

With that said, running a healthy system made of unpredictable parts is challenging, there is no way around it. All teams that have transitioned to microservices architecture have had to learn, adapt and become creative to face those challenges: service discovery, faulty services, stateless services, network latency, authentication across the system and many more.

The ecosystem is now ripe for running those systems with confidence and speed. I will now be working full steam on providing content towards running microservices hands on.

Running is one thing, as my friend Russ Miles often says, we must stress the system as it’s a living thing. Software do not live in a vacuum. Things go bad and we need to learn how to cope and respond fast with confidence. Russ and I will therefore talk more about how chaos experiments will make your teams stronger and faster at delivering great software.

Stay tuned because this is going to be fun as I will speak about Kubernetes, service mesh, logging, monitoring, storage and more! Embrace chaos as it is going the new order.

Let’s talk about microservices

Well well… let’s talk about microservices. I have been keen on following what was going on with microservices since its inception but I have never been its most vocal champion. I don’t believe in screaming microservices like a fanboy. It is mostly counter-productive.

With that being said, I see more and more people talking about them, but in many instances, it is approached like many other technologies or ideas in the past: “Oh look, it’s trendy, I must be doing it!” or “I am your manager, other managers tell me we should do microservices. So team, let’s do it!”.

This post will try to clarify a few things about why microservices may help you and some guidelines about building them.

Why do we need microservices in the first place?

Why indeed? Let’s be clear, microservices are not magical entities. If you believe switching to a microservices architecture will solve your problems, well, 99% of the time, you will be proved wrong. Let’s make it clear:

All or other things being equal, microservices will not benefit your project.

This does not start well, does it? Still, it is critical that we are honest about it.

Maybe it is the wrong question. A better one should be: “Why does our software fail to keep up with the business’s needs?

Indeed, this is real issue here. The feeling that your software eventually hinders your capacity to mov the business[1] forward. It’s a shame because it eventually ends up with some sort of bitterness and grief from within the company.

Obviously, hearing that microservices are small components that make you faster is attractive, but they can only help you if you understand that what truly matters is the shift towards designing for change.

In other words, we need microservices because we want to adapt our software more frequently to keep up with inevitable changes and the diruptive aspect of innovation. Inevitable doesn’t mean we should be defeated, quite the contrary! Indeed, the more we welcome and design for change, the more confident we will be in releasing fast and lean software that respond appropriately to its expectations.

What is a microservice by the way?

Russ Miles often plays the game of finding the wackiest definition of microservices, this one by Micah Blalock is certainly worth a read. Truth be said though, there is no strict definition of microservices and it’s a freaking good thing. The minute you have an industry-approved definition, you get ready for a feast much like the industry had with SOAP for instance.

Still, we can suggest some features that apply well to microservices:

They should strive at being simple and comprehensible. You can’t change something you don’t understand.

They are single-purpose components. In other words, they are small not because you count the number of lines but because they do one thing only. Usually, that one thing means your code-base for one microservice will remain small.

They should be treated like cattle not pets. Do not care for your microservices instances, care for your system’s general status. Kill and replace microservices that are unhealthy. This is quite a change when you come from a monolith application because you tend to care for its status much more.

A related one is that microservices ought to be independant. This will make it so much simpler to comprehend how to evolve your system.

A second related principle, microservices sould not share states. Decoupling your code while keeping coupling at the data level is a recipe for failure.

They should be keen on following Postel’s law: “Be liberal in what you accept but conservative in what you send”. The idea here is to play nicely with the change your system will constantly endure while reducing the coupling imposed by one microservice onto others.

They ought to be easy to roll out. Let’s recall that we want to keep up with fast pace of changes. We can’t achieve this if the release cycle is slow and complicated.

Honestly, there isn’t much more to it.

It is more the way they allow for an antifragile system to be built rather than how they are individually implemented. In other words, don’t over-think your microservices. Just make sure they help you design for change in a fast and confident manner.

 

How to build microservices?

At this stage, you should realise I do not believe there is one true way to implement microservices. However, there are definitely tools and approaches that will help along the way. I will enumerate a few of them, but please explore the field for what works for you. You would also do worse than reading Sam Newman‘s book “Building Microservices“.

First, you need to decide what are your microservices. Russ Miles describes a rather useful mechanism to scope your microservices using the Life Preserver diagram tool. You may even design around the CQRS pattern to further refine your microservices’ responsibilities.

Then, make a decision regarding the integration’s style you’re going to apply to your system. You don’t have to pick only one. Most of the time, you’ll hear about REST (or sort-of anyway) but you could also bet on a message oriented approach. If you use REST, you might want to use StopLight.io to design your API and even export it following the Swagger 2 format.

Recall that you never want to share state accross your microservices, at least not through a single common database for instance. What matters are the events flowing through. You may therefore rely on event-sourcing perhaps and reconstruct local states rather than care for consistency that is based on strong coupling. Finding the best event-store for your requirements and constraints may require some testing still but Kafka has gained some points.

Microservices are polyglot by nature. There is no need for you to keep implementing them using one single language. As per usual, chose the right tool for the job.

Please, do make sure you test your microservices.

Probably package up your microservices into portable images. This will reduce the complexity of dependency management and make it much simpler to distributed. You will likely benefit from Docker or Rocket.

Obviously, if you package up your microservices with Docker/Rocket, you will gain from managing their high-demand life-cycle through a service orchestrator. The field is quite busy but the major players are Google Kubernetes, Mesos/Marathon, Docker Swarm, Amazon ECS, CoreOS Fleet or Rancher. Really, this is worth taking the time setting a proper test for them because their documentation will not be enough to distinguish them.

You will then need a configuration service. There are various possibilities here but you may rely on a distributed filesystem or services like Consul or Kinto.

Make sure your microservices can be discovered! Rely on a service discovery tool such as Consul or etcd.

Ensure your microservices play nicely with external loggers like LogStash as well as monitoring tools like DataDog or Prometheus. Honestly, in both areas, there are tons of products so make time to review some of them.

You will likely want to set some sort of proxy before your services (to control the rate or secure the requests for instance or load-balance between microservice instances) using something like Kong or HAProxy.

From a security perspective, consider general best practices of web applications and make sure your team ponders on the right questions.

There are many related topics obviously but I will conclude with two critical aspects of building microservices properly:

  • As a developer, you own your microservices. This means that you can’t stop caring for them once you have pushed the code out. Your job is to follow-up to production. Talk to your ops. Actually, just don’t talk to them, make sure you work *with* them!
  • Stress your system! Remember you are building an antifragile system. You want to be confident you can test the plastic nature of your live system. Stress it!

That’s a lot to think about! Is there a simpler way?

Everything I have listed before should be on your shopping list when you build microservices. If you feel like it’s quite overwhelming, you will be happy to hear that platforms start emerging to take care of the entire operational aspect. For instance, AWS Lambda, Google Functions and (as it appears) the newest player in the field: Atomist. Make sure to review them.

Happy coding!

[1] Note that I am using “business” in a very broad sense here, don’t get hang on it too much.

dalamb – An AWS Lambda clone (sort of)

I was going through the AWS Lambda main presentation page yesterday and I started to wonder if it would be possible to create your own stack with the same featureset.

First off, AWS Lambda is a stunning programming model. A cross between stateless distributed functional programming at scale (say that fast ten times if you dare) coupled with zero-administration requirements. In other words, you write a function and AWS Lambda runs it at the appropriate scale for you.

What’s more is that, AWS Lambda can react to events following the event sourcing pattern. So you don’t keep any internal state, AWS Lambda executes your function with the event data.

Our job, as software developer, has just become almost too simple. The good news is that we can now focus on the value we are aiming for rather than the plumbing and machinery underneath. A whole new world.

With that said, AWS Lambda is brilliant but, well, is deeply tied to the AWS services and infrastructure. That’s sort of the point obviously, however, could we create a similar stack and run it ourselves? Because, why not!

Let’s therefore introduce “dalamb – An AWS Lambda clone (sort of)“.

dalamb properties and features

What would be dalamb properties? How could we design such a beast? Interestingly, we have most of the pieces at hand, at least from a technological and architecture perspective.

Let’s rewind a bit, AWS Lambda gives you the main following features:

  • extend AWS services with custom logic
  • perform funky operations triggered by AWS events
  • expose your function as a web service (insert mention to RESTful URL here)

All of this is backed by the powerful AWS infrastructure ensuring:

  • fault-tolerance
  • automatic scalability
  • a rich security model
  • pricing with precision

These features and properties tell us the stories dalamb should pursue:

  • The event-sourcing story: how to map function to events
    • The REST story: this one is just a specific case of the event sourcing story where the events are triggered by requests to a given endpoint
  • The delivery story: can we package up any function in any language?
  • The operational story: deploy, update, scale, keep available…

It’s all about events

The heart of the AWS Lambda platform is its event-sourcing feature. Events are everywhere, they actually define the system in itself.

As developers, what we build, when we forget about them, are fragile stateful monoliths that quickly fail under their own mass and incapacity to accept inevitable changes. Building discrete functions that react to events mean you reduce friction between changes and pressures on your system while, as a developer, keep a comprehensible view of the system.

Events have basic properties:

  • self-contained: the information enclosed within must not depend on moving external resources
  • immutable: events never change once they have been emitted
  • descriptive: events do not command, they describe facts
  • sequenced: event streams ordering must be supported
  • structured: so that rules and patterns can be matched against them

dalamb is not a re-write of the entire set of AWS services. It focuses on mapping discrete, simple, functions to events flowing through the system. For this reason, dalamb will be designed around event-sourcing patterns.

The goal will be to make it simple for external resources to send events and straightforward for functions to be matched to these events following simple rules.

dalamb will likely start using Kafka as a simple event-store backend.

Change is the nature of any live system

A live system is constantly changing due to internal and external stressors. Your functions are one of those stressors. In order for the system not to collapse onto itself, a certain level of automatic elasticity and draining are required.

Draining means that part of the system can be considered like wasteful and the system will find a way to clean them up. Elasticity means that the system must be able to absorb any load stress in a way that keeps the overall system functional.

AWS Lambda owns the responsibility of managing the system’s constant changing nature. So will dalamb. However, since dalamb will not own the infrastructure itself, it will be bound, initially at least, by the fixed size of the underlying infrastructure. However, within those boundaries, dalamb will ensure the system keeps its promises and stays live and well.

dalamb will likely take advantage of Mesos for resources sharing and Kubernetes or Marathon for manging the lifecycle of each function instances.

Geared towards developers

dalamb is meaningless if developers cannot express themselves though it. To achieve this, dalamb will not make any assumption regarding the actual code environment that is executed.

Indeed, in spite of being a powerful platform, AWS Lambda is currently limited by its selective targets. Oddly enough, if you are a Go or Ruby developer (for instance), you must call your code as a subprocess of the executed function.

dalamb takes the bet that the doors should be open to any developers with the same interface. To achieve this, dalamb will expect functions to be packaged up into container images, under the control of the developers. What dalamb should provide is the entry point definition for a container to be executed appropriately on the platform as a dalamb function.

Containers will be short-lived and their state will not be kept around for longer than the time required to run the function.

Developers will store their function on platforms like github or bitbucket and dalamb will simply react to events from those platforms to release to production new versions of a function.

What’s next?

dalamb is by no means properly defined or even designed. It is a story about developers achieving great values by relying on developer-oriented platforms. Whether it is Heroku or AWS Lambda, dealing with the complexity of this world has never been so accessible.