OTT Moves Toward Microservices as Speed Becomes an Issue
The process of instantiating a composition and ensuring it is available is given to a part of the overall MSA called orchestration. The orchestration commands (or “requests” in some architectures) the workflow be launched across suitable infrastructure and ensures it is operating, replaced in the event of a failure, and tidily ended when it is no longer required.
The other key component of an MSA is the scheduler. Often blurred with orchestration, the scheduler performs a different role—that of determining what function is required, when, and by whom—and is often the point of interface for the end operator’s business logic API.
Additionally, both orchestration and scheduling in the MSA approach usually share the responsibility for the critical decisions on where the function will be launched.
Orchestration can make fast decisions that ensure functional availability to enable self-healing and scaling, while the scheduler can make decisions tied to business logic, typically influenced more strongly by commercial forces than technical and operational ones.
KUBERNETES
Emerging from within Google, Kubernetes is really the go-to orchestration of choice for most organisations which are attempting to get some scale and operate with heterogeneous infrastructure. Kubernetes can orchestrate many types of containers, not just Docker, but others such as App Container (AppC), rkt, and more. Orchestration is central to the operation of an MSA, and having the ability to future-proof that layer as much as possible is a very important strategic advantage. Kubernetes has emerged as the leader.
DOCKER SWARM
Docker, of course, has its own orchestration capability, Docker Swarm. Swarm is a little more recently evolved and is ultimately playing “catch-up” with Kubernetes. There is tighter integration with Docker available from Swarm, and that is obviously a benefit, but should an operator ever wish to move outside the Docker relationship, Kubernetes offers that “diversity.”
MESOS
Mesos is the other giant in town. Though it is not specifically geared to container management, Mesos, with the addition of Marathon, makes a good container orchestration platform. It has numerous other capabilities too, largely focused on resource scheduling, and so Mesos is a very good single platform for managing both the fabric provisioning of the container hosts and the orchestration of the container functions out onto that fabric. Mesos could potentially run Kubernetes or Docker Swarm, and so Mesos is a significant player worth understanding.
A good comparison can be found at this blog post.
ERLANG
Another approach is one that my own company, id3as, has taken. We were looking for tools for developing an MSA for some of our clients in 2009/2010, a long time before many of the above strategies were mature. We turned to Erlang. Some 30-plus years old, Erlang was designed to deliver digital phone calls across Ericsson phone exchanges, and from the ground up it addresses some challenges that are all too familiar today, including the following:
- Redeploying function where infrastructure had failed
- Scaling up and down on demand
- Small function-focused paradigm (“functional programming”)
- Lightweight
- Isolated processes
- Orchestration and scheduling
An Erlang “program” is somewhat akin to a container, and indeed Erlang applications can be containerised and orchestrated by Swarm or Kubernetes. But in many cases, third-party scheduling, orchestration, and containerisation add no real benefit to our ability to schedule, orchestrate, and deliver function in pure Erlang, and usually introduce many unnecessary complexities. For platforms like Twitter and WhatsApp, Erlang works to deliver MSA extremely well, at scale, and with incredible uptimes.
Nonetheless, the widespread interest and adoption of Kubernetes and Docker in the recent 2–3 years mean that we increasingly see clients seeking APIs for communication between our Erlang-based id3as.media suite of functions and these other orchestration and scheduling environments. And one of the great things about MSA is its ability to take a single function and replicate it, vary it, and make it available to the operation.
Who Is Doing What?
But don’t take my word for it. Many companies in the streaming media space are using MSAs, and I spoke with a couple of exemplars.
“Our own internal direction is more toward a containerised/Kubernetes model,” says Dan Murray, product marketing manager, video monitoring at Tektronix. “We are not entirely there yet. But our customers are keen on that long-term fit. Externally, it tends to be a small number of customers [who] are pushing the edge and are doing their own Kubernetes/ MSA. Some of these guys want a similar delivery of technology.
“So customers don’t ask for orchestration or containers etc.; they typically ask to do things quickly, be it time to market or monitoring or the reactiveness of self-healing. When a client wants us to spin up picture analysis containers with the test being ready, it is highly beneficial to be able to spin that up quickly.
“We meet with many operators who say ‘We are heading that way, we are not sure we are ready yet, but we all want to go that way.’ So to some extent it’s about mindset.”
Harmonic is another company moving toward microservice architecture.
“Container-based micro-services represent a means to implement media processing functions for workflows, including OTT,” says Olivier Karra, director of global solutions architecture at Harmonic. “However, proposing these components at such a low level of the system stack is far from being sufficient, both from a functional layer and a system standpoint. Indeed, using micro-services alone implies significant additional work, resources and costs to deploy media services.
“In the case of VOS 360 SaaS, Harmonic oversees the deployment, management, and operations. This enables media services to be launched with minimal resources and time, increasing business agility, and reducing total cost of ownership. Based on the substantial benefits that it provides, we are seeing significant interest and adoption in VOS media processing from operators all across the globe.”
Numerous other companies are working out how to make their propositions available in some virtualised form or another, though as mentioned earlier, many of the demos shown at industry conferences—some from very large telcos and broadcast technology vendors—are a long way from production. A lot of the players in the space are trying to “retrofit MSA” to their legacy monolith software and development approach and may only realise how complex a problem they have made for themselves sometime after they are in production. They need to let go of these monolith approaches if they really want to benefit from MSA at scale and in production.
There are numerous stories of teams that have tried and been challenged by implementing MSA. While not a streaming operation, Segment shares a detailed account of a service where MSA was tried but was eventually replaced. It appears, though, that it lacked a number of skills necessary to translate to a full MSA from the outset; from what I can see, it had all the architectural principles worked out from a functional point of view, but it lacked planning for scheduling and orchestration, making tasks like deploying tests in isolation and ad hoc very complex.
In my own company, we design systems such that the customer simply defines their desired “state” through our API—for example, which channels they want running, what the multicast addresses are, what quality of video output they want, etc. We then make that happen on the pool of hardware we have available, and the customer does not need to know nor care which machine is performing which aspect of each job. That is handled by the orchestration and scheduling systems, which are then in a position to dynamically react to failures.
Once the state is active, the system actively monitors itself for faults and has native strategies for responding to those (potentially adding others that we may introduce for “specific business logic” reasons relating to a particular client).
What we see is that a number of traditional monolith software companies have wrapped their monolith in a Docker container, exposed a config API for the entire monolith, and then tried to use cloud-specific auto-scaling functions to launch and tear down copies of that monolith.
It works, at first. But over time we then hear stories about how MSA is challenging. Well, it might be a challenge for software vendors which are actually just gift-wrapping their traditional monolith software and hoping the cloud’s orchestration will give their clients all the control they want.
Auto-scaling is something that only works for a very small percentage of applications; for instance, a static website auto-scales very well. A pure transcoding engine that just does a single transcode, delivers the result to, say, S3 and then stops would also auto-scale well. As soon as you have any sort of “state” to maintain, it starts getting much harder. Auto-scaling a database is a non-trivial challenge. Auto-scaling a WebRTC session to allow more participants would not work so well.
Moving to MSA lets those parts of the application that can benefit from simple autoscaling do so. However, it’s no silver bullet, and it doesn’t make the bits that can’t auto-scale any easier at all. But at least it reduces the scope of the bottleneck problems that those auto-scale challenges cause.
So unlike the containerised monolith approach on display at various recent exhibition demos (to plug the companies’ knowledge of Kubernetes and Docker), if your software is born in the cloud and architected from its lowest level as a collection of properly designed microservices, then MSA is definitely the ideal way to keep large OTT systems agile, scalable, and highly available.
It is also, by nature, therefore quite difficult to actually demo a proper MSA on an exhibition stand because, to a user, a good microservices application should functionally do exactly what a traditional monolith does ... until you look at complex network scaling or fault tolerance, which you can’t typically do on the convention centre floor.
Conclusion
The MSA mindset is critical to modern distributed computing and network architecture. Even if some functions remain appliance-based for some time to come, all extension, renovation, and replacement will benefit strategically from endeavours to align the entire architecture to maintain a firm separation between function and hardware.
The question of when to fully migrate to MSA will largely be driven by economics. The urgency increases with the OTT operator’s move toward offering premium live services, since in that scenario, the service scaling and availability become mission-critical to meeting a commercial promise to the consumer with no second chances. The underlying platform must be instantly self-healing and rapidly responsive, if not preemptive, to scaling demands.
[This article appears in the Autumn 2018 issue of Streaming Media European Edition as "OTT Moves Toward Microservices Architectures."]
Related Articles
Can the internet keep up with the ballooning demands video is placing on it? And how can planners manage growth while keeping costs down? The correct answers are not always the obvious ones in this hall of mirrors.
28 Nov 2018
OTT services targeting kids need to keep four things in mind: content, convenience, user experience, and safety.
19 Jun 2018
Nagra details five trends it says will shape how companies deliver-and how consumers purchase-TV in the coming year.
21 Feb 2018
The subscription services are fueling their rise with lavish original content budgets, outspending traditional broadcasters.
13 Dec 2017