What is Orchestration?

Scott: Hey everyone, I’m Scott King, and I’m here with Chris Kraus. Today, we’re going to talk about orchestration and what it actually means. I did what most people do these days and asked ChatGPT for a definition. 

Here’s what it told me: ‘Orchestration refers to the process of arranging or adapting a piece of music for an orchestra or a large ensemble. This involves assigning different parts of a composition to different instruments to create a balanced and harmonious sound.

The term ‘orchestration’ can also be applied more broadly to describe the process of coordinating and blending various elements in other artistic or organizational contexts.’ I think that’s pretty cool. But ChatGPT didn’t stop there.

Chris: Is there something more than artistic orchestration?

Scott: What I found really cool was that the AI kept generating text even after my initial input. It was like it understood who I was and why I was there. It continued typing and generated a definition for orchestration in the context of computer systems and software. 

Essentially, orchestration refers to the automated coordination, management, and arrangement of complex computer systems, middleware, and services. This involves managing the interactions and execution of multiple tasks, sometimes across different platforms or environments, to achieve a desired outcome. When I saw that, I was like, “Wow, that’s awesome! I love it.”

Chris: The second definition is a bit drier, but I’m impressed that the AI was able to generate two examples of what orchestration means. It’s pretty impressive.

Scott: It was surprising to me that the AI generated two different definitions from one prompt. Since I knew we were going to discuss the DevOps example, I asked the AI what orchestration means in the context of DevOps. I expected it to provide a more contextual response, but it exceeded my expectations. 

Here’s the definition I received, “In the context of DevOps, orchestration refers to the automated management, coordination, and execution of various processes and tasks involved in the development, deployment, and maintenance of software applications.” DevOps is a set of practices that aims to bridge the gap between development and operations. It’s a bit jargon-y, but I suppose the AI learned that from the internet, as everyone seems to want to “bridge the gap.”

Let me get back on track. So as I was saying, DevOps aims to bridge the gap between development and operations teams by promoting collaboration and integration to streamline software delivery and improve overall quality. The AI also provided some examples, which I’ll include in the article with a screenshot. 

But let’s focus on orchestration, Chris. Some people may not fully understand what orchestration means in the context of DevOps, especially if they’re only involved in a single app or focused on infrastructure. So what does orchestration really mean in DevOps?

Chris: We always have a bit of a blind spot when we think about our step in a bigger process. When you say DevOps, there’s always an implied figure eight with continuous integration where we do the basics. We worry about how we compile the code, use source control, build it, and do unit tests. These are the things that developers tend to do, but they’re not the entire end-to-end cycle. Of course, developers love tools, and there are a lot of development automation tools that help with that part of it.

However, on the other side, when you get to continuous delivery or deployment, which I’ll just refer to as CD, that’s a whole other set of organization, different skills, and different parts of the organization. The goal is for everybody to work together to say, “Here’s your requirement, let’s code it, and let’s actually get it tested, moved into production, and monitor it.” Monitoring production is always the hardest part.

Many people just think of part of this process because that’s the part that their role is used to. Something needs to orchestrate and handshake from one part of the organization to the next. It’s difficult for people to think about it from the bigger picture. There’s a figure eight on the wall, but there are probably four different parts of the organization that have to help define what it is.

So let’s go code it. CD involves source control, compiling, unit testing, validation, non-functional testing, and sign off to put it in the customer’s hands. It’s a bigger process that has multiple steps, and it doesn’t work unless you have all of it. But it’s not all in the same toolset, and it’s not the same person doing everything.

Scott: Definitely, and it varies by organization. All the cool DevOps stories are about digital-first companies where technology isn’t as diverse.

Chris: Well, for security mandates, like hospitals and healthcare, they require a lot of sign-offs to ensure accuracy and validation. However, if you’re on a digital-first website with pop-up ads, their effectiveness is irrelevant if they don’t work properly. There are no federal or governance issues if an ad is not rendered quickly or accurately. In other parts of the organization, there are more validation steps involved in their processes.

Scott: So, what are the good parts of automation? Which tasks are more suitable for automation than others?

Chris: If you listen to people talk about Facebook and Amazon, they mention how many deployments they do and how much they build. Deployment is a broad term that can refer to the start of a process. From a developer’s perspective, we have source control, we extract, we compile, and we either build locally or on a server to ensure that the code is correct. We have tools to help us get requirements and put them into sprints. We’ve done a good job of dividing up the code, but after we’ve built it is where things get more complicated. The front end of continuous integration is something we’ve gotten good at because we’ve had years of experience and there are many tools available. However, when it comes to the next step, such as user acceptance testing or security testing, things become much more manual. These tasks are newer and it’s much harder to automate or orchestrate them.

Scott: Who’s in charge of this? If those manual steps take longer and it’s harder to orchestrate because maybe you’re involving people, and then you talk about the different rules from health care, security, and data governance, who’s in charge of that? And what are they looking into? This is a hypothesis, right? Are they looking into better automating what works well or addressing bottlenecks and constraints that slow the whole system down?

Chris: I think most people look to automate tasks that don’t require human intervention or feedback. For example, automating tasks in a script, like FTPing a file or starting a web server with a command line, are much easier to automate because humans aren’t involved. However, tasks that require human input, such as verifying if a system is functioning correctly or conducting performance testing, are much harder to automate. The challenge is that many of these additional steps don’t have a common system of record. Each person has their own place to work, whether it’s manual or automated, and they all use Excel spreadsheets or different ways to track their progress. So, the easiest tasks to automate are those that don’t require human interaction.

Scott: Yeah, or it’s a person, right? Like Brent from the Phoenix project. He knew how everything worked, and if he left, that would be a problem. So, with all these automation tools available, where do we need to orchestrate the next step? Going back to the orchestra example, when you listen to an orchestra, you hear the entire piece all at once, but there are different coordinated parts. It would be like if one part was missing or disrupted, like if you heard the orchestra without the strings and then they walked in from backstage and were getting settled, it would be kind of like that, right? Is human interaction often the hardest part?

Chris: Definitely. The challenge with human interaction is that it happens at different points in time, and when people are involved, they not only need to take action, but they also have to follow up and notify others of their results. Nobody likes filling out Excel spreadsheets and emailing them or uploading things to SharePoint. 

And then someone else has to go chase everyone around and ask if they’ve done what they’re supposed to do and what the results are. When we put people in charge of orchestration, like having someone do a manual test and then having a project manager come around and ask if everything is done, it becomes difficult. We used to have a person named Heather who would run around as a project manager and ask if everything had been done, including load testing, non-functional testing, and security scans. That’s the hard part because people are busy, sometimes they don’t get things done, they don’t have statuses, and there’s nothing coordinating and presenting the information in a single view in real-time.

Scott: That’s interesting, you’ve mentioned Excel three or four times already. I can’t imagine having to update an Excel spreadsheet to keep track of this. What I’m thinking about are the notifications on your phone. Your phone would just notify you if something changed, and you wouldn’t have to constantly check for changes, right? Instead of using an Excel spreadsheet, wouldn’t it be better if the DevOps orchestration process could simply notify me on my phone, like “Hey, do this”?

Chris: That is where orchestration is missing. We’re not very good at giving in the middle of a process. Kicking off a process and waiting for a result is easy, but saying “start functional testing” may mean we have manual testers offshore with a quality center, who have hundreds of manual tests to work through. Then someone has to get user acceptance from someone in the business who can say, “this does what it’s supposed to do.” Meanwhile, we’re doing regression testing and load testing in parallel. 

In the second half, multiple people are getting their jobs done, and we make a final go or no-go decision. We’re really orchestrating people, and that’s not something people have gotten good at. You can always find a niche tool that will track this, but in reality, no one’s gotten very seamless at including people to say, “I’ve signed off. The new functionality does what I expect it to.” Functional people have tested and regressed and said, “we haven’t broken anything unexpectedly.”

The security scans say that there are no open ports or problems like cross-site scripting, and the people doing vulnerability testing tend to work in parallel, which makes it harder to track. That’s why you need to lay an orchestration tool on top of it. It’s fine that they use Quality Center for their reports and load testing products for load testing. They can also activate service virtualization to simulate things for functional testing with specific data. But someone needs to help orchestrate and realize that a lot of it will happen in parallel. At the end, it’s about whether everybody checked in, gave their feedback, and gave their thumbs up or thumbs down. Then you go to the next step, which is coordinating with customer success to take this upgrade. It may require a little downtime, or there may be new functionality the customer needs to know about so they’re not surprised when it shows up. There are always multiple steps involved.

Just because we compiled the software and say it’s ready, it doesn’t mean we’re done. We have to notify customers and coordinate with them about the upgrade. We have to figure out when they want to take the upgrade, make sure they’re trained on it, and inform them of any changes in the screen or how to do something. It’s a whole other step that keeps coming. We’re not even done when we say the software is done. We now have to figure out how to tell the users we’re going to upgrade their environment. Orchestration can help us communicate better so that everybody knows what to expect. Then there’s a whole other checklist for deploying to production and deployment that we need to automate.

Scott: Yeah, I didn’t really consider the customer success aspect, that’s interesting because ultimately the goal is to get the software feature in the hands of the user. I was originally thinking more about companies like Facebook and Amazon, where changes are made and users just have to deal with it. But if you have your internal processes in order and then you have to involve the customer and have them check off on their UAT, that can be challenging because they have their own day jobs to worry about.

Chris: There are more steps involved because the outcome is not complete until the customer is using a feature and getting value from it. This is our ultimate goal, so we need to take all the necessary steps to get there.

Scott: Okay, so let’s say we have Heather who is having trouble orchestrating everything. What advice would you give her in this situation? As someone in charge, what should her focus be?

Chris: So, she has tribal knowledge of the process and the key roles of who to notify and get feedback from. When she puts that process in automation software and the software runs 10 instances of that process, it’s responsible for doing the checking up. This changes the way she works. Instead of working on Hassan and Bug people, she’s working on exceptions. She no longer worries about processes that work, but only manages the exceptions. If she gets notified that an exception has happened, such as security scans not being done in four hours, then she can focus on the important work of finding out why there is a delay. Her job is to take the steps of the process, the associated roles, put them in automation software, and let it do all the tedious work. She only expedites things that are broken or behind schedule, becoming the expediter for what’s gone wrong.

Scott: That makes sense to me. I hope it makes sense to everybody else. Just work on the exceptions. But, you have to build this orchestration for it to take care of all those automation steps.

Chris: And it’ll change over time. You’ll never build it once and never change it. There will be some changes where she’s going to update her automation. They may swap tools in the background, but that shouldn’t affect your process. Maybe there’s a new bit of process you have to cover. She’s going to upgrade and change that. It can’t just be static, one and done, and walk away.

Scott: All right. Heather, if you’re out there, give us a call. We’ll help you. But I appreciate it, Chris. Thanks for talking about orchestration and love the Dev Office example. Until next time.