This blog post is inspired by the discussion “Is GitOps an evolution of DevOps?” and reflects its main points.
Recently, there has been a significant interest in GitOps, a methodology introduced by Weaveworks in 2017. The growing number of dedicated conferences and summits makes it evident that GitOps is gaining momentum. But is it mature enough to be a mainstream, a default approach to be applied to every project? We try to get into the matter with our experts Volodymyr Tsap, CTO at SHALB, Stanislav Kolenkin, cloud architect in Critical Services at SoftServe, and Volodymyr Davydenko, cloud architect at SoftServe.
GitOps in a nutshell
Weaveworks defines GitOps as an operating model designed for Kubernetes and other declarative cloud native tools. By having the canonical desired system state versioned in Git, GitOps automatically reconciles divergences between the declared and actual states by running software agents. Having Git as a single source of truth, GitOps naturally stems from practices that developers have been using for years, transferring them from software development into the plane of infrastructure management. This way providing ops with all advantages of source code management, such as versioning, audit, compliance and preflight checks, and making it easier for devs to work with infrastructure, create temporary environments and deploy applications.
Volodymyr Tsap sees GitOps as a subset of CI/CD, namely, a cloud-native approach to continuous delivery: “With GitOps, we have splitted CI with its builds, tests and security checks, and CD. The deployment process implies that we have a state, i.e what’s running in a cluster, a declaration of our desired system in Git, and an operator or controller that reconciles them. And this is basically what GitOps does”.
Why has it become popular?
Infrastructure codification
The rise of declarative cloud native stacks dictates the need for infrastructure codification. As lines between software development and operations continue to blur, infrastructure engineers start to do more coding and as such need tools for this. Most frequently their choice turns next to Git-based VCS management tools like GitHub, GitLab, and Bitbucket.
Convenience of centralized management
Broad functional possibilities of Git allow you to have everything, from the structure of your organization to documentation, derived and managed from a single place. Having Git as a single source of truth minimizes chances of irrevocably losing your infrastructure: even if it has been lost because of some incident, you can always reproduce it from a cloned repository that contains your manifests.
Fast delivery of changes
Whether delivering a security patch or rolling out new updates, GitOps allows you to handle tasks on a large number of servers at once.
Stanislav Kolenkin says: “I like GitOps because it allows delivering changes fast. Suppose, you have a hundred servers. How are you going to manage them? Jenkins will go down the line successively until it reaches every node, which may take some time. With GitOps you can deliver changes and policies to any number of servers instantly”.
Developer-friendly toolset
GitOps propagates the concept that deployments should be as easy for an engineer as making a pull request. With the delivery pipelines centered around Git, developers can easily manage application deployments and Kubernetes operation tasks with their familiar tools, by making pool requests and committing to a repository.
GitOps pain points
Lack of feedback on deployments
The decoupling of CI and CD means that these processes are implemented by separated controllers and in different pipelines. This can cause an issue of not knowing exactly when your environment has finished with the deployment, and bring forth the necessity to write a wrapper that would inform you on the status of your deployment. With conventional systems like Jenkins you have CI and CD within a single pipeline, and the system informs you on whether your deployment was successful.
Security validation
Enforcing security policies together with using GitOps could come at odds and block your release. Suppose, we try to deploy and bump into a security policy that is managed by security engineers. As the controller tries to deploy again, it goes into a loop creating a security incident that gets blocked by a security team as a potential threat. That is why it is advisable to validate code on the CI level so as not to enforce security policies inside the cluster with Gatekeeper or Open Policy Agent.
Volodymyr Davydenko says: “At a certain point you can’t make security compliance checks on a cluster or infrastructure level. As successful cases prove it, security validation is better done at the CI level”.
Promotion between environments
It is a well-known issue that GitOps has no established practice for dealing with multi-environment configurations. Let’s say you need to promote an artefact, a container image from one environment into another without rebuilding it. How could that be implemented with GitOps, provided that the artefact’s URL changes when migrating from one container registry into another? There is no simple answer. The possible solutions could be:
- using a single repository with a branch for each environment
- using a separate repository for each environment
- using a single repository with a subfolder for each environment
with each of them entailing their own disadvantages.
Secrets management
Handling secrets is an important part of the software development, which is however left unaddressed in GitOps. According to Volodymyr Davydenko, GitOps tools lack functionality in building security policies: “Even being integrated with Hashicorp Vault and by making use of Vault Secrets Operator, both Flux and ArgoCD don’t match corporate security policies”. With GitOps tools providing no solution for this, it remains unclear how to deal with secrets in GitOps.
What’s next?
Volodymyr Tsap believes that GitOps is the next step in the technological chain that succeeds DevOps: “Linux, Git, Kubernetes, DevOps, GitOps – these are all stages of technological evolution. Today GitOps is a must-have skill that forms your value in the market”.
Stanislav Kolenkin: “GitOps as a solution will surely evolve further. The new functionality, all the issues that we have mentioned – this all will be growing in a snowball effect. Nevertheless, your choice must always be project-centric. That’s not to say that GitOps is a one-size-fits-all solution”.
To wrap it up, our experts came to a consensus that the future of GitOps lies in self-serviced automated platforms. Volodymyr Davydenko says: “I can see some companies already shifting from the GitOps pattern to standardized platform tools. During the onboarding process developers sign in to such a platform that provides them with repository templates, enables to choose a programming language, the way to build a Docker image and where to deploy it. But above all this platform defines a workflow that is triggered after the first commit is done into dev, and how this is then delivered to production”.
