Hello Kubernauts! My name is Jim, a Staff Project Manager at Harness, and today, I’ll be sharing my love letter to containers and their use in continuous integration (CI) pipelines.
As someone who has been responsible for CI pipelines since 2006, I’ve witnessed firsthand how containers have revolutionized the CI workflow. Before the era of container-native CI pipelines, build engineers and ops teams would set up build servers with specific dependencies required by the application and hand them off to the development team. However, this approach often led to complications, such as delayed access to necessary software, and an overall lack of control for the development team.
But then, container-native CI pipelines emerged, treating containers as the first-class unit of infrastructure. This allowed development teams to own their build environments, resulting in a much smoother workflow. One such tool leading the way in container-native CI is Drone CI, an open-source CI tool launched in 2012.
Drone CI, written in Go, offers a self-hosted solution that enables you to run it on your infrastructure of choice. Harness acquired Drone CI in August of 2020. Whether it’s your cloud service, data center, or even your home lab, you retain control over your source code and secrets, without relying on third-party exposure.
The real magic happens when Drone CI interacts with Docker containers to execute CI pipelines. Your Drone YAML file defines the steps of your pipeline, and each step runs inside a container as the entry point. With ephemeral builds, you’re guaranteed clean environments for each pipeline run, enhancing reproducibility.
Drone CI’s uniqueness lies in its plugins. Unlike other CI tools where plugins are installed on the server side, Drone CI plugins are Docker containers included as steps in your pipeline. These plugins can be written in any language that runs inside a container, providing a wealth of options for customization.
The official Drone Plugins Marketplace houses a plethora of community-developed plugins, making it easier for developers to find pre-built solutions for their needs. Furthermore, you can create your custom plugins, thus enhancing your CI/CD pipeline’s capabilities.
While container-native CI pipelines offer numerous benefits, it’s crucial to be aware of potential challenges. One such challenge was faced when Docker Hub introduced rate limiting on November 2nd, 2020. Docker Hub’s rate limiting affected CI pipelines, leading to failures and confusion among users.
To ensure smoother authentication to pull images from Docker registries, Drone CI provides a feature called “image pull secrets.” This allows users to attach Docker config JSON contents as secrets to repositories or organization-wide. With organization secrets, admins can generate read-only credentials and make them available to all pipelines.
By leveraging container-native CI pipelines with Drone CI, developers can achieve more efficient and reproducible CI/CD workflows. Embracing the power of containers and plugins empowers development teams to build, test, and deploy code confidently, knowing they have full control over their build environments and dependencies.
In conclusion, containers have become a pivotal technology in modern CI/CD practices, and tools like Drone CI provide the necessary container-native support to unleash the full potential of containers in software delivery pipelines. With proper authentication and the flexibility of plugins, teams can streamline their CI processes, promote reproducibility, and ultimately deliver software faster and more reliably than ever before.
