By Jonathan Boulle, December 13, 2016
Originally published on the CoreOS blog (archived copy) . Re-published with the authors' permission.
Since the release of rkt 1.0 at the beginning of this year, the project has powered ahead with over 20 new stable versions on a regular release cycle. The goal of rkt has always been to provide a container engine that is not only reliable but also composable and standards-driven, allowing easy operation and integration with other best-in-class tools in the container ecosystem. Today we wanted to provide an update on the ongoing work to integrate rkt with two such projects - the Kubernetes cluster orchestration system, and the Open Container Initiative (OCI) container standards - and chart the course for rkt's future in the year ahead.
From the start rkt was built as a pod native container engine. This means that the basic unit of execution is a pod, linking together resources and user applications in a self-contained environment. rkt's pods naturally follow the same pod concept popularised by Kubernetes.
To provide a smooth experience, rkt takes care to set up application context in a way that resembles as much as possible the usual Linux environment. Because of this, and the facilities that a pod-native container engine provides, an application developer doesn’t have to worry about:
At the same time, rkt offers a comfortable tool for SREs and systems administrators by supporting daily operational needs:
The technical implementation of rkt is even internally aligned with this concept, as the engine harnesses modular components, like the Container Network Interface (CNI), systemd for service management, and systemd-journald for logging. These modules are independently developed by dedicated communities to handle specific tasks.
Kubernetes is a cluster orchestration system, and upstream Kubernetes is a central component of CoreOS Tectonic. Kubernetes runs all applications in pods of containers, and it achieves this by delegating runtime tasks to a container engine. As Kubernetes has matured, users have requested the ability to use different execution engines like rkt or Hyper in a Kubernetes cluster. Earlier this year we introduced the initial version of rktnetes, a project to add support for rkt as the first alternative container engine. This process involved a considerable amount of work to make the Kubernetes Kubelet codebase less Docker-specific, removing assumptions and special cases from the source to form a truly modular abstraction.
This work combined with community effort and discussion led to the creation of the Container Runtime Interface (CRI), an API specification for low-level interaction between the kubelet and container runtimes. For more details on the history of CRI, see the original proposal. For more about what this means for rkt and Kubernetes, read on.
The introduction of the CRI into Kubernetes brings interesting new possibilities for pods, with granular control over the lifecycle of individual applications. This increased flexibility enables a variety of new use cases, like updating single applications within running pods, or dynamically injecting debugging capabilities. Notably, it implies that pods are now mutable, and that empty pods can both be created, and continue to exist after their applications exit. In CRI terms, this concept is called a “pod sandbox”.
Rkt has already introduced support for pod sandboxes. A new, experimental subcommand (currently called rkt app and enabled by an
RKT_EXPERIMENT_APP environment variable) allows the creation and manipulation of mutable pod sandboxes. It can start a new empty
environment, and then allow users to add, start, stop, and remove applications within a running pod sandbox. This was first introduced in
rkt 1.19.0 and it is currently on its way to stabilization, with more documentation following soon.
Image not preserved in the Internet Archive
Multiple applications are typically run side-by-side in a pod, with each application’s output and error stream (stdout/stderr) used for logging purposes. Historically, rkt has taken care of multiplexing the pod’s I/O to the outside world by using systemd-journald. Because of this, there was only limited support for attaching to applications directly, or redirecting their I/O.
The Kubernetes CRI allows for more sophisticated scenarios, like piping input to applications and attaching to running processes. To satisfy these requirements we contributed streaming support to systemd itself and are in the process of adding the following selectable I/O modes to rkt:
These modes are configurable per application, and individually for each app’s stdin/stdout/stderr streams, to offer the most flexibility. Look for this new feature in an upcoming rkt release in the near future.
At CoreOS we believe properly designed and employed standards are key to unlocking the power of open source. Software standards mean developers and teams can write software tools to compose and interoperate in predictable, consistent ways, without being beholden to particular implementations.
The Open Container Initiative (OCI) is a Linux foundation effort to create a truly portable software container. In the last twelve months we have seen the two key OCI specifications, "image-spec" and "runtime-spec", march towards their important 1.0 releases.
Pursuing our commitment to open specifications, rkt is currently transitioning its internal architecture to the new OCI standard - starting with the OCI Image Specification. rkt developers are ramping up efforts for native support of OCI images, including fetching, storing, and running. The upstream roadmap details technical adjustments that will happen in the following weeks, and a tracker project maps the ongoing effort status.
However, as we work through this transition internally, users can rest assured that our compatibility guarantees will still apply. Moving forward, we recommend the building and usage of images, but the rkt 1.x series will continue to fully support the retrieval and execution of ACIs the same as today.
The overarching goal is to remove internal ACI translation and embrace OCI natively as soon as the image-spec is finalized and the format stable enough for production usages.
In parallel to the image format, OCI is developing the so-called runtime-spec for describing the runtime execution environment that container engines should provide. This specification is being developed in close tandem with runc, a shared community effort at creating a reference implementation of the specification.
To provide the best possible support for the OCI runtime specification, rkt is gaining better integration with runc as its internal application executor. This is made possible by rkt’s modular architecture, which allows runc to be integrated as an alternative stage1 environment. This architecture allows the new implementation to be developed and utilized without any disruptive impact on the users, but it will allow rkt to reduce feature duplication and be better aligned with the rest of the ecosystem on its OCI journey.
rkt is home to innovative thinking about how a container engine works in the world of orchestrated clusters. From the CNI abstraction for making network interaction modular, to the CRI formalizing the way clusters interact with the container engine on each member node to run applications securely, simply, scalable, and reliably, rkt has been a center of cutting-edge code and a source of productive discussion with the wider community. Join us in improving both Kubernetes and the standards that define the containers and pods that package software today. If you’re new to rkt, check out these introductory rkt videos on our blog. If you’re a veteran container cluster admin or developer, take a look at the rkt documentation to start experimenting, or clone the rkt repository on github and start hacking.
