Ephemeral Execution Is the Future of Computing, but What About the Data?

Ephemeral processes like containers are increasingly being used to build and run applications because of their isolation, portability, and efficiency. But they're often bogged down by data gravity when handling data-intensive applications. If a process is executing only for a short time in isolation, how can it consume and produce data from and to a variety of locations, beyond the bounds of just databases?

In this talk, we will use Tekton as a case study to explore challenges of data gravity in ephemeral execution. Tekton is an orchestration engine for CI/CD that executes arbitrary user-supplied workloads as containers on Kubernetes. Tekton workloads need to securely consume data (such as source code) and produce data (such as binary artifacts), while avoiding limitations of the underlying architecture. To address these challenges, Tekton is exploring approaches including clean container injection mechanisms for backing immutable artifacts, and a secure server interface between orchestrating controllers and user-specified workloads for communicating trusted metadata.

Learn how to effectively use ephemeral processes to build and run data-intensive applications.