Reproducibility without compromise: Guix and ravanan

Abstract

We have a reproducibility crisis. Workflows that work on one machine at one time do not work elsewhere or later. Package managers like conda make it easy to install packages on different machines, but cannot help with reproducibility. Docker/Singularity images capture the software environment precisely, but do so by "shrink-wrapping" and leave us with a blob that needs to be stored forever more. Modern software has accrued incrementally over many decades and has become incredibly complex—a Gordian knot. Programming language-specific package managers (such as pip, cran, Julia's Pkg, etc.) appear to work well but choke up when dealing with multiple programming languages—the unavoidable reality of modern software environments. We need new tools that can straddle programming language boundaries seamlessly and describe workflows precisely. In this talk, I will introduce Guix and ravanan. Guix is a purely functional package manager that provides strong bit-level reproducibility guarantees and can control the software environment precisely all the way down to the lowest-level system libraries. ravanan is a Common Workflow Language (CWL) implementation that builds on Guix, bringing the same reproducibility guarantees to workflows and providing strong caching. ravanan caches not just on the workflow inputs and parameters but also on the complete software environment. A tiny change deep in the dependency graph invalidates the right cache entries and recomputes just the necessary steps—like a "make" that never needs a "make clean".