Outcome-as-a-service for multi-objective Dask-based Earth Observation processing

Abstract

Distributed Earth Observation (EO) workflows require cluster configurations that balance competing objectives (execution time, computational cost, energy consumption, and resource-use efficiency), yet practitioners typically provision resources using fixed heuristics that ignore most of these dimensions. This paper presents an outcome-as-a-service framework for Dask-based EO processing that jointly optimises all four objectives. A systematic measurement campaign on bare-metal infrastructure characterises execution time, processor and DRAM energy (via Intel RAPL), cost, and frugality across three spectral-index operators, five input data sizes, fifteen cluster shapes, and three CPU frequency governors. Configuration selection is formulated as a weighted multi-objective optimisation, in which users encode their priorities using a single weight vector across the four objectives. The resulting framework is delivered as a lightweight outcome-as-a-service that, given a workload description and user-defined weights, returns ranked configurations with predicted objective values in sub-second latency. An empirical evaluation against three deployable baselines and the best attainable configuration demonstrates that the service matches a biggest-tier oracle on execution time, cost, and energy while running up to roughly twice as fast as naive small-cluster and default provisioning, and improves frugality by up to 18% over every deployable baseline. All results are scoped to Dask-based EO workflows on homogeneous Intel Broadwell bare-metal hardware; monetary cost is computed from AWS c5 on-demand pricing as a reproducible proxy, and reported energy reflects CPU and DRAM domains measured via Intel RAPL.