Brazil Data Cube Workflow Engine: a tool for big Earth observation data processing
by Vitor C. F. Gomes¹, Gilberto R. Queiroz², Karine R. Ferreira², Edzer Pebesma³ and Claudio C. F. Barbos²
¹ C4ISR Division, Institute for Advanced Studies (IEAv), São José dos Campos, Brazil;
² Earth Observation and Geoinformatics Division, National Institute for Space Research (INPE), São José dos Campos, Brazil;
³ Institut für Geoinformatik, Westfälische Wilhelms-Universität, Münster, Germany.
DOI: https://doi.org/10.1080/17538947.2024.2313099
Publisher: International Journal of Digital Earth | Published: 9 February 2024.
Abstract
Earth Observation (EO) satellites have produced vast image collections that are freely accessible to society. However, handling these images often surpasses the capabilities of traditional hardware and software for EO data storage and processing, posing challenges for traditional Spatial Data Infrastructure (SDI). To overcome these challenges, innovative cloud computing and distributed systems have been developed, such as matrix databases, MapReduce systems, and web services. These technologies are now integrated into leading-edge platforms, forming a new generation of SDI for big EO data. These platforms have different characteristics in terms of governance, technologies, data access, infrastructure abstractions, data processing, and flexibility to extend their functionality. Our work contributes to the area of SDI for big EO data by proposing a server-side data-processing tool called Brazil Data Cube Workflow Engine (BDC-WE), based on workflow orchestration approach. BDC-WE provides a high-level interface using the openEO API for big EO data accessing and processing, allowing SDI maintainers to easily describe sequences of processes and integrate new algorithms. The architecture proposed in this study was implemented and the prototype was evaluated in two case studies described in this paper.
Keywords: Big data, Earth observation data, spatial data infrastructure, openEO, workflow orchestration
© This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
Share and Cite
Vitor C. F. Gomes, Gilberto R. Queiroz, Karine R. Ferreira, Edzer Pebesma
& Claudio C. F. Barbosa (2024) Brazil Data Cube Workflow Engine: a tool for big Earth
observation data processing, International Journal of Digital Earth, 17:1, 2313099, DOI:
10.1080/17538947.2024.2313099