IT services: consulting, software development, Internet and support

Tender description

This contract is for development of software for the Climate Data Store (CDS) Toolbox. The CDS is composed of a set of distributed data repositories containing all types of climate data and products ranging from very large datasets such as observation, reanalysis, climate projection, multi-model seasonal forecast products, to smaller and more elaborated value added products such as climate indicator maps, time series, tables, etc. 1 of the aims of the C3S is to provide an authoritative set of software (the CDS Toolbox) that will allow the users to develop applications that will make use of the content of the CDS. The Toolbox will be composed of: — Tools that perform basic operations on data, such as computation of statistics, sub-setting, averaging, value at points, etc.; — Workflows that combine tools by chaining them so that the output of some feeds as input to others; — Applications that make use of workflows and selected data and products of the CDS to build interactive webpages. These applications will be made available to users of the CDS via the C3S web portal, allowing them to perform data manipulation on the data and products of the CDS, and visualise their results. Users will also be able to share with other users of the CDS useful and quality controlled applications they have developed. The data available in the CDS will vary greatly in volume (kilobytes to petabytes), in format (GRIB, NetCDF) and in nature (observations, fields, time series). The contractor will: — Establish the minimum set of tools required (differences or re-gridding, as well as statistical computations such as means or standard deviations); — Review the existing software that could be used to implement this set of tools; — Design a common data model that can be used to combine data and products of different types; — Establish a method by which workflows using data and products that are held in various distributed data repositories can be orchestrated, allowing computation to be performed close to the data and data transfers to be minimised; — Implement a series of reference applications that will demonstrate all functions offered by the Toolbox and will be used as examples. The contractor will work closely with ECMWF and developments will follow an agile methodology. Technological choices will be based on existing open source software. Ownership of all rights in the resulting software will be assigned, through ECMWF, to the European Commission and ECMWF will release the resulting software, on behalf of the European Commission, under an open source licence. The architecture of the system will be open and extendible, and scale both in terms of the number of concurrent users and volume of available data.