Computer-related equipment

Tender description

Diamond Light Source (DLS) wishes to deploy a new platform for the collection and analysis of experimental data from beamlines and electron microscopes. The installation will be in our new data centre and will provide an integrated set of high performance compute nodes and a highly resilient, high performance file system. Diamond already operates a suite of high performance file systems and a HPC cluster in an existing data centre. However, this new system will not be “tightly coupled” with existing HPC and storage platforms. i.e. it is not planned to extend the low latency, high bandwidth current infiniband fabric across to the new data centre. This procurement is for the foundations of a new platform in a new datacentre, and it is required that this new system will be able to grow over time in both capacity and capability within the new data centre. The budget for the new system is 2 000 000 GBP ex VAT. For computational analysis, we require a system that in operation will be able to provide near real-time feedback on data capture at end stations, and provide for longer term post-processing of stored data. Our current applications utilise both standard x86 architectures, for which the majority are typically characterised as high throughput compute, single node type jobs, and GPU based applications that are developed internally and externally to utilise NVidia GPUs through CUDA. It is envisaged the compute nodes and the storage provided will be interconnected via a high performance, low latency fabric. The new compute nodes within the system will operate as an integral part of the overall solution, coupled via an expected high bandwith, low latency interconnect, to the proposed storage solution. — There is no minimum number of nodes or cores required. — The majority (>90 %) of the workload is characterised as high throughput or dependant on GPU for execution. — A balance between high core count and number of nodes is desired in order to maximise the overall throughput of jobs in the system. We typically operate shared workloads on a per node basis, but where GPUs are incorporated compute nodes generally are used exclusively on a per job basis. — Existing GPU nodes have a 1:1 ratio between CPU and GPU card per node. This is known to work well for our requirements though alternative proposals are acceptable. For storage, current experience is based on the use of both Lustre and GPFS, with each having different strengths and weaknesses. We intend to choose the file system to be deployed based on tests conducted in cooperation with the vendors. Alternative solutions to Spectrum Scale (GPFS) or Lustre are welcome providing sufficient supporting evidence of their usage in similar environments can be provided. In summary we require: — at least 5 PB of usable resilient storage, — at least 60 GB/s sustained aggregate read and write throughput to our clients, — at least 900 MB/s sustained aggregate read and write throughput from a single client, — sufficient meta data performance to allow fast data acquisition to be unaffected by data processing. We expect the system to support approximately ~ 2 x 109 files. We expect the vendors in the dialogue will have direct relationships with the compute and storage system hardware manufacturers, and also (possibly through these manufacturers) have demonstrated deployment experience with the proposed file system such as GPFS, Lustre technologies. Computer room constraints will require the compute and storage system to be installed in existing 19'' racks with an average power density of 15 kW per rack. Up to 3 racks may operate at a higher density of 20 kW per rack.