NERC Data Lab (UK SBS IT16263)

Tender description

Summary of the work Provide a flexible team of project manager plus software developers to support the initial phase of the Data Lab initiative. The initial phase will use 2-3 science case studies to demonstrate the technical feasibility of hosting a Data Lab in the JASMIN data analysis/cloud environment. Expected Contract Length 31st March 2018 Latest start date W/c 20 March 2017 Budget Range The budget is £300k (excluding VAT) Why the Work is Being Done The UK Research Councils have invested in the JASMIN data analysis facility at STFC to enable scientists to undertake computationally intensive research previously not possible within the environmental research community. This facility is largely tailored to meet the needs of those with a good grasp of IT and the required skills. NERC is seeking to lower the barrier to enable wider use of the resource and meet the growing need and expectation that scientists should be able to exploit and augment large volumes of data and undertake computationally intensive modelling to facilitate advances in reasearch across the NERC science portfolio. Problem to Be Solved Broadening access to data intensive computing for environmental scientists working across different disciplines will enable more efficient and innovative ways of collaborating in a secure virtual environment to deliver the outcomes of scientific research. From a strategic perspective, the project will support NERC’s aims of facilitating Big Data approaches by the NERC science community. “Big Data”, in this context, may be defined as those environmental data that are large in file size, inter-related data that may be complex and heterogeneous, or data that need to be brought together, manipulated or analysed very quickly, particularly in parallel or in real time. Who Are the Users The users of the system will primarily be NERC research scientists and NERC employees with responsibility for system administration. Others responsible for the underpinning IT infrastructure (IaaS) at STFC will also have a role in hosting the Data Lab service. Existing Team The project team includes representatives from NERC's Centre for Ecology & Hydrology (CEH), STFC's Centre for Environmental Data Analysis (CEDA) and end user scientists from academia and other NERC Centres. The Supplier will be working alongside the Project Partners responsible for the JASMIN infrastructure at The Centre for Environmental Data Analysis (CEDA). Current Phase Not started Skills & Experience • Expertise of managing large scale open source projects (~2M lines of code) and how you will apply it to this project • Expertise of understanding the proposed infrastructure (IaaS) at the JASMIN facility, STFC and how it will apply to this project • Expertise of common research tools (Matlab, IDL, Python, Java Applications, Audio visualisation techniques), data formats (CSV, GRIB, Binary, WAV) and processing needs and how it will apply to this project • Demonstrate how you will apply your skills of providing software solutions that meet complex and varied scientific requirements for exploration, manipulation and visualisation of data • Expertise of large data volume software solutions for batch and parallel processing and how you will apply it to this project • Expertise of leading and working in teams using Scrum and other Agile techniques and how it will apply to this project • Expertise of cooperating and collaborating with 3rd party organisations such as STFC (JASMIN) and how you will apply it to this project • Expertise of software testing (automated and manual testing) and how you will apply it to this project • Expertise of design of user interfaces and how you will apply it to this project Nice to Haves • Expertise of PRINCE2 methodology for managing issues, risks and reporting and how you will apply it to this project • Expertise in a scientific or technical background (e.g. a Scientific degree or higher) and how you will apply it to this project • Expertise of engagement with user communities and how you will apply it to this project • Expertise of business process planning and requirements to support provisioning, management and administration of a Data Lab and how you will apply it to this project Work Location The Centre for Environmental Data Analysis, RAL Space, STFC Rutherford Appleton Laboratory, Harwell, Didcot OX11 0QX. Some travel to customer sites at CEH Wallingford and CEH Lancaster will be required. Working Arrangments The Supplier will largely work on site at CEDA and report daily to the NERC Data Lab management team at the Centre for Ecology & Hydrology. A summary of progress and staff resources will be provided as a written report at regular intervals (end sprint/weekly). The Supplier should provide consistent representation in key roles throughout all stages of development although flexibility within the wider team - the exact size and skills mix may vary during the course of the project - may be necessary. Security Clearance Security clearance is not required. Suppliers should provide disclosure Scotland for their staff. Additional T&Cs The supplier will incorporate flexibility into the timetable to enable End User involvement to assist in the development of user stories and the testing of products. The Supplier will accommodate breaks in the timetable for customer led end user engagement, decision making and governance. No. of Suppliers to Evaluate 3 Proposal Criteria • The proposed technical solution • The proposed approach and methodology • How the approach or solution meets user needs • Team structure and organisational makeup • How they have identified risks and dependencies and offered approaches to manage them • Flexibility to delivery the proposed approach to ensure end user engagement Cultural Fit Criteria • Demonstrate how you will work as a team with members of the project and end users to produce the best outcome. • Demonstrate how you will work transparently and collaboratively • Demonstrate how you will keep good record keeping and tracking of progress. • Demonstrate how you will be comfortable discussing complex and varied end user requirements with scientists and technicians and confident in managing discussion and keeping focus. • Demonstrate how you will contribute to the ongoing development and improvement of the Data Lab • Demonstrate how you will take responsibility for your work • Demonstrate how you will Share knowledge with other team members • Demonstrate how you will Challenge the status quo • Demonstrate how you will work with clients with low technical expertise Payment Approach Capped time and materials Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers 1. Essential Skills mentioned are for a Software Developer nothing on the project manager. Can you please let us know the essential skills required for the project manager ? A good understanding of the technical requirements of the project and the proposed solution is required alongside experience of delivering software projects. The PM should be able to provide the necessary translation to facilitate communication between the customer and software developers. They should also be able to plan resources and manage work across the team to ensure targets are met and reported on at appropriate stages. An appreciation of Agile and PRINCE2 project management is required. Evaluation will be based on the criteria detailed wihin the requirement. 2. It is mentioned that project manager and software developers are required for this project. Could you please specify how many software developers are required ? We envisage that the team will comprise necessary expertise to interpret End User needs (Business Analyst); design and implement infrastructure compatible to the JASMIN architecture (Technical Architect); develop the necessary software solutions (Programmer); and design an intuitive user interface with remote and web access (Content Designer). We have not specified how many roles/individuals are required as that will be dependent on your proposed solution and what matrix of skills you have in-house to deliver it. We are not averse to a small team with capability across several areas or a larger team that introduces necessary skills at relevant stages. 3. An essential skill is "Expertise of understanding the proposed infrastructure (IaaS) at the JASMIN facility". Can you clarify what technology the IaaS facility is based on, e.g. OpenStack It’s currently based on VMware vCloud Director but we plan to migrate to OpenStack during this calendar year. 4. Is the Data Lab going to be open source? Open source is mentioned as an essential requirement. Yes, open source.