Data Migration Services for DWP DataWorks

Tender description

Summary of the work We want to make rapid progress with our Data & Analytics projects and have created an environment which supports new data, software and ways of working with appropriate security model and architecture. We need a supplier that can provide discrete outcomes within a fast-paced agile environment. Expected Contract Length up to 6 months Latest start date 12/12/2016 Budget Range We expect suppliers to quote competitive rates given the volume of work described in the Request for Proposal. Why the Work is Being Done DWP is creating a data analytics platform (“DataWorks”) within a hosting facility to cater for a number of discrete data-intensive projects that have been developed internally. We are seeking a supplier with experience of digital platform development and cloud migrations, particularly within the field of data science, to work alongside a growing in-house team to allow us to develop the DataWorks platform. Problem to Be Solved A Supplier is required to deliver packages of work agreed in advance in fortnightly Sprints. This includes work to develop and maintain the DataWorks platform and applications, migrate services from in-house facilities to a Cloud infrastructure, use data science skills to assist our user community to exploit the data to best effect, and to support the platform as it transitions to a business-as-usual Live Support framework. Who Are the Users As a DWP Technology Function I need to enable my user community to use increasing amounts of data in a highly secure way to obtain answers to complex questions that will reduce fraud & error and inform policy decisions. Work Already Done A high level security and architecture design has been produced and the majority of the infrastructure has been procured. Existing Team The successful supplier will work on discrete pieces of work alongside a delivery team made up of internal staff and other external suppliers. Current Phase Live Skills & Experience • Able to be based from Leeds 3-4 days per week and 1-2 days from London, or as required? • Prior experience of delivering accredited data-driven digital projects using agile processes for DWP or similar organisation? • Staff security cleared to SC or DV level or willing/able to undertake the clearance process to those levels? • Demonstrable expertise in data science using own in-house data scientists? • Experience of delivering development capability using agile methodologies across five or more concurrent projects with teams of 20 to 50 development resources 5.00% • Experience of aligning development activities with other parallel work streams across split sites, ensuring quality standards are maintained 5.00% • Have immediate capacity to supply technical skills (3-5 years’ experience) in Hortonworks Hadoop, OpenStack, Hive, Hue, Azkaban, Ambari, YARN, Jenkins, Splunk, PostgreSQL, QlikView 10.00% • Have immediate access to resource to undertake the following roles; Scrum Master, Big Data Architect, Hadoop Administrator, Hadoop Data Engineer, Integration Architect, Data Analytics Consultant, Data Consultant, DevOps 5.00% • Have experience of working with other suppliers to transition knowledge of systems and takeover responsibilities within demanding timescales (~2 months) 3.00% • Have capability to augment resources using a remote model (i.e. not at a DWP location), and have capability to deliver with compliance to ISO27001 standards 1.00% Nice to Haves Have experience of working within DWP or a comparable organisation within the last 3 years 1.00% Work Location The Supplier will be expected to be primarily co-located with the Customer on DWP sites based within Leeds and London, with the ability to augment resources using a remote model. Working Arrangments Onsite 4 days per week to work with and alongside existing team. A professional working day, hours and working pattern as needed to deliver the service in an efficient way as agreed by the DWP Product Owner and Delivery Manager. Security Clearance All resources must be BPSS (Baseline Personnel Security Standard) compliant or equivalent, and be prepared to undergo the clearance process to at least Security Check level. A small number of resources may need to obtain security clearance to Developed Vetting (DV) standard. No. of Suppliers to Evaluate 3 Proposal Criteria • Experience of your resources delivering major change via concurrent releases and Live Support services for a comparable customer 5.00% • Evidence of adoption and alignment to customer methodology and approach for offering suggestions for improvement 5.00% • Capability of standing up a team from the contract award date, managing resource scaling with related timeframes, managing staff rotation, managing attrition and managing under-performance 5.00% • Ability to gain, at pace, a detailed understanding of the solution to allow ongoing support and further development 5.00% • Evidence of identification of risks and dependencies and the approaches offered to manage them 5.00% • Inclusion of value for money proposals associated with provision of resources 5.00% Cultural Fit Criteria • Demonstrate flexibility (e.g. the ability to shift tasks to meet changing priorities) 4.00% • Demonstrate successful collaborative working as part of a multi-supplier delivery team sharing knowledge within the team 4.00% • Demonstrate simplicity (e.g. take simple steps to goals, do less but better, mitigate failures, talk straight with respect) 4.00% • Demonstrate excellence (e.g. set challenging targets, know users, deliver cost-effective high level customer satisfaction) 4.00% • Demonstrate enthusiasm (e.g. bring whole self to work, celebrate success, share passion and energy with others) 4.00% Payment Approach Time and materials Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers 1. When you state 'We expect suppliers to quote competitive rates given the volume of work described in the Request for Proposal.' We have not received any RFP... can you please clarify The indicative resources required per month (assumed number of full time equivalents averaging 20 working days per month) to deliver the services outline are as follows:Scrum Master 1 Big Data Architect 1 Hadoop Administrator 1 Hadoop Data Engineer 1Integration Architect 0.5 Linux Lead 1 Linux Admin 1 Senior Data Analytics Consultant 1Data Consultant 1 SME Support 0.25 2. Could you please confirm you require 10 people to work 3 days in Leeds and 2 days in London? Is there any flexibility around location? The notional resource requirement provided in answer 1 was 8.75 FTE was indicative only. Actual resources required will be determined by the supplier in response to the statement of work for a particular sprint.The requirement is for 3 days in Leeds and 2 days in London is the current expected working pattern, but this will be reviewed as the project develops. 3. Does a supplier have to bid for all the roles or can a supplier bid for a selection of roles? The Supplier must be able to provide services that will require all the skill sets listed. This is not an opportunity to deliver individual resources but to deliver a service that will include the skill set listed. 4. When DWP requests suppliers to complete the skills, experience and evidence document, does DWP intend to request suppliers to provide evidence for the specific people who will be doing the work or organisational capability? Evidence may be required to validate that the supplier has capability and capacity to deliver to the skill sets required during the term of this requirement. This may be in the form of CV’s 5. What are the expected timescales for deployment i.e. are the 10 resources expected to commence on the same date? The DWP are seeking to commence this contract on the 12th December, subject to completion of this process and internal approvals. The notional resource requirements are anticipated to be 8.75 FTE. 6. Can you specify the clearance level requirements for each role? The nature of the work means that we expect all resource provided by the supplier to either have, or be able to meet, Security Check level (SC) at a minimum. For some skill sets that involve access to particularly sensitive data, e.g. Linux Lead and Linux Admin, we would expect suppliers to propose individuals that would be willing and able to undergo enhanced vetting to DV level. 7. Can DWP confirm if any more information on requirements will be provided at the next stage of the procurement? The next stage will be based on the information and criteria already provided as part of this stage of the procurement. 8. Have the cloud provider / technology stack for "DataWorks" been selected? If so who are the providers ? Both the hosting provider and the technology stack have already been procured. The technology stack is based upon the information outlined, but includes Hortonworks Hadoop, OpenStack, Hive and Hue. 9. What data volumes do you expect to migrate onto the platform? (initially &ongoing) Initial data volume to be migrated is 6TB; at the end of 12 months we estimate data volumes in the region of 50TB 10. Once transitioned to the 'business-as-usual Live Support framework' support for this is planned to be delivered in -house team or by supplier? Business-as-usual Live Support would be delivered by an in-house team. 11. Is it envisaged that other systems will consume the data on the new platform or is it intended as an end point for reporting / analytics only? We anticipate that other systems would consume data on the new platform, however the detailed requirements would be outlined and agreed in advance with the chosen supplier in a statement of works. 12. What types of data will be migrated to the new platform? E.g. structured, unstructured (documents / media etc.) We are unable to disclose the precise data sources but this would be a variety of structured and unstructured data. This will be defined in the statement of works for the sprint. 13. Do you have a chosen provider to carry out IT testing during this Data migration project? This is out of scope of this requirement. We don’t have a requirement for dedicated testers. Work is fully tested within the DWP team and signed off by the user community before it is deployed.