Department for Education Data Engineering Capability

Tender description

Summary of the work We are looking for a development partner to help to continue to deliver our Collection service digital transformation, with a mix of continuous improvement to existing systems and delivery of new systems though GDS phases. The supplier will work alongside the in-house digital team and cover all digital service roles Expected Contract Length Two years Latest start date Thursday 14 May 2020 Budget Range Indicative budget could range up to a approx maximum of £4.5m consisting of multiple Sow's, but that scope could expand or contract. DfE does not commit to any minimum spend at this point. A pricing template will be given at the proposal stage to those suppliers we shortlist. Maximum day rate will apply for the following roles Software Engineer (Full Stack Developer) £800 Data Engineer (Back End Developer) £800 Infrastructure Engineer (Azure DevOps) £800 QA (Automation, data and software) £750 We would not expect any roles to exceed SFIA level 6. Why the Work is Being Done Department for Education (DfE) Data Directorate are looking for a supplier who will be able to provide all the required Data Engineering roles for delivering services, compliant with Government Digital Standards. This is a partnership on a range of digital and technical projects as part of the DfE’s Data Transformation Portfolio. You will provide expertise and specialism to boost capacity and enhance capability. You will work alongside our in-house teams. Each project will have SoW that will outline the work package and the specific capabilities that may be required Problem to Be Solved The DfE Data Directorate is looking for a supplier to support a range of user led data and digital projects, including both existing and new projects, from inception through to live. The supplier will work with blended teams, using agile to deliver outcomes based on user needs and, where applicable, government service standards. Each project will require a different mix of Service and Data Delivery capabilities and may not require the full complement of capabilities. Who Are the Users Internal Users Analysts, Policy team, Operations, Service Delivery team As a data collector working in Data Engineering and Operations division, I need to setup data collection in line with department policy Early Market Engagement Not applicable Work Already Done There are a number of projects and programmes that require data engineering resources , these range from the improvement of existing developed systems to transformation leading to new systems. The expectation is that the supplier will be able to provide the capabilities required to support end to end delivery from discovery through to live Existing Team The existing teams are comprised of multi-disciplinary agile delivery skills. These teams are involved in continual product improvement as well as providing brand new digital services. Typical team size is up to 12 with a skills mix such as • Product Owner • Service Manager • Scrum Master • Technical Business Analyst • Solution Architect • Data Engineer • QA • Data Analyst • Infrastructure Engineer • Security Architect Current Phase Not started Skills & Experience • Experience in designing and building data ingestion, transformation, validation, cleansing, migration and storage capabilities. Utilising industry standard tools for end to end development for data platforms projects for large organisation. • Experience in using agile tools and techniques to develop front-end web applications (full stack development), data validation and data storage in non-relational databases • Experience of end-to-end solution development and delivery of multi-strand digital and data programmes, setting up DevOps functions, building CI/CD pipelines and data testing capabilities within large organisations • Proven resourcing and engagement model to ensure the right quality of the team, ability to rapidly scale up & flex resources. • Experience of coaching & development of internal capability and upskilling of resources in large organisations • Experience of establishing and operating testing processes for data and software testing in large, complex, data intensive organisations within central government including regulatory and compliance requirements • Experience of working well in large public sector organisations, across organisational boundaries, and with multi-agency departments, ideally in government Nice to Haves • Experience of end-to-end solution development and delivery of multi-strand digital and data programmes, setting update DevOps functions, building CI/CD pipelines and data testing capabilities within large organisations • Experience and understanding of the Education sector. • Experience and understanding of educational datasets Work Location Primary location will be Bishop gate Office, Darlington. There is the potential for some work in Sanctuary Buildings, London and Sheffield. Any travel to other sites to be approved and will follow the DfE's Travel and subsistence policy. The primary base will be detailed in each Statement of Work (SoW) and expenses will not form part of the cost unless the supplier is asked to work from a secondary location instead during the execution of the work relating to the SoW. e.g. If SoW primary base is Darlington then no expenses will be paid for Darlington Working Arrangments This will be a 5 day per week (Monday to Friday) commitment and detailed working arrangements, including team location, size and make-up, will be contained within each statement of work. Day to day delivery will be governed by standard agile delivery techniques. These could be either a Kanban or a scrum approach (depending on the work) but the standard agile ceremonies appropriate to those will be undertaken. These will be monitored by the Delivery Manager and the Product Owner will be responsible for prioritising backlogs in refinement sessions. Security Clearance The successful supplier must be able to demonstrate that all proposed team members have been subject to Baseline Personnel Security Standards checks. However higher levels of Security Clearance maybe required dependant on work agreed. No. of Suppliers to Evaluate 5 Proposal Criteria • Demonstrate how you as a supplier have been fully accountable for delivery of a project or outcome and successfully delivered to specification, time and with value for money • Delivering business value – making sure business requirements are understood, validated via positive challenge and delivered against while understanding the value being generated (i.e. business intent) • Design for minimising build and run costs • Supplier teams are expected to use DfE’s collaboration software like Azure DevOps and Teams • Experience working with SQL server, Azure Blob, CosmosDB, ETL tools including Informatica PowerCenter/DEI, Informatica Data Quality • Ability to scale up team and flex resource when needed on-site Cultural Fit Criteria • At Data Engineering Division we want to grow our in-house skills: How would you support this? What skills, time and expertise would you make available and where? • Experience of collaboration with multiple organisational stakeholders with the aim to achieve a single objective. • Evidence how you would deliver in a multi-supplier environment: Ensuring DfE’s interest is reflected in the approach to work and outputs delivered, demonstrating openness and transparency in engagement • Demonstrate how you will ensure inclusivity and collaboration • Experience of transferring knowledge to permanent staff within a client organisation • Able to explain technical things to non-technical people, and non-technical things to technical people Payment Approach Capped time and materials Assessment Method • Case study • Work history • Reference • Presentation Evaluation Weighting Technical competence 55% Cultural fit 20% Price 25% Questions from Suppliers 1. What are the procurement timelines? Action DateIssue DOS stage 1 20th MarchDOS Stage 1 return 3rd AprilDOS Stage 1 evaluation and shortlisting 3rd April – 9th AprilDOS Stage 2 issue 9th AprilStage 2 return 24th AprilSupplier presentations 28th April (on line)Stage 2 final scoring 1st MayStage 2 award 1st MayStandstill period 1st May – 12th MayContract signatures 13th May 2. The ad says the main location is “London”, but the address for work says "primary location will be Bishop gate Office, Darlington" with "potential for some work in Sanctuary Buildings, London and Sheffield". Are many SOWs likely to be regularly needed on-site in Darlington (post COVID-19)? Darlington will be the main location with potential for some work in Sanctuary Buildings, London and Sheffield 3. Is it the case that all SOWs will be based in Darlington, but the teams may need to occasionally travel to Sheffield or London for meetings? The majority of SOW's will be located in Darlington, however some work packages may require a team based in either Sheffield or London 4. At stage 2, will suppliers be able to offer different rates for the different primary locations, or at least a different rate for London-based SOWs? Yes, we will ask for different rates for each primary location 5. Would the Dept consider accepting ‘blended’ day rates within future SOWs? Blended rates would not be the preferred option, however different rates per location could be an option. 6. Given the current working from home guidance from the government due to COVID-19, will the Dept allow remote working if travel restrictions are still in place when the first SOW is to be implemented? Yes, will be insist on following the current policy of working from home, and adhere to all Government policies in relation to home working, using where possible , collaboration tools and video conferencing. 7. We note the roles listed under the ‘Budget range’ section and their respective maximum day rates. Are there any other roles that might be included within future SOWs under this contract? We have included all anticipated roles, there maybe ad hoc requirments for additional roles, but these will not be numerous. 8. Is the primary location for this work Darlington or London? The DOS advert is confusing as it says ‘London’ as the main location in the Overview section – but Darlington as the main location in the ‘Address where the work will take place’ section. And what is the expected work split between Darlington, London and Sheffield? Darlington will be the main location 9. For the developers, could you elaborate on any particular tools skills etc that would be essential or preferred? Infrastructure/DevOps - Cloud infrastructure experience, Azure DevOps preferableData Engineers - ETL skills, experience of Informatica power centre/DEI, SQL skills, shell scripting skills Testers - Automated testing, manual testing - Data testing skills -SQL, Software testing skills - test cafe, javascript, selenium 10. Given the current situation regarding the Coronavirus is there scope to carry out the work from remotely, making use of collaboration tools and Video Conferencing for example? Please see the previous answer 11. What is the current technology stack? What are database & reporting/analytical services currently present & envisioned? Azure Dev Ops stack; Informatica, Databricks, Data Factory for ETL dev; Cosmos DB, javascript- for application development 12. Is there any incumbent supplier? Multiple existing contracts are vended by DFE. We are at the start of a major transformation, so we are engaging with a wider pool of venders to provide services 13. Are Informatica tools used in conjunction with the Azure technologies listed? Yes, please see previous response. 14. Can some or all of the work be carried out remotely, given the current social distancing instructions? Yes, please see response to previous questions. 15. Is it the Department’s intention that future SOWs be based on deliverables and outcomes? And potentially be fixed price / capped T&M? Or is the intention to issue SOW’s that request supplier resources to augment Data Directorate in house teams on a T&M basis? Future SOWs will be based on deliverables and outcomes and have a value cap. 16. The following question is listed twice: Experience of end-to-end solution development and delivery of multi-strand digital and data programmes, setting up DevOps functions, building CI/CD pipelines and data testing capabilities within large organisationsCan you please confirm what should be entered in the 'nice to have' version if we have already answered the 'essential' skill version If this question has been asked twice, then suppliers are able to give the same response to the duplicate question 17. Can you provide details of the current technology stacks that were used to deliver the existing applications, to include programming languages and frameworks? Please see response to previous question 18. The third question in Essential skills and experience and the first question in Nice-to-have skills and experience are the same; Experience of end-to-end solution development and delivery of multi-strand digital and data programmes, setting update DevOps functions, building CI/CD pipelines and data testing capabilities within large organisations. Could you please advise this is correct? If yes, do you require different answers to score full marks? We do not require a different response to both questions, the same answer can be provided for both questions 19. Can we avail of remote near shore to support this need? Under the current circumstances, all work will be carried out off site, however, in the future, there will be requirements for attendance at DfE offices. 20. What type(s) of applications have been developed so far, e.g. websites, APIs, mobile apps, dashboards? DfE is going through a transformation journey and would be initiating multiple programmes for data and digital transformation which would include areas on website, data and API’s and dashboard.