Data Architecture, Data Modelling, Data Governance, Data Quality, and Metadata Services

Tender description

About adding context and requirements test Pre-market engagement N/A Work done so far DfE are part-way through its journey of revising its conceptual and logical architecture to ensure industry best practices are followed, aligning with its policies and standards in creation of reusable patterns and components. DfE have various in-housing, modernisation and new service development projects which are at various stages covering conceptual, logical, and physical designs for operational and analytical data for DfE’s Data Directorate’s programmes (e.g., Data Transformation Programme & Learner Data Service) and digital services (Get Information About Schools (GIAS), Explore Education Statistics (EES), etc.). Metadata workstream has conceptualised and initiated construction of the DfE-wide Data Discovery Platform (DDP) to utilise the available metadata to enable discovery and access. The Data Governance function has undergone a recent transformation with refined internal policies, processes, and standards being established. We have also developed data ownership model and set out data quality profiling activities. Future work will focus on establishing governance forums, data management KPIs and a data maturity assessment. We also aspire to develop comprehensive training plans to enhance data literacy in DfE. The work for each workstream is at different lifecycle phases, with some being in early phases, whilst others are more mature in the process. Which phase the project is in Not applicable Existing team The current team is a combination of permanent and contract resources with varying levels of experience and expertise. Data architecture services support cross programme delivery through setting strategic design direction as well as supporting build team decision making and ensuring alignment with enterprise standards. The successful supplier will work within the Data Architecture and Governance Team, whilst collaborating with ‘virtual teams’ as part of service delivery. Teams are remote based and comprise of data service suppliers and permanent staff, and therefore capability and knowledge should be shared throughout the duration of the work. External services are currently providing a large team of data professionals. The successful supplier would need the capability to flex services accordingly in response to increasing or decreasing demand patterns throughout the contract term. Address where the work will be done No specific location as predominantly remote working. Expected the supplier will likely attend the office minimum 1 day per month. This could be any DfE office. Meet ups are typically in London, Leeds, Manchester, Darlington, Coventry, or Sheffield. Expenses will not be paid for these meet -ups. The primary location of work will be specified in each Statement of Work (SoW) and expenses cannot be claimed for travel to that location. Expenses can be claimed for travel to any secondary location of work as required and agreed. Expenses are capped to the value specified in the SoW. DfE Travel and Expenses policy applies. Working arrangements A SoW will be completed for each engagement, identifying primary location, working arrangements, and the expected deliverables and outcomes the supplier will be contracted to deliver. All work to be delivered to GDS service standard principles and meet DfE technical standards. Typically, 5 day working week (Monday to Friday) commitment and detailed working arrangements, including team location, size and make-up, will be agreed within each statement of work. Day-to-day delivery is governed by standard agile delivery techniques. These will be via scrum approaches and standard agile ceremonies will be undertaken. These will be monitored by the Delivery Manager and Product Owner, who will be responsible for prioritising backlogs in refinement sessions. There will also be regular architecture peer reviews and governance boards to ensure best practice and standardised patterns are adhered to. Occasional travel may be required to ensure good integration and skills transfer. IR35 determinations will be made for each SoW. However, we expect most requirements will fall inside of scope of IR35. Determination will be assessed on the working practises required for each SOW. Provide more information about your security requirements: Baseline Personnel Security Standard (BPSS) Provide more information about your security requirements (optional): DfE requires all supplier staff to have Baseline Personnel Security Standard (BPSS) clearance. There may be a requirement for some of the work to gain higher security clearance checks and / or enhanced DBS checks. The clearance needed will be detailed in the SoW. The supplier will need to be compliant with the DfE's Security Standards and respond to a security questionnaire. Latest start date 2023-07-24 Enter the expected contract length: 2 years Extension period: 1 year Special terms and conditions Bidders must respond to the opportunity using the response template found within the project attachments. Only essential and nice to have criteria require responses at stage 1. Bids received not having used the submission template will not be accepted Special terms and conditions In order to participate in this opportunity, DOS6 awarded suppliers should have followed all necessary registration steps through the Public Procurement Gateway, operated by Crown Commercial Service (CCS). Please follow instructions sent to you at the email address supplied during the DOS6 application process Special terms and conditions All Supplier Staff working on services in relation to this Contract will need to undertake, as a minimum, a BPSS security check. Security checks relevant to supplier staff will be indicated clearly through each Statement of Work issued under this Call-Off Contract. Any enhanced security requirements (e.g., SC etc) will be detailed in the Statements of Work' The Supplier shall ensure that no Supplier Staff who discloses that they have a Relevant Conviction, or who is found to have any Relevant Convictions (whether as a result of a police check or through the vetting procedure of HMG Baseline Personnel Security Standard or through the Disclosure and Barring Service (DBS) or otherwise), is employed or engaged in any part of the provision of the Services without the prior written approval of the Buyer. Subject to the Data Protection Legislation, the Supplier shall disclose the results of their vetting process, immediately to the Buyer. The decision as to whether any of the Supplier’s Staff are allowed to perform activities in relation to the Call Off Contract, is entirely at the Buyer’s sole discretion. The Supplier shall be required to undertake annual periodic checks during the Call Off Contract Period of its Staff, in accordance with HMG Baseline Personnel Security Standard so as to determine the Supplier Staff suitability to continue to Framework Schedule 6 (Order Form Template, Statement of Work Template and Call-Off Schedules) 5 provide Services under the Call Off Contract. The Supplier shall ensure that any Supplier Staff who discloses a Relevant Conviction (either spent or unspent) or is found by the Supplier to have a Relevant Conviction through standard national vetting procedures or otherwise, is immediately disclosed to the Buyer. The Supplier shall ensure that the individual staff member immediately ceases all activity in relation to the Call Off Contract, until the Buyer has reviewed the case, on an individual basis, and has made a final decision. Where the Buyer decides that a Supplier Staff should be removed from performing activities, as a result of obtaining information in relation to the Call Off Contract, or for any other reason, the Supplier shall promptly and diligently replace any individual identified. Special terms and conditions The location will be specified in the Statement of Work (Primary Location). Expenses cannot be claimed for travel to the Primary Location of work when in-office working is required. Some work may be UK wide, and travel may be required to other DfE offices. When travel to other DfE sites is required, all costs incurred must comply with the prevailing DfE Travel and Subsistence policy. Parking is not available on DfE sites. Should base location vary it will be outlined and agreed upon the individual statements of work Special terms and conditions The latest start date will be outlined in each individual statement of work. Except for in exceptional circumstances, and as agreed with the buyer Special terms and conditions Service provider resources must work within the United Kingdom unless agreed by the Department on an individual basis. Departmental equipment must not be taken abroad or access the departmental network whilst outside the United Kingdom unless agreed by the Department on an individual basis Special terms and conditions The Department’s standard approach to pricing mechanism will be ‘Capped Time and Materials’ (CTM) however we reserve the right to use any of the other approaches as listed in the call-off charges section Special terms and conditions General Information Standard Framework and Call Off Terms and Conditions will apply Suppliers must agree to all DFE security clearance policies and processes. Suppliers must comply with the Regulation (GDPR) Regulation May 2018; the (DPA) Act 2018, the Law Enforcement Directive and any subsequent amendments/changes to date including providing sufficient guarantees to meet the requirements of GDPR in line with Procurement Policy Note (PPN) 02/18 May 2018 which updates PPN 03/17 All Project-Specific IPRs (as defined) and the Deliverables shall vest in the Buyer unconditionally and immediately upon their creation Special terms and conditions Non-Solicitation Clause: 1.1 The Supplier shall not, either directly or indirectly, by or through itself, its affiliate, its agent or otherwise, or in conjunction with its affiliate, its agent or otherwise, whether for its own benefit or for the benefit of any other person solicit, entice or induce, or endeavour to solicit, entice or induce, any employees of the Buyer who are employed or engaged in the performance of the Services with a view to employing or engaging the employee of the Buyer during the Call-Off Contract Period and for a period of [9 months] thereafter. 1.2. Notwithstanding clause 1.1 the Supplier may employ or engage any employees of the Buyer which have responded directly to a bona fide recruitment drive either through a recruitment agency engaged by the Supplier or via an advertisement placed publicly by the Supplier (either in the press, social media, online or in trade and industry publications). Special terms and conditions Shortlisted suppliers invited to Proposal will be issued a Financial Viability Assessment Tool and a Security Questionnaire which must be completed. Are you prepared to show your budget details?: Yes Indicative maximum: 8500000 Indicative minimum: 0 Provide further information: These capability services are to be delivered over a contract period of 2 years and with an indicative contract value of approximately £8,500,000 (exc. VAT) for the full 2-year service delivery. Budget approval is subject to Cabinet Office spending controls, service assessments and DfE governance. Each SOW will outline the work required, associated budget and payment approach for each SoW will be capped time and materials. Suppliers will need to provide clear costs to enable tracking. DfE does not commit to any minimum spend on this contract. The cost mechanism will be Capped Time and Materials The supplier should be capable of flexing resource up and down to meet the Department's requirements throughout the contract term. Confirm if you require a contracted out service or supply of resource Supply of resource: the off-payroll rules may apply Summary of work Data Architecture and Modelling services will create conceptual, logical and physical layered designs to support reusable patterns and components. These artefacts may span across large scale transformations and modernisation projects. This will further include documentation of designs covering all architectural domains, including, but not limited to, ingestion, flow of near real-time data, transformation, validation, cleansing, and storage. The outputs from these services are also expected to produce option evaluation papers, alternation papers, reviews, presenting the findings to the departmental governance boards, and seeking approval to proceed. These services will also support live projects and programmes, working under tight deadlines in multi-disciplinary teams. Metadata Engineering and Management services will support the design and maintenance of the existing metadata catalogues and repositories to enable DfE to understand and be able to fully utilise its assets. Data Governance and Data Quality deliverables will support the development of policies and standards to support projects and BAU activities including, but not limited to, Data Ownership model and Data Quality profiling activities. All services are expected to provision training, coaching and mentoring for staff as and when needed throughout the duration of the contract. Where the supplied staff will work No specific location (for example they can work remotely) Who the organisation using the products or services is The Department for Education Why the work is being done The Department for Education's ambition is to reimagine its data collection services to allow greater insight into the sector by collecting richer data on a near real-time basis. This will allow the Department to gain valuable insight into patterns that occur within the sector to be able to provide timely interventions and policy changes that can help ensure better outcomes for our children and young adults. DfE also desires to reduce current system vulnerabilities to cyber-attacks; provide richer more accurate and timely data to aid the COVID recovery policies; free up admin time in the sector by reducing the collection burden; reduce DfE time spent in data validation and cleansing activity within the current process. This will help us achieve our strategic goal to offer ‘Data as a Service’, modernising our data collection services and laying foundation for good data governance practices; all of which will allow delivery of high-quality data into our estate enabling greater insight into the sector as well as enable data driven policy and operational decision-making. DfE Data Directorate are looking for a supplier who can provide data architecture, data modelling, data governance (incl. data ownership, quality, policies, standards etc.) and metadata services to support us achieve this ambition. The business problem you need to solve DfE’s data directorate have a large portfolio for data and it's getting bigger with more ministerial imperatives such as ‘Children’s Social Care’ and ‘Attendance’ being added. As Data Directorate get richer departmental data, there is a need to do more to match that data for insight e.g., Attendance and Children’s Social Care, and Attainment to see patterns for predictions for future interventions in policy. Agile services providing Data Architects and Data Modellers are needed to support these imperatives. Developing architectural designs, patterns and models for these core capabilities utilising the department’s processes for collecting, matching, processing, validating, and checking the department’s business critical datasets will be vital. There is a need to establish core capabilities for the Data Architecture and Modelling function in DfE Data Directorate to support operational and analytical data management and production across the Data Directorate’s portfolios of existing and new services. DfE requires creation and documentation of architectural patterns and designs through artefacts to support decision-making across business, application and technology domains. In addition to working on programmes and projects, data architecture and modelling services will involve development and enhancements of existing services such as in-year service enhancements. Metadata specialist services will facilitate establishment and development of metadata management within DfE. This will involve development of the enterprise data catalogue, dictionary and glossary to meet the Department’s needs. Metadata specialists will also be responsible for helping to deliver the existing BAU Metadata Management processes, ensuring that the technical team delivers at pace with the various requirements gathered from the business. Data Governance part of the contract will address development and enhancement of the growing data governance function within the Department including (but not limited to) development of data ownership/stewardship model, policies, standards, data quality management. To deliver these ambitions, DfE Data Directorate are looking for a supplier who can provide Data Architecture Modelling, Metadata and Data Governance (including Data Quality, ownership, policies, standards etc) services. First user type: Analyst First user type: Compliance Professional First user type: Data Operations Engineer First user type: External Users Enter more details about this user type: As an analyst, I need to be able to find and access trusted data through appropriate technological solutions for analysis and reporting purposes. Enter more details about this user type: As a compliance professional, I need to be able to agree and apply retention labels, monitoring, enforcing and reporting. I also need to limit the sharing and visibility of personal identifiable data to protect learners’ identities and only share when there is a permitted purpose. Enter more details about this user type: As a data operations engineer, I need to ensure the Databases are maintained so that we can share data within the department, across government, the sector and to the wider public domain. Enter more details about this user type: External users are the educational sector, other government departments consuming DfE data or sharing data with DfE. As an external data user, I need to be able to find relevant data, request and access it.