Data Matching and Enrichment Service

Tender description

Summary of the work To provide a skilled, data matching team to enable: •Support and enhancement of the existing IBM Big-Match implementation to enable near to real-time probabilistic matching. •Expansion of data matching customer base, to match at scale and across multiple entity types. •Continuous improvement of matching, including match accuracy, monitoring and reporting Expected Contract Length 24 Months Latest start date Wednesday 31 August 2022 Budget Range • Any day-rates within the contract must not exceed maximum Home Office DDAT target rate card value. This will be available as part of the ITT pack made available to down selected suppliers. Why the Work is Being Done Data Services & Analytics (DSA) is a centralised service within DDaT’s Enterprise Services and has been at the forefront of efforts to re-imagine the department’s relationship with data. It has led the way in professionalising our approach to the acquisition and analysis of data, promoting dynamic insight-led and evidence-based decision-making tools and techniques. The Home Office Data Matching provision is a component of the DSA Data Integration Service and is fundamental to the DSA data platform and delivers on DSA’s vision to be at the leading edge of use of data analytics in Government. Problem to Be Solved The Data Matching services deal with a significant amount of data from a variety of sources which require ongoing work to integrate into a combined, searchable dataset. The Home Office is looking for a partner supplier with specialist knowledge and experience of delivering data matching solutions in IBM Big-Match. The Home Office has determined that it requires a supplier to provide this service with a view to bringing this in-house in the future. Who Are the Users Users are one of: 1. Data Scientists and Data Analysts within the Home Office, who explore and use matched datasets to provide analysis, insight, reporting and data products for their customers, to support business operations, policy making and decision making. 2. Users of the DSA product suite including products for entity searching, data enrichment and triage, which integrates with the Matching Product. 3. End business users, who use DSA data products to access the persisted matched data. Early Market Engagement An Early Market Engagement event was held on 16th February and an invitation was made available to all suppliers on the DOS framework. Slides will be included as part of the ITT pack for down selected suppliers Work Already Done The data matching provision is a mature cloud-hosted implementation of IBM Big-Match, and is continuously evolving to support new Home Office data analytics products. It offers capabilities for: •Bulk Cross Match – match/re-match all data held in the Person, Object, Location, Event database (POLE) and persist results into POLE. •Delta Match – match new data changes against the matched entities in POLE and persist results into POLE. •Wash against static data – search for or “wash” entities against a matched data set without persisting in POLE. •Wash against streamed data The matched entity types include Person, Object and Location. Existing Team The provision works in a multi-discipline agile team combining Civil Servants, suppliers and a specialist group of matching experts. The team run and monitor matching jobs, fix problems, and develop new matching components for around 30-40 data sets with an expected growth rate of around 20% per year. The supplier is expected to stand up a team made up of the specialist skills required to deliver the outcomes. Incumbent service provider will support the Matching service during a transition period of one-month overlap with existing contract. This will allow handover and smooth transition from one contract arrangement to the new. Current Phase Beta Skills & Experience • Capable of analysing complex data, developing recommendations and presenting this back in a clear, concise convincing form to senior stakeholders. • Expertise in managing, configuring and improving IBM Big-Match and other similar probabilistic matching technologies • Experience of working with sensitive data • Experience of working with matching large-scale data in a cloud-hosted Production environment • Expertise of matching data across multiple data sources • Successful delivery of a data matching provision, including monitoring and reporting (evidenced with examples of complexity where appropriate). • Ability to manage and drive value for money of the existing service • Demonstrate a deep knowledge and understanding of the technology stack including: Java, Python, SAMZA, KAFKA, HBase & SQL • Experience of design/assurance of high-level and low-level designs for data matching • Expertise in working with data based on the POLE model • Experience of delivering matching in a fast-data pipeline Nice to Haves • IBM Accredited • Experience of operating data matching in a cloud-hosted environment for similar organisations • Knowledge of legal and regulatory requirements relating to data matching in the public sector • Experience of Configuration of IBM Big-Match on other accredited environments • Expertise in optimising a data matching service and responding to user feedback, with an emphasis on Analytical Quality Assurance • Staff with SC clearance already in place Work Location Due to Hybrid working arrangements, work will be carried out from the supplier's own premises or home locations, attending the Home Office premises where the activity requires this, primarily in Croydon. The Matching Product Manager will confirm this. Working Arrangments The supplier is expected to provide a specialist team to deliver the outcomes. Following Covid, the Home Office have moved to Hybrid working arrangements, the supplier will align with these and work from their own premises or home location, attending Home Office premises when necessary, for example, face to face meetings or where security considerations, i.e. restricted on site access to specific systems is required. Travel to regional sites may be required, expenses incurred from travel outside of the M25/Greater London will be subject to Home Office Travel and Subsistence Policy. It is expected this contract will sit within IR35. Security Clearance All staff must hold or be prepared to undergo SC Security Clearance to operate on this project, (with willingness to acquire resources with NPPV3 and DV clearance) Additional T&Cs The work will be subject to the terms of the DOS 5 framework and conducted under individual SOWs, subjected to either Fixed priced, T&M or Capped T&M, as required. No. of Suppliers to Evaluate 4 Proposal Criteria • Team members and experience • Team Structure • Mobilise team • Flexible resources • Approach and methodology • Value for money Cultural Fit Criteria • Work collaboratively with all teams in Data Services and Analytics to ensure smooth handoffs in an environment of continuous improvements and changes • Ability to function effectively and collaborate in a multi-supplier environment • Working successfully in complex public sector organisations • Approach to proactive issue management, problem resolution and improving ways of working • Improve Health and Wellbeing • Increase supply chain resilience and capacity • Addressing Equality and Diversity Payment Approach Capped time and materials Evaluation Weighting Technical competence 55% Cultural fit 15% Price 30% Questions from Suppliers 1. Which supplier is the current incumbent The incumbent of the current Data Matching contract is Valcon Group UK Ltd