Professional Services Customer Insight Platform (CIP)

Tender description

Summary of the work Realise strategic goals outlined in Problem to be Solved. Build, run, maintain and optimise HMRC Customer Insight Platform live services, including high availability Transaction Monitoring infrastructure (Splunk, Scala Apps, AWS S3/RDS/EMR + Spark clusters, Kafka, Jupyter Notebooks). Operating Continuous Integration-Continuous Delivery (CI-CD) capability in public cloud using open-source technologies. Expected Contract Length 12 months, with an optional extension of up to 3 months (if required). Latest start date Monday 19 December 2022 Budget Range Estimated £6.7m (excl. VAT) Why the Work is Being Done Customer Insight Platform (CIP) provides a platform and relevant data services currently hosted in AWS (London Region) and integrated with all of HMRC's modern, customer-facing digital services. We are looking for a partner with expertise to further grow its scope, quality and functionality in response to changing user needs. CIP went live in February 2014 and is always evolving. The platform vision and roadmap have set out a new future direction, centred on using transactional data to build a comprehensive portrait and understanding of customer's interactions with HMRC. This will be done by continually improving the quality, comprehensiveness, usability, operability and security of our data and tooling as well as simplifying the path to Production. Furthermore, we need to facilitate wider use of our platform for machine learning based production features across HMRC. Problem to Be Solved Our partner will maintain and enhance HMRC's CIP platform capability to serve up audit content and services to internal customers, while ensuring an optimal user experience. In particular, they will be instrumental in ensuring we meet the below strategic goals for the coming year: - Understand security posture at any given moment - Confident correct security controls are implemented - Data securely accessed - Personal data deleted within required legislative time frames - Data back-up is fully redundant - Improved, secure Platform-as-a-Service offering for applications - Breakdown CIP monoliths - Improved, secure Platform-as-a-Service offering for data - Create secure maintainable, valuable data products on our platform - Data governance as code - Easy search that covers all our data - Provide network of customers, timeline and entities - Provide performant search across 7 years of data history - Create consistent customer experience across CIP - CIP data used by customers through other systems - Provide an opinion about our data - Understand data quality health at any given moment - Data reflects reality and is consistent over time - Data products have SLOs and are meeting them - Transform relationship with data providers to improve data quality Who Are the Users As a user of services on tax.service.gov.uk: I need HMRC to protect my personal identifiable information, and the public revenue from unauthorised access and/or attack - So I can safely conduct my tax obligations with confidence. So I can accurately and simply provide authentication and attribute information. As a HMRC Officer I need accurate, reliable and true information about the activity of customers using our customer facing services. To enable me to: Detect, prevent, and pursue unauthorised and/or criminal activity. Assist customers with their interactions with HMRC services. Analyse customer activity and behaviours. Early Market Engagement N/A Work Already Done CIP went live in February 2014 and is now in its third major iteration: 2014 - 1 x SME cloud provider 2016 - 2 x SME cloud providers (active-active) 2017 - 1 x hyperscale cloud provider The platform roadmap envisages CIP's continued evolution as a cutting edge PaaS, so there is plenty of transformative feature work in addition to the live running aspects. Existing Team HMRC expect the Service Provider to work alongside other suppliers, internal staff, including apprentices, as part of blended teams. There are 13 teams, typically comprising a subset of the following roles: Data Architect Infrastructure Architect Security Architect User Researcher Service Designer Data Engineer Data Scientist DevOps Engineer Software Developer Frontend Developer Product Owner Delivery Lead Business Analyst Quality Assurance Engineer / Tester 1st Line Support There is an incumbent Provider. Current Phase Live Skills & Experience • Demonstrable proficiency in writing quality code (esp. in Scala or Python) to solve business problems. • Demonstrable experience using AWS to solve problems involving cloud compute, storage, queueing, data ingestion, and data processing. Be explicit about which tools you have used. • Demonstrable experience of using varied DevOps technologies for use cases such as Infrastructure as Code, build and deploy, observability and monitoring. Be explicit about which tools you have used. • Demonstrable experience of building and running a Platform as a Service hosting (web/API/etc.) applications at scale (i.e. 100s of servers). • Demonstrable experience collecting large numbers of interaction audits (300 million daily) from many sources. Including a range of different software-based collection methods, and how you tracked/minimised loss. • Demonstrable experience in ensuring data integrity by mitigating against tampering to enable use as evidence in a judicial setting. • Demonstrable experience in using tooling to aggregate, process, transform, and query petabytes of data. Be explicit about which tools you have used. • Example of how you have processed and used first, second, and third party data to profile customer interactions with risk scoring. • Demonstrable experience in administration of search, analysis, and management tooling for logs/audits (E.G. ELK, Opensearch or Splunk). • Demonstrable experience in normalising complex nested JSON data into flattened SQL data. • Demonstrable experience developing custom user interfaces that enable in-depth search and analysis of terabytes of data. • Demonstrable experience in using data tooling (and approaches) to solve problems involving data governance and standards, AI/ML, data cataloguing. Be explicit about which tools you used. • Demonstrable experience of building and working within a Data Mesh environment and the benefits it provided. • Fully outlined sensible and mature process in how you would translate business problems and user needs into live features. • Fully outlined ideal deployment process from code all the way to user need fulfilment for software. • Fully outlined ideal deployment process from code all the way to user need fulfilment for ‘data as a product’. • Demonstrated experience of how you have worked collaboratively with other development teams and or team members from either other suppliers or in-house. • Fully outlined processes/ways of working to ensure you only deliver the highest quality of code. Nice to Haves • Significant, similar Agile Delivery contract with a public sector body or in a highly regulated environment. • Superset administration and performance tuning. • Keycloak administration. • Data warehousing technologies including Redshift, Snowflake and Google BigQuery. Work Location Mainly remote but with two official office locations detailed below required for occasional face-to-face meetings. Primary London location: 14 Westfield Avenue, Stratford, London, E20 1HZ Primary Yorkshire and the Humber Location: 7 and 8 Wellington Place, Wellington Street, Leeds, LS1 4AP. Working Arrangments The supplier will be required to be able to perform a mixture of remote working with co-locating alongside existing teams - mixed teams comprising both internal staff and other contractors, 5 days per week. Occasional travel may be required to other HMRC Delivery Centres. Expenses will be paid in accordance with HMRC travel and subsistence policy. We expect the successful supplier to provide upskilling to permanent staff to increase internal capability. Security Clearance All supplier contractors must hold or be able to achieve Security Clearance (SC). These posts are NOT reserved to UK Nationals. To be suitable for security clearance contractors MUST have been continuously resident in the UK, without significant breaks, for 5 years. Eligibility for clearance can be found below: https://www.gov.uk/government/publications/national-security-vetting-advice-for-people-who-are-being-vetted Additional T&Cs All personnel engaged in the provision of this service must have a minimum of three years experience in their designated role/skill specialism, with an expectation of five years minimum experience for lead roles. DOS T&Cs. Additional HMRC specific terms in relation to tax compliance among other things will be added to the call off contract" No. of Suppliers to Evaluate 3 Proposal Criteria • Demonstrate your technical competence and high quality of your solution to the proposal that will be provided at stage 2. • Demonstrate a quality and holistic/sensible overall approach and methodology. • Demonstrate consideration for key project stages and outline your timeframes to complete various streams/facets. • Demonstrate a strategy to fulfil business requirements and a quality approach/solution that meets user needs. • Demonstrate knowledge, experience and documentation sharing. • Demonstrate how you will support the business/in-house teams/permanent staff capability in operating the delivered solution going forward. • Demonstrate and evidence quality control process/strategy in delivering the overall proposal. • Demonstrate how you will track, monitor, relay to and deliver across both the business and technologist tiers. • Demonstrate a quality communication strategy, approach and frequency both with the business and technologist tiers. • Demonstrate DevOps experience and feature release cadence. • Demonstrate value for money and analyse cost during service delivery. Outlining how you drive efficiencies, reduce total cost of ownership in all aspects of design/development in your solution. • Demonstrate your consideration of working with a legacy estate in complex mixed ecosystem environments and your approach on how these would be transformed into modern customer centric solutions. • Describe your approach to managing risks & dependencies in regard to the proposal provided, focusing on (a) Information Security risk management, (b) Project risk management. • Demonstrate your understanding of Transaction Monitoring capabilities and application in the business area. • Demonstrate knowledge of the General Data Protection Regulations and details of any challenges this may provide in delivery of capabilities and how this will be addressed. • Provide an example of an existing team, describing how you would ensure the right resource is available and demonstrate your capability to ‘scale up’ quickly to meet urgent requests. • Demonstrate a quality delivery strategy and how you adhere to the 'Principles behind the Agile Manifesto' and particularly comfortable with Kanban and Lean methodologies within a product-centric environment. Cultural Fit Criteria • Demonstrate how you will put the Buyer at the heart of delivery by delivering value early and often. • Demonstrate reflection on becoming more effective by adjusting/improving behaviours, operating a no-blame culture, taking responsibility for work whilst pairing/per reviewing by default, encouraging people to learn from their mistakes. • Demonstrate how you will collaborate and work effectively alongside permanent staff / in-house teams/suppliers/contractors, with considerations for (a) multidisciplinary teams (b) mixed ecosystem teams (c) multi-site teams. • Demonstrate innovation, promoting applicable ideas and suggestions. • Fully outline your recruitment process for the screening/ commissioning of staff/contractors, in terms of how this works and how calibre/quality is guaranteed. • Demonstrate how your organisation will provide additional and/or value-added activities when acting in the role of a partner. • Demonstrate how staff at all levels will communicate effectively and can demonstrate the soft skills to communicate effectively with the business and techs. • Social Value Question - Tackling Economic Inequality MAC 3.5: Demonstrate action to identify and manage cyber security risks in the delivery of the contract. Payment Approach Time and materials Assessment Method Case study Evaluation Weighting Technical competence 50% Cultural fit 20% Price 30% Questions from Suppliers 1. What is your expectation on onboarding/transition timeframes to a new vendor? We aim to have the supplier that wins the contract start somewhere around the 7th or 14th December (indicative & subject to change). They will have from this period up to 5th January 2023, when the contract ends with the incumbent supplier for onboarding/transition as required, working around limitations of the Christmas/New Year period. 2. What is your expected resource capacity for this programme Over the course of the contract, we anticipate equivalent resource capacity below:4xProduct Owners5xDelivery Leads0.5xService Designers0.5xData Architects0.5xTechnical Architects0.5xSecurity Architects1xUser Researchers0.5xContent Designers1.5xBAs0.5xTechnical BAs1xSoftware Engineer Technical Lead1xFull-Stack Engineer Technical Lead2.5xFull Stack Engineers0.5xFront-end Engineers3.5xSoftware Engineers4xQAs0.5xDevOps Engineer Technical Leads5.5xDevOps/Platform Engineers0.5xDevSecOps Engineers2.5xData Engineer Technical Leads4.5xData Engineers0.5xData Scientists0.5xSearch Analysts0.5xData Analysts0.5xData Analytics Engineers (GA)0.5xDevOps Engineer (OoH)0.5xData Engineer (OoH) 3. Please breakdown the requirements by roles for initial start date of 19th December? From 19th December, we will at least need the below: 2 BAs 5 DevOps Engineers 3 Data Engineers 0.5 Data Scientist 2 Full Stack Engineers (Scala) 1 Software Engineer (Scala) 2 QAs (Quality Engineer/Analyst with experience in Scala) 1 Delivery Lead 1 Full Stack Engineer Tech Lead (Scala) 4. What are your current challenges in delivery? Examples of current delivery challenges:Expanding to new user bases where we don't understand need/alignment with current Product offerings.Expanding to consuming new very complex/non-standardised data sets needing interpretation/normalisation.Replacing/migrating off legacy/existing monolithic solutions to new solutions.within contractual restrictions/timelines.limiting complexity.Catering for diverse features from diverse users - some we only learn about upon feature deprecation.Prioritising required functionalities.Managing expectations where direct replacement of existing features impossible.Coordinating with users to change how we work together and their working models to reflect changing landscape of our technology. 5. What is the DevOps toolset that is being used for code deployment? GitHub, Jenkins pipelines and Groovy, Terraform, Packer, Inspec/Test Kitchen, Artifactory, Python, Bash, Docker, ECR, ECS, S3, lbManager and bespoke Python tooling as management plane for container deployment across EC2/ASGs, Ansible in exceptional circumstances. 6. What tools are used for data cataloguing and curation? DataHub, Kafka, EMR/Hive, KeyCloak, Aurora Postgres, Docker, AWS OpenSearch, Splunk. We curate data working with our data-producing client upstreams act as owner of data quality, aligned with provided Schema Registry (in Data Mesh). We use bespoke microservices in our data pipeline, and software-based collection methods described below. These review ingested data to verify conformation to required standards, sending automated responses back to the data producers regarding non-compliance, and allow developers to verify against such standards directly. HMRC are also exploring other public data sources and making agreements with 3rd party/Governmental providers on the quality/standards of such data. 7. Can you please let us know the technology stack used for user interface development? Scala & sbt & Play Framework, Python, Docker (e.g. for hosting 3rd party software with relevant user interfaces) and local internet browser, NodeJS & npm, Selenium and Cypress testing, user research methods and other accessibility testing tools. 8. Could you please expand upon what is meant by software-based collection methods? Using bespoke microservices (typically Scala/Python, or upon relevant open-source technologies) and other bespoke/open-source tooling (proxies/queues/message bus/etc. - e.g. Kafka/EMR/RabbitMQ/etc.) collectively used to ingest data from data sources (from public internet, trusted networks, other Government data sets, etc.) into a data pipeline, carrying out interpretation/filtering/sanitisation/normalisation/transformation/censoring/virus-scanning/validation required before shipping into data storage used as backends for other services that provide insight of HMRC customers to other clients. Ingestion may be on a push model (i.e. audit data of a particular format received from a downstream client) or a pull model (microservice fetches/requests data from upstream), or a combination. 9. Are we able to partner with another supplier and respond to the 100 questions? If so, does the partner have to be DOS5 supplier? We are open to more than one supplier; if they work as a collaboration, we still expect a single set of responses, and all members should be a DOS5 supplier. Else, one supplier should act as the primary supplier and own the relationship and be responsible for commitments with HMRC including the response, and sub-contract out appropriately.The number of essential and nice to have questions is within the permissible range allowed. This is to ensure sufficient information is available to all potential suppliers to ensure a fair and open competition. 10. Please clarify your expectations on onboarding of SC clearable and SC cleared staff, and when they become chargable to HMRC, please confirm the end-to-end process and associated Staff that are not SC cleared will still be able to charge when they come to HMRC with the intention to get SC clearance as soon as possible. They will be able to develop and work on pre-Production/development systems in the duration they do not yet have SC. The supplier will be expected to work with HMRC and the relevant Government departments in processing the SC clearance process as required (collecting evidence and sending back relevant forms). Ideally there would be some level of pre-screening to validate they would be eligible and likely to pass SC clearance. 11. If HMRC was to sponsor new SC’s, what is the current end-to-end process and associated timelines from submission of request through to log-on credentials/equipment being issued and being billable? See above and process outlined here. The overall SC clearance may take months depending on whether the applicant currently or previously has had SC clearance and their recent living arrangements. However, the applicant will still be able to work on pre-Production/development systems in the meantime using devices that they provide themselves and relevant network access that we will provide, and may provisionally be able to work on Production systems under a grace period and specific conditions whilst their SC application is in progress subject to approval and risk being accepted by the business. 12. Who is the incumbent provider please? The incumbent supplier is Kainos. 13. In relation to the following question: ‘Fully outlined ideal deployment process from code all the way to user need fulfilment for software. Can you please confirm how you would like suppliers to respond to this? I.e do you want evidence of where we have done this with a client, or do you want suppliers to describe the process? We would like the supplier to describe how they would carry out the process for a client in an ideal world (without restrictions). If they were able to already do this process with a former/current client, such an example can be used as evidence, but they should include what they would have done differently should the implementation have deviated from the ideal due to project restrictions/limitations. 14. In relations to the following question “Fully outlined processes/ways of working to ensure you only deliver the highest quality of code.” Can you please confirm how you would like suppliers to respond to this? I.e do you want evidence of where we have done this with a client, or do you want suppliers to describe the process? Please provide a short summary of 'how/what to do', but back that up with evidence of where these theories have been put into practice. 15. Question 8 – could you clarify what you mean by first, second and third party data please? First-party data is data collected by HMRC. Second-party data is data collected by trusted partners of HMRC such as other Government departments with relevant data or parties commissioned by HMRC that provide data requested by HMRC. Third-party data are data sets that were collected independently of HMRC involvement/relation that typically is aggregated from many sources and not directly collected by the third party.