CAID Combined AI Classifiers Harness C21178

Tender description

Summary of the work Deliver a harness that integrates the predictions of 2 or more AI classifiers of Indecent Images of Children (IIoC) and produces a final more accurate outcome based prediction. Where optimal engagement is made of human experts to further enable the best outcome. Expected Contract Length up to 2 Years. Latest start date Monday 9 August 2021 Why the Work is Being Done The aim of the Child Abuse Image Database (CAID) is to continue to provide UK law enforcement with an image database and associated tools to aid investigations into indecent images of children. Over the coming years, the CAID Transformation Programme aims to expand CAID in line with policing priorities and provide new capabilities utilising AI and advanced automation to examine and categorise large volumes of images and videos to streamline the investigation and prosecution of offenders and help to safeguard children from abuse. Problem to Be Solved During a child sexual abuse investigation all files on a device not previously known to CAID need to be assessed for illegal imagery of children. An existing AI is already deployed that aids this process. We seek a supplier to deliver a combined AI classifier harness to bring together the predictive outputs of 2 or more classifiers to produce better classification outcomes with significantly less burden on officers. Who Are the Users The users of CAID include UK Police Forces, the National Crime Agency and the Internet Watch Foundation. Law Enforcement Organisations need to assess illegal images and videos of children so they can safeguard victims of child sexual exploitation and abuse and prosecute offenders. Once the assessment is complete victim identification officers review any new material. As an assessor in a child sexual abuse investigation I need to assess every previously unknown to CAID image and video on a seized device so that the offender can be correctly prosecuted to the level of their crime. As a victim identification officer in a child sexual abuse investigation I need to have access to correctly assessed images and videos, identified as new to CAID, so that I can help identify any new victims in order that they can be safeguarded. Work Already Done The CAID Transformation Programme has already procured an AI Classifier to assess indecent images of children (IIoC). The “1st AI Classifier”. There is a concurrent DOS Outcomes advert for the CAID programme entitled, 'CAID 2nd AI Classifier C21177' published at the same time as this DOS Outcome. Existing Team The CAID programme comprises a portfolio of projects across strategy and transformation and technical release workstreams. The service will primarily be client-side and operate in partnership with Home Office civil-servants, specialist contractor team members and policing. Current Phase Beta Skills & Experience • Experience of working with law enforcement or government and, with this, the knowledge of expected controls and governance arrangements • Experience of image analysis and the appropriate use of ML in relation to image analysis • Delivery of through life ML based capabilities, integrating AI algorithms to existing services, best hosting approaches for AI and the optimal approach to supporting and upgrading live AI systems • Experience of evolving and upgrading legacy systems, decoupling boundary interfaces to enable new capabilities to be integrated one new service at a time. • Experience of providing project leadership and support for projects with multiple and complex dependencies • Proven knowledge and expertise in data governance for the public sector where data sensitivity and security are a major consideration • Experience in providing a flexible resource profile that can efficiently adapt to the demands of the programme, meeting short- and long-term resourcing needs • Evidence of seeking the best capabilities available in the industry, looking to build on the best of open-source tech or potential consortium delivery with a specialist supplier • Experience of continuous delivery and how best to break down a project into tangible delivery steps, monitoring benefits realised and implementing continual improvement practices throughout • Experience of maximising knowledge retention and skills transfer, including advancing knowledge sharing, documentation and the use of enterprise repositories • Manage team effectiveness to deliver outcomes and necessary interactions with the broader programme • Experience in integration technologies, data processing pipelines and the load balancing of data processes Nice to Haves • Have capability in both the analysis of still imagery and video. • Human machine workflow optimisation • Specific experience in policing and/or CAID Work Location The CAID Team are based in Croydon. The team must be able to onboard and work remotely with occasional travel to other UK locations that include London, West Yorkshire and Croydon. Working Arrangments The supplier will provide team members to deliver and manage the required outcomes. Team members will be protected from exposure to the imagery stored on CAID but discussing the subject matter could be still distressing. The team must be flexible to work remotely or from Home Office locations within the M25 area. Some UK based travel may be required. Pricing will be based on a hybrid of T&M, fixed price and milestone payments. Day rates must be aligned with the Home Office-mandated rate card. Security Clearance The service requires resources who either have or are prepared to apply for and obtain SC or NPPV3 (Non-Police Personnel Vetting level 3) clearances. Additional T&Cs The Home Office expect to own all project specific foreground IPR including the source-code for the Combined AI Classifiers Harness. Any supplier bidding for the CAID Combined AI Classifiers Harness C21178 cannot also bid for the DOS Outcomes advert for the CAID. programme entitled, 'CAID 2nd AI Classifier C21177' published at the same time as this DOS Outcome. The Home Office will not pay expenses for travelling from Supplier work locations to Home Office or other locations within the M25 area. No. of Suppliers to Evaluate 5 Proposal Criteria • Approach to flexible working, assigning multi-skilled personnel to meet CAID goals. • Leverage 3rd party commercial, open-source and in-house capabilities in delivery • Approach to sharing knowledge and skills transfer with the customer. • Approach to through-life product management - in particular ambition toward evolving the capability as a long-term product. • Identification of risks and dependencies and offered mitigation approaches. • Technical competence in a range of relevant technologies; use of AI, machine learning, image analysis, data fusion, ensemble methods, running AI algorithms as services, APIs, software integration. • Demonstrate how you would understand CAID user and stakeholder requirements to deliver capability that meets business needs • Value for money • Skills and expertise in human factors, including any research or experience in combining AI with human expertise. • Overall technical solution • Source code handover Cultural Fit Criteria • Experience of working with public sector organisations • Share knowledge and experience with other team members • Genuine enthusiasm for tackling child sexual abuse • Work as a team across our organisation and with other suppliers • Be adaptable and flexible and able to deliver against changing priorities • Be inventive and look to build a solution that utilises the best of human experts with AI • Proactive issue management, problem resolution and improving ways of working • Be transparent, have a no-blame culture and encourage people to learn from their mistakes • A commercial desire to use this opportunity to develop or enhance a product that helps combat child sexual abuse, where consequently the product will be improved and evolved over time. • An operational staff wellbeing policy detailing the support available to staff involved throughout the length of the project Payment Approach Fixed price Assessment Method • Case study • Presentation Evaluation Weighting Technical competence 65% Cultural fit 15% Price 20% Questions from Suppliers 1. Are the existing classifiers crown IP and can the details of the algorithms be shared to allow an appropriate fusion method to be defined/developed? The Home Office has limited foreground IP on the existing classifier and only some general implementation details. Fusion methods that require deep integration should be ruled out. 2. Please can you provide the “Home Office mandated rate card" - Architecture: £600(SFIA 3); £750(SFIA 4); £950(SFIA 5)- Data: £550(SFIA 3); £650(SFIA 4); £900(SFIA 5)- Delivery: £500(SFIA 3); £700(SFIA 4); £850(SFIA 5)- Engineering: £450(SFIA 3); £650(SFIA 4); £800(SFIA 5)- IP Ops: £450(SFIA 3); £650(SFIA 4); £800(SFIA 5)- Product: £540(SFIA 3); £680(SFIA 4); £800(SFIA 5)- QAT: £460(SFIA 3); £575(SFIA 4); £800(SFIA 5)- UCD: £425(SFIA 3); £575(SFIA 4); £625(SFIA 5) 3. Will the harness work start after the 2nd classifier is built? No it will run in parallel with the 2nd Classifier. A minimum specification of the classifier output will be provided. 4. Is the objective of the harness to improve performance and also to deploy the classifiers? The harness will orchestrate the classifiers in that it will call them and manage that process, such as any asynchronous call-backs, restarts and where applicable handling concurrent instances of the classifiers. It will combine the outputs with the aim of improving classification performance. 5. What will the deployed status of the first and 2nd classifier be when the harness work kicks off? The 2nd Classifier will be under development.The 1st Classifier will be deployed as integrated into some policing classification workflows and associated tool variants. Other integration options for the 1st Classifier may still be at a pre-deployment status. 6. Will the output of the harness project dock into existing software used by officers? Effectively yes. The harness logic should look to best incorporate human input and oversight into the classification process. If existing software is deemed lacking in how it elicits such human interaction then extending the scope to include elements of user interface change would be considered. 7. Will the successful supplier have access to the outputs of the classifiers on a reference dataset? And how big is this reference dataset? The successful supplier will have access to the specification of the output of the 1st Classifier and will have opportunity to contribute to the definition of the output of a generalised Classifier.Classifier output reference examples and evaluation performance metrics can be provided.