CAID 2nd AI Classifier C21177

Tender description

Summary of the work Deliver an AI based classifier of indecent imagery of children (IIoC). Expected Contract Length Up to 2 years Latest start date Monday 9 August 2021 Why the Work is Being Done The aim of the Child Abuse Image Database (CAID) is to continue to provide UK law enforcement with an image database and associated tools to aid investigations into indecent images of children. Over the coming years, the CAID Transformation Programme aims to expand CAID in line with policing priorities and provide new capabilities utilising AI and advanced automation to examine and categorise large volumes of images and videos to streamline the investigation and prosecution of offenders and help to safeguard children from abuse. Problem to Be Solved During a child sexual abuse investigation all files on a device not previously known to CAID need to be assessed for illegal imagery of children. An existing AI is already deployed that aids this process. We seek a supplier to deliver the 2nd AI Classifier as part of this programme of work to produce better AI prediction outcomes when combined with the existing 1st AI Classifier to enable a solution that has significantly less burden on officers. Who Are the Users The users of CAID include UK Police Forces, the National Crime Agency and the Internet Watch Foundation. Law Enforcement Organisations need to assess illegal images and videos of children so they can safeguard victims of child sexual exploitation and abuse and prosecute offenders. Once the assessment is complete victim identification officers review any new material. As an assessor in a child sexual abuse investigation I need to assess every previously unknown to CAID image and video on a seized device so that the offender can be correctly prosecuted to the level of their crime. As a victim identification officer in a child sexual abuse investigation I need to have access to correctly assessed images and videos, identified as new to CAID, so that I can help identify any new victims in order that they can be safeguarded. Work Already Done The CAID Transformation Programme has already procured an AI Classifier to assess indecent images of children (IIoC) - the “1st AI Classifier”. There is a concurrent DOS Outcomes advert for the CAID programme entitled, 'CAID 2nd AI Classifier C21177' published at the same time as this DOS Outcome that will combine the outputs of all deployed classifiers. Existing Team The CAID programme comprises a portfolio of projects across strategy and transformation and technical release workstreams. The service will primarily be client-side and operate in partnership with Home Office civil-servants, specialist contractor team members and policing. Current Phase Alpha Skills & Experience • Experience of working with law enforcement or government and, with this, the knowledge of expected controls and governance arrangements • Deep knowledge and experience of image analysis and the appropriate use of Machine Learning in relation to image analysis • ML expertise: proven ability in the formal qualification of machine learning efficacy, evidencing ML fitness, data quality assessment and prediction accuracy • ML expertise: proven ability in data curation for machine learning in order to minimise bias and maximise business outcomes • ML expertise: deriving high-order insights from imagery knowing where best to use deep learning, multicue ML, data fusion and ensemble models • Delivery of through life ML capability, integrating AI algorithms to existing services, best hosting approaches for AI and the optimal approach to supporting and upgrading live AI systems • Experience of providing project leadership and support for projects with multiple and complex dependencies • Proven knowledge and expertise in data governance for the public sector where data sensitivity and security are a major consideration • Experience in providing a flexible resource profile that can efficiently adapt to the demands of the programme, meeting short- and long-term resourcing needs • Actively seek the best capabilities available in the industry, looking to build on the best of open-source tech or potential consortium delivery with a specialist supplier • Experience of continuous delivery and how best to break down a project into tangible delivery steps, monitoring benefits realised and implementing continual improvement practices throughout • Experience of maximising knowledge retention and skills transfer, including advancing knowledge sharing, documentation and the use of enterprise repositories • Manage team effectiveness to deliver outcomes and necessary interactions with the broader programme • Have capability in both the analysis of still imagery and video. Work Location The CAID Team are based in Croydon. The team must be able to onboard and work remotely with occasional travel to other UK locations that include London, West Yorkshire and Croydon. Working Arrangments The supplier will provide team members to deliver and manage the required outcomes. Team members will be protected from exposure to the imagery stored on CAID but discussing the subject matter could be still distressing, furthermore classifier machine learning training and evaluation will require the use of legal adult pornographic imagery. The team must be flexible to work remotely or from Home Office locations within the M25 area. Some UK based travel may be required. Pricing will be based on a hybrid of T&M, fixed price and milestone payments. Day rates must be aligned with the Home Office-mandated rate card. Security Clearance The service requires resources who either have or are prepared to apply for and obtain SC or NPPV3 (Non-Police Personnel Vetting level 3) clearances. Additional T&Cs The Home Office expect to own all project specific foreground IPR including the source-code for the Combined AI Classifiers Harness. Any supplier bidding for the CAID 2nd AI Classifier C2117 cannot also bid for the DOS Outcomes advert for the CAID programme entitled, 'CAID Combined AI Classifiers Harness C21178' published at the same time as this DOS Outcome. The Home Office will not pay expenses for travelling from Supplier work locations to Home Office or other locations within the M25 area. No. of Suppliers to Evaluate 5 Proposal Criteria • Approach to flexible working, assigning multi-skilled personnel to meet CAID goals. • Prior solutions in using AI to predict human actions and behaviours • Leverage 3rd party commercial, open-source and in-house capabilities in delivery • Approach to sharing knowledge and skills transfer with the customer. • Approach to partitioning background and foreground IPR in a manner that best enables long term evolution of capability, where all foreground IPR will be owned by the Home Office. • Approach to through-life product management - in particular ambition toward evolving the capability as a long-term product. • Identification of risks and dependencies and offered mitigation approaches. • Technical competence in a range of relevant technologies; use of AI, machine learning, image analysis, data fusion, ensemble methods, producing AI algorithms to run as services, APIs, software integration. • Demonstrate how you would understand CAID user and stakeholder requirements to deliver capability that meets business needs • Value for money • Ability to source and curate data, including a willingness to source ML training content that includes legal adult pornographic images and videos • WARNING: An early evaluation gate for the candidate classifier will require the supplier to use legal adult pornographic imagery. • Demonstrate how you deal with boundary cases where predictions are not self-evident Cultural Fit Criteria • Experience of working with public sector organisations • Share knowledge and experience with other team members • Genuine enthusiasm for tackling child sexual abuse • Work as a team across our organisation and with other suppliers • Be adaptable and flexible and able to deliver against changing priorities • Be inventive and look to build a solution that utilises the best of human experts with AI • Proactive issue management, problem resolution and improving ways of working • Be transparent, have a no-blame culture and encourage people to learn from their mistakes • A commercial desire to use this opportunity to develop or enhance a product that helps combat child sexual abuse, where consequently the product will be improved and evolved over time. • An operational staff wellbeing policy detailing the support available to staff involved throughout the length of the project Payment Approach Fixed price Assessment Method • Case study • Presentation Evaluation Weighting Technical competence 65% Cultural fit 15% Price 20% Questions from Suppliers 1. Is this role outside IR35? This is an Outcomes based contract which will be based on a hybrid of fixed-price/milestone and capped T&M priced Statements of Work. It is therefore presumed to be outside IR35. However the IR35 status will be confirmed prior to contract award. 2. What is the budget for this opportunity? An indicative budget is currently being agreed and will be confirmed before contract award. 3. Is the Authority looking for an independent supplier for the 2nd Classifier or are you expecting the existing supplier of the 1st Classifier to bid for either of the requirements? This opportunity is open to all suitably qualified suppliers and the Authority will consider all submitted responses. 4. Will the successful supplier have access to the same data that the first classifier was trained on? The successful supplier will have controlled access to CAID lab facilities where the real data is available for ML training. The 1st Classifier had the same access. 5. Will there be a briefing for shortlisted suppliers on the objectives of the 2nd classifier – and what the current classifier does? Yes. 6. Why is there a need for a 2nd classifier? Can the performance of the current classifier just be improved to increase accuracy? The business and stakeholders have committed to explore the benefits of multiple independent classifiers. Like any AI, classifiers are likely to be improved over time by increased access to higher quality training data. 7. Will the authority consider a solution that requires native cloud tooling, such that any application developed would need to be operated from a public cloud environment (e.g, use of Amazon Rekognition)? No. Much of the image data to be classified is criminal to possess or copy. The classifier must operate on premise including on machines that are air-gapped from the Internet. 8. Will the authority consider pre-existing commercial solutions/applications, that may require customisation/scripting to meet the brief? Yes. We expect there to be commercial products that are able to already classify some of the characteristics needed to perform classification of indecent images of children, such as age etc. 9. How accurate is the performance of the 1st Classifier? Some performance information can be shared with the successful supplier.No ML algorithm is perfect as is expected. Independence between the 2 classifiers is our goal with the hope that they make different mistakes. 10. Who’s infrastructure will this capability be built on? The classifier once approved and live will run on policing infrastructure. Final ML training on CAID data will be in the CAID lab on lab infrastructure. Prior to that all early development with benign data will be on the supplier's own infrastructure.Once live drift detection and an associated retraining capability should be considered.Policing infrastructure is typically MS Windows based. 11. Will this capability need to be hosted on Dark Web? The capability needs to be reasonably flexible on where it can be hosted.