Development of computer vision (AI) tools for REM of pelagic Fisheries

Tender description

This requirement is for the development of computer vision (AI) tools for pelagic fisheries CCTV. The project will consider the development of an automated image analysis system to isolate (and quantify) activities (and animals) associated with pelagic fisheries, with a focus on the capture of UCB species in CCTV-observed fishing events of a given subset of vessels subject to mandatory Remote Electronic Monitoring (REM). The project will focus initially on counts of UCB events, but the intention would be to gather biological data on the individual animals caught e.g. classification to the finest possible level and estimates of size. The results will be used by Scottish Government Marine Directorate (SGMD) Compliance (for regulation infringement detection) and SGMD Science for bycatch rate estimation, as part of wider observer programmes, and to inform future revisions to REM technical specifications. The ultimate goal of this development work is to design, implement and test a UCB detection and classification system, including the supply of relevant CCTV footage from vessels in Scottish Waters. Lot 1: This system should be capable of being integrated into REM review software to automatically process and analyse video sequences from fishing trips to identify relevant events, count UCB events, and subsequently identify, count and measure all UCB individuals. It is important to point out that such a system should also be able to handle a realistic range of environmental noise such as variability of the location within the field of view, due to differing vessel layouts, in which events / animals have been detected, occlusions, pose variations, and lighting conditions. It is envisaged that this will be done in a probabilistic way, and that human intervention and further analysis will be suggested when the uncertainty exceeds a preset threshold (as there will be sections of footage in which identification will be very difficult due to occlusion). The system should be easily maintained and upgraded and will remain the property of the Scottish Government. This project will consider the development of interim stages towards fully automated image analysis tools for identifying and characterising atypical fishing activity events indicative of UCB species presence. In addition, the AI tools will subsequently determine the number (and, potentially, the size) of a subset of UCB observations via cameras as part of the mandated REM of eligible pelagic fishing vessels in Scottish waters. Attention would focus initially on the recorded numbers of UCB events as well as counts and size measurements for sea mammals (inc. cetaceans, seals), sharks / skates / rays, seabirds, and non-target bony fish. The results / products will be used by SGMD scientists to increase the efficiency of video review tasks, optimise the technical specifications of electronic monitoring systems for this fishery, estimate bycatch mortality rates, and identify fishing practices where mitigation measures may be required to protect vulnerable species. Specifically, this study will aim to: a) Develop a graphical user interface (GUI) to facilitate the viewing, sharing, collation, analysis and annotation of CCTV video footage from pelagic fishing vessels; b) Instruct and assist with the calibration of camera systems (if required) to ensure field-of-view and lens distortion correction in length measurements; c) Develop detection algorithms to detect, classify (to the most detailed taxonomic level feasible), and measure individuals of UCB species passing through the dewatering separator from representative video footage; d) Develop detection algorithms to determine when activity is taking place on-deck or in-water in relation to catch handling and apply facial recognition to anonymise the recorded footage; e) Develop detection algorithms to detect, classify (at the minimum level of mammals, elasmobranchs, non-target fish, and seabirds) and measure (inter alia) individuals of ETP species on deck or in-water from representative video footage. Suppliers should provide imagery from a minimum of 3 vessels. However, it is desirable that imagery is inclusive of both freezer vessels and refrigerated / chilled water vessels, vessels with differing deck layouts and catch handling configurations; and vessels from multiple flag states. Imagery provided to Scottish Government for annotation within the GUI must be anonymised regarding its metadata and have had facial recognition blurring methods applied. Suppliers must arrange to have access to suitable CCTV footage from Scottish Waters, either within their own data storage architecture (if they are an REM service provider) or via partnership with an REM service provider / pelagic fishing boats. It is anticipated that the project will be entirely computer lab-based, as all required video footage is being collected on an on-going basis as a matter of routine.