FCO cloud analysis platform development

Tender description

Why the Work is Being Done The Open Source Unit's aim is to best use open data to help the Foreign Office deliver across the whole spectrum of its policy priorities (encompassing many spheres, such as: crisis response, consular services, security and sustainability). To enable this, we aim to construct a data platform that gives staff and collaborators easy access to a rich selection of open data, and a suite of powerful, user-friendly analysis tools they can use to unlock the most pertinent insights. The work must be completed by 31st March 2019. Problem to Be Solved Useful data is widely dispersed and cannot be accessed from a single point. Standards vary (e.g. different date formats and regional breakdowns), making it difficult to combine data from different sources. Data is in different formats (html, csv, APIs, etc.) requires different processing to make it useable on a common platform. Data is of different types: numeric, text, satellite imagery, metadata, etc. Specialist tools/code (R/Python) are currently required to perform analysis – we need to abstract this complexity to open analysis to a broader group of users. The platform must be secure and meet the NCSC’s requirements for official documents. Who Are the Users As an economic researcher, I need to combine and visualise data, for specific regions and time periods, from different sources (e.g. World Bank Open Data, and page view data from the Wikipedia API), so I can make informed inferences when making policy recommendations. I also need to locate pertinent insights from large collections of documents to create briefings, and identify/monitor patterns in time series data which imply that the probability of an event has increased. I need to collaborate effectively with colleagues while completing these tasks. Early Market Engagement OSU have developed a prototype component of the platform, focussing on text analysis. Early feedback has been positive, but a larger scale user testing exercise would be required. Prior development, we conducted a small user research exercise, the main conclusions of which were: People currently spend too much time traversing various different data sources – highlighting the need for more centralisation Use of numerical analysis is currently very limited – more access to tools that enable this would be beneficial. Work Already Done OSU have developed a prototype text analysis component of the platform. It uses Spacy to perform named entity recognition and tag documents, which were then indexed in Elasticsearch. The tool has a simple front-end, built with Vue.js, which allows users to explore the documents via geographical and metric based interface. OSU have also harnessed the open source collaboration product, Nextcloud, for inter-team working. We are interested in the possibility of using Nextcloud as an ecosystem for the wider platform, and adding analytical capability by developing additional Nextcloud apps. Existing Team OSU has three data scientists who would be involved with the data science aspects of the project e.g. machine learning / data wrangling. At this stage we do not expect any additional suppliers to be involved with the project. Current Phase Alpha Work Location Foreign & Commonwealth Office, King Charles Street, London, SW1A 2AH Working Arrangments We would like the supplier’s team to be present in our offices at least three days per week to facilitate effective communication with the FCO staff also involved. Security Clearance Existing SC clearance required for all individuals involved. Additional T&Cs Standard Contract Skills & Experience Demonstrate experience of developing user-friendly, visually engaging cloud appications Demonstrate experience of interacting with Elasticsearch Demonstrate experience of capturing, standardising and storing data from varied sources in a suitable container (e.g. MongoDB etc.) Demonstrate experience of developing data-pipelines, e.g. to provide updates from APIs Demonstrate experience of developing attractive, user-friendly UIs with (Vue/React/AngularJS), including data visualisations and geospacial representations Demonstrate experience of working in an agile team to deliver software based on user requirements Demonstrate experience of producing user documentation and maintenance documentation Demonstrate experience of government cyber security requirements for official data Demonstrate experience of developing secure user authorisation and user interaction masking Demonstrate experience of producing documentation to evidence security credentials Nice to Haves No. of Suppliers to Evaluate 5 Proposal Criteria Technical solution Approach and methodology How the approach or solution meets user needs •How the approach or solution meets your organisation’s policy or goal Cultural Fit Criteria Work as a team with our organisation Be transparent and collaborative when making decisions Have a no-blame culture and encourage people to learn from their mistakes Take responsibility for their work Share knowledge and experience with other team members Challenge the status quo Be comfortable standing up for their discipline Can work with clients with low technical expertise Payment Approach Fixed price Assessment Method Written proposal Presentation Evaluation Weighting Technical competence 50% Cultural fit 15% Price 35% Questions from Suppliers Budget range £300-400k based on 3/4 fulltime individuals for the duration of the project.