PU-17/0128 - Data Source Collection and Assessment

Tender description

Why the Work is Being Done ONS is creating a small number of platforms that will be shared across the organisation (and in some cases across Government) to deliver various data journeys across the statistical value chain (based on the Generic Statistical Business Process Model - GSBPM). This work concerns part of the collection platform and will enable the secure loading of data from (ultimately) a variety of non-survey (i.e., not interactively collected by ONS) external sources. Problem to Be Solved Create the alpha (prototype) of various products that constitute the non-survey data collection elements of the overall Collection Platform. Three products are involved: (1) Import External Data - import data source and execute malware checks,(2) Data Source Assessment - apply rule based structural data assessment and (3) Data Exchange - load and transform data for further processing. These are products within the ONS EA framework. The alpha stage is as defined within the GDS Service Design Manual. Who Are the Users As a(n) external supplier I need to electronically pass data for an approved data source and signed off data journey So that the data source can be made available for pre-processing and assessments. As a(n) internal user I need my data source and metadata to be loaded into an accessible repository if it does not contain malware So that the pre-processing stage can continue. As a(n) internal user I need to be able to confirm that the data are structurally correct So that downstream processing can proceed. Other user stories have been identified. Early Market Engagement n/a Work Already Done High level discovery has been completed which identified high level user needs, Epic stories, EA Product/Platform mappings and high level Service Definitions (which contain the high level user stories referred to above). Of course during the alpha phase more detail will be inked in with a focus on the minimum requirement. The Business Analyst involved in the discovery will carry forward into the alpha phase also. Existing Team The supplier will be collaborating with multiple existing (and to be created) multi-disciplinary product teams. The supplier will provide User Research resources as well as technology development capability Each team will include a Delivery Manager, Product Owner, Technical Lead, Business Analyst and Statistical Methodologist. The teams are made up of 3rd party delivery partners and ONS permanent staff. Current Phase Discovery Work Location The work will be based in ONS' Newport (South Wales) Office - there is a possibility some travel will be required to ONS' Titchfield site, Fareham, Hampshire. Working Arrangments The team must work alongside existing teams building products to strategic drivers and objectives. They will collaborate with other teams across the service and beyond and will invest time and effort into knowledge sharing, coaching and mentoring across individuals in order to ensure increased capability of ONS people. They will be co-located with technical and business colleagues, operate using Agile tools and techniques, actively participating in Agile ceremonies including service show and tells. They will be expected to be on-site 5 days per week. Security Clearance All team members must be in possession of a current Disclosure Scotland clearance prior to starting contractual work. Additional T&Cs Framework Ts and Cs will apply Skills & Experience Expertise in solutions that utilise SOA and micro services - 3 points. Expertise in development/delivery using Cloud Foundry (Python, Scala, Java) - 3 points Expertise in the design and delivery of Restful API's (inc: security functional and non-functional requirements) and use of appropriate tooling such as Swagger/RAML. - 2 points Expert in cloud technologies: design, planning. security, integration, whilst remaining Cloud provider agnostic - 2 points Experience of effective identification and use of open source software components - 2 points Experience in relational and non-relational databases in multiple formats - 2 points Experience in using automated (testing) environments and continuous integration, delivery & (potentially) deployment, particularly GitHub and Jenkins - 2 points Expertise in research and analysis of user needs - including those related to Assisted Digital and accesibility - and translating them into product needs - 2 points Expertise in Agile delivery (e.g., Scrum, Kanban etc.) as outlined in the Government Service Design Manual using BDD and TDD - 3 points Experience of delivery in Data Engineering and Data Management Services, especially in a sensitive data environment - 3 points Proven track record of delivery and integration of services in 'brownfield' & 'greenfield' estates - 2 points Expertise in coaching/mentoring less skilled people, leaving a legacy - 2 points Expertise in Full Cloudera (Apache Spark etc.) Stack - 2 points Expertise in workflow and integration solutions especially involving API gateways - 2 points Nice to Haves Experience of working across government agencies - 1 point Security knowledge of AWS, Azure or locally deployed on VMWare infrastructure - 1 point. Experience of creating solutions using 'R' - 1 point No. of Suppliers to Evaluate 3 Proposal Criteria Technical Solutions = 5 points Approach & Methodology = 5 points Approach the approach or solution meets user needs = 5 points How the approach or solution meets the organisation policy or goal = 2 points Estimated timeframes = 2 points Identify risks and dependencies, approaches to manage them = 2 points Team Structure = 2 points Value for Money = 2 points Cultural Fit Criteria Work as a team with our organisation and other suppliers - 5 point Transparent & Collaborative when making decisions - 5 points Have the user and customer needs at the core of decision making - 5 points No blame culture and encourage people to learn from their mistakes - 5 points Payment Approach Time and materials Assessment Method Written proposal Case study Work history Presentation Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers Budget range n/a