Improvements to statistical and geographical databases and open data publishing tools / processes

Tender description

Why the Work is Being Done The aim is to deliver a package of essential improvements to ADD's statistical and geographic databases and open data publishing tools / processes. This includes (1) various priority enhancements to existing relational (SQL Server) and LinkedData databases and associated data management processes / tools. (2) Strengthening and improving the overall data architecture and associated 'end-to-end' processes - including integrating with the Department's NoSQL (Mark Logic) technology and document stores. Problem to Be Solved (1) Enhancing ADD's open data publishing tools and data standards to enable more efficient and robust data release. (2) Extend ADD's specialist data stores and tools, so that analysts and data scientists are better equipped to respond to current high-profile demands for new analysis and insights. (3) Design and implement priority enhancements to existing statistical data stores and associated publishing tools. (4) Enhance current analytical data stores to incorporate data gathered from Local Authorities. (5) Support work to design replacement systems (utilising cloud-based technologies) and begin migrating current data. (6) Help develop and implement data acquisition and management strategies. Who Are the Users The Data, Analytics and Statistics (DAS) team are permanent Civil Servants who support and develop ADD's analytical databases and associated technology. Work to design and implement data acquisition and management strategies will involve collaboration with DCLG Corporate ICT colleagues, and the Department's Data Programme Board. Early Market Engagement Work Already Done The DAS team has undertaken preliminary work on high-level designs and blueprints for items 1-5 above. This has been partly informed by a recently completed Discovery exercise, which identified and recommended various opportunities to improve current statistical data processing and publishing tools and methods. This needs to be validated, leading to detailed design and implementation. Priority enhancements will be delivered via DAS's existing technical and database infrastructure, comprising Windows Server running SQL 2016. Databases are accessible to users via a mixture of server and desktop tools - including SQL reporting services (server), MS-Excel, Stata, ArcGIS, Quantum GIS, R-Studio and Python. Existing Team The supplier will be working with Data, Analytics and Statistics (DAS) ICT team. Current Phase Not applicable Work Location DCLG Headquarters, 2 Marsham St, London Working Arrangments This is envisaged as a time and materials engagement with agreed outputs. It is expected that some of this will be achieved working onsite with role leads. Remaining work can be offsite. Normal operational hours of Monday to Friday 9am to 5pm. Security Clearance Counter-Terrorism Check (CTC) Additional T&Cs All IPR associated with outputs and deliverables produced shall be wholly owned by DCLG. Suppliers will be expected to document all activities during the assignment, and handover (including transferring knowledge and skills) to the permanent team in a timely manner. Skills & Experience (1) Have proven experience delivering projects using the agile project management methodology (2) Demonstrate understanding and ability to deliver digital services / products to the Government Digital Service standards (3) Have a track record of successfully managing and assisting in delivering IT projects within public sector organisations (4) Have effective communications across organisational, technical and political boundaries, understanding the context (4a) Makes complex and technical information simple and accessible for non-technical audiences. Is able to advocate and communicate what a team does to create trust and authenticity. (5) Translate data into valuable insights that inform decisions. (5a) Involves teams in analytics and synthesis to increase consensus. Collects, collates, cleanses, synthesises and interprets data to derive meaningful and actionable insights. (6) Develop fit for purpose, resillient, scalable and future-proof data services to meet user needs (6a) Demonstable understanding of how to expose data from systems (eg. through APIs), link data from multiple systems and deliver streaming services (7) Produces data models and understands where to use different types of data models (7a) Understands different tools and is able to compare between different data models. Able to reverse engineer a data model from a live system. (8) Have knowledge of data engineering and specific technologies (particularly SQL Server) (9) Plan, design, manage, execute and reports tests using appropriate tools and techniques and works within regulations (9a) Ensures risks associated with deployment are adequately understood and documented. Nice to Haves (1) Demonstrate experience of working in a challenging and complex private or public sector technical environment with regularly changing priorities. (2) Show a strong knowledge of and experience working with IT technical services within the private or public sector No. of Suppliers to Evaluate 4 Proposal Criteria (1) Capability and experience - weighting 60% (1a) Team structure, capability and experience (1b) Proposed methodology and approach (inc. GDS Standards) (1c) How the process and end deliverables will meet the user needs (1d) Timeframe estimation (2) Cultural fit (see Cultural fit criteria for method) - weighting 20% (3) Pricing - weighting 20% (3a) Standard price calculation assuming a 200 day assignment for both roles (3b) EXAMPLE - Role 1: £700p.d x 200 = £140k and Role 2 £650p.d x200 = £130k - total price to evaluate £270k When bidding, suppliers should provide the following: (A) Full CVs for the proposed Data Architect and Database developer (B) A short (1 page max) summary outlining (1) understanding of problems to be addressed and outputs to be delivered (B)(i) A short (1 page max) summary outlining (2) suitability, skills and experience to successfully address problems and deliver outputs (C) Proposed day rates and duration for each candidate Cultural Fit Criteria (1) Capability and experience - weighting 60% (1a) Team structure, capability and experience (1b) Proposed methodology and approach (inc. GDS Standards) (1c) How the process and end deliverables will meet the user needs (1d) Timeframe estimation (2) Cultural fit (see Cultural fit criteria for method) - weighting 20% (3) Pricing - weighting 20% (3a) Standard price calculation assuming a 200 day assignment for both roles (3b) EXAMPLE - Role 1: £700p.d x 200 = £140k and Role 2 £650p.d x200 = £130k - total price to evaluate £270k When bidding, suppliers should provide the following: (A) Full CVs for the proposed Data Architect and Database developer (B) A short (1 page max) summary outlining (1) understanding of problems to be addressed and outputs to be delivered (B)(i) A short (1 page max) summary outlining (2) suitability, skills and experience to successfully address problems and deliver outputs (C) Proposed day rates and duration for each candidate Payment Approach Time and materials Assessment Method Written proposal Work history Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers Budget range we anticipate appointing a team of 2 to deliver the required outcomes, comprising of 1x senior and experienced database architect, and 1x database designer / developer. suppliers will be expected to provide timesheets for approval and invoicing only being permit for work actually undertaken. candidates will be assessed by the criteria set out below and we may also choose to invite candidates for face-to-face interviews