PU-17/0107 - Statistical Process Control

Tender description

Summary of the work ONS are looking for a dynamic, highly skilled, collaborative supplier to work on creating the alpha of SPC and components of the first data journey. Moving forward we need a supplier capable of delivering significant technical change at pace in a data focused environment using Agile techniques. Expected Contract Length Minimum 6 months. Latest start date 01/09/2016 Budget Range n/a Why the Work is Being Done ONS is creating a small number of shared platforms that will deliver various data journeys across the statistical value chain (based on the Generic Statistical Business Process Model - GSBPM). Key to the delivery of those platforms will be the means to define and execute those individual data journeys (and hence choreograph the various technical components that will deliver the statistical business products). Statistical Process Control (SPC) represents that 'beating heart' of the processing platforms and must deliver work flow configuration, business rules and management capabilities in various environments. Problem to Be Solved Create the alpha (prototype) of the Statistical Process Control solution to deliver the first data journey, bringing VAT Turnover data into the ONS National Accounts. Create (alongside other teams) alphas of the technical components required for the first data journey, using ONS' strategic technology stack. Ultimately SPC will coordinate all data journeys and all components required for all business products (including those related to the 2021 Census). The business products have been defined using a comprehensive EA approach and there is substantial collateral to describe them. Who Are the Users High level discovery has been undertaken, the initial stories are (As a user I need), TO BE ABLE to run my predefined statistical processes SO THAT all gaps in my data have been processed appropriately TO BE ABLE to amend the data generated by each stage (or step) of a data journey until data reaches 'ready' or 'published' status SO THAT I can easily modify and improve data quality and apply rules and QA for each stage TO BE ABLE to rerun any stage (or step) SO THAT I can validate that changes meet the rules within the rerun stage. Early Market Engagement n/a Work Already Done High level discovery has been completed which identified high level user needs, Epic stories, EA Product/Platform mappings and high level Service Definitions (which contain the high level user stories referred to above). Of course during the alpha phase more detail will be inked in with a focus on the minimum requirement. The Business Analyst involved in the discovery will carry forward into the alpha phase also. Existing Team The supplier will be collaborating with multiple existing (and to be created) multi-disciplinary product teams as the SPC product will coordinate all data journeys from collection (which is a separate platform - see other supplier opportunities) to dissemination (also a separate platform). The supplier will provide Product Owner and User Research resources as well as technology development capability Each team will include a Delivery Manager, Product Owner, Technical Lead, Business Analyst and Statistical Methodologist. The teams are made up of 3rd party delivery partners and ONS permanent staff. Current Phase Discovery Skills & Experience • Expert in delivery of solutions that utilise SOA and micro services - 2 points • Expertise in development/delivery using Cloud Foundry (Python,Scala,Selenium/BOSH/Manifest or docker, GRAILS, Spring) - 3 points • Expertise in the design and delivery of Restful API's (inc: security functional and non-functional requirements) and use of appropriate tooling such as Swagger/RAML. - 2 points • Expert in cloud technologies: design, planning. security, integration, whilst remaining Cloud provider agnostic - 2 points • Experience of effective identification and use of open source software components - 2 points • Experience in relational and non-relational databases in multiple formats - 2 points • Experience in using automated (testing) environments and continuous integration, delivery & (potentially) deployment, particularly GitHub and Jenkins - 2 points • Expertise in research and analysis of user needs - including those related to Assisted Digital and accesibility - and translating them into product needs - 2 points • Expertise in Agile delivery (e.g., Scrum, Kanban etc.) as outlined in the Government Service Design Manual using BDD and TDD - 3 points • Experience of delivery in Data Engineering and Data Management Services, especially in a sensitive data environment - 3 points • Proven track record of delivery services in 'brownfield' & 'greenfield' estates - 2 points • Expertise in coaching/mentoring less skilled people, leaving a legacy - 2 points • Expertise in Full Cloudera (Apache Spark etc.) Stack - 2 points • Expertise in workflow and integration solutions especially involving API gateways - 2 points Nice to Haves • Experience of working across government agencies - 1 point • Security knowledge of Cloud Foundry on AWS, Azure or locally deployed on VMWare infrastructure - 1 point • Experience of creating solutions using 'R' - 2 point Work Location The work will be based in ONS' Newport (South Wales) Office - there is a possibility some travel will be required to ONS' Titchfield site, Fareham, Hampshire. Working Arrangments The team must work alongside existing teams building products to strategic drivers and objectives. They will collaborate with other teams across the service and beyond and will invest time and effort into knowledge sharing, coaching and mentoring across individuals in order to ensure increased capability of ONS people (especially of Product Owners and Delivery Managers). They will be co-located with technical and business colleagues, operate using Agile tools and techniques, actively participating in Agile ceremonies including service show and tells. They will be expected to be on-site 5 days per week. Security Clearance All team members must be in possession of a current Disclosure Scotland clearance prior to starting contractual work. Additional T&Cs Standard Framework Ts and Cs will apply No. of Suppliers to Evaluate 3 Proposal Criteria • Technical Solutions = 5 points • Approach & Methodology = 5 points • Approach the approach or solution meets user needs = 5 points • How the approach or solution meets the organisation policy or goal = 2 points • Estimated timeframes = 2 points • Identify risks and dependencies, approaches to manage them = 2 points • Team Structure = 2 points • Value for Money = 2 points Cultural Fit Criteria • Work as a team with our organisation and other suppliers = 4 points • Transparent & Collaborative when making decision = 4 points • Have the user and customer needs at the core of decision making = 4 points • No blame culture and encourage people to learn from their mistakes = 4 points • Skills transfer and mentoring of team members = 4 points Payment Approach Time and materials Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers 1. What is the primary development language to be used? You mention Python, Scala, Grails which is Groovy language Framework, and Spring which is a Java language framework. Or can the partner choose the language from this list? The preferred development languages are Python, Scala and Java (in that order) with R for statistical development if required. The supplier technical lead would work with the ONS Architect to determine the most appropriate language(s) and framework(s) to use. 2. How many staff are you looking for and what is the mix? e.g. between developers and testers? Are all staff expected to start on 1 September 2016 or will there be a planned ramp up of staff e.g. seniors deployed first? This engagement is for a team to create the alpha of statistical process control and alphas for other components, ONS will supply business analyst, product owner, delivery manager and architect resources together with agile coaching support. In the first instance the team will be relatively small - user researcher, user experience designer, technical lead and one or two developers. The expectation is that testing will be automated and the developers will have those skills. contd. 3. How many staff are you looking for and what is the mix? e.g. between developers and testers? Are all staff expected to start on 1 September 2016 or will there be a planned ramp up of staff e.g. seniors deployed first? One of the outcomes of the alpha will be greater understanding of what is required from the technical team as well as the product itself but one can already predict that as the work progresses the team will inevitably grow. We are seeking (if practical) support for the product owners as it is a new role to the organisation. Start date is as close as is practical to 1 September. 4. You refer to "agile techniques" and specify scrum or Kanban. Who will be the scrum master for this work? Is that the same person as the delivery manager that you refer to? The delivery manager will fulfil the same role as a scrum master does. 5. You specify that a Product Owner will be part of the team provided by the partner. Do you mean a Proxy Product Owner? What resources will ONS provide to enable the Product Owner to develop the high level discovery phase that you state has been completed? The intention was to explore whether an experienced product owner could be provided to mentor and support ONS product owner (i.e., the ONS representative would operate more as a subject matter expert), you are right that this could only be a proxy. ONS will supply business analyst, product owner, delivery manager and architect resources together with agile coaching support. 6. You specify that a Product Owner will be part of the team provided by the partner. Do you mean a Proxy Product Owner? What resources will ONS provide to enable the Product Owner to develop the high level discovery phase that you state has been completed? The intention was to explore whether an experienced product owner could be provided to mentor and support ONS product owner (i.e., the ONS representative would operate more as a subject matter expert), you are right that this could only be a proxy. ONS will supply business analyst, product owner, delivery manager and architect resources together with agile coaching support. 7. You state that the partner should provide a Product Owner and User Research resources. A common and successful pattern is for the Product Owner and User Research resources to reside on client site (i.e. ONS site) and for the delivery team to be located elsewhere. The Product Owner and the Scrum Master will liaise to ensure delivery. This pattern should result in considerable costs savings for ONS and the tax payer. Is this onsite/offsite model something you can consider? ONS experience has been that this model has not worked successfully, we expect colocation of teams. 8. 1) We have assumed the following roles that the supplier should be providing – is this correct?a. Delivery Managerb. User Researcherc. UX Designerd. Software Engineere. Content Designerf. WebOps Engineerg. Product Owner The supplier should provide, Technical Lead, User Researcher, UX Designer, 1/2 Software Engineers although the team will ramp up as we understand more about what is required of the technical team. ONS will provide the Product Owners although it is a new role to the organisation and mentoring for them through an experienced Product Owner could be helpful. 9. Is a staggered start acceptable i.e. can S/w engineers / developers start later than other team members? Or everybody onsite from Day 1? Ideally the team should be complete with all roles covered as we will quickly get into sprints. 10. What is the main programming language you expect the developers to use? The programming languages are Python, Scala, Java and R for statistical programming if required although the team will work with the ONS IT architect to identify the most appropriate solutions. 11. Can you share any additional information regarding the high level discovery outputs at this stage? This can be shared with suppliers should you be successful in shortlisting.