Automation Skills Injection

Tender description

Why the Work is Being Done The testing capability within the Met Office is being transformed from a Quality Assurance (QA) to a Total Quality Engineering (TQE) model. This involves introducing new practices such as Behaviour Driven Development (BDD) and Specification by Example (SBE). These practices consider quality at the outset and support Continuous Integration which enables a ‘learn fast, learn often’ culture. The current test automation approach was found to be lacking and unable to fully support a Total Quality Engineering / Continuous Integration (CI) model. Problem to Be Solved A recent review identified the following challenges: • No common tooling strategy. • Duplicate tooling (e.g. 9 x Test Management tools). • Over reliance on unscripted testing (~60%). • Only ~20% of products have regression packs • Automation scripts not maintained post-delivery. • Emphasis on GUI automation. Addressing these challenges will: • Increase velocity. • Increase quality. • Increase repeatability. • Shorten feedback cycles (learn fast, learn often) • Reduce the overall time to market. • Reduce testing technical debt. • Drive a continuous improvement culture. • Link test artefacts and test execution results to requirements in a common tool. Who Are the Users Solution Engineering requires Supplier to: • Review test automation blueprint. • Establish and implement a common tooling solution (e.g. toolset, frameworks) • Generate scientific test data. • Provide skills injection (coaching). • Work with teams to deliver automated tests Delivery Team requires a tooling solution with ability to: • Stand-up and teardown a test environment. • Support various test phases and execution triggers (e.g. functional, non-functional, at code check-in, overnight). • Test at different levels of the technology stack (testing pyramid) and support BDD, SBE, TDD and CI. • Provide MI so that the Definition of Done can be assessed. Early Market Engagement Work Already Done The current test automation approach was found to be lacking and a new test automation blueprint has been developed. The blueprint includes an automation strategy and areas of testing technical debt to be addressed (e.g. unscripted regression testing). The blueprint recommends engaging a 3rd party to help implement the strategy and up-skill the current Met Office teams. Suggested team size is ~5: • 1 x Test Architect • n x Senior Automation Specialist(s) • n x Automation Specialist(s) The Met Office is in the process of purchasing OpenShift from Red Hat to manage some non-production environments and containerise applications. Existing Team • Delivery Teams – Delivery Manager, Technical Lead, Solution Engineers, Product Owner. • As a Service – Solution Architects, Test Architects, Product Managers, Technical Testers, PMO. • Automation capability to be provided by Supplier. Supplier to list roles. Current Phase Beta Work Location • Majority of engagements will be based in Exeter, UK. • Option to work remotely as appropriate. Working Arrangments Core team of ~3 to be based onsite with the potential for the other roles to work remotely. Supplier to rotate core team based onsite as appropriate. The Met Office is open to onshore / nearshore solutions. Security Clearance Please consider when responding, the necessity for all individuals to be willing and able to pass SC clearance if attending Exeter site. Additional T&Cs Skills & Experience Experience implementing an enterprise wide test automation strategy in an Agile environment. Delivering tooling solutions that support testing at different levels of the technology stack (testing pyramid) including data presented in graphical tiled format (e.g. weather maps). Experience with solution engineering practices such as BDD, SBE, TDD, CI. Working with projects during initiation translating strategic concepts and tooling architectures into practical implementations. Experience coaching delivery teams. Able to provide resource with practical experience in a large selection of automated tools and associated scripting languages to include Java. Delivery Centres based within the EU/EEA. Nice to Haves Able to provide a formal training capability. Experience creating scientific test data (e.g. 4 dimensional geospatial weather data, GRIB2, NetCDF, CFNetCDF). Able to offer additional services that the Met Office may decide to leverage at a future point (e.g. resource augmentation, fixed cost engagements). Creating a culture of continuous improvement in order to constantly enhance testing principles, processes, techniques, tooling and frameworks. No. of Suppliers to Evaluate 5 Proposal Criteria The proposed approach, methodology and experience. How the approach or solution meets user needs The proposed technical solution The lead time to commence the engagement The value for money of the proposal Cultural Fit Criteria Work as a team with our organisation and other suppliers Timeliness and effectiveness of communication Willingness to engage as a partner Openness and approach to knowledge transfer/training Payment Approach Time and materials Assessment Method Written proposal Case study Work history Presentation Evaluation Weighting Technical competence 60% Cultural fit 15% Price 25% Questions from Suppliers Budget range £200,000 to £400,000 in total. we have currently put “time and materials” as our price criteria. however, we are open to any suggestions from suppliers about the price criteria and are willing to review any alternatives. rate card guide (excluding expenses): • test architect - up to £400 • senior automation specialist – up to £350 • automation specialist – up to £300 supplier to provide the resource augmentation price required to deliver the needs outlined in this briefing document. the met office will fund any new tooling and/or infrastructure.