K280021932 - Test Route Transformation Discovery

Tender description

Summary of the work Test routes are the basis of practical driving tests and allow examiners to give candidates directions or ask them to follow Satnav directions. DVSA wants to discover the problems it faces designing and delivering test routes and gain confidence that technology could transform this in the future. Expected Contract Length 12 weeks with the option to extend for a further 2 weeks, if required. Latest start date Friday 10 February 2023 Budget Range This will be a fixed price discovery project up to the value of £115,000 Why the Work is Being Done DVSA has an ambition to use technology to improve the design and delivery of driving tests routes. The test route starts and finishes at the local driving test centre and is the basis of the main parts of the driving test. These are: • General driving ability where the examiner gives the candidate directions • Independent driving where the candidate follows directions on Satnav DVSA now wants to complete a 12-week Discovery, in 6 x 2-week sprints or as agreed with the supplier, that: • Ensures we understand the problem that needs to be solved • Revisits previous proof of concept work to give DVSA confidence that a working prototype can be built in Alpha Problem to Be Solved Test routes are designed and developed manually by the Local Driving Test Manager. It is time consuming to make sure each route meets all the legal requirements and flows consistently throughout the day, regardless of when the test is conducted and traffic conditions, so they complete reliably within test slot times. While important, learning test routes leads to inefficiencies. New examiners must learn routes before being able to test on them. This typically takes 3 or 4 drives around each route before an examiner knows the route well enough to give timely instructions to candidates. Existing examiners at a driving test centre must learn new or changed routes. DVSA does not publish test routes, but these can become common knowledge over time. Some driving instructors teach learner drivers the test route at the expense of learning to drive independently, regardless of test routes at a driving test centre. Anything that can be done to increase the variability of test routes leads to more of the latter, with road safety benefits. Who Are the Users The users are: • The driving examiners who deliver car practical driving tests • The local driving test managers who manage driving examiners and develop the routes followed on driving tests • Test candidates (or learner drivers) who take car practical driving tests DVSA expects initial test route transformation work to only impact examiners and local driving test managers. Test candidates and driving instructors are out of scope for Discovery. The scope of Discovery is car. Other categories of driving test – for example, motorcycle and lorry, bus – will be considered in Alpha and beyond. Work Already Done An algorithm-based proof of concept was completed 2½ years ago. This gave DVSA confidence in the general principle of a technology-based solution, although challenges remained about delivering a working prototype in Alpha. In addition, the proof of concept did not address typical Discovery questions, like user needs and policy intent. This is a rapidly developing technology field, and the landscape will have changed. Some documentation on previous work will be made available to shortlisted bidders. Existing Team The work will form part of the Flexible Test Delivery Service project. The dedicated team includes: • Head of Driver Transformation, the senior responsible owner (previously an Operations Manager with line responsibility for driving examiners) • Senior Delivery Manager (previously a local driving test manager) and Project Coordinator (previously a driving examiner) • Project Analyst, delivering business analysis for the project The project will also have access to Policy resource. Policy is responsible for National Driving Standards and the criteria on which test routes are based, and the underpinning regulatory framework. They sanction core changes to the driving test. Current Phase Not started Skills & Experience • Detail your experience of understanding and defining problems during Discovery • Detail your experience of understanding user needs and engaging stakeholders during Discovery • Detail your ability to work with geographic (or geospatial) data and information • Detail your ability to do algorithmic based work when delivering a proof-of-concept • Detail your experience of rapidly delivering technology based proof-of-concept work • Demonstrate your experience defining the inputs that structure the delivery of Alpha and the likely costs of doing so Nice to Haves • Detail your experience helping clients to support the move from Discovery to Alpha using business cases • Demonstrate your experience of applying the Technology Code of Practice https://www.gov.uk/guidance/the-technology-code-of-practice Work Location The DVSA is working on a ‘hybrid’ model with staff working at a combination of office locations and other suitable locations. Supplier staff may be able to carry out some collaborative work remotely but may also be expected to spend some time working at DVSA locations. These might be business locations and driving test centres. The DVSA key business location for this work is in Nottingham. Any visits to test centres are likely to be in a broadly similar part of the country. Working Arrangments Suppliers will be available Monday to Friday during normal office hours of 9.30am to 5.00pm Security Clearance Minimum requirement of BPSS. DVSA anticipates any proof-of-concept work is completed on supplier systems/devices. Bidders won't have access to live data and/or key front-facing systems. Bidders will be given access to current routes. These must not become public knowledge. Routes generated by the supplier must be shared securely with DVSA. Additional T&Cs Further, detailed information will be provided to shortlisted Suppliers at the relevant stage of the procurement process. Please provide non-DVSA examples when responding to the Essential and Nice-to-Have criteria. This will remove any incumbent advantage. If the response includes a DVSA example then it will be discounted. No. of Suppliers to Evaluate 5 Proposal Criteria • Demonstrate your approach/methodology to successfully delivering the outcomes and achieving value for money across the following areas: • Proof-of-concept • Problem definition • Moving from Discovery to Alpha • Capability and Structure • Social Value Criteria - Demonstrate action to support health and wellbeing, including physical and mental health, in the contract workforce. Cultural Fit Criteria Demonstrate your approach to working with DVSA openly and collaboratively. Payment Approach Fixed price Evaluation Weighting Technical competence 75% Cultural fit 5% Price 20% Questions from Suppliers 1. Will DVSA be using an e-sourcing portal for stage 2/proposals? We will be using the DfT e-sourcing portal (Jaggaer) for stage 2 of this opportunity. Please register using this link https://dft.app.jaggaer.com/ 2. Can DVSA please confirm their intended procurement timescales for shortlisting, proposal stage (start date and submission deadline) and contract award? 20/12 suppliers notified of stage 1 outcome 21/12 shortlisted suppliers will receive full statement of requirements and be invited to bid / launch of stage 2 proposal. This will be via the DfT e-sourcing portal (Jaggaer) 12/01 proposal deadline 24/01 shortlisted suppliers notified of stage 2 outcome 06/02 award contract to winning supplier, following standstill period Please note that the timeline is subject to change at DVSA’s discretion. 3. Can suppliers use multiple examples in their answers to questions? Suppliers can use more than one example to illustrate their experience, but should not exceed the number of words permitted for the response. Please note the information in ‘Additional Terms’ section that ask suppliers to use non-DVSA examples. 4. Could you please provide a timeline for the remainder of procurement? Anticipated timeline is as follows:20/12 suppliers notified of stage 1 outcome 21/12 shortlisted suppliers will receive full statement of requirements and be invited to bid / launch of stage 2 proposal. This will be via the DfT e-sourcing portal (Jaggaer) 12/01 proposal deadline 24/01 shortlisted suppliers notified of stage 2 outcome 06/02 award contract to winning supplier, following standstill period Please note that the timeline is subject to change at DVSA’s discretion. 5. 1) Could we request some information on the tech stack we will need to take into consideration for this project?2) Do you already have an incumbent providing these services required? If so, who are they please, and how long have they been providing the services to you?3) Can you please confirm there will be no presentation/interview as an additional assessment method? If there will be presentations, when are they expected to be held? 1)We are not building a working prototype in Discovery. As such DVSA is tech agnostic at this stage and its main concerns are understanding the problem and having confidence that a working prototype can be later built. Assuming we move to Alpha, a working prototype would start to take account of our tech stack.2) There are currently no services in place for this work. However, a proof of concept was completed for this project in 2019/20. The supplier who delivered this outcome was Atos.3) There will be no presentation/interview as an additional assessment method. 6. Regarding Essential criterion 4 – could you please confirm whether you are referring to data science algorithms or an algorithmic logic to running a proof-of-concept? We're not experts in this field. To be clear, running a proof-of-concept is not a subsequent and separate stage of work, but part of this contract. We expect to see a proof-of-concept that gives us confidence it’s worth building a working prototype. This doesn't mean a solution deployed in a live test. The proof-of-concept can be simple, for example an output that we manually load into our Satnav to allow local managers to validate an automatically generated route. It is for the supplier to consider whether they need data science algorithms or algorithmic logic or both to deliver the outcomes. 7. 1. Will suppliers be expected to budget for up to 14 weeks’ work within the quoted budget, or would the budget be extended should this further work be required?2. Did an external supplier carry out the previous PoC activity referenced? If so, who was this supplier?3. Given the Christmas period, it is likely that many suppliers will struggle to find available resource to complete Stage 2 (if shortlisted) by 12 January. Can the deadline be extended to account for this? 1. Suppliers are expected to budget for up to 12weeks’ work within the maximum budget stated in the listing. There is the option for a 2-week time-only extension. 2. A proof-of-concept was completed for this project in 2019/20. The supplier who delivered this outcome was Atos3. DVSA have considered the Christmas period and allowed for a longer period of time (almost 3 weeks) for stage 2 submissions to be completed. The timeline has been developed to ensure that the latest contract start date can be adhered to. 8. 1. The DOS ad mentions a PoC from 2.5 years ago. Will the new supplier be expected to resurrect and re-use the existing PoC in any way?2. The stage 2 Proposal criteria asks about the supplier’s approach to the PoC. Are you expecting a new coded PoC to be created during the 12 week Discovery? 1. The supplier will be expected to build a new proof-of-concept as the requirement has evolved and the technical landscape is likely to have changed since this work was carried out.2. Yes, new code will be required as part of the proof of concept. 9. 1. Please can you clarify the “…and likely costs” element of the final Stage 1 Essential question. Are you expecting suppliers to provide cost estimates as part of this answer, or are you just looking for evidence of experience of defining Alpha costs, or something else?2. Are you looking to purchase an “outcome”, and can suppliers suggest a shorter time-period, or is the 12 week duration fixed in your view? 1. We are not expecting suppliers to provide cost estimates as part of their response, rather demonstrate that they have experience of defining indictive costs for Alpha.2. We are looking for an outcome, as such suppliers can suggest a shorter time-period provided the outcomes can be delivered within this time. 10. 1. GDS typically expects a Discover to take 4-8 weeks. Is there a specific reason that you are proposing 12 weeks, e.g. due to limited availability of examiners etc? If so, are you expecting the supplier’s team to perhaps not operate full-time over the 12 week period?2. Is the winning supplier likely to be awarded the Alpha work, or will that definitely only be subject to a separate procurement activity? 1.Suppliers can suggest how best to fulfil the requirement to achieve the outcomes. For example, over a shorter time-period or not operating full-time over the 12 week period. 2. The next stage of the project (Alpha) would be the subject of a competitive procurement process and is out of scope for this contract. 11. The main “Location” on the advert states that all work can be done remotely from no specific location. However, the details in the Work Setup section explain that work is likely to be at least partly on-site in Nottingham. Please can you confirm which is more accurate? Most of the work can be done remotely. However, there is likely to be the need for some work to be carried out in test centres which will be in the Nottingham area. 12. Would DVSA be open to suppliers working with partners to deliver, if it can be demonstrated that a superior outcome can be achieved by using partners? Yes, DVSA are open to suppliers working with partners to deliver the outcome.