Automation Test Specialists

Tender description

Summary of the work Team of subject matter experts to work alongside internal testers to help build and maintain automation frameworks and scripts of new/existing systems. Also to train and support the current test team in BAU tasks as well as identifying areas of improvement and making recommendations and assisting in implementing said recommendations. Expected Contract Length 12 months Latest start date 01/04/2017 Budget Range £100,000 for the duration of the contract. Why the Work is Being Done To build test automation capability in-house and to increase the coverage of automation for internal and external facing systems across the organisation. Problem to Be Solved We have a large number of projects and systems that require a supplier to provide technical expertise in helping to build the automation capability within the testing team. We are therefore looking to engage a supplier who can act as a partner to the IPO to help increase the automation coverage of new and existing systems, but more specifically to up-skill our existing team and turning them into subject matter experts. Who Are the Users The Internal Testing Team require automation test capability to support the delivery of key new and existing projects as well as increasing automation test coverage of already live systems. To assist in this process of developing automated test capability we need a supplier to act as a partner and subject matter expert to deliver various pieces of work over the life of the contract that will facilitate this goal. Early Market Engagement Aside from a general awareness that this requirement has been fulfilled by suppliers in other government organisations, no formal market engagement has taken place. Work Already Done Some of our systems are automated, however, there is a need to increase the coverage across all working systems and systems in development. Existing Team The successful supplier's team/specialists will sit within agile multi-disciplinary teams but will be fully supporting the testing team requirements. Current Phase Not applicable Skills & Experience • Have extensive experience across industry recognised test automation tools • Have experience of using Selenium Webdriver with Visual Studio / Specflow / C# • Have experience of Meter / Visual Studio Performance Testing • Have Experience of designing and creating re-usable automated test scripts and procedures • Have good understanding of programming languages .NET/Java/Scala and database technologies MS SQL Server • Have experience of working and delivering within an Agile methodology • Have the ability to up skill and knowledge transfer • Have Agile methodology (Scrum / Kanban) skills and experience • Have BDD / Cucumber / Gherkin skills and experience Work Location Intellectual Property Office, Cardiff Road, Newport, NP10 8QQ Working Arrangments Individuals will work as part of an agile delivery team working to a delivery manager for tasks and a test manager for ways of working. Work pattern will be standard office hours Monday to Friday (40 hours per week), however it is expected that a flexible approach will be taken and agreed between the IPO and successful supplier as each SoW is agreed throughout the life of the contract. Security Clearance Supplier's teams will need to be SC cleared. Additional T&Cs N/A No. of Suppliers to Evaluate 5 Proposal Criteria • Detailed explanation of the proposed approach including timescales • Explanation of how knowledge transfer to IPO staff will be achieved to enable them to take on future testing and maintenance without 3rd party support • Description of Agile skills and the use of the defined testing tool set • Details of proposed resources including skill set and expertise • Details of resource capacity and how this is flexed in response to client demand • Case study evidence of delivery of similar services to similar clients • Commercial rates for proposed resource and how these demonstrate value for money Cultural Fit Criteria • have experience of working in a team • have excellent Communications skills • work in a transparent and collaborative manner • transfer knowledge and experience to civil servants to support learning from this engagement Payment Approach Capped time and materials Evaluation Weighting Technical competence 70% Cultural fit 10% Price 20% Questions from Suppliers 1. Any flexibility for the winning supplier to provide the testing service offsite (from supplier`s office) and then if needed onsite visits? No, as we are looking for a supplier to up-skill our internal test team and operate in the capacity of a partner to us for the duration of the contract, we do not believe the requirement is suited to providing an off-site testing service. As such the supplier will be expected to be on-site during the contract. 2. How many testers do you believe this will require? Would you accept individuals from separate companies or do you only want bids from suppliers who can provide an entire team? We are looking to work with a single supplier for continuity purposes over the duration of the contract. In terms of the number of testers one of the first pieces of work (SoW's) will be to work with the supplier to come in and review the test automation the IPO currently has, and then to come up with a plan of how we go forward over the duration of the contract. That will subsequently formulate how many testers we need and when. 3. The budget set at £100,000 for a team seems extremely low, is this an error, this equates to £8333 per month and would give you roughly 2 mid level testers a month for that, any clarification would be welcome including the size of the team you require? Whilst we are looking to engage a supplier for a year's contract, we do not anticipate that this will be on a full time basis, we view the contract as being a series of call-offs/SoW's based on specific needs over the life of the contract.In terms of the number of testers the first piece of work (SoW's) will be to work with the supplier to review the test automation the IPO currently has, and then to come up with a plan of how we go forward. That will subsequently formulate how many testers we need and when. 4. Can you advise on the IPO's position on IR35 relating to this requirement please? The IPO are currently considering their position on IR35 across all contracts that may be covered by the regulations. We are currently waiting for the publication of the HMRC tool that will provide indications as to whether contracts/individuals are within or IR35 or not. Until that is published we are unable to definitively confirm our position.However, based on our current understanding of the regulations and the way we intend for this contract to operate, I.e. outcomes based, we envisage this contract would likely be outside IR35. 5. You mention that consultants will have to be SC cleared, if we have suitable consultants without SC clearance, can we get clearance after approval? Yes SC clearance may be obtained for suitable specialists following contract award. 6. From the Q/A are you looking for different roles to start with ? What I understand is that there will be a test stratergy role early up followed by a doer role as mentioned in the job spec is there a understanding of the breakup and time expected for each of these roles ? The first piece of work (SoW's) will be to work with the supplier to review the test automation the IPO currently has, and then to come up with a plan of how we go forward. That will subsequently formulate how many testers we need and when. As such we expect one or two subject matter experts would be needed for approximately one month for this initial review and recommendations, we would then bring in the additional tester resources as and when required. 7. We would propose that part of the upskilling be done through standard training courses that our consultants deliver to other clients. Is this acceptable or would you always envisage bespoke training courses be delivered? In principal yes this would be acceptable, but we would first need to consider whether the courses are necessary and applicable before formally agreeing to this method of up-skilling. 8. Will the successful supplier be required to use equipment provided by IPO, e.g. laptops, in order to perform this work? If yes, will such equipment be available on the first day of work? Yes the successful supplier will be required to use equipment provided by the IPO and this will be available from the first day of work. 9. We would like to propose an initial Discovery to understand the state of play at IPO and then agree each month a monthly set of outcomes with IPO based on your current priorities and the overall roadmap. Would this be acceptable to IPO? Yes this would be acceptable, as per previous answers the first piece of work (SoW) will be to work with the supplier to review the test automation the IPO currently has, and then to come up with a plan of how we go forward. That will subsequently formulate how many testers we need and when. 10. Can you please confirm if the following question on the application is correct. "Have experience of Meter / Visual Studio Performance Testing". We are assuming you mean JMeter / Visual Studio. If you are able to clarify that would be appreciated. Yes, this should read JMeter 11. Can you say more about the size and shape of the development organisation? How many people in test roles, how many in development roles, what the mix of civil servant and supplier people is? We currently have 19 developers (including contractors), 14 Testers (including contractors) with 4 developers and 1 testers scheduled to start within the next month. 12. "we are looking for a supplier to up-skill our internal test team" What percentage of total working time have you allocated for up-skilling of your permanent employees, please? How regular is this up-skilling activity and how do you measure its success? This will depend on the initial piece of discovery and scoping work and how we will move forward as a partnership, as described in other answers above. We would expect the upskilling percentage to be high at the start of the engagement and then tail off as the team builds confidence over time. Upskilling will be done as an on-going activity and success will be measured by the ability for the team to write automated tests. 13. Can Travel and Accommodation expenses be claimed as part of this work? We would not expect to pay travel and expenses for this piece of work although in the event a supplier is shortlisted and asked to submit a proposal we are happy to discuss options around how this subject may be addressed over the life of the contract. Ultimately this will be considered as part of assessing a supplier's overall value for money offering. 14. What is driving your decision to invest in automated testing? What would you consider to be a success with this investment? There isn't a drive to invest in automation as we currently already have automation, the drive is to upskill the team to be able to support and write automation themselves. The success element would be to see a difference in the ability of the team and their ability to write automated scripts. 15. Are you open to restructuring how your test team is organised in relation to delivery teams as part of this work? As per previous questions the first piece of work (SoW) will be to work with the supplier to review the test automation the IPO currently has, and then to come up with a plan of how we go forward. That will subsequently formulate how many testers we need and when. As such if there was a recommendation of needing a restructure then this is something we would consider. 16. How do you see the balance of writing automated tests to coaching and supporting your existing team in writing automated tests. Are you primarily looking for additional test resources or people who upskill your existing team? We are primarily looking for additional testers as part of a partnership arrangement to upskill our existing team, however, as part of this arrangement there may be a need for the additional testers to 'get involved' in supporting the testing requirements of particular/specific projects. 17. Outcomes are unclear. Would you prefer a bid that delivers as much outcome using all of your budget to one that delivers less for less money? We would prefer a proposal that delivers best value for money. As part of any shortlisted supplier's proposal it is up to them to demonstrate they have they the required skills and experience to meet our requirements and can bring value for money by adding value to the engagement. In terms of outcomes we are primarily looking for a supplier to upskill our existing testers - the plan for doing this will be determined via the first SoW referred to in questions above (see questions 2 & 3). 18. “”individuals will work to a delivery manager and a test manager” this suggests that the supplier will not have complete control of their day to day work. We would expect our consultants to be experts and to be able to define how they deliver the necessary outcomes while working to IPO general roadmap and delivery plans. Can you clarify how the delivery and test managers will influence what the supplier team will work on a week-to-week basis. The anticipated nature of the agreement is that the successful supplier will be engaged as subject matter expert and as such will provide direction to the IPO. Whilst individuals will work to delivery and test managers this will not be for the purposes of line management activity or removing control from the supplier; it will be for the purposes of managing the IPO/Supplier relationship and the delivery of relevant milestones and outcomes to the desired quality as agreed in an SoW. 19. You mention an existing automation capability used in the live environment. Could you outline the technical skills needed to contribute to this, such as the languages and frameworks used? The skills are all listed in the Skills and experience section of the published requirements. 20. How much of the work do you anticipate being on new systems compared to released systems (with existing automation)? It will most likely be 70% existing 30% new although this may change as work on the new systems is likely to increase in the coming year. It is expected that the SMEs will work very closely with our testers as up-skilling of our the existing team is paramount 21. Can you say more about how the *anticipated* work is currently divided between open source tools and the Microsoft stack? Is there an anticipated requirement for more expertise to be required in one area than the other? Our current framework uses Selenium Webdriver with Visual Studio/Specflow/C# so it is a mixture of both 22. • Hi Team Regarding the 'Automation Test Specialists' opportunity , we've the queries listed below 1. Could you please let us know if the application under test is a web based or standalone or a hybrid application? 2. What technologies have been used in developing the application under test? 3. Are there any existing manual test cases which are planned for automation? If yes, how many manual test cases are available? 4. We see that JMeter/Visual Studio Performance Testing is mentioned under " Essential skills and experience". Does that mean Performance Testing is also under the scope of the project? The in house applications are both web and windows based, however, the requirement is to help support our test team in building their automation testing capability. 23. Can you please clarify the level of security clearance that will be required for this assignment. if we have people landing in the country and getting BGV checks done will that suffice ? We expect anyone that is supplied to be SC Cleared although as per question 5 this may be obtained following contract award.