Data Engineer

Tender description

Summary of the work The data engineer will be integral in designing and implementing data solutions for STA's portfolio of digital tests. You will lead on the development and implementation of data storage solutions and digital data flows. You will also provide learning and development to existing team members to upskill the team. Specialist role Data engineer Expected Contract Length Four months Latest start date Monday 19 July 2021 Maximum Day Rate £690 max excluding VAT including T&S Early Market Engagement Not applicable Who Speclialist Work With Working within the Standards and Testing Agency, an executive agency of the Department for Education, you will work with colleagues in the Assessment Development, Data and Digital teams as well as external contractors to apply your specialist knowledge of data engineering to the ongoing development of new digital systems. What Specialists Work On 1. Scope the requirements of current and future systems, as well as working with colleagues within the department to understand and align with the department's data strategy, 2. Develop data validation processes throughout data flow, working with colleagues to ensure their data needs are met, and deliver testing and assurance of data systems, 3. Implement data storage solutions (e.g. data warehousing) for existing digital platforms and any future developments, ensuring solutions are in line with departmental data strategy 4. Upskill existing team members in order to grow capability and resilience in this area. Skills & Experience • Have experience of leading, up skilling and motivating a team in a multi-disciplinary environment, delivering products within specific time- scales and working alongside multiple senior stakeholders • Be an accomplished ETL developer who has proven experience of delivering complex big data solutions and a track record of working with structured and unstructured data. • Have extensive working knowledge of Data Warehouse techniques for dimensional data modelling, relational database design, ETL techniques and data analysis • Have knowledge of data architecture concepts / technologies e.g. data warehouse, data lakes, big data • Have expert level knowledge of data reconciliation, ETL error handling and error logging mechanisms • Have advanced coding skills in SQL and knowledge of JSON or XML Nice to Haves • Have experience using R to extract, transform and validate data. Work Location Work is expected to be mainly remote but some on site working at either the London, Coventry or Sheffield office may be required. Working Arrangments To be agreed at start up meeting. Security Clearance BPSS clearance No. of Specialists to Evaluate 5 Cultural Fit Criteria • be able to deliver learning & development to a range of audiences and skill sets • work as a team with our organisation and other suppliers • be transparent and collaborative when making decisions Assessment Method • Interview • Presentation Evaluation Weighting Technical competence 75% Cultural fit 5% Price 20% Questions from Suppliers 1. Is there a current incumbent? No, this is a new post. 2. What is the IR35 status? IR35 does not apply. 3. Does the specialist require any Cloud-based experience? Can you specify which? Yes, familiarity with Azure cloud hosting. 4. What is the expected frequency of travel to Sheffield, Coventry and other locations? As needed. 1-2 times a week. 5. Could we have a tentative timeline for the procurement process please? CQs deadline 21 May. Bids deadline 4 June 2021. Target Award Date 18 June 2021. Contract start date 21 June 2021. 6. Can you please confirm which ETL tool will be used? The supplier will have to work across units in the DfE to understand what solutions are possible. 7. Travels to one of your offices generate additional costs. Do you want them to be included in the daily rate? We would have to assume specific number of travels per month. Or, do you accept additional costs (flights, trains, accommodation, per-diems)? Any additional costs - e.g travel, should be included in your daily rate and should not exceed the maximum daily rate of £690 as listed in the advert 8. Would you consider the “minimal number of travels” scenario, when the specialist spends a few days with you in the beginning, and a few days somewhere in the middle of the project? Everything else is done remotely. Yes. 9. Are you looking for literally one specialist or would you accept small team? If yes, is 690 GBP you maximum budget for daily expenses (i.e. whole team cost mustn’t exceed that value)? We are looking for 1 person. The £690 is the maximum budget. 10. Assumed duration is 4 months, but there’s holiday period coming. Have you taken that into consideration or 4 months could be exceeded? 4 months of working days are expected. 11. Is this position focused mostly on requirements gathering and system architecture design or mostly on development/implementation? it’s a mixture of both. 12. What part of the job is maintaining and supporting existing features versus implementing new ones? it’s a mixture of both. 13. Does this role include any work on frontend / UI / UX design or just pure implementation? There is no work on the front end but neither is it pure implementation. 14. Are there any rules regarding specific software/vendor being preferred or banned? The solution will depend upon the security measures and the tools within DfE. 15. Is the existing solution based purely on Azure cloud or it includes on premise infrastructure? It is cloud based. 16. What is the ETL tool currently in use? This is to be advised as this is a new project. 17. Can we split the requested role between two of our experts? No.