DWP Datawork Data Engineer

Tender description

Who Speclialist Work With Working as part of a multidisciplinary agile team maintaining and improving the Universal Credit Data Platform, they’ll work with Network Architects, Technical Architects, Senior Data Engineers, DevOps Engineers and Business Analysts within the team. What Specialists Work On They will identify, build, modify and manage data flows between complex systems using the appropriate solution. They will work on the design and implementation of a data streaming service, API's to expose data to our users and their systems. Use their Big Data knowledge to optimise the current platform to ensure it is working efficiently as possible. Manage code versioning using Git. Work in an agile environment following a prioritised backlog. Where the Work Takes Place One Leeds City Office Park Meadow Lane Leeds LS11 5BD Early Market Engagement Working Arrangements 5 days a week with minimum 3 days on-site in Leeds working as part of a multi-disciplinary agile delivery team with work being specified by a work management system (JIRA). You will attend daily stand-ups and relevant show & tells to monitor/discuss progress. Work Location One Leeds City Office Park Meadow Lane Leeds LS11 5BD Security Clearance Security Check (SC) Additional T&Cs Additional DWP Terms and Conditions will apply. Skills & Experience Designs, writes and iterates code from prototype to production-ready. Understands security, accessibility and version control. Can use a range of coding tools and languages. Has been involved in the introduction of a data stream service based on Java, Scala or python. Optimises the code to ensure processes perform optimally Has knowledge of specific technologies Hadoop, Flume, Sqoop, Map Reduce, Hive/Impala, Hbase, Kafka, Spark Streaming, experience with ETL tools incorporating Big Data (like Informatica, Talend etc...), Shell Scripting, Python Has a demonstrable understanding of how to expose data from systems (for example, through APIs), link data from multiple systems and deliver streaming services. Data Manipulation (SQL/No-SQL) DevOps (GitHub, Ansible, Continuous Integration, Test Driven Deployment) Produces data models and understands where to use different types of data models. Able to reverse engineer a data model from a live system. Nice to Haves No. of Specialists to Evaluate 3 Cultural Fit Criteria Work as a team with our organisation and other suppliers Be transparent and collaborative when making decisions Have a no-blame culture and encourage people to learn from their mistakes Share knowledge and experience with other team members Challenge the status quo Assessment Method Evaluation Weighting Technical competence 70% Cultural fit 10% Price 20% Questions from Suppliers Budget range the maximum rate dwp is willing to pay is £550 which is exclusive of vat and reasonable and agreed expenses up to a cap of £90 per day. the day rate will be evaluated as per the evaluation criteria %.