Developer - Scala (Play) and ETL

Tender description

Summary of the work Develop iterations of a consolidated pan-archival catalogue to meet users’ needs, including services/APIs in Scala (Play) and ETL processes. Specialist role Developer Expected Contract Length 6 months contract with possible extension of 3 additional months Latest start date Monday 31 October 2022 Maximum Day Rate Up to £700 per day Early Market Engagement We ran Discovery and Alpha phases. You can read more about the project: https://www.nationalarchives.gov.uk/about/our-role/plans-policies-performance-and-projects/our-plans/our-digital-cataloguing-practices/project-omega/ The Discovery phase produced a proposal for a new Catalogue Data Model using RDF, a new identifier scheme, and an Extract, Transform and Load process to migrate catalogue data from a relational database to RDF. We have built the first version of an API and are just starting to build the initial screens having held workshops identifying the key ways that staff managing the catalogue work with the data and what they would like in future. The archivist needs to search, analyse, add to, correct, edit, and enhance record descriptions individually and in bulk so that the catalogue is properly maintained. We now seek a specialist developer to speed up progress on the delivery of a Private Beta with the following work strands: • Data extraction and transformation into RDF • Cloud-based data services • Editorial web application services user interface for users to view, edit and save metadata • API services Who Speclialist Work With The specialist will work with a Technical Architect and two RDF Developers. The core in-house team is the Service Owner, Product Manager/Data Analyst, Delivery Manager, a Senior Archivist, and the Head of Cataloguing, Taxonomy and Data. The specialists will also work with a wider group of users including archivists and editors across the organisation responsible for the management of the catalogue. What Specialists Work On We are developing a Pan-Archival Catalogue and management system, bringing together record descriptions from multiple catalogues into a single new system. You will join a Technical Architect and two other Scala and RDF developers to work on multiple iterations of the Beta development. We will be continuing with the Extract, Transform and Load process and will be developing services in Scala, using Play Framework for the frontend, to enable users to search, select, add, edit, import, export and delete data via a UI. We will be developing further API services to enable integration between the Catalogue and other TNA systems. Skills & Experience • Have experience with using standards-based ontologies/vocabularies, such as W3C PROV data model and Dublin Core • Have experience of working with graph databases (for example AWS Neptune), SPARQL, and validating RDF data (for example using RDF SHACL). • Have experience, knowledge and understanding of building Extract, Transform, Load (ETL) processes • Have experience, knowledge of Scala and an understanding of working with mixed content in the context of large, semi-structured datasets Nice to Haves • Have experience developing a user interface/front end to support non-expert, editorial engagement with graph data • Experience of using Play Framework Work Location The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU Working Arrangments Remote working is the main arrangement for this project, with monthly team work sessions in person (2-3 consecutive days, occasionally up to 5 days) conducted at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU. Although this has been advertised as a resource, we have checked the role requirements (to the best of our knowledge) using the assessment tool found at https://www.gov.uk/guidance/check-employment-status-for-tax and, in our opinion, for the role(s) as advertised the intermediaries legislation does not apply to this engagement (outside IR35). Security Clearance Baseline security clearance will be required. No. of Specialists to Evaluate 4 Cultural Fit Criteria • Have collaborative and flexible working approach, e.g., working with in-house technical and other digital specialists, ready to attend monthly work sessions in person • Approach to supporting teams to adopt new technologies • Examples of delivering transition, knowledge transfer and handover of code • An appreciation for the importance of technical documentation as a means of ensuring ongoing maintainability of systems • Demonstrable commitment to working within a diverse working environment, with a team comprised of experts from a wide variety of backgrounds Assessment Method • Reference • Interview • Scenario or test Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers 1. just to confirm, this opportunity is Outside IR35? We refer you to the statement in the advertisement that, in our opinion, for the role the intermediaries legislation does not apply to this engagement (outside IR35).The HMRC CEST tool was completed by the contract manager making certain assumptions at this stage in the process. Should any of those assumptions prove incorrect, we reserve the right to re-run the HMRC CEST tool.HMRC remain the final arbiter as to whether IR35 applies or not. 2. Is there a current incumbent supplier? Yes, There is a supplier currently in the role whose contract ends on 30 September 2022 and is at liberty to re-apply. We wish to assure all potential suppliers that all bids will be treated on their own merit, and the incumbent will receive no special credit or favour. 3. If this process is to renew the existing current incumbent, why this role publishes on DOS for other suppliers to apply. How this could be a fair assessment ?Do you have preferred supplier?If we give a reduced margin, will you transfer current incumbent to our books? The 'incumbent' supplier's contract is coming to an end. The purpose of this exercise is not to 'renew' the existing incumbent; the current incumbent may or may not choose to apply. We will run this competition fairly and objectively, scoring all responses according to the framework methodology. There is no preferred supplier.We have no contractual ability to transfer the current incumbent.