RDF Developer (Omega)

Tender description

Summary of the work Develop a consolidated pan-archival RDF metadata catalogue to meet users’ needs and implementing this by building ETL processes, an RDF triple store and API services, underpinned by AWS Neptune and AWS Elasticsearch. Specialist role Developer Expected Contract Length From start date until 30 June 2022, plus an optional extension period of up to 6 additional months Latest start date Tuesday 1 March 2022 Maximum Day Rate Up to £749 per day Early Market Engagement We ran a Discovery phase leading to plans for an Alpha using AWS Neptune and Elasticsearch. You can read more about the project: https://www.nationalarchives.gov.uk/about/our-role/plans-policies-performance-and-projects/our-plans/our-digital-cataloguing-practices/project-omega/ The Discovery phase produced a proposal for a new Catalogue Data Model using RDF, a new identifier scheme, and transformation routines for the existing data to the new model. We have held workshops identifying the key ways that staff managing the catalogue work with the data and what they would like in future. The archivist needs to search, analyse, add to, correct, edit and enhance record descriptions so that the catalogue is properly maintained. The archivist needs to work with catalogue entries individually or as large sets, making (or reversing) bulk changes, so they can work efficiently. The archivists need to understand the version history of the catalogue so they can be confident about where the information has originated. We now seek a second development specialist to speed up progress on the following work strands: • data extraction and transformation • infrastructure • editorial workflow • API. Who Speclialist Work With The specialist will work with a Technical Architect and two RDF Developers. The core in-house team is a Data Analyst/Product Manager, a Senior Archivist, the Service Owner and the Head of Cataloguing, Taxonomy and Data. The specialists will also work with a wider group of users including archivists across the organisation responsible for the management of the catalogue. What Specialists Work On We are developing a Pan-Archival Catalogue, bringing together record descriptions from multiple catalogues into a single new system. We require a developer to join a Technical Architect and two other RDF developers to work on the Alpha development. You will join a small team to deliver a new catalogue management system. This will involve developing API functions to search/select/add/export/edit/import/delete catalogue data; developing search for use by expert users (using SPARQL with Elasticsearch); developing an Extract, Transform, Load process to migrate catalogue data from multiple relational database (SQL Server) and RDF databases to a cloud based native RDF database (AWS Neptune). Skills & Experience • Have experience with using standards-based ontologies/vocabularies, such as W3C PROV data model, Dublin Core and W3C ODRL • Have experience of validating RDF data, for example using RDF SHACL • Have experience of creating and working with RDF databases and SPARQL, for example AWS Neptune • Have experience, knowledge and understanding of building Extract, Transform, Load (ETL) processes • Have experience, knowledge of Java/Scala and an understanding of working with mixed content in the context of large, semi-structured datasets • Have experience, knowledge and understanding of implementing resilient and secure systems using IAM in a cloud context Nice to Haves • Have experience developing a user interface/front end to support non-expert, editorial engagement with RDF • Have experience, knowledge and understanding of EAD3 and EAC-CPF Work Location No specific location, remotely except from monthly team work sessions at The National Archives at Kew (a couple of days per month). Whilst mainly working remotely, the Catalogue, Taxonomy and Data team is based at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU Working Arrangments Given the situation with COVID-19, remote working will be the main arrangement for this project. Some work may be conducted at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU, but only when it is appropriate and safe to do so. Although this has been advertised as a resource, we have checked the role requirements (to the best of our knowledge) using the assessment tool found at https://www.gov.uk/guidance/check-employment-status-for-tax and, in our opinion, for the role(s) as advertised the intermediaries legislation does not apply to this engagement (outside IR35). Security Clearance Baseline security clearance will be required. No. of Specialists to Evaluate 6 Cultural Fit Criteria • Have collaborative and flexible working approach, e.g. working with in-house technical and other digital specialists • Approach to supporting teams to adopt new technologies • Examples of delivering transition, knowledge transfer and handover of code • An appreciation for the importance of technical documentation as a means of ensuring ongoing maintainability of systems • Demonstrable commitment to a diverse working environment, with a team comprised of experts from a wide variety of backgrounds Assessment Method • Reference • Interview • Scenario or test Evaluation Weighting Technical competence 60% Cultural fit 20% Price 20% Questions from Suppliers 1. Is there any incumbent? No. 2. Are you open to working with a developer who doesn’t have directly prior experience with (but can pick up in the course of the project) W3C PROV / W3C ODRL / Dublin Core, if the person brings experience in ETLs, data extraction and transformation, infrastructure, AWS databases, custom APIs? Yes, we would open to working with a developer with that experience. Please bear in mind that, f your candidate is shortlisted for interview, there will be a technical test. 3. As a small business we do not hold resource in bench and can onboard external resources only after issuance of contract.How quickly will you need all the resources to be deployed for the project from the contract signing date ? Would you allow staggered resource deployment ? No, the approach you suggest is not suitable for this contract. We are seeking an individual specialist and we want to interview that person – not the company that will employ them. On this basis, there is no 'resource' to 'stagger'.