Data services

Tender description

In 2013, the Office for National Statistics (ONS) undertook a competition in accordance with the Restricted Procedure pursuant to the Public Contracts Regulations 2006, in respect of the servicing, collection, cleaning and delivery of nationally representative price data to produce inflation indices, the Consumer Prices Index (CPI) and the Retail Prices Index (RPI) (the Contract). ONS awarded the contract to TNS UK Ltd (TNS). The term was advertised as 3 years, with an option for ONS to extend for a further 2 periods of 1 year. The contract commenced on 1 February 2015 and has since been extended until 2020. It has become necessary for ONS to extend the contract for a further 2 years for the reasons set out below. ONS therefore intends to extend the contract to 31 January 2022 in accordance with Regulation 72(1)(b) of the Public Contracts Regulations 2015 (PCR). ONS produces a wide range of key economic/social statistics. The contract involves the collection of nationally representative price data from retail outlets for the UK's monthly inflation indicators, the CPI and the RPI. Services provided by TNS include: 1) Item selection: the participation, with ONS staff, in the periodic review and selection of items for a basket of goods and services and of a subset of those items for which the service provider will collect prices; 2) Sourcing of products and outlets: finding products that comply with the item specifications, and outlets in which to price those products; 3) Location enumeration: the enumeration of, and supply of information on, locations where this is needed for outlet selection; 4) Collection of prices: the collection of representative price data and descriptions of the products being priced for the subset of items for which the service provider will collect prices; 5) Data cleaning: the checking of the data for quality; 6) Data delivery: the delivery of the data to ONS; 7) Associated services: including but not limited to market information on prices, retailers, goods and services, and the twice yearly collection of price data (in London) for the Purchasing Power Parities surveys. At the point, ONS applied for funding to facilitate the contract, HM Treasury required ONS to comply with a number of obligations. One of which included exploration of alternative innovative mechanisms to collect data, following expiry of the contract. ONS has, over the last few years, identified a number of technologies which may streamline and modernise the collection process. In particular, web scraping and scanner data. However, ONS has identified issues in respect of both methods. For example, in respect of web scraping, this method relies on the website structure remaining static, and that website terms and conditions do not preclude web scraping. ONS is concerned that in light of the levels of consistency required in order to generate the CPI and RPI, it would not be able to place reliance on this technology and it may not be possible to scrape prices providing little or no prior notice. In respect of scanner data, this would require retailers to agree to provide their data to ONS. Whilst some progress has been achieved in this regard, use of this method would be solely reliant on the continued cooperation of retailers, which will not provide the level of certainty required to maintain the CPI and RPI. For these reasons, ONS has struggled to find a mechanism which would suit its requirements. However, on 27 April 2017, the Digital Economy Act (DEA) received Royal Assent. The introduction of the DEA brings about a fundamental change to the availability of data from retailers. In short, the effect of the DEA is that ONS is able to compel retailers to provide specific pricing information, which will enable ONS to seek certainty in respect of the acquisition of regular and up-to-date data in order to maintain the CPI and RPI. This makes scanner data an attractive mechanism to replace the existing method of data collection. Before making any changes to the coverage or the basic calculation of the RPI, ONS must obtain the prior consent of the Bank of England pursuant to the Statistics and Registration Services Act 2007. This process therefore constrains how and when changes can be made to the RPI and its underlying data sources and requires a long lead time in order for the impact on the RPI to be established, the Bank of England to be properly consulted and consent to the change to be given. It has therefore become necessary to temporarily extend the current contract, as data collection in the future will fundamentally change, such that a different type of supplier is likely to be interested in the "new" contract, and if a new supplier was appointed on an interim basis (i.e. between 2020-2022), this would give rise to substantial cost and duplication on the part of ONS.