<aside> 👉 This page includes:
For organisations at the early stages of their digitalisation journey, manual data processing can be a simple way to collect and store information. Tasks like downloading datasets and entering data manually initially feel easy and be the path of least resistance.
As organisations mature, the limitations of manual data management surface and can become a significant barrier preventing an organisation from achieving its goals.
Beyond a certain point, it is better to invest in changing approach to data management, rather than continuing to use manual processes.
This page summarises key stages of data processing. In the sub-pages you’ll find practical worked examples that relate to energy data.
It can be hard to know where to begin at getting started with tasks like automating data ingestion. One effective and popular way to automate the process is to use Application Programming Interfaces (APIs).
APIs allow for seamless, programmatic access to data, eliminating manual downloading and data entry and so ensuring consistent and scalable data structures for process improvement.
We’ve produced a walkthrough showing how to interface with the England and Wales Energy Performance Certificate API.
<aside> 👉 A walkthrough on accessing the England & Wales EPC API data
APIs, an Energy Performance Certificate (EPC) example
</aside>
It is highly unlikely that the data we collect will come pre-packaged in the exact format we need it in. In order to extract the information we need from the data, we transform it into our required format.
Data transformation refers to the process of changing the format, structure, or values of data to fit the requirements of a system or task. This can involve tasks such as cleaning, standardising, enriching, and integrating data.
We’ve produced a walkthrough showing how to transform the data collected from the England and Wales Energy Performance Certificate API. This continues on from our previous walkthrough.