How Shaping Cloud helped build the Data Science platform for UK Agriculture
Context & Drivers
Agrimetrics is a joint venture between Rothamsted Research, University of Reading, theNational Institute of Agricultural Botany and Scotland’s Rural College. The model for the centre was developed with input from over 50 UK organisations, spanning the agri-food industry and adjacent sectors. Agrimetrics was granted Innovate UK funding in October 2015 and tasked with creating an International Centre of Excellence for data science and modelling of the agri-food system.
This data integration, modelling and analysis platform has several key success criteria.
- It will make existing data sets accessible through a single point as well as allow access to new data sets arising from sources including the other Centres for Agricultural Innovation. A wealth of data relevant to the agri-food sector is already in existence, but access to the data can be difficult, and its existence is not always known.
- Agrimetrics will add value to the data in the portal by enabling the integration of data from disparate sources, and by providing access to modelling and analysis tools to generate new knowledge and innovation. This data integration, modelling and analysis platform will provide users with the ability to undertake their own analyses and visualisations.
- The organisation will take a clear leadership role in the development of rigorous and internationally accepted metrics of sustainable intensification. There is huge difficulty in developing metrics that encompass different climates, cropping systems, and levels of technology.
- Processes will be put in place to allow for different access, charging and IP arrangements for the data sets identified, processed, curated and developed by Agrimetrics. Data sets (from third party data owners, or data products developed by Agrimetrics) will be made available directly, through data re-sellers, or via Application Programming Interfaces (APIs) to allow third parties to develop applications based using data on the platform.
- Agrimetrics will undertake a wide range of knowledge exchange activities with the aim of raising awareness and increasing levels of adoption of big data and informatics approaches in the agri-food sector.
- In order to stimulate the market, Agrimetrics will also look to develop first party apps and services that demonstrate the capabilities of the platform.
For a project of this size and ambition the key deliverable for Shaping Cloud was to lay a solid foundation for the platform in order to allow it to grow and scale over the next 3 and a half years, the first phase was split into 2 core deliverables:
- Implementing the SC:Transform methodology in order to design and build the Cloud Infrastructure that will act as the foundation for the ongoing development of the data platform, incorporating:
- The over-arching infrastructure and technical design of a hybrid cloud environment optimised for Linked Data and Data Science operations.
- Procurement & Installation of hardware in order to create an Agrimetrics private cloud.
- Configuration and setup of a public cloud environment built on Microsoft Azure.
- Completing the hybrid environment by connecting private and public clouds, leveraging the JANET network.
- An SC:Develop engagement in order to design & build a 1st party app (FarmAssist) that demonstrates the early potential of the platform.
- The primary goal of the app was to demonstrate to the farming community how through them sharing just a small amount of their data with Agrimetrics they are able to gain a greater insight into their business.
- The app allows farmers to compare their average yields against a set of industry benchmarks
- It also allows them to gain greater insight into how other crop types may fare in the weather conditions and soil type for their specific farm.
- The app pulls in data from a wide array of external and 3rd party providers that many farmers would not have direct access to.