Big data: big hope or big risk?

ID-100236071Hailed as the latest technological advance that could revolutionise development and agriculture (along with other sectors), “big data” has been the focus of several recent articles, most notably a series of articles published by SciDev.Net. In June 2013 a UN High level panel called for a “data revolution” emphasising the need for better data to track progress towards development goals. But what is big data and how can it aid poverty and hunger eradication?

Big data is not just large amounts of information but rather it’s about integrating infrastructure to collect data at every step of the development process and designing new data collection methods that can track development goals effectively. In particular, big data is being hailed as the big fix for the lack of reliable official statistics in developing countries. But there is no clear (agreed upon) definition of big data, one article stating “it is data generated through our increasing use of digital devices and web-supported tools and platforms in our daily lives”. Due to our increasingly digital society, the amount of data (from social media platforms, mobile phones, online financial services etc.) has grown enormously. A much quoted statistic states that up to 90% of the world’s data was created over just two years (2010–2012). The aim for big data is to use this sizeable knowledge source to add value to society. Driving interest in evidence-based policy making, big data is also being termed a movement, one that aims to turn data into decision making.

In May 2012 Global Pulse published a White Paper entitled Big Data for Development: Opportunities & Challenges, which highlighted the opportunities big data provides. In particular they explore the role of big data in describing what is happening, predicting what may happen and explore the reasons behind why things happen.

For agriculture, big data means information can be collected along the whole supply chain including from supermarkets, weather sensing equipment, digital images, and research papers. These data sets can then be transformed through analytics into actionable information. But this conversion is rife with complexities in terms of managing, processing, sharing and using huge amounts of data. [Read more…]