Is big data the answer?
Digitalizing agriculture to make it smarter
Ahead of the 2017 Climate Change Conference in Bonn, scientists across CGIAR identified digital agriculture as one of the “10 best innovations for adaptation in agriculture” that can help achieve food security under a changing climate, while also delivering co-benefits for environmental sustainability, nutrition and livelihoods.
Despite its critical role, agriculture has lagged far behind other sectors in the development and implementation of digital tools. A recent report by the management consulting firm McKinsey & Company found that, even in industrialized economies such as that of the United States, the agricultural sector ranks 23rd out of 25 industries in digitalization, and the rate of adoption is slow. When it comes to low- and middle-income countries, it appears that these innovations have an even smaller foothold.
If the agriculture sector is to overcome current challenges to increase productivity, adapt to climate change, implement environmentally sustainable solutions and, ultimately, achieve food security for the future, it needs to get smarter. It needs to get digital.
The CGIAR Platform for Big Data in Agriculture is well on the way to achieving its goal to positively disrupt agriculture research and democratise agricultural data, helping to generate impactful big data innovations that can revolutionize farming in developing countries.
Since its launch in 2017, the Platform has made considerable advances across each of its three strategic pillars.
CGIAR Platform for Big Data in Agriculture’s position as thought leader on the equitable use of big data in agriculture has been recognised repeatedly over the past year, appearing in internationally renowned publications such as Nature, Reuters, ABC Rural (AU), Food Tank and others, for its work and expertise in the sector.
Over the past year, the Platform has provided support to CGIAR and partners to fully comply with open-data and open-access principles, addressing the technical and organizational challenges of strengthening data analytical capacity and developing practical, big data-driven uses and methodologies.
Increased data capacity across our 15 partner centers
As a result of allocating a series of grants to each of our 15 partner centers, we have seen a series of inspiring trends emerge as each center implements strategies to mobilise data.
- Increased investment in developing data repositories and software infrastructure to build open data sharing and storage capabilities
- Investment in new staff roles for data curation, collection, and analysis, while providing additional training for current staff
- Reallocation of staff resources to collect, store, and unlock data
Responsible data sharing guidelines
While the enthusiasm for data sharing grows, we have been working to ensure that data sharing and use comply with ethical standards that protect those who could be vulnerable to exploitation. We have completed surveys of each of the 15 centers’ privacy and ethics standards, and are in the last stages of developing a set of guidelines to help researchers navigate the evolving implications of technology, confidentiality, intellectual property, consent, access and sharing of benefits.
In line with the aim that research data should be findable, accessible, interoperable, and reusable (FAIR), the Platform has built and launched a robust prototype of an open access, searchable data harvester, GARDIAN (Global Agriculture Research Data Innovation and Acceleration Network). The harvester, which spans databases across all CGIAR centers, applies semantic web methods that enables searching across domains.
More community for our Communities of Practice
Our Communities of Practice (CoPs) aim to increase the capacity for problem solving, across the CGIAR and partner network, through collaboration and the sharing of information and knowledge. To increase this working capacity of our CoPs, we established new communications strategies and digital tools that will open up communication channels within and between the communities, with the Platform itself, as well as with the wider scientific community.
During the past year, this platform has brought together big data practitioners, in partnership with global private sector brands, local entrepreneurs, universities, and others, in spaces that encourage interaction and produce innovative new ideas to solve development problems.
Big Data in Agriculture Convention: Alliance for a Data Revolution
On 19–22 September 2017, CGIAR gathered over 300 local and international researchers, non-profits, public and private sector actors for the Platform’s inaugural annual convention, hosted by the International Center for Tropical Agriculture (CIAT) in Cali, Colombia. The Convention marked the programmatic launch of the Platform, which aims to enable the development sector to embrace data and other digital technology approaches to solve agricultural development problems faster, better, and at greater scale.
New partnership to bridge data gender gap
The Platform has teamed up with CGIAR’s Collaborative Platform for Gender Research to co-design innovative uses of data science to help bridge gender divide in data; a key step to ensure women are not left behind in digital agriculture revolution and that the digital transformation is inclusive. The partnership will serve to ensure gender is embedded in the design of the work supported by the Big Data Platform, helping to both increase the impact of our work and advance CGIAR gender research.
Private sector partnership for geospatial analysis
The Big Data Platform negotiated an alliance with Amazon Web Services and imagery provider Digital Globe, enabling researchers across CGIAR to access high-resolution satellite imagery and run analysis on it in the cloud.
Last year we challenged partners, universities, and others to use our data to create pilot projects that democratize data-driven insights to inform local, national, regional, and global policies on agriculture and food security in real time. We received more than 120 proposals from applicants from 37 countries.
During our September convention, we awarded a grant of US$100,000 to each of the five winning ideas; Seeing is Believing: Personalized Crop Advice for Rural Farmers (CABI & IFPRI); Pest and Disease Monitoring Using AI (CIAT, CIP & IITA); Facebook Chatbot Livestock Advisory Service (FarmInk & ILRI); IVR Marketing Service (Voto & CIMMYT); and Real Time Diagnostics for Wheat Rust (EIAR & CIMMYT).
Each of the projects has shown significant progress and is now in execution and trial stages, with all expecting to begin preliminary in-field testing between July and September of this year.
Today, one in four people in sub-Saharan Africa is malnourished, and famine and food shortages are already affecting South Sudan, northern Nigeria, and Somalia.
Since at least the 1970s, food crises have seemed to strike sub-Saharan Africa with frequency. They often require marshaling a complex and costly international response and can produce a sense of hopelessness that this region is somehow fated to suffer an endless cycle of food-related disasters and millions of people will continue to face chronic malnutrition. The Food and Agriculture Organization of the United Nations (FAO) recently indicated that the number of people without access to adequate calories in the world has increased since 2015, reversing years of progress. Moreover, it is malnutrition and its consequences – not just bouts of extreme hunger and famine – that is the enduring problem.
The current global response to malnutrition is generally reactive rather than proactive; interventions are limited to households and community level instead of coordinated at a national level; data is lacking to enable decision-makers to act; and inadequate measures are used to detect often subtle factors leading to food shortages before chronic malnutrition sets in.
One in four people in sub-Saharan Africa is malnourished. Photos: Stefanie Neno / CIAT
The approach proposed by CIAT and its partners aims to fast-track solutions to meet global commitments to end hunger and malnutrition by 2030. It will roll out a Nutrition Early Warning System (NEWS), in support of the African Development Bank’s U$1.1 billion “Say No to Famine” program.
NEWS outlines an innovative way to fight food insecurity and malnutrition based on an artificial intelligence (AI) technique known as machine learning, in which computers process complex and constantly changing data in order to “learn” and make predictions. NEWS will apply this technology to understand the past and current world conditions, predict what will happen next, and give a menu of options on what to do based on these factors. For example, the system would search for early signs of crop failures, drought, rising food prices, and other factors that can trigger malnutrition. Over time it would become “smarter” and more accurate.
CIAT nutritionist Dr. Mercy Lung’aho, who has been leading the NEWS initiative, said: “In a rapidly changing and unpredictable world, a transformative agenda for nutrition requires that we explore new (disruptive) approaches for addressing malnutrition. CIAT is making the case that we need a paradigm shift in the way we are addressing malnutrition in both the presence and absence of crisis.”
In the context of crisis, such as drought and famine: Despite progress in strengthening early warning systems for food insecurity such as FEWSNet, current approaches to detect deteriorations in nutrition still tend to be ‘late’ warning systems, reliant upon indicators that are only able to detect a nutrition crisis after it has already taken hold. However, famine or drought, even mild, are shocks that add new cases of undernutrition to already existing ones. Therefore, a shift to preventative actions will require a shift in the way we conceptualize nutrition resilience, nutrition security, forecast nutrition-related threats and shocks among communities, identify the causal factors driving poor nutrition, and design nutrition-sensitive solutions that mitigate the impact of shocks and crisis on households and communities.
In the absence of ongoing crisis: We need a disruptor to better strengthen the evidence-base for action, inform decision- and policy-making, and track progress toward goals and commitments. Nutritionists agree that there are numerous factors that explain poor nutrition. We need to better define the problem of malnutrition, diagnose its root causes, design impactful interventions, and track progress in Africa. However, before developing new tools and platforms to enable collection of new data to improve its analysis and use, we need to ask if we should instead look at all the data that is relevant to nutrition. Have we fallen victim to either “functional fixedness” or “filter failure” – both limiting biases – where we only use data in the way it has traditionally been used or, in the case of filtering, where we are blind to or discount subtle signals that may indicate emerging causes or confounding factors for malnutrition or its interventions.
The fact that malnutrition has been increasing on the continent is evidence that existing tracking systems ignore or miss key details. This is where NEWS, big data and AI will help.
- I want to end malnutrition in Africa. That’s why I’m turning to big data and algorithms Mar 2018
- Artificial Intelligence and Big Data to fast-track the fight against malnutrition in Africa Sep 2017
- Enhancing Africa’s Nutrition Resilience through Artificial Intelligence (Report, Sep 2017)
- Good NEWS for the Fight Against Malnutrition May 2017
Since 2013, a team of researchers at CIAT and the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) has worked with local and national government institutions, national growers associations, and weather agencies in Colombia and Honduras to provide farmers with advice on which crops to grow, and if and when to plant.
The team first set up online systems to capture and disseminate crop and climate information. Then they used innovative approaches in modeling, climate forecasting, big data analytics, and artificial intelligence to develop recommendations for farmers. Subsequently, they discussed and analyzed these with local partners, who in turn shared them with farmers through an online platform, monthly agro-climatic bulletins, and regular meetings.
So far, this work has reached an estimated 300,000 farmers in the two countries. In 2014, the team’s work also helped prevent an economic catastrophe: 170 Colombian rice farmers dodged a staggering US$3.6 million in losses after following the advice of their national association, FEDEARROZ, of not to plant the crop in the first of two annual growing seasons. That advice was based on information from the bulletin.
Subtle shifts in rainfall plus more extreme climate events force Colombian rice farmers to toss aside familiar assumptions about what varieties to plant and when. Photo: Neil Palmer / CIAT
This groundbreaking work took the prize for the 2017 Momentum for Change Lighthouse Activities’ ICT Solutions category awarded by the United Nations Framework Convention on Climate Change (UNFCCC). The team also won the UN’s Big Data Climate Challenge in 2014 for early work on the project.
“As a result of our work, we’ve seen a fundamental change – a transformation, in the way farmer organizations plan their businesses. But beyond that, this award is proof of the hard work of our partners and our team of 30-plus researchers in making sustainable and effective climate services a reality for thousands of farmers,” said project co-leader and CIAT/CCAFS climate impact scientist Julian Ramirez.
For Ana María Loboguerrero, CCAFS Regional Program Leader for Latin America, working closely with local partners was essential to the success of the project.
“What CIAT and CCAFS are doing, in bringing institutions together with the purpose of enabling farmers to respond effectively to climate variability, is a first in Latin America. We work closely with farmer organizations and public institutions, and empower them to use tools and methods to deliver timely, actionable, usable, and accurate information on crop management. We are essentially bringing vitally important climate information closer to the people so they can use it to make better informed decisions.”
CIAT scientist Satoshi Ogawa likens the way a new gadget he has been testing, called e-kakashi, functions to how Facebook works.
“On Facebook, you can see what everyone is up to. It’s the same with e-kakashi, but only for farming. You can view instantly how your crops are doing, plot by plot, plant by plant,” says Ogawa, who is leading efforts to test the capability of the technology on CIAT fields.
So it’s like getting automatic status updates from your crops.
E-kakashi comes from Japan, and in Japanese, kakashi means scarecrow.
This modern version of a scarecrow, however, works differently. E-kakashi involves connecting a “mother” device to dozens of “baby” terminals that collect without delay large volumes of agricultural data from hundreds of sensors placed around the field, including air temperature, relative humidity, solar radiation, carbon dioxide levels, and soil temperature and humidity.
The device provides a concrete application of the “Internet of things” (IoT), or the ability of everyday gadgets to connect to the Internet and with each other.
CIAT established the trial with the e-kakashi platform in June 2017 and four mini terminals are currently being tested with rice. The objective is to collect enough data on the rice growing process and how it is affected by different factors, to build a model that can estimate when plants enter critical stages such as flowering, heading, and harvesting, and issue better recommendations to farmers on how much fertilizer to use, whether to increase irrigation, or if there’s a need for more labor.
The resulting model will form the basis of an app that can alert users when, for instance, water in the soil reaches dangerous levels.
Ogawa believes that installing more of these devices in Colombia would be critical to improve rice production. In Japan, over a thousand platforms of sensors have been installed, of which 300 are e-kakashi.
In fact, Japan’s Ministry of Internal Affairs and Communications and Colombia’s Ministry of Information and Communication Technologies recently signed a memorandum of understanding to launch pilot projects that take advantage of information and communication technologies in the field of agriculture.
Colombia’s Ministry of Agriculture and Rural Development (MADR) and the National Rice Growers Association (FEDEARROZ) have also agreed to assess the tests being run at CIAT.
During a visit to CIAT in January 2018, Japan’s State Minister of Internal Affairs and Communications, Sakai Manabu, said:
“We hope to see the results of [e-kakashi’s] implementation on the ground, because there are many possibilities for application in Colombia due to the great potential for improving agricultural systems.”
“We hope that the new research studies, now being reviewed, will build on the advances made so far, and that they will continue to receive CIAT’s support as they move forward,” he added.