CGIAR Research Program 2020 Reviews: Rice Agri-Food Systems (RICE)

Transcription

CGIAR Research Program 2020Reviews: Rice Agri-FoodSystems (RICE)Authors: Ravinder Kumar and Rattan YadavDecember 2020

This evaluation has been commissioned by the CGIAR Advisory Services Shared Secretariat (CASSecretariat) Evaluation Function (CAS/Evaluation). CAS/Evaluation encourages fair use of this materialprovided proper citation is made.Correct citation: CAS Secretariat (CGIAR Advisory Services Shared Secretariat). (2020). CGIAR ResearchProgram 2020 Reviews: Rice Agri-Food Systems (RICE). Rome: CAS Secretariat Evaluation Function.https://cas.cgiar.org/Cover image: Laguna harvest. Credit: IRRIDesign and layout: Luca Pierotti and Macaroni Bros

CGIAR Research Program 2020Reviews: Rice Agri-FoodSystems (RICE)Authors: Ravinder Kumar and Rattan YadavDecember 2020

AcknowledgmentsThis evaluation was prepared by a team led by Ravinder Kumar, providing senior evaluation expertise,and the subject matter expertise provided by Rattan Yadav.The two-person team worked under the overall direction of Allison Grove Smith, Director, CGIAR AdvisoryServices Secretariat. These reviews are being led by the CAS Secretariat Senior Evaluation ManagerSvetlana Negroustoueva. Ravi Ram, Senior Evaluation Consultant to CAS Secretariat, managed the firstcohort of three CRP2020 Reviews – A4NH, GLDC and WHEAT. Angela Giménez Barrera provided technicalevaluation expertise for the review process. Paolo Sarfatti served as senior advisor to the evaluativereviews project 2020. Gaia Gullotta and Max Runzel provided analytical support. Rodomiro Ortiz servedas a peer reviewer of the preliminary findings and final report.CAS/Evaluation and the review team gratefully acknowledge RICE CRP Team and the CGIAR SystemOrganization for their support: Julien Colomer and Hector Tobon.CAS DisclaimerBy design, the CGIAR Results Dashboard was a key source of data for the 2020 CRP Reviews. During thepilot phase of the CRP Reviews, issues with interoperability and resulting data quality between themanagement information systems (CLARISA and the Dashboard) and extracts from CRP systems (MARLOand MEL) were discovered. For harmonization, CAS engaged with the MARLO team and the CRP MEL focalpoints to conduct data cleaning and pre-analysis for CRP review teams. This exercise revealed thelimitations of CGIAR’s reporting/repository systems for evaluation purposes; these limitations weremostly due to changing reporting requirements and discrepancies in whether CRPs adopted MARLO orMEL systems. Moreover, in the case of peer-reviewed journal articles, the protocol used by the CRPreview teams to identify relevant publications differed from the guidance applied by CRPs (the CRP reviewteams’ bibliometric analysis used only publications indexed by International Scientific Indexing [ISI],available through Web of Science). Therefore, CAS acknowledges discrepancies between the CGIARResults Dashboard, and the data provided to the Review teams for their analysis, which should not beseen as a factor having influenced the analysis by the CRP review teams.i

ContentsExecutive Summary . 11 Background to the CRP 2020 Review . 41.1 Purpose and Target Audience of the Review . 41.2 Overview of the CRP and Its Context in Research for Development . 41.3 Scope of the Review and Review Questions . 51.4 Approach, Methods, and Limitations . 61.5 Management and Quality Assurance . 62 Findings . 72.1 Quality of Science . 72.1.1 Quality of Research Inputs . 72.1.2 Quality of Process (including Partnerships) . 72.1.3 Quality of Outputs. 82.1.3.1 Quantum of Research Outputs and Collaborations . 82.1.3.2 Quality of Researchers and Research Outputs . 92.2 Effectiveness . 92.2.1 Achievement of Planned Outputs and Outcomes. 92.2.2 Demonstrated Importance of Outcomes and Contribution of Outcomes to Broader Goals .132.2.3 CRP Management and Governance .162.2.4 Progress along ToC (CRP and Flagships) .182.3 Future Orientation .222.4 Cross-Cutting Issues .232.4.1 Capacity Development .232.4.2 Climate Change .232.4.3 Gender .242.4.4 Youth .243 Conclusions and Recommendations . 253.1 Quality of Science .253.1.1 Quality of Research Inputs .253.1.2 Quality of Process (including Partnerships) .253.1.3 Quality of Outputs.253.2 Effectiveness .253.2.1 Achievement of Planned Outputs and Outcomes. 253.2.2 Demonstrated Importance of Outcomes .253.2.3 CRP Management and Governance .263.2.4 Progress along ToCs (CRP and Flagships) .263.3 Future Orientation .273.4 Cross-Cutting Issues .273.5 RICE CRP-Level Recommendations .273.6 CGIAR System-Level Recommendations .284 Lessons Learned . 295 Annexes. 29ii

Find the Annexes and Brief here:CRP Reviews 2020: RICE CAS CGIAR Advisory Servicesiii

List of TablesTable 1 Gender analysis of RICE leadership at three CGIAR partner Centers . 7Table 2 Volume and share of peer-reviewed RICE papers in open-access and non-open-access journals . 8Table 3 Analysis of outcomes (as reported in the OICRs) contributing to broader goals .15Table 4 Progress on the CRP theory of change .19List of FiguresFigure 1: Proportion of RICE budget shared by RICE partners . 7Figure 2: Performance on 89 RICE milestones by flagship, 2017–19 .10Figure 3: Percentage of milestones contributing to a sub-IDO . 10Figure 4: Actual RICE expenditures compared with budgets in POWBs, 2017–19 .11Figure 5: Number of RICE innovations (n 224) by innovation stage, 2017–19.12Figure 6: Representation of countries among 42 OICRs reported in the RICE annual reports, 2017–19 .13Figure 7: Regions where RICE has informed or influenced policies (n 25), 2017–19.14Figure 8: Sources of RICE CRP funds, 2017-2019.17Figure 9: Assured future funding estimates, within and beyond CRP phase II (million US ) .22iv

Table of PVSQTLRCTRICESDGSLOSPIASRFSTRASASTRVAgriculture for Health and Nutrition CRPAfricaRice Centreadvanced research instituteAssociation of South East Asian Nationsalternate wetting and dryingBill & Melinda Gates FoundationCoalition for African Rice DevelopmentCGIAR Advisory Services SecretariatClimate Change, Agriculture and Food Security CRPCentro Internacional de Agricultura TropicalCentre de Coopération Internationale en Recherche Agronomique pour le Développementcluster of activities (under a flagship project within the CRP)Community of Practice on Crop ModelingClosing Rice Yield GapsCereal Systems Initiative for South AsiaCGIAR Research ProgramConsortium for Unfavorable Rice Environmentdirect-seeded riceEuropean CommissionEastern and South AfricaFondo Latinoamericano para Arroz de Riegoflagship programGrain quality-enhancer, Energy efficient and durable MaterialGrain Legumes and Dryland Cereals CRPGlobal Rice Science PartnershipIntermediate Development OutcomeIndependent Evaluation Arrangement of the CGIARimpact factorInternational Institute of Tropical AgricultureInstitut de recherche pour le développementIrrigated Rice Research ConsortiumInternational Rice Research InstituteIndependent Steering CommitteeIndependent Science and Partnership CouncilJapan International Research Centre for Agriculture Scienceskey informant interviewmonitoring and evaluationmarker-assisted breedingManaging Agricultural Research for Learning and OutcomesMulti-Environment Trialnational agricultural research systemNew Rice for Africanongovernmental organizationOutcome and Impact Case Report (part of annual reporting system of CGIAR)Philippines Rice Research Instituteplan of work and budget (annual planning exercise at CRP)Program Planning and Management Teampublic-private partnershipparticipatory varietal trialquantitative trait locusrandomized control trialRice Agri-Food System CRPSustainable Development GoalSystem Level OutcomeStanding Panel on Impact Assessment of CGIARStrategy and Results FrameworkStress Tolerant Rice for poor farmers in Africa and South AsiaStress-Tolerant Rice Varietyv

TNAUToCTORW1/W2W3viTamil Nadu Agriculture University, Indiatheory of changeterms of referenceWindow 1 or Window 2 fundingWindow 3 funding

CGIAR Research Program 2020 Reviews: Rice Agri-Food Systems (RICE)Executive SummaryBackground and Context on CRPRice farming is associated with several and deep structural challenges, such as diminishing availability ofresources (land, water, labor, and energy), climate change, and inequality. The CGIAR Research Program(CRP) RICE aims to address such challenges. RICE is led by the International Rice Research Institute(IRRI) and five other organizations with international mandates. RICE is organized under five flagshipprograms (FPs): FP1–Accelerating Impact and Equity, FP2–Upgrading Rice Value Chains, FP3–SustainableFarming Systems, FP4–Global Rice Array, and FP5–New Rice Varieties. RICE research is concentrated infive mega-rice growing environments: mega-deltas and coastal zones, irrigated systems, rainfedlowlands, uplands, and inland valleys.Purpose, Scope, and Questions of the CRP 2020 ReviewThe purpose of this review is to assess the extent to which the RICE CRP delivered quality of science anddemonstrating effectiveness in relation to its own theory of change during 2017 to 2019. The reviewquestions, which were set by the CGIAR CAS, are as follows: Quality of science: To what extent does the CRP deliver quality of science, based on its workfrom 2017 through 2019? Effectiveness: What outputs and outcomes have been achieved, and what is the importance ofthose identified results? Future orientation: To what extent is the CRP positioned to be effective in the future, seen fromthe perspectives of scientists and of the end users of agricultural research (such as policymakers,practitioners, or market actors)?Approach, Methodology, and LimitationsThe methodology employs mixed methods. Qualitative and quantitative data were collected andcombined in a process of triangulation to answer the three main review questions and all sub-questions.Key methods included analysis of program documentation and interviews with key RICE stakeholders.Forty-two interviews (including six women respondents) were conducted with RICE staff, partners, anddonors. The review relied on evidence mainly from the CGIAR Dashboard and CRP datasets as externalassessment data were limited. This is a rapid review, missing opportunities for in-depth data validation,field studies, and wider coverage.Key Findings and ConclusionsRQ 1 – Quality of ScienceThe RICE CRP has access to sufficient quality, diversity, and quantity of skills, infrastructure/facilities,and collaborators (more than 400) across the globe, to deliver its objectives. Several RICE researchershave h-indices of 25 or more, indicating their international standing and high-quality contributions.RICE’s leadership is strongly appreciated by researchers. A gender imbalance (highly skewed towardmen) was evident among the RICE management staff (10 men:1 woman) and among senior researchleaders (33 men:3 women). This imbalance was less evident among technical support staff (393men:274 women).The RICE partnership with CGIAR Centers, non-CGIAR institutions, and national agricultural researchsystem (NARSs) is working well, as is evident from the co-authored publications, germplasm, and otheroutputs/innovations. Research planning and development of projects (in consultation with stakeholders)appears function smoothly. Flagship programs (FPs) are interlinked and deliver joined-up science andtechnologies following appropriate ethics and internal review mechanisms. Owing to staff losses, FP4 hassuffered setbacks, especially on managing phenotyping activities; in the future this work can be bettermanaged by merging it with FP5. RICE’s efforts toward capacity development were found to beimpressive, in terms of both number (training more than 500 scholars every year) and the quality and1

CGIAR Research Program 2020 Reviews: Rice Agri-Food Systems (RICE)range of training opportunities offered (internships, on-the-job training, visiting scholars, BS, MS, PhD,and postdocs). However, NARSs still do not have the capacity to do such science on their own.RICE generates a notable number of peer-reviewed journal articles (about 250 publications annually).According to this reviewer, 75% of these articles are of high quality and published in appropriate journals(in the fields of plant sciences, crop science, agronomy, soil, and engineering). Outputs reviewed showedhigh relevance to users, novelty, rigor, and originality. Some FPs or partners are delivering a betterquality and quantity of research—for example, on average, 68% of RICE publications were generated byIRRI (FP1, FP2, FP5), 15% by AfricaRice (FP3), 7% by the Centre de Coopération Internationale enRecherche Agronomique pour le Développement (CIRAD) (FP4), 5% by the Institut de recherche pour ledéveloppement (IRD) (FP4), 4% by the Centro Internacional de Agricultura Tropical (CIAT) (FP4), and0.62% by the Japan International Research Centre for Agriculture Sciences (JIRCAS) (FP3). A pattern ofassociation of ‘publication in a high-quality journal’ and ‘winning bilateral funding’ was seen for lead andother partners of the CRP. Interestingly, scientific outputs matched exactly with the funding allocated tothem. About 30% of RICE papers published are not open access and not freely available to users.RQ 2 – EffectivenessThe RICE CRP completed 79% of its planned milestones on time. This finding demonstrates effectiveplanning and execution in the CRP. All flagships except FP4 have achieved most of their milestones in atimely manner. Insufficient finances and staff turnover are highlighted as reasons for noncompletion ordelays in achieving milestones. Overall, RICE received 12% less funding during 2017 to 2019 thanplanned in its plans of work and budget (PoWBs) and 17% less than planned in its 2016 proposal. Facingbudgetary uncertainty, all flagship budgets saw reductions. In the PoWBs for 2017–19 FP2 was allocateda small proportion (6%) of the total budget (US 230 million) and actually received 4% of totalexpenditure. Similarly, FP3 was allocated 28% of the total budget in the 2016 proposal (which wasrevised to 18% in the PoWBs) but received 20% of total expenditures. Lower planned allocations andlower actual expenditure than planned have compromised scaling-out activities in FP2 and FP3,potentially limiting the RICE’s achievements. Despite these budget cuts and uncertainties, the RICE CRPhas demonstrated stellar performance in discovery and delivery of science in rice agri-food systems.While a few innovations have achieved wide-scale application (such as NERICA, WITA9, sub1, satellitebased rice monitoring, smart valleys), adoption levels for several RICE innovations are still weak. Thisresult indicates that the seed system work requires greater attention than what has been possible.It is estimated that RICE impacted more than 2 million farmers during 2017–19 (the legacy impact couldbe around 10.4 million farmers over the Global Rice Science Partnership [GRiSP] period). This is about20% of the target of 11 million affected that RICE wanted to achieve by 2022. Many of the technologiesand innovations contained in Outcome Impact Case Reports (OICRs) are in the pilot or scaling phases.Several barriers to adoption and impact have been experienced, such as the limited capacity of the publicsector and limited engagement with the private sector, especially in Africa. Further, affordability is a bigissue for rice farmers regarding adopting several RICE innovations. These barriers would need to beovercome for RICE to improve its impact.RICE leadership and continuity at the helm are widely acknowledged by several scientists to be the CRP’smain driver of success. However, there is a perception that RICE lead center (IRRI) could have donemore to enable and recognize good work, foster efficient internal structures (rather than duplications),and facilitate smooth execution of work. The role of the Independent Steering Committee (ISC) wasparticularly appreciated for providing regular recommendations to flagship leaders.RICE realized the need for cross-CGIAR partnerships with A4NH, CCAFS, and PIM in several areas ofcollaborative research on nutrition, climate change and policy uptake, respectively. While partnerships(cross-CGIAR) worked in some instances, several RICE scientists consider this a missed opportunity.RICE’s theories of change (at the CRP and FP levels) are logically structured around four levels of impactpathways: discovery research, product, adoption, and outcome. RICE has achieved significant progresson TOCs at the CRP and FP levels, though more would need to be done in terms of scaling up RICEinnovations. However, the RICE CRP has not used ToCs for program reflection and reporting. In mostcases, impact evaluations of RICE work on different aspects (such as material, managementtechnologies, tools, policy influence) are led by FP1, with participation and advice from externalstakeholders. This peer review and evaluation system raises the prospect of (perceived or real) conflict ofinterest.2

CGIAR Research Program 2020 Reviews: Rice Agri-Food Systems (RICE)RICE has developed hundreds of quantitative trait loci (QTLs) or genes, models, products, andtechnologies that have good potential for achieving climate adaptation and resilience and contribute toclimate change mitigation as well. The RICE CRP has demonstrated its gender inclusiveness across allflagships, though the limited resources available in FP2 and FP3 are likely to have contributed to loweringexpectations around gender mainstreaming work. Youth programming in RICE is still nascent.RQ 3 – Future OrientationRICE, with IRRI, AfricaRice, and other partners, is well situated to progress strongly into the future interms of its future funding pipeline and its reputation for credibility. A major risk RICE will face in thefuture is shifting donor priorities away from cereal crops to larger integrated development research andinnovations to address grand challenges facing humanity (climate change; malnutrition; Covid-19, otherpandemics, and pandemic-induced poverty and hunger; sustainable intensification). Addressing thesechallenges would continue to require RICE research and upscaling of RICE innovations.RecommendationsFor the RICE CRP1. RICE should continue to strengthen its work on seed systems, with a dedicated team and partnershipnetwork. Investments in this area should be at least doubled in the future.2. To the extent feasible within the remainder of the CRP timeframe, all Centers (including non-CGIARcenters) should mobilize funds and capacities to the CRP and close the gaps between differences inresearch outputs (publications) and bilateral funding delivered by Centers part of the CRP.3. RICE should analyze mechanisms for its innovations to be taken to the scaling pathways. Severalstrategies for doing so are recommended in the detailed recommendations.4. RICE programming can better frame its sustainable intensification work. To the degree feasible in theremainder of the CRP timeline, RICE should increase its research on sustainable production tradeoffs.5. An in-depth, independent review of private sector engagement in RICE for scaling up and adoptioncan be carried out to guide engagement models for achieving wider adoption and deeper impact.6. ToCs at the CRP and flagship levels are, by their very nature, generic. While these are helpful,project-specific ToCs could be more granular and useful for tracking progress and identifying barriersalong the ToC.7. RICE should engage external team leaders in significant impact areas being assessed. This canprovide more accountability and given CGIAR donors confidence about impacts being achieved.8. In its 2020 annual reporting, RICE should consider capturing an extended summary of achievementsand gaps related to the ToCs, at both the CRP and FP levels.9. Improving the quality of science at the CRP would mainly require three interventions: (1) a genderbalanced ratio in staffing, (2) more open-access publications, and (3) a merger of FP4 and FP5, whichwould likely increase synergy and efficiency in operation.For the CGIAR System1. Greater CRP effectiveness is possible with better integration and a common framework of operationacross CRPs and Centers. One CGIAR should promote an approach of less entrenchment and morecollaboration, especially among CGIAR centers and between developing-country NARSs and CGIARCenters. Further One CGIAR should infuse private sector thinking.2. The System Organization (SO) should clearly lay out the functions of the Independent SteeringCommittee, which are aligned with Center-level structures.3. CAS should develop a project evaluation system within CGIAR. In the One CGIAR system, anindependent entity can take on this role of commissioning project evaluations of significant bilateralprojects. These evaluations should be utilization focused.4. CGIAR System Organization would need to review Managing Agricultural Research for Learning andOutcomes (MARLO) for its complexity and usefulness among the Centers and programs that haveadopted it.5. CAS (Evaluation Function) should complement the work of the Standing Panel on Impact Assessmentof CGIAR (SPIA) by supporting CRPs in structuring impact evaluation systems led by externalexperts, especially for significant impact areas.3

CGIAR Research Program 2020 Reviews: Rice Agri-Food Systems (RICE)1 Background to the CRP 2020 Review1.1 Purpose and Target Audience of the ReviewThe purpose of the RICE CRP review is to “assess the extent to which the RICE research program isdelivering quality of science and demonstrating effectiveness in relation to its own theories of change.”Key objectives are: To fulfill CGIAR’s obligations around accountability regarding the use of public funds and donorsupport for international agricultural research To assess the effectiveness and evolution of research conducted through the CRP in 2017–21 To provide an opportunity for programs under review to generate insights about their researchcontexts and programs of work, including lessons for future CGIAR research modalities.The study is accountability focused, but where lessons are identified these will be noted. Primary reviewusers will be the CGIAR System Council, with additional potential for the RICE program management todraw on the insights and lessons. Additionally, the lessons may inform the One CGIAR transition in 2022.The findings, conclusions, and recommendations aim to inform the CRP as it refines its 2021 plan of workand budget for the remaining program year.1.2 Overview of the CRP and Its Context in Research forDevelopmentRice is the world’s most important staple food and will continue to be so in the coming decades. A staplefood for some 4 billion people worldwide, rice provides 27% of calories in low- and middle-incomecountries. Global demand for rice is predicted to continue to grow in the time to come. With expectedpopulation growth and income growth global demand for rice will increase from 479 million tons of milledrice in 2014 to 536–551 million tons in 2030. Rice farming is associated with poverty. About 900 millionof the world’s poor depend on rice as producers or consumers, and of these, some 400 million poor andundernourished people are engaged in growing rice, mostly on landholdings of less than 2 hectares. Ricefarming is also associated with several and deep structural challenges, such as the following: Environmental quality is declining worldwide, with diminishing availability of resources (land, water,labor, and energy).Climate change is exacerbating the situation through the effects of higher temperatures, morefrequent droughts, and flooding, as well as sea-level rise, which threatens rice production in megadeltas.Women still face many barriers and inequality in access to and control over resources such as land,information, and inputs.Apart from increasing production and productivity, there is a growing demand that rice cultivation bedone in a more environmentally friendly manner, combining genetic traits and cultural methods in such away that poor farmers growing the crop have improved livelihoods and rice consumers gain better healthand nutrition.The CGIAR Research Program (CRP) RICE aims to address such challenges. As captured in the revisedCRP proposal (RICE, 2016), RICE aims to facilitate transition of smallholder rice farmers to modernbusiness entrepreneurship by exploiting opportunities offered by market diversification and theemergence of stronger consumer demand for high-quality and nutritious rice products. At the same time,it is helping poor farmers in hinterlands and less-endowed environments cope with extreme stresses andthe effects of climate change and other shocks. In doing so, RICE is at the cutting edge of science,mobilizing modern technological breakthroughs such as those offered by biotechnology, information andcommunication technology (ICT), and Big Data platforms. The revised proposal says that the RICE willharness 600 research and development partners from both the public and private sectors to delivermeasurable impacts on the overall goals of the CGIAR: reduced poverty, improved food and nutritionsecurity, and improved natural resources and ecosystem services. With appropriate technological,institutional, and policy support, rice farming, processing, and marketing could offer equal opportunitiesof employment for women and men and help to empower women, thus accelerating attainment of foodsecurity and poverty alleviation.4

CGIAR Research Program 2020 Reviews: Rice Agri-Food Systems (RICE)The RICE CRP is implemented by six organizations with international mandates and a large portfolio onrice: three members from the CGIAR—the International Rice Research Institute (IRRI, the lea

RICE is led by the International Rice Research Institute (IRRI) and five other organizations with international mandates. RICE is organized under five flagship programs (FPs): FP1-Accelerating Impact and Equity, FP2-Upgrading Rice Value Chains, FP3-Sustainable Farming Systems, FP4-Global Rice Array, and FP5-New Rice Varieties.