World leaders have set a goal of gender equality by 2030, yet experts caution that it can’t happen if data gaps are not solved. One noted, “Data saves lives. It captures the attention of policy makers and focuses their efforts on the right issues.” That is, as advocates put it, we must “count women” if we are to garner large-scale support and resources for women and girls. To learn what this emphasis looks like in implementation, I asked international development practitioners how gender-related measurement is carried out at their project sites—and with what effects? What I heard resonates with the words of former USAID Administrator Andrew Natsios: focusing on measurement is important, yet it also “ignores a central principle of development theory… programs that are [the] most transformational are the least measurable.”

My findings, based on interviews with sixty development professionals, suggest that when it comes to programming meant to engage and benefit women and men equally, the current policies and practices of measurement unfold in unexpected ways.

Although sex-disaggregated data is meaningful for funders, reporting practices take project-level attention away from understanding women within the intersecting structural inequalities of their everyday lives. In this formulation, measurement practices enhance upward accountability to donors, while reducing downward accountability to the women the programs are meant to benefit.

Among the international indices developed to measure women’s empowerment, we find the UNDP Gender Inequality Index (GII) and the World Economic Forum’s Global Gender Gap Index (GGI). In 2016, the top three bilateral donors of Official Development Assistance were the United States – USAID ($33.6b), Germany – BMZ ($24.7b), and the United Kingdom – DFID ($18.0b). USAID, BMZ, and DFID each have gender policies with attention to sex-disaggregated data and finding “what works for women and girls.” Data2x, a technical and advocacy platform, was expressly formed to ensure the collection of sex-disaggregated data. Yet, while these projects may succeed in making women and girls “visible” through data, how do these measurement practices translate into development programming for those women and girls we can now “see”?

To understand the impacts of measurement practices, it is important to analytically separate two interpretations of the word “development.”

Commonly, development is understood as an ideal, an intention, a moral good—the “better world” on the horizon. Yet development is also a multibillion-dollar industry employing thousands. It is a sector of jobs, contracts, and organizations in which professionals and organizations build their reputations on their “impact” on improving livelihoods in developing countries. Conceptualizing development as two-sided in this way maintains excitement for the ideal of development, while also allowing exploration of what the industry does to the people, places, and communities where development projects operate.

Again, development as an ideal posits downward accountability to the world’s poor through efficacious development, whereas development as an industry tends toward upward accountability to donors (and, by extension, taxpayers) through demonstrated results. The latter dominates accountability discussions in development circles. In 2005, for instance, the Paris Declaration for Aid Effectiveness articulated an international standard for finding what does and does not work for aid through five pillars, including a commitment to measuring and achieving results and establishing mutual accountability. But how are results defined? And accountability to whom? Results typically sit at the project level and include baseline data, progress reports, and performance plans alongside cost-benefit analyses, business cases, impact assessments, and randomized control trials. Evidence that demonstrates a return on investment is considered key in demonstrating accountability and transparency for taxpayer expenditures. In this context, measurement is key in determining organizations’ labor and budget decisions.

Time and again, we hear: “What gets counted, gets done.” However, the focus on measuring development ignores the very messy world of international development.

My findings from researching the evaluation system of a bilateral agricultural initiative suggest that when donors mandate the “counting” of women, the pressure to deliver results undermines reaching women in meaningful and sustainable ways – the “ideal” side of development.

During my research, multiple Gender Advisers told me that gender-related work items are deprioritized because other input-based initiatives, such as latrine building or immunizations, are easier to measure. When faced with an impending quarterly or annual report to the donor, quantifiable measurements become highly valued despite the difficulty of clearly capturing gender dynamics. Gender Advisers also noted that projects are not often designed with reaching women in mind. For example, women may need trainings delivered closer to their homes and with childcare provided to decrease their opportunity cost. Further, Chiefs of Party (project directors) cite the “pre-investment” of working with marginalized populations. Women and girls disproportionately fail to qualify to participate in projects requiring a high school education or fairly sophisticated levels of literacy, numeracy, and strategic thinking to write business plans. As they face the pressure of delivering results to donor, Chiefs of Party reported needing more time to slow down and build the capacity of women and girls—often structurally marginalized through social, cultural, economic, and/or political factors in their own communities—to benefit from the programs.

When development ideals are refracted through the politics of evidence, it creates a high stakes context in which organizations must prove their impact. Transformational goals such as women’s empowerment are, then, likely to be deprioritized in the face of easier-to-measure goals and objectives.

The next time the development sector asks for more data and measurement to help achieve gender equality or women’s empowerment as an ideal, practitioners and researchers should pause. Development as an industry, driven by measurement practices, may negatively impact how women are included in projects. Of course garnering political support and resources at a policy level is needed, and making women visible is a worthy venture—there are data gaps. But garnering meaningful engagement of women during implementation is equally important. This research suggests we should count women and, rather than count women period. This means developing alternative accountability structures, in which metrics are only one part. Reinvigorating and reimagining the project cycle to make space for the structural and normative realities that make up women’s everyday lives would assist in meaningful project delivery and results reporting. Longer term horizons to allow time for gender social norm transformation, integrated projects that help women gain foundational skills, and the adoption of change theories that directly address differential programming for women (including an understanding of the diversities within and between, for instance, married women and women-headed households) are all potentially fruitful adaptations. This research challenges us to think creatively about how to ensure that results-based development works for women and girls.