This fellowship has been a special and rewarding experience— meeting to meet and talk with scholars who are all working on projects to make academia better, more equitable, more diverse, more fulfilling … just wow! For me, the lessons have been both positive and negative—positive in terms of finding community, celebrating successes, and knowing that the work I’m engaged with isn’t unique. It gives me hope in collective power to coordinate and enact positive change. The negative parts have been in terms of seeing so many of us try, and not always succeed in our goals—to talk about the very real barriers that keep systemic power structures in place in academia, to hear about individuals committed to a system that advantages some while disadvantaging others and their research. It helps to hear others go through those struggles, but also makes change at the individual level feel like a hugely uphill battle at times.
With that, I’ll describe a bit about my own progress this past year and a half—the challenges, the successes, and the lessons learned, in the hopes that it can help reassure or inspire others to act.
I started my project as a HuMetricsHSS Fellow with the goal of integrating alternative impact measures into research evaluation at American University, a university in Washington, DC which leans heavily toward applied social sciences, especially in politics and policy. We have been steadily improving our research profile, but most departments still rely heavily on citation metrics for journal publications. In 2020, a university-level task force recommended the inclusion of DEI into tenure guidelines, and I saw an opportunity to incorporate more equitable evaluation measures. I created a resource guide, presented at a mini faculty retreat based on the guidelines, then reached out to several departments … and quickly found myself at a dead end, with only a couple of schools or departments reaching out to engage in further conversation.
So, how successful was I in accomplishing my original goal? I have no idea!! The guideline revision process continues, with an assessment likely coming after all schools and departments have completed their process. Within my unit, we updated library faculty tenure guidelines to broaden forms of tenurable scholarship from only peer-reviewed articles to include peer-reviewed works of any kind, including conference presentations, which are often impactful in librarianship, and to include lots of flexibility in measuring scholarly, practitioner, and other forms of non-scholarly impact using a range of measures appropriate to both the research output and audience(s). The guidelines leave the choice of metrics up to the individual, but allow for impact demonstration in line with the ACRL Framework for Impactful Scholarship and Metrics (that, full disclosure, I helped create!).
However, some other unanticipated opportunities happened to bubble up around the same time: I was able to advocate for the purchase of two ‘sister’ products that measure university-wide research impact as well as alternative metrics, and gave several presentations on the uses for these products to campus administrators, using some nifty graphics to show how we can not only benchmark based on traditional metrics like output, but can tell amazing stories about how our research is being used, shared, and cited in a non-scholarly context. Simultaneously, our reaccreditation self study was starting, and I ended up co-chairing the subcommittee responsible for writing about the university’s scholarship. As a result, I’ve been able to include some of these metrics, and combine them with interesting stories that showcase our university’s research in ways we’ve never gathered before. By doing so, we are helping to reinforce that alternate research impact measures and goals are well aligned with our university’s dedication to incorporation of DEI into everything we do, including our scholarship and its impact, but it also reinforces the American University value of community, by giving researchers more options to tell about the impact of community-based research in terms of community engagement and impact rather than scholarly impact, as well as the AU values of excellence and impact by defining AU’s excellence through diverse impact measures.
While our self-study is still ongoing, this has been enormously successful in helping administrators see what a more diverse set of research impact measures can look like at the university level and giving us new tools and vocabulary to both measure and talk about our research in ways that make sense for our university.
I think the next phase will be in educating our faculty about how to align their own research measures and stories with this university-level analysis, but I fully expect that as our university looks to our next strategic plan we’ll start to see an increased emphasis on not only research, community, and the scholar-teacher ideal (which are all components of our current strategic plan), but the ways in which diverse and alternative metrics can make connections between all of those ideas by demonstrating things like community and real-world impact that result from our scholarly efforts.
So, here is a short list of some of the lessons I’ve learned along this journey.
- Advocacy for a more holistic evaluation method is a process. Some efforts may not pan out, but over time you can build support across the institution.
- Sometimes, it’s luck. Sometimes, it’s being in the room and having an opportunity. Mostly, it’s communicating frequently and finding administrative advocates. For me, the opportunity to discuss and demonstrate the potential of altmetrics to track non-scholarly impact for deans and other administrators was key, as was the support of our Vice Provost for Research and Innovation, who have all helped continue the conversation around these issues and how they are integrated on campus.
- Metrics and tools help drive vision of what’s possible. For me, the purchase of new products was an opportunity to tangibly show the potential of altmetrics to track non-scholarly impact at the university level rather than talking about it as an abstract or author-centered concept.
- Alignment is key: our university’s strategic plan, mission and vision all help answer the questions of who we want to be and who we want to impact. A focus on DEI and on non-scholarly audiences such as policymakers helped ensure that altmetrics were helping show how we are fulfilling our mission, particularly for researchers whose impact are not well showcased by citation metrics alone.
- Not one size fits all: some areas of our university will greatly benefit from things like policy citations or news mentions, while others are well-served by more traditional metrics. Just like the broader incorporation of DEI into scholarship, diverse metrics help make more research understood and valued based on its own merits. They give researchers more freedom and possibility, but they are intended to enhance rather than replace.
- Most importantly, advocacy for more diverse metrics is not solo work, and collective action makes big change possible. At my university, this has meant growing support for a more unified vision of the role diverse metrics can play in describing our research and the university’s impact, but the more people and institutions that take on this work, the more we can change the existing attitudes and perceptions and shift the dominant paradigm. Some of the resistance I’ve faced comes from researchers worried that our path will take us out of alignment with other universities, while at other universities these same concerns are similarly expressed. At the end of the day, there is more support for change than is easily seen, and many opportunities to enact change, both big and small.
I am so grateful to HuMetrics for bringing the fellows together, and for helping inspire and envision a more values-oriented approach to scholarship. And for anyone who is interested in learning more about progress at American University, please don’t hesitate to get in touch!
Rachel Borchardt, Scholarly Communications Librarian, American University