Rethinking humane indicators of excellence in the humanities and social sciences

Tag: values

HuMetricsHSS Workshop: The Value of Values

HuMetricsHSS Workshop: The Value of Values

This 2.5-day intensive workshop at Michigan State University will bring approximately 25 humanities and social science (HSS) scholars and administrators at all levels and from all types of institutions into conversation with each other and with the project team. Working groups will focus on each of the proposed five values (Equity, Openness, Collegiality, Quality, Community) as they might relate to practices in academic HSS disciplines; the intention is not simply to reaffirm the values of the framework, but to interrogate, challenge, and revise them.

On Living Our Values While Under Stress (and Preparing for Workshop One)

On Living Our Values While Under Stress (and Preparing for Workshop One)

The real work of the HuMetricsHSS initiative begins in Michigan this week, when an insightful group of thinkers—faculty members of all ranks, teaching in any number of HSS disciplines at all kinds of institutions, along with administrators, graduate students, university publishers, and librarians—has agreed to come together to rip apart, interrogate, and rebuild that values framework, to come to a consensus on the values we share as a larger group.

If you’re wondering if the HumetricsHSS workshop is for you, the answer is yes!

If you’re wondering if the HumetricsHSS workshop is for you, the answer is yes!

This October, the HuMetricsHSS team is excited to bring together a diverse group of scholars, teachers, administrators, and students from a wide range of institutions for a topic that we believe will transform academia. Over the course of a two-day workshop, we’ll interrogate, brainstorm, break apart, add to, and thoroughly revise a proposed set of values to create the beginnings of a framework for assessing a more holistic, accurate interpretation of scholarly “excellence.”

The HuMetricsHSS initiative is based on the idea that humanists and social scientists should not content ourselves with existing, faulty evaluation metrics that are based on what’s easy to measure (citations, grant dollars awarded, Twitter mentions, etc.) rather than what’s important to measure (influence, quality, collaboration, etc). Instead, we believe that it’s important to think about what kind of values matter to those of us working in humanities and social science fields — our collective ethos, as it were — and to develop a flexible metrics framework that rewards work that embodies those values. See this post for an example of how this idea might apply to a typical scholarly object: the syllabus.

But we face a major challenge. We are all too aware that our team of six alone cannot give voice to the shared values of a huge and diverse community. That’s where you come in.

Thanks to a generous grant from the Andrew W. Mellon Foundation, we’re now able to ask students, deans, tenure-line faculty, contingent lecturers, alt-acs, Ph.D. candidates, department heads, and others working in HSS departments at any higher educational institution to apply to help us articulate the shared values of a more equitable and, frankly, excellent academy in East Lansing, Michigan, on October 5-7, 2017. We’ll be joined by workshop facilitators Dr. Cameron Neylon (Curtin University) and Dr. Stefanie Haustein (University of Ottawa).

Travel is covered, so if you care about bettering evaluation practices in the humanities and social sciences, please apply to join the workshop by midnight on August 6 by filling out this easy form.

Questions? Drop us a line at humetricshss@gmail.com!

Triangle and Beyond

When we first introduced the work of the HuMetrics group to the other TriangleSCI teams last October, we met with some resistance. Much like peer groups in the run-up to an election, we had become used to having our own ideas reflected and amplified internally. […]

The Syllabus as Scholarship

The Syllabus as Scholarship

“A critical component of our emerging #Humetrics conversation at Triangle SCI involves finding ways to expose, highlight, and recognize the important scholarship that goes into the all-too-hidden work of peer review, syllabus development, conference organizing, mentoring, etc. Our current metrics fail to capture what is […]

Nurturing Fulfilling Scholarly Lives

Nurturing Fulfilling Scholarly Lives

In Aristotle’s Nicomachean Ethics, there is a famous passage in which he reminds us that “to be happy takes a complete lifetime; for one swallow does not make a spring, nor does one fine day; and similarly one day or a brief period of happiness does not make a person blessed and happy” (Nic. Eth., 1098a16–20).

This passage came to frame our conversations around #HuMetrics at this week’s Triangle Scholarly Communications Institute, because it reminds us that a fulfilling life  —  what Aristotle calls, eudaimonia, happiness, that is, a life well lived  —  requires cultivated habits rooted in core values that, when intentionally practiced, shape the character of a good life.

In the end, what we value should be embodied in what we do, not once or twice, but regularly over the course of a lifetime.

In framing our conversation about #HuMetrics with this ancient conception of ethics, excellence, and character, we seek also to advance and reinforce the idea that a scholarly life can only be well lived in communities of practice with others.

For the #HuMetrics team, this year’s Triangle SCI experience was a swallow that signifies but does not yet fully manifest the coming spring. It opened for us a space for the flowering of a community of practice oriented toward the question of how we might more broadly cultivate communities of practice that embody the values of fulfilling scholarly lives.

Five Core Excellences of Enriching Scholarship

Working out loud together, we identified five core excellences of enriching scholarship:

Rebecca Kennison wrote about Equity.

Simone Sacchi wrote about Openness. His point was amplified further by Rebecca in her post about The Value of Openness.

I wrote about Collegiality.

Jason Rhody wrote about Quality.
Nicky Agate wrote about Community.
And Stacy Konkiel sought to tie things together by distinguishing between enriching and corrosive values.

In writing together in this way, we seek to embody the excellences for which we advocate.

The question that animates our work is this:

The Winter of Our Discontent

For too long, we humanists have been allergic to metrics. This allergy has prevented us from engaging in a serious and sustained conversation about what practices of scholarship we might want to cultivate and incentivize both through the activities we measure and those we celebrate.

As a result, a large and growing battery of metrics have been developed based on the practices of more scientifically oriented scholarship or simply on what it was possible to use our technologies to measure.

Current metrics of humanities scholarship have been shown to be too blunt to capture the multiple dimensions of scholarly output and impact (see Haustein and Larivière). In addition, the inappropriate nature of current indicators can incentivize perverse scholarly practices (see The Metric Tide, Wilsdon et al.).

A critical component of our emerging #HuMetrics conversation at Triangle SCI involves finding ways to expose, highlight, and recognize the important scholarship that goes into the all-too-hidden work of peer review, syllabus development, conference organizing, mentoring, etc. Our current metrics fail to capture what is most substantive about the rich life of scholarship we practice together in living academic communities.

In this context, our challenge and our responsibility is to articulate, incentivize, and reward practices that enrich our shared scholarly lives and expand our understanding of scholarship itself.

Without being naïve about how difficult it is to change culture, we hope to begin to reshape the conversation about metrics around the values of enriching scholarly practices and the communities in which they thrive.

Although our time together at the Triangle SCI was only one swallow that does not yet make a spring, the seeds planted there may begin to take root over the weeks and months to come, and the communities of scholarship that blossomed there just might be “made glorious by this sun” that shines when a broader public is invited to join the conversation.

The Excellences of Scholarship: Collegiality

The Excellences of Scholarship: Collegiality

We began the day with a walk. The morning was cool and fresh, and the grounds around the DuBoise House at the Rizzo Center offered the six of us on the #HuMetrics team at the TriangleSCI the peace and space we needed to think out […]

On Quality

On Quality

Today is the second full day of the Triangle Scholarly Communication Institute, where I am part of a team that’s focusing on HuMetrics: Building Humane Metrics for the Humanities. We have each agreed to quickly blog some thoughts as part of our process; warning: what […]

Community as a Humanistic Value

Community as a Humanistic Value

Today is the Day Two of the Triangle Scholarly Communication Institute, where I’m heading up a team that’s focusing on HuMetrics: Building Humane Metrics for the Humanities. Our team has focused a lot on the importance of working out loud, of process over product, and agreed that we would each take 20 minutes once or twice a day to blog, pomodoro style, about what we’ve been up to. In other words, don’t expect polished prose; this is pre-alpha humetrics-in-the-making.

We started off team time today by clustering the 20–30 values that we believe would enhance and enrich humanities scholarship under five headings: Equity, Openness, Collegiality, Quality, and Community. Taken together, these values could embody what the Greeks called arete, or excellence in practice; if we encompass them in our scholarly practice, they should have impact (see Stacy Konkiel’s post on influence vs. impact for what we understand by that).

I offered to write the post on community because it was a perceived lack thereof (in the humanities) that prompted me to leave academia. The infighting and backstabbing, the condescension and derision — these were not something I wanted to be a part of. And yet there’s so much potential for things to be better. That’s why we wanted to reverse engineer humanities metrics, to see if we could start from the things we value and then work out how to incentivize and measure practices that embody those values.

Under Community, we clustered engagement, network, holistic, attunement, and leadership. [Note: while I’m writing this, we’re continuing today’s discussion on Slack, and have decided, given conversations about posterity and future audiences that were part of this afternoon’s session, that preservation should also be included here.] It’s a diverse list, to be sure (I’m sure you’re scratching your head or stroking your beard or furrowing your brow just a little), but hopefully I can explain our thinking. That is, if I can remember our thinking — I’m beginning to believe there’s such a thing as too much brainstorming.

Paying attention to community in scholarly life means fostering, cultivating, and participating in relationships and networks to which one gives and from which one takes. Like collegiality, it’s about generosity and mentorship; it’s about knowing when to lead and when to listen. It encompasses attunement because it asks us to be intentional about the connections we make and the way we enact them. One might create a community in the classroom, or acknowledge the full roster of people that contributed to the making of a book (is there really such a thing as a monograph?), or orient a class around service learning in such a way that it benefitted the larger community, the one beyond the academy. One might participate in a community by sharing data or code or primary source materials, by adding certain materials to a syllabus, or by opening up one’s own creative process (rather than just the product) to critique and conversation. And being part of a community means thinking beyond the now, proactively considering the preservation of all elements of the scholarly record (from blog posts to conference papers to tweets and vines), thinking forward to the publics and communities that might find value or interest in our work ten, fifty, or one hundred years from today.

Follow team #HuMetrics as we wrestle with humanities metrics. We are Christopher Long, Rebecca Kennison, Stacy Konkiel, Simone Sacchi, Jason Rhody, and Nicky Agate, and we’ll be writing here all week.

On Openness

On Openness

Second day at #TriangleSCI with the #HuMetrics team. Today we focused on clustering our values into major value-categories (Equity, Openness, Collegiality, Quality, and Community) with the idea that excellence in scholarship is an expression and combination of these value-categories as they are embodied in scholarly […]

Equity as a Core Value

Equity as a Core Value

We’ve just completed Day 2 at #TriangleSCI, and more hard (but good) work is now behind the #HuMetrics team. Today we took our huge brainstorming list from yesterday and distilled the values we believe should underpin the development of “humane metrics.” We came up with […]

Influence vs. Impact: Which Are Humanists Really Trying to Achieve?

Influence vs. Impact: Which Are Humanists Really Trying to Achieve?

Apologies for the false dichotomy I’ve set up by my framing of this post in its title as “impact versus influence.” It’s a result of the quickblogging process, one that Christopher Long, Rebecca Kennison, Nicky Agate, Simone Sacchi, Jason Rhody, and I agreed upon as a means of digesting and sharing our daily work at the TriangleSCI meeting.

Yesterday, the #HuMetrics team spent the better part of our day articulating values, outputs, processes, and metrics that humanistic research results in. Our idea was that if we could “reverse engineer” metrics from values and practices, that we could come up with metrics that are more humane: they not only reflect and incentivize the practices that humanists value most, but also help humanists avoid the “impact trap” that many in STEM find themselves a part of.

Our team came up with a list of values that fall into several general areas: equity, openness, collegiality, quality, and community:

(It’s important to note that our list is by no means exhaustive, and for the most part it draws upon our own personal experience and is not as informed by the existing research in this area. We’re aware of that limitation  —  the list is mostly meant to be a starting point for thinking about metrics.)

As you can see, these core values are flanked on either side by two overarching desires: for research excellence and for research impact. It’s upon the latter point that I want to think aloud for a few minutes.

“Impact” is a term with very particular connotations, depending upon where you stand in the world.

From the STEM and social sciences perspective, it’s often related to measurable changes in the world that are attributable to research outcomes (nod to Cameron Neylon for that succinct definition). Much of the time, this results in an emphasis upon research commercialization, economic impacts, or public health impacts.

For the humanities, “impact” is also often tied to money: how many jobs the cultural sector produces, income related to cultural activities like the film industry or museum openings, and so on. But as the UK Arts and Humanities Research Council has pointed out, there’s a hierarchy of impact assumed in our current neoliberal environment, one that puts economic value above all other values  —  and that should not be the case, especially for the humanities. And as David Budtz Pedersen at the Humanomics Research Center at the University of Copenhagen has pointed out, “the humanities may find many pathways into society, some of which are deeply integrated in the functioning and affluence of modern liberal societies.”

We discussed the need to push back against the idea of “impact” as outcome oriented (especially as those outcomes relate to the economy), and to also reclaim the term “impact” to mean what humanists want it to mean  —  in all its messiness, and sometimes at odds with what’s demanded of researchers by the institutions, governments, private funders, and public that want to see an easy-to-digest statement of “return on investment” from the humanists whose work they support.

What remains to be seen  —  what we’ll tackle tomorrow  —  is whether it’s actually possible to find metrics to relate to less-tangible values, beyond economics: those that tell us whether humanities research is truly changing a discipline, affecting the way the public thinks, or having any other number of personal and societal impacts.

Perhaps a better way to think about what humanists wish to achieve is to use the term “influence” instead of impact?

Follow team #HuMetrics as we wrestle with humanities metrics. We are Christopher Long, Rebecca Kennison, Stacy Konkiel, Simone Sacchi, Jason Rhody, and Nicky Agate, and we’ll be writing here all week.

The Value of Openness

The Value of Openness

Day 1 of the Triangle Scholarly Communication Institute is underway, and our HuMetrics: Building Humane Metrics for the Humanities team (Nicky Agate, Simone Sacchi, Christopher Long, Stacy Konkiel, Jason Rhody, and me) is already hard at work. We began by putting aside (for the moment) […]

HuMetrics Values

HuMetrics Values

Today is the first day of the Triangle Scholarly Communication Institute, where I’m heading up a team that’s focusing on HuMetrics: Building Humane Metrics for the Humanities. Our team has focused a lot on the importance of working out loud, of process over product, and […]

Enriching vs. Corrosive Values in Academia

Enriching vs. Corrosive Values in Academia

As part of the TriangleSCI HuMetrics working group, I spent the better part of this afternoon brainstorming and debating academic values, products, processes, and metrics in an attempt to lay a foundation for this week’s attempt at articulating “humane metrics” for the humanities.

As our discussion wound down, it occurred to the team that we were working with an important assumption re: values: we had spent the day identifying only those “enriching” values that we wanted to encourage (collaboration, generosity, inclusivity, quality, etc.), rather than examining the current set of values we wished to discourage, those that in many ways are “corroding” academia (competition, bureaucracy, exclusivity, etc.).

That’s in large part due to what we’re here in Chapel Hill to do: define and promote a means of appropriately measuring and rewarding quality scholarship (if not excellence) in the humanities  —  a group of disciplines currently underserved by citation-based metrics.

Yet citation-based metrics are increasingly being requested in research evaluation scenarios, by tenure and review committees, granting agencies, university administrators who wish to benchmark their departments against those of other universities. And those metrics often unintentionally reinforce so-called “corrosive” values: as Haustein and Larivière point out, the over-reliance upon simplistic, citation-based metrics for evaluation has led to undesired practices in academia like “salami-slicing,” self-citation, and ghost authorship; in the humanities, Nederhof explains that a focus upon both citation-based metrics and a push toward publishing in “preferred” venues has led to changing publication practices that are out of step with the realities of the field.

As our work begins tomorrow, we’ll attempt to explore those values that “enrich” academia, hopefully allowing us to “reverse engineer” or map metrics that measure scholars’ ability to achieve values we wish to encourage. No small feat, for sure.

Follow team #HuMetrics as we wrestle with humanities metrics. We are Christopher Long, Rebecca Kennison, Stacy Konkiel, Simone Sacchi, Jason Rhody, and Nicky Agate, and we’ll be writing here all week.

From Metrics to Values

From Metrics to Values

Recognizing that metrics drive practices in the academy (and elsewhere), we on the HuMetrics: Building a Humane Metrics for the Humanities team at this year’s Triangle Scholarly Communications Institute decided to approach our work by thinking first about the values that inform enriching scholarship. This […]