The “scholarly blog”: PressForward’s many paths to success, and how to measure them

 By Kimberly Coulter

It has been five months since Ant Spider Bee relaunched its site with the WordPress web aggregation and publication plugin PressForward. Thanks to a generous grant from the Alfred P. Sloan Foundation, we have been able to pilot this tool as a partner of the Roy Rosenzweig Center for History and New Media. PressForward helps us review a collection of relevant RSS feeds, nominate the posts we deem of greatest interest to our readers, and repost excerpts. By doing the aggregating work of the “ant,” it helps us be the “bee”…allowing more resources for digestion and cross-pollination.

“Digestion” and “cross-pollination” are two main functions of scholarly blogs, electronic publications that may build community and curate, contextualize, or comment on issues in a field of study. Discussing how a tool like PressForward is relevant for scholarly communication, and in particular blogs that engage an academic community with relevant news along with original “gray literature” (meaning not peer-reviewed), was a focus of the PressForward Institute for Scholarly Communication at George Mason University from 13-14 August. We enjoyed meeting RRCHNM staff and learned how to better manage our feeds and share them with others. It was great to meet the other three original pilot partners [PLoS, MicroBEnet, CitizenScienceToday (Zooniverse)] and other participants (including Woods Hole Marine Biological Laboratory and the Association of College and Research Libraries’ dh+lib).

In these five months since adding PressForward we’ve seen our traffic increase 138%. Still, we ask: what does a successful scholarly blog look like for us, what role does PressForward play, and how can we measure this success?

Models

As PressForward project co-director Stephanie Westcott pointed out, attracting researchers’ online engagement means different things to different communities. PressForward aims to do this by semi-automating information-gathering, and in making it easy for community members (“editors-at-large”) to participate in information curation. PressForward’s “test blog” is Digital Humanities Now,  to which visitors turn to for job postings, calls for proposals, news, and conversations relevant to digital humanities. The editors-at-large (particularly graduate students) view the work as career-relevant service. RRCHNM’s Lisa Rhody reported that the “editor’s choice” featured post has even become a quality stamp; a selected author sometimes includes the republication on his or her cv.

“Editor’s Choice” feature in PressForward’s test publication, Digital Humanities Now. (c) Digital Humanities Now. https://digitalhumanitiesnow.org/

While an aggregated feed may be presented as a stand-alone offering, as in Digital Humanities Now or Jon Christensen and Ursula Heise’s attractive blog Environmental Humanities Now, which generally feature “editor’s choice” posts selected from the aggregated feed, we also discussed other models. These alternative models generally fell into three categories: using the feed to 1) augment an original content publication with related postings that together may frame problems and offer resources; 2) emphasize community curation; and 3) contextualize a large body of scholarly literature.

At the workshop, pilot partners and others described diverse goals for using the aggregated feed. Using a dynamic feed to augment a regular original publication is what we’re trying to do with Ant Spider Bee: ultimately creating a collection of short essays reflecting on the role of the digital in environmental humanities. We hope our feed not only serves its scholarly community but also draw it to the site. Ideally feed content should infect its original content as well, making it more current and engaged. Community building is a tacit goal of most scholarly blogs, but as a focus, it has potential for great impact. By casting a broad and inclusive net, presenting cool new projects from the community as news, and analyzing them–ideally in visual ways–it’s possible to use PressForward to explicitly cultivate and strengthen a scholarly community. Popular also are aggregated opportunities and job postings, as are emphasized in Digital Humanities Now and dh+lib. Rosalind Reid, executive director of the Council for the Advancement of Science Writing and project director of New Horizons in Science, spoke about plans for a curated Compendium of Best Science Writing. In cases with large scholarly corpuses with a high number of users, it’s possible to use aggregated feeds to pull in materials that contextualize peer-reviewed papers. The Public Library of Science (PLoS) has ambitious plans to create a blog for each thematic journal collection, to which PressForward feeds would aggregate related news and discussions.

Scholarly blog editors need expert understanding of the field, but also understanding of its experts’ information-seeking and communication behaviors. In general their role serves a “meta” research function: contextualization, curation, and programming. In contrast to peer review’s stamp of experts’ quality assurance, content curation may mean an expert’s assemblage of objects to narrate or interpret a story. While scholarly blogs generally do not perform rigorous peer review, they may influence a field’s direction by defining its problems, programs, and roles (see Michel Callon on “Domestication of the Scallops”). Aggregation is a convenience, made possible by technology, that enables curators to cast a wider information net and filter it more finely. Curation itself cannot be automated, and it can serve a powerful function.

In addition to taking a big picture and influencing the framing of issues, scholarly bloggers are often invested in a project’s extra-disciplinary, educational, or public value (open access, etc.). Yet there is usually a mismatch between this work and the traditional metrics. How can we measure the impact of such work?

Metrics

Different stakeholders measure the value of scholarly work with different metrics. It may be important that the work is heavily cited, gains funding, is adopted by policymakers, reaches the public, picked up by mainstream journalism, informs public debate, builds community, provides visibility for the field, or even sells books, magazines, or museum tickets. Thus it is essential to be clear about the goal before choosing what to measure. For example, Google Analytics offers tools more oriented to measuring stickiness (keeping users on a page) and sales, while, as Rhody points out, a successful feed aggregator should have a high “bounce rate,” showing the site’s success in directing users to relevant external content.

“Alternative” metrics measure high quality, high impact digital and other nontraditional outputs that may not be captured by traditional metrics. Tools like surveys might help capture “soft” outcomes like resulting collaborations, incorporation of recommendations into policy, public engagement, journalistic coverage, media appearances, speaking engagements, invitations to editorial roles, invitations to contribute to an edited product, growth of visibility of a community, visibility for the field, educating the public. In a metrics panel discussion on the workshop’s second day, Robin Champeaux of Oregon Health and Sciences University introduced some alternative metrics resources but also pointed out that those committed to work with nontraditional value may need to do their own curation work to document its success with alternative metrics; a great deal of contextual knowledge is necessary to quantify alternative outcomes.  and quantifying alternative outcomes requires a lot of contextual knowledge.

For measuring impact in the humanities, Web-sourced data can only grow in importance, as Stacy Konkiel writes in her post “We’re overdue on altmetrics for the digital humanities.” The Altmetric Explorer is one tool that aggregates webbased attention data, including an API to big policy indexes to track ideas’ adoption into policy. It lets users monitor, search and measure conversations about publications, such as blogs, news, Wikipedia, or Twitter, offering an alternative to traditional journal-to-journal referencing via citations. It does, however, charge 3000GBP annual license for a team of five. Another tool that helps scientists share the diverse impacts of their work is Impactstory, which measures things like saves and code forks, and how frequently work is saved, viewed, and discussed. Users can contribute data from tools like Figshare, which makes research outputs sharable and citable; Publons, which hosts and aggregates open peer reviews; or LinkedIn’s presentation sharing service SlideShare.

“The Altmetric Explorer demonstrates that articles from the Journal of American History are consistently referred to in blogs (yellow), on Twitter (light blue), in mainstream news sources (red) and on Wikipedia pages (dark grey).” Fran Davies, Not just science articles – tracking other disciplines and other research outputs” August 6, 2015. https://www.altmetric.com/blog/not-just-science-articles/

Alternative metrics can help to differentiate a candidate in a competitive field; they may lead to, or indicate strong networks that lead to success in hard metrics like funding. Citation takes long times to build. The web can be complementary, and provide immediate impact.

While it’s important to document successes, and to lobby universities to recognize alternative media scholarship for tenure and promotion, that is not the only avenue for thinking about “what counts.” As tenure-track positions—even tenure itself—become less and less common, early career scholars are well-advised to broaden their avenues. As one workshop participant concluded, even if it’s not about tenure and promotion, we will always be serving different masters. Building a community, connections, visibility, and exchange of ideas keep us adaptable and focused on our missions.

We came away from the workshop with a long list of ideas for Ant Spider Bee. To name a few: we’ll be pruning our RSS feeds and welcome your recommendations of further feeds to follow; deploying a PressForward outbound OPML to make it easy for our users to subscribe to our same feeds; investigating alternative metrics tools; considering an ISSN for our featured posts; and starting in October, I’ll be teaching Digital Environmental Humanities at the University of Munich and asking my students to contribute to Ant Spider Bee‘s PressForward feed curation as “editors-at-large.” We hope Ant Spider Bee’s feeds are relevant for its audience, and that our posts influence the thoughtful framing and use of digital tools in our fields. We would be delighted if they made a small contribution to the advancement of digital and nontraditional scholarly projects and professional recognition for their authors. Whether or not we succeed in measuring this, we find the effort worthwhile.

Kimberly Coulter directs the Environment & Society Portal in Munich. The Portal is the digital publication platform and archive of the Rachel Carson Center for Environment and Society, a nonprofit joint initiative of the LMU (University of Munich) and the Deutsches Museum.