Is the NTRO the NITRO that will Blow Up the Humanities?
Presentation to Deans of Arts, Humanities and Social Sciences (DASSH) Conference, Customs House, University of Queensland, 18-19 September 2014
Terry Flew, Professor of Media and Communication, Creative Industries Faculty, Queensland University of Technology
I would like to that the Board of the Australasian Deans of Arts, Social Sciences and Humanities (DASSH), the Board President, Professor John Gertov, and the Conference Convenor, Professor Fred D’Agostino, for inviting me here to speak this morning. When this topic was first proposed, I had anticipated being in an after-lunch session, rather than opening the event itself. I would hope that my contribution is neither to incendiary to establish a conflictual tone for these two days, nor too soporific in its use of data to leave you wishing you had arrived at Customs House a bit later in the morning, and dealt with those emails in your hotel room for a bit longer.
What both Professor Ross Woodrow and myself have been asked to consider by Professor D’Agostino – Fred – was to consider the question of non-traditional research outputs (NTROs). This was from the perspective of valuing the impact of such work, as measured through such exercises in developing research performance metrics in the previous and forthcoming Excellence in Research for Australia (ERA) rounds. It also reflected the more pragmatic requirement to develop benchmarks for performance in relation to creative outputs that are comparable to those in the more traditional humanities and social science disciplines, for reasons such as annual personal promotion rounds.
In this talk, I will not be discussing the question of quality in relation to NTROs, as this will be the focus of Ross’s talk. I also should add that I am not a creative practitioner myself. As a Professor of Media and Communications, I am situated within what Toby Miller terms (in his 2013 book Blow Up the Humanities) Humanities 2, or the applied fields of communications and media, that have tended to co-exist with media and creative production programs of various kinds. As someone who writes about the creative industries, I am situated within what Miller terms Humanities 3, albeit with claims to knowledge in critical cultural studies, or what Miller terms Humanities 4.
Over the last three years, I have also been actively engaged with the Humanities and Creative Arts (HCA) panels of the Australian Research Council. This has been both as a member of the College of Experts reviewing Discovery and DECRA applications, as well as the Indigenous Discovery program. I have also been on the Research Evaluations Committee for the 2012 ERA round. Over the last two years, I have also participated in the Linkage Infrastructure, Equipment and Facilities (LIEF) scheme, as the somewhat lonely representative of the HCA disciplines, in a program dominated by the physical and human sciences, and by the information technology and engineering disciplines. It is on the latter basis that I can illustrate a point about measuring and evaluating NTRO outputs.
In assessing the 2014 LIEF round, I was working with a colleague from the Biological Sciences as joint carriage embers for an application from HCA that had a collection of leading arts and design academics and practitioners. My co-assessor noted that the CIs seemed top produce relatively few outputs relative to funding, giving as an example Australia Council grants of $20,000 that had only led to one major research output. My point was that this would be exactly what would be expected from such funding: a grant is given for a particular project (performance, exhibition etc.), and is subsequently acquitted. My point was that the number of research outputs relative to funding can be expected to be lower, as each “output” has a large time and infrastructure requirement behind it that is more akin to setting up a lab in the sciences than it is to producing an equivalent journal output.
Non-traditional research outputs have been a feature of both the 2010 and 2012 ERA rounds. At first glance, there appears to have been a decline in the number of NTROs submitted in the 2012 ERA round, which is clearly at odds with the general trends with research outputs, particularly books and refereed journal articles. But the decline in the total number of NTROs submitted of 32% disguises the other major change, which is an increase in the number of portfolios of creative work submitted of 56%, and an increase in the number of works submitted within portfolios of 75%. I would suggest that this is evidence of a turn to quality within the NTRO category, as leading creative researchers increasingly submit their work as part of a portfolio of work, rather than as a discrete set of original works.
Within the NTRO category, the largest number of works – both in absolute terms and as percentages – are in Field of Research (FoR) 19 Studies in the Creative Arts and Writing. There are also a modest number submitted in FoR 12 Built Environment and Design.
In the 2012 round, NTROs consisted of: (1) original creative works; (2) live performance of creative works; (3) recorded or rendered creative works; and (4) curated or produced substantial public exhibitions and events. For the 2015, there will be an additional category of Research Report, that can be expected to have a particularly strong impact in the social science disciplines.
Turning to the question of those FoRs where NTROs are most significant, we can contrast FoR 19 with FoR 12 and parts of FoR 20 to give an illustration of the dynamics at play:
Percentage of research outputs in selected 4-digit FoR codes (%)
Books Book chapters Journal articles Conference papers NTROs
1201 Architecture 2 17 20 34 27
1203 Design Practice & Management 2 9 20 43 26
1901 Art History and Theory 5 30 30 10 25
1902 Film, Television & Digital Media 3 13 20 10 54
1903 Journalism & Professional Writing 3 6 22 9 60
1904 Performing Arts & Creative Writing 1 7 12 6 74
1905 Visual Arts & Crafts 1 4 4 3 88
2001 Communication & Media Studies 5 29 42 21 3
2002 Cultural Studies 6 34 46 10 3
There are several possible points that can be taken from this data. First, we can see considerable diversity among the FoR 12 and 19 codes where NTROs are high. FoR 1901 is closest in shape to a ‘conventional’ humanities discipline such as Communication & Media Studies or Cultural Studies, but even here 25% of submitted outputs were NTROs. The share of NTROs in total submitted outputs across FoR 19 had a range from 25% on 1901 to 88% in 1905 Visual Arts and Crafts.
Second, it is also the case that even in these codes where NTROs are particularly important, it is also the case that more conventional research outputs are as well. If we take FoR 1902 Film, Television & Digital Media as an example, 46% of its submitted outputs took conventional forms. This would come as no surprise to those working in such fields, where film and television studies, or game studies, are well-established research areas with a significant history in the humanities. But there are interesting balances among those institutions that submit as a Unit of Evaluation within such a code, between those that emphasise their ‘theoretical’ elements and those which emphasise their ‘practice’ elements.
A third point is that, because of the complexities involved in organising various forms of creative practice, as well as the trend towards portfolios among the leading researchers, there may not be the same volume of outputs as one will find in more traditional HASS disciplines. This will mean that the quality measures will become more important. Without revealing any trade secrets from the 2012 ERA, it would be fair to say that there is a degree of ad hocery in the benchmarking of quality in creative work in some of the practice-based fields at present.
There have already been ways in which the rise of NTROs has challenged more traditional research performance metrics. One concerns the relationship between the ERA process and the HERDC collection. Whereas the four HERDC categories set criteria for the inclusion or exclusion of work, NTROs are largely outside of the HERDC collection, so the question of how they constitute research is largely left to the submitting institutions. At the same time, they can clearly be defined as research according to the OECD’s definition of research as:
Any creative systematic activity undertaken in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this knowledge to devise new applications (OECD Glossary of Statistical Terms, http://stats.oecd.org/glossary/, 2014).
How the relationship between these categories will be sorted out into the future remains a policy challenge. It is one that will intensify, not only as NTROs related to artistic and creative practice grow in size and significance, but also as new forms of research activity are incorporated into the ERA, such as the preparation of research reports for government, industry, the professions, and others.
In Australia, as in the UK, the creative practice fields have taken on research evaluation exercises with some gusto. In a presentation to QUT staff in 2012, Jonathan Pitches, Professor of Theatre and Performance at the Unviersity of Leeds, explained that a credo that had informed their approach to the then-RAE (now REF) was that:
There is nothing special about practice-led research, no need for special pleading, nor for ring-fenced monies to fund the work.
This avoids the problem that Miller identified with the U.S. humanities, where arguments about the special nature of the humanities disciplines had led the fields into a funding cul de sac. It adopts a position of a confidence from which to speak to research councils, evaluators and collaborators, as well as those who doubt the “research” dimensions of creative practice. The corollary is that evidence of the rigour of the work in creative practice fields has to be of the highest quality for this position not to backfire on us, when in direct comparison with more established academic disciplines.
Another issue is where creative practice will sit within the portfolio of activities of different universities, and how the ERA process may shape that. For the marginal cost of submitting NTRO outputs is significantly higher than that of books, journal articles etc. This is partly because the work of explaining how and why this output constitutes research needs to be undertaken by the institution – and anyone who has viewed such statement will know that they are highly variable – but is also reflective of the need to develop robust platforms that best present the work in a digital form to its assessors. Given that individual institutions have incentives to render their own work more compellingly to its assessors, this will force institutions to make decisions about how much they wish to invest in those FoR codes where NTROs are central to determining overall research productivity.
The question of investing in creative practice provides a good note on which to finish this presentation. As any Dean will know whose Faculty has strong creative practice programs, they are not cheap to offer. At the same time, they cannot be offered on the cheap, as the students drawn to them have high expectations.
Their growing attractiveness relates to a point that authors such as Gunther Kress were making in the mid 1990s, as the implications of the digital revolution were only beginning to become apparent. Kress argued that for the contemporary humanities student, critique would no longer be enough. It would be increasingly accompanied by a desire to make things, and to intervene directly in the dominant symbolic systems.
The fact that Miller’s binary between “old” humanities and cultural studies misses this emergent phenomenon is actually quite surprising. At the risk of pointing towards “Humanities 5”, it was a critique made of Blow Up the Humanities in the U.K. context, where it was noted that so-called “elite” programs were deeply infused with an awareness of how digital technologies were transforming their objects of study, and had already largely taken on the critiques associated with cultural studies, which have been around in some or other form since the 1980s.
Similar issues arise with developing the creative practice disciplines in the context of the ERA. My expectation would be that the “participation bar” to engaging effectively with discipline codes with a high NTRO component is likely to rise significantly, and that will present new challenges for those institutions that wish to establish themselves as leaders in these fields. But there will also be the question of the opportunity costs for the future in not investing in creative practice activities in the present.