Published in Brian Martin, C. M. Ann Baker, Clyde Manwell and Cedric Pugh (editors), Intellectual Suppression: Australian Case Histories, Analysis and Responses (Sydney: Angus & Robertson, 1986), pp. 50-58
Prejudice in granting research grants - in pdf
Go to
in his well-documented article (C. Manwell, "Peer Review: A Case History from the Australian Research Grants Committee", Search, vol. 10, no. 3, March 1979, pp. 81-6.) Clyde Manwell not only questions the validity of anonymous peer review in general, but describes in detail his own experience with the Australian Research Grants Committee (ARGC) which in 1971 abruptly terminated his grant for a zoological project, and refused to renew it on two subsequent applications. Professor Manwell offers three hypotheses for his failure: (1) Natural selection: poor research; (2) Random drift: the uncertainties of the system; (3) Unnatural selection: deliberate prejudice. He allows readers to draw their own conclusions, presenting in fact compelling reasons for the third hypothesis. His story is so frightening in its implications for the future of higher education in this country that it behoves all academics who have had similar experiences to publicise them in the hope of reform.
Most academics are reluctant to admit such failures or difficulties, which might be used against them in subsequent applications for grants or promotion. It is, as Professor Manwell points out, difficult to present an objective statement of experiences highly damaging to morale or self-respect. I believe, nevertheless, that the documents in my own case are worth citing. History at first sight appears more subjective than the sciences, and more likely to be influenced by nonacademic factors. But, interestingly enough, most criticism of peer review seems to come from scientists. There is a clear need for more humanities-based statements on peer review.
As an historian I have specialised on Irish overseas influence in the nineteenth and twentieth centuries. This topic is inextricably linked with state aid for private schools, a particularly contentious contemporary problem. In the late 1960s, while I was completing my research on the Irish in New Zealand, state aid achieved a breakthrough in Australasia as state and national governments vied in increasing subsidies to private schools. Opponents of state aid, like the subsequent Defence of Government Schools, were generally portrayed as cranks or bigoted extremists. Unfortunately - perhaps because of my lack of church affiliation - it seemed to me that the anti-state aid arguments, generally accepted in the nineteenth century, were not inevitably ridiculous. Meanwhile, the increasing sectarian violence in Northern Ireland led to widespread debate there on the dangers of denominational education. Some of the tactics adopted by private school supporters appeared to me highly dubious. I felt obliged to publicise some of the old controversies before the decisions on state aid in Australia were finalised.
My situation, though tenured as a lecturer in Tasmania, was thus a little precarious. Nevertheless, before the end of the 1960s, I established a satisfactory reputation as a researcher. I published several articles in Ireland, New Zealand and Australia, plus a short book on the politics of state aid in Tasmania. My Ph.D. thesis on Irish and educational controversies in New Zealand was, however, savagely attacked by an anonymous referee (later found to be a member of a church education society) when submitted to the Melbourne University Press. The anonymous referee complained that I had ignored vast quantities of archival material, but much of this I demonstrated to be non-existent. Another publisher was found. In 1971, after returning from leave in Ireland, I was promoted to senior lecturer.
By this time several members of my department had been awarded ARGC grants. As my publication record was superior to many, I anticipated no difficulty in obtaining finance for a study of Irish influence in the old Commonwealth countries during the Irish revolutionary period, 1916-22. The work had clear contemporary relevance in that it sought to analyse the methods of overseas pressure groups, simultaneously justifying Irish guerrilla action and state aid for local private schools. I had already written on Irish and educational issues in Australia and New Zealand; an article on the Irish in a Canadian province, where I had spent some of my 1970 leave, was about to appear; comparative imperial studies had long been my interest. Moreover, when lecturing in New Zealand in the 1960s I had obtained, after interview, a small grant for local travel from the New Zealand ARGC; in 1970, the University of Manitoba gave me a research fellowship worth about A$5600, with no teaching required.
My Australian ARGC application in 1971 simply cited two or three articles directly connected with my proposed comparative study; I did not mention other writings with marginal relevance. I was very surprised in October when my application was rejected without interview. Other colleagues who had published little or nothing were sometimes embarrassed by the continuation of grants after their research efforts had petered out. I assumed that there had been some flaw in my mode of application; it was after all a period when funds were flowing relatively freely. Perhaps I should have cited all publications? Maybe I had failed to itemise my financial requirements with sufficient precision?
In 1974 I was granted six months' leave by the University of Tasmania. My original project still interested me. I hoped by concentrated research to accumulate sufficient material, when combined with my previous work, to write a short comparative volume on Irish support groups in the British Dominions during the Irish Revolution. With a grant from the University of Tasmania and cheap accommodation in Ireland, provided by a relative, I could afford to spend my leave in that country. My comparative study, however, required some time in Ottawa, and possibly a short visit to South Africa to track down some elusive material. I calculated that with $1000 from the ARGC - precious little by the requirements of scientific research - the book might be written.
This time I took no chances. My application cited everything already published, plus all my forthcoming work. I had then to my account one book and twelve reasonably substantial articles. Three other articles, including a 22,000-word chapter on Tasmania for D. J. Murphy's Labour in Politics[12], were on the way. Three full-length books[1, 2, 3] were in the press in Ireland, New Zealand and Tasmania. All these publications touched in some way on my comparative project. I also nominated two personal referees, both former supervisors of my research and now sympathetic advisers on my current projects. One was Emeritus Professor W. P. Morrell, author of a number of classic studies of British colonial policy and comparative Commonwealth development. The other was Professor F. S. L. Lyons, soon to be elected to Ireland's most prestigious academic position, the provostship of Trinity College, Dublin, and without doubt the most eminent Irish historian of his time. It seemed appropriate that a comparative study of the Irish in the Commonwealth should be supported by such a distinguished Irish historian and so outstanding an analyst of the Commonwealth.
My precautions were in vain. Without even an interview, the ARGC secretary, Mr K. E. Creech, again regretted on 26 October 1973 that my request for support had not been successful. The cyclostyled letter informed me as usual that the committee had "a longstanding policy of not discussing the reasons for the failure of an application" to preserve anonymity and because there was insufficient money to finance all worthy projects. But, it suggested, don't be downhearted, read the Chairman's Advice to Australian Research Grants Committee Applicants and try again.
I was, however, already acquainted with this document, especially the section suggesting that if a disappointed applicant wrote nicely to the Chairman, asking what was wrong with his work, the Chairman would endeavour to explain. I accordingly wrote nicely to the Chairman, asking what was wrong with my work. A few weeks later I received a roughly typed letter, signed for Secretary Creech. The letter invited me to "appreciate that these assessments must remain confidential to the Committee". As consolation, however, he enclosed a copy of ... the Chairman's Advice to Australian Research Grants Committee Applicants!
My reply of 29 November was not quite so nice. I requested Mr Creech to refer to the Chairman's Advice, p. 12, and read: "If the unsuccessful applicant writes to me in a spirit of bewilderment rather than belligerence, I try to be as helpful as I can". I referred to the specific examples of the sort of help the Chairman was prepared to give. If policy had changed, I argued, the current Advice should be withdrawn immediately and future applicants informed that they would be at the mercy of a committee "which issues no explanations of any sort". In the present context Creech's reply and enclosure seemed "not only unsatisfactory but positively insulting". I again requested "some explanation".
My 29 November retort achieved results. A letter from Creech informed me tartly that there had been no policy change and that the Advice still reflected the committee's policy. He then proceeded to do what he had previously declared impossible - summarise the conclusions of "independent assessors":
Each of the assessors recognised that the project you proposed to complete presented the possibility for an interesting and original study. However, they were in agreement that a comparative approach coupled with considerable insight and organising ability would be required to make the project worthwhile. The comments made by the assessors indicated that they were not convinced that you had appreciated the intrinsic demands of the project and consequently they were quite unimpressed with the research proposals that you outlined in your application.
What can be made of this? First, it is interesting to note that despite the enormous difference between Professor Manwell's projected study of genetic variation of proteins in Australian native and imported animals and my comparative study of Irish influence overseas, the ARGC response to both had common elements. There was an initial reluctance to supply any information - in my case a Kafka-like bureaucratic stalemate based on patent self-contradiction - followed by a report whose damning conclusion was offset by initial admissions as to the value of the project. As in Manwell's case, there was the implicit double standard: despite the failure of numerous ARGC projects to bear fruit, we were both condemned on the assumption that our good projects might not be successfully completed because of our lack of ability as researchers.
Let us look more closely at the report on my comparative Irish influence project. The "interesting and original study", the assessors argue, would require "considerable insight and organising ability". One would surely expect this of any academic, no matter how lowly. To deny their existence in the case of a candidate with a number of publications would surely require some extremely hard evidence - harder, it is hoped, than the evidence of the Melbourne University Press referee for my New Zealand book who cited non-existent archives. On the former occasion I saw the original report and was able to make what was conceded by the deputy director of the Melbourne University Press (18 February 1969) to be an effective reply, necessitating another opinion (ultimately not granted). I had of course no guarantee that the MUP referee had not been employed by the ARGC. Presumably, the "independent assessors" considered my publications in detail. How did they receive my highly critical article on state aid in the Australian Humanist? Were they invariably better qualified to demonstrate the inadequacy of my writing than those who had reviewed it or accepted it for publication? Is it just to deny a grant on the evidence of published work when many grants are awarded to those who have published nothing?
The only stated justification for denying me finance was a belief that I had failed to appreciate "the intrinsic demands of the project". Yet the assessors admitted that the project was original in that no one had attempted anything like that before. How then could they, who had not worked on the project, know in advance "the intrinsic demands" better than myself? Is there indeed any way of convincing an anonymous and unaccountable assessor that anyone knows the "intrinsic demands" of their subject when previous publications and the opinion of the most respected scholars can be dismissed as worthless or irrelevant, without any possibility of a reply? What sort of "academic" procedure is this? In history, at any rate, the anonymous citation of a bogus archive or two can apparently sink any applicant without trace. If ever an interview were justified, it was here: a little cross-questioning would soon have discovered my understanding of the "intrinsic demands" of my chosen field,
So that was that. My experience in 1974 convinced me that I had been black-listed by a rival and could expect no ARGC grant at any time in my subsequent career, unless, of course, I moved to some entirely different field of study. This was, no doubt, the precise objective of the "independent assessors". Professors Lyons and Morrell, who repudiated the ARGC explanation as completely opposed to their recommendations, advised me to battle on without the grant. In late 1974 I took study leave in Ireland with sufficient money to research there, but not enough to visit Canada or South Africa. Before leaving, however, I attempted to force the issue into the open by a direct challenge to the academic whom I believed responsible for my double rejection by the ARGC. Though supplied with a prepublication copy of my article[7] criticising his methodology and published work in detail, the pundit turned down a request from the editor to write a reply for the next issue.
My six months' leave in Ireland was not quite as productive as I had hoped. Lacking the necessary source material from South Africa and Canada, instead of a book I could publish only a 6000-word article on Irish support groups in the Dominions during the Irish Revolution. I did, however, prepare papers on other aspects of the Irish question, published in England[15, 17], Bangladesh[16], the United States[19], and Australia.[14, 18, 20] More significant was the Dublin Historical Association's publication of my 48-page pamphlet[13], summarising my earlier research on Sinn Fein founder, Arthur Griffith, and incorporating the results of Irish research of 1974. All in all, a lengthy pamphlet and seven or eight papers seem a reasonable return for six months' study leave, despite my lack of ARGC funding. How many ARGC grantees do better? But who cares about the actual writings of an academic who has been proved anonymously to be inappreciative of "the intrinsic demands of the project"?
Between 1974 and 1975, I also published four books whose research had been completed in previous years. Reviews appeared in Ireland, England, Canada, Australia and New Zealand. Some were very good, many were satisfactory, but a few were disappointing. My Irish Issues in New Zealand Politics[3], nearly aborted in its early stages, was satisfactorily received by the Times Literary Supplement and excellently reviewed in a local journal by the head of a New Zealand university education department. In acclaiming the book as "a masterly and lucid treatment", Professor Rollo Arnold praised the "researches which have ranged through Ireland, North America and Australia, as well as New Zealand, to give depth and perspective" - a view hard to reconcile with the ARGC assessors' complaint that I was unaware of the comparative implications of my subject. Even the New Zealand Marist Messenger, despite some reservations, concluded that Irish Issues should be studied in all Catholic seminaries. But the pundit who had refused my direct challenge gave it an atrocious but quite unspecific review in an Australian periodical whose editor obligingly refused to publish my reply. With mates like this, who needs anonymity? Book reviews, demonstrating the wide range of academic opinion on most academic issues, are one of the strongest arguments against the current system of anonymous "peer" review which enables one or two "faceless men" to destroy potential rivals without any risk to themselves.
Early in 1976 I was promoted to Reader for the "excellence" of my research in Irish history. For once the academic game of Russian roulette had worked in my favour: instead of blasting my reputation to tatters, the anonymous assessors clicked their chambers benignly. "Excellent" in 1976, I was inappreciative of "the intrinsic demands" of my chosen field in 1973, judgement in both cases being based on precisely the same work. As for money, my request for a single grant of $1000 was dismissed with ridicule in 1973 by the ARGC; three years later the taxpayer was called on to pay an annual increment of twice that sum for the rest of my career. The original $1000 might have produced a book to enlighten the general public; the continuing increment simply raised my personal standard of living.
Like Professor Manwell, I leave it to the reader to decide whether my disgrace at the hands of the ARGC was the product of natural selection (my incompetence), random drift (my bad luck), or unnatural selection (calculated dishonesty). The decision must be based on the question: how independent were the assessors? My own view comes through strongly in the foregoing pages, but my experience may have made me neurotic or paranoid. My departmental juniors, almost invariably successful in their applications, seek my unprofitable advice on the completion of ARGC forms: simultaneous with my second failure in 1973, a Ph.D. candidate, then under my supervision, obtained a most satisfactory ARGC boost. Should I apply again? Until some reforms are made in the system, my time, financed by the taxpayer, could clearly be spent more profitably.
What sort of reforms? First, the abolition of anonymity. Assessors' reports must be sent to applicants, and an appeal system, headed by an academic ombudsman, can easily be established.[21, 22] Second, actual research needs to be monitored and the results published after five years. If these suggestions are considered too time-consuming and expensive, why not draw the names of applicants out of a hat? Though far from ideal, it would be an improvement on the present system.
Books and articles published in 18 months after refusal of second ARGC application
BOOKS
1. The Tasmanian Gallows: A Study of Capital Punishment (Hobart: Cat & Fiddle Press, 1974), pp. xv, 119.
2. Arthur Griffith and Non-Violent Sinn Fein (Dublin: Anvil Books, 1974), pp. xxi & 253.
3. Irish Issues in New Zealand Politics, 1868-1922 (Dunedin: University of Otago Press, 1974), pp. x, 248.
(Reviews mentioned: (i) Times Literary Supplement, 13 February 1976; (ii) ANZHES Journal, vol. 5, no. 2, Spring 1976; (iii) Marist Messenger, NZ, December 1975; (iv) Journal of Religious History, vol. 9, no. 2, December 1976, pp. 216-17.)
4. A Guide to the State Aid Tangle in Tasmania (Hobart: Cat & Fiddle Press, 1975), pp. viii, 118.
ARTICLES
5. "New Zealand Liberal Legislation and Manitoba Labour, 1894-1916", W. P. Morrell: A Tribute - Essays in modern and early modern history presented to William Parker Morrell, Professor Emeritus, University of Otago. Edited by P. S. O'Connor and G. A. Wood (Dunedin: University of Otago Press, 1973), pp. 169-83 and 289-94.
6. "Irish Catholics and the Manitoba School Crisis, 1885-1921", Eire-Ireland, St Paul (USA), Autumn 1973, pp. 29-64.
7. "Patrick O'Farrell and Irish Secular Nationalism - a Reconsideration", Tasm. Hist. Res. Ass. Papers and Proceedings, December 1973, pp. 221-38.
8. "General Histories of Ireland published in the 1970s", Historical News, Christchurch, May 1974, pp. 9-11.
9. "Tasmania and the Irish Revolution, 1916-22", Tasm. Hist. Res. Ass. Papers and Proceedings, June 1974, pp. 69-88.
10. "C. S. Parnell, Cecil Rhodes, Edmund Dwyer-Grey and Imperial Federation", Tasm. Hist. Res. Ass. Papers and Proceedings, September 1974, pp. 125-32.
11. "Arthur Griffith and the Decline of Secular Nationalism", Church and State, Dublin, Autumn 1974, pp. 6, 36-40.
12. "Tasmania", in Labour in Politics: The State Labor Parties in Australia 1880-1920, D. J. Murphy (ed.), University of Queensland Press, 1975. pp. 387-444, 464-6.
Books, articles and papers published as a direct result of the six months' leave in Ireland; refused ARGC funding on grounds of lack of appreciation of "intrinsic demands" of my field:
BOOK
13. Arthur Griffith (Dublin Historical Association Pamphlet, 1976), pp. 48.
ARTICLES
14. "The Self-Determination for Ireland Leagues and the Irish Race Convention in Paris, 1921-22", THRA Papers and Proceedings, vol. 24, no. 3, September 1977, pp. 88-104.
15. "The Advocacy of Passive Resistance in Ireland, 1916-1922", Anglo-Irish Studies, III, Cambridge, 1977, pp. 35-55.
16. "India in Irish Revolutionary Propaganda, 1905-1922 ", Journal of the Asiatic Society of Bangladesh, vol. XXII, no. 1, April 1977, pp. 66-89. (Also given as Conference paper, New Zealand Historical Association, Dunedin, August 1975.)
17. "Arthur Griffith: Architect of Modern Ireland: Part II: 1916-1922", History Today, London, vol. 29, no. 4, April 1979, pp. 248-56.
18. "Bertrand Russell and Ireland", Australian Journal of History and Politics, vol. 26, no. 2, August 1980, pp. 279-83.
19. "Ulster Protestants and the Sinn Fein Press, 1914-1922", Eire-Ireland, St Paul (USA), Winter, 1980, pp. 279-83.
20. "Catholic Education and Irish Nationalism: O'Connell to Community Schools", ANZHES Journal, Adelaide, vol. 10, no. 1, Autumn 1981, pp. 1-12.
Articles advocating a higher moral code for academics:
21. "Anonymity: The Cancer of Academia", Education Research and Perspectives, Perth, vol. 6, no. 2, December 1979, pp. 3-11.
22. "A Hippocratic Oath for Academics?", Vestes, vol. 24, no. 2, 1981, pp. 155-9.
Dr Davis has applied twice for support from the ARGS and has been unsuccessful, but it is not my intent to engage in personal debate as to the reasons for the denial of funding. I am unable to agree, however, on a number of inferences that are offered about the scheme in relation to his case. It is untrue, for example, that grants are not given "on the evidence of published work when many are awarded to those who have published nothing" (p. 52). ARGS grants are very competitive and publication records play an important part in guiding the committee to decide whether or not the research that is planned is likely to be successful and communicated in major publication outlets. Second, I have seen no evidence that people who publish little are embarrassed by the continuation of their grants when their research efforts have petered out (p. 51.) The scheme monitors its grants through progress reports, interviews and peer assessment and evidence of this kind of situation is very likely to surface. And if it does, a grant would not be awarded.
There are a number of specific issues raised by Dr Davis's case that are important and worthy of comment. These relate to the strengths and weaknesses of the peer review system, confidentiality of reporting, the function of nominated versus committee- selected assessors, procedures available for requesting feedback, the intent of the scheme's interview mechanism, and suggested reforms.
Peer review, confidential reporting and nominated assessors The peer review mechanism is not perfect and much has been written about chance and consensus in peer review. It is possible, for instance, for experts in a field to disagree among themselves about the same proposal and questions can certainly be raised about the reliability and validity of the assessment. Peer assessment, however, does provide critical evaluation by those familiar with the issues and the science that is being used. Such comment can "drive" research, isolate the "cutting edge" of a research field, and highlight the best proposals.
Dr Davis is too extreme when he talks about faceless men destroying rivals without any risk to themselves (p. 53). Destructive comment of this kind is likely to be detected by the committee which acts as impartially as it can to interpret the evidence and assess its relevance to the proposal that is being considered. The issue of whether that comment should be confidential or not is another matter. Confidential assessment is requested in order to guarantee that frank and honest comment will be given. Obviously there are assessors who, for whatever reason, would make quite different comment when offering it anonymously than when offering it when identified. It is my experience, for example, that nominated assessors can readily convey excessive enthusiasm to support an applicant.
Requesting feedback The account Dr Davis gives of requesting feedback sounds unfortunate and I can only comment on what I know now to be the case. Unsuccessful applicants can write to the committee and request feedback and the committee when giving that feedback attempts to give as detailed a response as possible. Rather than summarise what has been said by the assessors, the committee tries to cite directly from assessors' reports.
The interview mechanism It is impossible for me to assess in the present instance whether "if ever an interview were justified, it was here" (p. 53) but it is important to reinforce the value of the interview in clarifying doubts in the committee's mind about the adequacy of the research. An interview serves many purposes. It would be conducted if clarification was needed from applicants but not if the committee thought it would add nothing to their case as stated by themselves and judged by the assessors.
Suggested reforms Reforms, of course, should be considered in the system, and Dr Davis suggests two that require comment. First, the suggestion is made that anonymity be abolished, but Dr Davis's paper begs the question of whether identified comment will be entirely frank. Second, it is suggested that actual research needs to be monitored and results published. The committee, however, does monitor its grants closely. It requests regular and detailed progress reports from all grantees, who also have to submit a final report and state the benefits of their research. The committee considers it important that the people it funds are accountable for whatever support they have received.
Grant writing is a frustrating exercise, especially when the application is unsuccessful, and it is of course likely that some proposals funded by the scheme have been judged incorrectly. But the scheme operates as fairly and as objectively as it can to use peer assessment to help it reach a decision about the worth of the research. "Calculated dishonesty" (p. 54) is a judgement that is just not appropriate.
Professor Sheehan adds little or nothing to the debate. He produces no evidence to back the implication that all is for the best in the best of all possible academic worlds. His picture of a splendid body of men and women operating justly in conditions of perfect secrecy is charming, if naive. Professor Sheehan expects us to believe that a biased assessor would be immediately exposed by his peers. Unfortunately, life isn't that simple.
Two of Professor Sheehan's points are worth considering. First, he denies, albeit with a non sequitur assertion about future, not past, results, that applicants without publications obtain grants. Evidence of grants to unprolific researchers is readily available. An interesting case was the successful applicant in 1972 who had not published for five years and has produced nothing since. Some monitoring! No one wishes to point the bone at colleagues, but the government should require a detailed public statement, after a suitable interval, juxtaposing each individual's grant and his or her subsequent publications.
Second, Professor Sheehan believes that I beg the question on anonymity. Could anyone be frank if anonymity were removed? Of course they could. Most professional people, even academics, are forced to make unpopular public decisions. What anonymity does is to provide excellent cover for academic factions to manoeuvre undetected. With organisations like the National Civic Council boasting of their academic operations and potential assessors asserting openly that "faith has, of its nature, the special status of offering extra and clearer knowledge, a situation not applying to unbelief"[1] the federal government must act to ensure that justice is seen to be done.
Professor Sheehan carefully skirts my suggestion of an academic ombudsman. Yet an ombudsman of sufficient credibility could easily be appointed to guarantee that public money would always go to active researchers and not to the nominees of aggressive interest groups.
1. P. J. O'Farrell, "Historians and Religious Convictions", Historical Studies, Melbourne, vol. 17, 1976-77, p. 289. Such views, while stimulating in open debate amongst equals, are dangerous in secret committees.