This post is one in a series, beginning with an Introduction, on my experience as a PhD student in the School of Social Work (SSW) at IU. This series proceeds in approximately reverse chronological order. The preceding post in this series is at the very end of the story: it contains a manuscript that I wrote about the failing grade given to my qualifying paper.
That manuscript summarizes a memorandum that I submitted to the SSW’s PhD Committee via Dr. Margaret Adamek, chair of the PhD program, after my qualifying paper was flunked. The memo presented five petitions and the arguments presented in support. This post contains the full text of that memo. It was a long memo; it may not particularly interest most readers.
Among other things, as detailed below, the memo pointed out that the graders did not explain their grades, as required by the grading rules. There did not appear to be good explanations for those grades.
Dr. Margaret Adamek of IUSSW informed me that the PhD Committee would not be taking any action on my petitions. At this writing, more than a year has passed since I submitted it. In that year, not one member of the administration or faculty has contacted me to ask whether I would be submitting a rewrite within the one-year time limit permitted by the rules.
The Committee’s nonresponse, and the grader’s grades, were consistent with a pattern of harassment going back several years, as noted in the next post in this series.
* * * * *
To: PhD Committee, School of Social Work (SSW)
Indiana University – Purdue University – Indianapolis
From: Ray Woodcock
Date: February 11, 2011
Re: Five Petitions Regarding PhD Qualification
This memorandum presents five petitions relating to the qualifying paper process within the School of Social Work (SSW) at Indiana University – Purdue University – Indianapolis (IUPUI), as described in the SSW’s Ph.D. Student Handbook 2009-2010. Each petition is supported, in the following pages, by a relatively comprehensive discussion of supporting evidence.
What is advocated here is a better approach to the PhD qualification process. These changes could yield a process that exemplifies the best in PhD education and significantly improves the marketability and morale of this program’s students.
It may not be necessary for most readers to plow through every line of this document. If the reader finds that the reasons provided in support of a petition are persuasive, s/he is invited to skip ahead to the next petition. One hopes, however, that readers who are responsible for maintenance and revision of the Handbook will pay some attention to relevant remarks throughout this memo.
The titles of the sections (below) presenting these five petitions are as follows:
Petition 1: Clarify Qualifying Requirements
Petition 2: Revise Evaluation Criteria
Petition 3: Count Certain Publications Toward the Qualifying Requirement
Petition 4: Discard My Reviewers’ Grades
Petition 5: Deem My Qualifying Requirement Satisfied
These petitions progress from those of general interest to all PhD students to those addressing my particular case. Even the latter may have relevance to other students, however.
In the PhD qualification process as presently designed, there is no clear connection between students’ actual competence and the grades they receive. At least one PhD graduate who has joined the SSW’s faculty was flunked in her own first try at the qualifying requirement, due to the poor guidance she received. Research into past “failing” grades would likely reveal related stories among other previous students within this program.
The reader may agree that all five petitions have merit – that, indeed, it is remarkable that I should even have to point out some of these things. The arguments in support of these petitions are compelling. I realize they may not make entirely pleasant reading for the faculty member(s) who are responsible for some of the problems critiqued here. I am sorry that things must be that way. It would have been better all around if faculty had been responsive to previous attempts, by me and others, to obtain appropriate action on these and other objectionable aspects of the PhD student experience within the SSW.
Petition 1: Clarify Qualifying Requirements
Pages 23-25 of the Handbook (attached) contain a section entitled “Qualifying Exam Guidelines.” In this Petition 1, I am asking the SSW to revise that section, so as to give students and graders (who are also referred to, here and in some other related materials, as “reviewers”) clear and appropriate guidance in connection with the preparation and grading of qualifying papers.
The following paragraphs provide my arguments in support of that petition. These paragraphs are arranged into several sections. Each section discusses a topic area in those pages in the Handbook. The gist of these arguments is that the Handbook provides unacceptably poor guidance in each of the following topic areas.
The “Purpose and Rationale” section begins at the bottom of Handbook page 23. It does not clearly distinguish purpose from rationale. This discussion uses the former term as shorthand.
Although that section begins at the bottom of that page, there are actually two previous sentences that also seem to state the purpose of the qualifying process. Handbook page 23, second paragraph, reads as follows:
The qualification experience is designed to provide the student with an opportunity to demonstrate that she/he has mastered the body of knowledge and skills necessary to engage in independent research and knowledge development.
The “qualification experience” (or, as it is called later on page 23, the “comprehensive/qualification process”) is not defined. The next paragraph states that this qualification experience
provides each student with an opportunity to profess to his/her colleagues in a scholarly format that demonstrates the student’s ability to apply, in an integrative fashion, knowledge from practice, policy, human behavior and research as well as the student’s external minor to problems or issues of significance to social work.
Yet while both of these quotes seem to be describing the purpose or goal of the qualifying process, they point in different directions. Is the process oriented toward the entire “body of knowledge and skills,” or just upon those that the student will be applying to one or more “problems or issues of significance”? This question becomes more prominent, but also more confused, in a third statement of purpose, appearing at the bottom of page 23:
The comprehensive/qualification process for students enrolled in the Indiana University School of Social Work PhD Program is intended to serve both a “summative” as well as a “foundational” purpose. It is summative in the sense that it provides students with an opportunity to synthesize or pull together in a “comprehensive” manner the knowledge, values and skills that have been accrued as a result of their formal doctoral studies. Within this context, it is intended to be extensive and inclusive of content from both the social work and external minor areas of the student’s doctoral studies. It is foundational in the sense that it also provides the student with an opportunity to analyze in a very focused way the meaning and relevance of that body of knowledge, values, and skills in terms of an emerging research agenda. Within this more formative context, the qualifying process is intended to be intensive and specific with respect to the student’s knowledge building interests.
Upon comparing this lengthy version against the first two statements of purpose, the student discovers that they are substantially different. This third description makes no reference to key terms from those previous descriptions. There is no specific reference, here, to “independent research,” mastery, “integrative” effort, “policy,” or even “problems” or “issues of significance to social work.” Conversely, this third description’s numerous emphasized terms (e.g., “summative,” “foundational,” “comprehensive,” “extensive,” “inclusive,” “intensive,” “specific,” “synthesize”) do not appear at all in those first two descriptions.
At the top of Handbook page 24, the student encounters a fourth description of the purpose of the process. Here, it is said that
The comprehensive/qualification process . . . is intended to provide tangible documentation that the student has indeed attained the requisite knowledge and demonstrated the expected competencies as prescribed by the program. It is designed to address issues that are related to all aspects of the Ph.D. Program, including, but not limited to, requirements established by the School of Social Work; the external minor; and the student’s research internship.
This statement, unlike those discussed above, goes beyond knowledge areas to include “all aspects of the Ph.D. program.” As such, it requires content pertaining to “expected competencies” and to “the student’s research internship.” But the Handbook does not, in fact, “prescribe” any “expected competencies.” Its only reference to competencies is to research competencies, in connection with the research internship. There is no explanation, here, of how those competencies are to be brought into the qualifying paper – which, as described in this section of the Handbook,certainly does not sound like a research paper. Nor is there any explanation of what the student should do if s/he winds up in a research internship that has nothing to do with his/her area of research interest. Competencies in this sense seem different from the “skills” cited in the paragraph at the bottom of page 23. That earlier paragraph’s repeated references to “knowledge, values, and skills” seem to relate, instead, to the 27 credit hours required in Component I of the student’s coursework (Handbook pg. 3).
It must be obvious to most readers, by this point, that if the SSW cannot provide a single, clear statement of the purpose of the qualifying requirement, students are going to be at a loss as to what their qualifying papers are supposed to be demonstrating. It is troubling that students have visibly struggled with these matters, and that in some cases their papers have been given failing grades, and yet that the SSW has not rectified the problem.
The Handbook’s assorted descriptions of the purpose of the qualifying process seem to be driven by a vague sense that there are lots of things that PhDs might wind up doing. But one can say that about adults in general. The PhD in social work does tend to have some pretty specific purposes. Yet even those seem premature at this point. In practical terms, the purpose of the qualifying exam is not to prepare or test the student to function as a PhD in society. That is the purpose of the PhD program as a whole. The purpose of the qualifying process seems to be to determine whether the student is prepared to advance to candidacy.
The Handbook does not presently have a section that explains the purpose and process of candidacy. This seems to be an oversight. Students might appreciate the addition of such a section. With such a section, perhaps this qualifying section of the Handbook could be refocused upon the more specific question of what is necessary to insure that the student is ready to become a candidate.
A Publishable Paper
This discussion has identified four different statements of the purpose of the qualifying process, within the pages of the Handbook devoted to that process. There is yet another such statement. This fifth statement appears in the Format section, on page 24 of the Handbook:
The comprehensive/qualification process culminates in a written substantive paper of publishable quality (approximately 40-60 pages in length). The style and format for the presentation should follow the guidelines established in the most recent edition of The American Psychological Association Manual. . . . . Copies of written comprehensive/qualification materials that have been approved in the past are available for review by students in the Doctoral Lounge.
As an aside, this text should be revised to explain what is meant by the word “substantive.” To my knowledge, “substantive” tends to be distinguished from “procedural.” I am not sure how a PhD student would write a procedural qualifying paper. (Of course, it is redundant to require a “written”substantive paper.)
This quotation operationalizes the purpose of the qualifying process. In the terms used in this paragraph, the goal of that process is not like what was said in the previous statements. It is simply to produce a publishable paper and to complete an oral component. As PhD students learn, and as this quote seems to say, being “of publishable quality” means more than being presented in a nice format, else the second sentence of the quotation would be redundant.
But this yields a quandary. For practical purposes, papers of 40 to 60 pages in length are not publishable. This conclusion is based, in part, upon my viewing of several thousand publications related to social work over the past six years of master’s and doctoral education. Papers of 40 to 60 pages are too long to go into journal articles or book chapters, and too short to become full-length books, and these are the forms of publication with which social work graduate students tend to become acquainted. One would hope that this program has taught students not to crank out overlength papers and then wait in vain for someone to publish them. Our papers have almost invariably been expected to be of publishable length, in the vicinity of 15 to 20 pages.
It may be instructive to compare how it is done elsewhere. In the social work PhD program at the highly ranked school of social work at the University of Washington, “The Qualifying Scholarly Paper . . . must be submitted for publication.” In lieu of a qualifying process that yields a professionally useless 40- to 60-page monstrosity, the PhD student at Washington has a good chance of winding up with a publication on his/her vita. Moreover, faculty are involved throughout the qualifying process, beginning with the student’s submitted paper outline and continuing through feedback and revision stages. This is a world away from the IUPUI process, where there seems to be a desire to snare and defeat the student. S/he has to figure it out by him/herself – and if s/he gets it wrong, s/he has just one chance to guess again. IUPUI expressly discourages Washington-style interaction: “Faculty members should not provide a review of multiple drafts of the qualifying exam” (Handbook, p. 25).
Other Students’ Papers
That fifth statement of purpose of the qualifying process (above) alludes to previous approved papers, available for review in the Doctoral Lounge. In my last visit, the relevant filing drawer in the lounge did not contain copies of all of the passing papers that students had submitted to date. It was not clear whether the papers provided there had been specially selected, or just happened to be some that somebody put there. Dr. Adamek has indicated that she, herself, has not added any papers to that drawer for a number of years – that, indeed, the SSW does not even retain copies of the successful or failing papers that students have submitted.
The papers in that drawer vary remarkably in quality and other characteristics. Among the papers I saw during my last look at that drawer, every one disregarded the 40- to 60-page limit stated above. When converted to 12-point type, all consisted of more than 60 pages of text, plus additional pages for references and appendices. One student’s paper ran to 88 pages. As shown below, the actual scoring process for the qualifying paper does not enforce a 60-page limit. In short, faculty and the Handbook have countenanced a process in which students have displayed an impression (indeed, a fear) that even 60 pages may not be enough.
In case it needs to be stated, the effort to produce a high-quality paper of this length can be tremendously time-consuming. The student who wrote that 88-page paper was a University Fellowship recipient, as I am. No doubt he spent weeks on his paper, if not months. Other students, for whom research and writing are challenging, may struggle with theirs for even longer periods. This is a very large chunk of time and effort to waste on an unproductive pursuit.
Handbook pages 23-25 contain a section listing Substantive Areas (below). None of the seven papers that were in the drawer conformed to that list. Six of the seven did clearly divide their pages into several major parts; but among those six, no major heading name (e.g., “Literature Review,” “Epistemology”) appeared consistently throughout all papers. Some papers provided reviews of literature in two or more sections.
These observations support several impressions. First, students are not sure how they are supposed to structure their papers, even on the most general level. They do not know how long they are supposed to be, but they seem to understand that the Handbook is not a reliable guide on that question. As supported by conversations among students, the papers in the drawer demonstrate that students have been flying blind – have been developing idiosyncratic organizational schemes, that is, and just hoping that their reviewers would approve. (Conversations have also indicated that students have sometimes been unpleasantly surprised by professors in this regard.)
The drawer in the PhD student lounge does not contain rejected papers, nor does it contain qualitative information regarding traumatic aspects of the qualifying process. The impression of unnecessary confusion and dismay would surely be enhanced if research were done into the actual qualifying experiences of students in this program.
A Problem of Generality
The third statement of purpose quoted above envisions that the student will “synthesize or pull together in a ‘comprehensive’ manner” manner the entire “body of knowledge and skills necessary to engage in independent research” – that s/he will produce an “extensive and inclusive” compilation of the “knowledge, values, and skills” accrued throughout his/her years of study.
One wonders – students wonder – what this requirement could possibly mean. Few if any human beings will be able, within their lifetimes, to comprehensively pull together and present, in writing, the body of knowledge of social work. Even if restricted to that fraction of social work knowledge actually covered in our years of doctoral coursework, such content would obviously spill far beyond the limit of 40 to 60 pages.
The Handbook, as just quoted, puts quotation marks around “comprehensive.” Why? It is not clear. Possibly the quotation marks mean that the qualifying paper is not really supposed to be comprehensive. Maybe the emphasis is meant to be on “synthesis”; maybe the student is just supposed to produce a comprehensive synthesis of “the knowledge, values and skills that have been accrued.” But this is not realistic either. Synthesizing materials does not mean discarding or simplifying them. A synthesis just combines things into a unified statement. A textbook might be an example of a synthesis. No students – indeed, few if any members of this school’s faculty – are ever going to produce, by themselves, a comprehensive, competent textbook synthesizing the broad variety of matters listed in the foregoing descriptions.
The expectation of a global synthesis is not consistent with the structure of PhD coursework. Doctoral study in this school does not consist of a two-year course called Social Work 101 (or 701). Our courses address a variety of discrete subjects on micro and macro levels. If such subjects are somehow synthesizable, we have not studied or been shown how to achieve that. There seems to be good reason for not making any such effort. In our studies, an item such as “An understanding of the process of enacting legislation at the state level” (from the syllabus for the S730 course) has rightly been segregated from, say, Gestalt therapy (which was presented in S710). In our training, there is no basis for a belief that these sorts of items can and should be synthesized within a general paper covering the gamut of social work knowledge.
At a certain point, some readers may begin to wonder what kind of educator would be making the outlandish demands quoted above. There is a fair question, not only from these materials but from others discussed below, as to the real objective being pursued. The Committee is urged to consider whether the SSW seeks to model a positive learning environment that follows best practices and enhances interest in learning about social work. This would be very different from a place that tries to intimidate students and that makes them jump through hoops, diverting their energies away from productive activities. Such concerns, recurring throughout this petition, suggest that stakeholders will benefit if the SSW’s better educators are given the lead in the writing and application of the Handbook.
Summative Yet Foundational
The paragraph at the bottom of Handbook page 23 says the qualifying process “is summative in the sense that it provides students with an opportunity to synthesize or pull together” the vast quantities of material just mentioned (emphasis in original). This quote may mean that the student is actually expected just to provide a summary. If so, it is not clear why s/he is simultaneously expected to provide a synthesis. There seems to be a confusion of two concepts here. Summary and synthesis are not the same thing. For instance, one can summarize the existence of incommensurable paradigms by saying that there are incommensurable paradigms. But it may be impossible to synthesize incommensurable paradigms.
Hopefully a capable editor can eliminate one such demand or the other, thereby clarifying what students are really supposed to be producing. Again, unless the SSW intends to require volumes of material in students’ qualifying papers, the demand for a “synthesis” will have to be deleted. Summaries can be of any length, from multivolume compilations right down to a single sentence – though it would still be difficult if not impossible to summarize the spectrum of social work knowledge, in anything but the most superficial terms, in 40 to 60 (to 80) pages.
Dropping “synthesis” and going with “summary” would raise another problem. Within the peer-reviewed professional literature that students study, and in which they strive to have their work published, there is not much demand for summaries. There does not appear to be any demand at all for summaries that grossly aggregate a plethora of social work topics, as the Handbook seems to expect. In other words, the Handbook’sdemand for publishability is rather sharply opposed to several of the terms that appear in the purpose statements found on Handbook page 23.
As page 23 admits, the demand for a summary is also juxtaposed against the “foundational” requirement found on that page. The foundation “provides the student with an opportunity to analyze in a very focused way the meaning and relevance of that body of knowledge, values, and skills in terms of an emerging research agenda.” This very puzzling. Is the student to devote the first half (or some other fraction) of the paper to a summary, and then devote the rest to creating a foundation? In that event, to revise some of the foregoing statements, the student does not even have 40 to 80 pages to provide the required summary. Now it seems s/he must summarize the field of social work in roughly 30 to 40 pages. Or is the paper instead supposed to proceed with a grand summary of the entire field while simultaneously drilling down into focused research? Either way, it sounds like a formula for a manuscript that is unpublishable, not only because it is too long and too general, but also because it is too disjointed.
Confronted by these imponderables and impossibilities, students have had to temporize. They have obviously not been able to provide comprehensive or synthetic statements, or meaningful summaries, of the entire body of social work knowledge. So they have had to guess how much attention they should devote to those demands, and how much they should devote to their own specialized topic. As noted above, the SSW does not provide full information on the variety of approaches that students have tried, successfully or not. But the few papers available in the PhD student lounge (above) indicate that some successful papers have simply ignored all of this stuff about comprehensive and synthetic and summative statements, and have focused solely upon a single topic of interest. In effect, with the blessing of their professors, these students have produced a glorified research paper, with lots of extra material that would have to be trimmed out to make it publishable.
Judging from an informal poll taken in one course, almost none of the SSW’s PhD students have published. So they could use some support and encouragement from the SSW in that regard. The Handbook does require that the qualifying paper be publishable, but there almost seems to be a sense that students’ papers don’t really get published, so there is no point in taking that requirement seriously. Rather than sell students short, however, there are some good reasons to treat publishability as an important criterion for qualifying papers. The possibility of a publishable article to show to others and list on one’s vita is much more motivating than the chore of writing yet another paper – a really long one – that will be read by almost no one. Having a specific topic, to be addressed within a relatively brief space, promotes focus in the treatment of the subject. The reviewer’s critique may be more connected with the real world if s/he is evaluating a manuscript for its publishability. And the SSW would surely benefit from being able to say that some recognizable percentage of its PhD graduates have published.
This is not an argument that the qualifying paper should not be long, though there are reasons to think that is true too. As noted elsewhere in this memo, leading schools do not tend to impose a long qualifying paper requirement. Their approach may be to treat the dissertation proposal as the place where voluminous writing becomes appropriate. If so, revision of the Handbook to clarify and flesh out the nature of doctoral candidacy may facilitate a shift of some of the burden away from the qualifying stage.
Even if the SSW decides not to emulate other leading schools in this regard – if, that is, the Handbook continues to require PhD students to produce lengthy manuscripts at the qualifying stage – there is no clear reason why such manuscripts must consist of one linear treatment. It would be possible to court publishability by breaking the long paper into manageable, publishable parts. The student could still combine or draw upon those parts to generate a more monumental document (notably, the dissertation) as the pieces continue to come together. It would surely be more conducive to publication, and to the quality of written work produced in this process, if students were encouraged to begin by developing manuscripts of roughly 15 to 20 pages, each tackling a part of the specific topic, rather than taking the roundabout approach of piling up all those excessive pages and then having to take them down again if s/he does wish to wind up with a publishable article.
To recap, it seems fairly obvious that students will tend to produce better qualifying papers if they stick with the format of the standard-length research or conceptual paper; that doing so will better comply with the Handbook’s requirement that the qualifying paper be publishable; that this is the orientation of some other leading social work PhD programs; and that, in any case, at least some accepted qualifying papers in this SSW have ignored the Handbook’s verbiage about comprehensive, synthetic, or summary treatment of the entire field of social work.
It is difficult, then, to identify specific reasons why it would be wrong for a student to break his/her qualifying paper into several pieces, and to approach each as a publishable manuscript. This seems, rather, to be a sensible innovation, one that a good educator would applaud, not punish. But with that remark, I am getting ahead of the story.
“Four” Substantive Areas
As noted above, the purpose statements on Handbook page 23 refer to a broad variety of subjects in social work. On page 24, the Substantive Areas section provides a list of four substantive areas that are to be addressed, in some sense, in the qualifying paper.
Interpreting the Header
For purposes of the qualifying paper, it is difficult to understand what the relationship is supposed to be, between this Substantive Areas section and the broad purpose statements on page 23. The Substantive Areas section opens with a bewildering spray of verbiage:
Within the context of these two major organizing foci (i.e., summative and foundational), the written document is expected to demonstrate the student’s capacity to critically analyze, synthesize, apply, and evaluate the contents of her/his educational experience in relation to the following substantive areas:
1. A summary synthesis of the various components of the student’s educational experience in relation to a specific area or issue of central importance to social work. This component must make clear how the completion of an external minor has helped inform the student’s conception of the identified issues and the theoretical and empirical questions that need to be addressed.
It seems the student is expected to provide a critical analysis, synthesis, application, and evaluation of the contents of his/her educational experience in relation to a summary synthesis of the components of that educational experience in relation to a specific area or issue, within the context of summative and foundational foci. But I do not know what that means.
So I will try to interpret that quotation. First clause: whatever we are talking about here, it happens within the context of the summative and foundational foci. Second clause: the qualifying paper should demonstrate the student’s capacity to do something. Specifically, it should show that s/he can critically analyze, synthesize, apply, and evaluate something.
Time for a pause. It is easy enough to roll the words off one’s tongue: analyze, synthesize, apply, and evaluate. But what do they mean? Is it even possible to conduct all four operations on any particular datum? Consider, for example, the present memo. It consists predominantly, one might think, of an analysis. What would it mean to request that it be synthesized? The questions might be, synthesized with what? and why? The answers to such questions would determine the nature of the output. One cannot synthesize everything with everything, and the contents of any particular synthesis will depend upon its purpose.
Similarly with the other terms. One could say that the analysis applies X and evaluates Y; or one might prefer to say that this is an evaluation that analyzes A and applies B. But is there some particular reason to pile up verbs – some legitimate, intelligent purpose, I mean, as distinct from ostentation or intimidation? Or can we just pick one verb – “discuss,” perhaps, or “examine” – and get on with it?
So: to return to the paraphrase. How about this reformulation: “Within the context of summative and foundational foci, the qualifying paper must demonstrate the student’s capacity to discuss the contents of his/her educational experience in relation to the following areas.” It is still very murky. What difference does the context of the foci make? How would this demonstration differ if it were conducted outside of the context of the summative and foundational foci (whatever that may mean)?
The summative and foundational foci seem to refer to just about everything, so perhaps we can dispense with that part. In that case, a glimmer of intelligibility emerges. “The qualifying paper must demonstrate the student’s capacity to discuss what s/he has learned in relation to the following substantive areas.” This may not be what they want, but at least it makes some sense. So now (whew!) the question is, what are those substantive areas?
Numbered Paragraph 1
The substantive areas are stated in four numbered paragraphs. I have quoted (above) the first of those four. That numbered paragraph 1 requests “a summary synthesis.” So the student is to “analyze, synthesize, apply, and evaluate” in relation to a synthesis. This, again, is inscrutable. Even the paraphrased version is unclear: “discuss what s/he has learned in relation to a summary synthesis.”
A reasonable guess is that the summary synthesis amounts to a discussion or presentation. So now we’re down to a simple redundancy: they want a discussion of a discussion. Eliminating that, we have a request for a discussion of those various components. Putting it together, the assignment posed in that first sentence seems to be this: “The qualifying paper must demonstrate the student’s capacity to discuss what s/he has learned in relation to a key issue.”
Then again, the capacity will be demonstrated by the act. The real point appears to be, simply, “The qualifying paper must discuss what the student has learned in relation to a key issue.” That seems to be what the header and the first sentence of numbered paragraph 1 are trying to say.
Now that one plausible translation of the Sanskrit heaves into the light of the current day, some readers may be scratching their heads. Why would the SSW be asking for something regarding the contents or components of the student’s educational experience? Are we to understand that the qualifying paper should consist of a mere recitation of what we studied in our various courses, as applied to a particular issue? Is there not supposed to be any additional research into the question? The sort of semester-end summation that seems to be requested here would be a far march from the glorified research papers (above), with a hundred references or more, that students have been submitting for their qualifying papers.
I am not sure, but it appears that the intended purpose of numbered paragraph 1 may have gotten lost in the fog of verbality.
Numbered paragraph 1 continues on, in an additional sentence that calls for an explanation of the conceptual contribution made by the student’s external minor. This poses the question of what contribution the minor should have made. The message here seems to be that the student had darned well better come up with some story about the minor, even if it was actually a bust, or if it proved not to be as relevant to the student’s future research plans as s/he originally expected.
The centrality accorded to the minor at this qualifying point is not entirely consistent with Handbook page 6, which yields contradictory indications that the minor is intended to provide “a specialized focus” for research and yet is also intended to “broaden” the student’s “locus of inquiry.” The former seems dubious; it is unclear why a social work PhD student should be required to seek his/her “specialized focus” in some area outside of social work. “Broadening” seems more consistent with IU’s University Graduate School 2009-2010 Academic Bulletin(p. 18), which says that “A minor program provides additional breadth and depth to the individual’s program.”
There does not seem to be a basis for the Handbook’srequirement that a minor, often chosen years earlier, must necessarily have “helped inform the student’s conception” of the issue on which s/he has now decided to specialize, in the dissertation and thereafter, for years to come. The Handbook should be revised to accommodate the fact that the dissertation topic may appropriately diverge from the minor as additional learning experiences exert their influence.
It does not seem that the social work major and the student’s minor should have been combined into a single Substantive Area. That may have been the original intent. But they are substantively distinct, except to the extent that the student can, and wishes to, identify some theme they share. Hence numbered paragraph 1 contains more than one distinct topic. This mixing may be the reason why numbered paragraph 1 begins by speaking of a single “specific area or issue,” but ends by referring to multiple “identified issues.”
Numbered Paragraph 2
Numbered paragraph 2 likewise combines several topics. This provokes uncertainty as to how the student is to construe these four numbered paragraphs. Does the SSW consider them equally important? Should their constituent topics be kept together in every qualifying paper? Or are they just casual groupings without any particular significance? The Handbook does not say.
Paragraph 2 reads as follows:
2. A critical discussion of the underlying epistemological issues and their relationship to the development of theory and research related to the student’s specified area of social work interest. This should include an analysis of the prevailing paradigms that have shaped the debate in relation to the student’s specified area of social work interest.
This raises another question about the general structure. Didn’t numbered paragraph 1 already cover the entire range of subjects studied in this program, in its reference to “the various components of the student’s educational experience”? Or is this paragraph saying that numbered paragraph 1 requires only a recitation of classroom learning, but now we are proceeding on into new material that calls for additional research? If so, why privilege epistemology that way? Like any number of other subjects, it too was one of the things we covered in class. And if epistemology is supposed to be treated as a special subject, why have some accepted student papers been able to get by with as little as one page on it?
Paragraph 2 begins with the expectation that social work students can competently provide a “critical discussion of the underlying epistemological issues” related to their area of interest. I suspect that a philosopher would laugh at the conceit implicit in this expectation. Armed with the additional information that students have tried to do this in just a page or two, a philosopher might ask whether social work educators care at all about their profession’s reputation.
The concept here appears to be, not that the student will actually know anything about epistemology, but that s/he will toss in a few remarks that paraphrase how other social workers have interpreted what someone like Denzin or Guba says about it. I suggest there is a better way. Epistemology, like research methodology, legislative process, or financing, is undeniably central in some academic debates, and no doubt its potential relevance could profitably be recognized in others. But unless one takes a parochial (e.g., purely philosophical, methodological, legal, financial) worldview that would require all questions to be examined through one particular lens, a good PhD should recognize that questions can almost invariably be circumscribed, revisited, or circumvented in multiple ways. It may make good sense to expect the PhD student to identify something in the nature of a foundational perspective underlying his/her work, or to indicate why talk of foundations is not suited for the particular question. But to assume that the philosophy of science provides the foundation for all questions in social work is like assuming that they are all ultimately matters of chemistry and physics. This may be true, and may even be crucial for some purposes, but it is largely irrelevant to social work.
In other words, numbered paragraph 2 depends upon a reductionist assumption that does not have much practical usefulness for social work PhDs. We are training to be scholars. Competence for this purpose requires that we develop some awareness of when we might be out of our depth. In that sense, the students who have submitted one page on epistemology, thereby implicitly rejecting the prominence accorded to the subject in the Handbook’s paragraph 2, have been tending toward the more intelligent solution.
Numbered Paragraph 3
The next numbered paragraph in the Substantive Areas section requires
3. An analysis of the identified area of social work interest in terms of its implications for the development of theory and research related to social work practice, policy and human behavior. This component must include content specifically related to issues of ethics, diversity and oppressed populations.
This paragraph is as hard to understand as paragraph 1. Its first sentence assumes that areas or issues have implications. Is that so? Poverty is an area or issue. What are its implications? It’s hard to say. There are people who take vows of poverty for religious reasons. At the PhD level, identifying an implication, or even an issue, requires a great deal of specificity. But as we get into an issue with enough precision to warrant talk of implications, we find that we are not dealing with a general “area” anymore, and we are far away from a general-purpose discussion of “the various components of the student’s educational experience,” never mind a comprehensive discussion of social work knowledge. This seems to be talking about an entirely different paper.
Paragraph 3 has other problems. It is not clear why the first sentence distinguishes human behavior from social work practice and policy, or why it does not include other kinds of social work knowledge (e.g., theory). Ordinarily, one might think that practice and policy cover the micro-macro spectrum. What is the reference to “human behavior” intended to add to that? If “human behavior” were not there, one could understand that the paragraph is demanding a potentially infinite list of implications – the better word would probably be “speculations” – regarding the development of theory and research across the micro-macro spectrum. But with the addition of that reference to “human behavior,” the expectation becomes undefined.
The second sentence of numbered paragraph 3 distinguishes ethics from diversity and oppression. No such distinction appears in the NASW’s Code of Ethics, so it is not clear what the SSW has in mind there. It is also unclear what counts as “content” related to these three words (i.e., ethics, diversity, oppression). Judging by some previous students’ papers, it has been sufficient to just add a page or two providing a few brief remarks on one or more of these topics. So here, as with epistemology, the listing of the topic in one of the four numbered Substantive Areas paragraphs does not seem to connote much significance, raising again the question of why the topic is included at all.
It is unclear what the two sentences of paragraph 3 have to do with one another. No doubt every subject in social work can be related to every other one. But the codes of ethics that I have seen do not privilege the subjects described in that first sentence. This matters because, as in the other numbered paragraphs, there is a claim that numbered paragraph 3 constitutes a “component.” What sort of entity would it be, in which “implications for the development of theory” would be tightly interwoven, together with ethics, diversity, and oppression, into a single “component”? This is the question of what kind of paper is expected. It remains a mystery.
Numbered Paragraph 4
The fourth numbered paragraph in the Substantive Areas section requires the student to articulate his/her own general research agenda. This is a sensible requirement, but it is not clear why it would be treated as part of a paper on which faculty are discouraged (above) from reviewing drafts. In a supportive environment, one would think, to the contrary, that the student’s research agenda would be developed and revised over time, with faculty input. It would make more sense to require the student to submit a one- or two-page agenda summary, along with but distinct from the qualifying paper.
That suggestion would also resolve the concern that the proposed agenda is not really a substantive area of knowledge and, as such, does not belong in the Substantive Areas section. Putting the agenda into paragraph 4 conflicts with the end of the first paragraph on Handbook page 24, which states that the qualifying paper “lays the foundation” for the student’s developing research agenda – implying that the agenda is not, itself, part of that paper.
Numbered paragraph 4 requires the student to provide “a critical review of the relevant literature” pertaining to his/her proposed research agenda. This seems to be why some students’ papers have more than one literature review. Again, they are confused. They have provided all of the information required by the Handbook’s various descriptions of the purpose and contents of the qualifying paper, including the contents of numbered paragraphs 1 through 3, and are now apparently required to provide another literature review related to the research agenda. Here, again, one sees the need for some editing, but also for some empathy, when the educator can plainly see that students are getting poor guidance.
Recap and Further Observations
As in other sections of the Handbook’s instructions regarding the qualifying paper, the Substantive Areas section gives the student poorly conceptualized and poorly written guidance. Contrary to the assertion at the end of the section’s first paragraph, the four numbered paragraphs do not present four significant and distinct substantive areas. Rather, paragraph 1 requires the student to talk about everything in relation to his/her area of specialization and also his/her minor; paragraph 2 presents an epistemological requirement that neither the SSW nor, apparently, most students take seriously; paragraph 3 requires unbounded speculations about implications, and also requires “content” about ethics; and paragraph 4 does not identify any particular substantive area of knowledge, but rather calls for a research agenda.
It is interesting to observe what this list leaves out. The numbered paragraphs in the Substantive Areas section that address specific substantive topics seem to be linked to the materials that students study in certain required PhD courses. (See Handbook p. 61.) That is, paragraph 2 (regarding epistemology) seems to relate to S720, Philosophy of Science & Social Work, and at least the first sentence of paragraph 3 seems to relate to S710, Social Work Theories of Human & Social Behavior. This raises the question of what happened to the other required courses in the PhD program. Students typically take several courses in quantitative and qualitative research methods. If the paper is to be as comprehensive as some of the Handbook’s pronouncements emphasize, surely the general area of research methods would merit a numbered paragraph of its own. Likewise, where did S740, Social Work Practice: Theory and Research go? The answer may be that the cursory reference to “theory and research” in paragraph 3 is intended to draw in that entire course – in which case the same paragraph’s one-word mention of “policy” may also be intended as an entrée for S730, Proseminar in Social Policy Analysis.
Is the student to understand, from this arrangement of topics within the numbered paragraphs, that all of the subjects balled up into paragraph 3 (including human behavior, theory, policy, and ethics) deserve no more attention, combined, than epistemology (paragraph 2) deserves by itself? That cannot be, else no accepted papers would have devoted a mere page or two to epistemology.
In short, there does not appear to be any guiding logic behind the selection, description, and arrangement of topics within the numbered paragraphs in the Substantive Areas section. This is not to say the section’s development was completely illogical. The argument being presented here is that, in this area as in others discussed above, students have understandably found it very difficult to know what is expected.
Handbook page 25 contains a section listing 14 “evaluation criteria” and stating that these criteria are to be graded on the Qualifying Exam Rating Form provided in Handbook Appendix 3 (pg. 48) (copy attached). As page 25 and the Rating Form indicate, each of these 14 criteria is worth a maximum of five points. A score of zero points is called “Unsatisfactory.” The other point-word associations are: 1 = Marginal, 2 = Satisfactory, 3 = Good, and 4 = Excellent. There are to be three reviewers: two from social work and one from the student’s minor area. The student will pass only if (a) each reviewer awards a score of at least 28 and (b) the average of all reviewers’ scores is at least 36.
Page 25 does not say whether the minor advisor’s interpretation of the social work content of the paper should be included in the calculation of the average. A reader might think that the minor advisor, who may know nothing about social work, would not be qualified, by him/herself, to prevent a PhD student from proceeding to candidacy; but the SSW does not take that position explicitly. As noted on Handbook page 23, the SSW does have the option of not requiring a qualifying exam covering the minor area, and apparently also has latitude to decide the role of the minor reviewer, either by a blanket rule or in the individual case.
Based on the information just stated, the maximum possible score is 56 (i.e., 14 x 4). For the student to achieve a passing score of 28 per individual reviewer, the reviewer would have to award an average score of Satisfactory (i.e., 2 points) across the 14 criteria. A passing average of 36 across all reviewers requires an average per-item score of 2.57 (i.e., 36/14), which is one point above the midpoint between Satisfactory and Good (i.e., 2.5 = 35/14). It is not clear why the SSW designated 36 as the passing average. There is no indication of what happens if a reviewer leaves one or more items ungraded, or demonstrates some misunderstanding or incapacity to provide a competent grade on any item or on the paper as a whole.
Based upon information provided to students, it does not appear that the SSW itself evaluates the assumptions, usefulness, or effectiveness of its rating approach. This apparent failure to implement best practices appears to be responsible for the perpetuation of a number of rather glaring flaws in the evaluation system.
First, according to Handbook page 25, “Each criterion is equally weighted.” This is an interesting approach. Consider the first of the 14 criteria: “Accuracy with respect to the application of the concepts and principles selected for discussion.” The student could score zero on this item – could be completely inaccurate throughout the paper – and yet, if s/he scored well on most other criteria, s/he could pass, because accuracy is just one criterion. Similar observations apply to other items, such as criterion 7, “Understanding of research concepts, methods, and related issues.” In fact, the 36-point threshold implies that the student could score zero on five items – items that may be fundamental to competence at the doctoral level – and still pass.
Rather than subject the student (and the reviewer) to an undifferentiated laundry list of 14 equally weighted items, it may be advisable to prioritize some criteria as essential for any PhD. It does seem that accuracy or, to cite another example, “Scholarliness as reflected in intellectual discipline, logical consistency and critical judgment” (criterion 6) would usually be considered more vital than, say, “Recognition of the implications for related research” (criterion 10).
The list of 14 criteria raises another question: where did this list come from? The question is, in other words, whether this list comprises a fair and appropriate device for evaluation of doctoral qualifying papers. One hopes that the SSW did not merely throw together a bunch of items that sounded like good ideas at the time – that this is based upon research into the makings of a good PhD. The Handbook cites no scholarly sources or even Indiana University authorities, though, and my own inquiries have not yet led to anything resembling this particular list of criteria.
The Evaluation Criteria section on Handbook page 25 begins by saying, “Student performance is evaluated in relation to the substantive areas discussed above in Section III.” That is not correct. The Rating Form is the only place where points are calculated, and its contents (reflecting the Evaluation Criteria) are inconsistent with the Substantive Areas (above). Starting with Substantive Areas numbered paragraph 1, the criteria do not ask about either the minor or the area of specialization. With respect to numbered paragraph 2, the criteria do not ask about epistemology or paradigms. With respect to numbered paragraph 3, the criteria (items 8, 9, and 12) do not ask about policy or human behavior. Finally, with respect to numbered paragraph 4, the criteria do not ask about the literature review or other characteristics of the research agenda, aside from expressing a seemingly premature concern with human subjects issues that will arise as the student pursues that agenda in the dissertation phase.
Criterion 7, unlike the Substantive Areas section, does ask whether the paper demonstrates an “Understanding of research concepts, methods and related issues.” The criterion does not require any particular kind of demonstration or discussion with respect to such matters, however. This leads into a concern spanning the criteria generally: the Rating Form and related materials in the Handbook provide no indication of how either the student or the reviewer should distinguish among Excellent, Good, Satisfactory, Marginal, and Unsatisfactory performance. Using criterion 7 to illustrate, it could easily happen that the student has a solid grasp of research concepts and methods, but decides that his/her particular qualifying paper does not call for a lengthy discussion of those topics – and could then be docked points for making that reasonable decision. Returning to “accuracy” as another example, what if the reviewer is not current in the literature cited by the student, or subscribes to a dissenting view or belief? In the present arrangement, the reviewer’s view is taken as gospel, even if it may in fact represent a minority of learned opinion. Or, verging into the area of accountability, what if the reviewer decides to save time and avoid disagreements by just going with the flow – by largely copying, that is, the scores awarded by another reviewer? Nothing in the present arrangement protects students from the possibility that some reviewers may engage in unethical forms of collaboration or free-riding. Abuses do occur. Good practice calls for something more than treating every reviewer’s last word as though it were inscribed in granite.
The Rating Form does require the reviewer to provide “a written justification for your judgment” with respect to any criterion graded Marginal or Unsatisfactory. Yet as discussed below, the SSW does not consistently enforce this requirement; and in any case this post facto faultfinding is no substitute for clear and specific guidance that should be available to students when they are writing their papers.
To summarize this subsection of this petition, the rating system used by the SSW to evaluate PhD students’ qualifying papers does not appear to be based upon educational research pertaining to doctoral students within the university. The rating form and the accompanying grading system do not appear to be of a quality that would support publication, were they to be included as a data gathering tool in a manuscript submitted to a peer-reviewed journal. Information available to the student suggests that the rating criteria consist of general ideals (e.g., creativity) arbitrarily mixed with a spotty selection from certain substantive areas – while other substantive areas, prominently featured in the Handbook’s discussion of such areas, are inexplicably ignored. There is no rubric to guide reviewers or students in the awarding of points for any particular matters, and no accountability to protect students against lazy, cynical, or otherwise abusive grading practices.
In overall summary, the foregoing arguments have been provided to support this petition’s request that the SSW revise the “Qualifying Exam Guidelines” section of the Handbook, to give students and reviewers clear and appropriate guidance in connection with the preparation and grading of qualifying papers.
Petition 2: Revise Evaluation Criteria
A PhD student, facing the confusion and uncertainty described in the foregoing critique of the Handbook’s instructions regarding the qualifying paper, might decide that the best course would be to ignore everything else and just focus on the criteria listed on the Rating Form (Handbook Appendix 3; copy attached). The Form, after all, is where the points are. Surely, s/he might reason, it would not be possible to fail, if one could just figure out what the Rating Form wants.
In light of such practicalities, I am petitioning the SSW to revise the evaluation criteria so that they provide a complete and appropriate guide to what is expected in the qualifying paper.
This request may seem obvious on its face. Apparently not everyone agrees, however, else the matter would already have been taken care of. The following pages therefore provide arguments in support of this petition.
To facilitate the presentation of these arguments, the reader is asked to assume a scenario in which the evaluation criteria are divided into two groups. In one group are those criteria that have to do with considerations of quality applicable throughout the qualifying paper (e.g., accuracy, organization). In the other group are those that identify discrete substantive areas (e.g., theory, diversity). This scenario demonstrates one way to resolve some of the editorial difficulties identified above.
In that scenario – to adapt an observation offered in support of Petition 1 (above) – it will probably not be feasible to equate quality criteria, on a point-for-point basis, with substantive criteria. For instance, it seems unlikely that the SSW will be able to support its present, implicit claim that “Organization and integration of content as specified in guidelines” (criterion 2) is exactly as valuable as, say, “Recognition of issues related to diversity, oppressed populations, and social and economic justice” (criterion 13). These are apples and oranges. There is no way to determine that both are worth four points within a 56-point scheme. What seems more workable is to require separate passing grades in terms of both quality and substance, but not otherwise to link these two dissimilar groups of criteria.
Distinguishing these two groups would free the SSW to require any number of criteria pertaining to quality, or to substance, without worrying about an unbalanced numerical impact upon the overall grade. In the present arrangement, that is, if there were ten quality-oriented criteria and only four substance-oriented criteria, a student could pass by writing a high-quality paper devoid of substance. In the proposed scenario, the student would have to demonstrate both quality and substance, regardless of the numbers of criteria involved in each such demonstration.
Presumably the criteria of quality should be based on published empirical and/or conceptual research. For instance, the criteria that the SSW ultimately favors may derive to some extent from the GADE Guidelines for Quality in Social Work Doctoral Programs. Whatever the quality criteria may be, evaluation to determine their presence in a given qualifying paper is likely to require an approach that differs from the evaluation of substantive content. For one thing, a paper may be organized overall and yet disorganized in one paragraph, and as such may be considered Good or Excellent in terms of organization. Substance is not like that. A substantive area (e.g., theory, practice, method) tends to be treated in some parts of a paper and not in others. A substantively bad paragraph could severely damage a paper’s score on the corresponding substantive criterion.
Presumably the SSW intends its reviewers to be accountable. Accountability in substantive matters is relatively easy: the reviewer can cite sources or counterarguments pertaining to a specific assertion within the paper. It is much harder to be accountable for general judgments of quality that span the entire paper. How, for example, is accuracy to be determined? Does the reviewer get to seize upon a few instances that s/he considers noteworthy, for better or worse, and to ignore the level of accuracy pervading the balance of the paper? Or must s/he award accuracy points for every citation and assertion?
Accuracy, in particular, may be a poor example, insofar as it does not seem to belong in the list of quality criteria. Presumably it will be adequately addressed by the evaluation of substance: the student is, or is not, accurate with respect to e.g., theory, ethics, or policy. Whatever the case with accuracy, the observations offered in the preceding paragraph do seem to apply generally to other quality judgments. Reviewers are not awarding points for “organization” or “creativity” line by line, as they go through the paper. They are forming general impressions.
With respect to those general impressions, academic quality might call for a high bar and an exacting rubric, as noted above. Unfortunately, in my own experience with the qualifying exam grading process, reviewers are not willing to be held accountable for the specifics underlying their general impressions on quality criteria that span the entire qualifying paper. In that event, protection of the student seems to require a relatively low bar, such that a failing grade on quality must be based on a rather glaring (i.e., readily demonstrable) failure, well beyond the potential for arbitrariness on the part of reviewers.
The choice of substantive matters to be covered in the qualifying paper depends upon the paper’s purpose. Since there is no clear statement of that purpose, it remains unclear how substantive criteria are, or should be, selected.
The discussion under Petition 1 identified the possibility that substantive criteria for the qualifying requirement were dictated by the courses that PhD students here are required to take. That discussion noted, however, that the correspondence between courses and substantive criteria is patchy. Even if such correspondence were perfect, in one sense its citation here would beg the question. Why are PhD students required to take those particular courses?
Some faculty may feel that students should not ask such questions. That stance would seem hostile to change and, as such, not necessarily consistent with the general orientation of the profession. If one takes empowerment seriously, the better approach would seemingly be to model processes in which students are encouraged to take an active role in the development and evolution of this program and of their opportunities and obligations within it. In any event, a PhD student who is focusing on social work education, as I am, would seem justified in trying to understand the school’s approach, even if his/her own progress did not depend upon it.
Some faculty may also feel that it is inappropriate even to imagine that the program’s courses and its substantive qualifying criteria should be changed just because one student doesn’t like the way things are. This perspective, prioritizing privilege and tradition, is regularly rebutted when laws change. Programs do, and should, change to accommodate just one woman, or just one member of a minority, or just one person with a disability – because, behind that person, there are others who would be here, but for the institutional resistance that has kept them out heretofore. As often as not, salutary change does begin with one person. From a perspective of social work values, that person is doing the right thing if s/he raises fair questions about the status quo.
Courses required in social work PhD programs are not mandated by any accrediting agency. It certainly would be possible for the SSW to alter its list of required courses. Alteration could be especially advisable if, for example, courses are required (and corresponding qualifying paper criteria are imposed) merely because certain professors want to teach them, or because PhD students are assumed to require an inordinate amount of hand-holding.
Not counting courses in statistics and research methods, which are required in virtually any social work PhD program, this SSW requires all PhD students to take five different courses. The University of Illinois and the University of Michigan require only three. The University of Chicago largely gives students “the flexibility to pursue their own scholarly interests,” as one would expect at the highly specialized doctoral level. Ohio State requires quite a few courses, but Ohio State is not very highly ranked, and my impression so far is that its researchers are not very visible in the profession’s literature.
None of these programs, including Ohio State’s, requires all students to take a course in the philosophy of science, like this SSW’s required S720. (See also the discussion of the epistemology requirement in support of Petition 1, above.) None requires integrative seminars like S791 and S792. Leading schools (including not only those just mentioned, but also Columbia and Berkeley) do not require all students to sit through three survey courses on theory and policy. While the SSW does not appear to have undertaken serious research to justify its approach, conversations with students suggest that most dislike these courses, and would greatly prefer to be doing focused research and developing credentials in their areas of interest. In short, there are good reasons for the SSW to change its course requirements, and therefore the subjects that must be covered by the evaluation criteria on the Rating Form.
If the SSW does continue to impose what appears to be an excessive number of survey courses on all students, the next question is why it should be necessary for the qualifying paper to provide a rambling overview of them all. This, again, is not the approach taken at leading schools. As noted in the argument supporting Petition 1 (above), it is not done that way at the University of Washington, which U.S. News & World Report ranks fifth among social work programs. At Berkeley, the qualifying exam focuses upon three narrow fields chosen by the student. At Michigan, the qualifying (“preliminary”) exam is solely oriented toward the student’s specialization. At Chicago, the qualifying exam is focused on “2 of 8 conceptual domains that inform direct practice, policy, or organizational research” and consists of “a take-home, open-book examination conducted during a 1-week period” and “a short (15-page) paper.”
As with the quality criteria (above), accountability is a serious issue with respect to the substantive criteria listed on the Rating Form. First, because the SSW does not conduct rigorous evaluation of its own practices, the qualifying paper grading process allows a reviewer to render judgment even if the student has superior knowledge of the particular area. This is an admittedly difficult problem. But it is not solved by simply forging ahead with an assumption that the reviewer nevertheless should cast such judgment. A consultative, inquisitive, or otherwise adaptive (and, let one say, respectful) evaluation style may be superior to the present one-way flow of critique from professor to student.
Assuming the reviewer does have competence to evaluate an item, the present substantive criteria are very vague. For example, looking at criterion 9, what does it mean to evaluate the qualifying paper for its “Recognition of the implications for the development and revision of theory”? The criterion seems to suffer from misplaced concreteness. As Daley et al. (2006, p. 1) admit, there is not even any agreement on what theory is. The thoughtful PhD student may sensibly decide not to follow as Daley et al. disregard cautions cited by the likes of Thyer and Payne, and proceed “Regardless of the ongoing debate about whether to value theory or not” (p. 2).
The relevance and the contours of discussions of theory, and of other substantive matters, depend upon the particulars of the student’s inquiry. Assuming the SSW continues to insist upon criteria that are not adapted to those particulars, it may still be possible to improve significantly upon the terms used on the Rating Form. For instance, it may be reasonable to replace criterion 9 with something like this: “Paper explains and applies theories presently cited in relevant literature, proposes one or more theories if the literature contains none, and/or defends a rationale for proceeding in his/her discussion without reference to or reliance on any explicit theory.”
Generally, if a rigid set of substantive criteria (regarding e.g., theory) is to be applied across the board to all qualifying papers, most if not all such criteria would benefit from expansion and clarification into something resembling a detailed and frequently updated rubric. The list of criteria should also include all criteria that are taken into account in the actual grading. For example, reviewers should not penalize students “off the books” for falling short of the page limit, or for going beyond it. If it is an important criterion, it should be stated on the list of criteria. Likewise with the minor, which sounds crucial on page 24 of the Handbook but is completely ignored on the Rating Form.
In short, students should be able to identify plainly, from the list of criteria, what they are expected to do. They should not have to guess, and should also not be directed circularly back to other text in the Handbook,as criterion 2 does explicitly and as a reviewer may believe other criteria do implicitly. There is no guarantee that all reviewers and students will find the specific Handbook text being referenced. If the criterion is that complex, it probably should be broken out into multiple criteria or elaborated in subparts within a rubric.
On the basis of the foregoing arguments, then, this petition asks the SSW to revise the evaluation criteria so that they provide a complete and appropriate guide to what is expected in the qualifying paper.
Count Certain Publications Toward the Qualifying Requirement
As noted above, at leading schools whose qualifying processes call for a paper, the expectation is that the student will submit a publishable paper. In this petition, I ask the SSW to deem certain publications sufficient to satisfy this school’s requirement for a qualifying paper.
This petition meets the test of common sense. If the goal is to produce a publishable paper, the publication of a paper tends to demonstrate satisfaction of the requirement. If a student can manage to pull together a publishable paper during his/her years of coursework, s/he should be encouraged to send it off for publication promptly, rather than wait and risk that the passage of time and the development of other scholarship will pre-empt it or render it obsolete. The student who does send an article off for publication, and who gets that article published, should not be punished for failing to withhold it until it can first be used to satisfy a qualifying requirement. If the SSW can use the qualifying process to stimulate the production of publishable articles, reflecting well on the student and the SSW, it should do so.
If this petition is approved and implemented in the Handbook, certain caveats may be in order. Presumably the Handbook should specify that such publication will be in a peer-reviewed journal. It may seem advisable to specify a list of journals that count for this purpose, supplementable upon petition. An article that has been accepted for publication but not yet published would presumably count. If the SSW requires a long qualifying paper that contains both global and focused aspects, it may decide to count the published paper only toward the portion of the requirement that the paper satisfies.
As noted in the argument supporting Petition 1 (above), students in this PhD program are almost invariably unpublished. The prospect of being able to meet at least part of the qualifying requirement by getting a paper published could have a number of positive effects. In addition to improving the credentials of school and student, such an option could help to improve doctoral student focus upon a potential dissertation topic early in the program. Students who have achieved one publication may be encouraged, and may encourage their peers, to try to achieve additional publications. Students would predictably use papers assigned in their coursework as opportunities to develop suitable publications, thereby enhancing the quality of those papers and providing an automatic means of keeping the bar for assigned papers at an appropriate level of quality. Paper assignments used for this purpose would then draw upon the professor as a first-round editor, rather than leaving students to fly blind into the qualifying stage.
I am able to cite those various benefits because I have personally experienced some of them. I did write two articles, during my years of social work coursework, that were accepted for publication and that have now been published in a major social work journal. I can testify that the experience of being published does enhance a student’s sense of competence to engage in social work research and publication. The student who achieves publication may acquire some pride in being able to add such an item to his/her vita. For some students, this kind of tangible encouragement could make the difference between remaining in the program or dropping out; for others, it will likely affect employability.
Petition 4: Discard My Reviewers’ Grades
On December 31, 2010, I submitted a qualifying paper to the three members of the committee duly created for purposes of grading that paper. In an email message to me on February 4, 2011, Dr. Adamek of the SSW informed me that the last of those three reviewers had finally returned his grade to her. As indicated in that message and the accompanying materials (attached), the reviewers unanimously agreed that my paper deserved a failing grade.
In this petition, I ask the SSW to discard the grades provided by the reviewers, on the grounds that the grading of my paper was not conducted fairly and competently. The arguments advanced in support of this petition entail critique and rebuttal of the grades and remarks provided by the reviewers.
Under the SSW’s qualifying paper procedures, the failing grade given to me means that I have used up one of my two chances for the qualifying paper. If I submit and am failed again, that is the end of my years of doctoral study in social work. The poor grading and virtually nonexistent guidance provided by my reviewers place me at substantial risk of being flunked a second time, again for reasons that may not stand up to scrutiny. Moreover, the duty to completely rewrite a 72-page paper should not be lightly imposed, and should not be imposed at all if such a duty is excessive or inappropriate under the circumstances.
Along with Dr. Adamek’s February 4 email and the grading materials that accompanied it, I have attached a February 7 email in which Dr. Adamek partly clarifies the identities of the reviewers. As shown on that email, I had asked her why the reviewers’ names were not entered in the spaces provided for their names on the Rating Form. Having selected the reviewers myself, I knew who they were; I just wanted to know which reviewer had made which remarks. Dr. Adamek did not answer that question. She seemed to think that my purpose in asking was to “follow up with the graders,” and stated that this was not “typically” done.
The reader might notice that the Rating Forms sent to me are not the original rating forms from the faculty members. They are not signed, and they appear to have been prepared by a typist, not by two very busy professors who are very unlikely to fiddle around with typing Xs next to the numbers, in the same way on both forms, instead of just circling them. I did wonder why anyone would go to this trouble of having the forms retyped, instead of just sending me scans or photocopies of the professors’ actual Rating Forms.
In her February 7 email, Dr. Adamek stated, “It is permissible to submit an outline of your new paper to the committee members for their input before you develop a full paper.” I appreciated this new information, but I found it remarkable. Why did the Handbook not say this? Why did Dr. Adamek not say it when she sent me the scores on February 4? What if I had not sent her another message that prompted her February 7 reply – would she then not have bothered telling me at all? Were there other unwritten rules I should know about – that, perhaps, the more favored students in the PhD program did know about? Why would submission of an outline be permissible on the revision, but not on the first submission? Or was it perhaps permissible the first time around, too, but nobody told me?
What Dr. Adamek did confirm unequivocally, in her February 7 email, was that Reviewer 1 submitted the January 3 Rating Form, and Reviewer 2 submitted the January 25 Rating Form. As I examined the materials in more detail, however, I discovered that this was not likely. Reviewer 2 complained that I provided “No articulation of research agenda,” whereas the person who submitted the January 25 Rating Form gave me a 3 out of 4 (“Good”) on criterion 14, regarding the research agenda. In other words, it would not have made sense for Reviewer 2 to praise my recognition of ethical issues on the research agenda, when Reviewer 2 believed that my research agenda did not exist.
I called Reviewer 2 and verified that, actually, he had submitted his Rating Form on January 3. In other words, Dr. Adamek sent the reviews to me with incorrect associations of Rating Forms to reviewer comments, and incorrectly insisted that she had done it right when I followed up to verify whether she was sure that Reviewer 1 was the one who submitted the January 3 form.
It was odd that Dr. Adamek named the January 25 reviewer as Reviewer 1, when the January 3 reviewer was not only the first to submit his grades but was also the chair of my committee. The Rating Forms prepared by the typist did appear to have been prepared at different times – using different versions of the Form, that is – so it did not seem that one typist had accidentally mixed them up. There may be other explanations, but it does seem advisable, under the unfolding circumstances, to mention that there were some irregularities here.
I have to confess, I wondered whether Dr. Adamek was deliberately trying to confuse me as to which reviewer submitted which comments. I wondered about this because I had become acquainted with another bizarre quals grading situation at Indiana University. I won’t comment on that one now, because it’s a good story and it deserves its own chance to breathe, and perhaps I will be able to give it that chance – sometime, but not now. Right now, the point is just that, as things developed with Reviewer 1 especially (below), and as I encountered these repeated indicia that Dr. Adamek was not exactly helping me to associate the reviewers with their comments, it did begin to seem that there might have been an attempt to obscure the matter.
To prevent further confusion, as shown on the attachment, I have added “Reviewer 1” and “Reviewer 2” in Courier type on the top right-hand corners of the proper Rating Forms. Dr. Adamek said that Reviewer 3 did not submit a Rating Form, and that this was “not uncommon.” So apparently the Handbook does not necessarily describe the way the process actually works. She did not explain how average scores could be computed, for purposes of the averaging calculation required by the Handbook, if not all reviewers submitted scores.
With that background, the following paragraphs proceed to critique the several reviewers’ scores and remarks.
The entire written comments provided by Reviewer 1 are as follows:
I have decided to go with my initial review of the comps – it is a fail since Woodcock did not address the criteria for a comp exam laid out in the PhD Student Handbook. The guidelines asked for one thing and Ray produced something entirely different.
I hope that in the retake he will decide to address the criteria and stay within the page limit.
To emphasize, this is all that Reviewer 1 wrote, to explain a failing grade on a 72-page doctoral qualifying paper.
It does offer a few hints. Before pursuing them, let us observe that he finds my paper entirely different from what he thinks I should have written. So he is demanding a complete rewrite.
But about those hints. For one thing, I was interested in that reference to his “initial review.” It seemed to say that he had later revisited the paper, and had entertained an alternate view in which the paper did not fail. Regrettably, just as the process did not allow me to consult with him in advance, so also it did not allow him to communicate or to engage in dialogue with me or with the SSW regarding his indecision.
This reviewer’s taciturnity says something in itself. Why would he not provide a reasonable explanation of his thinking – some remarks about specific sections, some identification of major problems, some suggestions for improvement? He is not tongue-tied. He is a professor. He knows how to talk to students. The most likely answer seems to be that his own circumstances entail some political complexities (below).
For whatever reason, politics or otherwise, his written remark suggests that he considered it necessary to choose a black-and-white solution: either pass big-time or fail big-time; nothing in between. His total score of 23 was dramatically lower than the overall average of 36, across all reviewers, that I needed to pass. At that rate, what kept him from giving me a score of 5, or 10? Maybe he felt that my paper was actually pretty good, as failing papers go; or maybe he felt that a score much lower than 23 would have put his own grading in the spotlight.
Why did the grading process allow these sorts of possibilities? Arguments provided in support of the preceding petitions in this memo have recurrently suggested that grades should be guided and constrained by a carefully constructed rubric. In other words, the grader can’t just toss out a superficial suggestion that I did a poor job on, say, criterion 2 (“Organization and integration of content as specified in guidelines”), which seems to be the focus of the one written remark that Reviewer 1 does provide (above). A suitable rubric would inquire into that. Did the paper have major sections? Did it include a References section? and so forth. With a rubric, it would not have been so easy to get away with giving me a score of zero on criterion 2.
Rubrics are not a new concept. Faculty are familiar with them. There is a reason why they are not already part of the SSW’s qualifying paper grading process. But what is that reason? Based upon the limited information available, part of the answer seems to be that accountability is too much work. This is understandable, in a sense. We are all busy. But this is a decision on someone’s career, and these are professors within a helping profession. If academic pressures reach the point where reviewers cannot take the time to do a proper job on something this important, then something needs to change.
Reviewer 1 gave me scores of zero (i.e., Unsatisfactory) on two items and scores of 1 (i.e., Marginal) on four more items. Those items were criteria 2 (“Organization and integration of content as specified in guidelines”), 5 (“Evidence of a constructively critical and objective approach to the subject matter”), 7 (“Understanding of research concepts, methods and related issues”), 8 (“Recognition of the implications for social work practice”), 11 (“Recognition of the implicit values and assumptions operating in the approach to the subject matter”), and 12 (“Ability to analyze, synthesize and evaluate the practice, theory, and research issues identified”).
The Rating Form (not to mention common sense) requires each reviewer to provide “a written justification” for each criterion rated Marginal or Unsatisfactory. There should therefore have been six distinct justifications in the written comments provided by Reviewer 1 – one for each of the six items just listed.
His page limit complaint does not address a criterion on the Rating Form. So, as shown above, he provided only one explanation. In other words, Reviewer 1 did not comply with the duty stated on the Rating Form. It is an important duty. The student would obviously appreciate specific guidance in what the reviewer considers wrong about the paper, so that s/he can make sure to get it right next time. And the reviewer might be wrong – this reviewer, especially. He has just recently come into the SSW by adoption. He does not have a social work degree. His department was simply merged into the SSW. I know more about social work than he does. So of course I would like to see his reasons for scoring so many items so poorly.
The situation seems to be that Reviewer 1 found himself in a bind. This will become clearer in connection with the discussion of Reviewer 2 (below). For now, let it suffice to say that this was the first time that Reviewer 1, a recent graduate himself, had had to grade a qualifying paper, and he appears to have looked elsewhere for guidance.
The focus turns, then, to the SSW. How did it handle this situation, in which the reviewer was inexperienced, new to the department, did not have a social work degree, and did convey, in writing, a hint of ambivalence? The SSW received this reviewer’s unexplained negative ratings and treated them as fact: it failed the student without explanation or intervention. It did not insist on the safeguard of even obtaining, never mind verifying, the required explanations for each item graded poorly, and it also did not take steps to address either the reviewer’s stated ambivalence or the student’s apparent confusion. In short, the SSW had no protections in place that would have identified a potential irregularity and would have slowed down the process, to make sure things were being done fairly and appropriately. For practical purposes, the SSW did not mind if this reviewer failed my paper, regardless of what the reasons might have been.
As noted above – to emphasize his leading role – the person whom Dr. Adamek designated Reviewer 2 should have been designated Reviewer 1. He was the chair of my committee, and he submitted his review on January 3, well before the second reviewer’s January 25 submission.
The written comments provided by Reviewer 2 are as follows:
Ray Woodcock fails to address the issues and criteria clearly laid out in the Ph.D. qualifying exam guidelines supplied by the IUPUI School of Social Work. The series of very loosely connected papers submitted for consideration, while fairly coherent and well-written, did not meet the guideline criteria. Most importantly, synthesis, critical analysis, substantive research agenda, and applicability to a Ph.D. project were lacking. Woodcock criticizes existing approaches at a surface level, misuses certain terms and concepts, and generally fails to point to anything new that his own writing or research would add to the field.
There was, in summation:
1. No synthesis of educational experience and only a vague relation of the external minor to a social work agenda.
2. An inadequate discussion of prevailing paradigms.
3. No real identification of an area of research.
4. No articulation of research agenda.
So here we have clear, confident statements of the reasons for at least some of the items on which this reviewer considered my work Marginal or Unsatisfactory. And which items were those? The question has already been answered, for the most part: as with Reviewer 1, the failed criteria were 2, 5, 7, 11 and 12. Altogether, if we include the items on which the two reviewers gave me passing marks, their grades were identical on criteria 1, 2, 4, 5, 7, 10, 11, and 12. They differed by just one point (e.g., giving me a 3 instead of a 2) on all of the others except 6 and 14. So they really saw my paper in very much the same way.
There are two ways to interpret such close agreement between reviewers. One is that brilliant minds think alike. The hypothesis here would be that the decades of study and experience that went into the social work career of Reviewer 2 ultimately left him no more able to grade my paper than Reviewer 1, who is much his junior and who has, again, no social work degree or experience. The question, in that event, is why my classmates and I are getting PhDs in social work, if we are going to be just as well trained for social work academia with a PhD in sociology or some other field.
The alternate hypothesis is that Reviewer 1 took the practical step of essentially copying the scores provided by Reviewer 2, making a few minor changes here and there. He did hint that he felt there might be another way of viewing the paper, but he was not very well positioned to defend that view. He may also have felt some political pressure to go with the flow. Several administrators and faculty members have a history of documented hostile behaviors toward me.
The problem there (in case the reader is curious) seems to be that I am willing to inquire into and, when there is no alternative means of reaching a solution, to highlight abusive practices within the SSW. This memo illustrates the point. But that story is best told elsewhere.
The relevance of that hostility, for present purposes, has to do with something Reviewer 1 told me. After I had formed this committee of reviewers, he said that he had heard things about me from others, and that as a result he might not be willing to continue with me on to the dissertation stage. I do not know whether anyone actually suggested that he would enjoy a better fit within the department if he failed my paper. Such an explicit statement would not have been necessary. He had learned that I was disfavored. One can safely say that the desire to be accepted in his new department and, naturally, to advance his career, did not give him incentives to indulge a positive view of my paper. In other words, the SSW’s arrangements for grading qualifying papers do not insure that students will advance on the basis of merit, as distinct from popularity.
Reviewers 1 and 3 had time to become aware of the grade submitted by Reviewer 2. He submitted that grade on January 3, 72 hours after I had turned in the paper. It was a long paper; it is unlikely that all three reviewers were reading it on New Year’s Day. It would be weeks before reviewers 1 and 3 would turn in their own scores. In short, there was apparently no significant committee interaction before Reviewer 2 submitted his grade. Instead, as the dominant member of the committee, Reviewer 2 seems to have decided the fate of my paper singlehandedly.
Given the Handbook’s rule that a paper cannot pass if any reviewer provides a score of less than 28, the die was cast as soon as Reviewer 2 submitted a score of 27. At that point, a dissent (that is, entry of a significantly higher score) by Reviewer 1 would not have changed the outcome. It would merely have put him in the spotlight, where his colleagues in the SSW could find fault with his grading and use that to support his dismissal from the SSW. After all, there were budgetary pressures. His department had been merged into the SSW, as I say, at the very time when he came onboard. There had been a shakeout; faculty from his old department had left; there were fears that others would have to follow.
The odds of just coincidentally producing a score of 27 are small. The written comments of Reviewer 2 certainly do not convey any whiff of ambivalence, of weighing different factors pro and con. He had a strong, simple reaction to the paper, and there is every sign that he felt quite sure, on that basis, what the outcome should be. Indeed, the written comments demonstrate that both Reviewers 1 and 2 – and, no doubt, many other reviewers as well, for other students – have treated the Rating Form as an after-the-fact elaboration of a decision based upon their general-purpose reading (or at least skimming) of students’ papers.
In other words, reviewers are free to make a global decision about the paper as a whole, and then fill in the Rating Form afterwards. It is not a question of whether a paper really deserves to fail, according to criteria carefully selected by the SSW. It is a question of whether the reviewer wants the paper to fail. Once s/he decides that, it is a simple matter to fill in the form to support the conclusion. In that case, the Rating Form may indicate what the reviewer felt was best and worst about the paper, within his/her own concept of what counts as a failing paper. But it does not indicate whether the paper deserved to fail according to criteria decided by the SSW.
It seems, then, that implementation of a detailed rubric could significantly change how qualifying papers are graded in the SSW. It would not only require more work; it would require reviewers to expose their unspoken decisions to daylight. If this sort of accountability were what the SSW wanted, this would be the practice by now. Instead, as shown in my case, the SSW has opted for a process in which the student is vulnerable to a single negative vote by any reviewer who wishes to penalize the student for any reason, and the true rationale for that vote will not be explored.
The essential fact behind this reviewer’s scores and remarks may be simply that he did not like what I wrote. I was critical of clinical social work education, and he is very much invested in clinical social work education. But if this is the real problem with my paper, a rewrite would still fail, unless I changed my arguments to conform to his beliefs.
To illustrate one alternative, if the SSW does insist upon taking a rigidly quantitative, one-shot, no-dialogue approach to the grading of the qualifying paper, the reviewer could be required to enter responses to specific questions, but not allowed to cast a vote on the fate of the paper as a whole. Where possible, the questions would be hard to fudge (e.g., “Does the paper have a References section?”). This, coupled with an enforced rule requiring persuasive explanations of failing items, would go far to discourage abuses like those that are evident in my case.
Again, it would require some work. And it may still not be the best approach. If the SSW is simply not going to insure that a quantitative evaluation is done well, then some other approach may be warranted.
As noted above, the specific criteria on which Reviewer 2 gave me a grade of Marginal or Unsatisfactory, and was therefore obliged to provide a written justification, were 2, 5, 7, 11, 12, and 14. He does not provide his comments in item-by-item form, so it is not possible to be sure what he was responding to, in any particular remark. But one can guess that this is his response to criterion 2 (“Organization and integration of content as specified in guidelines”):
“Ray Woodcock fails to address the issues and criteria clearly laid out in the . . . [Handbook]. The . . . [papers were] very loosely connected . . . . There was . . . [no] synthesis of educational experience.”
His reference to “papers,” plural, has to do with the structure of the paper I submitted. It consisted of four distinct, publishable articles. These articles contained a total of 72 pages of text (16 pages in paper 1, 18 in paper 2, 18 in paper 3, and 20 in paper 4). As such, the total page length was shorter than papers that other students have submitted and that have been approved. It was well short of the 88 pages of text found in one of those approved papers.
I presented those four articles as a single qualifying paper. They were bound together by a five-page epilogue. The first paragraph of that epilogue includes this statement:
The [qualifying] paper consists of this epilogue plus the four preceding chapters . . . . The purpose of this epilogue is to highlight some links among the articles, so as to explain where they will lead in my future research.
The fourth article addressed the doctoral minor. I will talk about it in connection with Reviewer 3 (below). The epilogue discusses each article individually, and then provides a synthesis of my educational experience by explaining how article 3 is the culmination of articles 1, 2, and 4.
Reviewer 2 does not seem to have noticed the epilogue. His comments, as just quoted, provide no indication of how the epilogue failed to provide an adequate connection among articles or failed to qualify as a synthesis of my educational experience. To the contrary, when he claims that I failed to identify an area of research, he is confirming that he did not realize I discussed precisely that topic in the epilogue.
I introduced the qualifying paper with a cover letter. The cover letter provided a list of some of the points of confusion I had encountered as I attempted to understand the Handbook. That list – a brief version of the elaboration presented in support of Petitions 1 and 2 (above) – documented a significant level of uncertainty regarding the Handbook’s vague and confusing expectations.
As with the epilogue, Reviewer 2 did not seem to notice the cover letter. His remarks provide no indication that he was familiar with its contents. It was attached to the paper, but he seems to have blown right past it.
It is possible that Reviewer 2 did take both the epilogue and cover letter into account, but for some unexplained reason decided to reject or simply disregard them. Nonetheless, he failed – more to the point, the SSW did not require him – to engage in a careful process that took full account of the materials submitted and that sought to insure that he understood those materials and that the student was treated fairly. Therefore, he was able to doom the entire paper and, moreover, to set up a situation in which the other reviewers would not need (indeed, within their political circumstances might not be able) to do anything more than join in the gang effort, adding virtually no comments or explanations beyond his own.
I say it is possible that Reviewer 2 took the epilogue and cover letter into account. But it is extremely unlikely. I contacted him on February 7, slightly more than a month after he submitted his grade. In a brief phone conversation, I asked whether he recalled that I had submitted a cover letter and epilogue. At first, he did not recall the epilogue, but then after a moment he thought that he might have seen that. It certainly did not seem to have made an impression. He did not recall the cover letter at all. I said I was sorry that he had found my paper to be so poor. He was able to respond with remarks similar to some of his written comments (above). So he did remember the paper. But, to emphasize, he had nothing whatsoever to say about the cover letter and the epilogue.
One thing is clear: for whatever reason, Reviewer 2 ignored my expression of profound uncertainty as to what the Handbook was trying to say. And the SSW, in present terms, is fine with that. There are no safeguards against this sort of thing. Dr. Adamek got the scores and just sent them right on to me. Contrary to everything that a PhD student learns about human beings in the pursuit of knowledge, the SSW’s existing procedure for grading qualifying papers treats the reviewer’s infallibility as not only unquestioned but unquestionable.
The reviewers would have been able to read the cover letter, recognize my uncertainty about what was expected, and either answer my questions about what was expected or refuse to proceed until they or someone else had resolved that uncertainty. Now that I have taken the time to explore and present a comprehensive critique of the Handbook, of course, there seems to be quite a bit that needs to be resolved. But at the time when I submitted my paper, as I say, the cover letter stated a brief list, offered in explanation of the path I had selected. It was not unreasonable to expect that list of concerns to be taken into account in deciding the paper’s merits.
It may be that Reviewer 1, unlike Reviewer 2, did read and weigh the cover letter and/or the epilogue. This could explain his apparent hesitation before finally damning the paper. He may have assumed that Reviewer 2 had weighed those documents in reaching his conclusion; it may have been inconceivable to him that a reviewer would have just skipped over some contents of a doctoral qualifying paper. Reviewer 1, himself, did not articulate any reason for rejecting or declining to address the cover letter and the epilogue, so perhaps he, too, somehow overlooked them. But even if he had noticed, read, understood, agreed with, and actually stood up for those documents vocally in protests to the other reviewers or the SSW, the Handbook provides no way in which he or anyone else could force an inquiry into the situation, much less a change in the outcome. The grading scheme – which the Handbook itself calls “a juried process” – is actually solo judgment without the possibility of appeal, by the student or even by the reviewers themselves.
Turning to criterion 5 (“Evidence of a constructively critical and objective approach to the subject matter”), the relevant written justification provided by Reviewer 2 seems to lie in his comment that “critical analysis [was] lacking” and that “Woodcock criticizes existing approaches at a surface level.” Unless these statements are intended to contradict each other, the reviewer’s contention seems to be, not that critical analysis was lacking, but that it was superficial. The existence of a latent contradiction in those statements raises a concern that the reviewer did not have a clear rationale in mind, but was concerned rather with simply aggregating a set of derogatory statements in support of a preconceived outcome. Here, as elsewhere, it would have been helpful, for purposes both of revision and of verifying the quality of such comments, if the SSW had required them to be addressed specifically to the grading criteria.
That is not the only inconsistency in this reviewer’s remarks on the subject of critical thinking. In contrast to his written statement that “critical analysis [was] lacking,” consider his grade on criterion 6 (“Scholarliness as reflected in intellectual discipline, logical consistency and critical judgment”). Note particularly that criterion’s reference to “critical judgment.” On criterion 6, this reviewer rated my paper as Excellent. The message seems to be that I was poor at critical analysis, but excellent at critical judgment. The reviewer does not explain his reasons for reaching such starkly different opinions about the critical thinking evident in my paper.
The reader might also compare this reviewer’s written complaint that the paper “criticizes existing approaches at a surface level” against his grade on criterion 4 (“Documentation and use of relevant literature and other academic and professional resources”). He felt the paper was excellent in that regard as well. So the paper’s assertions were very well documented, but they were superficial. Is the reviewer suggesting that the paper should have set out the contents of each source document at length? This would have defeated both the complaint of superficiality and the possibility of proceeding from the documented points to the larger argument being advanced. It is not clear why a writer would present a lengthy (i.e., non-superficial) recitation of what the sources have said. The reader is able to consult them for him/herself if there is doubt that the paper’s brief account reflects them accurately.
I am rarely accused of being uncritical or superficial. The present document may illustrate the point. Maybe the reviewer got a mistaken impression by doing most of his reading in the first of the four articles contained within the qualifying paper. That particular article contains a survey of certain topics. Possibly he inferred that all four articles would be like that. Otherwise, it is hard to see how the qualifying paper is either superficial or uncritical.
Perhaps what is happening here is that people who have spent years grading papers tend to develop a stock-in-trade of rarely challenged clichés that roll off the tip of the pen at a moment’s notice. But I am a PhD student. I should be thinking critically about the pedagogy of social work. It does not take much exposure to Foucault to stimulate concerns about the inherently judgmental, potentially damaging, and frequently mistaken conclusions reached in hasty assessment, whether it is done in clinic or classroom. The people with whom social workers interact deserve better.
To continue down the list, the reviewer’s only remark that seems relevant to his failing grade on criterion 7 (“Understanding of research concepts, methods and related issues”) is that “Woodcock . . . misuses certain terms and concepts.” But don’t we all? Again, without a rubric, the Handbook provides no indication of what would be required for a failing grade on this criterion. Is one mistake enough? Generally, failure calls for much more than “certain” or occasional errors – not that I am aware of any in my paper, obviously, else I would have corrected them prior to submission. If this criticism were to warrant a failing grade, it would have to show that the paper was plagued with egregious blunders.
Or perhaps I err in thinking that the reviewer intended his finding on criterion 7 to be supported by the language just quoted. The criterion actually seems to be asking about errors with respect to research concepts and methods in particular. The reviewer says nothing at all about that particular kind of error.
The written remarks provided by Reviewer 2 are likewise silent on criterion 8 (“Recognition of the implications for social work practice”), unless perchance this is what he means to address with his statement, “Woodcock . . . generally fails to point to anything new that his own writing or research would add to the field.” It is not clear why he offered this criticism. None of the criteria call for a demonstration of how my writing or research would add something new to the field. The reviewer seems to be thinking of the dissertation stage. If my peers are not expected to propose novel research at the qualifying stage, it is not clear why the SSW is enforcing his expectation that I do so. Moreover, this criticism is puzzling. The reviewer gave me a score of 3 (Good) on criterion 3 (“Creativity, imagination and insight with respect to the presentation of ideas”). His view seems to be that my work in this paper is creative and imaginative, but completely lacking in novelty. I have no idea how to make sense of that.
As far as I can tell, Reviewer 2 offers no comments pertaining to criterion 11 (“Recognition of the implicit values and assumptions operating in the approach to the subject matter”). On criterion 12 (“Ability to analyze, synthesize and evaluate the practice, theory, and research issues identified”), all he has to say is that synthesis was lacking. This unsupported claim is vulnerable to the kinds of rebuttals presented (above) with respect to his remarks about critical analysis, as well as to the problems with the Handbook’s concept of synthesis as explored in support of Petition 1 (above). On criterion 14 (“Recognition of appropriate ethical issues related to the identified research agenda, especially those involving human subjects”), his sense that my treatment was Marginal differed notably from Reviewer 1’s belief that my paper was Good. I have not decided whether my doctoral research will entail ethical issues pertaining to human subjects, and the Handbook does not require any such decision at the qualifying stage.
Reviewer 2’s written comments (above) call for “applicability to a Ph.D. project.” Neither the text of the Handbook nor the Rating Form require anything pertaining to a project. This appears to be another instance in which the reviewer confuses the qualifying stage with the dissertation proposal.
As a demonstration of some of the problems identified in the analysis supporting Petition 1 (above), Reviewer 2 imposes several demands that do appear on pages 23-25 of the Handbook but that are not required by the Rating Form. These include his objections that I provided “only a vague relation of the external minor to a social work agenda,” “An inadequate discussion of prevailing paradigms,” and “No articulation of research agenda.” If these objections have not already been rebutted and/or undermined by remarks presented in support of this and earlier petitions within this memo (including his neglect of the epilogue), suffice it to point out again that the paragraph at the bottom of Handbook page 23 expects a “general,” “emerging” research agenda, not a finalized one.
As noted above, Reviewer 3 did not supply a Rating Form. We have only his written remarks, which, in their entirety, are as follows:
I am in full agreement for a grade of unsatisfactory for this submission. Although Ray has clearly spent a considerable amount of time crafting the document(s) and uses a vast pool of words and phrases, it lacks an equally vast amount of cohesion and depth. In particular, on the section on Leisure, the use of the Oxford English dictionary is highly inappropriate while also failing to cite/critique some of the most established researchers on the leisure-work dichotomy is troubling.
It will take only a slight sense of irony to observe the absence of cohesion and depth in these remarks. Somehow, the use of the OED is believed to constitute an instance “in particular” of a lack of cohesion and depth. Whatever that is supposed to mean, the reviewer does not seem to notice that I critiqued the use of dictionaries to define leisure. Apparently he considers it “highly inappropriate” to cite dictionaries even for illustrations. The basis for that view is not stated.
The reviewer states his agreement with a failing grade for “this submission.” His comments indicate, however, that he is referring only to the fourth article within my qualifying paper. In other words, he has failed to read the qualifying paper as a whole and provide a grade upon the whole. His opinion about one portion that I targeted specifically toward the minor is not dispositive of the question of whether the entire “submission” fails.
This reviewer – another junior faculty member who is plainly aware of and following the lead of the others – fails to specify the gap in my manuscript that would have called for citation to (never mind critique of) certain other researchers whom he declines to name. (Evidently the student is supposed to guess which researchers he has in mind, for purposes of a rewrite.) He posits the existence of a “leisure-work dichotomy,” apparently not recognizing that the sources I cite question the viability of that dichotomy. He makes no comments at all pertaining to the substance of the first half of the paper, nor to the paper’s central theme.
I waited five weeks for this reviewer to pen two poorly written sentences that provide no meaningful guidance and evince no comprehension of the paper’s thesis. This review is defective to the point of irresponsibility.
Remarkably, the SSW has accepted this reviewer’s analysis, too, at face value. That is remarkable not only because of the deplorable quality of what he wrote, but because of the facts that he neglected to mention. Going beyond other students whose papers have been accepted, I submitted an entire 20-page paper on the minor aspect alone. I also incorporated considerations relevant to the minor in the third paper. His claim is essentially that a publishable paper, which he does not seem to have read very carefully, is insufficient to meet the requirements of the Handbook, though he does not specify which such requirement was left unmet. Again, a questioning person at the SSW, possessed of a reasonable concern for students’ career progress, might have paused for a moment to look into this before endorsing it.
In overall summary, the Rating Forms and written comments submitted by these three reviewers are rather incredible in their incompleteness, indefensibility, and in some instances outright incoherence. This is not a close case, where at least one reviewer would have provided a carefully reasoned critique with clear reference to specific problems, full explanations of failing grades, and viable suggestions for revision. It certainly is not a case where all three reviewers did their jobs well. Accordingly, this petition asks the SSW to discard the grades provided by the reviewers, on the grounds that the grading of my paper was not conducted fairly and competently.
Petition 5: Deem My Qualifying Requirement Satisfied
In this petition, I ask the SSW to decide that I have satisfied the purpose of the qualifying requirement, and that I should therefore be deemed to have met that requirement. The following argument presents the support for this petition with respect to the major and minor areas of study.
Social Work Major
The preceding Petitions demonstrate that the SSW’s qualifying paper requirement is in disarray, to the point of leaving students vulnerable to confusing and potentially abusive instructions and treatment. My case illustrates some of the abuses that can take place.
This should not be allowed to continue. The SSW is in a university. As such, it is always at risk of indulging in university-style politics, and of indulging in the kinds of abuses that sometimes take place in universities. But social work is a profession based upon values, and what has been happening here is not consistent with those values.
The purpose of a qualifying requirement, as implemented at leading universities, is essentially to determine whether the student is prepared to advance to the dissertation phase. This question is not difficult to answer affirmatively in my case.
Leading schools that require a paper in their qualifying processes require it to be publishable indeed. As shown above, I have been flunked because I did not produce a paper that would have been unpublishable on grounds of excessive length, useless generality, and vague purpose.
Before writing my paper, I reviewed the conflicting and in some regards impossible expectations stated in the relevant pages of the Handbook. On the basis of my review, on which I was not able to obtain faculty input, I submitted a publishable paper that combined breadth and focus. These seemed to be the Handbook’sprincipal expectations.
I notified my reviewers of my reasons for choosing the format I chose. I pointed out specific parts of the Handbook’sinstructions that I found confusing. The reviewers could have resolved my confusion; they could have postponed grading until that confusion was resolved. Instead, they chose to ignore what I sent them. This does not justify punishing me.
Before undertaking this qualifying assignment, I had already exceeded the requirements of producing a publishable paper. As noted in defense of Petition 3 (above), I have published two papers in a major social work journal. Moreover, I have submitted two of the four articles contained in my qualifying paper to peer-reviewed journals, and the editors of those journals have sent those articles out for review.
I came to the SSW as a University Fellow in recognition of my academic capabilities, and I have demonstrated those capabilities by contributing to the profession’s store of knowledge. There are no plausible reasons to continue to prevent me from proceeding to the dissertation phase.
According to the Handbook (p. 6), “The External Minor is intended to provide students with both an interdisciplinary perspective and a specialized focus for their research. It is intended to broaden and enrich the student’s locus of inquiry.” Breadth is not an issue in my case. In addition to my present minor in leisure research, I have graduate degrees in law and business.
Other schools whose materials I have surveyed in the process of preparing this memo do not even mention the minor in connection with their qualifying requirement. Here, by contrast (as noted above), the SSW tries to make the minor into a part of the major field. In doing so, it exceeds the purpose of the minor as contemplated by the university’s Academic Bulletin.
Remarks made by some faculty in the SSW seem to link this overuse of the minor to an infatuation with the University of Michigan’s double-major PhD program. (Michigan is the alma mater of a couple of influential members on the SSW’s faculty.) Michigan’s program makes sense. But Michigan’s approach is not typical among social work PhD programs, and Indiana University has not yet adopted it. A doctoral minor – basically, an obligation to take a few courses in another field – cannot play the role of a PhD major. The attempt to create an ersatz Michigan program based upon the minor yields an overstatement of the minor’s role at the expense of the major, with unnecessary complications for the student at the qualifying stage.
Even if the Handbook’s expectation for the minor were viable – that it will provide “a specialized focus” for the social work PhD student’s research – I would have met it. I am admitted to the bar in two states and have years of experience in legal practice. The influence of this education and training is visible in the two social work journal articles that I have already published.
If it were necessary to ignore the intellectual and experiential breadth and depth provided by my legal training and experience, and their demonstrated relevance to social work, I would still have exceeded the minor requirement with respect to my present minor in leisure research. I have not merely taken the requisite four or five minor courses and submitted a qualifying paper that provides the requested information with respect to that minor. I have, in fact, completed the coursework for a PhD in that minor field. And I did not just include some remarks about the minor in my qualifying paper: I submitted an entire publishable article on the minor topic by itself. In addition, I have published an article in a peer-reviewed journal in that field.
These considerations support the purpose of this petition, which asks the SSW to decide that I have satisfied the purpose of the qualifying requirement, and that I should therefore be deemed to have met that requirement.
 E.g., http://grad-schools.usnews.rankingsandreviews.com/best-graduate-schools/top-social-work-schools/rankings. Note: some other schools in this region that are not mentioned here (e.g., Washington University in St. Louis, Case Western) do not seem to provide the relevant information on their websites.
 See http://socialwelfare.berkeley.edu/AcademicPrograms/PhDHandbook.shtml (pp. 6, 34-38).
 Daley, J. G., Peters, J., Taylor, R., Hanson, V., & Hill, D. (2006). Theory discussion in social work journals: A preliminary study. Advances in Social Work, 7(1), 1-19.
 The journal is Families in Society (FIS). My articles appear in FIS 89(4) (2008) and 92(1) (2011). FIS is commonly considered one of the top social work journals. See e.g., Sellers, Mathiesen, Perry, & Smith (2009), Journal of Social Work Education, 40(1), 143-160.