Grading of a Qualifying Paper, Indiana Style

This post is one in a series that examines my experience as a PhD student in the Department of Recreation, Parks, and Tourism Studies (RPTS) at Indiana University (IU).  Starting with an introductory post, this series provides information about the termination of my participation in that doctoral program.

The central event in that termination was the flunking of my doctoral qualifying paper.  The preceding post, and the ones before it, convey a sense of what a qualifying paper is, and otherwise frame the situation.  Now, in this post, I turn to the actual grading of that paper.

As a preliminary note, some may consider it unfair that this post would refer to “Indiana-style” grading, as distinct from grading within RPTS.  Why criticize the whole university for the failings of just one department?  The answer is that the whole university was implicated.  Specifically, the university’s processes for correcting these sorts of abuses was broken.  A student could easily have similar experiences in any number of departments, and could find, as I had found, that the university as a whole offered no recourse.  That conclusion is supported by extensive personal experience on both of the university’s two largest campuses, as indicated in a later post in this series.

As in the preceding post, some readers may find that the conclusion offered here is obvious within the early paragraphs of this post.  In this case, the conclusion I am putting forth is that the grading of my qualifying paper was corrupt.  Readers who do not need the full treatment in order to reach that conclusion are welcome to move along at any time to the next post in the series.  Of course, even those who share my conclusion may still find it worthwhile to read what is written here.

Dr. Ruth V. Russell was the chair of my qualifying committee in RPTS.  On May 3, 2010, Ruth sent me an email reading as follows:

Hi Ray,

Here are the necessary parameters for the quals paper:

* forward new thinking, or old thinking reposited

* broadly focused on leisure, as in a new summary (the point of quals in the first place)

* grounded in philosophic and/or research literature

* with suggestions for research verification

* submitted for publication to either the Journal of Leisure Research, Leisure Sciences, or Leisure Studies

Have fun!

Ruth

On June 30, 2010, I submitted copies of my qualifying paper to Ruth and also to Marieke Van Puymbroeck of RPTS and to Margaret Adamek of social work.  Marieke was the second grader in my RPTS major and Margaret was my advisor in social work, which was my doctoral minor for purposes of the RPTS PhD major.

On September 4, 2010, Alan Ewert sent me a letter.  His letter purported to convey the grades and comments submitted by Ruth, Marieke, and Margaret.  It was not clear how or why Alan was allowed to get involved.  He had been involved in previous retaliatory actions against me.  I had made a conscious point of switching away from him, choosing Ruth as my doctoral advisor.  As hinted in the foregoing email message, Ruth took a positive and encouraging approach to her students, and had chaired a number of doctoral students’ committees in recent years.

Alan did not forward the original grading sheets prepared by Ruth, Marieke, and Margaret.  Instead, he sent me a document that appeared to have been prepared by a single typist.  The first grader’s remarks ended on page 2, and the second grader’s remarks took up immediately at that point, not on a separate sheet.  There was no explanation for why it would have been necessary or appropriate for someone to reformulate the grading information, rather than sending me photocopies of what my graders actually submitted.

I belabor these details for a reason.  I will explain shortly.

Collusion Between Graders

On the document that Alan sent me, each grader’s scores and remarks were presented within a form that roughly matched the criteria stated by Ruth (above).  That is, there was a row for the “forwards new thinking” criterion, another row for the “broadly focuses on leisure” criterion, and so forth.  Each row contained three columns:  one for the criterion, one for the score, and one for comments.  The document stated that the score should range from zero to 4, where a passing score would be 2 (“marginally met the requirement”).

Here is the completed grading form, attributed to Ruth, that Alan sent to me:

Grading Form Attributed to Ruth Russell

Let me be clear about that.  Ruth invented the grading criteria (e.g., “forwards new thinking”); Ruth designed a form whose instructions required that graders should “add comments where requested”; but then, if I was to believe what Alan was telling me, Ruth failed to supply the comments requested by her own form — even when she was supposedly giving the student a failing grade.  That seemed odd.

I think Ruth’s students would find it odd too.  It may be interesting to view an example of the kind of feedback that one could expect from her — the kind of feedback that she had been giving me for years.  By way of contrast, here is a page from a paper graded by both Alan and Ruth, in a course they co-taught.  Alan’s comment on this page is the red one at upper left.

Alan was not always this bad.  But he was almost never as discursive as Ruth was.  This graded page shows the kind of extensive and thoughtful feedback that she tended to provide, throughout the papers that I submitted to her.  She was a hard worker.

In other words, it seemed unlikely that the nearly blank grading form shown above conveyed grades that were actually decided by Ruth Russell.  It appeared, in other words, that someone was fraudulently purporting to provide grades that did not actually come from the chair of my qualifying committee.

Now, that may seem farfetched.  And yet, for a number of reasons, it appears to be the most plausible conclusion.  It will take a bit of time to unfold those reasons.  But by the end, I believe most readers will have serious reservations about the grading process to which I was subjected.

First, there is the question of what happened to Ruth.  Why couldn’t I just call or email her and ask what was going on?  That is a good question.  It will have to wait for a later post, where I look at it more carefully.  For the moment, suffice it to say that I was informed that she was no longer available.  I was supposed to believe that the chair of my qualifying committee graded my paper; yet she had mysteriously and completely vanished.  For my purposes, that is.  She was in touch with other people, but she was no longer in touch with me.

There is also the question of who would have wanted to inject a fraudulent failing grade.  I told Alan, based on additional comments (below), that it looked like his grading.  He was not always brief, negative, and dismissive, but there certainly was a tendency in that direction.  He denied that he had graded my quals paper, pretending to be Ruth.  He did acknowledge, though, that someone else, other than Ruth, might have been involved in her part of the grading.

It is not so difficult to answer the question of why someone would have wanted to give me a failing grade, where Ruth herself may not have been inclined to do so.  A previous post examines some factors that may have contributed to a punitive mindset within the RPTS department.

For such reasons, the following discussion does not assume that Ruth Russell actually graded my paper.  Rather, the discussion refers to the person who provided the foregoing scores as simply Grader 1.

Now, let us compare the scores submitted by Grader 2 — who, I think, probably was Marieke.  By interesting coincidence, the scores from Grader 2 were almost identical to those from Grader 1.  They differed only on the third item, where Grader 2 gave me a 2 rather than a 1 for the use of philosophical and/or research literature.  Like Grader 1, Grader 2 also provided no line-by-line comments to explain these grades.

Each grader did provide a short paragraph following the grading chart.  First, Alan’s letter presented the following remarks from Grader 1:

After moving administrative mountains to meet Ray’s request for an alternative form to the written qualifying exam, I am deeply disappointed in his submission.  The paper carries a sarcastic tone, makes fun of academic pursuits, and in the second half makes absolutely no coherent argument(s).  The essential assignment was for a “refereed journal submission” and I feel embarrassed by this.  While I know Ray’s potential is for excellent writing, this manuscript, with a few clever turns of phrase not-withstanding, satisfies nothing of the assignment, and masks his ability.  I honestly can’t decide if he is eligible for a re-write attempt.

Notice the bombastic and belittling rhetoric:  “absolutely no coherent argument(s)”; “a few clever turns of phrase not-withstanding”; “satisfies nothing of the assignment.”  That is not Ruth.  The remarks admit that Ruth took special efforts (not remotely amounting to “administrative mountains,” but nonetheless requiring her to go out of her way) to set up an experimental qualifying process in which I would “have fun” (see above).  A professor with an established record of supporting students, who had made a special effort to encourage an unorthodox paper, was not likely to simply write me off after my first try.

The person who wrote those words apparently did not even bother to read Ruth’s grading criteria.  That is, this grader gave me scores of 1.  The instructions on Ruth’s grading form made clear that a score of 1 (as distinct from a score of 0) meant that the submission failed but that “a revision of the work should be allowed.”  So, having already made that decision, why would Ruth write, here, that she “honestly can’t decide if he is eligible for a re-write attempt”?

Alan also provided remarks from Grader 2:

Recognizing that I agreed to serve on this committee to see how Ruth would handle such a non-conventional qualifying exam, I had hoped that this paper would in fact, be useful for the field. However, I feel that it is entirely sarcastic, mocking of academia and the field of leisure behavior, and in fact, not useful. While critical examinations are welcome if they are grounded in existing literature, research, and offer suggestions for growth, this examination is critical (although the arguments focus more on sarcasm than substantial information) without any real recommendations. The paper seems to be a joke to him, and I am embarrassed that it was submitted for review.

Hmm.  Does any of that sound familiar?  Notice, in particular, that the second sentence, in both of these two quotations, contains almost exactly the same complaints:  sarcastic, mocking, and useless/incoherent.  Those complaints are even presented in the same order.  In both cases, there is the further claim that the writer is “embarrassed.”  Isn’t it interesting that two different people would arrive at just those words, in just that sequence?

Well, no, not at all, according to current RPTS department chair Bryan McCormick.  In a letter discussed in the first post in this series, Bryan claimed that this highly similar phrasing just proved that both graders had the same view of my paper.  Imagine:  among all of the things that a reviewer could say about a 7,000-word document, both of these independent thinkers, from two different generations, came to almost exactly the same thoughts.  If we take Bryan at face value, he finds it encouraging that faculty in his department repeat each other.

But can Bryan really believe that?  Consider this scenario:  a teacher gives her students an opportunity for extra credit.  Two students decide to pursue this opportunity.  The extra-credit assignment requires each student to read an article and then write a review of it in his/her own words, making it as long or short as s/he chooses.  Each student is to work independently, not collaborating or sharing notes with the other student.  A few days later, the two students hand in their assignments.  The teacher sees that the reviews are of almost identical length (100 words for Grader 1, 111 words for Grader 2).  The teacher also observes that, in their characterizations of the assigned reading, those two students communicate several of the same thoughts, using the same words, in the same sequence.  Shouldn’t a capable teacher be concerned about the possibility of cheating?

In fact, there is no mystery here.  Marieke Van Puymbroeck was up for tenure at exactly this time.  That is, senior professors were in the process of deciding whether she would be invited to become a permanent member of the faculty.  People up for tenure are notoriously eager to show that they are not a risk or a threat to anyone.  They become ultra-conformists.  So it was not actually necessary for Grader 1 (or anyone else) to coerce Marieke into writing the same things in her review.  All that was necessary was for Marieke to get a glimpse of what Grader 1 wrote — and there were no protections in place to insure that she would not get any such clue.  Regardless of whether Marieke knew or suspected that Ruth was not actually Grader 1, a person in Marieke’s position would be strongly tempted to echo whatever the senior faculty member seemed to be saying.  And the striking similarities between the remarks and scores submitted by the two graders strongly suggest that this is exactly what happened.

I sent letters pointing out these facts to Bryan and, before him, to interim RPTS department chair Dave Compton.  Neither replied.

Bias

The foregoing remarks have raised the problem of unethical collaboration or conspiracy in a false or misleading grading scheme, subsequently covered up by senior departmental figures.  There is a second problem.  The criticisms provided by the graders are of such poor and uninformed quality as to indicate prejudgment.  In other words, there seems to have been a prior decision that my paper would fail; and for that purpose, the less said, the better.

What I received from Alan in this case was like the story told by one of my RPTS PhD classmates, who said he spent a huge amount of time on a long and difficult paper, submitted it for grading, and got it back from the professor with just one mark.  At the bottom the final page, he saw one letter:  B.  No explanations, and certainly no praise.  Just a grade of B for all his hard work.  He was being punished.  But maybe he should have been thankful.  That punishment was much gentler than that experienced by another of my doctoral classmates in the IU parks & rec program.  That student’s advisor simply stopped talking to him for an entire year.  These professors actually figured they could get away with this stuff.  And I guess they figured right.  At IU, they could.

A closer look at the few comments provided by Graders 1 and 2 reveals a number of ways in which the graders were grasping at straws to justify indefensible conclusions.  Consider, for example, the belittling tone of Grader 1, quoted above, with its hostility toward sarcasm, making fun, and using “clever turns of phrase.”  The clear impression is that Grader 1 — supposedly Ruth Russell — was a serious individual who had no use for this sort of thing.  There certainly are professors in RPTS who take themselves very seriously.  But I did not choose those professors for my qualifying committee.  For one thing, good leisure literature is (as one would expect) not hostile toward humor and entertainment.  Ruth, in particular, had said this, in one email to me:  “I am fascinated about the idea of writing a book with you.  I find your writing to be fun to read – a sliver of humor supports its intellect.”  In another communication, she encouraged me to consider submitting a novel as my dissertation.  A work of fiction!  The pompous individual who played the role of Grader 1 does not seem to have shared Ruth’s outlook toward my qualifying paper, as developed with me over a long period of discussion.

As shown above, the graders accused me, in particular, of mocking academic pursuits.  They did not say what, specifically, they considered to be mocking.  Perhaps they were referring to the part where I cited research indicating that the the field of leisure studies seemed to be closed in upon itself, getting little attention from and not having much impact on the outside world.  It is troubling to observe that some professors in RPTS consider a new PhD to be “mocking” when he cites the field’s own authorities to demonstrate a need for change.

Under such circumstances, taking oneself too seriously appeared to be part of the problem.  What seemed to be called for was, rather, a bit more self-scrutiny and openness to critique within the field.  I provided that.  To the self-satisfied, what I wrote may have been uncomfortable; if so, that was not new.  As noted above, I had apparently been making professors uncomfortable with my willingness to advance heterodox perspectives, long before this qualifying paper episode.

The graders didn’t seem too eager to question themselves in this instance.  But a critical thinker might ask:  if I were really mocking academia and the leisure field — if I thought it was ridiculous — why would I be spending years to get an advanced degree and become a professor in it?  The situation appeared to be, rather, that the graders were echoing Bryan McCormick’s view that PhD students in IU’s parks & rec program needed to realize that the professors were right and that they, the students, were wrong — and therefore that a critique of any form, serious or humorous, would generally be unwelcome.  If they hadn’t chosen to complain that I had written something entertaining, they would apparently have complained that what I wrote was undesirable for some other equally specious reason.

Perhaps the most remarkable thing is that neither of the two RPTS graders even bothered to critique my paper’s substance.  Their remarks indicate that they flunked it because they didn’t like its tone.  The grading that I received was evidently that of a person who doesn’t really care what the paper contends.

It is worth comparing Graders 1 and 2 against the third grader.  Grader 3 was Margaret Adamek, chair of the social work PhD program.  Margaret was, by this point, committed to the effort to have me removed from the university.  Predictably, she flunked my paper too.  Yet she did so without baldly copying over the scores and comments provided by the other graders.  She also did provide line-by-line remarks on the form that the other graders left blank.

Margaret may have seen that it was unnecessary for her to flunk my paper.  That question was already decided by Graders 1 and 2.  For whatever reason, she gave me at least marginally passing scores on all items except the one regarding suggestions for research verification.  This may seem to imply that she passed my paper, contrary to what I just said; but despite passing scores on four out of five items, she checked the box marked “do not advance to scheduling the oral exam.”

Along with scores of 2 on most other items, Margaret gave me a 3 out of 4 on the “grounded in philosophical and/or research literature” item, observing that “There are nearly 130 citations so the paper is well-documented with both recent and seminal work.”  To be sure, she wasn’t qualified to make that observation.  She had no doctoral-level training in parks & rec.  I was never entirely sure why a minor advisor could vote on the suitability of a qualifying paper, in a field she knew virtually nothing about.  Even so, her remark does raise the question of how the first two graders could have concluded that the paper was useless or embarrassing.  One might reasonably ask what kind of instructor would ridicule a paper in which a student has labored to support his arguments with 130 citations.

As Ruth required (above), I did submit my qualifying paper to the Leisure Sciences journal.  The editors of that journal did not decide to publish my paper.  Indeed, they did not even send it out for review.  This was not a surprise, and it was not required by the assignment.  The purpose of the assignment was not to write the kind of relatively safe piece that such a journal would publish.  It was to put together something respectable that would push the boundaries.  As Ruth stated in the email shown at the top of this post, her first command was to reposit old thinking or put forward new thinking.  This, she indicated there, was different from “the point of quals,” which in her view was to provide a broad summary.

My paper succeeded in the mission of pushing the boundaries.  In a reply that was not at all demeaning, the editors of Leisure Sciences said this:

Your paper was certainly enjoyable to read, and it contained a wide variety of insights about the concept of leisure and the field of leisure studies.  But, the manuscript does not yet seem to be fully developed as a publishable scholarly paper.

As Ruth told her students, the next step after a rejection at one journal is to revise the paper and submit it elsewhere.  She had experienced rejection too.  If RPTS had not curtailed my prospects of a PhD in leisure studies, I would by now have revised and resubmitted that paper; it would likely have been published somewhere.

To recap, this post suggests that my qualifying paper received a failing grade in a grading process that was tainted by unethical collusion and skewed by inappropriate prejudgment.  Evidence for collusion emerges in a number of similarities in the remarks submitted by major graders, where Grader 2 had powerful incentives to copy Grader 1.  Evidence for prejudgment arises from contrasts between the major graders, on one hand, and the minor grader and journal editors, on the other.  Evidence for prejudgment also arises from lack of support for the major graders’ criticisms, and from indications that those graders did not even seem to know whether the submitted paper was actually consistent with Ruth’s expectations.

Meanwhile, this post has proceeded in something of a vacuum where Ruth is concerned.  As noted earlier, that’s because she, herself, proceeded into something of a vacuum.  The next post turns to the mystery of what happened to her.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s