Patrick West

Recorded-Voice Formative Assessment for Creative Writing Students: A Case Study

Abstract:

Assessment has been called the most important element of pedagogy. The significant personal investment that Creative Writing students often have in their submissions introduces another factor for consideration. For this cohort, if not more generally, written feedback would appear to be inferior to assessment provided in recorded-voice form. Analysis of the results of a small-scale case study conducted by the author supports this claim. A proposition by Susan Sontag, that hearing provides a better way of knowing than seeing, generates a theoretical expression of the same idea. Sontag links knowing through hearing to the notion of responsiveness: hearing is more personal than seeing. Developers of effective assessment packages should take this into account. My study also underscored the importance of process for Creative Writing students in the assessment situation.

Teaching and learning are activities with many complex and interwoven threads. Effective assessment packages pull all the threads together for the maximum pedagogic benefit of students. You cannot hope to generate a tight fit between course objectives and student outcomes if the assessment regime is poorly conceived. John Biggs argues that "assessment is the senior partner in learning and teaching. Get it wrong, and the rest collapses." (Biggs 160). Paul Ramsden adopts a similar position: "the underlying reasons for students' failure to learn effectively can probably be found in the ways that teachers and other educators currently think about teaching and assessment" (Ramsden 15). The fact that assessment can also be very traumatic for students personally is another significant dimension of the problem - one that deserves greater consideration. University teachers have an ethical and pastoral responsibility to reflect frequently on their chosen assessment packages. Almost twenty years after Peggy Nightingale wrote about an educational experiment with recorded feedback and pronounced herself "delighted with the results", my article revisits the territory in the different context of Creative Writing pedagogy (Nightingale 38).

The discipline of Creative Writing is a special case with respect to issues of assessment. According to Jeri Kroll, "here students produce and are therefore in charge of the content to some degree; they are personally involved in what comes from their own imaginations" (Kroll). This observation captures two elements that set the typical Creative Writing student apart from his/her peers in other areas of study (paradigmatically the Sciences). The vexed question of the relationship between objective assessment criteria and insistently subjective artistic productions cannot be completely disentangled from a consideration of the personal angst often caused when red pen comes into contact with printed page of poem, short story or film script. My intention in this article is nevertheless very specific: I plan to engage with Kroll's notion of "personal involvement" and the question of what it means for formative assessment. The focus will be on the problem of the university teacher's responsiveness to Creative Writing students as (mainly) young men and women with what sometimes amounts to an overweening personal stake in their imaginative submissions.

My interest in experimenting with a different method of providing feedback to my Creative Writing students can be traced to nothing less inimical to precise definition than a vague sense of dissatisfaction with the written mode of assessment. The exact nature of the problem eluded me at first, nor could its magnitude be accurately quantified. If pressed at the time, I suppose I would have said that workload constraints were starting to mean that written comments were no longer a practical instrument for communicating a kindly, sensitive and productive appreciation of a student's creative efforts. Only the process of conducting the empirical research itself showed me exactly what issues should be central to my inquiry - not to mention just how much, in fact, they were matters of significant concern to my students. My case study transformed into a thesis my hypothesis that one mode of formative assessment was better than another. I do not resile from the observation that the small sample of students involved (ten) means that further empirical work is needed in order to confirm or qualify the findings of what was to some extent only a pilot investigation.

In a homage to the European writer and philosopher Elias Canetti, Susan Sontag argues that "Canetti equates knowing with hearing, and hearing with hearing everything and still being able to respond" (Sontag 196). In this biographical observation, hearing is privileged over seeing; the ear is given precedence over the eye. As Sontag implies, "restating the archaic gap between Hebrew as opposed to Greek culture, ear culture as opposed to eye culture" means that we have effectively returned to an ancient hierarchy (196). Canetti's ethics of the ear provides valuable theoretical scaffolding for my work.

"The voice for Canetti stands for irrefutable presence. To treat someone as a voice is to grant authority to that person; to affirm that one hears means that one hears what must be heard" (Sontag 196-7). All that requires to be added to Sontag's formulation is an awareness on the part of the speaker (the assessor) of the added responsibility that comes with this power and potential. Crucial to the role of being an educator engaged with assessment, is a willingness to speak no less "responsively and attentively" than one wishes to be heard: this is how I interpret the force of Sontag's point about hearing "what must be heard" (Sontag 197). The creative products of our students on the page deserve to be heard as voices: let us risk the only seeming absurdity of suggesting a reading with the ear not the eye. Excellent formative assessment is about talking while carefully listening, and about carefully listening while talking - at the very least, feeling genuinely able to talk. Neither teacher nor student should have superiority over the other in this pedagogic arrangement.

The course Effective Writing is offered in the School of Arts at my university in the first semester of each year. I have been the convenor of the course in 2001 and 2002. Approximately 250 undergraduate students (mainly first years) are given the opportunity to experiment with a variety of fiction and non-fiction genres such as the academic essay, the critical review, the short story, and the film script. The teaching team places a premium on developing generic writing skills with broad application. Staff-student contact happens in a one-hour lecture and a two-hour workshop every week.

Assessment is by three folios: a "Creativity & Writing" Folio (Fiction and/or Non-Fiction Engagement); a "Writing for Other Spaces" Folio (Performance Engagement); and finally, a "Writing About Writing" Folio (Non-Fiction and Fiction Engagement). The focus of the assessment criteria is the ability to "engage with the topic." Each folio totals 1500 words; at least two discrete pieces of writing are required. The value of the folios for final course-grading purposes is 25%, 35% and 40%. We encourage students to ask why this percentage increases while the word length remains constant. This detail of the assessment package is intended to communicate the proposition that a key element of developing professionalism and expertise as a writer is a willingness to engage with exponential intensity in successive writing tasks. According to Marlene Scardamalia and Carl Bereiter, "expert writers generally are found to work harder at the same assigned tasks than nonexperts, engaging in more planning and problem solving, more revision of goals and methods, and in general more agonizing over the task" (Scardamalia & Bereiter 172) There can be little doubt that a serious writer's notions of what is possible in terms of excellence and imagination expand as he/she grows in experience. Where better to make this point to students than embedded in the actual assessment package, where it will count for the most?

It was during the 2001 offering of Effective Writing that I did the empirical research for the current project. I decided to align my small-scale case study with the feedback protocols of the second item of assessment: "Writing for Other Spaces" Folio (Performance Engagement). For this folio, all work submitted would fall into the fiction category: for the most part, film scripts, poetry, and performance poetry (an example, one might say, of voices on the page). There were nine tutorials in the course that year, of which I conducted three. I only needed to mention what I was doing in two of my classes to obtain ten student volunteers. I marked their folios first, returning them in the usual fashion via the assignment return boxes. All that I wrote on paper was the mark and the grade. The rest of the feedback was provided on cassette tapes: one per student. (Throughout this project, I have attempted to disarticulate the summative dimension of assessment from the formative.) The students would collect their cassette tapes along with their folios. I had asked them to listen to my feedback and then complete a short questionnaire.

The preamble to the questionnaire was designed to direct the participants' attention to the difference between written and spoken feedback in the context of assessment items in a Creative Writing course: "in answering the following questions, I would like you to be thinking about your experiences of receiving assignment feedback for Assessment Items 2 and 3 in Effective Writing (i.e. the first two Folio submissions)." A short series of questions then followed:

1. What are the advantages of receiving feedback in traditional written form?

1a. And the disadvantages?

2. What are the advantages of receiving feedback on a cassette tape?

2a. And the disadvantages?

3. Would you be happy to receive feedback in taped form for every Assessment Item? Why or why not?

4. Are there any other comments that you would like to make?

The task of generating feedback in recorded-voice form for the first time was quite a challenge. It became apparent almost immediately during my engagement with the first of the ten folios that I could not hope to generate my spoken comments fluently in real-time: that is, speaking as I actually read, analyzed and pondered - not hitting the pause button. One element of the assessment process that has perhaps been insufficiently discussed in the literature to date, is precisely this problem of "compression and summarization". How do university teachers actually bring together all of the potentially infinite affects and meanings of a piece of writing (and a folio of writings plural is an even more complex entity) in a relatively short space in time?

At least one other issue can be raised here. A common fault with beginning and emerging creative writers is a relative blindness to issues of pace and tempo in their artistic productions. With all of the silences and abrupt interjections that it might perforce entail, there would perhaps be some value in real-time formative assessment with the intention of underscoring this dimension of a student's work. In other words, we could help students develop for themselves the "reading-through-with-temporal-awareness" strategy of writing production - the struggle with word limits is not a problem that disappears in the professional context. And how many of our students, after all, are we certain ever actually follow our advice and recite their work out loud prior to final drafting? The simple act of reading a student's work back to them, either directly in person or by employing a mechanical aid, would be no bad thing.

In the end, the approach I adopted was to read through each folio while jotting down ideas and responses when they came to me, linking these by page number or other means to particular places in the text. Having switched on the cassette recorder, I would then work my way through the entire folio again, on this occasion committing my comments permanently to the tape. You could say that I was providing about double the normal amount of feedback. What the students would end up hearing, were both fluent versions of my written ideas, and either extensions of these ideas or explanations of the assumptions of writing practice/theory that are seen by experts to underpin them. An advantage of engaging with the student's work twice, which allowed for reciting from a preliminary written draft of my responses, was that I did not feel the necessity of editing my tapes in any way.

I am making a simple distinction between the redaction of my written notes for verbalization and a quantity of ad libbed additions direct to tape. This distinction can be refined. One element of responsiveness from this second category started to make its presence felt in particular once the cassette recorder was switched on. To a significant extent, I found that I was actually commenting on the process of my examiner's reading: typically, I would say something like 'and now I've come to the third stanza of the poem and the beginning is just as obscure' or 'this melodramatic outburst doesn't follow on very effectively from the previous snatch of realistic dialogue'. It was not very difficult to replicate my first reactions the second time around - this became one of the new skills that I developed as an educator confronted with a pedagogic challenge. (In a moment, we will attend to the very substantial importance of process to the project as revealed in the questionnaire responses.) Crucial to the integrity of the entire exercise, from my point of view, was that I did something more than just read out written comments that - if we leave to one side for the moment Elias Canetti's generic distinction between the ear and the eye - could almost as well have been simply returned to the students in written form.

The questionnaire responses can be analyzed both quantitatively and qualitatively.

Quantitative results were extremely encouraging: only 20 per cent of students (two) said that they would not be happy to receive feedback in taped form for every piece of assessment. 80 per cent (eight) were either positive or very positive about my experiment. But this statistical analysis is not even the more interesting half of the story.

Engagement with the qualitative dimension of the questionnaire responses allows us to hear the individual voices of students with striking specificity and clarity. (I record them here verbatim.) What was appreciated above all was the more personal feel of the spoken feedback, in comparison to written comments. "More intimate," said one. "Very personal . . . as if your tutor was straight in front of you," commented another. Again: "it feels more personalised" and "more personal". "I discovered," writes Peggy Nightingale "that I was more pleasant as a talker than as a writer. . . more personal, 'chatty'. . . students seem to believe I'm really trying to help them" (Nightingale 39). On the flip side, with respect to the disadvantages of receiving feedback in traditional written form, one student observed that "sometimes it can be impersonal". Students also appreciated the added nuances and inflections of meaning that are possible in speech. "Tone of voice gives further depth and meaning to the comment", was a typical response. Additionally: "added perspective of tone of voice for more understanding". More extensive than standard written comments, my taped feedback can also be characterized as deeper, richer and more intensive. How is information ever actually imparted? I suggest that detail is as much a function of the rhythm and melody of a voice as of the accumulation of (additional) phrases and sentences. Quantity and quality might be seen to complement each other here.

Jeri Kroll's point about Creative Writing and "personal involvement" generates an interesting juncture with these student voices. According to John Singleton, however, there are "two major aspects of Writing - the notion of process and the notion of the personal" (Singleton 69). The first of these can also be extrapolated as a productive framework for interpretation of my students' qualitative reactions. A great many respondents talked about thoroughness and detail: that is, companion concepts of process. For example: "more thorough", "very thorough" and "it is more thorough." Also: "more detail" and "more exacting". Other observations fell comprehensively within the ambit of process. "I like to have the verbal explanation of how the mark came to be," noted one student. "It was very helpful to hear the first reactions," wrote somebody else, allowing us to infer that a more extensive dynamic of process was in progress. "A step by step breakdown," was another phrase employed. Again: "I felt like you had really read my folio not just looked through it." And: "I feel that you got a more clearer understanding on what the mentor is trying to get across." With regard to written feedback, one student characterized a disadvantage like this: "no step by step breakdown of the marks." Another complained that "I feel that you can't really understand how the mentor tries to portray their thoughts through brief notes and marks." By contrast, with taped feedback, "a step by step breakdown is given and you can really grasp what is being told."

The notion of process connects with the personal to the extent that positive interaction with students is effectively facilitated through process as a means of engagement that - with regard to their lives - is not abstract, pre-emptive or out of proportion.

Students also brought into sharp relief what I had originally only been able to roughly adumbrate as the practical advantages of recorded-voice feedback over the written mode of formative assessment: "the spoken word conveys more info. quicker and in more depth than the written." Also: "you can say more on tape than you can write in comments." Indeed, one student even felt moved to observe that "I like the simplicity of written comments, the taped comments are very complex and quite indepth for grading." This is, with the recognition of a certain reversal or idiosyncrasy in the comment, an observation that actually resonates positively with the practice of recorded-voice feedback.

The economies of time and effort associated with this method of assessment are made precise in Peggy Nightingale's short piece. "In terms of my time, recording comments is quicker than writing them. Marking a 3000-word literature essay takes me something like half an hour on average, and with the recorder I save five to ten minutes per paper" (Nightingale 38). She goes on to say, "I also give the students much more commentary; because you can talk so much faster than you can write, students get much more information" (38). The logistical advantage for me over the course of this pilot study was probably similar; I think it would become even greater with more practical experience of the method. But I am concerned to avoid reductionism. It is not just number of words (or even time) that matters. Susan Sontag's recognition of the somewhat indeterminate but positive qualities of hearing and the voice must not be forgotten.

"I would really like to see taped feedback being used in the future in writing classes." If most students would have gladly given their assent to this comment, there were also a couple of dissenters. They objected mainly to what their peers had been so enthusiastic about. We should listen to what they have to say too. "The tone of your voice," said one student, "can not be made neutral and for some people that might be disparaging and hard to take especially if they have not done well." Another respondent wrote that he would like "to keep the relationship/communication impersonal/objective." This is a quite stark response to the problem of creative submissions being typically both personal and subjective. Yet another student, although himself in favour of receiving feedback in this form, observed altruistically that it might be "possibly too confronting for some people" and that it is "harder to take criticism 'in person' as it were." Also: "criticisms face to face can cause uncomfortable feelings."

This more circumspect dimension of the questionnaire feedback reminds us that no change to any assessment package will ever be an unqualified good. Indeed, buttressing the comments just cited is the noticing of other positive dimensions to the employment of written feedback: the consensus of response to the first question was that this method of formative assessment is more immediate, comprehensive, to the point, and with the additional advantage of being sensitive to the spatiality of the page. For example: "it is easy to see instantly the exact mistake and to scan over the result." Also: "it is all 'pinpointed'" and "straight to the point." Access to the "overall impression" was commended. And, of course, "the feedback can be written in the area of the paper that it applies to."

I acknowledge that many teachers are now using computers to generate written comments. Software packages for the efficient dissemination of basic learning points have also been developed. I suppose that one issue with respect to these technologies concerns the personal touch that might be inherent in writing by hand; is the pen or pencil, in comparison to the keyboard, more an analogue of the voice? As I have just noted, writing by hand also allows for maximizing any exploitation by the marker of the spatiality of the page and the words upon it. (I am sure, all the same, that every student would quite simply appreciate never having to decipher the scribblings of academics - some are more guilty here than others.) Part of the argument of this paper, of course, has been that the advantages of recorded-voice assessment are not limited to any unproblematic quantification of word length.

How might we "walk the talk" of these findings about assessment issues? Perhaps some sort of a mix of taped feedback - verbal, repeatable and private - with the traditional written type of formative assessment, could be introduced into our assessment packages. Or perhaps spoken feedback could be generated, but with explicit attention to the problem of being "too personal" and thereby inadvertently hurtful. I take it as beyond reasonable doubt, at least, that if only one mode is to be used then - on the basis of this preliminary, small-scale case study - it should be the spoken one. This proposition can be supported both on the basis of a cost (time)/benefit analysis and with respect to the ethics of pedagogy.

References

Biggs, John. Teaching for Quality Learning at University. Buckingham: Society for Research into Higher Education & Open University Press, 1999. Return to article

Kroll, Jeri. "A or C: Can We Assess Creative Work Fairly?" TEXT (Vol. 1, No. 1; April 1997) <www.griffith.edu.au/school/art/text/april97/kroll.htm> Return to article

Nightingale, Peggy. "Recorded Commentary." Improving Student Writing. Green Guide # 4. Kensington, NSW: Higher Education Research and Development Society of Australasia, 1986. Pp 38-39. Return to article

Ramsden, Paul. "Studying Learning: Improving Teaching." In Paul Ramsden (ed). Improving Learning: New Perspectives. London: Kogan Page, 1988.) Pp 13-31. Return to article

Scardamalia, Marlene and Carl Bereiter. "Literate Expertise." In K. Anders Ericsson and Jacqui Smith (eds). Toward a General Theory of Expertise: Prospects and Limits. Cambridge: Cambridge University Press, 1991. Pp 172-94. Return to article

Singleton, John. "Creative Writing and Assessment: A Case Study." In Moira Monteith and Robert Miles (eds). Teaching Creative Writing. Buckingham: Open University Press, 1992. Pp 66-78. Return to article

Sontag, Susan. "Mind as Passion." Under the Sign of Saturn. London: Vintage-Random House, 1996. Pp 181-204. Return to article

Dr P.L. West is a lecturer in Creative Writing in the School of Arts, Griffith University, with a special interest in the assessment of creative submissions at tertiary level. His PhD is from the University of Melbourne.

Return to Contents Page
Return to Home Page