Translation Evaluation in Educational Setting for Training Purposes: Theories and Application
BEHROUZ EBRAHIMI
Introduction
The field of Translation Assessment is problematic, and it is often difficult to tell the difference between, e.g., ‘translation evaluation’, ‘translation criticism’, ‘translation assessment’, and ‘translation quality assessment’. Some scholars are concerned with developing models that satisfy the needs of practitioners, thus an empting to bridge the gap between theory and practice. Others attempt to draw up ‘objective’ translation assessment criteria by means of incorporating conventional frameworks of educational measurement, such as reliability, validity, and objectivity, into their overall structures.
There are various theories and applications about the evaluation of students' translations. It is claimed that the field of Translation Quality Assessment is problematic, especially when the text is long.
Body
Farahzad (1992) maintains that two main features are to be checked in scoring for each unit of translation (she suggests that sentence and clause might be the units of translation) and they are:
1. Accuracy: the translation should convey the information in the ST precisely i.e. the translation should be close to the ST norms.
2. Appropriateness: the sentences sound fluent and native, and are correct in terms of structure.
She declares that unnatural translations which convey the source text's meaning receive half a score, whereas inaccurate translations receives no score, no matter how appropriate and natural the target texts sound.
In error recognition items, one score is given for spotting the error, and another one for correcting it.
Farahzad believes that scoring the long text can be done in several ways:
A: it can be scored holistically: since the item assesses a wide, variety of competencies, the examiner may find it convenient to approach the text as the unit of translation and adopt this system, especially with a large number of students.
The examiner may, for instance, come up with the following scheme:
1. Accuracy 20 percent
2. Appropriateness 20 percent
3. Naturalness 20 percent
4. Cohesion 20 percent
5. Style of discourse/choice of words 20 percent
B: it can be subjected to objectify scoring. In this system the target text must be read two times, first to check the accuracy and appropriateness, then for cohesion and style. Albeit time-consuming, this system is more reliable.
Farahzad suggests that sentence and clause might be the units of translation. Thus each verb in the source language text marks a score. The main clause receives one score and each sub-clause another score. So the accuracy and appropriateness are checked in each sentence and clause.
Cohesion and style cannot be checked and scored at the sentence and clause level. The elements of cohesion (e.g. transitional, appropriate use of pronouns, linkages, etc.) are spread all over the text as are the elements which form the style of discourse (choice of words, grammatical structures, etc.)
If, for instance, the source text is fairly neutral, one may allot a smaller number of points to it than in other cases where the preservation of style is important.
Farahzad's method seems a holistic method and it may cause some problems in evaluation of translations. Hence, it seems that Waddington's method might complete the Farahzad's method in assessment.
Waddington (2001) indicates that almost all the contributions in Translation Quality Assessment have been descriptive or theoretical and have centered mainly on the following themes:
(i) Establishing the criteria for a “good translation” (Darbelnet 1977, Newmark 1991);
(ii) The nature of translation errors:
• Defining the nature of translation errors as opposed to language errors (House 1981, Nord 1993, Kussmaul 1995, Gouadec 1989);
• Drawing up a catalogue of possible translation errors (Gouadec 1981);
• Establishing the relative, as opposed to absolute, nature of translation errors (Williams 89, Gouadec 89, Pym 92, Kussmaul 95);
• The need to assess quality not only at the linguistic but also the pragmatic level (Sager 1989, Williams 1989, Hewson 1995, Kussmaul 1995, Nord 1996, Hatim & Mason 1997);
(iii) Basing quality assessment on text linguistic analysis (House 1981, Larose 1989);
(iv) Establishing various textual levels on a hierarchical basis and linking the importance of mistakes to these levels (Dancette 1989, Larose 1989);
(v) Assessment based on the psycholinguistic theory of “scenes and frames”
(Dancette 1989 and 1992, Bensoussan & Rosenhouse 1994, Snell-Hornby 1995).
In order to find out the kind of translation exam and the kinds of methods of correction currently in use in Faculties of Translation, Waddington sent out a questionnaire to 48 European and Canadian universities. A total of 52 teachers replied from 20 of these universities and their answers reflected the following situation:
(i) All the teachers said that they require the students to translate a text, although over half also include other complementary tests.
(ii) As far as methods of evaluating student translations were concerned, 36.5% of the teachers use a method based on error analysis, 38.5% use a holistic method, and 23% combine error analysis with a holistic appreciation.
In accordance with these findings, he considers the validity of the results obtained through applying these different types of method to the correction of translations of part of an authentic text done by students under exam conditions.
Then, Waddington introduces four methods of assessment. The first method (method A) is more known than other methods and is functional in translation classes.
Method A is taken from Hurtado (1995); it is based on error analysis and possible mistakes are grouped under the following headings:
(i) Inappropriate renderings which affect the understanding of the source text; these are divided into eight categories: contresens, faux sens, nonsens, addition, omission, unresolved extralinguistic references, loss of meaning, and inappropriate linguistic variation (register, style, dialect, etc.).
(ii) Inappropriate renderings which affect expression in the target language; these are divided into five categories: spelling, grammar, lexical items, text and style.
(iii) Inadequate renderings which affect the transmission of either the main function or secondary functions of the source text.
In each of the categories a distinction is made between serious errors (–2 points) and minor errors (–1 point). There is a fourth category which describes the plus points to be awarded for good (+1 point) or exceptionally good solutions (+2 points) to translation problems. In the case of the translation exam where this method was used, the sum of the negative points was subtracted from a total of 110 and then divided by 11 to reach a mark from 0 to 10 (which is the normal Spanish system). For example, if a student gets a total of –66 points, his result would be calculated as follows: 110-66=44/11=4 (which fails to pass; the lowest pass mark is 5).
Jamal Al-Qinai in Translation Quality Assessment: Strategies, Parameters and Procedures (2000) indicates that translation is a complex hermeneutic process in which intuition plays a crucial role in interpreting the intentions of the ST writer. Further, languages vary in their choice of lexical connotations, sentence structure and rhetorical strategies, the only tangible tools for assessment. It is prudent, therefore, to talk about the adequacy of a translation rather than the degree of equivalence. Quality is relative and absolutes of accuracy cease where the end user (i.e. client) imposes his own subjective preferences of style in TT. Standardization of quality is thus a fuzzy grey area. For instance, does accuracy and good translation mean that a shoddy poorly-written, poorly-structured ST be reproduced as a shoddy poor TT? Is it professional for a translator to act as a filter, an advocate of ST? Alternatively, should a translator produce a ‘straight’ translation rather than a ‘sanitized’ one? (ITI Conference 1994: 72-3).
However, Al-Qinai in his study concerns to textual/ functional (or pragmatic) compatibility (i.e. quality of linguistic conversion) rather than to the logistics of management and presentation (i.e. quality of service). He points out that the ultimate end-users are interested in the quality of the product and not the means sought to serve its creation.
Al-Qinai Sets-up a model for translation quality assessment. He writes:
The assessment of a translated text seeks to measure the degree of efficiency of the text with regard to the syntactic, semantic and pragmatic function of ST within the Cultural frame and expressive potentials of both source language and target language.
Al-Qinai states that since no two languages are identical, either in meaning or in form, the best we can hope for is an approximation given the following variables:
a) Nature of ST message.
b) Purpose and intent of ST producer.
c) Type of audience.
Julia Sainz (1992) discusses a student-centered approach to correction of translations. She believes that teachers must make it clear that there are no right or wrong answers to the questions and that the students' answers are going to be used only as feedback for discussion later on.
The process which Julia Sainz suggests for correction of translations comprises five stages:
1. Development is a stage during which intended to understand and anticipate students' needs in order to those needs more efficiency.
2. Implementation is a stage during which students get the "correction chart" shown on the following:
Mistakes Possible Correction Source Type of Mistake
Under "Mistakes" students write the word, phrase or sentence which was understand as incorrect in their translation.
Under "Possible Correction" they try to produce an "error free" version.
The source of the answer for students' correction is entered under the column "Source" as: 'Myself'; 'Peer'; 'Dictionary'; 'Teacher'.
The column "Type of Mistake", filled in by the students, can become a good exercise to help students recognize what types of mistake they are making and consequently eliminate them.
3. Monitoring is a stage during which teachers can monitor the process in order to make adjustments as the course unfolds, on the basis of the information they retrieve from the 'Correction Chart'.
4. Integration is a stage during which teachers can fill in their own chart of "Types of Mistakes" for a particular translation piece.
5. Self-monitoring is a stage during which students can check their own progress in the course, at the same time, become critical about their learning.
At the bottom of the 'Correction Chart', students are asked to circle the figure, ranging from +3 to -3, which they think best matches their idea about their performance in that particular translation passage and to make any other comments.
A student-centered correction of translation is very useful in translation classes. By this careful system, the students are subject to constant revision and changes in order to be improved. Small changes can sometimes create great effects. This method based on having a class.
Carol Ann Goff-Kfour in 'Testing and Evaluation in the Translation Classroom' discusses reliability and validity in evaluation:
Evaluation depends on the reliability of the test instrument. Reliability refers to the test's consistency.
Goff-Kfour, then, states the types of assessment:
A placement test, Progress tests, Achievement tests, Formative assessment, Summative assessment, Process assessment, and Portfolio assessment
Among the suggested tests and assessments, portfolio assessment seems one of the best methods. It is a new technique to aid students in tracking their progress. Not only do the students track their own level but also the instructor is able to judge the student's work in reference to past assignments. The portfolio method is time consuming for instructors who have large classes, but the advantage is that instructors can gauge the progress of the student by actually consulting the work done by the student at the beginning of the course or in the middle rather than only consulting the marks in their book.
References
Al-Qinai, J. (2000) 'Translation Quality Assessment. Strategies, Parameters and Procedures ' Meta, XLV, 3, 2000
Farahzad, F. (1992) 'Testing Achievement in Translation Classes' Amsterdam/Philadelphia. John Benjamins Publishing Co.
Goff-Kfouri, C. (?) 'Testing and Evaluation in the Translation Classroom'
Julia Sainz, M. (1992) 'Student-Centered Corrections of Translations' Amsterdam/Philadelphia. John Benjamins Publishing Co.
Waddington, C. (2001) ' Different Methods of Evaluating Student Translation: The Question of Validity' Meta, XLVI, 2, 2001