Principles of good assessment practice
-Lynn Vos
Principles of good assessment practice
The bulk of educational research concerns assessment principles and practices – how to structure assessment, how to improve it and how to ensure that students learn what they are intended to learn. This post summarises eight principles of good assessment practice that together have come to be known as ‘authentic assessment’ (AA). It is adapted from a paper that I wrote on how tutors who use simulation games apply these principles when assessing student learning. While not yet adequately researched to demonstrate its superiority to forms of assessment practice, the individual principles are supported by much evidence and when woven together can lead to a much stronger assessment regime in which students become more fully engaged and challenged, gain a greater understanding of the value of the tasks they are required to do, and have the opportunity to build increasing competence over the semester – all desired outcomes of assessment.
This post starts with an overview of the trends in assessment research since the late 1980s as educators and researchers sought more meaningful and effective ways to measure student learning than what standardised testing could achieve. It then considers the eight principles of authentic assessment that tutors can use to develop their assessment regime for a particular module or course.
Trends in assessment research
Over the past three decades, research into assessment in higher education has tended to fall within three main themes: design, measurement and validity of assessment instruments; evaluating and providing feedback on student performance; and alternative forms of assessment to develop the learner.
The rise of alternative forms of assessment emerged out of the debates in the 1980s and 1990s over the educational value of standardised testing (also known as traditional assessment). Many argued that large scale testing forces teachers to narrow what is taught in order to meet test requirements so that assessment tends to encourage and instrumental approach to learning where the goal is to reproduce what was taught rather than to develop a deep understanding of concepts and the ability to think independently.
Debate and criticism of assessment practices have come from within and outside of academia. Within academic governing bodies such as the UK Higher Education Quality Council, discussion moved from qualifications to the skills and abilities that university graduates should possess upon completion of studies. This was the beginning of a growing trend for greater transparency and accountability in higher education – a trend that has brought both good and more questionable practices into HE, the latter including such ‘measures of quality’ as the NSS (National Student Survey) and more recently, the Teaching Excellence Framework (TEF).
With respect to assessment, in both the US and the UK, parents, employers, and other external parties sought evidence that traditional assessments and standardised testing were really contributing to student knowledge and skill development. A contingent debate emerged about the role of higher educational institutions in preparing students for employment, leading ultimately to a major emphasis on instructional and assessment practices that embed employability skill development.
One consequence of the testing debate was a shift into the foreground of alternative philosophies of education and assessment. Traditional assessment practices founded on objectivist theories of learning gave way to assessment research based on cognitive/constructivist and sociocultural theories. The objectivist tradition generally views knowledge as existing independently of or outside the knower and it is the teacher who conveys meaning. Constructivists, on the other hand, see learning as a process of active engagement through which learners construct meaning and new ideas, taking into account their current and previous knowledge. Learning is not, therefore, imposed or transmitted by direct instruction but emerges out of the learning context and within the learner him or herself. Schell (2000) explains the constructivist perspective as follows:
"The teacher provides the roadmap … while allowing students to construct their understanding of the topic. Learners assume increasingly more control over … learning and are free to explore the various … details of the topic. They can build their own mental frameworks in ways natural to them". (p. 10)
Socio-cultural theories of learning, many of which are also constructivist in orientation, have been informed by the work of Dewey (1916), Vygotsky (1978), and more recently by Lave and Wenger (1991). All argue for the importance of context in learning. Vygotsky (1978) and Lave and Wenger (1991) situate learning within interactive groups, or as Wenger (1998) called them, ‘communities of practice’. This research has implications for how learners transfer knowledge learned in other contexts, as well as for the value of realistic or ‘real world’ learning situations.
The rise of constructivist thinking and practice and of Biggs' (1996) now widely adopted approach to curriculum design has led to a proliferation of forms of assessment alternative to traditional testing. Among the most widely discussed are competency based, performance-based, direct and authentic forms of assessment. While each of these has unique attributes they share a number of characteristics: all types require students to actively engage with concepts, ideas and projects, involve ‘real world’ or professional contexts, have clear assessment criteria, are challenging and develop higher level thinking skills, in addition to assessing not only knowledge, but competencies and skills.
Authentic assessment
The alternative approach that has received the most attention in the literature is authentic assessment. Over the past two decades a number of conceptualisations of authentic assessment have been developed, all with the main core objective of setting out processes and practices that are meant to better prepare students for future occupations than traditional assessment are able to do and to allow students to develop a deep sense of the material, but also key skills and competencies that can be transferred to new learning situations. While a consensus on its characteristics is yet to occur, the following eight principles incorporate most frameworks or discussions of authentic assessment:
1. The real world value of the assessment task;
2. Students perform or create a product as output;
3. Challenge and complexity of tasks and issues of transfer;
4. Known criteria and assessment literacy;
5. Developmental opportunities with formative assessment and regular feedback;
6. Sufficient and varied activities to make up the whole;
7. Opportunities for reflection; and
8. Interaction and collaboration.
A brief overview of each principle and its characteristics follows.
1. The real world value of the task
For many researchers of authentic assessment practices, the real world value of the assessment task or tasks is the main determinant for authenticity. Building on both constructivist and ‘assessment for learning’ theories authentic assessment calls for learning activities that mirror how performance would be undertaken in professional settings or in the ‘real-world’, particularly where the learning intention is for students to demonstrate a skill or the application of knowledge. This does not mean 'training' students how to do employment tasks, but actively involving them in actual and challenging situations, cases and problem solving.
The real world requirement emerged out of the original definition of authenticity created by Archbald and Newmann (1988) in their counterpoint to the perceived failures of traditional assessment in preparing students for future work or learning. The authors, who first coined the term authenticity, wrote:
"What counts for success in school is often considered trivial, meaningless, and contrived by students and adults alike. Ultimately then, the quality and utility of assessment rest upon the extent to which the outcomes measured represent appropriate, meaningful, significant and worthwhile forms of human accomplishment. We synthesize these qualities into one idea: authenticity". (Archbald & Newmann, 1988, p. 71)
Initially, therefore, the call was for more meaningful and significant assessment tasks, and not specifically for those representing professional tasks. Over time, the greater focus in higher education on preparing students for employment has strengthened the importance of assessment that related to the world of work and to more complex problems.
Much research has demonstrated that a focus on real world activities makes assessment meaningful, more motivating and compelling to students, and this, in turn, supports higher levels of motivation that can translate into better outcomes. In general, students are also more likely to take a deep approach to learning if they are intrinsically motivated and see the relevance and importance of the task to their future.
2. Performance or product as output
The constructivist perspective that underlies authentic assessment views learning as the active creation of knowledge and a process that as Biggs (2003, p. 36) noted “…changes the students' perspectives on the world so they behave differently”. So assessment should require students to provide an active demonstration of that learning either through a performance, a variety of performances or the creation of a product as output. In demonstrating the behaviours or practices required, students show that they have acquired not only knowledge but requisite skills and competencies. Note that this does not limit assessment to active demonstrations of knowledge in the form of, say, presentations or carrying out specific tasks, but can also include written examples, as long as the tasks reflect the kinds of understanding required in the discipline or more broadly develop transferable skills. Examples include: portfolios, open book exams, take away exams, projects and investigations, varied writing assignments, oral assessment, problem solving tasks, simulations, self, peer and co-assessment. Wherever students can engage in active learning, students have the opportunity to practice and develop knowledge and skills.
3. Challenge, complexity and transfer
Authentic assessment should be challenging and represent the complexities and ambiguities of real world decision making as well as the potential for multiple solutions/perspectives.
After all, the kinds of problems that graduates are likely to encounter in their work as well as the more complex problems facing humanity require higher order thinking skills -- analysis, evaluation and critical thinking-- in addition to broad multi-disciplinary perspectives. Thus, assessment should allow students to practice and develop these skills and to bring in ideas learned elsewhere.
Getting students to effectively transfer knowledge from one context to another is an aspect of learning that has confounded teachers and researchers. Constructivists have argued that the transfer of knowledge from one domain to another is difficult, particularly given that learning is context-based For example, students who learn math in a math class may not be able to use those skills in completing a personal budget. Successful transfer of knowledge requires scaffolding, discussion with others, and opportunities for reflection, in addition to a good understanding of the material that is being transferred. None of this is easy and requires teachers to think carefully about how they structure and scaffold learning. As we will see below, opportunities for practice are required. Effective transfer of knowledge rarely occurs the first time around.
The work on flow by Csikszentmihalyi (1987) demonstrates that motivation, affect, arousal and concentration are highest when levels of challenge and skill requirements are perceived as high. Alter and colleagues (2007) showed in their research on the disfluency effect – a metacognitive regulation process – students will assign their cognitive resources depending on the perceived difficulty of the task, i.e. they will work harder if they see the task as challenging them.
Interestingly three new and one rephrased NSS questions reflect this thinking: The course is intellectually stimulating; my course has challenged me to achieve my best work; and my course has provided me with opportunities to bring information and ideas together from different topics. The latter relates to transfer of and integration of knowledge. With respect to the second principle above on product as output, the NSS question – my course has provided me with opportunities to apply what I have learned – reflects assessment that is applied and involves active learning.
4. Known criteria and assessment literacy
In authentic assessment, teachers will expose students in advance to the main criteria upon which they will be evaluated and offer opportunities for students to become more ‘assessment literate’ through the use of exemplars or a dialogue to create a shared understanding of what is expected.
A social constructivist perspective argues that “meaningful understanding of assessment requires some kind of active engagement with the criteria by both tutors and students” (Rust, O'Donovan, & Price, 2005, p. 234), because “to truly understand…the criteria and standards being applied requires tacit as well as explicit knowledge” (p. 231). In other words, simply telling students the bases on which they will be assessed is not enough. Students need to themselves be put in the position of the assessor and grade others work for themselves (peer assessment) in order to inculcate the tacit knowledge involved in assessment that is difficult to convey with words alone.
Furthermore, as Gulikers et al. (2004) argue, in professional situations employees are generally aware of the criteria on which their work will be judged (although tacit criteria are also at play at work, as Polanyi notes), so it should also be transparent in the learning context. Their study on the benefits of authentic instruction did point out, however, that too many criteria can actually reduce motivation and learning so teachers have to get the balance right.
An addition point needs to be made here. As teachers, we often assume that our colleagues or teachers on our team understand the assessment criteria in the same way that we do. This is not always the case, however as demonstrated by Webster et al. (2000) in their study of how 55 different members of staff interpreted the terms ‘analyse’ and ‘evaluate’ as used in assessments at the school. The wide range of different interpretations provided by staff shows that teaching teams need to also share understanding of what the task requires if they are to guide and grade students work in a consistent manner.
5. Developmental opportunities with formative assessment and regular feedback
Where assessment seeks to measure competency of a range of knowledge and skills, students need formative developmental or practice opportunities and be provided with regular feedback on their progress during completion. After all, giving students the opportunity to practice a skill or a set of actions is one of the main opportunities that formal education should provide and practice allows students to gain greater levels of competence over time. For each stage of practice, tutors can give feedback that will help students to understand where they need improvement and how they might move forward. Also, given that authentic assessments may involve a series of stages or steps, and that the success of future stages may depend upon earlier successes, practice and feedback are extremely important.
Furthermore, practice, followed by feedback either from the tutor or peers will allow students to demonstrate, over time, greater levels of competence and the application of higher levels of thinking.
Time is short in most modules (10-15 weeks of formal classroom time) and work loads of tutors is high so a key question is how to provide sufficient opportunities for practice and formative feedback that allows student to move onto the next assessment with greater understanding and competence. Peer assessment is one method as are learning analytics tools increasingly available from publishers.
Of all the principles of authentic assessment this one can be the most resource intensive so teachers need to think very creatively about how to provide these kinds of scaffolded opportunities.
6. Sufficient and varied activities make up the whole
Authentic assessment should require students to engage in sufficient and varied activities to ensure that they cover all the associated learning outcomes or intentions rather than allowing them a choice of what to be assessed on (e.g. final exam with a choice of questions). According to the principles of constructive alignment, assessment should cover all intended learning outcomes. Furthermore, complex tasks require students to demonstrate a range of competencies that cannot be judged in a single test or activity so assessment “should involve a full array of tasks and multiple indicators of learning in order to come to fair conclusions” (Gulikers et al., 2004, p. 80).
In addition, as noted above, assessment should involve scaffolding, such that students are exposed to incrementally more challenging problems requiring progressive application of higher order thinking skills, and therefore more than one task is generally required.
Another benefit of sufficient and varied activities in the assessment regime is that students with different levels of knowledge and skills may perform better on one activity than another.
7. Opportunities for reflection
Reflection is a critical aspect of an authentic learning environment and one that should be encouraged and guided. Boud, Keogh, and Walker (1985) see reflection as a natural process of having engaged in meaningful experiences. During reflection, students are exploring what they have done or learned in order to make new connections, form new understandings, but also to identify weak or missing links in knowledge or skills. In addition to the cognitive-structuring benefits of reflection, it allows students to consider their own approaches to learning. The latter, ‘thinking about one's thinking’, is termed metacognition and is important in assessments involving an array of tasks, complexity and formative feedback. It is also considered by many advocates to be essential to improving critical thinking processes. However, Rule (2006) cautions that metacognitive skills do not come naturally to all, however, and reports on a study by Kramarski, Mevarech, and Arami (2002) showing that those who received instruction in reflection and metacognition within a cooperative learning project outperformed those who had no such guidance. Although not writing on authentic assessment, Jennifer Moon (1999, 2004) has also written extensively on the importance of helping students to develop these skills prior to asking them to undertake reflective activities.
8. Interaction and collaboration
A number of researchers include collaboration as a necessary requirement of authentic assessment
and base this requirement on social constructivist theories such as that of Vygotsky (1978). Vygotsky argued for the importance of social and cultural influences on learning and of social interactions as the means by which individuals make sense of the world. Interaction and discussion with more experienced peers and with tutors help students move towards greater levels of understanding and performance than could be achieved independently particularly if those peers/tutors can scaffold the learning appropriately for the level of the student. Vygotsky (1978) termed the gap between current levels of understanding and what is possible with effective collaborative instruction as the zone of proximal development. Not all advocates of authentic assessment see collaboration with peers as a necessary requirement, however. For example, Gulikers et al. (2004) argue that students should only be working with others if the real world scenario upon which the assessment is based also calls for team working. On the other hand, they do agree that the tutor has an important role in scaffolding information and in providing feedback.
An important point to raise here relates to group work. I caution that while group work can provide the benefits of collaboration it can also hinder learning where team members are either not all committed or have different levels of skills and knowledge. In addition, where group work is used for an assessment task, it is very difficult to fairly conclude that each student has achieved all the learning outcomes in the module if they were only preparing one part of the assignment. I recommend that group work should be used for discussion and support but students should produce independent pieces of work for grading.
Summary
These eight principles summarise the main characteristics of authentic assessment as it has developed in the literature over the past twenty plus years and provide a framework for the tutor. Most principles represent good stand-alone practices but when interlinked with the others can provide greater learning benefits. Biggs (2003), who considered authentic assessment to be a good example of constructive alignment, also noted that its effectiveness is enhanced when it is aligned with authentic instruction and in fact the two are often combined into authentic instruction or pedagogy or termed an ‘authentic learning environment’ (Newmann et al., 1996; Sambell et al., 2013). The underlying premise is that instruction and assessment are interdependent and for assessment to be effective it needs to be grounded within an instructional pedagogy.
To date, empirical research into the effectiveness of authentic assessment in terms of student learning gains as compared with other forms is somewhat limited. More studies are needed into whether authentic assessment contributes to greater learning gains than other forms of assessment as it has not yet been demonstrated convincingly. McAlister (2000) notes that one of the key challenges in measuring the performance-related benefits of authentic assessment is the existence of so many small but noteworthy differences in how it is conceptualised. Studies are also needed that consider how other factors may affect the success of these kinds of assessments. For example, only a few studies have looked at the role of the student.
In their research, Fook and Sidhu (2010) found that most students valued authentic assessment as they felt it helped them to develop more skills than traditional tests and was more beneficial to their future work, but many students also admitted that they did not really put in much effort to get the best out of these assessments. Others found that assessments of this kind ‘were a sheer waste of time’ and that with portfolio type assessments, for example, ‘they did not know what to write … [and] were most of the time repeating the same thing’ (p. 158). Tutors also found that many students remained quite passive and resistant to the active involvement required, that plagiarism was an issue, and not all members of a group assignment did their part. In summary, additional studies on how best to implement authentic assessment and on ways to overcome barriers to its effective use are also needed.
The much more prolific non-empirical studies on authentic assessment e many of which have been investigated to generate the eight principles above have sought to conceptualise the concept, link it to learning theories, develop models, build upon previous theoretical frameworks, and to bring in empirical research from other studies to show where various characteristics have demonstrated a contribution to student learning gains (Ashford-Rowe et al., 2014; Cumming & Maxwell, 1999; Custer, 2000; Keyser& Howell, 2008; Myers & Nulty, 2009; Newmann, Secada, &Wehlage,1995; Newmann&Wehlage, 1993; Petraglia, 1998; Reeves et al., 2002; Rennert-Ariev, 2005; Rule, 2006; Saunders et al., 2001; Schell, 2000; Swaffield, 2011). Many of these studies are of the benchmark or best practice kind that tutors can use to compare their own assessment values and practices against (Price et al., 2011), while others are meant to stimulate debate and discussion (Boud & Falchikov, 1999) or to identify appropriate methods to use (Darling-Hammond & Snyder, 2000; Galarneau, 2005; Herrington & Herrington, 2006; Lebow & Wager, 1994), among them simulation games.
The learning theories and perspectives that underpin each of the eight principles are outlined in Figure One below:
Figure One: Authentic Assessment theoretical underpinnings
A key theme in assessment research is student involvement in the entire assessment process – from design and development of assessment tasks, to understanding the rationale for each assignment and its contribution to their learning, to acting as assessors themselves in order to inculcate both the explicit and tacit criteria on which they will be graded and through active collaboration and interaction with peers and tutors. The more we educate and involve students in assessment processes – what we do, what they do, and their own understanding of personal strengths and weaknesses (reflection/metacognition), the greater ownership they will take and the greater the learning gains.
My full paper ‘Simulation games in business and marketing education: How educators assess student learning from simulations, International Journal of Management Education, 13 (2015) looks at how tutors apply authentic assessment principles in measuring learning with simulations. It also considers some of the challenges they face in applying these principles. You will also find the full list of references.
Below is an example of how the principles of authentic assessment can be applied to the assessment regime in a module that uses a marketing simulation game as the key learning tool.
1.See for example: Ashford-
Rowe et al., 2014; Cumming & Maxwell, 1999; Keyser & Howell, 2008; Lebow & Wager, 1994; Lund, 1997; Maina, 2004; Mueller, 2005; Reeves, Herrington, & Oliver, 2002; Reeves & Okey, 1996; Renzulli, Gentry, & Reis, 2004; Savery & Duffy, 1995; Tanner, 1997; Wiggins, 1993
