Where's the [Any] key?

Link to resource: Assessment Research, Resources and Rationale

 

Assessment is a crucial component of learning used in both K-12 and post-secondary education.  In order to support other educators, we wanted to examine assessment through the lens of an online learning environment.  Assessment is meant to focus on “student involvement and authentic, meaningful assessment, leading to the development of a variety of assessment forms” (Weurlanderet al., 2012).  Quality assessment requires knowledge of provincial curriculum frameworks, district vision and goals, as well as teachers’ expertise with traditional and online tools to support it. Furthermore, there is an emphasis on competence-based curriculum focusing on “capability rather than factual knowledge” (Weurlanderet al., 2012).  The resources we curated as well as the resources we created are supported by research and support our learning outcomes. 

Formative and summative assessment are vital aspects of all learning environments. Formative assessment informs teachers of students ongoing learning and understanding, whereas summative is a final capturing of students overall learning. According to Weurlander et al. (2012) formative assessment can influence “students’ learning in a number of ways. It sends messages about what counts as important knowledge; it has an impact on students’ approach to learning and gives feedback to students about their learning” (p. 749). Additionally, they included research that found formative assessment “positive[ly] impact[s] student learning” (p. 749). When considering or creating assessment, it is important to include a variety of “assessment tasks [as they] have the potential to support student learning in different ways” (p.758). Moreover, a multitude of assessment strategies, including collaborative and blended learning approaches, increases student engagement (Vaughan, 2014). 

Summative assessment is meant to be a final snapshot of students learning.  However, it can exclude learners or create anxiety when the stakes are high for a singular assessment for both in-person and online learning environments states that, 

“[h]igh– stakes tests are inevitably designed to be as ‘objective’ as possible, since there is a premium on reliable marking in the interests of fairness. This has the effect of reducing what is assessed to what can be readily and reliably marked. Generally, this excludes many worthwhile outcomes of education such as problem-solving and critical thinking” (p. 209).  

Another concern about high-stakes summative assessments is that they may encourage cheating, a pressing concern for teachers, particularly in an online environment (Mellar et at.,2018).  Creating a variety of authentic summative assessments and giving students choice can help to ensure their validity (Mellar et al., 2018). Therefore, it is imperative that students are provided with a variety of summative assessments with low stakes when teaching online. 

When integrating formative and summative assessment strategies, teachers should consider the quantity, as creating too many assessments to track student progress can have a negative impact (Vaughan, 2014).  This also restricts teachers from providing timely feedback to their students, a key component of successful online learning (Gaytan and Mcewen, 2007). “Feedback is therefore a key component in formative assessment, and students need to understand and be able to act on the feedback they receive in order to improve their learning” (Black and Wiliam 1998; Nicol and Macfarlane-Dick 2006; Sadler 1989, 1998; as cited in Weurlander et al.,2012., p. 748).  Furthermore, Shea and Bidjerano (2010) discovered that teaching presence along with teaching students the technological skills they need to complete and navigate given tasks leads to “successful online learning” (p. 1727).  Thus, well planned online learning experiences along with feedback, help and encouragement also improves learning success for at risk learners (Hughes, 2007). 

In conclusion, when considering online or in-person teaching environments, summative and formative assessments are vital.  Formative assessment should be used to continually check in with students about their understanding of content, leading into a summative review of their learning. When assessing students, a variety of tools and strategies should be used in order to reach all learners for both types of assessments. Overall, the focus of this project was to provide teachers with a realistic and comprehensive overview of what assessment in classroom and online environments could entail.  

Co-Authored by Faune Nicholas, Jerry Chien, Leanne Huston, and Rochelle Smith

 

References 

• Gaytan, J., & Mcewen, B. C. (2007). Effective Online Instructional and Assessment Strategies. American Journal of Distance Education, 21(3), 117–132. https://doi.org/10.1080/08923640701341653
• Harlen, W. (2005). Teachers’ summative practices and assessment for learning – tensions and synergies. The Curriculum Journal, 16(2), 207–223. https://doi.org/10.1080/09585170500136093
• Hughes, G. (2007). Using blended learning to increase learner support and improve retention. Teaching in Higher Education, 12(3), 349-363. https://www.tandfonline.com/doi/abs/10.1080/13562510701278690 
• Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz, A., & Yovkova, B. (2018). Addressing cheating in e-assessment using student authentication and authorship checking systems: teachers’ perspectives. International Journal for Educational Integrity, 14(1). https://doi.org/10.1007/s40979-018-0025-x 
• Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learning environments. Computers & Education, 55(4), 1721–1731. https://doi.org/10.1016/j.compedu.2010.07.017 
• Vaughan, N. (2014). Student Engagement and Blended Learning: Making the Assessment Connection. Education Sciences, 4(4), 247–264. https://doi.org/10.3390/educsci4040247 
• Weurlander, W., & Soderberg, M., & Scheja, M., & Hakan, H., & Wernerson, A., (2012). Exploring formative assessment as a tool for learning: students’ experiences of different methods of formative assessment. Assessment & Evaluation in Higher Education. 37.(6). 747-760, DOI: 10.1080/02602938.2011.572153

When I first began the course at the start of the month, I had [commented] that to me, curriculum is like drawing an Ouroboros; the snake devouring its own tail. The comparison is that like the artist contemplating to begin creating the artwork starting from the head versus the tail, discourse in curriculum follows a similar vein where the argument is over designing curriculum based on “what should be taught” versus “how it should be taught”.

Egan (2020) commented on that former approach in our curriculum where we focused on delivering breadth of knowledge, at the expense of spending time combing the depths of select topics, had a negative impact on students’ engagement. He is clearly favoring a “how it should be taught” approach with this mindset. Having students be able to discover discipline in engaging deeply with a topic will allow them to develop self-motivation and ultimately find joy being immersed. Yet this could be flipped around to justify a “what to teach” approach by narrowing down the list of topics to what students are interested in, or what the teacher feels to be vital tenants of a course. Which leads back to my metaphor of the snake drawing. Curriculum (or course) design is circular in nature as it is difficult to isolate one view from the other if teachers are invested in the students’ best interests. At least, this was my original view.

In “Procedures of Power in Curriculum Discourse: Conversation From Home”, Blades (1995) illustrates how the curriculum designers at Alberta Education gave the final decision of “what should be taught” to representatives from post-secondary institutions. This happened despite having talks with various stakeholders such as teachers and practicing professionals, but curiously, did not include those who would be most affected by the change: the students. Given that BC has recently re-designed our curriculum, I can’t help but wonder if similar exclusionary politics were involved as well. Luckily, us teachers have respectable amount of autonomy in our classroom and are not forced to strictly adhere to every part of the curriculum. However, this simply transfers that privilege of power from the ministry to teachers, where we have our own explicit and implicit biases. The most notable of which is how comfortable we are with inclusion of Indigenous ways of knowing.

Most educators were raised and trained in the predominant Western education system; many of us were uncomfortable implementing Indigenous perspectives into our classrooms. Donald (2009), humbles us by pointing out common fallacies used in resistance to teaching Aboriginal Education (AbEd): disqualification by lacking subject knowledge, fear of disrupting of current practices, or taking a neutral perspective as all views are equal. I admit, I have found myself using some, if not all, of these excuses in my practices thus far. The first rationale is faulty as many teachers are tasked with teaching subjects outside of their specialty or knowledge anyways. Recusing yourself from teaching AbEd is simply what Felman calls “Ignorance… nothing more than a desire to ignore” (Nahachewsky and Slomp, 2009). The second point frames Western and Indigenous education as being antagonistic with one another. This is born from the preconception that there can only exist a singular answer or one correct form of knowledge. Yet much like what we see in art or literature, there can be multiple explanations such as implied or inferred meanings which can further enhance our understanding, as opposed to devalue it. A similar comparison can be made from Western medication and Traditional Chinese Medicine (TCM). Growing up in an Asian household, I’ve been taught about TCM perspectives like how certain foods have specific properties and should be not consumed together if they are opposites. An example would be to never mix spinach and tofu. This interaction was eventually explained using Western scientific knowledge of how oxalic acids in spinach binds with calcium to form kidney stones. While the latter explanation is much more specific and “precise”, due to our bias of science being holier than anecdotal stories, the TCM lens is still valid and has continued to be valuable over the last few centuries. This helps prove that Western education and AbEd can co-exist together to enhance our and our students’ understanding in “what should be taught”.

Shifting focus to “how things should be taught”, this issue has become even more complex with our current restrictions living under the threat of Covid-19. Traditional brick-and-mortar classrooms pivoted online, and teachers raced to digitize their classroom in order to allow students to continue learning. This led to the realization of how schools are ill-equipped to adapt to the current level of available technology, as issues such as access to devices or even internet connection made it difficult to have a “general” approach to online education. At my school, we already offered blended learning so our students had a mixture of in-person and online classes, and should better adapt to the times. Reality was quite different as I discovered providing avenues for synchronous online meetings to substitute the regular in-person class time and keeping the rest of the class resources, were not adequate as students who performed well previously had started to struggle. I realized in reading Ted Aoki’s (1993) explanation of the “lived-curriculum” and “curriculum-as-planned”, that my current courses were framed largely around the latter aspect. There lacked the personal aspect, also called “lived-curriculum”, which I would provide during in-person classes where the class community would chat about the topic, or off-topic, to help make the content more enjoyable and relatable. This brings us back to the Indigenous perspective of how individual identity being constructed by how they contribute towards the community. My isolated online class did not promote social connections between the students, thus isolating them from each other. Class engagement dropped as a result because they could not see how the material is relevant to them, or had the social outlet to at least collaborate to get through it. While the district believes simply providing generic online courses and resources would solve the issue of physically distanced learning, it fails to recognize those products are purposely created without considering the teacher nor students’ identities. Thus, it is incumbent for the teacher applying the course to recognize this “course-as-planned” and adapt it to also incorporate students’ “lived-curriculum”. In essence, being able understand and plan around “how to teach” and “what to teach”.

To summarize, curriculum has much more depth to what I had originally stated. The Ouroboros appears to have an oversized head and tail, disproportionate to the rest of its body. Yet if I consider the body as the bridge of discourse of details between the two aspects of “what” and “how” to teach, the structure becomes more like the infinity symbol (∞). Each loop would represent one of the aspects and all the considerations necessary within it, but then intersect with the other loop to create a holistic structure overall. Thus, in creating future courses, self-reflection will be paramount as I will need to cross-check whether I have successfully connected both views, and whether one has been emphasized over the other.

 

 

 

 

References:

Aoki, T. (1993). Legitimating lived curriculum: Towards a curricular landscape of multiplicity. Journal of Curriculum and Supervision, Spring 1993, Vol. 8, No. 3, 255-268

Blades, D. (1995) Procedures of Power in a Curriculum Discourse: Conversations from Home. JCT, 11(4), 125-155.

Donald, D. (2009). “The Curricular Problem of Indigenousness: Colonial Frontier Logics, Teacher Resistances, and The Acknowledgment of Ethical Space”. In Beyond ‘Presentism’. Leiden, The Netherlands: Brill | Sense. doi: https://doi.org/10.1163/9789460910012_004

Egan, K. (2020). “Learning in Depth in a Franciscan Friary Cell.” www.educationthatinspires.ca/2020/02/06/learning-in-depth-in-a-franciscan-friary-cell/

Nahachewsky, J., & Slomp, D. (2009). “Sound and Fury: Studied Response(S) of Curriculum and Classroom in Digital Times”. In Beyond ‘Presentism’. Leiden, The Netherlands: Brill | Sense. doi: https://doi.org/10.1163/9789460910012_013

Due to the policies established to prevent the spread of Covid-19, traditional brick-and-mortar teaching were forced to pivot to online teaching models. The majority of teachers had to cope with digitizing their resources and learning how to navigate learning management systems (LMS). Teachers teaching at schools that already offer blended or online learning, like myself, continued to utilize pre-built online courses on established LMS, thus can focus on other aspects of online teaching. The direction I decided upon was towards how I could enhance students’ participation in active learning in online environments.

Along with my fellow teachers at our high school, I’ve noticed that students across all subject areas and grade levels were more engaged with the courses during in-person attendance when compared to learning online at home. This issue was further highlighted when we saw students who performed strongly before Covid-19 mandates, begin to slip both in terms of class engagement and academic performance. This discrepancy can be explained using Aoki’s (1993) idea of “curriculum-as-planned” which does not incorporate the “living-curriculum”. While the pre-built courses I use are created by BC Teachers and reflect the new BC Curriculum, they were still designed for “faceless people in a homogenous realm” (Aoki, 1993). When implemented as-is into the classroom, some students would find it difficult to digest the information either due to abstract language or concept, or find it demotivating as they are unable to see how it connects to their lives. Students possess their own living-curriculum” according to Aoki, which is based on their individualities and interests. Previous face-to-face instruction time were successful because I was able to act as the bridge between the two curricula, in order to keep students engaged and motivated. When things moved to being fully online, this connection was disrupted and rarely did the latter curricula present itself. My management of the digital classroom started with good intentions, but ultimately exacerbated the issue of disengagement when students were learning at home.

One of the biggest perceived advantages of online courses is the flexibility of learning anytime and anyplace. Sheail (2018) and Kirkwood (2000) argues against that notion as students have limits in both time and space for which they can participate in online learning. Some challenges include having proper hardware like laptops and a stable internet connection to participate in digital classrooms. Students need to parley with their family on when and who has the highest priority in terms of accessing those resources. With these barriers in mind, I created optional drop-in times for synchronous meetings to avoid conflicting schedules for students and their families. They would interact with course readings and activities asynchronously as before and could ask me through video chat if they needed individual support. Yet this resulted in students becoming isolated in the online classroom. The First Nation’s perspective of societies is that its members see themselves as being “synecdochic rather than more… metonymic (Weaver, 2000)” (Donald, 2009); individual identity is driven by how a person see themselves contribute to community. My methods had accomplished the opposite where students only see and interact with the course content. They were simply another person in a course, like a replaceable cog in a machine. Lack of motivation to engage in the course becomes understandable as they continue to treat the concepts as being abstract, with no cause or effect in their daily lives. If I were to work backwards from the end goal of having students actively engaged in online learning, I need elicit a sense of curiosity or interest in the topic at hand. To do this, I need students to see how the topic connects to our daily lives in some form. Lastly, I can draw upon the collective class community to help students feel connected and share their living-curriculum as a starting point to guide discussions towards course topics.

Reflecting upon my failure in the past school year, my shortcomings have been rationalized through perspectives shown from the literature. Offering freedom to engage in the class asynchronously and the lack of promoting shared community spaces for class use were the most likely reasons for why students did not succeed online. The latter factor will require exploring meaningful, easy-to-use, district-approved tools that allow for both synchronous and asynchronous discussion; and promote its use in the course. The former point will require scaling back the freedom by designating short, but mandatory, meetings times to ensure students are not being isolated in the online classroom. I strongly believe that these two changes will be beneficial in engaging students in active learning online, both during and after this period uncertainty what education will look like going forward.

 

 

References:

Aoki, T. (1993). Legitimating lived curriculum: Towards a curricular landscape of multiplicity. Journal of Curriculum and Supervision, Spring 1993, Vol. 8, No. 3, 255-268

Donald, D. (2009). “The Curricular Problem of Indigenousness: Colonial Frontier Logics, Teacher Resistances, and The Acknowledgment of Ethical Space”. In Beyond ‘Presentism’. Leiden, The Netherlands: Brill | Sense. doi: https://doi.org/10.1163/9789460910012_004

Kirkwood, A. (2000). Learning at Home with Information and Communication Technologies. Distance Education, 21(2), 248–259. https://doi.org/10.1080/0158791000210204

Sheail, P. (2018). The Digital University and the Shifting Time–Space of the Campus. Learning, Media and Technology, 43(1), 56–69. https://doi.org/10.1080/17439884.2017.1387139

As educators, assessment is a large part of our job and there are a vast number of tools and pedagogies available. Therefore, as a group we chose to look at the specific aspects of assessment and online assessment tools for the current time we are in, during the COVID-19 pandemic. We first created a set of learning outcomes for assessment (see below), then our curated list and finally we collaborated on a blog post. The Key to Unlocking Assessment seeks to provide educators with ideas of how to transition into the blended and online learning environment that will likely be our reality in the fall. These sources, strategies, and tools were chosen with a critical lens in order to provide practical resources for the everyday educator. The remainder of this post provides our rationale behind this curation. We focused on a range of resources from connections to ministry curriculum, different types and formats of assessments, tools to conduct them, and opportunities for professional development. To achieve this, we examined government and university articles, commercial enterprises, as well as public/private blogs. The CRAAP test was applied to each entry, to evaluate whether they were: Current, Relevant, Accredited, Accurate, and Purposeful. There were only a few resources in our curated list that did not meet every aspect of the test, however they were included for the following reasons.  

1. Old ministry document: This document is not considered current as it was created in 1994. However, the PDFs are still useful as a starting point for educators; although some adjustments will need to be made in order to address updated curriculum. Included in the list are the updated curriculum resources. 

1. Raising Standards Through Classroom Assessment: A 2001 summary of a 1998 literature review that provides a detailed outline of the important building blocks of assessment by answering the questions of why assessment is important and how it aids learners. It focuses specifically on formative assessment for learning. Although the literature review was written over 20 years ago the information provided is still applicable to today’s learners and educators.  

1. Assessing the Online Learner: Does not contain a specific date but does contain referenced work from 2007, therefore we can reasonably conclude that it was written sometime after this. The information shared is applicable, purposeful, reliable, accurate and could be a very useful resource for teachers.  

Learning Outcomes for Assessment: 

• Identify self-assessment tools that allow students to engage with content through:  
  • Explanation 
  • Justification 
  • Personal connection  

• Build and apply rubrics and scales to assess curricular competencies.  
• Investigate and utilize various forms of formative and summative assessment for online learning environments.  
• Develop a variety of assessment methods to engage students in a subject area using technology. 
  • Consider teaching/learning environment (online vs blended) 
  • Personalized to consider student needs.  
  • Voice and choice 
  • Use assessment to inform instruction 

• Construct timely assessment or appropriate support to ensure student success.  
  • Check-ins 
  • Support Blocks 
  • Office Hours  
  • Communication Home 

 What’s Next? 

Our next step is to create resources and evidence informed content that will authentically assess student understanding, in our new learning context. Additionally, we aim to aid educators in developing their own assessment skills, strategies, and tools, that will fit both in-class, blended and online environments.  

 

Co-Authors: Faune, Jerry, Leanne, Rhyanon and Rochelle.  

 

With the majority of districts shifting towards distance learning thanks to Covid, there have been plenty of interest and questions around how we implemented our Blended and Online program. The quick answer is – it’s a work in progress. The framework or platform (Learning Management System, or LMS) may be in place, but it is not ideal. The simple solution to all would be providing the association which we get our pre-made courses from (it’s Western Canadian Learning Network, aka WCLN) and offering troubleshooting support for those learning to operate Moodle as an LMS. This “here is the resource, good luck figuring it out” is basically the same treatment given to us when we started.  I always wonder why some teachers think we’re better off because “we have everything online already”. There is nothing stopping them from simply scanning and uploading their worksheets, nothing fancy about rotating the camera on your phone to record your lectures, and nothing different about recommending Youtube videos for students to watch. If they’re not interested, then they too can access the same readily available online courses we do and join us in suffering the same issues: restrictions to content customization, broken links, random updates, missing/incorrect (teacher) resources, and of course, good intention for changes which lead to unintended catastrophic failures. It reminds me of a Reddit joke:

99 little bugs in the code,
99 little bugs in the code.
Take one down, patch it around,
123 little bugs in the code.

We’re not even that well versed in synchronous online meetings. Our blended program has mandatory face-to-face instruction so students who are struggling with learning at home still come in-person every week for help. I imagine this is the same issue that most brick & mortar teachers face; students who don’t do any work at home so that 70-80min block every day is the only time they make any progress. We have it worse – that one block once a week is the only time we connect with them. With Covid moving these supports online, these students lose out even more because they never engaged in at-home learning in the first place. Our district also recommends teachers to use Microsoft Teams of BlueJeans or Zoom for these online meetings. Not only are we unable to help you operate those proficiently, we also can’t help with Teams because we were introduced to it at the same time as everyone else.

Then there is the myth that our students are somehow better equipped to deal with distance learning because they’re already doing it. Students who enroll in these program are not all tech wizards. Like all other students, they exhibit a diverse spectrum of technological literacy. We have students who download, print, and submit hardcopies despite our marking being done online. We have students who swears by Apple devices and struggle to hand in assignments because they don’t know how to convert it to Word files; even though our school provides 1:1 Microsoft device. They also complain that resource links are broken despite repeated reminders of DON’T USE SAFARI. I even had a student who came in late-May and asked why I was surprised they didn’t  Teams setup; they thought everyone was just going to pass thanks to Covid.

To me, curriculum is Art. If one sits there contemplating what it could look like, they will have nothing but empty space to show for. Recognize there is discourse in curriculum design regarding what to teach versus how to teach it. Experiment with it; use different lens and reflect on the results. Seek feedback and consider what improvements can be made. The final product might be a marvelous masterpiece, or it could be an ugly blob – and that is okay. Teachers do not start out perfect, otherwise we would all be at highest salary on the pay scale. We learn from our mistakes and seek ways to improve our practice over time.

 

 

 

 

 

 

 

 

 

 

 

The Ouroborous design shown above was specifically chosen in this metaphor because curriculum design feels circular in nature as well. It symbolizes what Egan (2003) touched upon in the history of curriculum discourse where the focus should be on “what should be taught” versus “how it should be taught”. Do I begin with drafting the head or the tail of my snake? Pottle (as cited in in Egan, 2003) cautions that over-contemplation would waste more time than simply doing both. So, let’s take a chance start from the head followed by the body and tail in this piece, which are the contents of a curriculum. The rationale behind it could be as simple as wanting students to be able to distinguish and create different parts of the body. Afterwards, if they wish to create a palindromic creature or repeated array of a single part, that would be perfect and creative extension of the knowledge they have learned. With a clear picture of where the head, body, and tail are, I can then flesh out the details such as the position of parts, the specie being drawn, or perhaps texture of the model. This is the “how” portion of the curriculum. For instance, I could provide students with a blanket circle and ask them to arrange three unique parts within the circle and the connect them. Or, I could provide them with pictures and ask them to sketch or label the parts.

Reflecting on which approach had “better” results becomes easier as you obtain data, be it measured or anecdotal. There should also be student feedback to help direct the goal or process of the curriculum as they are the ones most impacted by it (Blades, 1997).  While us teachers in BC may not have a voice in our curriculum, if the same process occurred for us as it had with Alberta Education then we at least have the autonomy to designate outcomes we want to include or exclude in the classroom.