An Overview of Digital Assessment and Education
Perspective
Overall, the experience of learning about assessment and all that it entails became more intriguing as we moved further through the semester. Assessments have been part of my educational career from the beginning. As a student I was assessed using various methods which included the basic scantron based exams to the more robust essay-based assignments. Each of these have their place in assessing students and how they are progressing through the semester.
As an instructor, I have given many assessments and like most, started with the traditional multiple choice/true-false option. These are the mainstay for most classes; however, they are not always the best method of measuring students and how they are moving along. As digital assessments became more prominent, they provided many more options for gathering data at the class level, the department level, as well as the institutional level. The assessment process is all about obtaining data and using to increase student success.
Student success is driven by understanding where they are and where they may end up. One of the key concepts that I found to interesting was Interim Assessments, the process of smaller more frequent assessments to have a better understanding where our students stand. These interim assessments allow instructors to adjust and revise their teachings to ensure that students are keeping up with the material and more importantly, understanding the material, rather than just memorizing. Interim assessments allow intervention of the instructor before it is too late.
I have come to enjoy developing new ways to measure student knowledge and how they are progressing. Moving away from the traditional paper exam and into the digital world has allowed new methods of assessment. I can now mix the low stakes weekly quizzes, response-based discussions, and higher cost exams all in the same class using the tolls included in the Learning Management System (LMS). The ability to deploy an assessment at any time in the LMS has provided more class time for instruction, follow up, and discussion on topics that may have been missed using traditional assessments.
For example, using digital assessments has provided me two more weeks of engagement that would not have been possible using traditional in class exams. Furthermore, using a digital assessment provides almost instant feedback to students and greater flexibility to complete online exams, quizzes, and/or discussions at their own pace and schedule. Most of the feedback provided by students is that they felt more at ease taking the exam or completing the assignment at their discretion with less distraction.
Overall, assessing students and using the data we gather should allow us to further improve our methods. Using this data to drive our decisions and how to best teach our material should lead to a more streamlined process, more engaging content, and less time spent sifting through unused data at the institutional level.
The Importance of ESSA
Assessing a student’s performance each semester is an integral part of education and like everything else in society, is always changing. From the traditional paper based assessments to the ever evolving digital assessment, measuring a student’s progress throughout a class is an important task. Over the course of the session, we have been provided readings concerning the importance of the Every Student Succeeds Act (ESSA) and its evolution from the No Child Left Behind Act (NCLB).
ESSA was signed into law by President Obama in 2015, as a direct successor to the NCLB, as it was no longer able to meet the needs of a changing population, and better prepare students for future career needs (Anonymous 1, 2020). The initial plan for the ESSA was to develop a higher level of assessments that would provide better data to educational institutions, as well as local stakeholders such as parents and the surrounding the community (Anonymous 2, 2020). ESSA also provided states the ability to create their own assessment methods that best fit their student populations and their needs. As states are now able to create their own assessments, they are able to meet the needs of their communities rather than being restricted by national assessment methods.
Computer Based (Digital) Assessments
Simply, a computer-based assessment is one that a student completes on some form of a computer. This can include a desktop, laptop, tablet, or their smartphone. The use of a digital assessment removes, for the most part, the pencil and paper of traditional exams. Digital assessments have replaced the pencil and paper methods in much of academia as well as our daily lives (Mishra, 2023). The benefits of moving to computer based (digital) assessments are varied. These include convenience, ease of use and distribution, efficiency, and analytics (Mishra, 2023). As noted previously, digital assessments allow for a variety of different data collection methods.
We can offer students options on how they demonstrate their knowledge of a subject using such things as online portfolios, video submissions, blogs, and the development of online resources. Differing the options to students and how they submit an artifact allows for the different learning styles. As for institutions, the move to digital assessments creates the opportunity to save time and money on printing, copying, grading, and handing out the results back to students. as well as money on each of the resources used to create the assessments. Instructors are able gain the time back from "test day" and move assessments online for students to take at their discretion.
Standards Based Grading
Standards Based Grading (SBG) in the process to measure student's proficiency in meeting outcomes rather than earning specific letter grades (Townsley, 2014). The move from the traditional points based scoring system and overall grading to the standards based grading process will be a fundamental change in how we view students’ knowledge on a topic. SBG is a holistic view of the level of knowledge a student has obtained over a certain topic or subject.
Rather than being relegated to a time-based format, students are assessed using mastery of a concept as a goal. Using standards-based grading, we allow students to show exhibit creativity, individualization, and create a positive learning environment (Townsley, 2014). Standards based grading represents a paradigm shift in how students are measured and provides teachers flexibility in how they assess.
Online Cheating
When given the choice to discuss Online Cheating during the Week 7, I was more than willing to discuss some of the techniques I have utilized over the course teaching online. I will summarize a theory from Robert Merton, that I included in the written discussion that week. He developed a theory referred to as Strain Theory or Goal Attainment Theory. He argues that when an individual cannot attain or achieve their goals, they will find alternative means. Merton believes that when an individual finds the traditional means of attainment either too restrictive or too cumbersome, they will look for other ways to reach their desired end. Cheating becomes an option when students know that they can just reach for their phone and Google It (Ferrell, 2015).
For students, the end goal is to earn a passing score. Students in face-to-face classes tend to be restricted on how they can cheat, but online students have more resources available to them. As I went online early in my career, I tried to find ways to restrict the student's behavior; however, I soon learned that that it was going to be a hard road ahead. I decided to just let them have at it and do what I could to lessen the impact of cheating and allow students to use what they wanted. There are several techniques that can used to help alleviate the need to cheat, these include: limiting time frames for completing an exam or quiz, use class-based discussions as references for questions, and use questions that cannot be easily looked up (Ferrel, 2015).
These techniques can be useful when trying to deter cheating but will not guarantee that they will not cheat. I argue that the more often a student looks at the relevant material, they will learn it or at least become more familiar with it. Letting students know that they can use whatever material they have may have a positive impact when preparing for an exam, quiz, or whatever type of assessment is being given.
Computer Based Text Analysis
Computer Based Text Analysis (CBTA) or computer assisted grading has been referred to as many things including, Automated Essay Scoring (AES), Automated Essay Grading (AEG), and Automated Writing Evaluation (AWE). The idea behind this technology is to use computers to aid in grading by using text recognition by scanning the data and providing results based on pre-defined content (Sinclair & Rockwell, 2012).
Within the world of CBTA, there are two types of analysis – Text in Isolation (TII) and Corpus Based Assessments (CBA). Briefly, TII does not require a large body of previously graded materials, while CBA does. TII programs look for specific characteristics from previous works such as essay length, specialized words, and the use of pronouns, amongst other things (Aken,2017). However, CBA utilizes large bodies of graded material to compare and will score certain essays based on similarity in work. These two approaches to text-based analysis and assisted graded can take advantage of modern computing devices, provide speedier feedback to students, and allow instructors to spend more time in curriculum development.
Development of Outcomes
Developing outcomes to be used in a classroom is a collaborative effort that needs to be completed by a group of educators. This process can be daunting when trying to create specific set of outcomes for a department as well as multiple instructors. I was fortunate to have a set of outcomes already created due to teaching Introduction to Sociology at a local community college.
Using these outcomes, I was able to create a small set of questions, ten in total, and align those questions to the outcomes using Canvas’ built in alignment tool. However, this tool does not allow for aligning a specific question to a single outcome, but you are given the ability to use the outcomes as a rubric for a question that needs graded after it has been submitted, for example, an essay or short answer-based assessment. The use of outcomes allows instructors to have an end goal in mind and teach towards meeting those outcomes.
Once students have completed the assessments and the instructor has parsed through the data, they are able to determine if the student have met the appropriate level of mastery based pre-defined characteristics. Using the compiled data for each outcome, the instructor is given a glimpse into what material will need to be rehashed or revisited.
Using the same data for multiple classes we can see what instructor may be lacking in covering the material. This encourages collaboration among instructors, to share resources and methods to ensure success among the students. Below is an example of gathering data from multiple classes using an external site. Each response is anonymous response buy provides data across the department.
We can see that 93% of those that have completed the assessment have answered correctly, which meets the standards developed by the department. Withing each outcome there should be a threshold of what is considered a success and if the results show that students are not meeting that threshold, this is a reason to revisit this information going forward and delve into what material need to be addressed.
In conclusion, developing and measuring outcomes allows educators to make sure that what is expected to be learned over the course of a semester is being learned. We can use the outcomes as a measure of what students have learned, but also where our own deficiencies are. Assessing outcomes is not just a measure of student success, but also a measure of instructor success.
To use a personal example, we are required to assess specific outcomes each semester. As a program coordinator, it is my responsibility to coordinate this assessment and report the results to administration. We typically assess four courses and a specific outcome associated with that specific class. I will ask for input, usually in the form of multiple choice/true false questions, from instructors who teach those classes. I will then use those questions to build an online survey and provide the link to the instructors. Using the online survey site cuts down on the time required by instructors, as all they need to do is provide a link to the students and the results come back t0 me. Once the semester is over, as a department, we can review the results and look for deficiencies. I have provided a sample from our Fall 2022 Assessment for Sociology 1301 - Outcome #4 (SLO#4).
Closing Remarks
The process of assessing students is a complex one. There are several questions to address such which methods of assessments are suitable, how to assess differing learning styles of students, and how to measure the outcomes. Assessment has become a hot topic among academic institutions as well as the state. Using standardized outcomes is a way to measure various levels of academia to make sure all populations of students are meeting expectations. This type of measurement attempts to make sure that students of certain population groups are not left behind relative to other populations.
Revisiting each of these topics over the course of this blog, reminded me that the assessment process is a complex one. There are many facets to assessments that have to be taken into consideration. Reading about the various approaches to the assessment process, only showed me how much I need to learn. From the move to ESSA from the NCLB, the transition to Digital Assessment, Standard Based Grading, Computer Based Text Analysis, and Developing Outcomes each of these topics is a unique part of the assessment process and they will continue to change over time and will need to be re-addressed.
Works Referenced:
Aken, A. (2017, October 9). An evaluation of assessment-oriented computer-based text analysis paradigms. Higher Education Research. Retrieved March 15, 2023, from https://sciencepublishinggroup.com/journal/paperinfo?journalid=296&doi=10.11648%2Fj.her.20170204.12
Anonymous 1. (2020, April 14). The Elementary Secondary Education Act (the every student succeeds act of 2016). Home. Retrieved March 14, 2023, from https://www2.ed.gov/policy/elsec/leg/essa/index.html
Anomymous 2. (2020, October 28). What is the Every Student Succeeds Act? Office of Elementary and Secondary Education. Retrieved March 14, 2023, from https://oese.ed.gov/families/essa/
Farrell, M. (2015, March 19). Want to stop cheating on online quizzes? ....... let them cheat! OLC. Retrieved March 15, 2023, from https://onlinelearningconsortium.org/want-stop-cheating-online-quizzes-let-cheat/?
Mishra, P. (2023, January 30). What is a computer-based assessment? https://www.proprofs.com. Retrieved March 14, 2023, from https://www.proprofs.com/quiz-school/blog/what-is-computer-based-assessment/
Sinclair, S., & Rockwell, G. (1970, January 1). 10. teaching Computer-Assisted Text Analysis: Approaches to learning New Methodologies. Digital Humanities Pedagogy - 10. Teaching Computer-Assisted Text Analysis: Approaches to Learning New Methodologies. Retrieved March 15, 2023, from https://books.openedition.org/obp/1644?lang=en f
Townsley, M. (2020, February 27). What is the difference between standards-based grading (or reporting) and competency-based education? Aurora Institute. Retrieved March 15, 2023, from https://aurora-institute.org/cw_post/what-is-the-difference-between-standards-based-grading/
Blog Video Recap
Canvas Assessment Questions and Outcomes
Real World Assessment Outcome Results
Fall 2022 Houston Community College Sociology SLO#4
Comments