How to Assess the Learning Process, Not the Final Product
Let’s talk about assessment, and grading. There’s something going on, and it is impacting all kinds of institutions.
The crazy thing is, it's not only Harvard. Head over to https://gradeinflation.com/ and check out all of the statistics. Grades have been steadily "improving" every single year with some very big jumps.
Compare that with the average SAT scores, which was documented in the book, Measuring Success: Testing, Grades, and the Future of College Admissions edited by Jack Buckley, Lynn Letukas, Ben Wildavsky -- and what you'll see is a trend that is alarming.
Average GPA goes up! Average SAT goes down...in the same time period.
Why is this so important (and why does it bug me so much, when maybe it shouldn't)?
Because much of our K-12 system and Higher-Ed is based around one specific type of measure: Final Product Grades.
These summative grades are given at the end of a test, quiz, unit, project, essay etc.
While the grades keep rising, what we are really missing out on is whether or not real learning is occuring.
The learning happens during the actual process, not in the result of a final product.
When we only assess the final product, we miss so much value in the learning -- and ultimately fall short of providing valuable and useful feedback that is needed to support learning progress.
Now enter Generative Artificial Intelligence into the mix. It’s never been easier to create final products such as essays, slideshow presentations, videos, and so much more.
The problem, is if we only grade the final product, we can’t be sure what learning happened with our without the help of artificial intelligence.
In a lot of the work I do with schools, we look at the role of Project-Based Learning, Design Sprints, and other Inquiry-Based learning experiences that are not so easily assessed as multiple-choice tests, quizzes, and five-paragraph essays.
Some of the best ways to assess this type of learning, is during the process. Here are seven ways you can assess the process and not the product:
1: GRIT Rubric
I’m a big fan of project-based learning, problem-based learning, and inquiry-driven learning (hey, I even wrote a book about inquiry). But there is a secret (and big) problem with project-based learning.
Chris Lehmann (Principal at Science Leadership Academy) famously said, “If you assign a project and get back 30 of the same thing, that’s not a project, that’s a recipe.” And therein lies the problem with projects. When we assign projects we tend to have an assignment with guidelines, steps, and a rubric for how it is going to be graded. Often we show examples of previous projects that received high marks.
Think about what you would do if you were a student….no, really think about it for a moment. Yep, it is exactly what I would do. I’d create a project that met the specifications and guidelines, and hand in something I knew would get a high grade on the rubric. And then I’d have a project like everyone else.
I flipped that entire project-based learning process on it’s head when I did the 20% Time Project with my students. They had to create their own project (and product) and document the process. But I didn’t grade their final product. I did not give them a rubric for that final product. Instead, we graded students on the process.
For this, we used the G.R.I.T. Rubric developed by College Track (it’s way too awesome). This assesses students on the PROCESS not the final PRODUCT by measuring their Guts, Resiliency, Integrity, and Tenacity while working on the project:
2: Screencasting Your Thinking and Your Doing
Imagine getting a sheet with 20 math problems, solved, and with the steps. Maybe 10 years ago this would show the thinking and work, but now with PhotoMath, Wolfram Alpha, and ChatGPT -- all those steps could be AI-generated.
Enter screencasting.
Here you have students screencast them solving 2-3 math problems, talking through their learning process, showing their work, and where they are struggling. By hearing, and seeing, student learning in action - we are able to assess where they are at right now --- way more accurately than seeing only the final product.
3: Video Annotations on Any Work
Let's take screencasting to the next level. Video annotations can take any student's work (lab report, essay, analysis, project, etc) and allow the students to talk through their learning.
They can annotate with their voice, and using drawings, stickers, and other video editing tools. The goal is not for the video to come out as a beautiful final product, but instead be a reflection on their learning so far.
4: Create Process Rubrics and Conferences
Using detailed learning goals as the formation for these rubrics, learners will be able to continually see (both through self-assessment and conferencing) where they are on a continuum of proficiency and mastery.
Honestly, ChatGPT has been fantastic at helping me create these types of process rubrics. It will even put it in a table for you!
Then you have conferences with students to assess and give feedback on their learning during the process.
I'm a big fan of my co-author John Spencer's "4 Types of Learning Conferences" and have used these with all kinds of students over the years:
5: Failing Reports
When my students were doing the 20% Project (Genius Hour) in my class we had an “Epic Failure Board” (inspired by a number of people) where they would pin up some their biggest fails and epic risks.
The 20% project required each student to challenge themselves. They were learning and creating with a purpose, often with lofty expectations and goals…and failure came at every step.
In the first month of the project, I could sense hesitation from many students who did not want to give 100% effort with the possibility of 90% failure. The Epic Failure Board changed the classroom culture from one that shied away trial and error, to one that supported and even praised risk-taking.
I know there are many different definitions of failure, but as a class we adopted a mantra: Sometimes you win, and sometimes you learn.
This is a fantastic way to frame mistakes, missteps, and struggles. I’ve also used this in my personal and professional life as an author, speaker, husband, and father.
Have students share what went well, what they learned, and where they struggled -- you'll learn more about them and their learning through this process.
6: Benchmark Shark Tank Presentations
I used to think Shark Tank presentations were for the end of the unit/project. However, when you put them right after the idea generation phase of a project -- it changes everything.
When students in my school have their Shark Tank pitch day, they get to share with the entire class what they are working on. Publicly announcing what they are trying to accomplish makes the goal real. Students get to see what their peers are working on and want to make sure their project stands up to the rest of the class.
7: Portfolios
The final (and maybe best) way of assessing the process is through portfolios.
George Couros shares that there are two types of portfolios that work really well to assess learning:
When I started to look at what was the best place to start a digital portfolio, we looked at two types of portfolios; a learning portfolio and a showcase portfolio. Here is the easiest way to differentiate the two in terms of student learning:
Learning portfolio – If a student were to take a video of them reading in four consecutive months, you would see all readings over time to see development and growth.
Showcase portfolio -If a student were to take a video of them reading in four consecutive months, they would pick the best one from the four samples. What is beautiful about using a blog as a portfolio is that you do not have to choose; you can do both.
There are many different options for portfolios, but I think the seven questions George lays out in this blog post, set the stage for what it can do for both you and your students:
Is this a learning portfolio, showcase portfolio, or combination of both? – Does this show the student’s progression over time (learning), or just the best stuff (showcase). There are huge benefits to both for learning and opportunities over time. A combination of both in my opinion is best.
Who owns the learning? – Is this a portfolio that only shows “school” work, or does the student have the opportunity to display what they are passionate about, or is it simply for items to be displayed based on what the teacher wants? Is it a combination of both? If the student feels no ownership over the process and product, the results will not be as powerful as if they do.
How will it be exported after the process? – For starters, see the question above. Secondly, if there is no plan to ensure that students have the opportunity to put all of this learning into their own space eventually, you are missing another opportunity that digital provides.
How will you make the audience eventually go global? – A lot of parents and educators are worried about the work of a student getting “out there” (for various reasons), but if the portfolio is only available upon request, we are taking a very “paper” mentality, to a “digital” platform. This does not meant the whole world has to see everything from the beginning, or the student needs to share it with the world if they do not want to, but the progression plan to share it with the world should be there. Will the audience be limited long term?
What brings people to the portfolio? – Is there any mechanism that brings people to the portfolio other than telling people to come? Simple things like email help to build an audience. Is the portfolio more likely to be seen and more valuable to the learning if it goes to people, other than people coming to the portfolio?
What impact will this have on the learner’s digital footprint? – Will Richardson suggests that by the time kids graduate grade 12, you should be able to google them and find good stuff about them (see image at the top of the post). Does the portfolio help in this endeavour when every student we work with now will be googled for jobs, university, or a myriad of other things.
What about next year and other classes? – This is a HUGE question. If the portfolio only lasts for one year, then you are missing a great opportunity. What professional learning is in place for teachers to support a connection of learning over time for the students? What will the students' work look like over time and how will they be able to google or search for their own learning? If the plan is not in place to grow this over time, we lose so much from the process.
You can also add all of the above ways to portfolios to mix and match!
Closing Thoughts
This isn't easy. It is much easier to talk through a powerpoint, give a multiple-choice test, grade the answers, and move on.
This isn't perfect. Assessing the learning process is often subjective and very human. Conversations have to happen. The continuum may look different each day.
But it is worth it.
In a world of artificial intelligence, our learners have so many tools at their fingertips to help them showcase amazing final products. It can take them minutes to do what it used to take hours and days to do just a few short years ago.
If we want to continue to challenge and engage our learners, we have to not only provide meaningful learning opportunities but also relevant ways to assess the process (not just the final product).
