Tuesday, 28 May 2013

Damned Statistics & Digital Meta-cognitive Opportunities

Tweeting my mouth off
Education would rather  focus on arbitrary and fabricated data, like graduation rates.  It's easy to increase graduation rates, just lower standards.  It has been working for Ontario Education for years now.  You barely have to even attend a class now to get a credit, and if you fail? A teacher not even qualified in the subject area will pass you along; we call that student success.  The grade eleven university level English paper with no less than three grammar and punctuation errors IN THE TITLE was an example I saw of this.  It was given a 78% by the credit recovery teacher grading it.  That failing student will now go on to university thinking that they are an 'A' student (they went from failing badly to 80%!).

There is another way.  Rather than chasing our own tails by trying to improve statistics that we create ourselves, why not start harnessing data that is actually useful and relevant to students beyond the context of education?  Digital technology offers us a fantastic and under utilized avenue for collecting meaningful data on student learning; data that might actually help them beyond the walled garden of education.

Rather than addressing the distraction caused by digital devices, we ignore them, or try to ban them.  Even at our best we only tentatively use digital tools, and when we do we ignore the data they could be providing on student activity.  Digital devices could shine a powerful light on student learning, instead we call them a distraction and let students abuse them into uselessness.  Effectively harnessing educational technology could give us granular, specific data about student activity in the classroom, yet we choose to wallow in darkness.  Really useful data-driven learning is only a decision away from implementation.

Education, like so many other sectors, has become increasingly interested in data driven management.  I don't have anything against that on principal, in fact, I'd rather be managed according to logic and fact than the usual management ethos (egomania and paranoia).  Where we go wrong with data driven educational reform is where human beings are involved.  Education, more than most fields, prefers not to reveal its inner workings.  The choices made on what data to collect and how to present it usually revolve around a sense of self preservation rather than a focus on student success.  The only data we collect is data we can control for our own ends.

The intent of the education factory is to reduce something as complex as human learning down to a percentage.  That in itself is about the biggest abstraction you could devise, what Twain would call a statistic in the truest sense.  Those numbers are ultimately useless in anything other than education.  The only time in your life your grade will ever matter is if you're transferring from one educational institution to another.  No will ever ask what your marks were once you're out of school.  They don't even ask teachers what their marks were before hiring them; even educators realize how meaningless grades are.

Instead of spending all our energy fabricating meaningless statistics in the form of grades, imagine harnessing all the data that flows through education technology and presenting it in a radically transparent reporting system that connects students to their lives after they graduate. That system would provide students with a powerful tool for metacognitive review around their own learning, and their use of digital tools.  Instead of reductive grades and empty comment banks, why not offer an insightful statistical analysis of how a student uses digital tools as they learn?  The tools themselves are eager to share this data, it is only educators who are stopping it.

A student who is shown, in specific detail, why they failed a course (but watched oh so many fascinating youtube videos), is being shown their own poor choices in stark detail.  One of the great joys I have in elearning is showing students their analytics.   When I get the, "I don't understand this!" line, I ask for specifics, which usually gets me a, "I don't get any of it!"  I can then pull up an analysis of what lessons the student has attempted.  The student who didn't bother to actually even try any of the lessons gets wonderfully sheepish at this point.

With meaningful data on hand about their poor choices, education's arbitrariness instead becomes a metacognitive opportunity to adjust learning habits; something we seem loath to do on digital tools, even as we criticize how students use them.

Collecting meaningful student data would allow us to connect the abstract world of education with what students will face on the other side of graduation, especially if we continue to collect data after they move on.  Ever wondered what high school courses are actually useful (and I don't mean in graduating, I mean in finding work, being useful, living a good life)?  How about a live stat attached to each showing employablity based on course choice?  Think you'll move over to applied level English because your friends are in there and you don't like doing homework? Welcome to a 14% higher unemployment rate, and a 6% higher criminality rate!  Imagine what parents and students could do if this kind of data were available.  Realizing that there are real world consequences to your educational choices would do much to remove apathy and a lack of engagement on the part of students.  Education has very real consequences beyond school but we seem intent on trying to remove any obvious connection between education and the rest of a student's life.  With open learning data we'd have way fewer students who have missed the starting gun.

Last year my school talked about creating a cosmetology program.  This would be a hugely expensive undertaking requiring changing the face of the school.  That was OK though because the board was willing to throw tens of thousands of dollars at an increase in graduation rates for at risk girls.  What would they do with it once they were out?  It made me want to start up a video game program, not because it would do anything helpful, but because it would fill sections.  We subvert usefulness in a desperate attempt to game graduation statistics.

I couldn't help but think of the college computer engineering program I'd been to see a few months before.  They had a 100% placement rate for grads with starting pay well above the Canadian average, but they couldn't find enough people interested in the field to run a full course each year.  They didn't have any females in the course at all, and were desperately trying to get more women interested.  I can't find enough kids in my high school to run more than one combined senior computer engineering course... in a field that all but guarantees a good job when you graduate and is about as future proof as you could wish.  I don't imagine cosmeticians are walking into that kind of employment certainty at high rates of pay, but a future out of school isn't what we're aiming students at, we're just concerned with graduating them.

It sounds harsh, but one of the reasons students are so disengaged from school is because they recognize the cognitive dissonance between the world beyond school and the fabricated reality we keep them in until they turn eighteen.  If you want students to engage in their educations provide them with metacognitive data that actually helps them.  Education has gone to greater and greater lengths to try and protect students from themselves and the 'real' world, all to chase fictional statistics.

Digitization in the classroom offers us access to meaningful data on student learning behaviour that was impossible even ten years ago.  Instead of being ignored and treated as a distraction, we should be harnessing digital technology and communicating that data.  A student who spends less than 10% of class time working on their project before failing it?  If that data were included in assessment, a student would have a metacognitive opportunity to understand the mechanics of their own failure.  They might then also begin to harness digital tools rather than being distracted by them.  Digitization shouldn't be an escape from accountability, it should amplify it.

In such an environment, assessment might become something more than a damned statistic.


I didn't even get into how this data could serve employment after school.  Detailed data on how students tackle work would be of great interest to employers.  Even the basics like attendance and ability to focus on work would be of more interest to employers than any grade.

Imagine an Ontario Student Record that offered employers an automated resume that included attendance and other useful details like ability to complete work in a timely fashion, group/team skills, communications and approach to new challenges.  Instead of hiding education behind a curtain of graduation, we could begin to make it immediately and obviously connected to future success.