Okay everyone, take your seats — we have a lot of material to get through today. What’s that? When’s recess? Where do you think you are, Finland?
Today, as promised in the headline, we’re going to look at the promise and perils of Big Data in the education sector.
Common System
Nova Scotia officially launched its student information system — brand name iNSchool, because nothing screams “education” like jamming two words together and randomly inserting a capital letter — under Darrell Dexter’s NDP government in the fall of 2009.
Preparations for the launch had begun as early as 2005, with school boards working with the education department to develop the “functional and technical requirements” and deciding to “acquire commercially available systems.” In her overview of the system for the Standing Committee on Public Accounts in April 2015, Sandra McKenzie, deputy minister of Education and Early Childhood Development, explained:
Program goals and objectives were agreed to, common processes documented and data standards were established so that a common system could be developed for our more than 400 schools.
The first two applications chosen for the iNSchool system were PowerSchool and TIENET (Technology for Improving Education Network). These applications which, in passing, are now owned by the private equity firm Vista Equity Partners, were fully implemented by June 2013.
The best description of iNSchool I’ve found came from the May 2014 report of Nova Scotia’s Auditor General:
- PowerSchool: This application manages core student information, such as report cards, demographics and schedules. It includes an internet-accessible portal enabling parents and students to see current information on matters such as attendance, grades, assignments, teacher’s comments, and upcoming school events. The system maintains records for approximately 120,000 students.
- TIENET: This application manages extended services for students with additional needs, such as individual program plans and information from the SchoolsPlus program, which provides additional supports and services to students.
- Learning Management and Reporting System: The scope of the Learning Management and Reporting Project is to support the learning and teaching function with a complete and integrated solution. It includes curriculum and resource management; instruction, classroom assessments and evaluation; Provincial and common assessments; and teacher professional learning. This application is expected to be implemented at a later date.
The AG had audited the planning, design, procurement and implementation of iNSchool and whether the system and the information in it was “adequately controlled.” The answer to that latter question was, “no.”
We were able to gain unauthorized access to many iNSchool user accounts and student information contained in them. We found several appropriate network controls, but we found security weaknesses at the operating system, database and application levels.
This is what I thought I was going to be writing about this week — secure storage of student records. But while you can never be completely sanguine about the safe storage of sensitive information, it turns out that this is the least problematic aspect of data collection in N.S. Schools. Therefore, I will discuss proper storage of student information later in the series and focus instead, today on:
The Promise of Big Data
While establishing data standards and a common data system for the province’s public schools seems reasonable enough (Nova Scotia’s roughly 14 private schools — including hockey schools and religious schools — do not use iNSchool, nor do homeschoolers), the AG’s description of the system makes it clear iNSchool is intended to do a whole lot more than simply standardize record keeping.
Here’s Monica Williams, executive director of the Department of Education’s Centre for Learning Excellence, answering a question from a member of the Public Accounts Committee:
Through the Action Plan [Nova Scotia’s Action Plan for Education 2015] we are going to be registering all students who are born in the province, in an electronic file through the school system, education system, and we’re going to start the screening of our children, our prospective students, at 18 months of age, 36 months of age, and six months, before they start school, to identify early any challenges that they having and to direct the families and the children to supports that they may need. Early identification of risk is something that is a very high priority in the department.
I don’t know how the Department of Education plans to “screen” 18-month-olds, but it might want to consider what happened in Maryland in 2014, as recounted by Susan Ochshorn, founder of ECEPolicy Works:
Last December, in the wake of a survey of kindergarten teachers, the Maryland State Education Association called for immediate suspension of the state’s readiness assessment. The teachers had delivered an hour-long test to five-year-olds whom they barely knew on skills that they had not yet taught. Nearly 30 percent of the students were unable to understand and use the technology required by the exam. Many teachers lamented the loss of critical time for bonding with their eager, young learners. And 63 percent of them reported that they had received no meaningful data to inform instruction from administration of the exam.
It sounds like a crazy thing to have even attempted, but such is the promise of Big Data, it makes frustrating five-year-olds seem worthwhile. Education tech types don’t see five-year-olds, they see little bundles of facts, just waiting to be gathered, stored and funneled through the algorithm, which will tell you which five-year-olds will struggle with math, which will love reading and which will become radicalized.
It’s worth noting here that in addition to the information gathered by teachers and entered into databases like PowerSchool, there are many, many other ways in which student data is collected. As Stephanie Simon wrote of the education tech company Knewton in a 2014 Politico article:
By monitoring every mouse click, every keystroke, every split-second hesitation as children work through digital textbooks, Knewton is able to find out not just what individual kids know, but how they think. It can tell who has trouble focusing on science before lunch — and who will struggle with fractions next Thursday.
Or so it claims.
The Reality of Little Kids
Valerie Steeves, professor of criminology at the University of Ottawa and co-leader of The eQuality Project, a Canadian research and advisory group focused on youth and the internet, is not so sure.
Steeves was one of four speakers from The eQuality Project to present at a January conference called “Privacy Implications in the Networked Classroom.”
[J]ust to bring it down to really basic terms, the idea is to collect everything that I do — whether it’s on social media, or when I’m at school and I’m using Blackboard to report on my students or whether I’m emailing my husband — if you just collect all that data on Val…the algorithm will look and say, “Well, okay, what has Val done for the last 57 years?” [laughs] Okay, they didn’t have technology tracking me for the first 30 at least, but, “Look at everything Val’s done and we’re going to assume she’s going to do exactly the same thing in the future.”
So it’s an inherently conservative form of prediction, it’s based on past behavior. That worries me…because, what happens to human agency? Especially if you’re 13 and 14 and 15. Think of the changes you went through between 13 and 23…You’re constantly getting into new stages and your interests change, and your moral view can develop and grow. So I find this problematic just in that it’s an inherently conservative understanding of human behavior.
As for the notion that the algorithms that interpret data are “neutral” or that math “gets rid of that human propensity to discriminate,” Steeves points out first, that the categories to which children are assigned are established by institutions for their own purposes and second, that a child’s learning process is a very difficult thing to measure, so companies like Knewton choose “awkward” measurements as “proxies” for learning — mouse clicks, eye movements, finger tensility. Said Steeves:
Both through the way we categorize people and the kind of proxies that we pick, we end up creating algorithms that reproduce these stereotypical understandings of life.
She cited the work of Harvard professor Latanya Sweeney, whose study, Discrimination in Online Ad Delivery, analyzed the type of ads that appeared on Google when certain names were searched for and discovered that names associated with African Americans (DeShawn, Darnell, Jermaine) were more likely to produce ads suggesting criminal activity (like criminal record-clearing sites).
(My own skepticism about algorithms stems from LinkedIn’s insistence I join both the Hispanic and Asian journalists’ societies, Netflix’s certainty that I will enjoy the One Day at a Time reboot and Google’s belief that I’m always in the market for the thing I just bought.)
What all of this does is raise doubts about the third claim made for Big Data — that it is able to “personalize” things.
And so…actually, I can tell by your eye movements and finger tensility that you’re no longer really paying attention, so why don’t you take a little juice break and we’ll meet in the next section in 10 minutes.
Electronic Record or Millstone?
Elana Zeide is an associate research scholar at Princeton, a visiting fellow at Yale Law and a Microsoft research fellow at NYU School of Law. In addition to being a lawyer with a specialization in tax law, she has an MFA in non-fiction writing and a BA in American Studies.
I take all this to mean that when she focuses her attention on a subject, it’s probably worth listening to what she discovers and lately she’s been focusing on the use of technology in education.
The first thing to say about her work is that she seems to put more faith in Big Data than Valerie Steeves does. In fact, one of her articles is entitled “19 Times Data Analysis Empowered Students and Schools.” Here’s an example:
2. Sending Kids Home – Means They Might Not Come Back
In 2014, researchers at the Center for Civil Rights Remedies analyzed information from across the US — including “K-12 suspensions, self-reported delinquency, arrest and incarceration records” and found that high suspensions “did not improve school outcomes or deter future misbehavior,” writes Zeide. “Instead, they found that students suspended from school are less likely than their non-suspended peers to obtain a high school diploma and to obtain a bachelors degree by their late 20s, and are more likely to be arrested, become multiple offenders, and be sentenced to confinement in a correctional facility. These findings contributed to the nationwide reevaluation of “zero tolerance” school discipline policies, culminating in the Obama administration advising schools to find alternatives to policies that promote suspension.
What’s interesting about this — and what brings up a point about Big Data I hadn’t actually considered before — is that Zeide introduces the case study by saying:
Critics have long challenged suspension-oriented disciplinary policies as counterproductive to improving student performance and reducing racial inequities. Until recently advocates and researchers struggled to support these claims with empirical proof…
So the data, in this case, didn’t reveal anything close observers of the system didn’t already know, it just gave them ammunition to use against those who, presumably, were refusing to believe them.
I wanted to mention that article because I feel I’ve been hard on Big Data and Zeide tells some good stories about its use. But what I really want to discuss is a presentation she gave to Princeton’s Center for Information Technology Policy (CITP) in October 2016 entitled, “Ethical Use of Big Data and the Rise of ‘Algorithmic Credentialing.’ (It’s about a half-hour long, I assume you will watch it after you finish this article, but if you prefer to watch it now, fill your boots — this is a free range classroom.)
In this talk Zeide, who believes in the good Big Data can do, cautions against some of its not-so-good aspects. She considers the “chilling” effect that constant surveillance and monitoring have on students (a subject Valerie Steeves also discussed). Studies have apparently shown that students who know they are being observed “are less expressive…take fewer risks intellectually.”
Similarly, when asked an opinion they know is being recorded, students “will articulate viewpoints that are more close to what they think the mainstream viewpoints are, without necessarily realizing that they’re doing so.” This, says Zeide, runs counter to the traditional notion of the classroom as a safe space where students are encouraged to take intellectual risks and where they know “not all actions have immediate consequences.”
And then there’s the issue of “embedded assessment” — tests that must be passed before you move on to the next chapter of your interactive textbook or the next unit of your online course. As a diagnostic tool, it’s great, but as a part of a student’s permanent record? She tells the story of a good student who has failed the same in-course test seven times. Eventually, she says, he’ll pass and move on to the next level but he knows that somewhere in the system it’s been recorded that it took him multiple tries to get there — what if, she suggests, one day, during a “close hire,” that test becomes a factor?
And as though the sheer heft of the digital trail following children through life were not enough, Zeide discusses experiments with “blockchain” storage of student records — that is, using the technology behind the digital currency bitcoin to create “immutable” records or “an open-source permanent record of everything students have done,” something that might, she says, be great on an administrative level but is surely less so on a psychological, pedagogical or social level.
Finally, she considers what it means to allow educational tech companies to define what education is — because when they define “knowledge metrics” and “outcomes” they are, in effect, defining education. And if education becomes that which can be measured — if “metrics win the day” — then what happens to creativity? Zeide suggests that for cash-strapped institutions and school boards, it could become something only privileged students with resources can nurture.
It’s a big responsibility to abdicate to the tech vendors, even if most are “idealists.” Says Zeide:
[W]e have debates in larger society about what education should be about. Is it about getting a degree to get a job? Is it about [John] Dewey’s version of intellectual self-fulfillment and exploration? Is it about the labor market or is it about creating a citizenry that is tolerant of different viewpoints, that is able to self-govern? These are tensions within the system. But those tensions end up getting worked out invisibly when they’re encoded into technology.
She ends with the “heretical” suggestion that perhaps, when it comes to education technology, less is more.
Suomi
Which brings us to… Finland.
Elementary School in Vaskio, Salo, Finland. (Photo by Kotivalo, own work, CC BY-SA 4.0, (http://creativecommons.org/licenses/by-sa/4.0 via Wikimedia Commons)
C’mon, you knew we were going to get there eventually. You can’t talk about education in 2017 without mentioning Finland, I think that’s actually been codified into our free trade agreement with the European Union.
Finland, for those of you who don’t know, is the educational chou-chou of the industrialized world. A mysterious, Nordic country whose 15-year-olds have been acing the PISA tests (international language, math and science tests) since 2000 although they only have to take one — ONE — standardized test during their whole P-12 career.
Standardized tests, by the way, are another pillar of the Nova Scotia education system, which believes children are both unique individuals whose data must be captured so their learning plans can be tailored precisely to them AND interchangeable members of a particular grade level who can be evaluated by means of a single (preferably multiple-choice) test.
Call it the triumph of the Standards & Accountability crowd, a group that has been stomping all over US education since the ’90s. People who believe you can a) determine what a child should know in a given grade, b) test for that knowledge and c) (the fun part) hold teachers and schools and school boards accountable for those children who cannot read or do math at grade level.
In Nova Scotia, we’ve been spared the ugliest aspects of this kind of thinking — like the teacher value-added ratings published by American newspapers, including the New York Times and The Los Angeles Times a few years back (a practice they seem to have discontinued) — but we haven’t escaped completely. And should we ever decide we want to rate teachers in this way, I’m sure there’s an app for it that could be integrated seamlessly into iNSchool.
Or maybe…we could be more like Finland. Writing in The Atlantic magazine in 2011, journalist Anu Partenen noted:
…Finland has no standardized tests. The only exception is what’s called the National Matriculation Exam, which everyone takes at the end of a voluntary upper-secondary school, roughly the equivalent of American high school.
Instead, the public school system’s teachers are trained to assess children in classrooms using independent tests they create themselves. All children receive a report card at the end of each semester, but these reports are based on individualized grading by each teacher. Periodically, the Ministry of Education tracks national progress by testing a few sample groups across a range of different schools.
Partenen’s article (What Americans Keep Ignoring About Finland’s School Success) argues that, at its core, the Finnish system seeks to achieve not excellence but equality. And yet, somehow, by ensuring each child in the country receives the same, quality, public school education (Finland has no private schools), it has also achieved excellence.
As for teacher accountability, Partenen quotes Pasi Sahlberg, director of the Finnish Ministry of Education’s Center for International Mobility:
Accountability is something that is left when responsibility has been subtracted.
