Search This Blog

Friday, March 2, 2012

Criticisms of Learning Analytics

Gardener Campbell, Virginia Tech www.gardenercampbell.net
Criticisms of Educational Experience in Learning Analytics


Moving seamlessly between areas is not trivial
?of domain crossing-Stephen Johnson calls Adjacent Possibility
Danger
increased by reductive learning analytics
lock ourselves into a model that will take us backwards

"Perhaps the final claimant for the title of ultimate thory of the universe is M-theory....M-thory is not simple. You can't print it on a T-shirt...M-thoery is not a single theory. It is a collection of theories. Hawking describes them as a "family of theories". Each member of the family is a good description of observations in some range of physical situations, but none is a good descripton of observations in all physical situations. Non can account for 'everything'.... No single flat map is a good representation of the EArth's surface. Just so, no single theory is a good representation of all observations."
Kitty Gerguson, Stephen Hawking: An Unfettered Mind

Analogy to Learning Analytics? why is this relevant today? This is a conclusion we've reached provisionally about the cosmos--probably the second most complex thing in our universe. The first most complex thing in the universe is our brains. Why would we think that any kind of approach to learning would be any less complicated? How do we get at measuring this? Carefully. With a degree of humility. We will inevitably try to get to some sort of reduction to be able to understand, to wrap our heads around it. Minds-trillions of interconnections among neurons.

Our Shared Reality, 2012
"We are living in the middle of the largest increase in expressive capability in the history of the human race."
Clay Shirky, Here Comes Everybody


Siri now. This suggests the scale of what we are trying to map. Any attempt at quantifying or qualifying has to answer to this scale. We can't organize ourselves just around what we can measure.

"Learning analytics in the academic domain is focused on the learner, gathering data from course management and student information systems in order to manage student success, including early warning processes where a need for interventions may be warranted."
--Analytics in Higher Education: Establishing a Common Language, Angela van Barnevald, Kimberly E. Arnold, and John P. Campbell, ELI Paper 1:2012, January 2012


ACKKK!! This is much closer to the Cartesian coordinates than it is to the M-thory and expressive capability.

TED talk: Sebastian Seung: I am my connectome

Course management systems have nothing to do with the expressive nature of the self. It has nothing to do with the self, identity, or with the complexity of being human. This should be at the heart of what we talk about when we talk about learning or understanding. This is about schooling, very different from learning. Much schooling is simply transactional. We build industrial models of education, assembly lines that cannot be stopped. We measure the cosmos from Flatland. The structures we are now quantifying via LA are a set of phenomena that are just about getting from one end to the other. Monkey see, monkey do: writing down what the teacher writes on the blackboard.

Gardner gives us four strong cautions with regard to LA:
•"Student Success"
•Complexity
•Points of "Intervention"
•The "Third Wave"

Student success is not the same thing as success as a student. There was a student quote:

"I have no clue about what I want to do with my life: I have no interests because I saw every subject of study as work, and I excelled at every subject just for the purpose of excelling, not learning. And quite frankly, now I'm scared." _-"Here I Stand", Erica Goldson, June 25, 2010
http://americaviaerica.blogspot.com/2010/07/coxsackie-athens-valedicatorian-speech.html

I have been a lifelong educator. Yet, this quote is of no surprise to me. I have always taught my students that school is a game with rules and you have play by the rules, and then you need to learn on top of that. I told my son as he was going through school that he needed to excel and do well to get to each successive step, BUT that he MUST learn IN SPITE of the system. People can easily go through the system and get all A's, excellents, great recs, but not learn, or learn to learn, or learn to think, or learn to be creative at life, liberty, and the pursuit of happiness, and problem solving.
If we apply the reductive model of analytics, we will have our top students, top engineers, etc. A set of students who finish school not even sure what they want to do with their lives.
Standards and rubrics don't seem to get to the deeper pieces of learning. Standards are set to map cognition in a Cartesian coordinate plane. The factory approach puts even the well meaning people to sleep. Cognition exists in the design of the analytic systems as well.

reddit.com check out the FAQs
I want to learn. Can we look not just at student compliance, but at student contributions and not just what they write in answer to a prompt but how they link to the world outside. Forum: IWTL: How to Learn This is a self-organizing system--not like a learning management system. Prompt: You learn something new every day; what did you learn today? Submit interesting and specific facts that you just found not (not broad information you looked up, TodayILearned is not /r/wikipedia.
What a great forum! What an enlightened place for students, learners, users, whatever we want to call them. I have been an educator because I was unhappy with the system that existed. I tried to make changes constantly so that students, learners, children, all could LEARN, UNDERSTAND, THINK. I agree completely with Gardner problems with the learning management systems. I'm glad that this conversation is finally coming up within the LAK class.

John Naughton, From Gutenberg to ZuckerbergL What you Really Need to Know about the Internet (2012) --
Complexity is the new reality.
•non linear, unpredictable--butterfly effect
•feedback matters--a lot-- must be loops that feed the complexity
•systems demonstrate self-organization (like reddit.com above)
•EMERGENCE-- synergies--new phenomena -- then we have to learn about the new phenomena and then we have to engage in double loop learning. How do we deal with paradoxes?
What is Double Loop Learning? This refers to success being defined as more than positive outcome "relative to pre-established targets." This defines success allowing learners "need to reflect on the appropriateness, in the light of unfolding events, of the assumptions (the mental model) used to set up those actions and targets." Naughton Analytics mean little more than positive outcome. Double loop would allow a true analytics system that would itself be able to learn--to adapt to the changes, to reflect and make appropriate change. (just as math is not equal to calculating)
Student success is not the same thing as success as a student.
Complexity-- set up target, and then figure out if the target has remained the same.
Intervention- typically LA intervenes when it thinks there will be risk of failure-- but learning doesnt proceed from confusion> lightbulb > clarity, or thesis and antithesis resulting in sythesis--it is instead a set of nodes and connections between these, like a concept map--this is a truer picture of learning.

LA system of suitable complexity--intervenes when the student begins to understand something and validates that what you are thinking is interesting, you are on the edge of finding out, you are on the right path here. building the system around interventions around the point of about to fail is not as interesting as intervention around you are about to understand.

connectome
waves-ELI director, Malcolm Brown A third wave, integration of social media-2-integration of LMS into learning 1.
have we really integrated social media--reddit, twitter, blogs, if we had engaged wide-open social media we would see how complex learning really is--let's question the third wave because we're routing around complexity. What is the goal?

quasi-demo of this kind of system- www.wdyl.com (what do you love?) the icon for a search is a heart-- put in learning--
framed around invitations that will begin in delight and end in wisdom. the google page is like a project based, inquiry based
more coming soon
understand the complexity of lerning. there is a healthy humility about what we don't know--measures curiosity but also arouses it--
free range student writing-auto ethnography - so that sparked my curiosity--link to a website--student is giving info about what it is the student is doing within the learning envirionment
How do you measure wisdom? through Chris Dede wrote article about wisdom inlearning, he pointed to the point--A Handbook of Wisdom, Cambridge U Press, 2005--several strands of reasearch now devoted to wisdom-id cross cultural consensus, traditional learning models, things they don't include, awareness that within each cultures reporting self-awareness there is a strong strand of wisdom -- Wisdom is a core human value, it has to do with learning, but not just learning, has to do with doing, but not just doing, why not ask questions about the process. A small pieces loosely joined web presents an interesting possibility for connections that would increase the chances we could find wisdom in our collective conversation.

look up other references:
"Paradox of the Active User", John Carroll & Mary Beth Rosson, 1987
(Is Effective learning possible?)
John Naughton, From Gutenberg to Zuckerberg: What you Really Need to Know about the Internet (2012)
World Wide Mind Stephen Churros

No comments:

Post a Comment