Search This Blog

Wednesday, February 29, 2012

Multi-level analysis of distributed learning, uptake

Dan Suthers, A unified framework for multi-level analysis of distributed learning LAK11: http://blip.tv/solaresearch/8_dan_suthers-5692149

Dan works mostly in computer supported collaborative learning. This work came out of doing analysis of interactions in small groups, face to face, online, multiple media and put this together somehow? He was also supported an online community of teachers (TappedIn. Representation of data and analysis enabled the scaling up.

Many theories about how learning happens in social settings:
1. the individual is stimulated by the social setting-social as stimulus to social entity as learning agent
2.Castell's networked individualism
3. Jeremy Rochelle's maintaining a joint conception of the problem
4. how much coupling is needed between learners
diffusion of innovations
knowledge building
they all have in common--contact between people must occur for learning to happen
interaction-not necessarily conversation--maybe Uptake better concept

UPTAKE-the act of the actor taking something relevant from what someone has done before as being relevant for your own ongoing activity (downloading files, etc.)
evidenced by what we can see directly-people's actions in their environment

Analytic Challenges
interaction between the individual, small group and collective agency
requires multiple level analysis
practical challenge-- studyuing learners working in whiteboard and a threaded discussion--interaction is spread or distruted over multiple media--put it back together
traces of activity are fragmented-how to create the whole in the analysis?

ties--not at all like log data-want to unpack what the ties mean
want to put together this trace of interaction

system: that collects things together in and puts things together in a single
annotated artifact and analyses it.

using db or log files (http) want to figure out what is happening
1. understand what entities we want to see and the relationship between them
ie threaded discussion, has some features to illustrate
2.the granularity at which things are recorded at may be different than the granularity of what you want to look at
message may have 3 logs for it, threading relationship
person 2 has written msg 2 and written msg 3 snd then posts something--now more abstract version as a transcript rather than log files-- wikis etc -unify the record of different media.

3. what about interaction? adjacency carries-assumption-
4. construct contingency graphs-- identify empirical relationships between events that collectively evidence uptake
called contingencies after Garfinkel's "contingently achieved accomplishments" how actors draw on the evolving context.
not truly proof of interactions, when people do things they draw on the resources of their environments

5. did by hand, now figuring out how to automate:
contingencies: media dependency

6. contingencies: Lexical or Semantic Overlap: also contingencies between the read events and writing of messages --events that are near each other may be related, events by the same actor,
for example, reuse of noun phrases ( contingency graph showing contextual action mode) graph of entity-relations of discussion 1 and all actors.

7. end up with: Contingency Graph as Contextualized Action Model
•analytically relevant manifest relationships between the actor's actions and other events that have been recorded
•Next:raise the analytic level of description to latent relationships and higher order structures.

8. be selective in what contingencies you put in--could become pretty complex
all above are data structures that are manipulated computationally.

•interpret collections or subgraphs of contingencies as corroborating evidence for uptake
•supports sequential analysis of interaction

Uptake Graph--an Interaction Model
possible automated way to find the uptake in discussions-- way to find the potential for learning- a structure you can look for in the data structures
micro analysis of transactions- manually, now trying to automate-especially highly interactive discussion

Next Layer--abstracting away from the sequentiality of the events-
affiliations of people through media-- Accociograms
directed affiliation network of actors and artifacts
mediation model: how actors' associations are mediated Latour
this largely factors out time--looks at mediated interactions--finds round trip between actors-- wouldn't see this in threaded discussion structure-round trips are important-dialogue is how learning happens in groups

Relationships
patterns of mediated associations reveal relationships
dialogue pattern-round trip
consumer pattern P3 reads everything P2 produces

Multi-media associations
characterize pairwise relationships in terms of distribution across media
compare roles of various media in supporting associations (suthers and Chus, networked learning, 2010)

cluster analysis
compare roles of media bridging between groups
transitive closure,
Ties--
straightforward to collapse into sociogram by transitive closure or similar computations
mediated associations
SNA methods can be applied to the sociograms

this framework allows potential automation of representation of data to do analyses on-- interpretation of analyses
multi-level analysis

Prior research
contingency graphs are used for:
microanalysis of process through which learners achieved an insight
semi-automated analyses of graph manipulations to find pivotal moments

currently applying this to TappedIn, longest running network of educators

latours idea-following the actors
Advantages of this framework:
as a data representation
integration of distributed data: uncloak distributed interaction
common format for reuse of algorithms
as an analytic framework
multi-level multi-theoretical analysis possible
multiple ontologies allow for mapping between interaction, mediated affiliation and tie levels of analysis

Attention data and Attention Trust

Thank you Erik Duval for pointing out in our session with you last week about the Attention Trust:

"APML is about something bigger than technology. It’s about your right to take ownership and control of your Attention Profile so that you can share it with the services you know and love.

From AttentionTrust – your Attention rights include:
Property: You own your attention and can store it wherever you wish. You have CONTROL.
Mobility: You can securely move your attention wherever you want whenever you want to. You have the ability to TRANSFER your attention.
Economy: You can pay attention to whomever you wish and receive value in return. Your attention has WORTH.
Transparency: You can see exactly how your attention is being used. You can DECIDE who you trust."

The Attention Profiling Markup Language (APML) is a standard format from which we can extract our own attention data-- to extract the music we listen to, photos, tags, bookmarks, blogs, tweets, and then we can personally aggregate our own data and analyse it for reflection!

The Attention Recorder:
"AttentionTrust, has developed a free, open source piece of software that allows you to record information about the websites you visit and pay attention to. This "Attention Recorder" makes it possible for you to store, analyze and share this data with anyone you choose." (http://www.attentiontrust.org/services)

Here is a blog entry discussing AttentionTrust as returning attention to its rightful owner. Things to think about while we are looking at creating a document of ethics for the field of Learning Analytics: Bokardo

Tools: DuckDuckGo

One of the blogs in the MOOC this week talked about using the the goal / hybrid-search engine DuckDuckGo. http://duckduckgo.com/

Apparently, it has a transparent workspace minus the advertising and a clear privacy policy that guarantees anonymity of the searches and control of the browser settings. This may be why DuckDuckGo was in the list of top 50 Web sites conducted annually by Time magazine.

This is comforting in light of the new Google privacy policies going into effect tomorrow. Google isn't really free, we give up our personal information and the right to it's usage.

Wednesday, February 15, 2012

Hans Rosling, Data Visionary, TED talk June 2006

My husband Dan showed me this talk by Hans Rosling. Rosling was a doctor in rural Africa who co-founded Doctors without Borders, and later served as a professor at the Karolinska Institut in Stockholm. He founded Gapminder (gapminder.com) and developed visualization software you'll see in the talk. It was bought by Google in 2007.

TED TALK: Hans Rosling shows the best stats you've ever seen
http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen.html

Although he seems to have slightly preceded the "Age of Analytics", his visualizations make sense of large and complex data sets that otherwise present contrasting views of the world. He espouses look at the data to actually see the changes, and to not just take what he calls the "average data". He shows breaking open the data sets into more specific entities that give more information and make better sense.
JUST WATCH HIM!!

Learning Analytics MOOC first 4 weeks readings and video links

Week 1
Baker, S.J.D., Yacef, K. (2009) The State of Educational Data Mining in 2009: A Review and Future Visions: http://www.educationaldatamining.org/JEDM/images/articles/vol1/issue1/JEDMVol1Issue1_BakerYacef.pdf
Why the current interest in analytics in education?
Technology and the completion agenda:
http://www.insidehighered.com/news/2010/11/09/completion
Untangling the social web-Economist:http://www.economist.com/node/16910031
Marisa Mayer (Google), The Physics of Data:http://www.parc.com/event/936/innovation-at-google.html
Industry partnerships: http://www-03.ibm.com/press/us/en/pressrelease/36384.wss
Big data stupid decisions: http://www.youtube.com/user/OreillyMedia#p/c/2543D0F253DD85CE/25/LXDFwphuwCs
Big Data: the next frontier: http://www.youtube.com/user/OreillyMedia#p/c/2543D0F253DD85CE/39/C5VB0E1bWiI
Week 2
Goldstein, P. J. (2005) Academic Analytics: Uses of Management Information and Technology in Higher Education http://net.educause.edu/ir/library/pdf/ecar_so/ers/ers0508/EKF0508.pdf

Siemens, Long: http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume46/PenetratingtheFogAnalyticsinLe/235017

What is a career in big data? http://www.youtube.com/user/OreillyMedia#p/c/2543D0F253DD85CE/68/0tuEEnL61HM

Baker, R. S.J.d. Data Mining for Education: http://users.wpi.edu/~rsbaker/Encyclopedia%20Chapter%20Draft%20v10%20-fw.pdf

Analytics: The widening gap (a bit of a flash back to last week, but compartmentalized curriculum isn't good :)): http://sloanreview.mit.edu/feature/achieving-competitive-advantage-through-analytics/ (you will need to register (free))

Week 3
Knewton: Adaptive learning system: http://www.youtube.com/watch?v=LldxxVRj4FU
Signals: Applying academic analytics
Transforming learning through analytics - Rio Salado (starts at 19 min): http://educause.mediasite.com/Mediasite/Play/bdac04349c7246a6b6f949ded44397501d

UMBC: Check my course activity: http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/VideoDemoofUMBCsCheckMyActivit/219113

Week 4
Semantic Web: An Introduction: http://infomesh.net/2001/swintro/

Ray, K (2009) Web 3.0 http://vimeo.com/11529540

Berners-Lee, T. (1989) Information Management: A proposal http://www.w3.org/History/1989/proposal.html

Tim Berners-Lee talk http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html

Hilary Mason, Machine Learning: http://www.infoq.com/presentations/Machine-Learning

Jovanović, J., Gašević, D., Brooks, C., Devedžić, V., Hatala, M., Eap, T., Richards, G., "Using Semantic Web Technologies for the Analysis of Learning Content," IEEE Internet Computing, Vol. 11, No. 5, 2007, pp. 45-53, http://goo.gl/eouEW

Ali, L., Hatala, M. Gašević, D., Jovanović, J., "A Qualitative Evaluation of Evolution of a Learning Analytics Tool," Computers & Education, Vol. 58, No. 1, 2012, pp. 470-489, http://goo.gl/2gTd4

LAK12- MOOC course- 4 week check-in

Methodology of the course: One of the critical literacies in a digital networked era-the ability to navigate and make sense of what is going on in distributed settings.
In the course, most are now able to have tolerance of ambiguity, and understand that learners control their learning, learning is around other learners, distributed,learners control their space of learning.
twitter, the daily(aggregates blogs, twitter, etc.) Now we all know there is a boundary-less structure.
Three weeks of discussion--what is LA, big data, society is more complex, how do we make sense of it.
First week: We need to use our human cognition to navigate the distributed nature of data. Huge surge of interest in big data, stats, AI. We begin to look at a hybrid human cognition model as data is too overwhelming in our modern world.
Second week: Started to look at what analytics are-business intelligence, educational data mining community. Hard to come to a definitive definition of what LA are. Looking at the data that learners generate-the trail that can be measured and analyzed to personalize adapt the learning for the individual. holistic
Third week: Cases and examples of analytics
universities, John Campbell and SIGNALS at Purdue, Rio Silato, what are the factors that improve or influence learner success? Much is centered now on the online experience. At this stage, much of LA occurs online because that is the data trail that is left.
Week Four: Semantic Data Link Data (course turn) intelligent curriculum
Not just evaluating what students are doing, but what the curriculum changes in response to what the students are doing. Dragan. Develop the knowledge architecture of different fields--each student receives their own course.

Remaining course presentations/discussions:
Ethics-what can we and what should we do? Practicalities-tools and techniques, opportunity to play around with analytics approaches and tools. systems and course level
Last two weeks- concept paper around Open Learning Analutocs-what shouild LA serve for a researcher - SoLAR -

FYI-Multiples of bytes-- BIG DATA is BIG!!!

Multiples of bytes
SI decimal prefixes Binary
usage IEC binary prefixes
Name
(Symbol) Value Name
(Symbol) Value
kilobyte (kB/KB) 103 210 kibibyte (KiB) 210
megabyte (MB) 106 220 mebibyte (MiB) 220
gigabyte (GB) 109 230 gibibyte (GiB) 230
terabyte (TB) 1012 240 tebibyte (TiB) 240
petabyte (PB) 1015 250 pebibyte (PiB) 250
exabyte (EB) 1018 260 exbibyte (EiB) 260
zettabyte (ZB) 1021 270 zebibyte (ZiB) 270
yottabyte (YB) 1024 280 yobibyte (YiB) 280

A petabyte (derived from the SI prefix peta- ) is a unit of information equal to one quadrillion (short scale) bytes, or 1000 terabytes. The unit symbol for the petabyte is PB. The prefix peta (P) indicates the fifth power to 1000:

1 PB = 1000000000000000B = 10005 B = 1015 B = 1 million gigabytes = 1 thousand terabytes

But, in traditional binary usage, there are 1 125 899 906 842 624 bytes in 1 petabyte.

The pebibyte (PiB), using a binary prefix, is used for the corresponding power of 1024, which is more than 12% greater.

The unit petabyte is sometimes used for the corresponding power of 1024, following traditional designations of computer storage size, a practice superseded by modern conventions.

Learning Analytics--New Discipline and Semantics -- session with Dragan Gasevic

I actually missed the live session of this talk, as it was scheduled at 11am MST instead of the usual 1pm. The email came in the morning during meetings, and by the time I tuned in, I had missed it. Thank you George for recording these sessions and making them available the next day!

Age of Social Media and Learning
Slidea of this prsentation
How do we know how well we are doing using any technologies? How successful are we in learning and how effective our institutions are?
Topics discussed/reactions and questions:

*Learning - today, tomorrow...
Today's idea of learning and general ed is a pizza box. Come to the institution, we will give you a textbook, study guide, instructor, and final exam. The final will be the info in the textbook.
Agree that this is not the idea of modern education that should include higher level learning, application, thought, analysis.
See Dragan's Learning Paths:
learning doesn't just happen in the box--formal and informal
idea of Educational Ecosystems
reusability
adaptivity
evolution
collaboration with educators and students
The authoring of these systems with reusability is it's packaging. to create
LEARNING AND COLLABORATION
mobile
administration
community
peer-review
presenting (youtube, recording and posting, blogging and discussing)
learners

students using many tools etc--still creating a bigger pizza box-within the boundaries of the institution. These BOUNDARIES ARE LIMITING WHAT WE WANT TO ACHIEVE.

How do we capture all the learning happening EVERYWHERE?
How can we connect all these tools in which students are linking with all the formal tools that students are using?
Having a formal or commonly shared model may be important--to integrate these systems

Three Generations of Distance Education Pedagogies
Connectivism
open idea-but boundary of institution that requires more formal course
border of connectivism and social constructivism
keep integrating
How can we facilitate learning analytics for constructivism and connectivistm -- while behaviorism is important--these had some analytics.
not to set back learning to behaviorism, but to learn about LA in ways to learn more about constructivism and connectivism

Part II: Learning Analytics

A systematic change is needed!
LA- What?
Measurement, collection, analysis, and reporting of data about learners and their contexts (SoLAR)
LA--Why?
Understanding and optimising learning and the environments in which learning occurs
and then how can we optomize the environment sna tools to make learning more effective
SoLAR paper about OpenLearningAnalytics
differentiation between learning analytics and academic analytics
LA benefits learners, educators, teaching staff, whereas academic analy benefits administrators, funders, marketing, governments, etc.
EVIDENCE BASED EDUCATION
As the integration of best research evidence with practioner expertise and stakeholder values (based on Evidence Based Medicine)
use the research and LA as a critical enabler to proceed toward evidence based ed

Criticism about LA-- concern that analytics will reduce the learning experience to equations if we return to behaviorism
LA involves many statkeholders not involved in pedagogical underpinnings of ed
accusations can be useful, we don't need behaviorism fears? because analytics can only be applied if we keep track of learners BEHAVIORS,
but this is more than just BEHAVIOR--one trace is action recording, go deep down and analyze blog text to see the cognitive presence of the person and measure critical thinking--machine algorithm analyzing the cognitive function and looking at constructivistm
When we look at the analysis and the social metric and person is involved in and their community, we are then talking about learning analytics which will support these more advanced pedagogies.

??-- To follow the clinical / medical analagy, how to perform analytics in a controlled manner? For example, software may differ among so many dimensions, e.g. UI, UX, content, learning paradigm employed, what is going on in the classroom that is prompted by the software. Where a given material subject to analytics may be aiding learning along some of these dimensions, it may be harming along others. The studies of learning materials I have seen don’t even beging to adress these issues and tends reduce results to a binary ‘the software improves / does not improve learning’.

Part II
Learning Analytics
Approaches and Challenges
with bits of semantics


BIG DATA-- need machines, data overload, we are producing more content on the web than we are ever able to keep up on--now connectivism- ? is how can we deal with these?

LA CHALLENGE: WHAT AND HOW TO COLLECT?

Ubiquitous Learning analytics?--Tool and format independent--aggregates and integrates
how can we gather data from ALL formats, and aggregate to give feedback to learner and others in the stakeholder process

SEMANTIC WEB
Ontologies-- interconnecting application--shared domain conceptualizations
Peter Metcalf paper-- ontologies
Linked Data Graphic
DPedia-- rdf or other techs --all the underlying structures and concepts in wikipedia extracted and processed by machines to easily navigate from one concept to another concept--
therefore, when we are analyzing blogs in LA, we may be
"A crazy problem requires a crazy solution!" --Griff Richards, 2005

NEED AN ONTOLOGY TO CAPTURE ALL THESE PEDAGOGICAL ISSUES
Learning Context Ontology: LOCO captures many kinds of data:
content structure ontology
content type ontology
user model ontology
learning design ontology
domain ontology
Jovanovic, J., Knight, C., Gasevic, D., Richard, G., "Learning Object Context on the Semantic web." 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, The Netherlands, 2006 pp.669-673.

good paper for ontologies
http://www-ksl.stanford.edu/people/dlm/papers/ontologies-come-of-age-mit-press-%28with-citation%29.htm
another good paper on ontologies - general overview: http://www.jamesodell.com/Ontology_White_Paper_2011-07-15.pdf

EDUCATION SOCIAL SEMANTIC WEB
Combing the Social and Semantic Web

Formative Evaluation
Ali, L., Hatala, M., Gasevic, D., Jovanovic, J. (2012). A Qualitative Evaluation of Evolution of a Learning Analytics Tool. Computers and Education, 58(1) 470-489, http://goo.gl/ICvMT

LA HOW AND WHAT TO REPORT?
Visual Learning Analytics
Ubiquitious LA impossible without visual because of information overload
how we can inform our educators better to different conditions students are demonstrating and compare students to each other--overload?
provide ed insights about cognitive functions
wanted to compare types of social activites--kinds of posts, different kinds of conceptualizations being discussed
types of sociographs related to different concepts
allow students to do social bookmarking and highlighting, and see what concepts students

Learning Analytics Acceptance Model graphic
Inspired by Venkatesh, V., Morris. M. G., Davis, G. B., & Davis, F. D., 2003 User acceptance of information technology: Toward a unified view. MIS Quarterly, (27:3), pp.425-478.
a model to predict learner potential
analyze different LA related to quality of concepts (course, modules, class), with types of student interactions (engagement in lessons, discussions, general comprehension), see how all are associated with LA and ease of use--study which provided and identified correlations-- one critical points about general student interactions is strong association of social analytics--most important is the social interaction analytics that lead to the overall perceived usefulness of the analytics and the tool.
The adoption of a tool is based on the context, policies, cultural,

LA WHAT TO MEASURE?
Measurement, collection, analysis, and reporting of data about learners and their contexts

How can we increase level of critical thinking? community of inquiry used as a measure of critical thinking--
cognitive level can be associated with different kinds of teaching, social presence,

LEARNING ANALYTICS FOR COMMUNITY OF INQUIRY
EFFECTS OF INSTRUCTIONAL INTERVENTIONS
Example: Role playing (invited expert and moderation) with explicit instructions how to contribute.
how do I maintain and identify that this interaction works?

How can we identify different kinds of metacognition?

Social Learning analytics for self-regulated workplace learning--LearnB-- for each
another paper Dragan
for differnt types of artifacts, we want to provide different streams--if I'm studying something, don't show me something distantly related
what kinds of events are around certain competencies? how to facilitate with visualization with the level of peer knowledge, if I created a new competence and shared it with everyone, I can see my performance and sharing with the rest of my peers--only me to see this--I can regulate my own behavior

MEASURING COGNITIVE PRESENCE
TEXT MINING AND LINKED DATA
A very similar text-mining problem is spam classification.
How id cog pre AUTOmatically
by analysis of the text--.57-.74
we have anti spam 95% or higher
these are the kinds of algorithms to auto classify msgs based on cognitive presence

COGNITION AND META-COGNITION
Discovering learning processes
to see indiviudal numbers, but see the flow of activities, first msgs in infor sharing, then usually see resolution cog presence
area of process mining
chart flows--
SNAPP
http://goo.gl/jtO31
http://nodexl.codeplex.com/

Technology Acceptance Model as a model for the above

Learning Analytics
MORE
meaningful and ubiquitous
with semantic technologies

to label and not do spam detection or labeling--do text analysis and determine
triggering event
exploration
inspiration
resolution
other

Monday, February 6, 2012

Knewton Adaptive Learning Platform

Knewton: the dream of personalized education

I was intrigued with the Knewton system shown in the YouTube video: It's a highly structured environment, but it's data mining of student actions in the system does provide the student herself, teacher, and parents personalized "next steps". These recommendations rely on more than probability, looking at the way each student best learns to give the uniquely personal responses.
Looking up a bit more background than in the video, I discovered Jose Ferreira, a former Kaplan exec, founded the company in 2008. He's lately received millions of dollars toward funding more R&D research in his "adaptive" system.
It even offers GMAT course reviews!
Knewton is a learning platform that uses algorithms to achieve a level of customization previously only attainable for the teacher in very very small groups. Customize both the content and context, the latter in that it identifies the pairs with the same learning style. Traditional educational environments, as it tells a teacher, is a dream.

I think the question arises whether it is worth keeping these traditional environments based on the logic of the industrial age. Some people argue that each human brain is wired in a way that even as individual learning styles and multiple intelligences are able to map it. If we take these arguments seriously, probably not worth keeping, for example, segregation by age or content.

But it is questionable regarding Knewton, which among its many advantages as well promising a lifelong relationship between people and publishers, is who will own the data it collects. Do you are the Shareholders or stakeholders of Knewton?

This is a great tool for individuation in learning. It adapts, and feeds back. But it has the one dimension only of "delivering knowable curriculum" to the student. I think learning is more than that.
I think we need to ask ourselves: How does it really work? How is coding working to send differentiated content to each learn. Do they Classify learners?
The other piece I think we need to explore is that this seems to be test prep work. It cannot replace classroom work, discussion, discourse, debate, creativity. I worry that failing school districts could view this as a solution to their problems, inserting students into computer stations with teachers and parents just monitoring progress and adding what the system recommends.

Thursday, February 2, 2012

Tools for Data Visualization and Analysis

The Computerworld article, "22 free tools for data visualization and analysis
Got data? These useful tools can turn it into informative, engaging graphics",
by Sharon Machlis, presents a wonderful array of FREE tools with varying skill levels. I've taken just the skill levels one and two here for my own beginning interest and exploration.
1. Users who are comfortable with basic spreadsheet tasks
2. Users who are technically proficient enough not to be frightened off by spending a couple of hours learning a new application.
Google Fusion Tables, Many Eyes, VIDI, OpenHeatMap, TimeFlow, DataWrangler, Google Refine, Zoho Reports, Google Chart Tools, and IBM Word-Cloud Generator.
To begin, I'll explore
Many Eyes
and OpenHeatMap.