Research: INTE 6720 – Data Literacy Project

Data Literacy Project

Rather than diving into reading existing research for my data literacy project I chose to do a little investigating of my own. About two years ago CU Boulder made the decision to migrate their learning management system from D2L to Canvas. Now that we are a couple of years into the migration and adoption of Canvas I decided to interview staff to glean real time information on how they have collected and used data to support the adoption of Canvas as a learning management system as the university transitioned from D2L to Canvas.

Data has become a vital part of our modern technical world so much so that data drives many decisions made in society today. Data touches almost every aspect of our world and education is no different. Students, faculty, and staff are affected by data driven decisions. Within the Infrastructure and Sustainability division, where I currently work at CU Boulder, we have a mentor program that uses Canvas to communicate with the mentor teams. Most of the mentor teams consist of staff members who are not familiar with the Canvas Learning Management System (LMS). Although I have worked extensively with Excel, formulas, and programming I have yet to take a look at data as it relates to learning management system and the adaptation of users to new systems and trends.

With this in mind I decided to turn to some people on the Boulder campus who have been integrally involved with facilitating learning and the adoption of the Canvas LMS on the Boulder campus. My first interview was with Viktoriya Oliynyk who works with faculty in a Canvas support role on the Boulder campus. I met Viktoriya during one of the CU Boulder Academic Futures talks and saw her Cidi Labs presentation at the COLTT Conference in August, 2019 where she acts in a Canvas support role on the Boulder campus. Her focus on Cidi Labs as a tool for instructors to build more engaging courses piqued my curiosity, that along with her role as direct support for tools used on campus brought her to mind when considering people to speak with for this project. We talked about the challenges of the change from the D2L learning management system to Canvas, the impact of change itself on adoption, faculty adoption rate, data available directly in Canvas, and other options for data collection. Listen to an audio portion of our interview below.

After speaking with Viktoriya I was able to confirm that additional conversations would be beneficial in order to dive more deeply into the data based decisions on campus. In the next interview Mark Werner, the Associate Director for Academic Technology, pointed me to Aisha Jackson, the Director of Academic Technology Applications Design.

During my interview with Aisha we discussed the different angles one can look at data from; the student perspective, the instructor perspective, and the institutional perspective. Aisha noted that reporting data in Canvas is limited and although they use Canvas reporting pull data out of Canvas to evaluate it outside of Canvas itself. In addition surveys and discussions are used to compliment data pulled from the system.

Aisha noted that Canvas provides some information on students and courses, like student activity via click data, and course information, such as the number of courses and early adopters. However the capability to build data driven tools that integrate with Canvas is an important part of data analytics for CU Boulder. For example a web-grading tool, built to make moving grades into the Registrars’ system more efficient, is an example of a dashboard tool using Canvas and API’s. Use of these types of tools and reporting provide valuable feedback on usage that can alert them to a need for reminders to be sent to faculty based on usage, or lack thereof. I wasn’t familiar with API’s but learned that unlike an Excel export they are more dynamic than a one-time data export. You can indicate how frequently you want the data to be pulled and integrated into the dashboards you build out. This allows for more dynamic reports to be built based on institutional need. Things that rise to the surface for CU Boulder analysis are:

Which departments are adopting?
Where are course counts?
Where are student counts?
Which tools are being used, or not used?

Part of analytics includes knowing about who is using the system, this is information that comes both from within and outside of Canvas. Both Aisha and Viktoriya noted that Survey’s are another source of valuable data. Recent surveys have revealed there are a couple of tools where users feel there is room for improvement, this led them to the areas they are focusing on now. Aisha was also able to share a bit about the current adoption rates for Canvas as a whole. I learned that the goal was 90% adoption, updated figures indicate they are currently at an adoption rate of 92%. This is exciting news just a few years into the transition.

In Summary

The three conversations I’ve had so far with those on campus who support Canvas opened my eyes to some sides of data analytics that, although seemingly obvious now, I had not consciously been considering. It was interesting to see the different sides of data and how focus changes based on perspective. One common thread was that a well-rounded set of data includes more that just reporting from an LMS. Although there are important aspects to be considered from a data draw one should not underestimate the importance of getting the human perspective through the use of surveys and user response groups.