S1 Introduction

Introduction to the hidden stories of the M4 podcast

Learning Analytics after the Pivot-to-Online

June 30, 2021Uncategorizedlearning analyticspentadij@gmail.com

A whole host of posts/MOOCs have been published about the ‘pivot to online’ – generating online versions of face-to-face teaching to cope with lockdown. One thing that has not been discussed that much is the use of the logs of students using the VLE in this situation, IMHO this analytic technique has great potential to support tutors moving online to track how their new materials are being used. Analysis of such Learning Analytics can answer questions such as:

What proportion of students have accessed key materials on the VLE by a certain date?
Which students haven’t engaged with the content of the VLE and are in danger of dropping out completely?
What items in the VLE are proving unexpectadly popular/unpopular?

Given the effort invested and the speed of generating materials, it seems important to me to answer these questions . It’s possible and advisable to directly ask students via feedback how they are using materials but in most situations, that leads to somewhat vauge and incomplete data to analyse. Learning Analytics do not replace feedback as a way of gauging how students are using learning materials, they are a useful addition and both data sources should be used to assess newly created online materials.

Most discussion I’ve seen (such as UCL’s Connected Learner Extensions Open Courseware) concentrate on question 2 and other questions related to student progress. That is important but in this post I’ll be concentrating on the use of Learning Analytics to inform about the learning design (e.g. questions 1 and 2).

Use of learning analytics is not universally welcomed, some educational technology colleagues regard it as unwarrented student surveillance. I address some of these ethical points in the last section of this post.

Data Visualisation

My experience of using VLE Learning Analytics visualisation ‘out of the box’ is not good: the reporting doesn’t usally visualise the data in a useful way. However, within Moodle (and maybe other VLEs), it’s possible to download the raw Learning Analytics data which can then be visualised using other tools. In order to explore questions [2] and [3] above I wrote a script in R to analyse and visualise the data (download script here). The download is in R Markdown, allowing me to write all the instructions you’ll need to do the analysis on your own Moodle course in the same file as the code. It produces two visualisations:

Cumulative first-hits graph:

To illustrate the script I have used data from a large scale (100+ students) MSc module on the Wolfson Institute (part of Queen Mary’s University London) Psychological Therapies MSc on which I work.

The first shows the first-time hits on a resource against time: a student accessing a resource on a given date contributes to the count if it is the first time they have accessed that resource – returning to a resource at a later date is not recorded. The plot is cumulative, it therefore shows the proportion of students who have accessed a resource by a certain date. In flipped learning this plot can be read to find out what number of students accessed a resource before the face-to-face session, what number accessed it for the first time on the day and how many didn’t look at it until after the related face-to-face session.

Click here to expand the graphic into a new window.

In the example shown above, four resources from a flipped presentation module are shown, two of them (purple and cyan) have had annotations added to illustrate what you can find with this visualisation.

Interpretation: Students are advised to access the resources before the related face-to-face session, the materials are released a week before the class session. A major issue is that students are not using the flipped learning material as designed with the cyan resource – they are skipping the pre-session material before day before the teaching session because they have assignment deadlines to meet. However, the majority of them catch up accessing the resource later.

Grade vs Item heatmap

This visualisation shows the proportion of students who have accessed an item by final course grade. It obviously can’t be used until after the module has ended and marks have been collated. Students are sorted into grade bins (rows) and the columns represent different course items. The colour of each cell represents the proportion (0=none, 1=all) of students in that grade bin accessing the module item. The columns are ordered by when the students should be accessing them as the course unfolds.

Interpretations:

It can generally be seen that a lower mark correlates with lower proportion of students accessing items.
Some items are universally unpopular (dark blue vertical lines) and popular (vertical yellow lines).
As the course continues, the difference between weaker students and stronger students becomes less defined – even good students are being strategic with their time and skipping materials
Some items show a strong variation between mark classes in behaviour, these items are worthy of looking at to try and work out why and maybe advise students.

Notes:

The 40-50% bin is anonomolous because only two students were found in this bin and they appeared to have accessed a lot of the popular elements. This illustrates an issue with the visualisation: you need a reasonable number of students in each class to make it representative. It is less useful for small courses (the course illustrated had about 110 students on it)

Ethics

A lot of learning technology colleagues are wary of learning analtyics as they regard them as ‘survellience’, and consider them unethical. I think that viewpoint has value but it doesn’t capture the full story. Certainly, Learning Analytics can be used to spy on students and the data used to in unethical ways. An example would be a commerical web service that gathers personal information on students and sells that information on in a non-trasparent way. However, other uses appear to me to be far more benign, an example is where students group behaviour looking at materials over time is measured (the two visualisations are examples) – the individual student cannot be identified and all data is kept within an institution. If students are made aware of this use of their data, I think it represents an ethical use.

That being said, there are other ethical dangers than the misuse of personal data. Considering just learning analytics could lead to a behaviourist-centric educational approach – for example, did intervention X drive more use of a forum? This is a valid measure of engagement and is easy data to gather but it may fail to measure deeper, more important educational activity. In most situations its more helpful to focus on the depth of learning illustrated by the students posts in the forum. We need to be aware of where our attention is being driven as educators considering Learning Analytics.

In short, analysing learning analytics where students cannot be identified avoids most ethical issues, but not all. More detail.

River Chess Storymaps

February 2, 2021Uncategorizedpentadij@gmail.com

View of the Chess in Winter

“A fantastic endeavour! Congratulations. I was really down there in the water with the spawning trout.”
Member of the Public responding to the project, January 2021

So along with a core team of colleagues from UCL and QMUL Geography and a whole host of partners* I’ve been working on a collection of esri Storymaps since March about the Chess River.

Access the River Chess Observatory with a navigational introduction

Go straight to the River Chess Observatory skipping the introduction

We’ve released them as a informal learning resource for the public’s consumption guided by our partners. The Chess river is an interesting topic to tackle: its a chalk stream, these are typically rich in fauna and flora. However, its subject to a number of issues and pressures common to chalk streams generally.

In this post I’ll be considering how Storymaps have worked as a medium for telling a ‘spatial story’ such as the Chess: one where maps are a key tool for explaining the concepts and problems.

WordPress with Map Widgets

One way to characterise a Storymap is that its very like WordPress, or other cloud based text editors such as Sway but with added widgets (called blocks as in WordPress) that are focussed on map based presentations. For example, one really useful block we’ve used is the map slider (press play on the video):

Video of me using the map swipe block in Storymaps

This allows you to put two maps together so you can compare them, in the above situation, we’re showing how the urban areas in the Chess catchment around Chesham have increased in the modern era (right hand map) compared to the 1940s (left hand map).

We’ve also made use of the sidecar block to present maps/graphs in an explanatory fashion (sensu Andy Kirk in Data Visualisation) where the main point is that we’re doing some interpretation and are presenting it using a data visualisation of some kind. In the example below we’ve used a sidecar block to produce what I call a ‘build’ animation where layers of data are added one by one with relevant explanation as you do it (again, its a video):

Both of these blocks (sidecar and slider) are pretty easy to put them together and esri thoughtfully provide a library of useful maps that you can call on with the map slider.

Collections and Google Analytics

Another neat function that esri have added to Storymaps is collections: you can split your content into multiple Storymaps and present them in a collection, arranged as a set of tiles which the user simply clicks to enter an individual Storymap. In our collection we’ve created four ‘narrative’ Storymaps that logically lead on one to the other like chapters. We’ve also added two Storymaps that act simply as frames to hold data dashboards that my colleague Hefin Rhys (a volunteer from Chesham in the Chess valley!) has programmed using R Shiny. The dashboards hold graphs showing up to date measurements from the Chess, some from the Chesswatch sondes. The idea behind this is that users can come and view the narrative Storymaps and look at the dashboards too but if they want to return to look at the current state of the river (dashboards) they can do that and skip the narrative parts completely.

All of the Storymaps are tagged with Google Analytics codes so that we can track use by the public. This is another advantage of splitting content into multiple story maps, we can see how far users got into the content whereas with just one, big Storymap we’d only know that they accessed it, not where they got to.

*partners, to whom we owe a great deal of thanks include: Esri, Thames water, Affinity water, The River Chess Association. Other help has come from BGS, Hefin Rhys and the reviewers of the materials.

What Makes a Good Spatial Story?

January 19, 2021Uncategorizedpentadij@gmail.com

Since March last year I’ve been working on a project using esri StoryMaps. Its a neat tool which requires no coding to use, designed to enable you to produce a scrollytelling story with a spatial slant – mostly by enabling you to produce useful maps/map-based-animations. A scrollytell animation uses the simple action of scrolling to drive an unfolding story with text, video and animations, arguably, they date back to 2012 and a story called ‘Snow Fall’, a Purlitzer winning story and graphical delight (nice history and future of scrollytelling).

Here’s a video showing how easy StoryMaps are to create:

Telling a spatial story on the web using animated maps isn’t limited to StoryMaps, there are other free tools such as Google Earth Tour builder (tutorial howto) which will allow you to produce a different type of spatial story. I use the term ‘Atlas Tour‘ to describe these different types of platforms.

When I was teaching at Southampton University, I got undergraduate students to create an Atlas Tour (using Google Earth Tours which was state of the art at the time). I allowed students to choose their own story, I quickly discovered was that students sometimes struggled with choosing a good topic for an Atlas Tour. Students often identified a topic that had a map that goes with it or involves spatial data but that still didn’t make for a great Atlas Tour, the spatial component of the narrative has to form the ‘spine’ of the story – it has to be the main story.

A lovely example of an Atlas Tour we’ve already mentioned is ‘Snow fall’, if you’ve never read/watched it you’re in for a treat.

An example of a non-Atlas Tour would be a topic that my partner works with as a medical statistician: the performance of surgeons at different English hospitals over a year. You have data about where the surgeons work so in theory you could make a map and plot on it performance data in some way. However, the story is really about how individual surgeons are doing, its about people not place so it doesn’t make a great Atlas Tour.

A topic that does make a great Atlas Tour can involve data that isn’t based around maps at first glance. Take the case of basketball, players take shots from different parts of the court. These shot locations can be collated and analysed using GIS techniques producing ‘court maps’ that make great Atlas Tours. Here’s an example story that could be put into an Atlas Tour platform like StoryMaps.

Identifying good topics for an Atlas Tour isn’t easy for the inexperienced but, if I were to put it in a nutshell, I’d say it boils down to :

“Is there a map that is the essence of the story?”

Ep5] Cleaning Data, loops and Vectorisation

cleaning data, loops, R programming, vectorization

2] Cleaning Data: To remove a column ‘col’ in a dataframe ‘data’ using null, the command is:

data$col <- NULL

How to apply hash strings (a series of unreadable text) to replace important data e.g. full names. https://cran.r-project.org/web/packages/openssl/vignettes/crypto_hashing.html

3] For Loops:

The code I discussed

For (i in 1:10){

commands to be written here in a series of lines

}

4] Vectorisation: The explanation of apply (and related commands like ‘lapply’) I used is here https://www.guru99.com/r-apply-sapply-tapply.html

Podcasting and Education

August 12, 2020Uncategorizededucation, podcast, R programmingpentadij@gmail.com

You may have noticed that podcasting is having a bit of a moment: it used to be a faff to download podcasts to your computer then sync with your iPod/other player. However, the rise of the smart phone has solved that issue with software handling automatic downloads from the web so the latest podcast in a series is always on your phone waiting for you.

This has enabled a steady year on year rise in the numbers using podcasts over the last decade, users are listening on commutes and other dead time when they can’t read or watch anything.

A characteristic of this audio consumption is that it holds the user’s attention better – people can easily click away when they’re watching a video in a browser or scanning some text but this happens less to those listening to audio. These factors mean that podcasts are ripe for use in education, uses in language learning are well known and common but podcasts could serve a number of purposes in all topic areas:

Extra material for a module especially one-on-one discussions or interviews
Summarys of teaching sessions or of feedback to a whole class on an assignment
Revision
Individual feedback on an assignment (although this is not strictly podcasting but just use of audio)

With that in mind I’ve been experimenting by producing a podcast about learning R programming to investigate the practicalities in education (webpage and feed for putting into your podcasting app).

In educational terms, this is what I’d term a ‘sidecar’ podcast, its meant to cover general topics of interest to anyone learning R programming for the first time, its not specifically tailored to a particular course (in falls in section [1] of my list above). Because of the organisation involved and because of Covid lockdown, I’ve done it on my own. I think a two way discussion between presenter and interviewer or just a two way discussion would be a nice alternative. My ex-colleagues at the OU run a lovely example combining learning about pedagogy through the lens of TV/film with two main presenters and the occaisional guest expert.

Practicalities:

Podcasting is relatively simple to produce compared to video or multimedia. However, it certainly isn’t as simple as text. There are plenty of resources on how to do it but here are some quick pointers I picked up:

Quality Mic: Invest in a quality microphone, listening to poor quality audio, especially on a commute with lots of background noise, is tiring for the student/user. A blue Yeti second hand will set you back about 60 pounds.
Summarising or repeating content (in some way that avoids being boring) is helpful for understanding in a podcast – users tend not to track back to listen to something again in the way they would with text.
Intonation: Speak in an informal way and try and vary intonation: users are missing the visual clues of your face mouthing the words and this makes up for it. Unless you really know what you’re doing, avoid reading from a script, it comes out dull and lifeless.
Editing Sofware: To edit out ‘umms’ and ‘errs’ and prepare your content, Audacity is free and has proved fine for my use.
Hosting: I hosted Learning R on my own WordPress site, using Seriously Simple podcasting. For a first go podcast, there are plenty of free hosting packages available such as podbean.

Shut Up and Write: Students meet up to write at the same time

August 3, 2020Uncategorizedmasters, students, synchronous, writing sessionpentadij@gmail.com

My colleagues Maria Turri and Bridget Escolme are running writing retreats for their Creative Arts and Mental Health Masters students currently working on their dissertations without other teaching. The basic idea is that everyone gets together and works in a supportive atmosphere with some supportive group and buddy chatting at defined times. The students really appreciate it and it obviously fits with these Covid19 times where students don’t have a physical shared space to socialise in. The following text is the outline used for an upcoming session, it neatly provides more detail.

Text shared with students

For those of you who have not attended before, be reassured that this is an opportunity to get on with some writing, with no scrutiny about what you have written or about your progress. The benefit comes from having a structured time where everyone writes (and struggles!) together.

An indicative timetable is here below. Take care and see you then, Maria

10.00 – 10.10 (afternoon: 2.00-2.10)

Maria sets up the retreat, explains the principles for anyone new: switch off your devices, do not consult papers or other sources, try and be in a space where you can work uninterrupted (unless you have essential duties to perform or attend to).

10.10- 10.30 (afternoon: 2.10-2.30)

– 20 minutes free writing on any dissertation-related topic.

10.30 – 10.40 (afternoon: 2.30-2.40)

– check in with partner (break out groups in pairs) – let them know what you wrote about in the free writing time and what your goal is for the next writing time, e.g. ‘to write something about…’

10.40 – 11.40 (afternoon: 2.40-3.40)

– 1 hour writing time, writing something under the topic you discussed with your partner.

11.40 – 11.50 (afternoon: 3.40-3.50)

– check in with partner. Tell them how you did with your goal and what your next one is going to be.

11.50 – 12.10 (afternoon: 3.50-4.10)

– break. Please take this whole 20 minutes as a break and if it’s possible to do so, go outside or into another part of your living space. At least be sure to turn away from the screen. Please don’t switch on your phone or check any emails or social media.

12.10 – 12.50 (afternoon: 4.10-4.50)

– 40 minute writing time, writing something under the topic you discussed with your partner.

12.50 – 1.00 (afternoon: 4.50-5.00)

– check in with partner, or with whole group if we’re only a small group. Tell them how you did with your goal.

1.00 (afternoon: 5.00)

– quick return to whole group to say goodbye.

ABC Workshops at distance via padlet

July 24, 2020UncategorizedABC workshop, learning design, padletpentadij@gmail.com

The ABC Workshop is a well known learning design activity to help module teams design modules based on face-to-face interaction and writing things on cards and moving them around on boards. I did one for some colleagues a couple of weeks ago but we had to do it remotely because of lockdown. I thought I’d share my findings and thoughts on what software to use and how it went.

My first thought on how to do it was to organise a web conferencing session with a shared whiteboard and use the whiteboard in the same way as the cards and sheets are used in a face-to-face ABC workshop. At QMUL I had the choice of using Blackboard Collaborate or MS Teams. With a bit of experimenting it soon became apparent that they weren’t up to the job so I moved onto thinking about using web conferencing with an external platform. Here are the platforms I considered:

Miro: Looked nice as you could create virtual postit notes on a drawn background but it requires users to create IDs and login so I rejected it.

Trello: Allows a set of columns to be set up and cards to be moved around on them just as in a workshop. A nice template to use if you’re interested. The problem is that you need to login to interact as users and also it isn’t accessible so I rejected this.

Padlet: Allowed users to access a board and interact without logging in, this seemed helpful and to avoid messing about at the start of the session. Although it isn’t accessible, they’re working on it which is very good to hear. I chose to set up the board in columns which would represent the weeks on a standard ABC sheet. The disadvantage is that dragging cards from column to column is not possible, you have to go through a multi-click operation to move cards from one column to another.

This is the padlet I used for more detail.

My approach has been done in more detail by Laurea University who have written it up in detail. However, there are some significant differences:

Whole workshop on padlet: Instead of using padlet just for the main storyboard, I used it for the whole workshop. The early radar graph and tweet activity sheet were converted to columns to fill in (column [2] below. I think the radar graph is unecessary, a simple system of stars on cards is readable in much the same way. The advantage of doing this is that there is less messing about in the workshop switching between media.

Screenshot of the padlet used for the session, first two columns.

Who are your students?: Inline with the Open Universities learning design workshops I have an activity based around thinking about who your students are (column [1] above).

How did it go? The ABC workshop worked well, the technology wasn’t a problem and numbering the columns helped with moving the team on to keep within 90 minutes. The team I was working with were very well motivated and I’d kept the numbers low (two of them) deliberately to limit complications.

Wrapping up: A nice thing about the padlet is that the notes from the session are a lot easier to write up afterwards, no deciphering scribbled hand writing! Its also important to set the privacy (Share (top right corner of the padlet screen) > change privacy) of the padlet so your users can create and edit padlet cards. The UCL ABC team have written about delivering workshops in lockdown.