University of Edinburgh and the Royal College of Surgeons of Edinburgh have jointly established a suite of online distance learning programmes to support surgical trainees and to prepare them for independent surgical practice and for their future academic development. The ChM Masters in Surgery programmes utilise asynchronous discussion boards to teach and evaluate student understanding of complex surgical cases. An example of such a discussion board can be seen in figure 1.

Student attendance, engagement and in-course assessment has traditionally been used as a model to evaluate a students performance. However, we found that using these measures of performance would not allow us to predict a students exam and overall programme performance.

Using social network analysis of discussion board interactions we are able to better predict final exam marks. In this study we will highlight one particular student (purple) who scored poorly in the final exam assessment but would not have been flagged by the normal metrics associated with student performance.

data were collected from five student discussion groups from two ChM programmes. Discussion board posts were collected for a full academic year and imported in to Excel. Where students mentioned an individual by name a link was created between the two individuals. For each mention the weight of the link was incremented by one unit. Network graphs were generated and analysed using Gephi software 1. Linear regression analysis was performed using SPSS

We found little correlation between the quantity of posts, quality of posts (inferred from discussion board mark) and attendance with exam performance  (Figure 2a, 2b, 2c). In contrast using social network analysis and the pagerank ² algorythm showed some degree of correlation.

The network graphs generated in this analysis highlight the relative influence of each student in the discussion and the central role of the etutor (Figure 3). 

Multiple linear regression using post number, discussion board marks and pagerank value gave a statistically significant relationship with exam marks and an adjusted R² value of 0.3

Using social network analysis of online asynchronous discussion boards we have demonstrated that the role a student plays in a discussion may be more important in predicting exam performance than simple measures of participation volume or in-course assessment marks. Further, we demonstrate how this data would allow the prediction of students at risk

1. Bastian M., Heymann S., Jacomy M. (2009). Gephi: an open source software for exploring and manipulating networks. International AAAI Conference on Weblogs and Social Media
2. Page, Lawrence; Brin, Sergey; Motwani, Rajeev and Winograd, Terry (1999). The PageRank citation ranking: Bringing order to the Web.