It is expected that the discussion boards in online CS projects provide a space for knowledge building. An exploration of interactions within such discussion spaces also taking into account the participants’ roles, e.g. moderators, scientists and volunteers, can provide us with insights on how role assignment influences knowledge-building discourse.
Modelling discourse using Epistemic Network Analysis (ENA) provided initial insights regarding different forms of participation and expertise of different user roles, e.g., volunteers, moderators and scientists, in the online discussion spaces of CS projects.
Initial findings indicated that the moderators engage in fueling the discussions while scientists engage in sharing knowledge and providing important updates that are necessary to carry out intended CS tasks.
Introduction
In the context of online CS projects, different participant groups such as scientists, moderators and volunteers are contributing to knowledge building discourse using online discussion forums. Previous studies have used classical frequency based measures such as counting approaches (Aristeidou, Scanlon, & Sharples, 2017) and discourse analytics (Huang et al., 2018) techniques to explore the profiles and levels of engagement of different participants in such online spaces. However, the dynamics of interactions among different participant groups and the type of knowledge each group brings into the collective discussion have not been studied in detail also considering the temporal perspective in the discourse data. To this end, as a particular case of relevance, we retrieved publicly available forum data (130 posts) from the talk pages of the Chimp&See project hosted in the Zooniverse platform. Individual discussion posts retrieved from projects’ discussion boards, i.e., Help Board, Chat Board and Science Board, were then coded using a custom coding scheme. In summary, we used two different codes related to data collection and data analysis. We also used two codes to indicate help seeking and help giving. Other codes reflected discussion participation, providing updates, initiation of new ideas, sharing knowledge and organising project related activities. The coded dataset was then used as an input to the ENA webtool to retrieve ENA models.
ENA aids in quantifying the connections among pre-specified codes in qualitative datasets and visualises the structure of connections among such codes using dynamic network models (Shaffer et al. 2016). The nodes in ENA models represent the pre-specified codes in discourse and the network edge weights represent the relative frequency of co-occurrences between two codes.
Results
In the following, we describe the results obtained by applying ENA technique to the aforementioned forum dataset.
As shown in Graph 1 in the Help Board, volunteers and moderators exhibit similar behaviours as they often made connections among help seeking (Resques.Help) and help giving (Giving.Help). The scientists’ network showed a different structure. The connections to codes Updates and Sharing.Knowledge indicates that in these types of discussion spaces, scientists provide important updates and share knowledge that is necessary to carry out the intended CS tasks. Regarding the Chat Board (see Graph 2) the relatively strong co-occurrences observed between Initiating, Discussion and Giving. Help codes can be interpreted as discussion leading to the proposal of new ideas. The networks generated for scientists and moderators show a similar network structure that can be better interpreted by referring to the difference network shown in Graph 4. In the science board (see Graph 3) relatively strong co-occurrences between codes such as Data.Collection, Discussion, Data.Analysis, Giving.Help and Initiating indicate that volunteers express doubts and as they engage in data analysis, which also seems to co-occur with strong help giving. Such tasks also seem to lead to the initiation of new ideas.
Graph 1: Epistemic Networks generated using discussion posts from Help Board.
Graph 2: Epistemic Networks generated using discussion posts from Chat Board.
Graph 3: Epistemic Networks generated using discussion posts from Science Board.
Graph 4: Difference network generated considering discourse of moderators and scientists in (a) chat and (b) science boards.
The difference network was generated by subtracting the average connection strengths in the Chat Board from the average connection strengths in the Science Board. As it can be seen in Graph 4 the strong connections between Discussion, Data. Analysis, Data. Collection and Giving. Help codes in both chat and science boards indicate the important role moderators play in such discussion spaces by providing inputs and support to data collection and analysis, all the while taking part in different discussions to ensure the continuous flow of discussions. Finally, regarding the scientists, the strong connections to code Sharing.Knowledge in both Chat and Science Boards indicate the important role scientists play in these discussion spaces by sharing important knowledge which is essential to carry out CS activities.
Conclusions and Future Work
ENA analysis present in this study aimed at providing insights regarding how different roles contribute to the knowledge-building discourse within discussion forums in CS projects. Initial insights derived from this analysis indicate that moderators facilitate the continuous flow of discussions as expected while scientists share important knowledge and updates related to CS tasks. Volunteers seem to find the discussion spaces useful to express doubts and to propose new ideas as they engage in data analysis. A limitation of this study constitutes the limited number of posts considered for manual coding, therefore, generating networks using ENA. In the future, we are planning to extend the analysis by considering a larger number of posts and multiple projects from the Zooniverse platform to obtain insights that can be generalised regarding the contribution of different roles towards knowledge-building within online discussion forums of CS projects. These insights can provide information about the effectiveness of discussion boards as an incentive mechanism for social interaction and learning gains. Moreover, exploring how promotion (based on role assignment changes, e.g., assignment of moderator role to a volunteer) influences volunteers’ contributions towards knowledge-building discourse would help to uncover how such incentives enhance participation.
References
This work was derived from our previous work reported in the following conference paper:
Amarasinghe, I., Manske, S., Hoppe, H. U., Santos, P., & Hernández-Leo, D. (2021, August). Using Network Analysis to Characterize Participation and Interaction in a Citizen Science Online Community. In International Conference on Collaboration Technologies and Social Computing (pp. 67-82). Springer, Cham.
Aristeidou, M., Scanlon, E., & Sharples, M. (2017). Profiles of engagement in online communities of citizen science participation. Computers in Human Behavior, 74, 246-256.
Huang, J., Hmelo-Silver, C. E., Jordan, R., Gray, S., Frensley, T., Newman, G., & Stern, M. J. (2018). Scientific discourse of citizen scientists: Models as a boundary object for collaborative problem solving. Computers in Human Behavior, 87, 480-492.
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45.
