Is Participating in MOOC Forums Important for Students? A Data-driven Study from the Perspective of the Supernetwork

To ascertain the influence of forum participation on student grades, we first obtain students’ course information, including userID, scores for tests, exercises, final exams, and discussions, as well as total scores and discussion activities (e.g. the number of comments, replies, and posts). The Information Retrieval course examined in this study opened in May 2014 with 3,231 registrants.

In this course, the full marks for test scores, exercises scores, final exam scores and the full score for discussions are 113, 30, 30, and 30 respectively, accounting for 30 percent, 10 percent, 30 percent, and 30 percent of the total score. Especially, if the number of student posts, comments, or replies is higher than three, the full discussion scores can be identified (Table 1).

Table 1

Descriptive statistics of student scores and forum activities.

Students in this study are divided into a discussion group and non-discussion group according to whether they participate in the MOOC discussion forum, where differences in scores between the two groups include test statistics and the nonparametric analysis results (Table 2).

Table 2

Test statistics.

P-values in Table 2 indicate that the test scores, exercises scores, exam scores, and total scores of the two groups have significant differences. And we can infer from the mean rank in Table 3 that grade of the discussion group is better than the non-discussion group, so participating in a discussion forum can truly improve student achievement in MOOC.

Table 3

Results of nonparametric tests.

Besides the nonparametric tests, multiple linear regressions are used to investigate the relationship between forum activities and scores. Taking the numbers of comments, posts, and replies as independent variables, the dependent variable is the total score weeding out the discussion part that students participating discussion forum get in the MOOC course. Comment refers to other students’ comment on one student, while reply is the one student’s reply to others, and post refers to the students’ posting topics in the forum. The proposed regression model is as follows:

Score=Constant+β<!-- ? -->1Comment+β<!-- ? -->2Reply+β<!-- ? -->3Post+ε<!-- e -->.$$\begin{array}{} \displaystyle \rm Score = Constant + {\it \beta_{1}}\,\,Comment +{\it \beta_{2}}\,\,Reply +{\it \beta_{3}}\,\,Post+\varepsilon. \end{array}$$

The numbers of comments, replies, and posts are found to reflect MOOC users’ involvement in the forum, where students can exchange ideas and benefit from co-construction of knowledge by their participation. We therefore suggest that forum participation will have a significantly positive influence on student grades. To verify our assumption, we do multiple linear regression analysis (Table 4).

Table 4

Regression analysis results.

The regression gets good fitting results (adjusted R² = 0.657, p = 0.000). Table 4 indicates that comment (β₁ = 0.879, p = 0.004), reply (β₂ =3.937, p = 0.000) and post (β₃ = 1.776, p = 0.000) all have significantly positive effects on student grades. Participating in forums is found to contribute to communication, where discussions with others can help deepen the understanding of the course content. This means that the MOOC forums can reinforce learning, help students to better master knowledge, and thus play an important role in enhancing grades.

Based on above research, we find that forum participation activities do have a positive influence on students’ scores, as discussion group members (people who participate in the forum) get higher scores than those in the non-discussion group.

Yosef Sheffi (1985) first put forward the concept of the “supernetwork” model as a suitable technique for integrated system modeling. Nagurney et al. (2002) referred to the supernetwork as being above and beyond existing networks. Single networks cannot completely depict characteristics of the real world and relations between networks, whereas a supernetwork can describe and express network information more comprehensively and clearly.

In research on supernetwork, the use of the hypergraph is frequently employed. In mathematics, a hypergraph is a “generalization of a graph in which an edge can join any number of vertices… a pair that is a set of elements called nodes or vertices, and is a set of non-empty subsets of called hyperedges or edges” (Wikipedia, https://en.wikipedia.org/wiki/Hypergraph). The defining formula of the hypergraph is as follows (Berge & Minieka, 1973):

Assuming V = {v₁,v₂,…, v_n} is a finite set, if

E_i ≠ ∅ (i = 1, 2, …, m),

then the binary relation H = (V, E) is called the hypergraph. V = {v₁,v₂,…,v_n} is called the hypergraph vortex. E = {e₁,e₂,…, e_m} is the edge set of hypergraph, and set E_i = {v_i1, v_i2,…,v_ij}, (i = 1, 2, …,m) is called hypergraph edge, which is the superedge, exhibited in the following formula and Figure 1):

Figure 1

V={v1,v2,v3,v4,v5,v6,v7}E={e1={v1,v2,v3},e2={v1,v4},e3={v2,v3},e4={v3,v5,v6},e5={v4,v7}}$$\begin{array}{} \displaystyle V = \{v_{1}, v_{2} , v_{3} ,v_{4} , v_{5} ,v_{6} , v_{7}\}\\ \displaystyle E = \{ e_{1} = \{ v_{1},v_{2} ,v_{3} \}, \,\,e_{2} = \{ v_{1} ,v_{4} \}, \,\,e_{3} = \{ v_{2} ,v_{3} \}, \,\,e_{4} = \{ v_{3} ,v_{5} ,v_{6} \},\,\, e_{5} = \{ v_{4} ,v_{7} \} \} \end{array}$$

In MOOC, there are two networks, the people network and knowledge network. People of all ages, with different education backgrounds coming from different countries, consist of the people network. Different fields of knowledge offered by MOOC platforms constitute the network. The people and knowledge networks interlace and build connections with each other, so that they constitute multi-style, multi-level, and multi-dimensional supernetworks. In this complicated supernetwork, we employ some parameters to depict it, which can describe the forum’s activity levels more clearly and help us better understand active levels of participants as well as the degree of the threads’ significance.

Studies about supernetworks are still in the development stage. Although the concept of the supernetwork is defined, the parameters to depict it still lacks unified and specific definitions. Taking properties of the super-node and super-edge into consideration, we define the concepts and algorithms of super-node degree and super-edge degree (collectively called “super degrees”) based on the hypergraph, as follows:

Node degree: In the hypergraph, node degree of super-node v_i, is defined as the sum of super-edges, including v_i, noted as d_ej (v_i);

Edge degree of super edge: In the hypergraph, the edge degree of the super-edge e_j is defined as the sum of super-nodes belonging to e_j, noted as d_vi (e_j);

Super node degree: In the hypergraph, the super-node degree of super-node v_i, is noted as d_H(v_i), where the formula is:

dH(vi)=dej(vi)×<!-- ? -->∑<!-- ? -->idvi(ej)∑<!-- ? -->jdvi(ej)$$\begin{array}{} \displaystyle d_{H}(v_{i})=d_{e_{j}}(v_{i}) \times \frac{{\mathbf\sum}_{i}d_{v_{i}}(e_{j})}{{\mathbf\sum}_{j}d_{v_{i}}(e_{j})} \end{array}$$(1)

(Note, ∑_i, d_vi (e_j) means the sum of the edge degree of super-edges including v_i, and ∑_jd_vi (e_j) means the sum of the edge degree of all super-edges);

Super edge degree: In the hypergraph, the super-edge degree of super-edge e_j is noted as d_H(e_j), where the formula is:

dH(ej)=dvi(ej)×<!-- ? -->∑<!-- ? -->jdej(vi)∑<!-- ? -->idej(vi)$$\begin{array}{} \displaystyle d_{H}(e_{j})=d_{v_{i}}(e_{j}) \times \frac{{\mathbf\sum}_{j}d_{e_{j}}(v_{i})}{{\mathbf\sum}_{i}d_{e_{j}}(v_{i})} \end{array}$$(2)

(Note, ∑_j, d_ej (v_i) means the sum of node degree of super-nodes belonging to e_j, ∑_id_ej (v_i) means the sum of node degree of all super-nodes.

To better understand the definitions of these degrees, take Figure 1 for example. The node degrees of super-nodes v₁, v₂, v₃, v₄, v₅, v₆, v₇ are 2, 2, 3, 2, 1, 1, and 1, respectively. The edge degrees of super-edges e₁, e₂, e₃, e₄, e₅ are 3, 2, 2, 3, and 2, respectively. The super-node degrees of super-nodes v₁, v₂, v₃, v₄, v₅, v₆, v₇ are 0.83, 0.83, 2.00, 0.67, 0.25, 0.25, and 0.17, respectively. The super-edge degrees of superedges e₁, e₂, e₃, e₄, e₅ are 1.75, 0.67, 0.83, 1.25, and 0.50, respectively. The functions of these parameters in MOOC are as follows:

Node degree denotes the frequency in which a certain person participates in the forum threads. The larger the node degree value, the more active this participant is, and the better he or she facilitates knowledge sharing.

This study uses a crawled dataset retrieved from the discussion forum of the course “Information Retrieval.” There are many threads in the forum, and students can post to ask for help, to discuss the lectures, to share learning notes, and so on.

Among the data crawled, we pick up five threads (marked as threads 1 through 5) that not only have more participants and the number of posts but also have significant gradients, that of “Week 7–Interaction Assignment,” “Week 4–Interaction Assignment,” “Week 2–Interaction Assignment,” “Week 1–Interaction Assignment” and “Information retrieval and information selection”. The post numbers of these six threads are 106, 97, 85, 42, and 31, respectively. The weight of super-nodes and super-edges are not taken into consideration when calculating parameters of the supernetwork. We therefore adjust the numbers of people who post several times at the same threads. This means that if someone posts several times at the same thread, we only mark it as one time. Then the adjusted numbers of threads are respectively 99, 86, 78, 36, and 24.

We calculate the numbers of participants in these six threads, and pick up students who participate in more than one thread. The finding is that 3 people (e.g. “Viannn” username) participate in five threads; 7 people (e.g. “mooc 15951364231” username) participate in four threads, and the rest participate in three or fewer threads. The majority of people only participate in one or two threads. Selecting 10 typical participants, Table 5 depicts their thread participation levels.

Table 5

Participants and threads they participate in.

The relevant parameters of above participants and their threads is then calculated, where node degrees and super-node degrees are calculated (Table 6).

Table 6

Node degree and super-node degree of participants.

Node degrees and super-node degrees all indicate participants’ MOOC forum activity levels. The larger the value is, the more active this person is. Nevertheless, super-node degrees may be different even though people have the same node degree. For example, “Day@4” and “mooc15951364231” have the same node degree, but the former has a larger super-node degree. This means that he or she participates in much more threads with higher participation rates, and knowledge shared or opinions expressed may influence more people. The Super-node degree of “Viannn” and “Winner” rank top 2, which indicates that they have higher activity levels and thus a larger potential to acquire and share knowledge. But “Shadowfollower” et al. have relatively smaller values, indicating that they are less active in the forum and may exert a smaller influence on others.

Considering the entire forum, however, participants are far fewer in number than mere course registrants, where the numbers of threads that most people participate in are very small (one or two). Most students in the course do not participate in the forum at all. So the majority of people have smaller node degrees and super-node degrees than participants listed in Table 7. Edge degrees and super-edge degrees of the six threads are shown in Table 7.

Table 7

Edge degree and super-edge degree of threads.

Edge degree and super-edge degree both indicate the threads’ participation levels. The larger the value is, the more people the thread attracts. What’s more, the larger super-edge degree indicates that this thread could allow more influential people with higher participation rates to join in to facilitate knowledge sharing. Threads 1 to 5 represent the participation rates with different levels. Threads 1 and 2 have larger super-edge degrees than others, which means that they are more important to participants. Threads 3 and 4 have the same edge degree but different super-edge degrees, where Thread 3’s larger super-edge degree indicates that it attracts more people with higher participation rates and thus contributes more to knowledge sharing.

Nowadays, most MOOC platforms only offer the number of viewers and comments to students, where only a small percentage also provide numbers of up-votes, which represent positive (negative) votes for threads. But it is not enough to manifest the quality of threads. As some researchers said, the more we enable participants to know about their interaction information and quality of threads, the more they will be motivated to join in the forum. In the same vein, the more students depend on interaction information, the more actively they would participate in the forum discussion (Zhan et al., 2015). It is therefore important to improve forum settings and offer suitable assessments of student (and teacher) contributions as well as the quality of threads.

The study parameters of super-node degree and super-edge degree can therefore act as an assessment of activity levels of forum participants as well as degrees of thread significance. It is easy to understand that one thread may have many posts, but if most posts are off-topic or worthless, the number of posts cannot truly reflect the value of this thread. But one thread with a higher participation rate is more likely to be a valuable thread, where super-edge and super-node degrees can reveal this latent or hidden information.

Above all, the super degrees can be used in MOOC platforms to act as a kind of assessment, for it can partly express activity levels as well as the significance and quality of participants’ threads.