Advances in Economics, Business and Management Research (AEBMR), volume 60

4th Annual International Conference on Management, Economics and Social Development (ICMESD 2018)

Learning Analytics: Online Higher Education in Management

M. Eulàlia TORRAS-VIRGILI1,a, *, Andreu BELLOT-URBANO1,b
1
OBServatory, OBS Business School, Avinguda Josep Tarradelles 171, planta 5,
08901 L'Hospitalet de Llobregat, Barcelona, Spain.
a
*[email protected], [email protected]

Keywords: Management, Online, Education, Learning, Analytics, Teaching.

Abstract. Learning analytics are a set of methods and techniques that collect, process, inform and
successfully produce machine-readable data continuously to analyze the educational process and
allow taking improvement decisions. The research aim is to develop and verify an instrument, based
on theoretical contributions accepted by the international community, can evaluate the quality of
online higher education in management based on the Learning Analytics methods. The online
teaching-learning model in management proposed that students’ actions focused on an educational
activity should be in the central position. This online teaching-learning model in management have
been proposed according five components: the management of the institution, the technological
presence, the instructional design, the intra-psychological component and the inter-psychological or
social component. Based on the Teaching and Learning in Online Education Institutions in
Management has been created and provided an instrument that collects the dimensions, categories
and variables of each component. The components analyzed show that the dimensions, categories and
variables for each are adequate to support the analysis of online higher education institutions in
management.

Introduction
Learning analytics are a set of methods and techniques that collect, process, inform and successfully
produce machine-readable data continuously to advance the educational environment and reflect on
learning processes [1].
In general, these procedures initially emphasize the measurement and collection of data and
preparation for processing during learning activities. These procedures are focus on the analysis, the
presentation of data and the interpretation of results. Which then informs and empowers apprentices,
instructors, and organizations on the performance and achievement of objectives and facilitates
decision making accordingly [2].
Learning analytics and educational data mining share similar objectives and focus on the
intersection of learning science and data-based analytics but show differences in their origins, in
techniques, in fields of emphasis and in types of discovery.
The main objective of learning analytics is to extract information from educational data to support
education related to decision making. The information can be oriented to several interested parties [3].
Thus, teachers can receive feedback to evaluate the structure of their courses and the proper
development of the learning process. These data can help to quickly identify students who have
difficulty following the course, as well as the topics that generate the most confusion.
Students can receive recommendations on resources according to their performance, goals and
motivations, can graphically analyze the results of their learning process, compare them with those of
the rest of the class, and observe the performance and contributions related to collaborative activities.
Managers can use information to design a better allocation of human and material resources to
improve the overall quality of their academic experience [4].
In general, the fundamental difference between learning analytics and educational data mining is
that the latter focuses more on techniques and methodologies, while learning analytics deals more
with applications. However, these differences seem to be less and less noticeable as both fields evolve
over time. It seems that in the future it is very likely that the limits between learning analytics and data
mining in education will disappear.
Copyright © 2018, the Authors. Published by Atlantis Press. 270
This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Advances in Economics, Business and Management Research (AEBMR), volume 60

Knowing student's preferences is a complex task. Simply asking students what they want is too
invasive and prone to error, however, monitoring behavior discretely and finding meaningful patterns
is difficult and computationally slow.
However, capturing the user's exact preferences is an essential task because the systems of
tomorrow must respond dynamically to the changing needs of their students [5].
Recommendation systems learn the preferences of their students over time through both discrete
monitoring and relevant feedback. The construction of accurate user profiles is a difficult process. In
general, a recommendation system must work with variables that generate noise while also producing
meaningful information in regards to the interest of the student, with web browsing records or product
purchase records. To be successful, must be select a profile representation suggesting for the domain,
and the adequate profile techniques employed.
The recommendations are formulated by finding similar users and recommending the materials
they may like, which is known as collaborative filtering or filtering based on the content. Ontology is
a conceptualization of a domain in a comprehensible environment with a machine-readable format
consisting of entities, attributes, relationships and axioms. Ontology can provide a rich
conceptualization of the work domain of an organization, representing the main concepts and
relationships of work occupations [6]. These relationships could represent isolated information, such
as a phone number in the employee's home, or could represent an activity such as authoring a
document, or attending a conference.
Ontology is commonly used in knowledge-based systems to define the classes and structure of
supported domain models. Ontology is used in such a way that we can move forward using inference,
assignments to external knowledge bases and mappings for human comprehensible concepts.
There are a variety of techniques with this objective. Some of them come from classical statistics,
such as the Pearson-r Correlation, techniques include the Probabilistic Methods based on the Bayes
Theorem, the Analysis of Major Components, and others. Some are more recent such as learning
analytics and educational data mining [7].
As we have pointed out above, one of the biggest challenges that teachers have today is to evaluate
the students’ learning while they develop learning activities in the virtual environment. For this,
teachers usually choose information and communication technologies that allow them to have
evidence of the work carried out and thus be able to make an equitable evaluation, especially when
these activities require collaborative work.
The interaction of students in the virtual environment through online applications (social networks,
forums, virtual teaching and learning environments, etc.), provides data that is recorded in the servers,
called system logs. More and more LMS platforms and other technological environments make it
easier to obtain statistics on student performance and the degree of interaction with colleagues, for
example the number of connections or the number of messages sent. Even so, part of the data is not
stored in the logs of the system and are lost.
Thanks to the indicators and metrics analyzed we can have information about the performance of a
student throughout an activity and help, preventively, to achieve the learning objectives if any of the
indicators shows a deviation with respect to them [8]. The virtual teaching and learning environments,
Moodle, Canvas, Blackboard, etc. incorporate complements of learning analytics to present very
useful information about the activity that students develop on the platform. Some of these LMS
platforms provide data in the moment that the teacher can consult to answer different questions of the
teaching-learning process. For example, if the student formulates a doubt in relation to a concept, the
teacher can access the indicators and metrics to verify if the student has reviewed the material in
which the concept is developed. Otherwise, the teacher's response will be to indicate to the student
that they reads material reinforcing the key points.
The implementation of indicators and metrics in LMS platforms solves the key issue of automating
information and dispose of it in the moment. That is, access to data is not only more complete than
without learning analytics, it is also simpler and faster to access said data [9].
The teaching-learning process is developed beyond the LMS platforms. For example, cooperative
work among students can be enriched if the interaction is mediated by open resources. The activity of

271
Advances in Economics, Business and Management Research (AEBMR), volume 60

the students in these open resources is not stored the logs of the system, therefore, it is lost. Learning
does not occur only in closed and formal environments, but by definition exceeds these virtual
environments and it is neither appropriate nor possible to limit the activity to the technological
platform.
Learning analytics offers a more personalized learning because they allow students to have more
effective learning experiences [10]. This personalized learning experience begins with a development
by the common designers that is approaching the rhythm of each student from the use of data that is
automatically collected by most LMS platforms. Therefore, the interaction of the students during the
course leads them to be active agents in the development of their own curriculum.
In view of the above, we consider it interesting to use Learning Analytics to analyze the actions of
online higher education institutions in management. For this, it is necessary to be based on a
theoretical model that includes the main contributions at online higher education. This theoretical
model will allow identifying the components, dimensions, categories and variables to be used in
Learning analytics.
The research aim is to develop and produce a model based on theoretical contributions accepted by
the international community and an instrument to evaluate the quality of online higher education in
management based on the Learning Analytics methods.

Methodology
Our proposal was to develop an instrument for analyzing online learning environment quality. To
obtain evidence of this, we opted for a Learning Analytics methodology based on a theoretical model.
This methodological option was chosen because the research aim involved the process of teaching
and learning. Experimental manipulation and strict control were not involved because learning must
be considered in context and it is important to get direct data from the interaction among participants.
This instrument has been proved at OBS Business School. OBS Business School is an
international online higher education institution in management based in Spain that offers online
Master's Degree. Visualization is the learning analytics method chosen for this first approach. The
methodology is specified in three phases:
Phase 1. Development of the Theoretical Model on Teaching and Learning in Online Education
Institutions in Management. This theoretical model is based on the main international references.
Phase 2. Produce an instrument based on the components, dimensions, categories and variables of
the Theoretical Model on Teaching and Learning in Online Education Institutions in Management.
Phase 3. Exploratory phase to collect evidence of the adequacy of the instrument. The collection
will focus on one of the components.
Units of Analysis
It is essential to apply learning analytics based on an online higher education educational model in
management. The reason for this is that the key in the use of Learning Analytics lies in the
interpretation of the data obtained. Only if the data provides elements that can be analyzed together,
then the learning analytics can be the basis of appropriate decisions.
In this sense, the online teaching-learning model in management places student's actions focused on
an educational activity in the central position. Therefore, the cognitive, social and affective
dimensions are fundamental. These actions make sense with the interaction. The interaction with the
physical environment, the technologies and the materials, and the interaction with the social
environment, the teacher and the peers, allow us to convey the teaching-learning process.
The unit of analysis was divided into five components in relation to the online teaching and
learning process theoretical bases (see figure 1):

272
Advances in Economics, Business and Management Research (AEBMR), volume 60

Fig 1., Teaching and Learning in Online Education Institutions in Management.

• Component 1. The management of the institution- Management is based on standards

(standardizations, schedules and subject standards and evaluation standards), in the community
(the different groups that perform the functions) and in the division of labor (with the elements of
coordination included) (see figure 2) [11, 12].
• Component 2. The technological presence- The technological presence is defined as a wide range
of behaviors that constitute the online interaction styles of the students. Based on a selection of the
records of these behaviors that we look for in the online learning environment is what constitutes
the technological presence [13, 14].
• Component 3. The instructional design- Three primary levels of interactions illustrate the
relationship between learning theories, instructional strategies and student interactions with
human and non-human resources that include the use of emerging technologies to facilitate
electronic learning. Three primary levels are cognitive dimension, social interaction and human
and non-human interaction [15, 16].
• Component 4. The intra-psychological component- The analysis of the psyche, and therefore, the
analysis of the interior of the person when he learns can start from the distinction of the cognitive
dimension (perception, language, thought, reflection, etc.), the affective dimension (the emotions,
feelings, affections, etc.) and the motivational dimension (goals, expectations, achievements, etc.)
[17, 18].
• Component 5. The inter-psychological or social component- Inter-psychological or social
presence is the ability of learners to project their identities into the learning community of inquiry
emphasizing the traits that allow progress in learning [19, 20].
Model for Learning Analytics
Each component is associated with dimensions, categories and variables that allow a general level of
maturity of each component. These dimensions, categories and variables allow us to analyze and
describe the events of the classroom and the institution. Therefore, these dimensions, categories and
variables allow to operationalize theoretical constructs and specify them in appropriate factors and
statistics. Figure 1 shows an example: dimensions, categories and variables for the component 1
called the institutions management.

273
Advances in Economics, Business and Management Research (AEBMR), volume 60

Fig 2., Dimensions, categories and variables for the component 1 the institutions management.

Instrument
Based on the Teaching and Learning in Online Education Institutions in Management an instrument
has been created that collects the dimensions, categories and variables of each component. This
instrument should be used as a basis to analyze with Learning Analytics. At the same time, the
instrument allows obtaining data on the quality of the institution for an analysis prior to the use of
Learning Analytics. Table 1 shows the number of dimensions, categories and variables for each
component:

Table 1, the number of dimensions, categories and variables for each component.
Component Dimension Category Variable
The institution management 3 6 60
The technological presence 2 5 57
The instructional design 3 6 35
The intra-psychological component 3 6 42
The inter-psychological or social component 3 6 43
Total 14 29 237

274
Advances in Economics, Business and Management Research (AEBMR), volume 60

The structure of the scale is based on the methodology used by [12], for the evaluation of higher
education institutions. Said methodology proposes the following five levels of analysis: not, a few,
some, enough, many. The instrument is public at: https://es.surveymonkey.com/r/7DPWGD2

Results
The results have been obtained with the Atlas/ti. The results of the Master's Degree classroom
analysis show that the inter-psychological or social component can be adequately analyzed based on
the dimensions, categories and variables proposed at the Model Teaching and Learning in Online
Education Institutions in Management. The identified variables are shown at the figure 3.

Fig 3., Inter-psychological or social component net

Discussion and Conclusions

The components analyzed show that the dimensions, categories and variables for each are adequate to
support the analysis of higher education institutions in management. We consider that the
identification of the statistics is valid to sustain an evaluation process whether this process is based on
the Learning Analytics or based on the use of the instrument in its version of the questionnaire. In
future research, it is planned to use the Teaching and Learning in Online Education Institutions in
Management model to evaluate European and South American institutions.

Acknowledgement
This research was financially supported by OBServatory, OBS Business School.

275
Advances in Economics, Business and Management Research (AEBMR), volume 60

References
[1] R. Baker, Educational Data Mining: An Advance for Intelligent Systems in Education. IEEE
Intelligent Systems, 1541-1672, (2014) 78-82.
[2] J. Heath, E. Leinonen, An Institution Wide Approach Learning Analytics. In Anderson, M.
Developing Effective Educational Experiences through Learning Analytics, Hershey, IGI Global,
2016.
[3] H.I. Alabi, A Lenguaje and Space: Visualizing Learning Online. In Anderson, M. Developing
Effective Educational Experiences through Learning Analytics, Hershey, IGI Global, 2016.
[4] Z. Papamitsiou, A. Economides, Learning Analytics and Educational Data Mining in Practice: A
Systematic Literature. Review of Empirical Evidence. Educational Technology & Society, 17/4,
(2014) 49–64.
[5] S. L. Newman, B. Neimeyer, T. Seddon, A. Devos, Understanding educational work: exploring
the analytic borderlands around the labour that enables learning, Globalisation, Societies and
Education, 12/3, (2014) 65-73 https://doi.org/10.1080/14767724.2014.916609
[6] J. Watson, B. Kelly, R. Callingham,, J. M. Shaughnessy, The measurement of school students'
understanding of statistical variation. International Journal of Mathematical Education in Science and
Technology, 34 (2003), 1-29. 10.1080/0020739021000018791.
[7] N. Noss, L. Hoyles, Learning, By Constructing and Sharing Models. Mathematical Modelling,
Education, Engineering and Economics–ICTMA 12, (2007) 79-88.
https://doi.org/10.1533/9780857099419.2.79
[8] A. Pardo, G. Siemens, Ethical and privacy principles for learning analytics. British Journal of
Educational Technology, 45/3 (2014), 438-450. http://dx.doi.org/10.1111/bjet.12152
[9] J. Minguillón, C. Appel & F. Santanach, Learning analytics in practice: setting up a laboratory for
action research at the Universitat Oberta de Catalunya. In: Eighth EDEN Research Workshop, Oxford,
26th October, 2014.
[10] B. Dietz-Uhler, J. E. Hurn, Using Learning Analytics to Predict (and Improve) Student Success:
A Faculty Perspective. Journal of Interactive Online Learning, 12/1, (2013) 713-726.
[11]. Y. Engeström, Developmental Work Research: Expanding Activity Theory in Practice. Berlin:
Lehmanns Media, 2005.
[12] J. J. Alcolea et al, White book of institutional intelligence at universities, Madrid, OCU, 2013.
[13] E. Gregori, E. Torras, T.Guasch, Cognitive attainment in online learning environments:
matching cognitive and technological presence. Interactive Learning Environments,8 (2011), 1-17.
https://doi.org/10.1080/10494820.2010.531026
[14] L. Rourke, H. Kanuka, Barriers to critical discourse. International Journal of Computer
Supported Collaborative Learning, 2/1, (2007), 105–126.
[15] N. Sclater, A. Peasgood, J. Mullan, Learning Analytics in Higher Education, Bristol, JISC,
(2016).
[16] A. Hirumi, A framework for analyzing, designing, and sequencing planned elearning
interactions, Quarterly Review of Distance Education 3/2, (2002), 141-60.
[17] S.L. Jack, L.A. Anderson, Entrepreneurship education within the enterprise culture: Producing
reflective practitioners. International Journal of Entrepreneurial Behavior & Research, 5/3, (1999)
110-125. https://doi.org/10.1108/13552559910284074
[18] L.S. Vygotski, Thinking and Speech. New York, Plenun Press, (1997).
[19] B. Bloom, Taxonomy of educational objectives: Handbook I, The cognitive domain,
Nueva York, David McKay & Co., (1956).
[20] R. Bodily, K. Verbert, Proceedings of the Seventh International Learning Analytics &
Knowledge Conference, 309-318, Vancouver, British Columbia, Canada, March 13th, (2017).
[21] S. Ryza, U. Larenson, S. Owen, J. Wills, Advanced Analytics with Spark, O'Reilly, Boston,
(2017).

276