Multimodal Learning Analytics (MMLA) systems hold immense potential for understanding and shaping learning experiences. However, the lack of standardized design models hinders the consistent and effective development of these systems. This systematic review addresses this gap by identifying and analyzing existing MMLA design models and frameworks. We employed a rigorous search strategy aligned with established guidelines to identify relevant studies published in the past decade. Following a qualitative approach, the review combined narrative synthesis and thematic analysis to extract and synthesize key findings. Our analysis revealed a diverse landscape of MMLA design models and frameworks, varying in their scope (specific learning activities vs. comprehensive MMLA system design), level of detail (high-level guidance vs. specific steps), and development process (theoretical foundations vs. practical experience). Notably, several models addressed key design considerations and core commitments emphasized by recent research (e.g., data privacy, learner agency, inclusive learning environments). More importantly, the aggregation of these identified models suggests promise for the development of a more comprehensive design model. This is because individual models cover distinct areas and aspects with some intersections. The review also identified recurring themes related to factors influencing MMLA system design, including usability, scalability, and ethical considerations. Finally, we provide a discussion on potential strategies for a concrete development, offering valuable insights for researchers, developers, and educators seeking to harness the power of MMLA to improve learning outcomes.

In the dynamic landscape of digital education, the Glocal Classroom (GC) stands out as a multifaceted smart learning environment. The integration of Multimodal Learning Analytics (MMLA) comes as an intriguing proposition, promising insights into learning dynamics and enhancing educational outcomes. Encountering numerous interdependent considerations involved in the design and integration of MMLA systems, the MMLA design framework (MDF) addresses this challenge, providing a systematic approach. MDF consists of a phased and iterative method for designing MMLA systems. In this study, we delve into the details of the fifth phase, focusing on the development phase. Our primary objective is to assess and refine the applicability of MDF, by taking the integration of MMLA in GC as a use case. We analyze GC's technological infrastructure, evaluating existing hardware, network capabilities, and potential challenges. The central emphasis is on the technical architecture, specifically the hardware components supporting MMLA. By focusing on the technical complexities, the study provides insights into challenges and opportunities associated with MMLA implementation. The outcomes will deepen our understanding of technology in education and refine the MDF model, making it more effective for designing MMLA systems. 

Multimodal Learning Analytics (MMLA) systems integrate diverse data to provide real-time insights into student learning, yet their design faces the challenge of limited established guidelines. This study investigates essential design considerations for MMLA systems during the research and development phase, aiming to enhance their effectiveness in educational settings. A qualitative approach employing semi-structured interviews was conducted with a diverse group of researchers in the MMLA field. Deductive and thematic analysis were used to identify key design considerations, including technology integration, constraints and learning scenarios. The analysis further revealed intersections between various design considerations, both confirming existing themes and highlighting new emergent ones. Based on the findings, the MMLA Design Framework (MDF) was developed to provide a structured approach to guide the design and development of MMLA systems. This framework, along with the identified design considerations, addresses the lack of conventional practices in MMLA design and offers practical insights for practitioners and researchers. The results of this study have the potential to significantly impact both research and educational applications of MMLA systems, paving the way for more effective and informed designs.

This study explores the integration of Multimodal Learning Analytics (MMLA) within the dynamic learning ecosystem of the Glocal Classroom (GC). By employing the MMLA Model for Design and Analysis (MAMDA), our research proposes a conceptual model leveraging the GC’s existing infrastructure into an MMLA system to enrich learning experiences and inform course design. Our methodology involves a case study approach guided by the six phases of MAMDA. Building on previous studies, including a systematic mapping of MMLA research and an investigation into MMLA system design. We seek to employ MMLA insights to comprehensively understand the learning experience, identify issues, and guide improvement strategies. Furthermore, we discuss potential challenges, mainly focusing on privacy and ethical considerations. The result of this work aims to facilitate a responsible and effective implementation of MMLA systems in educational settings.

This study investigates the application of Multimodal Learning Analytics (MMLA) in language practice, specifically within the authentic and dynamic environment of language café settings. The MMLA Model for Design and Analysis (MAMDA), a design science approach, is utilized to systematically explore the requirements for designing the MMLA system. We identify and map three elements: 1) Learning indicators, referring to spoken language learning signs, such as tone, amount and frequency of speech, and pronunciation. 2) Respective modalities and sensors, referring to the format of data to be collected and 3) Analytics models, including NLP models, that can be employed to identify and process the modalities. We propose a conceptual system that utilizes AI voice assistant while simultaneously collecting audio data for MMLA to enhance language learning experiences. The system is meant for providing insights into learning patterns, participant engagement, and the overall effectiveness of language practice strategies. While presenting a novel system showcasing the use of AI and data analytics in a unique educational setting, the study's central focus is to test and critically reflect on MAMDA as a framework for designing and analyzing MMLA systems.

The conveniences of remote work are various, but a surge in cyberthreats has heavily affected the optimal processes of organizations. As a result, employees’ cybersecurity awareness was jeopardized, prompting organizations to require improvement of cybersecurity processes at all levels. This paper explores which cybersecurity aspects are more relevant and/or relatable for remote working employees. A qualitative approach via interviews is used to collect experiences and perspectives from employees in different organizations. The results show that human factors, such as trust in cybersecurity infrastructure, previous practices, training, security fatigue, and improvements with gamification, are core to supporting the success of a cybersecurity program in a remote work environment.

Understanding and improving education are critical goals of learning analytics. However, learning is not always mediated or aided by a digital system that can capture digital traces. Learning in such environments can be studied by recording, processing, and analyzing different signals, including video and audio, so that traces of actors’ actions and interactions are captured. Multimodal Learning Analytics refers to analyzing these signals through the use and integration of these multiple modes. However, a need exists to evaluate how research is conducted in the emerging field of multimodal learning analytics to aid and evaluate how these systems work. With the growth of multimodal learning analytics, research trends and technologies are needed to support its development. We conducted a systematic mapping study based on established systematic literature practices to identify multimodal learning analytics research types, methodologies, and trending research themes. Most mapped papers presented different solutions and used evaluation-based research methods to demonstrate an increasing interest in multimodal learning analytics technologies. In addition, we identified 14 topics under four themes––learning context, learning process, systems and modality, and technologies––that can contribute to the growth of multimodal learning analytics.

Designing MMLA systems is a complex task requiring a wide range of considerations. In this paper, we identify key considerations that are essential for designing MMLA systems. These considerations include data management, human factors, sensors and modalities, learning scenarios, privacy and ethics, interpretation and feedback, and data collection. The implications of these considerations are twofold: 1) The need for flexibility in MMLA systems to adapt to different learning contexts and scales, and 2) The need for a researcher-centered approach to designing MMLA systems. Unfortunately, the sheer number of considerations can lead to a state of "analysis paralysis," where deciding where to begin and how to proceed becomes overwhelming. This synthesis paper asks researchers to rethink the design of MMLA systems and aims to provide guidance for developers and practitioners in the field of MMLA.

The use of artificial intelligence and machine learning techniques across all disciplines has exploded in the past few years, with the ever-growing size of data and the changing needs of higher education, such as digital education. Similarly, online educational information systems have a huge amount of data related to students in digital education. This educational data can be used with artificial intelligence and machine learning techniques to improve digital education. This study makes two main contributions. First, the study follows a repeatable and objective process of exploring the literature. Second, the study outlines and explains the literature's themes related to the use of AI-based algorithms in digital education. The study findings present six themes related to the use of machines in digital education. The synthesized evidence in this study suggests that machine learning and deep learning algorithms are used in several themes of digital learning. These themes include using intelligent tutors, dropout predictions, performance predictions, adaptive and predictive learning and learning styles, analytics and group-based learning, and automation. artificial neural network and support vector machine algorithms appear to be utilized among all the identified themes, followed by random forest, decision tree, naive Bayes, and logistic regression algorithms.

This abstracts presents MBOX, a scalable and lightweight system that integrates data collection, data analysis and instructive feedback to evaluate participants’ engagement levels of learning activities.

Smart Learning Environments and Learning Analytics hold promise of providing personalized support to learners according to their individual needs and context. This support can be achieved by collecting and analyzing data from the different learning tools and systems that are involved in the learning experience. This paper presents a first exploration of requirements and considerations for the integration of two systems: MBOX, a Multimodal Learning Analytics system for the physical space (human behavior and learning context), and SCARLETT, an SLE for the support during across-spaces learning situations combining different learning systems. This integration will enable the SLE to have access to a new and wide range of information, notably students’ behavior and social interactions in the physical learning context (e.g. classroom). The integration of multimodal data with the data coming from the digital learning environments will result in a more holistic system, therefore producing learning analytics that trigger personalized feedback and learning resources. Such integration and support is illustrated with a learning scenario that helps to discuss how these analytics can be derived and used for the intervention by the SLE.

Drones have been discussed frequently in both governmental and commercial sectors for their normalization in the airspace. Nonetheless, drones bring diverse privacy concerns to users. In this paper, we explore the ramifications to data privacy from the perspective of drone owners. To investigate privacy in this context, four experiments targeting a commercial drone were conducted using open-source software. The experiments identified personal data (e.g., audio, video, and location) that are at risk of being compromised particularly through the execution of a basic deauthentication attack launched at a commercial drone. Our findings indicate the severity of risks affecting commercial drones. This makes the case for more effective privacy regulations and better guidelines suitable for securing drones.

Multimodal Learning Analytics (MMLA) provides opportunities for understanding and supporting collaborative problem-solving. However, the implementation of MMLA systems is challenging due to the lack of scalable technologies and limited solutions for collecting data from group work. This paper proposes the Multimodal Box (MBOX), an IoT-based system for MMLA, allowing the collection and processing of multimodal data from collaborative learning tasks. MBOX investigates the development and design for an IoT focusing on small group work in real-world settings. Moreover, MBOX promotes adaptation to different learning environments and enables a better scaling of computational resources used within the learning context.

Web accessibility is becoming a relevant topic with an increased number of people with disabilities and the elderly using the web. Numerous legislations are being passed that require the web to be universally accessible to all people, regardless of their abilities and age. Despite this trend, university curricula still teach traditional web development without addressing accessibility as a topic. To investigate this matter closely, we studied the syllabi of web development courses at one university to evaluate whether the topic of accessibility was taught there. Additionally, we conducted a survey with nineteen students who were enrolled in a web development course, and we interviewed three lecturers from the same university. Our findings suggest that the topic of accessibility is not covered in web development courses, although both students and lecturers think that it should. This generates lack of competence in accessibility. The findings also confirm the finding of previous studies that, among web developers, there is a low familiarity with accessibility guidelines and policies. An interesting finding we uncovered was that gender affects the motivation to learn about accessibility. Females were driven by personal reasons, which we attribute to females having an increased sense of empathy. Finally, our participants were divided in their opinions whether accessibility contributes to usability.

While security is often recognized as a top priority for organizations and a push for competitive advantage, repeatedly, Internet of Things (IoT) products have become a target of diverse security attacks. Thus, orchestrating smart services and devices in a more open, standardized and secure way in IoT environments is yet a desire as much as it is a challenge. In this paper, we propose a model for IoT practitioners and researchers, who can adopt a sound security thinking in parallel with open IoT technological developments. We present the state-of-the-art and an empirical study with IoT practitioners. These efforts have resulted in identifying a set of openness and security thinking criteria that are important to consider from an IoT ecosystem point of view. Openness in terms of open standards, data, APIs, processes, open source and open architectures (flexibility, customizability and extensibility aspects), by presenting security thinking tackled from a three-dimensional point of view (awareness, assessment and challenges) that highlight the need to develop an IoT security mindset. A novel model is conceptualized with those characteristics followed by several key aspects important to design and secure future IoT systems.

Today, the Internet of Things (IoT) is mainly associated with vertically integrated systems that often are closed and fragmented in their applicability. To build a better IoT ecosystem, the open IoT platform has become a popular term in the recent years. However, this term is usually used in an intuitive way without clarifying the openness aspects of the platforms. The goal of this paper is to characterize the openness types of IoT platforms and investigate what makes them open. We conducted a systematic mapping study by retrieving data from 718 papers. As a result of applying the inclusion and exclusion criteria, 221 papers were selected for review. We discovered 46 IoT platforms that have been characterized as open, whereas 25 platforms are referred as open by some studies rather than the platforms themselves. We found that the most widely accepted and used open IoT platforms are NodeMCU and ThingSpeak that together hold a share of more than 70% of the declared open IoT platforms in the selected papers. The openness of an IoT platform is interpreted into different openness types. Our study results show that the most common openness type encountered in open IoT platforms is open-source, but also open standards, open APIs, open data and open layers are used in the literature. Finally, we propose a new perspective on how to define openness in the context of IoT platforms by providing several insights from the different stakeholder viewpoints.

The Internet of Things (IoT) market is growing rapidly, allowing continuous evolution of new technologies. Alongside this development, most IoT devices are easy to compromise, as security is often not a prioritized characteristic. This paper proposes a novel IoT Security Model (IoTSM) that can be used by organizations to formulate and implement a strategy for developing end-to-end IoT security. IoTSM is grounded by the Software Assurance Maturity Model (SAMM) framework, however it expands it with new security practices and empirical data gathered from IoT practitioners. Moreover, we generalize the model into a conceptual framework. This approach allows the formal analysis for security in general and evaluates an organization’s security practices. Overall, our proposed approach can help researchers, practitioners, and IoT organizations, to discourse about IoT security from an end-to-end perspective.

The traditional way of teaching may no longer be sufficient to cope with current requirements specifically in the Internet of Things (IoT) domain. The case for this paper is related to an introductory programming course on JavaScript for the period of 2016-2018. In this study a multi-method approach for data collection is utilized. Project-Based Learning (PBL), Scrum and rapid prototyping are utilized to support student projects over the three years. Students developed a number of prototypes for various IoT domains related to ongoing research projects within our research center. The results show that students could easily use their JavaScript knowledge for any type of IoT development. PBL, Scrum and rapid prototyping help addressing uncertainties during the projects and balancing the team efforts for learning, development, problem solving and creativity. One of the outcomes of this paper confirms that smaller team sizes of students perform better during the project lifetime. In conclusion, focusing on knowledge increase, teamwork, collaboration, interaction, constant feedback, and adaptability should be considered a priority while exploiting teaching approaches such as PBL, Scrum and rapid prototyping for IoT development.

The objective of this paper is to understand the benefits of Web Technologies for Prototyping the Internet of Things (IoT), particularly JavaScript. We conducted an exploratory case with our students in introductory programming course on JavaScript. Scrum methodology and rapid prototyping is utilized to guide the students final project work. This work was conducted as a part of an ongoing research project on Smart Homes. Within five weeks the students developed fully working prototypes. The results show that students could easily use the JavaScript knowledge both for web and physical IoT-device programming. Additionally, Scrum methodology and rapid prototyping aspects provided a more structured process that helped the students in making quick design decisions, an important aspect specifically considering the constantly emerging IoT technologies. In conclusion, we believe that by simply leveraging well known mechanisms and architectures that Web already has in stake today we can easily build and deploy smart things in the IoT area.

In this paper we propose a model for Internet of Things (IoT) practitioners and researchers on how to use security thinking in parallel with the IoT technological developments. While security is recognized as a top priority, repeatedly, IoT products have become a target by diverse security attacks. This raises the importance for an IoT security mindset that contributes to building more holistic security measures. In understanding this, we present the state-of-the-art in IoT security. This resulted in the identification of three dimensions (awareness, assessment and challenges) that are needed to develop an IoT security mindset. We then interviewed four security and IoT-related experts from three different organizations that formed the basis for our pilot study to test the model. Our results show that the identified three-dimensional model highlights continuous security thinking as a serious matter to sustain IoT development with positive outcomes for its users.

Fast growth in a number of connected devices, heterogeneity, constrained resources, privacy, software upgrades and operational environment create important security related challenges in Internet of Things (IoT) domain. In this research, a literature survey of the state of the art is conducted related to security aspects. The results are validated by conducting qualitative interviews with IoT practitioners. The efforts have resulted towards identifying several security trends and challenges and some design aspects to be considered by IoT practitioners. The outcomes point towards that security is not only a technical problem but is more of an awareness, mindset, people and process issue. In this paper, a novel model is conceptualized with openness and security characteristics followed with several key aspects important to design an IoT system. This model emphasizes the human in the loop aspect, which would help to determine the dynamic requirements and design principles of IoT systems in a more open, common and secure way.

IoT systems provide new possibilities for augmenting daily activities of the people, which can be embedded in different settings and across contexts. Proprietary technologies have been extensively deployed throughout multiple IoT platforms, where such systems are closed in many cases. This closed nature complicates the development and customization processes for developers and end-users. To our knowledge, existing solutions do not address the sustainability requirements for the IoT architecture in terms of orchestrating smart services and devices in a more open and common way. Having a lot of standards brings us to the situation that we cannot utilize any standard into the design and development of IoT systems. Thus, we believe that we should start thinking less about standardization, and more about open approaches within IoT domain. In this paper, we use the concept of an open architecture that supports flexibility, customizability, and extensibility as a basis to address some of the above mentioned challenges.

Multimodal Learning Analytics (MMLA) captures, integrates and analyzes learning traces from different sources in order to obtain a more holistic understanding of the learning process, wherever it happens. MMLA leverages the increasingly widespread availability of diverse sensors, high-frequency data collection technologies and sophisticated machine learning and artificial intelligence techniques. The aim of this workshop is twofold: first, to expose participants to, and develop, different multimodal datasets that reflect how MMLA can bring new insights and opportunities to investigate complex learning processes and environments; second, to collaboratively identify a set of grand challenges for further MMLA research, built upon the foundations of previous workshops on the topic.

The Swedish construction industry is set to expand in the coming years between 2015-2019. The growth of the construction sector requires new workers across the broad spectrum of society. Therefore, an educated workforce is critical to the construction industry for security, professional development, quality of work, efficiency, and recruitment. However, creating an educated workforce for the construction industry requires the effective use of ICT to provide quality educational opportunities that engage and motivate people. The report explores three different education levels, from high school, university, and higher vocational education plus an industrial partner. The project conducted interviews, workshops, and surveys guided by an in-depth literature review that looked at research and state of the art. The results from the project show that teachers and students have digital competency, but learning materials and processes are open for improvement. Additionally, the findings from industry show a strong interest in better leveraging information technology for competence development. The report warrants a more detailed investigation in how to improve education in the construction industry.

Requirements gathering are an important aspect of application development, especially when users are people with special needs. Traditionally, this process is being conducted using conventional methods, such as interviews, workshops and questionnaires. These approaches, however, are unable to grasp the full context when collecting data from the communities of people with special needs, mainly because of the difficult access to participants and incomprehensiveness of the data gathered. To mitigate such issues, in this position paper, we argue that existing traditional methods could be complemented by means of Internet of Things. The immense amount of data gathered from various devices interconnected could help generate meaningful data that will complement the usually insufficient amount collected using traditional methods. This new approach is, however, associated with challenges that are discussed along with a possible scenario on how data complementing from traditional and the indirect method could be done.

This poster explores how to develop a working framework for STEM education that uses both human annotated and machine data across a purpose-built learning environment. Our dual approach is to develop a robust framework for analysis and investigate how to design a learning analytics system to support hands-on engineering design tasks. Data from the first user tests are presented along with the framework for discussion.

Access to web content continues to be a challenge for the visually impaired, as the needs of such community are very diverse. The access is further hindered by the fact that designers continue to build websites non-compliant with Web Content Accessibility Guidelines (WCAG). To better understand the needs of the visually impaired community, three workshops were organized with various stakeholders coming from three different countries. The results from the workshops suggest that one-solution-fits-all model is inadequate without considering the levels of visual impairment when providing customized web experience. A set of requirements devised from the workshops guided the process of building a middleware prototype. Using eight adaptation techniques, the prototype provides the required user experience based on users level of visual impairment. Preliminary evaluation of the middleware suggests that several adaptation techniques perform better with non-WCAG compliant websites compared to those being compliant.

Continuous change changes everything; it introduces various uncertainties, which may harm the sustainability of software systems. We argue that integrating runtime adaptation and evolution is crucial for the sustainability of software sys- tems. Realising this integration calls for a radical change in the way software is developed and operated. Our posi- tion is that we need to Design for Sustainability. To that end, we present: (i) the AdEpS model (Adaptation and Evo- lution processes for Sustainability) to handle and mitigate uncertainties by means of integrating runtime adaptation and evolution, and (ii) a set of engineering principles to de- sign software systems that facilitate the application of the AdEpS model to build sustainable software.

Software development today is based on a set of Agile approaches in the mindset of Lean. These emphasize the need for team collaboration and communication, rapid feedback and continuous learning. This creates the need for software intensive companies to educate their developers to these ways of working in a manner that allows fast adoption of the acquired skills also in professional capacity. As Agile emphasizes learning, teaching agile development can rely on modern, learner-centric approaches such as situated learning and the idea of a ipped classroom. In this paper we present and reflect upon a case study of two courses of teaching distributed agile development for software professionals based on modern learning theories engaging the learners directly in practice.

The continuous evolution of learning technologies combined with the changes within ubiquitous learning environments in which they operate result in dynamic and complex requirements that are challenging to meet. The fact that these systems evolve over time makes it difficult to adapt to the constant changing requirements. Existing approaches in the field of Technology Enhanced Learning (TEL) are generally not addressing those issues and they fail to adapt to those dynamic situations. In this chapter, we investigate the notion of an adaptive and adaptable architecture as a possible solution to address these challenges. We conduct a literature survey upon the state-of-the-art and state-of-practice in this area. The outcomes of those efforts result in an initial model of a Domain Specific Architecture to tackle the issues of adaptability and adaptiveness. To illustrate these ideas, we provide a number of scenarios where this architecture can be applied or is already applied. Our proposed approach serves as a foundation for addressing future ubiquitous learning applications where new technologies constantly emerge and new requirements evolve.