Publikationer från Malmö universitet
Endre søk
Link to record
Permanent link

Direct link
Publikasjoner (10 av 14) Visa alla publikasjoner
Tegen, A., Davidsson, P. & Persson, J. A. (2024). Activity Recognition through Interactive Machine Learning in a Dynamic Sensor Setting. Personal and Ubiquitous Computing, 28(1), 273-286
Åpne denne publikasjonen i ny fane eller vindu >>Activity Recognition through Interactive Machine Learning in a Dynamic Sensor Setting
2024 (engelsk)Inngår i: Personal and Ubiquitous Computing, ISSN 1617-4909, E-ISSN 1617-4917, Vol. 28, nr 1, s. 273-286Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The advances in Internet of things lead to an increased number of devices generating and streaming data. These devices can be useful data sources for activity recognition by using machine learning. However, the set of available sensors may vary over time, e.g. due to mobility of the sensors and technical failures. Since the machine learning model uses the data streams from the sensors as input, it must be able to handle a varying number of input variables, i.e. that the feature space might change over time. Moreover, the labelled data necessary for the training is often costly to acquire. In active learning, the model is given a budget for requesting labels from an oracle, and aims to maximize accuracy by careful selection of what data instances to label. It is generally assumed that the role of the oracle only is to respond to queries and that it will always do so. In many real-world scenarios however, the oracle is a human user and the assumptions are simplifications that might not give a proper depiction of the setting. In this work we investigate different interactive machine learning strategies, out of which active learning is one, which explore the effects of an oracle that can be more proactive and factors that might influence a user to provide or withhold labels. We implement five interactive machine learning strategies as well as hybrid versions of them and evaluate them on two datasets. The results show that a more proactive user can improve the performance, especially when the user is influenced by the accuracy of earlier predictions. The experiments also highlight challenges related to evaluating performance when the set of classes is changing over time.

sted, utgiver, år, opplag, sider
Springer, 2024
Emneord
machine learning, interactive machine learning, active learning, machine teaching, online learning, sensor data
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-17434 (URN)10.1007/s00779-020-01414-2 (DOI)000538990600002 ()2-s2.0-85086152913 (Scopus ID)
Merknad

Correction available: https://doi.org/10.1007/s00779-020-01465-5

Tilgjengelig fra: 2020-06-07 Laget: 2020-06-07 Sist oppdatert: 2024-03-06bibliografisk kontrollert
Tegen, A., Davidsson, P. & Persson, J. A. (2023). Human Factors in Interactive Online Machine Learning. In: Paul Lukowicz; Sven Mayer; Janin Koch; John Shawe-Taylor; Ilaria Tiddi (Ed.), HHAI 2023: Augmenting Human Intellect: . Paper presented at HHAI 2023, the 2nd International Conference on Hybrid Human-Artificial Intelligence, 26-30 June 2023, Munich, Germany (pp. 33-45). IOS Press
Åpne denne publikasjonen i ny fane eller vindu >>Human Factors in Interactive Online Machine Learning
2023 (engelsk)Inngår i: HHAI 2023: Augmenting Human Intellect / [ed] Paul Lukowicz; Sven Mayer; Janin Koch; John Shawe-Taylor; Ilaria Tiddi, IOS Press, 2023, s. 33-45Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Interactive machine learning (ML) adds a human-in-the-loop aspect to a ML system. Even though the input from human users to the system is a central part of the concept, the uncertainty caused by the human feedback is often not considered in interactive ML. The assumption that the human user is expected to always provide correct feedback, typically does not hold in real-world scenarios. This is especially important for when the cognitive workload of the human is high, for instance in online learning from streaming data where there are time constraints for providing the feedback. We present experiments of interactive online ML with human participants, and compare the results to simulated experiments where humans are always correct. We found combining the two interactive learning paradigms, active learning and machine teaching, resulted in better performance compared to machine teaching alone. The results also showed an increased discrepancy between the experiments with human participants and the simulated experiments when the cognitive workload was increased. The findings suggest the importance of taking uncertainty caused by human factors into consideration in interactive ML, especially in situations which requires a high cognitive workload for the human.

sted, utgiver, år, opplag, sider
IOS Press, 2023
Serie
Frontiers in Artificial Intelligence and Application, ISSN 0922-6389, E-ISSN 1879-8314 ; 368
Emneord
interactive machine learning, online learning, human factors
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-61687 (URN)10.3233/faia230073 (DOI)001150361600003 ()2-s2.0-85171485242 (Scopus ID)978-1-64368-394-2 (ISBN)978-1-64368-395-9 (ISBN)
Konferanse
HHAI 2023, the 2nd International Conference on Hybrid Human-Artificial Intelligence, 26-30 June 2023, Munich, Germany
Tilgjengelig fra: 2023-07-06 Laget: 2023-07-06 Sist oppdatert: 2024-02-26bibliografisk kontrollert
Persson, J. A., Bugeja, J., Davidsson, P., Holmberg, J., Kebande, V. R., Mihailescu, R.-C., . . . Tegen, A. (2023). The Concept of Interactive Dynamic Intelligent Virtual Sensors (IDIVS): Bridging the Gap between Sensors, Services, and Users through Machine Learning. Applied Sciences, 13(11), Article ID 6516.
Åpne denne publikasjonen i ny fane eller vindu >>The Concept of Interactive Dynamic Intelligent Virtual Sensors (IDIVS): Bridging the Gap between Sensors, Services, and Users through Machine Learning
Vise andre…
2023 (engelsk)Inngår i: Applied Sciences, E-ISSN 2076-3417, Vol. 13, nr 11, artikkel-id 6516Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

This paper concerns the novel concept of an Interactive Dynamic Intelligent Virtual Sensor (IDIVS), which extends virtual/soft sensors towards making use of user input through interactive learning (IML) and transfer learning. In research, many studies can be found on using machine learning in this domain, but not much on using IML. This paper contributes by highlighting how this can be done and the associated positive potential effects and challenges. An IDIVS provides a sensor-like output and achieves the output through the data fusion of sensor values or from the output values of other IDIVSs. We focus on settings where people are present in different roles: from basic service users in the environment being sensed to interactive service users supporting the learning of the IDIVS, as well as configurators of the IDIVS and explicit IDIVS teachers. The IDIVS aims at managing situations where sensors may disappear and reappear and be of heterogeneous types. We refer to and recap the major findings from related experiments and validation in complementing work. Further, we point at several application areas: smart building, smart mobility, smart learning, and smart health. The information properties and capabilities needed in the IDIVS, with extensions towards information security, are introduced and discussed.

sted, utgiver, år, opplag, sider
MDPI, 2023
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-60144 (URN)10.3390/app13116516 (DOI)001004726600001 ()2-s2.0-85163091186 (Scopus ID)
Tilgjengelig fra: 2023-06-07 Laget: 2023-06-07 Sist oppdatert: 2023-09-05bibliografisk kontrollert
Tegen, A. (2022). Interactive Online Machine Learning. (Doctoral dissertation). Malmö: Malmö universitet
Åpne denne publikasjonen i ny fane eller vindu >>Interactive Online Machine Learning
2022 (engelsk)Doktoravhandling, med artikler (Annet vitenskapelig)
Abstract [en]

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be mobile and of different types, i.e. there is a need to handle streaming data from a dynamic and heterogeneous set of sensors. In machine learning, the performance is often linked to the availability and quality of annotated data. Annotating data is in general costly, but it can be even more challenging if there is not any, or a very small amount of, annotated data to train the model on at the start of learning. To handle these issues, we implement interactive and adaptive systems. By including human-in-the-loop, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used in interactive machine learning is typically annotations of the data, i.e. correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy.

In this thesis we explore which interactive learning strategy types are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on the performance. We also study how a user who is not always reliable, i.e. who does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. Simulated experiments are compared to experiments with human participants, to verify the results. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when current estimations are incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data. The robustness of the machine learning algorithms differs, where e.g. Naïve Bayes classifier is better at handling a lower reliability of the user. We also present a systematic literature review on machine teaching, a subfield of interactive machine learning where the human is proactive in the interaction. The study shows that the area of machine teaching is rapidly evolving with an increased number of publications in recent years. However, as it is still maturing, there exists several open challenges that would benefit from further exploration, e.g. how human factors can affect performance.

sted, utgiver, år, opplag, sider
Malmö: Malmö universitet, 2022. s. 209
Serie
Studies in Computer Science ; 18
Emneord
Interactive Machine Learning, Active Learning, Machine Teaching, Online Learning
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-51987 (URN)10.24834/isbn.9789178772810 (DOI)978-91-7877-280-3 (ISBN)978-91-7877-281-0 (ISBN)
Disputas
2022-06-23, HS aula samt livestramas, Jan Waldenströms gata 25, Malmö, 10:00 (engelsk)
Opponent
Veileder
Merknad

In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Malmö University's products or services. Internal or personal use of this material is permitted.

Paper VI and VII appear in dissertation as manuscripts.

Tilgjengelig fra: 2022-06-03 Laget: 2022-06-02 Sist oppdatert: 2023-09-05bibliografisk kontrollert
Tegen, A. (2022). Machine Teaching: A Systematic Literature Review.
Åpne denne publikasjonen i ny fane eller vindu >>Machine Teaching: A Systematic Literature Review
2022 (engelsk)Manuskript (preprint) (Annet vitenskapelig)
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-52023 (URN)
Tilgjengelig fra: 2022-06-03 Laget: 2022-06-03 Sist oppdatert: 2024-01-16bibliografisk kontrollert
Tegen, A., Davidsson, P. & Persson, J. A. (2021). Active Learning and Machine Teaching for Online Learning: A Study of Attention and Labelling Cost. In: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA): . Paper presented at 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Pasadena, CA, USA, 13-16 Dec. 2021. Institute of Electrical and Electronics Engineers (IEEE)
Åpne denne publikasjonen i ny fane eller vindu >>Active Learning and Machine Teaching for Online Learning: A Study of Attention and Labelling Cost
2021 (engelsk)Inngår i: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Institute of Electrical and Electronics Engineers (IEEE), 2021Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Interactive Machine Learning (ML) has the potential to lower the manual labelling effort needed, as well as increase classification performance by incorporating a human-in-the loop component. However, the assumptions made regarding the interactive behaviour of the human in experiments are often not realistic. Active learning typically treats the human as a passive, but always correct, participant. Machine teaching provides a more proactive role for the human, but generally assumes that the human is constantly monitoring the learning process. In this paper, we present an interactive online framework and perform experiments to compare active learning, machine teaching and combined approaches. We study not only the classification performance, but also the effort (to label samples) and attention (to monitor the ML system) required of the human. Results from experiments show that a combined approach generally performs better with less effort compared to active learning and machine teaching. With regards to attention, the best performing strategy varied depending on the problem setup.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers (IEEE), 2021
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-51515 (URN)10.1109/icmla52953.2021.00197 (DOI)000779208200189 ()2-s2.0-85125866078 (Scopus ID)978-1-6654-4337-1 (ISBN)978-1-6654-4338-8 (ISBN)
Konferanse
2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Pasadena, CA, USA, 13-16 Dec. 2021
Forskningsfinansiär
Knowledge Foundation
Tilgjengelig fra: 2022-05-18 Laget: 2022-05-18 Sist oppdatert: 2023-09-05bibliografisk kontrollert
Tegen, A., Davidsson, P. & Persson, J. A. (2020). A Taxonomy of Interactive Online Machine Learning Strategies. In: Hutter F.; Kersting K.; Lijffijt J.; Valera I. (Ed.), ECML PKDD 2020: Machine Learning and Knowledge Discovery in Databases. Paper presented at European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (pp. 1-17). Springer
Åpne denne publikasjonen i ny fane eller vindu >>A Taxonomy of Interactive Online Machine Learning Strategies
2020 (engelsk)Inngår i: ECML PKDD 2020: Machine Learning and Knowledge Discovery in Databases / [ed] Hutter F.; Kersting K.; Lijffijt J.; Valera I., Springer, 2020, s. 1-17Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

In interactive machine learning, human users and learning algorithms work together in order to solve challenging learning problems, e.g. with limited or no annotated data or trust issues. As annotating data can be costly, it is important to minimize the amount of annotated data needed for training while still getting a high classification accuracy. This is done by attempting to select the most informative data instances for training, where the amount of instances is limited by a labelling budget. In an online learning setting, the decision of whether or not to select an instance for labelling has to be done on-the-fly, as the data arrives in a sequential order and is only valid for a limited time period. We present a taxonomy of interactive online machine learning strategies. An interactive learning strategy determines which instances to label in an unlabelled dataset. In the taxonomy we differentiate between interactive learning strategies when the computer controls the learning process (active learning) and those when human users control the learning process (machine teaching). We then make a distinction between what triggers the learning: active learning could be triggered by uncertainty, time, or randomly, whereas machine teaching could be triggered by errors, state changes, time, or factors related to the user. We also illustrate the taxonomy by implementing versions of the different strategies and performing experiments on a benchmark dataset as well as on a synthetically generated dataset. The results show that the choice of interactive learning strategy affects performance, especially in the beginning of the online learning process, when there is a limited amount of labelled data.

sted, utgiver, år, opplag, sider
Springer, 2020
Serie
Lecture notes in computer science, ISSN 0302-9743, E-ISSN 1611-3349 ; 12458
Emneord
interactive machine learning, active learning, machine teaching, online learning, streaming data
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-17435 (URN)10.1007/978-3-030-67661-2_9 (DOI)000717542900009 ()2-s2.0-85103280211 (Scopus ID)978-3-030-67660-5 (ISBN)978-3-030-67661-2 (ISBN)
Konferanse
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases
Tilgjengelig fra: 2020-06-08 Laget: 2020-06-08 Sist oppdatert: 2024-02-05bibliografisk kontrollert
Tegen, A. (2020). Approaches to Interactive Online Machine Learning. (Licentiate dissertation). Malmö: Malmö universitet
Åpne denne publikasjonen i ny fane eller vindu >>Approaches to Interactive Online Machine Learning
2020 (engelsk)Licentiatavhandling, med artikler (Annet vitenskapelig)
Abstract [en]

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem.

To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy.

In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

sted, utgiver, år, opplag, sider
Malmö: Malmö universitet, 2020. s. 129
Serie
Studies in Computer Science ; 10
Emneord
Machine Learning, Interactive Machine Learning, Online Learning, Active Learning, Machine Teaching
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-17433 (URN)10.24834/isbn.9789178770854 (DOI)978-91-7877-084-7 (ISBN)978-91-7877-085-4 (ISBN)
Presentation
2020-06-18, 10:15 (engelsk)
Opponent
Veileder
Forskningsfinansiär
Knowledge Foundation, 20140035
Tilgjengelig fra: 2020-06-09 Laget: 2020-06-09 Sist oppdatert: 2024-03-05bibliografisk kontrollert
Tegen, A., Davidsson, P. & Persson, J. A. (2020). The Effects of Reluctant and Fallible Users in Interactive Online Machine Learning. In: Daniel Kottke, Georg Krempl, Vincent Lemaire, Andreas Holzinger & Adrian Calma (Ed.), Proceedings of the Workshop on Interactive Adaptive Learning co-located with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2020): . Paper presented at Interactive Adaptive Learning 2020, Ghent, Belgium, September 14th, 2020. (pp. 55-71). CEUR Workshops
Åpne denne publikasjonen i ny fane eller vindu >>The Effects of Reluctant and Fallible Users in Interactive Online Machine Learning
2020 (engelsk)Inngår i: Proceedings of the Workshop on Interactive Adaptive Learning co-located with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2020) / [ed] Daniel Kottke, Georg Krempl, Vincent Lemaire, Andreas Holzinger & Adrian Calma, CEUR Workshops , 2020, s. 55-71Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

In interactive machine learning it is important to select the most informative data instances to label in order to minimize the effort of the human user. There are basically two categories of interactive machine learning. In the first category, active learning, it is the computational learner that selects which data to be labelled by the human user, whereas in the second one, machine teaching, the selection is done by the human teacher. It is often assumed that the human user is a perfect oracle, i.e., a label will always be provided in accordance with the interactive learning strategy and that this label will always be correct. In real-world scenarios however, these assumptions typically do not hold. In this work, we investigate how the reliability of the user providing labels affects the performance of online machine learning. Specifically, we study reluctance, i.e., to what extent the user does not provide labels in accordance with the strategy, and fallibility, i.e., to what extent the provided labels are incorrect. We show results of experiments on a benchmark dataset as well as a synthetically created dataset. By varying the degree of reluctance and fallibility of the user, the robustness of the different interactive learning strategies and machine learning algorithms is explored. The experiments show that there is a varying robustness of the strategies and algorithms. Moreover, certain machine learning algorithms are more robust towards reluctance compared to fallibility, while the opposite is true for others

sted, utgiver, år, opplag, sider
CEUR Workshops, 2020
Serie
CEUR Workshop Proceedings, E-ISSN 1613-0073
HSV kategori
Identifikatorer
urn:nbn:se:mau:diva-17673 (URN)
Konferanse
Interactive Adaptive Learning 2020, Ghent, Belgium, September 14th, 2020.
Tilgjengelig fra: 2020-07-03 Laget: 2020-07-03 Sist oppdatert: 2023-12-28bibliografisk kontrollert
Ghajargar, M., Persson, J. A., Bardzell, J., Holmberg, L. & Tegen, A. (2020). The UX of Interactive Machine Learning. In: NordiCHI 2020, 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society: . Paper presented at NordiCHI 2020, 11th Nordic Conference on Human-Computer Interaction. New York, USA: Association for Computing Machinery (ACM), Article ID Article No.: 138.
Åpne denne publikasjonen i ny fane eller vindu >>The UX of Interactive Machine Learning
Vise andre…
2020 (engelsk)Inngår i: NordiCHI 2020, 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, New York, USA: Association for Computing Machinery (ACM), 2020, artikkel-id Article No.: 138Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Machine Learning (ML) has been a prominent area of research within Artificial Intelligence (AI). ML uses mathematical models to recognize patterns in large and complex data sets to aid decision making in different application areas, such as image and speech recognition, consumer recommendations, fraud detection and more. ML systems typically go through a training period in which the system encounters and learns about the data; further, this training often requires some degree of human intervention. Interactive machine learning (IML) refers to ML applications that depend on continuous user interaction. From an HCI perspective, how humans interact with and experience ML models in training is the main focus of this workshop proposal. In this workshop we focus on the user experience (UX) of Interactive Machine Learning, a topic with implications not only for usability but also for the long-term success of the IML systems themselves.

sted, utgiver, år, opplag, sider
New York, USA: Association for Computing Machinery (ACM), 2020
Emneord
UX, User Experience, Machine Learning, Artificial Intelligence, Interaction Design
HSV kategori
Forskningsprogram
Interaktionsdesign
Identifikatorer
urn:nbn:se:mau:diva-24079 (URN)10.1145/3419249.3421236 (DOI)2-s2.0-85123040796 (Scopus ID)
Konferanse
NordiCHI 2020, 11th Nordic Conference on Human-Computer Interaction
Forskningsfinansiär
Knowledge Foundation
Tilgjengelig fra: 2020-10-31 Laget: 2020-10-31 Sist oppdatert: 2024-02-05bibliografisk kontrollert
Organisasjoner
Identifikatorer
ORCID-id: ORCID iD iconorcid.org/0000-0002-3155-8408