The concept of smart building is based on the deployment of Internet of Things (IoT) technologies to develop various building applications and services. Aided by the proliferation of smart devices, research in building automation has grown significantly. Nevertheless, these smart devices are integrated with sensors that can collect and share sensitive data and private information related to the building occupants, exposing them to a variety of privacy threats. Although research efforts to promote the development of privacy-aware solutions for smart buildings have been on the rise, a comprehensive review that summarizes these studies is lacking in the literature. This paper provides an extensive review of the studies related to sensing in smart buildings. It highlights privacy issues connected to sensing in smart buildings, provides mitigation strategies that can be deployed to minimize occupants’ privacy invasions, and discusses future research directions towards realising privacy-aware smart buildings. To fulfill the aim of this study, five research questions are formulated, which enable systematic navigation through existing studies related to the topic. These research questions are directed to providing answers to privacy related to data leakage, privacy connected to sensor types, privacy related to different applications, privacy concerns with sensor deployment locations and building types, privacy issues with data processing methods, and to highlight mitigation strategies for reducing privacy invasion. It further discusses the technical approaches, general principles, and design choices for privacy-aware applications which are relevant for guiding relevant stakeholders.

Occupancy estimation in smart buildings is essential for optimizing resource usage and enhancing operational efficiency. Existing estimation methods predominantly rely on cameras or advanced sensor fusion techniques, which, while accurate, are often expensive, invasive, and raise privacy concerns. Additionally, these approaches frequently require extra hardware, increasing installation complexity and operational costs. A significant gap in the literature lies in the limited use of existing smart building infrastructure, such as detection systems and booking data, for people counting. This study addresses these limitations by exclusively utilizing two binary PIR sensors (in-door and in-room) and booking data. Since PIR sensors and booking systems are already integrated into most smart building infrastructures, leveraging these existing resources helps reduce costs and simplifies implementation. The primary goal is to estimate the number of people between each in-door sensor trigger using machine learning models by incorporating people counting levels and time thresholds. Among the evaluated machine learning algorithms, the Extra Trees Classifier delivered strong performance, achieving 68.5% accuracy when the estimated occupancy differed from the actual count by at most one person, and 81.56% with a tolerance of two. These results are based on periods when the room was occupied. When both occupied and unoccupied periods were included, the accuracy was 96.10% for ±1 tolerance. Moreover, incorporating booking data enhanced people counting accuracy by 4%. The study also explores the method's ability to identify underutilization and overutilization by comparing estimated occupancy with booking records and seating capacity, thereby supporting enhanced space management in smart buildings.

People counting in smart buildings is crucial for the efficient management of building systems such as energy, space allocation, efficiency, and occupant comfort. This study investigates the use of two non-invasive binary Passive Infrared (PIR) sensors for estimating the number of people in seven office rooms with different people counting intervals. Previous studies often relied on sensor fusion or more complex signal-based PIR sensors, which increased hardware costs, raised privacy concerns, and added installation complexity. Our approach addresses these limitations by utilizing fewer sensors, reducing hardware costs, and simplifying installation, making it scalable and flexible for different room configurations, while also ensuring high consideration of privacy. Additionally, binary PIR sensors are typically part of smart building systems, eliminating the need for additional sensors. We employed several machine learning methods to analyze motion detected by binary PIR sensors, imp roving the accuracy of people counting estimates. We analyzed important features by extracting event count, duration, and density from sensor data, along with features from the room’s shape, to estimate the number of people. We used different machine learning models for estimating the number of people. Models like Gradient Boosting, XGBoost, MLP, and LGBM demonstrated superior performance for their strong ability to handle complex, non-linear relationships in sensor data, high-dimensional datasets, and imbalanced data, which are common challenges in people counting tasks using PIR sensors. These models were evaluated using performance metrics such as accuracy and F1-score. Additionally, the results show that features such as passage events and the number of detected events, combined with machine learning algorithms, can achieve good accuracy and reliability in people counting.

The integration of Internet of Things (IoT) networks and anomaly detection systems can enhance and improve spaces such as smart offices. By including anomaly detection, the reliability can increase and systems can become more useful. In this paper, we present an unsupervised transformer-based model capable of detecting anomalies in multivariate IoT sensor data, optimized for edge deployment. Through the use of real-world data provided by Sony's smart office system called Nimway, the model applies a reconstruction-based approach with multi-head attention to capture temporal and contextual dependencies. The model has been evaluated on test dataset, full dataset, and a synthetic anomaly dataset. The proposed transformer detected 65 anomalous data points distributed in 7 anomaly groups on test dataset, and 169 anomalies in 23 anomaly groups in the full dataset. By incorporating a dynamic quantization, model size has been reduced by 75% (to 1.14MB), which makes it more suitable for edge deployment and enables faster inference in realtime smart office environments. The quantized model achieved the test loss of 0.0495 (Mean Absolute Error). Moreover, models with and without quantization achieved the same performance by correctly flagging 10 out of 15 anomalies, indicating that the size reduction did not affect the model's accuracy. Preliminary results of attention-based feature importance visualization offer early insights into the features that explain the anomalies. Although the work is still in progress, this study addresses important aspects of IoT anomaly detection such as resource constraint, unsupervised learning and contextual anomalies.

The advances in Internet of things lead to an increased number of devices generating and streaming data. These devices can be useful data sources for activity recognition by using machine learning. However, the set of available sensors may vary over time, e.g. due to mobility of the sensors and technical failures. Since the machine learning model uses the data streams from the sensors as input, it must be able to handle a varying number of input variables, i.e. that the feature space might change over time. Moreover, the labelled data necessary for the training is often costly to acquire. In active learning, the model is given a budget for requesting labels from an oracle, and aims to maximize accuracy by careful selection of what data instances to label. It is generally assumed that the role of the oracle only is to respond to queries and that it will always do so. In many real-world scenarios however, the oracle is a human user and the assumptions are simplifications that might not give a proper depiction of the setting. In this work we investigate different interactive machine learning strategies, out of which active learning is one, which explore the effects of an oracle that can be more proactive and factors that might influence a user to provide or withhold labels. We implement five interactive machine learning strategies as well as hybrid versions of them and evaluate them on two datasets. The results show that a more proactive user can improve the performance, especially when the user is influenced by the accuracy of earlier predictions. The experiments also highlight challenges related to evaluating performance when the set of classes is changing over time.

The objective of this study is to provide a better understanding of the potential of using Artificial Intelligence (AI) to improve Public Transport (PT), by reviewing research literature. The selection process resulted in 87 scientific publications constituting a sample of how AI has been applied to improve PT. The review shows that the primary aims of using AI are to improve the service quality or to better understand traveller behaviour. Train and bus are the dominant modes of transport investigated. Furthermore, AI is mainly used for three tasks; the most frequent one is prediction, followed by an estimation of the current state, and resource allocation, including planning and scheduling. Only two studies concern automation; all the others provide different kinds of decision support for travellers, PT operators, PT planners, or municipalities. Most of the reviewed AI solutions require significant amounts of data related to the travellers and the PT system. Machine learning is the most frequently used AI technology, with some studies applying reasoning or heuristic search techniques. We conclude that there still remains a great potential of using AI to improve PT waiting to be explored, but that there are also some challenges that need to be considered. They are often related to data, e.g., that large datasets of high quality are needed, that substantial resources and time are needed to pre-process the data, or that the data compromise personal privacy. Further research is needed about how to handle these issues efficiently.

Buildings are rapidly becoming more digitized, largely due to developments in the internet of things (IoT). This provides both opportunities and challenges. One of the central challenges in the process of digitizing buildings is the ability to monitor these buildings' status effectively. This monitoring is essential for services that rely on information about the presence and activities of individuals within different areas of these buildings. Occupancy information (including people counting, occupancy detection, location tracking, and activity detection) plays a vital role in the management of smart buildings. In this article, we primarily focus on the use of passive infrared (PIR) sensors for gathering occupancy information. PIR sensors are among the most widely used sensors for this purpose due to their consideration of privacy concerns, cost-effectiveness, and low processing complexity compared to other sensors. Despite numerous literature reviews in the field of occupancy information, there is currently no literature review dedicated to occupancy information derived specifically from PIR sensors. Therefore, this review analyzes articles that specifically explore the application of PIR sensors for obtaining occupancy information. It provides a comprehensive literature review of PIR sensor technology from 2015 to 2023, focusing on applications in people counting, activity detection, and localization (tracking and location). It consolidates findings from articles that have explored and enhanced the capabilities of PIR sensors in these interconnected domains. This review thoroughly examines the application of various techniques, machine learning algorithms, and configurations for PIR sensors in indoor building environments, emphasizing not only the data processing aspects but also their advantages, limitations, and efficacy in producing accurate occupancy information. These developments are crucial for improving building management systems in terms of energy efficiency, security, and user comfort, among other operational aspects. The article seeks to offer a thorough analysis of the present state and potential future advancements of PIR sensor technology in efficiently monitoring and understanding occupancy information by classifying and analyzing improvements in these domains.

A 3D Scene Graph (3DSG) is a hierarchical 3D representation of a physical environment. Hydra is a promising real-time spatial perception framework for 3DSG developed by MIT Spark Lab. It uses sensor and camera data from an agent to construct a graph of the environment with five layers and continuously updates it. Albeit its utility and efficiency in generating and updating real-time 3DSG using visual-inertial sensors, Hydra and many other 3DSG frameworks ignore violating to identify private sensitive objects by segmenting, identifying, and reconstructing everything in data. Therefore, in this paper, we enhance Hydra to preserve sensitive data and increase the privacy-awareness of the framework. Accordingly, we propose a Privacy-aware Hydra (Pa-Hydra) framework, which integrates a state-of-the-art Object Detection (OD) algorithm that utilizes a proposed Filter Algorithm to cover sensitive objects with black boxes and prevent them from being constructed by Hydra. We implemented the framework with two popular OD algorithms with two different pre-trained models: You Only Look Once version 9 (YOLOv9) and Real-time Detection Transformer (RTDETR). We evaluated the algorithms using the COCO2017 validation dataset and observed the efficiency of the proposed framework on the uHuman2 (uH2) dataset.

Segmenting images is an intricate and exceptionally demanding field within computer vision. Instance Segmentation is one of the subfields of image segmentation that segments objects on a given image or video. It categorizes the class labels according to individual instances, ensuring that distinct instance markers are assigned to each occurrence of the same object class, even if multiple instances exist. With the development of computer systems, segmentation studies have increased very rapidly. One of the state-of-the-art algorithms recently published by Meta AI, which segments everything on a given image, is called the Segment Anything Model (SAM). Its impressive zero-shot performance encourages us to use it for diverse tasks. Therefore, we would like to leverage the SAM for an effective instance segmentation model. Accordingly, in this paper, we propose a hybrid instance segmentation method in which Object Detection algorithms extract bounding boxes of detected objects and load SAM to produce segmentation, called Box Prompted SAM (BP-SAM). Experimental evaluation of the COCO2017 Validation dataset provided us with promising performance.

Our dependency on Global Navigation Satellite System (GNSS) for getting directions, tracking items, locating friends, or getting maps of the world has increased tremendously over the last decade. However, as soon as we enter a building, the signal strength of the satellites is too low, and we need to resort to other technologies to achieve the same goals. An Indoor Positioning System (IPS) may utilize a wide range of methods for positioning a device, such as fingerprinting, multilateration, or sensor fusion, while using one or several radio technologies to measure Received Signal Strength (RSS) or Time of Arrival(ToA). Sensor fusion is an efficient approach where an Inertial Measurement Unit (IMU) is combined with, e.g., RSS measurements converted to distances. But this approach has significant drawbacks in areas where, e.g., walls or large objects obstruct the signal path, which introduces bias in the distance estimates. This paper addresses the bias caused by signal path obstruction by compensating the measured RSS with localized RSS attenuation adjustments and thereby increasing the accuracy of the sensor fusion model significantly. We also show that a system can learn the compensation parameters over time, reducing the installationefforts and achieving higher accuracy than a fingerprinting-based system.