1. INTRODUCTION

The usage of electricity has always been a concern for business and home. As electric costs, ease of use and automated controlling has been rising; there is a need for a smart system that controls the power system to manage the power consumption and how to use electrical devices most efficiently. One of the practical ways that can do these issues is by using an energy management system that can help to control overall process in an automated way with the ability to the user to monitor the most power distribution and total cost as well as the ability to changes some settings as he desired [1]. However, the effective smart power system should be taken in consideration the three main aspects: first, is the power consumption when the system should accurately control the electrical devises in such way that reduced power consumption and offer fully controlling to electric devise by utilized artificial neural network to controls all operations. This is what our research aims at.

Recently, there are many types of research in the field of power management systems and most of these researches focus on a utilized low-cost component such as microcontroller units and single board computers along with low-cost sensors which become very popular to design and implements monitoring and control systems [2]. Some of these studies focused on uses microcontrollers and sensors that can detect motion (motion sensor) or light (light sensor) and then switch on a device such as lighting when it detects motion or light variation. These methods have an advantage of simplicity and low in cost. Goswami et al. (2009) [3], was designed and implemented an experimental modal for a power management system that utilized a simple microcontroller and sensors to monitoring and controlling lights and temperature. They test the accuracy of the system from the results obtained in overall processes. The accuracy is calculated via how intently the sensors can gauge the genuine or real world parameters values. The more accurate a sensor is, the better it will perform. Samadi et al. (2010) [4] proposed a smart power infrastructure that equipped with energy consumption controller; their work is focused on the interactions between energy provider and smart meters and they provided an algorithm to deal with the energy consumption. In any case, there is a disadvantage that these techniques are not very efficient because it depends on physical changes, for example, a motion sensor detects any movement in its sensing field but it cannot detect if that human, cars, or other things so the switch can be on even not need to be switch on when anything still moving such as animal. In another hand, the light sensors detect variation of light and it power on the light in case were the light is low and power off the light when light becomes high. Kleiminger et al. (2013) [5], point out that power management system in buildings and households can be achieved by using motion sensors and cameras, and they proposed a method that combined motion sensors with energy consumption. For their works, Passive Infrared Sensor (PIR) sensors have been installed in homes in addition to smart meters and plugs to measured power consumption. The had also used machine learning to recognize whether anyone is at home or not. They also used an Android application to gather data, which was installed on a tablet in the homes. This means that users had to explicitly change states from being home or away through the tablet. The reason for doing this was because they needed to label their gathered data in order to know whether users were home or away, since they used a supervised machine learning approach. However, one cannot be certain that the users always remembered to change states whenever they left their homes or arrived at home. Their users also admitted to forgetting it sometimes. They only focused on electrical power consumption, which means heating is not considered.

To overcome on the previous challenges, a new study focused on use image processing techniques such as a camera to sense varieties such as motion and light along with utilized artificial intelligence for object detection such as support vector machine (SVM), artificial neural network (ANN), etc. [6]. The SVM is needed low computational recruitment than ANN but it has lower recognition accuracy than ANN. ANN assumes a critical job in a wide scope of application and it has been utilized to serve the power management systems in recent years. A convolutional neural network, which is commonly referred to by conversational neural network (CNN), is another and precise kind of artificial neural network that can be used in image recognition. The CNN was published in 2012 by Krizhevsky et al. [7] (also referred as AlexNet), it gives a big improvement in the field of picture order and object localization that improved the classification performance and it applicable to a wide range of visualization tasks [8]. These techniques, however, is dependent on the computational requirement and typically needs a moderate processing specification in order to work smoothly which is not available in most Microcontroller Unit (MCU) and single board computers and in most cases required a PC based system in order to achieve that purpose [9]. Hence, in order to use the ANN with low processing units (MCU or single board computers) there is a need to design controller programs that can make the overall processing operation fast enough to work. This work is focused on used CNN to detect some important object that can be served in power management application such as controls lighting, TVs, computers, etc. Since the theory of object recognition accuracy is highly dependable on processing an image (enhancement and segmentation) to extract important features, in such way that with more details of image and efficient segment the objects inside the image and isolated it, high accuracy of recognition can be obtained. In the case where the image has many details and objects and when there is a need to process multi images in same time such as when need to recognized multiple objects from video streaming that may stream 25 fps (frame per second) that means the system should processing 25 images per second to recognize the desired objects. This heavy process needs to use a high computational requirement in order to recognize the object with high correctness in real time. However, the use of CNN with microcontrollers or even with single board computer (such as Raspberry Pi) to recognized objects in real time is a difficult task, because the low computational specs that make the process slow and not practical. Thus, this work has investigated the ability of improves the recognition in both accuracy and processing time by utilized saliency map to extract desired features. In Section 2, the brief review of recent artificial neural network based low-cost power management methods for household and buildings. In Section 3, the proposed methodology and its design described.

2. RELATED WORK

In this segment, we present the survey of current works that designed for low-cost energy management using the different approaches. In [10], author introduced the self-learning home energy management system. They demoralized the demand and supply side administration framework and coordinated for ongoing activity of a savvy home utilizing the neural networks for price forecasting, power alert functionality, price clustering, etc.

In [11], author proposed the control the executives framework with consistent learning. The smart fitting is outfitted with a sensor that can be traded on/off consequently as per nature, what's more, the sensor can be changed at whatever point for different conditions. Added to this, the profound learning model is carried into the entryway with territory care which is the possibility of haze registering.

In [12], another energy management method introduced for Hold Loads as well as large industrial loads. The Non-Intrusive Load Monitoring (NILM) algorithm presented by really actualizing sub-metering system for each store to estimate its modern improvement dependent on receptacle pressing calculations and criticism frameworks constrained by the Artificial Intelligence (AI) Calculation to finish up with a vitality proficient keen home and savvy lattices.

In [13], author designed optical stream criticism convolutional neural system according to the video stream to screen the home for energy efficiency. Their model uses rule-based channels before a data convolutional layer and the recorded optical stream for managing the optical progression of assortment. Distinguishing human stance is a key factor while fall events take after a falling stance. By sequencing housings of action it is possible to see a fall.

In [14], Internet of Things (IoT)-based home vitality the executives framework proposed with lightweight photovoltaic (PV) framework over powerful home territory systems, which bolsters the development of home scalable, reusable, and interoperable energy management method. They proposed methods with lightweight PV framework over powerful home territory systems, which bolsters the development of.

In [15], dispersed home vitality the board system and a conveyed calculation of machine booking presented for interest response, in view of the displaying of the cost capacity.

In [16], author presented an equipment plan and execution of keen minimal effort home vitality the executives framework with the elements of detecting, correspondences, and AI.

In [17], another solution for demand response for private clients proposed. A versatile neural fluffy derivation framework (ANFIS)-based home vitality controller was created to gauge vitality request as indicated by the client's way of life and natural elements identified with vitality utilization. In [18], author proposed the system for multiplatform area dependent on remote sensor and actuator systems for empowering productive and successful structure the board. In this research work, our objectives and methods are differing with focus on minimum cost and maximum accuracy using the deep learning technique. The methodology is described in next section.

3. APPROACH

The point of this work is to propose a smart low-cost power management system that can control power consumption and includes some appliances that can serve that purpose. The system can control the electrical devices such as switching lights, TV; air-condition units, ventilation units, and other such devices on/off depending on some aspects such as the presence of individuals, temperature, odors. This system should control all these aspects in a smart automated way for most building area, such as rooms, halls, corridors, etc. The system is composed from control unit which control all process, sensors (such as light sensors to detect light intensity, temperature sensors to detect temperature in desired area, ultrasonic sensor to detect motion and computer distance, an odors sensors to detect the odors), the power meter to measure power consumption, a camera to streaming constant video so as to distinguish the presence of individuals and their activities. Figure 1 demonstrates an example of the power the executives framework in buildings.

Figure 1

A sample of a power management system that covers all buildings area.

As shown in Figure 1, the power management system has multiple sensors distributed to cover all area. These type of sensor used is dependent on section need to control its power, for example light sensor used to control lighting that will switch on light when low light intensity detected, however, after switch lights on the system program start to analyze the area to identify if there is an individual in that area, if it recognized human, the lighting continue to switch on, otherwise the system will switch off lighting. Additionally, in the case of the large-scale room such as halls, the lighting needs to be in multiple groups and the program starts to compute the distance of person/persons from the lighting area. For example, if hall has three lighting groups (A, B, and C), in case when the person exists in the area of group (A) then the system will switch on group A lights and switch off lights of groups B and C, in case there is many persons that located in the area of group A and B the system will switch on lights of group A and B and switch off group C. Figure 2 illustrated the switching light strategy.

Figure 2

Switching light methodology by the proposed power management system.

The other task that can be achieved by used recognition process is to analyze human activity in order to control switching devices automatically in an intelligent way. For example, the system can predict that the person needs to watching TV, this can be achieved by our proposed method that assumes in case when person exists in hall and he set on sofa or chair with face directed to TV and the distance in desired range then the system is switch on TV and when person left the area the counter start and it then shut down TV after desired time (such as 3 minutes). This strategy can be also serving in some other application such as switching on/off computers, etc. Figure 3 illustrated the strategy of switching on/off the TV.

Figure 3

The smart power management system. (a) Switching light methodology, (b) TV on/off methodology.

The other control strategy of the proposed system is to control air conditioner units; this done by first indicates if there is a person in the room/hall area. In case there is a person, the system will be gathering temperature readings from a temperature sensor in that area, and then it switches on the air conditioner with ability to modify the temperature, mode, and fan direction to satisfying results. The sensors such as gas sensor and odors can be used from the power management system to control ventilation strategy that will start vent fan. Furthermore, to archives good power combustion some parts of the system such as camera not works until the system detect motion in a specific area then the system will power on all component located in that area to start analyzing the process. Additionally, the proposed system can be operating as a central unit or in dedicate units that work separately for each building section.

3.1. Hardware Design

In this method, we used the below hardware components with its configurations:

3.1.1. Raspberry Pi

A single board computer such as Raspberry Pi offers more computational specs than traditional MCU and it also available at low cost. Raspberry Pi, is a Mastercard measured PC initially intended for instruction, motivated by the 1981 BBC Smaller scale. It was intended to offer an ease gadget that would improve programming aptitudes and equipment. With its specs of small size and low price, it had been widely used to design low-cost control systems, especially for an application that needs more than a basic MCU. However, Raspberry Pi is slower than a cutting edge PC or work area yet is as yet a total Linux PC and can give all the normal capacities that suggest, at a low-control utilization level [19]. In this work, we have planned a brilliant power the board framework that employee Raspberry Pi as a detected, central processing control unit in order to design smart human detection system that can fully automated managing power such as switching lights on/off turn on the TV, etc. In this work, we have utilized Raspberry Pi 3 Model B, which has a central processing unit (CPU) type Broadcom BCM2837 that is a 64 bit Quad-core CPU and it operating in 1.2 GHz, it has 1GB RAM, Ethernet, wireless LAN and Bluetooth. It also has 40-pin extended GPIO, HDMI, CSI camera port to connect the camera to board, and DSI show port for interfacing a Raspberry Pi contact screen show. There is a Miniaturized scale SD port for stacking the working framework and putting away information.

3.1.2. Raspberry Pi camera

In this work, we used Raspberry Pi Camera V2 in order to stream live video from room to process the images and detect the presence of a human in the room. The camera underpins 1080p30, 720p60, and VGA90 video modes, just as still catch. It connects by means of a 15-cm strip link to the CSI port on the Raspberry Pi.

3.1.3. HC-SR04 ultrasonic sensor

The HC-SR04 ultrasonic sensor is an entirely moderate vicinity/remove sensor that has been utilized primarily for article shirking in different ventures. The HC-SR04 Ultrasonic Module has four pins, Ground, VCC, Trig and Reverberation. The Ground and the VCC pins of the module should be associated with the Ground and the 5 volts sticks on the raspberry pi board individually and the trig and reverberation pins to any Advanced I/O stick on the raspberry pi Board. The fundamental of operation is based on emits ultrasound at 40 000 Hz which goes through the air and if there is an article or hindrance on its way It will bob back to the module. Considering the movement time and the speed of the sound you can ascertain the separation. Figure 4 illustrated the ultrasonic sensor operation.

Figure 4

HC-SR04 ultrasonic sensor operation.

3.1.4. Temperature sensor

A Temperature/Humidity Sensor DHT11 has been used to monitor the temperature of each building section. This DHT11 Sensor can sense temperature and humidity and output signal with digital information. This sensor is low cost and offers high reliability with high performance, fast response, excellent quality, and anti-interference ability. The sensor has four-pin single row pin package and needs 5 V to operate, and it can detect the temperature in the range between from 0 to 50°C with error ±2, and it can detect humidity in the range from 20% to 90% RH with ± 5% RH. Figure 5 illustrated the ultrasonic sensor operation.

Figure 5

DHT11 temperature/humidity sensor.

3.1.5. Odors sensor

The MQ-4 methane gas sensor has been utilized to detect the bad odors and then power on vent fan to get rid of them. This sensor is compressed natural gas (CNG) sensor, which is ideal for sensing natural gas concentrations in the air (where the most important gas that needs to being de detected by this sensor is Methane-CH4). This sensor can detect concentrations of natural gas in the range from 200 to 10000 ppm. It has a fast response time with high sensitivity. This sensor needs a 5 V in addition to a load resistance, then it should be connected its output to an ADC. Figure 6 shows the MQ-4 gas sensor.

Figure 6

MQ-4 methane gas sensor.

The hardware component has been connected in the following steps. The camera has been connected with Raspberry Pi via onboard CSI camera port. The HC-SR04 is connected to Raspberry Pi pins where VCC connected to pin 2, the echo with pin 32 (GPIO 12), GND to pin 34, and trig with pin 36 (GPIO 16). The relay is also connected with Raspberry Pi pins where VCC connected with pin 4, GND with pin 6 and the TV to pin 3 (GPIO 2) and light to pin 5 (GPIO 3). Figure 7 shows the proposed system setup and pins connection.

Figure 7

Raspberry Pi pins, where blue box represents a connection with a relay, the yellow box represents the pin connection with the HC-SR04 sensor.

3.4. Experimental Results and Analysis

The test has been made with different parts. At first, we checked the operation of the power management system to manage all operation in real operation conditions. For this purpose, we instill the smart power management system in the building (apartment) that has two rooms, one hall, one bathroom. We distribute sensors separate lighting to two groups A and B. Then we have placed sensors (light, temperature/humidity, odors, and ultrasonic) sensors in the desired location within buildings area, where each room and hall have three sensors (light, temperature/humidity, and ultrasonic) sensors, the bathroom has three sensors (light, ultrasonic, and odors) sensors. And we placed a camera and processing unit in the hall. In addition, we connect TV to switch that connected to the system via relay as well as vent fan to control switching process. At first, we left the apartment from the main door, then the power management system successfully turns off lights and all connected devices. When open the door the system detects motion and switch on lights in corridors. When entering the first room the system has been detected the motion and switch on lights in the first room the temperature sensor starts measure room temperature and then switch on the air conditioner unit. When the second person enters, the second room the system switch on lights and air-conditioner unit too. Figure 8 shows the operation process when a person enters and left the room.

Figure 8

Light on/off operation based on human detection and distance measurement.

In the hall, same as in room when a person enters the hall, the system detects motion via ultrasonic sensor and directly switches on lights of groups A and B. The system also starts gathering temperature from sensor and switch on air conditioning unit. In the same time the system power on the camera and start recognized the person at the same time the ultrasonic sensor starts to compute the person located in the room. When person set on the sofa that is in front of the TV, in which the person becomes in A region, the system switches off group B lights, in the same time the system analyzed hall object and it successfully detects person and sofa as a result it switches the TV on. Figures 9 and 10 shows the detection process, recognition accuracy, and switching operation.

Figure 9

Human and chair/sofa recognition when human set on chair and sofa.

Figure 10

Human distance detection when moving in the room and switching operation.

As shown in Figure 9, we have tested the operation when human enter room. The system is successfully detecting human entering the room and isolated it from another object in the room. The system computes distance and switches on the light when human pass door and continue switch light on a human presence in the room then it has switched off when human move out the room through a door when distance reaches over limits. Figure 10 shows the test of TV switching on/off process. At first, the system detects the chair in the room and compute a distance between chair and TV, and if there is no human presence, then the system stays switch TV off. When human within the room it detects the human activity, if the human sat on the chair or sofa then it will switch the TV on, and if he stands off the system then put 1 minute then switch off the TV. Figure 9 shows the human distance calculation and switching operation as a human move over the room and sat in a chair or sofa. The results show that the precision of revealing is very high and it detects human and chair, sofa, in 100% from other room objects. In addition, the accuracy of recognition of separated object reach up to 99% and in relatively good response time about 0.9 seconds. The system also detects successfully detect human set in chair or sofa and detect both with high accuracy of detection reach up to 99% and in relatively good response time about 0.9 seconds. The ultrasonic also works well and detects distance but its value has some errors. The operation of switching light and TV works well and it runs in fast and smoothens.

The high accuracy of recognition as well as fast processing speed is related to utilized saliency algorithm with CNN. The main benefit from using of saliency algorithm is to extract only the most important objects from sense and it successfully segments these objects with good details that progress the presentation of the neural network in addition to reduce quality and as a result reduce computational requirement and increase training and recognition speed. Figure 11 shows the future extracted from high details image that has many objects. The results showed that this method efficiently removes foreground object from other objects in the sense with high details that can be detected easily and accurately with many recognition techniques.

Figure 11

Processing results. (a) Original image, (b): the feature extraction of saliency objects.

4. CONCLUSION AND FUTURE WORK

In this work, we have projected a smart power management system that utilized low-cost computer unit and neural network in order to control power consumption in the buildings and household. We have utilized low-cost component to design this system, in which, a single board computer that is Raspberry Pi has been utilized as a control and processing unit. Multiple sensors has been also utilized to serve the power management strategy, such as lighting sensor to detect lighting intensity, temperature/Humidity Sensor to detect room temperature, odors Sensor to detect bad odors, ultrasonic sensor to detect motion as well as determine person distance. The real operation test results show that the system worked smoothly and efficiently and it was successfully controlled different tasks at the same time without any failure. In the other hand the recognition process has been run in the fast-accurate way in order to control power system such as switching the device on–off in a smart way. The artificial intelligence part of the proposed system differs from other traditional approaches it makes the decision is more efficient than traditional methods based on sensor or human tracking. The main difference is both approaches based on tracking moving an object that can switch the device on–off even with any moving object such as animals or flying object, while the proposed system can detect human and some other object such as chair and sofa and detect human activity such as when human sat on the chair. The other important improvement of the wished-for method it can distinguish a human in 100% as well as in fast processing timeless one second even with a low computational computer such as Raspberry Pi. The improvement of human detection and recognition time is based on the utilized two effective methods. The first is the feature extraction method that depends on the theory of object detection-based attention and computes foreground of segments for an image. This model can automatically detect the distribution of foreground objects that can be found in the connected regions. Compared, our method doesn't depend on any learning; therefore, it doesn't need image retrieval and annotation. On the other hand, we wholly depend a shifted set of default bouncing boxes, of various scales and angle proportions to guarantee most items could be caught. They've demonstrated this foreground depending on these highlights beats standard data amplification frontal area and normal ghastly recurring foreground for that task of the detection foreground object. The experimental results show that the system runs smoothly and efficiently control power switch in real time and it is relatively got a fast processing time which gives the ability to use the Raspberry Pi as a defecated processing unit that makes the power management system low cost and practically useful. For future work, we like to explore the diverse profound learning and neural system classification techniques.