Recognition of Real-World Activities from Environmental Sound Cues to Create Life-Log
Mostafa Al Masum Shaikh, Keikichi Hirose, Mitsuru Ishizuk · InTech eBooks · 2011
There are several studies that collect and store life-log for personal memory.This chapter explains about a system that can create someone's life-log in an inexpensive way to share daily life events with family, friends or care-givers through simple text messaging with a notion to remote monitoring of someone's wellbeing.In the developed world where people are usually busier than ever, ambient communications through mobile media or the Internet based communication can provide rich social connections to their loving ones ubiquitously whom they care about by sharing awareness information in a passive way.For users who wish to have a persistent existence through ambient communication -to let someone else to know about their daily activity -new technology is needed.Research that aims to simulate virtual living or logging daily events, while challenging and promising, is currently rare.Only very recently the detection of real-world activities has been attempted by processing multiple sensors data along with inference logic for real-world activities.Detecting or inferring human activity using such simple sensor data is often inaccurate, insufficient and expensive.Therefore, this chapter discusses a technology, an inexpensive alternative to other sensors (e.g., accelerometers, proximity sensors etc.) based approaches, to infer human activity from environmental sound cues and common-sense knowledgebase of everyday objects and concepts.A system prototype to log daily events to infer activities in 'as you go' manner from environmental sound cues is explained with a few case studies.The input of the system is the patterns of sounds that are usually produced from activities (e.g., toilet flushing), occurring environmentally (e.g., road sounds) or due to interaction with the objects (e.g., cooking utensils clattering).A robust signal processing processes the input sound signal and Hidden Markov Model (HMM) classifiers are developed to detect predetermined sound contexts.Based on the detected sounds and along with the commonsense knowledge regarding human activity, object interaction, ontology of human life (e.g., living pattern of a single old man, or an old couple) and temporal information (e.g., morning, noon etc.) inference engine is employed to detect the activity and the surrounding environment of the person.Preliminary results are encouraging with the accuracy rate for outdoor and indoor related sound categories for activities being above 67% and 61% respectively. www.intechopen.comThe Systemic Dimension of Globalization 174 Environmental sound cues and life-logAlthough speech is the most informative acoustic event, but environmental sounds are also useful to process because those can provide useful information regarding context of the environment.In a given environment human-activity can be reflected by a variety of acoustic events, either produ c e d n a t u r a l l y o r b y t h e h u m a n b o d y o r b y t h e o b j e c t s manipulated or interacted.For example: Jingling sound of cooking utensils (like cooking pan, spoon, knife etc.) may lead to infer someone's cooking activity, likewise vehicle passing sound may lead to infer that someone is on the road, etc.Many sources of information for sensing the environment as well as activity are available (Chen et al., 2005;Philipose et al., 2004;Temko and Nadeu, 2005).In this chapter, we consider two objectives namely, soundbased context awareness, where the decision is based merely on the available acoustic information at the surrounding environment of the user and automatic life-logging, where the detected sound context infers an activity to be logged along with temporal information.Acoustic Event Detection (AED) is a recent sub-area of computational auditory scene analysis (Wang and Brown, 2006) that deals with the first objective.AED processes acoustic signals and converts those into symbolic descriptions corresponding to a listener's perception of the different sound events that are present in the signals and their sources.Life-log is a chronological list of activities performed by the user with respect to time.Such a list might indicate the user's well-being or abnormality according to the consideration of the person's self assessment or by someone else who cares about the person (e.g., relatives or care-givers).Therefore we apply the concept of AED to perform automatic generation of life-log.This life-log can be transmitted autonomously as a simple text message to someone else with the notion of ambient communication.Life logs include people's activities performed in specific locations at a specific time and it can be collected from various sources.We envisage that with the proliferation of computing power of hand held devices (HHD), availability of the Internet connectivity and improvements in communication technologies ambient communication will find a universal place at our daily life and allow us to realize virtual living through ambient social communication.Let's consider the following scenario of a globalized family.Scenario 1: Rahman family (Mr. and Mrs. Rahman) lives in Khulna, one of the metropolitan cities of Bangladesh.They have three sons living overseas, one in Texas, another in Ottawa and the youngest one in Bonn of Germany.Both Mr. and Mrs. Rahman are now at their age of over 50 and Mr. Rahman had a massive heart operation last year.Mrs. Rahman is also ailing from several sicknesses like diabetics, high blood pressure, etc.The three sons are always worried regarding the well being of their parents and consequently they often talk to their parents over the phones to know their whereabouts.Though calling to Khulna, Bangladesh from USA, Canada, and Germany is relatively cheaper now-a-days than before, but having a phone conversation with their parents is not always possible due to various reasons, for example, due to inconvenience in time differences (e.g., when it is 10 am in Khulna it is 11:00 pm in Texas, 12:00 am in Ottawa and 6:00 am in Bonn) that is, when the sons have convenient time to call, their parents are usually sleeping or resting.But they are often worried to know at least how their parents are doing everyday.Therefore, let's imagine that Rahman family has internet connectivity at their home and installed an inexpensive system capable of doing the followings.The system makes automatic lifelogging of daily activities by detecting and recognizing sound cues from their surrounding environments and sends email message(s) to their sons reporting their daily life-sketch.