We propose two compression methods for the human motion in 3D space, based on the forward and inverse kinematics. In a motion chain, a movement of each joint is represented by a series of vector signals in 3D space. In general, specific types of joints such as end effectors often require higher precision than other general types of joints in, for example, CG animation and robot...
Facial paralysis is a condition causing decreased movement on one side of the face. A quantitative, objective, and reliable assessment system would be an invaluable tool for clinicians treating patients with this condition. This paper presents an approach based on the automatic analysis of patient video data. Facial feature localization and facial movement detection methods are...
We present a method for characterizing human activities using 3D deformable shape models. The motion trajectories of points extracted from objects involved in the activity are used to build models for each activity, and these models are used for classification and detection of unusual activities. The deformable models are learnt using the factorization theorem for nonrigid 3D...
Scale-invariant feature transform (SIFT) transforms a grayscale image into scale-invariant coordinates of local features that are invariant to image scale, rotation, and changing viewpoints. Because of its scale-invariant properties, SIFT has been successfully used for object recognition and content-based image retrieval. The biggest drawback of SIFT is that it uses only...
The automated analysis of activity in digital multimedia, and especially video, is gaining more and more importance due to the evolution of higher-level video processing systems and the development of relevant applications such as surveillance and sports. This paper presents a novel algorithm for the recognition and classification of human activities, which employs motion and...
The ability of detecting human postures is particularly important in several fields like ambient intelligence, surveillance, elderly care, and human-machine interaction. This problem has been studied in recent years in the computer vision community, but the proposed solutions still suffer from some limitations due to the difficulty of dealing with complex scenes (e.g., occlusions...
We present results of a study into the performance of a variety of different image transform-based feature types for speaker-independent visual speech recognition of isolated digits. This includes the first reported use of features extracted using a discrete curvelet transform. The study will show a comparison of some methods for selecting features of each feature type and show...
This article presents a framework for the design of concept detection systems for image and video indexing. This framework integrates in a homogeneous way all the data and processing types. The semantic gap is crossed in a number of steps, each producing a small increase in the abstraction level of the handled data. All the data inside the semantic gap and on both sides included...
In the growing market of camera phones, new applications for the visually impaired are nowadays being developed thanks to the increasing capabilities of these equipments. The need to access to text is of primary importance for those people in a society driven by information. To meet this need, our project objective was to develop a multifunctional reading assistant for blind...
Vision is one of the main sources through which people obtain information from the world, but unfortunately, visually-impaired people are partially or completely deprived of this type of information. With the help of computer technologies, people with visual impairment can independently access digital textual information by using text-to-speech and text-to-Braille software...
This paper presents an approach to movie video indexing that utilises audiovisual analysis to detect important and meaningful temporal video segments, that we term events. We consider three event classes, corresponding to dialogues, action sequences, and montages, where the latter also includes musical sequences. These three event classes are intuitive for a viewer to understand...
This work introduces two novel approaches for feature extraction applied to video-based Arabic sign language recognition, namely, motion representation through motion estimation and motion representation through motion residuals. In the former, motion estimation is used to compute the motion vectors of a video-based deaf sign or gesture. In the preprocessing stage for feature...
This paper proposes an automatic eye-wink interpretation system for human-machine interface to benefit the severely handicapped people. Our system consists of (1) applying the support vector machine (SVM) to detect the eyes, (2) using the template matching algorithm to track the eyes, (3) using SVM classifier to verify the open or closed eyes and convert the eye winks into a...
Data association and model selection are important factors for tracking multiple targets in a dense clutter environment. In this paper, we provide an effective solution to the tracking of multiple single-pixel maneuvering targets in a sequence of infrared images by developing an algorithm that combines a sequential probabilistic multiple hypothesis tracking (PMHT) and interacting...
This paper describes a rarity-based visual attention model working on both still images and video sequences. Applications of this kind of models are numerous and we focus on a perceptual image representation which enhances the perceptually important areas and uses lower resolution for perceptually less important regions. Our aim is to provide an approximation of human perception...
The goal of the See ColOr project is to achieve a noninvasive mobility aid for blind users that will use the auditory pathway to represent in real-time frontal image scenes. We present and discuss here two image processing methods that were experimented in this work: image simplification by means of segmentation, and guiding the focus of attention through the computation of...
We present a novel method to recognize a multiview face (i.e., to recognize a face under different views) through optimization of multiple single-view face recognitions. Many current face descriptors show quite satisfactory results to recognize identity of people with given limited view (especially for the frontal view), but the full view of the human head has not yet been...
A major challenge faced by the blind and visually impaired population is that of wayfinding—the ability of a person to find his or her way to a given destination. We propose a new wayfinding aid based on a camera cell phone, which is held by the user to find and read aloud specially designed machine-readable signs, which we call color targets, in indoor environments (labeling...
This paper presents a novel approach for face recognition based on the fusion of the appearance and depth information at the match score level. We apply passive stereoscopy instead of active range scanning as popularly used by others. We show that present-day passive stereoscopy, though less robust and accurate, does make positive contribution to face recognition. By combining...
The use of wavelets in the broad areas of source coding, communications, and networks is surveyed. Specifically, the impact of wavelets and wavelet theory in image coding, video coding, image interpolation, image-adaptive lifting transforms, multiple-description coding, and joint source-channel coding is overviewed. Recent contributions in these areas arising in subsequent papers...
Three different localized representation methods and a manifold learning approach to face recognition are compared in terms of recognition accuracy. The techniques under investigation are (a) local nonnegative matrix factorization (LNMF); (b) independent component analysis (ICA); (c) NMF with sparse constraints (NMFsc); (d) locality-preserving projections (Laplacian faces). A...
This paper presents a framework of content-adaptive packetization scheme for streaming of 3D wavelet-based video content over lossy IP networks. The tradeoff between rate and distortion is controlled by jointly adapting scalable source coding rate and level of forward error correction (FEC) protection. A content dependent packetization mechanism with data-interleaving and Reed...
This paper describes a behavioral model for affective social agents based on three independent but interacting parameter spaces: knowledge, personality, and mood. These spaces control a lower-level geometry space that provides parameters at the facial feature level. Personality and mood use findings in behavioral psychology to relate the perception of personality types and...
A quadratic image interpolation method is stated. The formulation is connected to the optimization of lifting steps. This relation triggers the exploration of several interpolation possibilities within the same context, which uses the theory of convex optimization to minimize quadratic functions with linear constraints. The methods consider possible knowledge available from a...
This work investigates the use of the 3D dual-tree discrete wavelet transform (DDWT) for video coding. The 3D DDWT is an attractive video representation because it isolates image patterns with different spatial orientations and motion directions and speeds in separate subbands. However, it is an overcomplete transform with 4: 1 redundancy when only real parts are used. We apply...
