Video Activity Recognition

Published Conference Proceedings

  1. Jacoby, A., Pattichis, M., Celedón-Pattichis, S., and LópezLeiva, C. (2018). Context-sensitive Human Activity Classification in Collaborative Learning Environments. IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 141-144, 2018.
  2. Shi, W., Pattichis, M., Celedón-Pattichis, S., and LópezLeiva, C. (2018). , Robust Head Detection in Collaborative Learning Environments using AM-FM Representations. IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 65-68.
  3. Eilar, C. W., Jatla V., Pattichis, M. S., LópezLeiva ,C., and Celedón-Pattichis S., “Ditributed video analysis for the advancing out of school learning in mathematics and engineering project” , 2016 50th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 2016. pp. 65-68.


Hand Movement Detection in Collaborative Learning Environment Videos

This thesis explores detection of hand movement using color and optical flow. Exploratory analysis considered the problem component wise on components created from thresholds applied to motion and color. The proposed approach uses patch color classification, space-time patches of video, and histogram of optical flow. The approach was validated on video patches extracted from 15 AOLME video clips. The approach achieved an average accuracy of 84% and an average receiver operating characteristic area under curve (ROC AUC) of 89%.
UNM Digital repository Download Software

Context-Sensitive Human Activity Classification in Video Utilizing Object Recognition and Motion Estimation

This thesis explores the use of color based object detection in conjunction with contextualization of object interaction to isolate motion vectors specific to an activity sought within uncropped video. Feature extraction in this thesis differs significantly from other methods by using geometric relationships between objects to infer con- text. The approach avoids the need for video cropping or substantial preprocessing by significantly reducing the number of features analyzed in a single frame. The method was tested using 43 uncropped video clips with 620 video frames for writing, 1050 for typing, and 1755 frames for talking. Using simple KNN classification, the method gave accuracies of 72.6% for writing, 71% for typing and 84.6% for talk- ing. Classification accuracy improved to 92.5% (writing), 82.5% (typing) and 99.7% (talking) with the use of a trained Deep Neural Network.
UNM Digital repository Download Software

Human Attention Detection Using AM-FM Representations

The thesis explores phase-based solutions for (i) detecting faces, (ii) back of the heads, (iii) joint detection of faces and back of the heads, and (iv) whether the head is looking to the left or the right, using standard video cameras without any control on the imaging geometry. The proposed phase-based approach is based on the development of simple and robust methods that relie on the use of Amplitude Modulation - Frequency Modulation (AM-FM) models.For the students facing the camera, the method was able to correctly classify 97.1% of them looking to the left and 95.9% of them looking to the right. For the students facing the back of the camera, the method was able to correctly classify 87.6% of them looking to the left and 93.3% of them looking to the right. The results indicate that AM-FM based methods hold great promise for analyzing human activity videos.
UNM Digital repository

Distributed and Scalable Video Analysis Architecture for Human Activity Recognition Using Cloud Services

This thesis proposes an open-source, maintainable system for detecting human activity in large video datasets using scalable hardware architectures. The system is validated by detecting writing and typing activities that were collected as part of the Advancing Out of School Learning in Mathematics and Engineering (AOLME) project. The implementation of the system using Amazon Web Services (AWS) is shown to be both horizontally and vertically scalable. The software associated with the system was designed to be robust so as to facilitate reproducibility and extensibility for future research.
UNM Digital repository

Lesson Plan and Workbook for Introducing Python Game Programming to Support the Advancing Out-of-School Learning in Mathematics and Engineering (AOLME) Project

The Advancing Out-of-School Learning in Mathematics and Engineering (AOLME) project was created specifically for providing integrated mathematics and engineering experiences to middle-school students from under-represented groups. The thesis presents a new approach to introducing game programming to middle -school students that have undergone AOLME-training while still maintaining a fun and relaxed environment. The thesis provides a discussion of three different educational, visual-programming environments that are also designed for younger programmers and provides motivation for the proposed approach based on Python. The thesis details interactive activities that are intended for supporting the students to develop their own games in Python.
UNM Digital repository