Projects
Human Behaviour Understanding
Human behaviour understanding from videos is a complex problem in Computer Vision that is currently an actively researched area. It combines methods such as pose estimation, action recognition and making predictions on future behaviours. It has a wide variety of applications in areas such as assistive patient care, autonomous driving and even sports analysis [link].
Anomaly Detection
Considering the details of online, credit or debit transactions, such as location, merchant, device, amount and so on, AI algorithms can make predictions about the legitimacy of the transaction, and identify fraudulent pattern.
Aerial Image Analysis
Analyzing images from an eagle’s eye view can give us unique insights to better understand large areas that need monitoring, assessing environmental impacts and analysis for a number of industrial use cases. Mapping and modelling techniques applied to aerial imagery have huge potential in areas such as agriculture, climate, wildlife habitats, and more [link].
As a side project, I have explored the application of Computer Vision techniques to perform some detection and counting tasks which could be useful for industrial applications, such as in forestry and marine environmental monitoring.
Below are some results from an effort to solve the problem of automated counting for bundles of logs scattered over land and/or water at logging sites located in northern BC.
Counting of Small Objects from Point Annotations
The image below shows an attempt to annotate each log bundle with a rectangular bounding box, which is typically the format needed for application of popular object detection algorithms like Yolo. However, this annotation process quickly turns tedious and very challenging due to many overlaps of small objects in the sample images.
It is a lot easier to complete annotating the logs with a point-based annotation format. The point annotation is created at the center of each log bundle, to mark it as an object to be counted. The open-source tool used here is called ‘DotdotGoose’, which is easy and intuitive to use to create point annotations. This format would be much easier to scale when creating a larger number of manual annotations for the object to be counted.
The image on the left shows the results of training and applying the model LCFCN [1] , as a size-agnostic detection-based model, on test images to obtain an approximate count. The right image shows the results obtained from another method [2], which takes a query image together with a few exemplar objects from the query image (provided through the bounding box annotations in the top left quarter) and predicts a density map for the presence of all objects of interest in the query image.
Aerial Maritime Object Detection
The Aerial Maritime Dataset is an open-source dataset released on the Roboflow platform. The dataset, collected by a drone flown at 400ft, contains 74 images of aerial maritime photographs taken with via a Mavic Air 2 drone and 1,151 bounding boxes, consisting of docks, boats, lifts, jetskis, and cars. Some of the results (below) show near-perfect detection of the 5 categories of objects with a trained Yolov5 model.
Wildfire Smoke Detection
Similar to detection of maritime objects, we are also able to detect the presence of smoke in an aeria image. This application could be critical to alerting wildfires or other causes of large fires in remote areas with limited manual monitoring. It would be relatively easy to deploy this smoke detection model on an edge platform deployed at a remote observation site, to constantly monitor any fire and smoke appearances, and can be critical to active wildfire prevention through real-time alert generation.
MULTIMODAL RETRIEVAL AND GENERATION
Multimodal methods involve relating information from multiple sources. For example, images and text representations are matched for tasks such as retrieval and captioning. Image generation is also a very popular research area at the moment with the effectiveness of image generation from only a text description can lead to surprisingly good outcomes.
Autonomous Surface Water Navigation
Obstacle detection and avoidance in autonomous surface water navigation for small sailing vessels. Potential obstacles such as floating logs and other debris can be identified from video streams. This application would be crucial to generate timely alarm and prevent collisions for an autonomous marine navigation system.
Underwater video analysis
Underwater imagery poses unique challenges for automated analysis specific to the environment. Some challenges are a result of technical issues with underwater camera systems in ONC’s observatories. For instance, the camera’s auto-focus, which is designed for operation in air, may fail sometimes. Intermittent issues with the pan and tilt mechanism may cause the camera to rotate from its specified field of view, resulting in loss of relevant footage. This work aimed to recognise each experimental field of view (FOV) and the background for camera pan-tilt failure detection, loss of auto-focus and absence of lighting.
References
[1] Laradji, Issam H., et al. “Where are the blobs: Counting by localization with point supervision.” Proceedings of the European Conference on Computer Vision (ECCV). 2018.
[2] Ranjan, Viresh, et al. “Learning to count everything.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.