SYSTEM AND METHOD FOR CRIME SCENE EVIDENCE ANALYSIS

Abstract

The present invention relates to a system (100) and method for crime scene evidence analysis, comprising training (200) and real-time testing (300) by detecting a living being in a sample video by mapping contour and invariant moment similarities of the living being against one or more reference images using a fast DTW technique; detecting an activity by the living being by calculating a pairwise distance between a plurality of landmarks for a plurality of postures of the living being; recognizing the activity as a crime based on a set threshold using a dynamic time wrapping classifier; identifying one or more weapons in the sample video using freeman chain code method, and predicting (400) one or more crime activities in the real-time video by mapping similarity index between the one or more crime activities in the real-time video and the sample video using statistical template matching approaches.

Specification

Description: As used in the description herein and throughout the claims that follow, the meaning of "a," "an," and "the" includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.
If the specification states a component or feature "may", "can", "could", or "might" be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.
Exemplary embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments are shown. This disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the disclosure to those of ordinary skill in the art. Moreover, all statements herein reciting embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).
Various terms as used herein are shown below. To the extent a term used in a claim is not defined below, it should be given the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.
In some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that can vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the invention are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable. The numerical values presented in some embodiments of the invention may contain certain errors necessarily resulting from the standard deviation found in their respective testing measurements.
The present invention relates to a system and method for crime scene analysis that, based on human activity, assists in the identification, prediction, analysis and risk assessment of crime evidence and criminals in real-time situations.
Referring to FIG. 1, a block diagram of a system (100) for crime scene evidence analysis is illustrated, according to an embodiment of the present invention. The system (100) comprises of a training module (200) and a testing module (300). Each of the training module (200) and the testing module (300) have a presence detection unit (201, 301) for detecting a living being in a sample/real-time video, a movement detection unit (202, 302) for detecting an activity by the living being, an activity recognition unit (203, 303) for recognizing the activity as a crime, and a weapon recognition unit (204, 304) for identifying one or more weapons in the sample video. Further, there is a prediction unit (400) for finally predicting one or more criminal activities in a real-time video.
Referring to FIG. 2, a flow diagram of a method for crime scene evidence analysis is illustrated, according to an embodiment of the present invention. As depicted in FIG. 2, in the training module, the presence detection unit (201) detects a living being in a sample video by mapping contour and invariant moment similarities of the living being against one or more reference images using a fast DTW technique. The DTW technique involves employment of one or more HU-moments in appearance, character, visual pattern recognition, and a set of seven digits, for example H0 to H6.
According to an exemplary embodiment, the Fast DTW technique calculates the Euclidean distance between the Human Template Image vector (test data vector) and assault, abuse, robbery, fight, vandalism, shooting, shoplifting, and non-human (blank) images vector (train data vector). The maximum threshold set may be set to between 3 to 4. The minimum distance value means a higher similarity. According to an embodiment, the living being may detected in scenarios including but not limited to abuse, arrests, arson, assault, car crashes, burglaries, explosions, fights, robberies, shootings, thieving, shoplifting and vandalism.
As a second step, the movement detection unit (202) detects an activity by the living being by calculating a pairwise distance between a plurality of landmarks for a plurality of postures of the living being using a NovelHAD technique. According to an embodiment, the plurality of landmarks are defined by left hip, left hand elbow, right hip, right hand elbow, left leg ankle, and right leg ankle and alike. According to an embodiment, the plurality of postures includes kicking, dragging, beating, seated shooting, standing shooting, and alike.
In an embodiment, the moments toward a person's hands and legs are indicative of human activity. A novel technique called NovelHAD is used based on the human kinematic prototype anatomy and the MediaPipe Pose (MPP) landmark detector feature extraction technique. The NovelHAD technique is a new approach based on pose landmarks of human objects to classify two categories of human activity, i.e., 'no activity (class-1)' or 'activity (class-0).
According to an exemplary embodiment, in Python, the computer vision MPP library has been used to estimate the plurality of postures and the plurality of landmarks of the living being in a sample video, wherein, MPP extracts 33 (0 to 32) - 3D ((x, y, z) and also visibility-v) landmarks or important points (feature vectors) on the human body. As a next step, A pose detection model may be used to detect the presence of bodies with a key pose landmark out of the plurality of landmarksto predict the human pose.
As an exemplary embodimemt, 13 distinct criminal sample videos have been experimented, whose landmarks and human positions are identified, Out of the 33 expected landmarks for the human stance, six landmarks related to the human pose can capture human movements. Those are "1) Left_Hip 2) Left_Hand_Elbow 3) Right_Hip, 4) Right_Hand_Elbow 5) Left_Leg_Ankle 6) Right_Leg_Ankle". Based on the equation 1 below, Euclidean distance metric is used to measure pairwise distances between (Left_Hip and Left_Hand_Elbow), (Right_Hip and Right_Hand_Elbow), and (Left_Leg_Ankle and Right_Leg_Ankle).

Euclidean distance d=√((x_2-x_1 )^2+(y_2-y_1 )^2 ) (1)

According to an exemplary embodiment, the three values mentioned above are utilized to identify the existence of a activity by the living being. When the left hand or right hand or both hands are moved at the same time or at different times in any direction then the value of the pairwise distance (Left_Hip_to_Left_Hand_Elbow_dist), (Right_Hip_to_Right_Hand_ Elbow_dist) is high, otherwise low. A low distance value, or <0.20 meters, has been seen when there is "no movement" in the hands. A high distance value of >0.20 meters has been found if there are "movements" in the hands. Next, for both the left and right hands, the pairwise distance value falls within the low and high ranges, namely (Left_Hip_to_Left_Hand_Elbow_dist = 0.00 to 0.20) and (Right_Hip_to_Right _Hand_Elbow_dist = 0.00 to 0.20).
According to an embodiment, the pairwise distance value between (Left_ Leg_Ankle_to_Right_ Leg_Ankle_dist) is high when the left, right, or both legs are moved in any direction; otherwise, it is low. A low distance measurement of <0.02 meters is achieved when there is "no movement" in the legs. A high distance value, or >0.12, has been found if there are "movements" in the legs. For legs, the pairwise distance value is (Left_Leg_Ankle_to_Right_Leg_Ankle_dist = 0.02 to 0.12), for both low and high ranges.
According to an embodiment, a decision parameter to identify the presence of activity by the living being may be created based on the aforementioned ranges for both hands and legs. "No activity" (Class-1) is the outcome when hands and legs are motionless, Euclidean distance values are low, and all internal conditions of the decision parameter are met. The result is "activity detected" (Class-0) if any one of the decision parameter's internal conditions is false. In this case, the hands' and legs' Euclidean distance values are high.
As a next embodiment, the activity recognition unit (203) is meant for recognizing the activity as a crime when the pairwise distance between the plurality of landmarks reaches/goes below a set threshold, wherein the recognition is accomplished using a dynamic time wrapping classifier. Dynamic Time Warping is an approach to assess the similarity/distance of two-time series or vectors. DTW first finds an optimal alignment between two vectors allowing one vector to stretch or compress to best fit the other vector.
According to an exemplary embodiment, the kicking test image may be matched against the feature vectors, containing video frames of human activities (P1 to P16). The threshold set for identifying human activity is 2.0. Table 8 shows the recognition of the human activity posture P1 as kicking since it has yielded the minimum distance with the template vector. Hence, classifying it as a posture for kicking.
According to an embodiment, the weapon recognition unit (204) for identifying one or more weapons in the sample video is a supplementary factor in the recognition of the activity as the crime using freeman chain code method. In an instance, different sizes and orientations of the weapon templates and non-weapons templates are considered. The Freeman Chain Code method is implemented by using OpenCV on contour images of the templates. According to an embodiment, the minimum distance value is considered the best DTW distance measure.
According to an embodiment, the testing module (300) having the above for units (presence detection module (301), movement detection unit (202), activity recognition unit (203), activity recognition unit (203), and weapon recognition unit (204)) for following the above 4 steps for one or more suspected living beings in a real-time video, wherein the detection of the one or more suspected living beings is accomplished based on proximity analysis using a kalman filter technique.
In the testing module (200), the Kalman Filter (KF) technique may be applied to UCF-101 public dataset videos (videos 1 to 6) for multi-object tracking to predict a video frame that contains very near to each other state (or position) humans in the video frames.
KF works with object position in 1st frame as input it tracks the object in subsequent frames. After the object is detected, precise localization makes it possible to refine the regions and encapsulate the object using bounding boxes (tracking window) and the centroid position of the object. According to an embodiment, the pose detection model is used to detect the presence of bodies with the plurality of landmarks and the plurality of postures.
In an instance, 0 to 32 expected landmarks for the human stance, six feature vectors related to the human posture are used to predict human activity. Those are 1) Left_Wrist 2) Right_Wrist 3) Left_Hip 4) Right_Hip 5) Left_Ankle 6) Right_Ankle. To quantify hand and leg movements, pairwise distance values are employed. Pairwise distances between landmarks are calculated using the Euclidean distance metric. Euclidean distance between pairs of landmarks of train data is calculated using Equation (1).
Finally, the prediction unit (400) predicts one or more crime activities in the real-time video by mapping similarity index between the one or more crime activities in the real-time video and the sample video using statistical template matching approaches.
The statistical template matching approaches include Cross Correlation (CC) 2) Normalized Cross Correlation (NCC) 3) Sum of Absolute Difference (SAD) statistical approaches. It has been analyzed that From three of the TMSA similarity measure values, the NCC has the highest similarity measure value to predict shooting, fighting, and kicking crime activities in humans. Therefore, the use of NCC calculation is very fast and efficient and highly recommended to statistically evaluate the similarity between datasets.
Referring to FIG. 3A, FIG 3B, and FIG 3C, sample images for mapping contour in a sample video using a fast DTW technique; exemplary images for plurality of landmarks for a plurality of postures of the living being using a NovelHAD technique; and a sit shooting weapon posture and weapon identification therewith are presented, according to an embodiment of the present invention.
Further, Referring to FIG. 4, a plurality of performance metrics analysed on implementing the system and method for the crime scene evidence analysis are shown, according to an embodiment of the present invention. Crime Scene Evidence Analysis Report has been generated for human detection, presence of human activity detection, human activity recognition with weapon detection, human activity prediction, and utilized UCF-101 surveillance CCTV footage crime dataset. There are 950 videos in the dataset. Each video has 24 frames per second. The 13 real-world anomalies that are captured on camera include abuse, arrests, arson, assault, car crashes, burglaries, explosions, fights, robberies, shootings, thieving, shoplifting, and vandalism. The dataset videos are in various complex situations such as occlusion, cluttered background, low-resolution, illumination, all the people not precisely facing the camera, low video quality and image quality, overlapped humans, moving humans, small objects (humans), different poses, people wearing different clothes, people laid down on the floor, night videos, various weather conditions.
Human detection (HD) tests were done on 100 video frames (both human and blank) using the Fast DTW technique based on Contour and Invariant moments. 50 human video frames (class 1) and 50 non-human (blank) video frames (class 0) were taken into consideration for the implementation. FIG. 4 presents test results for human detection including 97% accuracy, 96% precision, 97% recall, 3% FPR, and 2% FNR.
The presence of human activity detection (HAD) tests were done on 1000 video frames using the NovelHAD technique based on MediaPipe Pose and human landmarks. From those 500 video frames which were related to 'no activity,' (class-1) and the remaining 500 video frames contained a human 'activity' (class-0). FIG. 4 presents the presence of human activity detection (Yes or No) in video sequences with 95% accuracy, 94% precision, 96% Recall, 6% FPR, and 4% FNR.
Human activity recognition (HAR) (type of activity) tested over 1500 video frames with 750 crime activities frames and with 750 no crime activities frames by using the DTW classifier with MPP Framework. Kicking, dragging, beating, and shooting (sit, stand) activities are recognized with 98.6 % accuracy, 98.6% precision, 98.6% Recall, 1.4% FPR, and 1.4% FNR.
From 750 crime activities recognized in video frames, 400 with weapons and 330 without weapon video frames were considered. Gun Weapon Detection (WD) has been done by using the Freeman Chain Code and DTW technique with 99% accuracy, 98.75% precision, 99% Recall, 1.4% FPR, and 1% FNR.
Human activity prediction (HAP) tested over 200 videos to predict very near position human frames by using Kalman Filter technique. From the predicted frames 100 crime activities video frames and 100 no crime activities video frames were considered. MediaPipe Pose (MPP) Framework and Template Matching Statistical Approaches (TMSA) were used to predict shooting, fighting, and kicking activities with 95% accuracy, 94.11% precision, 96% Recall, 6% FPR, 4% FNR.
It should be apparent to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted. Moreover, in interpreting both the specification and the claims, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms "includes" and "including" should be interpreted as referring to elements, components, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced. Where the specification claims refers to at least one of something selected from the group consisting of A, B, C … and N, the text should be interpreted as requiring only one element from the group, not A plus N, or B plus N, etc. The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of the appended claims.
While embodiments of the present disclosure have been illustrated and described, it will be clear that the disclosure is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the scope of the disclosure, as described in the claims. , Claims:We Claim:
1) A system (100) for crime scene evidence analysis, comprising:
a training module (200), further comprising:
a presence detection unit (201) for detecting a living being in a sample video by mapping contour and invariant moment similarities of the living being against one or more reference images using a fast DTW technique;
a movement detection unit (202) for detecting an activity by the living being by calculating a pairwise distance between a plurality of landmarks for a plurality of postures of the living being using a NovelHAD technique;
an activity recognition unit (203) for recognizing the activity as a crime when the pairwise distance between the plurality of landmarks reaches/goes below a set threshold, wherein the recognition is accomplished using a dynamic time wrapping classifier;
a weapon recognition unit (204) for identifying one or more weapons in the sample video as a supplementary factor in recognition of the activity as the crime using freeman chain code method;
a testing module (300) having the above for units (301-304) for following the above 4 steps for one or more suspected living beings in a real-time video, wherein the detection of the one or more suspected living beings is accomplished based on proximity analysis using a kalman filter technique; and
a prediction unit (400) for predicting one or more crime activities in the real-time video by mapping similarity index between the one or more crime activities in the real-time video and the sample video using statistical template matching approaches.
2) The system (100), as claimed in claim 1, wherein the activity may be including but not limited to abuse, arrests, arson, assault, car crashes, burglaries, explosions, fights, robberies, shootings, thieving, shoplifting and vandalism.
3) The system (100), as claimed in claim 1, wherein the plurality of landmarks are defined on left hip, left hand elbow, right hip, right hand elbow, left leg ankle, and right leg ankle and alike.
4) The system (100), as claimed in claim 1, wherein the plurality of postures include kicking, dragging, beating, seated shooting, standing shooting, and alike.
5) A method for crime scene evidence analysis, comprising training for the evidence analysis, further comprising:
detecting a living being in a sample video by mapping contour and invariant moment similarities of the living being against one or more reference images using a fast DTW technique;
detecting an activity by the living being by calculating a pairwise distance between a plurality of landmarks for a plurality of postures of the living being using a NovelHAD technique;
recognizing the activity as a crime when the pairwise distance between the plurality of landmarks reaches/goes below a set threshold, wherein the recognition is accomplished using a dynamic time wrapping classifier;
identifying one or more weapons in the sample video as a supplementary factor in recognition of the activity as the crime using freeman chain code method;
following the above 4 steps for one or more suspected living beings in a real-time video, wherein the identification of the one or more suspected living beings is accomplished based on proximity analysis using a kalman filter technique; and
predicting one or more crime activities in the real-time video by mapping similarity index between the one or more crime activities in the real-time video and the sample video using statistical template matching approaches.
6) The system, as claimed in claim 1, wherein the activity may be including but not limited to abuse, arrests, arson, assault, car crashes, burglaries, explosions, fights, robberies, shootings, thieving, shoplifting and vandalism.
7) The method, as claimed in claim 1, wherein the plurality of landmarks are defined on left hip, left hand elbow, right hip, right hand elbow, left leg ankle, and right leg ankle and alike.
8) The method, as claimed in claim 1, wherein the plurality of postures include kicking, dragging, beating, seated shooting, standing shooting, and a like.

Documents