Threat detection

Passive Millimeter Wave (PMMW) imaging occurs through the detection of radiometric temperature differences of the various objects in the scene. Unfortunately, due to their characteristics, the quality of the acquired PMMW images is very poor, see Figure 1. MMW images can be used to detect weapons or contraband concealed on subjects both in active and passive systems. They are been increasingly used in security applications to protect the security of public areas and services like trains and underground stations, airports, government buildings, military facilities, and checkpoints.

Figure 1

Unfortunately, since very frequently automatic detection algorithms produce very high false alarm rates, threats in the images must be detected by an operator. However, it has been estimated that the maximum operator screening rate is 120-140 person/hour while the Transportation Security Administration (TSA) aims at a processing speeds of 240 passengers per hour.

Passive MMW images have inherits limitations which make the detection of hidden threats a hard problem:

Previous works have addressed the automatic threat detection problem on MMW images. In "Image analysis for object detection in millimetre-wave images, two segmentation" methods are presented: K-means and Active Shape Models (ASM). K-means is applied over the intensity histogram to classify the background, the body and threats. Results show unconnected classifications and the need to introduce heuristics in order to determine the number of clusters. Classification is then done by performing maximum-likelihood estimation with the precomputed Probability Density Functions (PDFs) of background and foreground. To improve the connectivity of the segmented body an ASM is used over the results obtained using k-means. However, modelling the full range positions of the body with an ASM is a very complex task. In “Image processing techniques for metallic object detection with millimetre wave images”, weighted mixture models are estimated depending on the scenario. To classify objects, the Expectation Maximization (EM) algorithm is applied. Results show an improvement on the accuracy of the segmentation but still the method has problems with unconnected segmentation. In "Concealed object detection and segmentation over millimetric waves images,", a two stage algorithm is applied. Firstly, an Iterative Steering Kernel Regression (ISKR) or a Non Local Means denoising algorithms are applied to MMW images. Secondly, a Local Binary Fitting (LBF) algorithm is utilized to separate background, body and threats. Results show that the combination of ISKR and LBF algorithms obtains accuracy values around 90%. However, ISKR and LBF algorithms are time consuming algorithms that cannot be used in real time applications. We focus on the design of a real time algorithm to detect threats in MMW images. We propose an algorithm based on two phases: denoising and classification. In order to make our algorithm applicable in real time we propose a convenient combination of denoising filters to locate the possible threats on the image and the use a very simple but effective classification rule.

Database description

Figure 2: Examples of the captured images.
Left: with a threat on the leg.
Right: without threat

On July 2015, we worked with a passive wave millimiter camera, named "Wavecamm", in our lab at UGR. We have captured thousand of images to create a PMMWI threats database.



Figure 1: Capture Process

A comprehensive dataset of PMMWIs has been created. It consists of 3309, 125×195 images of 33 different people. Hidden objects are in the range 35×39 to 10×10 which in our images correspond roughly to 2752.39cm2 and 201.64cm2 respectively, smaller hidden objects are not considered to be relevant.

We took pictures of each person wearing 12 different objects in 10 body locations: forearm, chest, stomach, thigh, ankle (front), waist (side), armpit (side), arm, ankle (lateral) thigh (lateral), and 2 images without any objects. Images of people wearing simultaneously two objects in different locations were also taken. In summary the dataset consists of 463 pictures of people with no object, 2144 containing one object and 702 containing 2 objects.

Objects were simulated by bags containing substances with different millimeter wave responses: a cutter, 325g of gel, a 200g clay bar, a simulated gun, 200g of sugar, 200g of frozen peas, 150ml of cologne, 160g of gel, a bag with metal pieces, 200g of flour, a 50cl water bottle, and a 250ml hydrogen peroxide bottle, see figure 3. Notice that different object sizes were used.

Figrue 3: Simulated threats in the dataset: a cutter (1), 325g of gel (2), a 200g clay bar (3), a simulated gun (4), 200g of sugar (5), 200g of frozen peas (6), 150ml of cologne (7), 160g of gel (8) , a bag with metal pieces (9), 200g of flour (10), a 50cl water bottle (11), and a 250ml hydrogen peroxide bottle (12).

Figure 4 shows PMMWIs of subjects with simulated threats on different locations, the corresponding visual images are shown in figure 5. It is important to note that although in the experiments objects are visible and not hidden under clothing, this is irrelevant for PMMW sensors.

Figure 4: Examples of PMMWIs with (hidden) objects. Red boxes indicate objectlocations. Objects to be detected correspond to whiter areas on the body.

Figure 5: Corresponding visual images of the PMMWIs' examples in figure 4.

Visual images were taken at the same time as the millimeter ones. The objects are labeled on the visual images by its smallest bounding box. To transfer the object bounding box from the visible images to the PMMW ones a homography estimated from the visible image plane to the PMMWI one using a calibration pattern was applied. These bounding boxes are used to assess the performance of classifiers.

The database can be download here as a HDF5 file containing the following:

Blind PMMW Image deconvolution

It has been observed that PMMW cameras has a limited depth-of-focus and that, since the SNR of the images is very low, it is very difficult to adjust the focus correctly. We think that using blind image deconvolution techniques will help, first, to recover a sharper image where the hidden threats will be clearer and, second, increase the SNR making the detection process much easier.

PMMWI Super-resolution

Due to the small size of the PMMWI, they can take advantage of super-resolution (SR) techniques. In this project we have taken several sets of pictures, with small rotations and translations, to test SR techniques on them. Our hypotesis is that SR will increase the quality of the captured images and will help to detect the threat.