Hand-Held Multispectral Camera for Crime Scene Investigation

John Federici

I. Technical Goals – Achievements and Failures
II. Hardware Development Summary
III. Characterization of Spectral Properties of Latent Blood
IV. Detection of Latent Finger Prints
V. Impact of the project and Recommendations Concerning Further Technology Development

See also the instructional videos on "Crime Scene and Evidence Photography"

I. Technical Goals – Achievements and Failures

The goal was to develop a compact, lightweight camera which would be able to simultaneous capture 20 wavelengths from the visible to the near infrared (400-1000nm) at video rates. The proposed approach focused on the camera development on the use of commercially available “monochrome” CMOS detectors. We used commercial-off-the-shelf (COTS) silicon CMOS detector arrays which will enable spectral detection throughout this range. The use of all COTS components during the hardware development enabled us to minimize the cost of development as we optimize the optical design, minimize size, weight and power requirements, and develop the software interface.

Receive our free monthly newsletter and/or job posting alerts Click to sign up

The overall goals from the original proposal were: Year 1 – Design, develop, and construct a prototype multispectral camera and illuminator. Year 2 – Demonstrate the utility of the camera system on latent blood stains and finger prints. Year 3 – (Option) Design and fabricate miniaturized packaging of optical components and electronics into a field deployable hand-held unit. Details for Year 3 are discussed in Section V.

Below we list some of the technical goals and metrics for the multispectral camera as discussed in the approved application as well as their final status to assess technical success.

Demonstrate acquisition of de-multiplexed images for 20 different colors in the 400-1000nm range. UPDATE: This goal included some failures as well as significant successes. The most significant failure was that the SI-Cube optical design which was envisioned in the original proposal had significant technical limitations as described in the final report. However, while the SI-Cube design approach failed, the research team was able to develop two other optical designs (the dichroic camera and Grid camera designs) which enabled the team to meet or exceed essentially ALL of the technical metrics for the multispectral forensic camera prototype. Specifically, for the target of 20 different colors in the 400-1000nm range, we have demonstrated 16 different colors in the 400-1000nm range using a Dichroic camera approach. The limitation of 16 spectra colors is predominately limited by the number of USB3 camera sensors which can be connected using the computing hardware and still operate at video rates.

Demonstrate a spectral resolution for each color of <30nm. UPDATE: EXCEEDED design specification. Narrow band filters in each spectral channel to achieve ~10nm resolution.

Demonstrate multispectral movies at ~5Hz frame rates for all 20 colors. Demonstrate higher frame rates as the number of color channels is reduced. UPDATE: COMPLETE. We have demonstrated VIDEO frame rates >10Hz with 9 channels in 640x480 pixel formats. We have demonstrated a 5Hz frame rate with 16 colors simultaneously in 640x480 pixel formats.

Design and Demonstrate small SWaP (Size, Weight, and Power) requirements for the camera system. The target weight for the handheld camera is <1 kg. UPDATE: COMPLETE. The weight of the camera system is dominated by the weight of the NUC microcomputer which is 520 gm. The weight including the housing, cables, USB port, etc., is approximately 1 kg.

Using 15M-pixel detector arrays, demonstrate multispectral, large pixel format images for latent evidence collection. UPDATE: COMPLETE. As described in the final report, for ‘still’ pictures, the format size is 2592 x 1944 for 16 spectral channels.

Demonstrate capability of accessing up to 80 spectral bands from 400-1000nm. UPDATE: COMPLETE. For both the GridCamera and Dichroic camera design, one can change the spectral bands simply by changing the 10nm spectral bandpass filters in any of the spectral optical channels.

Demonstrate a small SWAP illuminator utilizing xenon flashlamps and augmented by IR LEDs. UPDATE: COMPLETE. We find that xenon flashlamps are not required. Broadbanded illumination from quartz halogen lamps is sufficient for the 400-1000nm spectral range.

The main technical issue in Year 2 was to demonstrate the utility of the camera system on latent blood stains and fingerprints. Multispectral images were acquired under various conditions for various surfaces (eg. dark cloth, leather, paint):

Before the blood dries (no cleanup attempt). UPDATE: COMPLETE. Blood stains are visible on light colored substrates as well as dark substrates.

After the blood dries (no cleanup attempt). UPDATE: COMPLETE. Blood stains are visible on light colored substrates as well as dark substrates. Measured spectra on light color fabrics is consistent with prior reports of blood spectra. We observe a change in the spectra as the blood oxidizes. If the blood stains are deposited on lightly colored substrates (eg. white or yellow tee shirt), it is possible to specifically identify the stain as blood through the reflectivity spectra. If the blood stains are deposited on dark substrates (eg. a black tee shirt), it is difficult to separate the influence of the substrate’s spectra to positively identify the stain as blood. In this case, the multispectral camera functions as a ‘stain detector’.

Excess blood will be wiped off the various surfaces (basic cleanup). Images will be acquired before and after blood dries. UPDATE: COMPLETE. Blood stains are still detectable even on dark fabrics.

After an attempted cleanup of evidence to remove the stains using a variety of cleaning agents including bleach, and other household cleaners, multispectral images will be recorded. UPDATE: COMPLETE. Washing of stains on cloth in household washing machine completely removes evidence of stain except for stains on white fabrics. Less aggressive forms of cleaning leaves sufficient residues of the stains such that they are detectable with the multispectral camera.

Understand and optimize the illumination conditions for both blood stains and latent prints. UPDATE: Quartz halogen lamps provide broad coverage in the visible to near-infrared spectrum. For blood stains, the illumination angle is not critical. For latent prints, the angle of illumination is an important consideration for good contrast between the latent prints and background.

Develop or adapt image processing algorithms which construct false color images to help the investigator identify the presence of latent evidence. UPDATE: Complete for blood stains. The multispectral camera which was developed in this project CAN minimize confusion and clutter in what is presented to the user. This can be done in two modes: for targets with ‘light colored’ backgrounds (eg. white or off-white), there is sufficient spectral content in the visible range to spectroscopically identify blood stains. For ‘hard to see (visible light)’ backgrounds such as black fabric, one can use the near-infrared colors and appropriately use a false color for the background material to provide a false-color RGB image which provides good contrast between the stain and the background. However, due to the fact that the reflection spectrum of blood in the 800-1000nm spectral range is essentially featureless, while a stain can be isolated, it is not possible to spectrally identify that stain as blood.
COMPLETE for Latent Prints. There is sufficient pixel resolution in acquired fingerprint images to enable computer-based algorithms to match fingerprints. Under certain illumination conditions which are highly dependent on the substrate, latent prints can be imaged with the multispectral camera. However, there does not appear to be a significant advantage in multispectral imaging of fingerprints in the 400-1000nm spectral range.

After acquiring sets of prints from 10 different volunteers for testing. One set of prints will be examined using the illumination source and multispectral camera. The other set will be processed using the ‘gold standard’ of fingerprint powder. UPDATE: When the multispectral camera exhibits sufficient contrast in the print and background material, one is able to process the acquired fingerprint images to match the image to a library of fingerprint images.

II. Hardware Development Summary

In the original proposal, a new optical approach (Spectral Imaging Cube (SI-Cube)) was proposed to separate incoming white light images into multiple spectral mages. However, in the course of developing this design and applying it to our needs for a multispectral camera, several technical shortcoming of this approach became apparent. In summary, the restrictions imposed by the SI-Cube design (eg. restricted field of view, limited spectral tuning, difficulty in aligning relay mirrors) could only be rectified by increasing the physical size of the SI-Cube. However, if one increases the size of the SI-Cube to accommodate improved imaging, then one could use alternative optical designs. Rather than pursuing the SI-Cube option, we instead focused on both the dichroic camera and grid camera approaches which should provide similar performance to what was originally anticipated for the SI-Cube.

Figure 1: (a) A set of targets including leather (far left of middle row) and various patterned cloth upholstery stained with pig blood. The white disks in the image are used to calibrate the spectral content of the illumination source. Image acquired with a cell phone camera. (b) False color RGB image which retains the Blue (420 nm) and Green (500 nm) channels but replaces the Red channel with IR spectral data at 780nm. The contrast in the blood stains relative to the black shirt and patterned materials improves relative to ‘normal’ RGB images.

III. Characterization of Spectral Properties of Latent Blood

As an example, we created a ‘crime scene’ of various targets in Figure 1a with patterned and/or textured materials including leather and various cloth upholstery samples. Each target was stained with pig blood. The spectral data of this set of targets was acquired with the 8 spectral channel dichroic camera. Using our very basic analysis of replacing the ‘red’ channel with the infrared data from 780nm in an RGB false color image, one can improve the contrast in the blood stains on many of the fabrics as seen in Figure 1b. For example, the blood stain on the black shirt, which is invisible in Figure 1a, is now readily apparent in Figure 1b. Moreover, the blood stains in many of the patterned materials which are harder to visualize in RGB are readily apparent in Figure 1b.

Further exploration to determine what the spectral limitations in detecting blood stains on a variety of substrates yields interesting conclusions concerning the ability of multispectral imaging to detect latent blood stains. When the blood stains are on white or off-white substrates, there is sufficient ‘color difference’ between the substrate and blood stain to identify not only the presence of the stain, but also spectroscopically identify the stain as blood. As one would expect, the difficulty in identifying a stain as blood arises on darkly colored substrates. The central point is that ‘difficult to see stains’ result from the fact that the spectra of the substrate and the blood are similar. The technical challenge then is to de-convolve the ‘blood’ feature from the substrate spectral feature [1, 2].

For ‘hard to see (visible light)’ stains on dark clothing, the magnitude of the infrared reflection from blood is significantly different from the reflectivity of the substrate fabrics so that one can use the infrared reflectance to improve the contrast in the images to visualize the presence of the stain on the fabric. However, the reflection spectra of blood in the near-infrared range from 700- 1000nm only varies slowly with no distinguishing features: While multispectral imaging of blood stains in the 700-1000nm range can be used to detect the presence of a stain, there are not any distinguishing spectral features which enable identification of the stain as blood. Unless the IR spectra extends beyond ~1600nm, there are not any spectral features in the 700-1000nm near-IR spectral range in which our multispectral system operates which can identify the chemical composition of the stain as a blood stain (this is the same conclusion of Refs. [1, 2]).

Using the multispectral camera, we were able to demonstrate video-rate ‘stain detection’. The complete videos are included as electronic attachments to the final report. The person in the image has a blood stain on his black tee shirt just to the right of the armpit area. Again we emphasize that these video demonstrates the ability of the multispectral camera (a) to function in a ‘stain detection’ mode (b) to acquire video-rate multispectral images and (c) survey a large area in a prospective crime scene for evidence of stains. In this example, due to the video-rate acquisition, the stain detection can be performed on a moving test subject.

See also the instructional videos on "Evidence Photography — Fingerprints"

IV. Detection of Latent Finger Prints

The results with our multispectral camera on latent fingerprints exhibit some successes, some failures, and some inconclusive results. One success is that the spatial resolution/ pixel resolution of the monochrome cameras have more than enough spatial/ pixel resolution such that the images can be compared using computer software to a library of fingerprints.

The failure in the multispectral methodology for fingerprints is generally there is no advantage in synthesizing a multispectral (400-1000nm) false color image in our attempts to achieve better contrast in identify fingerprints. The first limitation is that optimal contrast depends strongly on the angle of illumination of the target relative to the background surface. This limitation renders our envisioned modality of a ‘quick survey’ of a large area to identify the presence of latent prints problematic. More importantly, since the reflectivity spectra of fingerprints is essentially featureless in the 400-1000nm range, the contrast is entirely predicated on the spectral content of the background material. Unfortunately, since the ‘color’ of the background material is highly variable, one can not a priori pick a single color for which one achieves the best contrast. For some backgrounds, a single monochromatic image may be ideal in improving the contrast between the fingerprint and the background substrate on which it is located. On other background materials, there is little difference in the choice of spectral color. The choice of the color to achieve the best contrast will be substrate dependent. Given the wide variety of background/ substrate materials and the need to adjust the illumination conditions to obtain the best contrast, multispectral imaging of fingerprints in the 400-1000nm does not appear to be a promising technology for crime scene investigation.

V. Impact of the project and Recommendations Concerning Further Technology Development

In our initial proposal, our work was inspired by previous multispectral camera development of Teledyne Scientific & Imaging (TSI) in 2012 [3, 4] : “… crime scene investigators are tasked with detecting and collecting physical evidence present at the scene, including body fluids, hairs, fibers and latent prints. These are often difficult to distinguish from the background and may present a formidable challenge to detect and identify.” Our view in the original proposal was that despite the clear value of multispectral detection capabilities, until now, multispectral technology has not been available in a form suitable for deployment to the crime scene: the technical challenges to be overcome are weight, cost, complexity, ruggedness, portability and user friendliness.[4]

In comparing our multispectral camera to that which was developed by Teledyne, our multispectral camera shows several major improvements compared to Teledyne: (1) the utilization of 16 colors (ie. multispectral) acquired simultaneously from the visible into the near-infrared and processed into real-time images. Teledyne’s system had only a maximum of 6 colors[4], which are acquired sequentially and require after-the-fact processing. (2) acquiring the multispectral images at video-rates. (3) with the advance in monochrome cameras with increased sensitivity, we did not need to build a specialized multicolor illumination source as was done by Teledyne, but instead could use inexpensive halogen lamps (4) the camera design ensures a small SWaP camera.

We found that our work on utilizing multiple cameras simultaneously was at the fore front of technology. There appears to be a push by vendors to solve many of the computer/ camera problems which we encountered and eventually solved: For example, E-CON Systems (which supplies our cameras) and NVIDIA Jetson (one of vendors whose minicomputer we evaluated) has recently teamed up to stream 6 synchronized cameras on a single Jetson. Our technical assessment is that with the continued development in the integration of multiple cameras with minicomputers, a small SWAP, hand-held multispectral camera is achievable in the next phase.

Earn a Degree in Crime Scene Investigation, Forensic Science, Computer Forensics or Forensic Psychology

The merit of the decision to further miniaturize our approach for multispectral imaging depends more on the utility of the multispectral approach to crime scene investigators in the field than on the technical feasibility of a small SWAP camera. Specifically, as detailed in the final report, spectroscopic identification of blood stains using the multispectral camera is not always possible due to the strong impact of the substrate’s color on the spectral analysis. However, the multispectral approach from 400-1000nm would make rapid ‘stain detection’ viable. Essentially the merit of further miniaturization depends on the answer to the following: Would the rapid (ie. video rate) survey of a crime scene for ‘stains’ which might possibly be blood (or semen or other stain of interest) be valuable to the crime scene investigator? While multispectral imaging of blood stains in the 700-1000nm range can be used to detect the presence of a stain, there are not any distinguishing spectral features which enable identification of the stain as blood. Essentially, if the blood stains are on light colored materials, the blood stains can be spectrally identified as blood. However, if the stains are on dark clothing, the multispectral forensic camera can detect the presence of a stain, but not positively identify the stain as blood.

In a field crime scene investigation, immediate feedback is required to quickly identify the presence and location of latent evidence. There is a huge practical advantage for efficiency in allocation of an investigator’s time and reduced cost in rapidly performing forensic imaging at the crime scene. This trend for rapid screening and investigative feedback can be accelerated by the deployment of our handheld multispectral camera. We envision two modes for our new forensic multispectral camera: Stain mode and Spectral Detection mode. In the ‘Stain’ mode, the camera system processes the multispectral data to enhance the contrast and identifies the presence of stains. While this mode does not identify the type of stain, we envision that this mode will enable the rapid processing of a crime scene to identify the location of stains for further analysis. Once the locations of stains in a crime scene have been identified, the camera system can also operate in a ‘spectroscopy’ mode to attempt to spectroscopically identify the composition of the stain and determine if it is a blood stain.

As described in Refs [1, 2], if one could acquire multispectral data in the 1600-2500nm range (rather than the 700-1000nm range of our instrument), one could spectroscopically identify blood stains even on dark clothing. In principle, our instrument design could be easily modified for the 1600-2500nm range. In that range, the predominate hardware changes would be the appropriate bandpass filters (which is straight forward) as well as monochrome cameras which are sensitive in this wavelength range. The issue for infrared cameras beyond 1600nm is one of cost and image format size. If as a rough bench mark, we use the parameters of a commercially sold 1600nm camera, we expect that there would be a significant increase in cost compared to our system. Infrared cameras (Edmund Optics) operating up to 1600nm cost approximately $2000 each: replacing each of our 16 monochrome cameras (cost about $200 each) with longer wavelength IR cameras would push the price tag of the multispectral camera system to a cost roughly ten times higher than an equivalent camera operating up to 1000nm. Moreover, the pixel size of the 1600nm cameras is only 768 x 494 pixels which is significantly less than the 5M format which we are currently using. The illumination source is not a problem. Halogen lamps have sufficient brightness in the 1600-2500nm range.

Lastly, consider the prospects for using our 400-1000nm multispectral camera for detection of latent evidence other than blood stains. If one were to use UV lights for illumination, our multispectral camera could easily detect the fluorescence of stains thereby enabling their detection. For example, stains of sweat, saliva and semen could be detected with our multispectral camera with a UV light source. Unfortunately, blood stains do not fluoresce. With regards to latent finger prints, due to the featureless spectrum of fingerprint oils in the visible to near-infrared range and the strong influence of the spectral content of the substrate material, the utility of multi-spectral imaging of fingerprints in the 400-1000nm range is questionable.

REFERENCES

G. Edelman, Spectral analysis of blood stains at the crime scene,PhD. 2014, Universiteit van Amsterdam.
G. Edelman, Identification and Age Estimation of Blood Stains on Coloured Backgrounds by Near Infrared Spectroscopy. Forensic Science International, 2012. 220(1-3): p. 239-244.
National Institute of Justice. 2012; Available from: http://nij.gov/topics/forensics/appliedforensics-awards-2012.pdf.
M. Mahajan, K. Zachery, W. Zhang, A. Brackley, and X. Long. Day and Night Real Time Signature Enhanced Crime Scene Survey Camera, Final Report: NIJ Award 2010-DN-BX-K144. 2013; Available from: https://www.crime-scene-investigator.net/CrimeSceneSurveyCamera.pdf.

The Crime Scene Investigator Network gratefully acknowledges the Office of Justice Programs’ National Criminal Justice Reference Service for allowing us to reproduce the article .