SkImager : a concept device for in-vivo skin assessment by multimodal imaging

A compact prototype device for diagnostic imaging of skin has been developed and tested. Polarized LED light at several spectral regions is used for illumination, and round skin spot of diameter 34 mm or 11 mm is imaged by a CMOS sensor via cross-oriented polarizing filter. Four consecutive imaging series are performed: (1) RGB image at white LED illumination for revealing subcutaneous structures; (2) four spectral images at narrowband LED illumination (450, 540, 660, and 940 nm) for mapping of the main skin chromophores and diagnostic indices; (3) video-imaging under green LED illumination for mapping of skin blood perfusion; (4) autofluorescence video-imaging under UV (365 nm) LED irradiation for mapping of the skin fluorophores. Design details of the device and its software as well as preliminary results of clinical tests are presented.


INTRODUCTION
* Skin diagnostics is often performed qualitatively, based on subjective perception and professional experience of the doctor.Several optical imaging devices are available commercially; however, many of them are narrowly focused to one defined skin pathology, bulky designed and too expensive for small clinics.Dermatologist surveys indicate that there are two most popular commercially available spectral imaging devices for skin diagnostics.MelaFind [1], a recently FDA-approved device, developed more than a decade ago, consists of multi-spectral illumination equipment, hand-held probe for taking skin images, and a computer with software for analysis of these images.The obtained spectral images provide information on the lesion's boundary, size, and morphology.SIAscope [2] predicts the skin structure from skin colours taken by a digital camera.Calibrated images captured in the red, green, blue, and near-infrared parts of the spectrum are computer-reconstructed into four images, reflecting the distribution of epidermal * Corresponding author janispi@latnet.lvmelanin and blood within the papillary dermis, as well as the thickness of collagen layer in the papillary dermis and the presence of dermal melanin.
Several alternative cost-efficient designs for spectral imaging of skin have been offered recently [3][4][5][6][7].This proof-of-concept study aims at the development of a more advanced, compact and handy wireless skin diagnostics and monitoring device, based on inexpensive components and smart software.Multimodal imaging is the key approach, including capture of a number of spectral and video images from the pathology area, with subsequent extraction of clinically significant information.Polarized LED light is used for illumination, and round skin spot of a diameter of 34 mm is imaged by a CMOS sensor via a cross-oriented polarizing filter.Four consecutive imaging series are performed in order to collect (1) RGB image at white LED illumination, helping to reveal hidden subcutaneous structures; (2) four spectral images at narrowband LED illumination for mapping of the main skin chromophores; (3) videoimages under green LED illumination for mapping of skin blood perfusion; (4) autofluorescence video-images under UV LED irradiation for mapping of the skin fluorophores.To improve the reliability of diagnostics, manipulation with maps of different parameters (i.e.extraction, summing, and division of images) is proposed as well.The first prototype version has been described earlier [8].Design details of an advanced prototype device with preliminary brand-name "SkImager" are discussed in this paper.

DESIGN OF THE DEVICE
The 2nd generation prototype is a battery-powered fully wireless device.Its main building blocks are CMOS image sensor, LED illumination system, on-chip microcomputer, touchscreen, memory, and rechargeable battery.The functional diagram of the device is presented in Fig. 1c.SoC (system on chip) module Nvidia Tegra 2 T20 with a dual-core ARM Cortex-A9 processor (clock frequency 1 GHz) is used as a central processing unit.It provides smooth operation of all components of the device.A 3 Mpix RGB CMOS matrix with 3.2 micron pixel size (MT9T031) serves as the image sensor; it is connected to the central processor via a 10-bit parallel line.Removable SD memory card stores the image information that can be transferred to external processor, e.g., PC.It can be done also via Mini-USB connector, which also ensures software installations and updates from outside.So, on-board and off-board calculations can be performed to extract parametric maps of the examined skin area.The main information input-output device is the built-in 4.3 inch/480 × 272 pixel touchscreen.It displays the operation mode and state of the device (battery charge level, clock, state of the memory card).Power switch button is placed on the side above the slot of the memory card (Fig. 1b).
The device has a holder with integrated contacts for battery charging (Fig. 1a).The Li-ion battery (3.6 V 4.6 Ah) ensures up to 15 h of operation, providing about 100 full measurement cycles.The power consumption under maximum load (display switched on, spectral, and video image recording and processing) is 3 W, or 1 W in the waiting mode.Dimensions of the device are 121 × 205 × 101 mm, weight about 440 g.
The specific skin illumination is performed by a ring of LEDs, surrounding the objective of CMOS (Aptina) image sensor (Fig. 1b).In total, 24 LEDs are operatedsets of 4 diodes emitting at five various wavelength bands (peaks at 365, 450, 540, 660, and 940 nm -Fig.2), and 4 white LEDs.Each set of equal LEDs is powered separately by the 6-channel LED driver (Fig. 1c), in order to provide the same illumination intensity and constant signal output by the visible and NIR diodes.The 3-colour RGB CMOS sensor has a resolution of 2048 × 1536 pixels with maximum frame rate 25 per second.The spectral sensitivity bands of the CMOS sensor are illustrated in Fig. 2b.
In order to minimize detection of surface-reflected light, crossed film polarizers cover the LED ring and camera objective (S-Mount M12 × 0.5), respectively.To keep fixed 55 mm distance to the skin, two easily changeable conical tips are used, with the target field diameters 34 and 11 mm, respectively.The latter is intended for more curved skin locations, e.g. on the nose.Internal surfaces of the tips are black coated and multiple-step shaped, in order to suppress any sidereflected light.

SOFTWARE
Operating system Windows Embedded Compact 7 (CE 7.0) was adapted for this device (Fig. 3).Specific graphic interface and management/processing software were originally developed in programming languages C and C++.Visual Studio 2008 and ActiveSync system is used as a development environment.Modular DirectShow multimedia framework technology is applied for recording, processing, compressing and storage of images (including video-images).The Nvidia developed DirectShow plug-ins are used; these take advantage of the GPU (graphics processing unit), in order not to overload the CPU (central processing unit).
The Toradex developed software libraries are used for management of the LED system and image sensor configuration.
The application GUI (graphical user interface) is designed to ensure easy use of the touchscreen.The central and largest part of the screen displays the taken images and the calculated parametric maps of skin (Fig. 4).The right part of the screen provides information on the device state; it comprises a digital clock, charge level of the battery, and fulfilment of the memory card.Four general purpose buttons are placed on the left side of the touchscreen; they change their function depending on the menu entered.Also a brief sound signal is generated at each press.The PC version of image processing was developed in the "Matlab R2011b" environment.It was based on the previous version [8] with several improvements.Interface of the application is shown in Fig. 5. Measurement numbers are first correlated to the patient data.Then the polarized white-illuminated image can be viewed, along with six other images of the same skin spot: redilluminated, green-illuminated, blue-illuminated, and infrared-illuminated, fluorescence video-image under UVA-irradiation and photoplethysmography video-image, reflecting the superficial microcirculation.Besides, eight parametric maps of the illuminated skin spot are available: epidermal melanin distribution, dermal melanin distribution, total haemoglobin distribution, bilirubin distribution, erythema index map, fluorescence intensity distribution, map of fluorophore clusters (derived from the image of photobleaching rates [9]), and skin blood perfusion map.
Algorithm of the image processing is illustrated in Fig. 6.Each measurement is recorded with number, name of patient, and diagnosis.The selected image cube is loaded in the data array and multi-images are converted to RGB single data format.Then the background and illumination uniformity corrections are applied.The skin chromophore maps are calculated from the four spectral images, and the perfusion map and fluorophore distribution map -from the video-images taken at green and ultraviolet illumination of the skin, respectively.Manual segmentation function is integrated in the application in order to identify skin areas with similar spectral properties; several images can be overlapped to obtain more detailed diagnostic information, if necessary.

Calculation of skin chromophore maps
Figure 7 shows images under polarized white, red, green, blue, infrared, and ultraviolet illumination.Monochrome images are used for calculation of skin chromophore maps related to hemoglobin, bilirubin, and melanin.
The combinations of monochrome images were transferred into false-color images.The chromophore maps reflect the spatial chromophore concentration distribution in skin.From spectral images three types of chromophore maps are calculated: hemoglobin, bilirubin, and melanin.The hemoglobin (HB) map can be calculated by comparison of skin optical density in the green (540 nm, where hemoglobin absorption is high) and red (about 650 nm, where hemoglobin absorption is low) spectral bands [3,4].The bilirubin map (BL) can be calculated by comparison of skin optical density in the blue (450 nm) and green (540 nm) spectral bands [7], melanin map (ML) can be calculated by comparison of skin optical density in the red (660 nm) and near infrared (940 nm) spectral bands [7], and the dermal melanin map (MD) can be calculated from the near infrared (940 nm) spectral image.Following equations describe the chromophore maps: , image. 1 , C are the normalizing coefficients that can be found by calibration of the device using white reference.Registered signal depends on the LED emission spectrum , L sensor matrix sensitivity , M and spectral optical density of the skin : S For highlighting the skin malformations, region of interest (RoI), comprising the malformation, is manually selected.Then map thresholding is applied; the algorithm calculates all image regions, where the concentration of chromophores exceeds the specific threshold value.Finally, the thresholded map is added to the skin image, taken under white illumination.Some of the clinically obtained skin chromophore maps are shown in Fig. 8.The hemoglobin and bilirubin maps reflect more the intradermal structures of skin while the melanin maps highlight the upper pigmented malformations.

Calculation of skin fluorophore maps
The comparison of skin autofluorescence intensity image and image of autofluorescence photobleaching rates, both extracted from the skin video at UVillumination, is illustrated at Fig. 9a.Skin autofluorescence photo-bleaching map (Fig. 9b) is calculated from the fluorescence intensity decrease at selected pixel groups by comparing the consecutive frames [9].Comparison of fluorescent intensities of the first and second halves of the set of frames is performed as , where is an image cube with intensities at the i-th pixel in the column, j-th pixel in the row and k-th video frame, N is the number of frames.

Calculation of skin microcirculation maps
A snapshot of a video frame of palm skin is shown in Fig. 10a.The processing algorithm is based on temporal analysis of the video near the heartbeat frequency (~ 1 Hz), and as a result visualization of blood microcirculation amplitude in each pixel of the skin image becomes possible.The map shows higher signal intensity values in the regions where skin perfusion is better (Fig. 10b).Skin perfusion map is calculated using Fourier analysis of the temporal component of the image cube in the following way:

Fig. 1 .
Fig. 1.The prototype device in its battery-charging holder (a), some internal design details (b), and functional scheme (c).

Fig. 2 .
Fig. 2. The measured emission bands of the used LEDs (a) and spectral sensitivity bands of the CMOS image sensor (b).

Fig. 4 .
Fig. 4. Interface of the device with hemoglobin map.
cube with intensities in i-th pixel in the column, j-th pixel in the row and k-th frame, C is a normalization coefficient, Fourier transform of the image cube in time domain.The map represents spatial distribution of skin pulse wave amplitudes and may characterize skin vascular malformations[10].

Fig. 10 .
Fig. 10.A microcirculation video-image frame (a) and the calculated skin perfusion map (b).
An operating prototype of a new concept device for skin diagnostics by means of combined multimodal imaging has been developed and clinically tested.It shows a promising potential for clinical use by dermatologists, cosmetologists, plastic and regenerative surgeons, forensic experts, and other professionals dealing with the assessment of in-vivo skin.