Imaging system for nanosatellite proximity operations

This paper presents a novel low-power imaging system for nanosatellite proximity operations. A robust independent camera module with on-board image processing, based on the ARM Cortex-M3 microcontroller and fast static random access memory, has been developed and characterized for the requirements of the ESTCube-1 mission. The imaging system, optimized for use in a single unit CubeSat, utilizes commercial off-the-shelf components and standard interfaces for a cost-effective reusable design. The resulting 43.3 mm × 22 mm × 44.2 mm (W×H×D) aluminium camera module weighs 30 g and consumes on the average of 118 mW of power, with peaks of 280 mW during image capture. Space qualification and stress tests have been performed. A detailed case study for the ESTCube-1 10 m tether deployment monitoring and Earth imaging mission is presented. For this purpose a 4.4 mm telecentric lens, 10 bit 640×480 pixel CMOS image sensor, 700 nm infrared cut-off filter and a 25% neutral density filter are used. The resolution of the assembled system is 12.7 mm and 1 km per pixel at distances of 10 m and 700 km, respectively. Custom on-board image evaluation and high dynamic range imaging algorithms for ESTCube-1 have been implemented and tested. Optical calibration of the assembled system has been performed.


INTRODUCTION
Recent years have seen a significant increase in the use of nanosatellites, as a platform for small-scale space missions.The emergence of the CubeSat standard [1] and the growth of the community around it has provided an accessible and cost-effective method of testing new technologies in space [2][3][4][5][6][7].
As more and more nanosatellites need to deploy systems outside of the satellite or interact with objects in close proximity to the satellite [8,21,22], there is an emerging demand for space qualified camera modules to monitor these operations.However, very little has been published on the subject of small camera modules, optimized for CubeSat proximity operations.
The cameras used in current nanosatellites range from very simple commercially available camera modules [11,13,15,27] to custom-built telescopes [8,18,25,26,28,29].However, most of the existing systems are not built as mission-independent modules with custom image processing capabilities and internal storage resources.As such, most of the cameras presented so far require both extensive interfacing and dedicated software (and resources) in the on-board computer.
In this work we present the design and characterization of an independent and robust camera module for use in nanosatellites.A general overview of the modular hardware design is given.Software used for camera control and image processing is outlined.The relative spectral responsivity of the optical system, power consumption and sensor noise are characterized.Results from end-mass imaging and stress tests are presented.
The monitoring of tether deployment and Earth imaging on-board ESTCube-1 CubeSat are presented as the main case study [30,31].However, the system is designed to be easily adaptable for use in various different satellite missions, both in terms of hardware and software.

REQUIREMENTS
One of the main design drivers for the system was the need for a robust space qualified stand-alone camera module design that could be modified or extended, based on the requirements of the specific mission.However, as a secondary, supporting payload of a nanosatellite, the dimensions, mass, and power consumption have to be minimized without compromising the performance.In the case of ESTCube-1, the requirements were set to 5% of the total volume and mass of a single unit CubeSat.According to the power budget of the satellite, the peak power consumption was limited to 350 mW [32].
The ESTCube-1 mission has been designed for a 700 km Sun-synchronous polar orbit.The primary objective of the mission is centrifugal deployment of a 10 m tether, which consists of 50 µm and 25 µm aluminium wires, with a spherical matte aluminium endmass of 12 mm diameter [30,33].The tether deployment process is monitored by the imaging system.The same system should enable Earth imaging for outreach and educational purposes as the secondary objective of the mission.Data downlink of ESTCube-1 is limited to 9.6 kbit/s.Using a single ground station in Tartu, Estonia, communication with the satellite can be established 7 times per 24 hours with an average duration of 10 minutes per contact.
Arising from the mission objectives and limitations, the additional mission specific requirements for the imaging system have been set as follows.
• Angular resolution of 275 arcseconds, enabling the detection of the end-mass during the entire tether deployment process.• Field of view at least 45 degrees to enable imaging of Estonia.• Radiometric sensitivity to enable detection of the endmass fully illuminated by the Sun; the brightness of the end-mass at maximum distance is 7.3 apparent stellar magnitudes.• On-board storage of and capability to download unprocessed sensor data.On-board image processing for tether deployment monitoring has to be capable of filtering out noise and evaluate the images for possible locations of the endmass.For Earth imaging, on-board processing should provide the capability to select and characterize images based on quality and content.High dynamic range imaging should be implemented.
In addition, the imaging system has to meet the qualification requirements from the launch service provider.

HARDWARE DESIGN
The resulting module, with the dimensions 43.3 × 22 × 44.2 mm (W × H × D) and mass of 30 g, uses aluminium and glass as its main structural materials.The image sensor is mounted on a separate printed circuit board with a 30 pin connector to the main module (Fig. 1), to allow for easy installation and swapping of the sensor.To further increase the modularity and aid with sybsystem level testing, the design features two on-board voltage regulators, which allow for a flexible input voltage range of 3.0 to 6.5 V.A separate voltage regulator enables the system to switch off the image sensor completely in order to conserve power and protect the sensor.Current is measured separately before each of the regulators, to provide more detailed houskeeping data and detect possible problems.
To keep power consumption to a minimum, low power components such as the 120 MHz STM32F217ZGT microcontroller unit (MCU) and 2 MB IS61WV102416BLL-10TLI external static randomaccess memory (SRAM) module are used [34,35].The microcontroller features a digital camera interface (DCMI), direct memory access (DMA), and a flexible static memory controller (FSMC), which in combination allow for image transfer at a pixel clock of 13 MHz directly to the external memory without a need for additional hardware buffers (Fig. 2).Using SRAM this way reduces power consumption; however, the storage capacity of the system is limited.This can be mitigated by using a fast interface to the on-board computer or additional external storage.On ESTCube-1 universal synchronous/asynchronous receiver/transmitter (USART) is used to connect all subsystems (at 19.universal serial bus (USB) interface was used due to its faster transfer rates of up to 12 Mbps [36].In addition to the SRAM, a small 16 kB external ferroelectric random access memory (FRAM) module is used to store critical data and settings for the camera.
Based on temperature readings from other satellites and thermal simulations, the components of the system have been chosen to operate within the -20 to +60 • C range, and the system was tested with a range of -35 to +75 • C. For reference the design features an internal temperature sensor in the MCU and an external temperature sensor behind the imaging sensor.Both sensors were calibrated to an accuracy of ± 1 • C.
For the ESTCube-1 mission, a 4.4 mm telecentric lens with a standard M12x0.5 thread, 9 mm aperture and a depth of field of 0.4 to ∞ m is used.In combination with the 640 × 480 Aptina MT9V011 image sensor [37], this gives a resolution of 262 arcseconds per pixel and a field of view of 46 × 35 degrees.
For Earth imaging purposes, a 700 nm infrared cutoff filter was included, because the Bayer filter on the image sensor is effectively transparent at near-infrared wavelengths.An additional 25% neutral density filter was used to mitigate the potential overexposure of the Earth.The aluminium optics housing was designed to enclose the MCU for a compact design and also for an additional layer of protection from radiation (Fig. 1).

SOFTWARE DESIGN
A general overview of the ESTCube-1 camera system firmware is given in Fig. 3. Custom image processing software is used in both of the ESTCube-1 mission objectives.Images directly from the sensor are stored with a bit depth of 16 bits resulting in an image size of 600 kB.During Earth imaging the data is scaled and compressed to create small 1.5 kB lossy thumbnails that can be downloaded in about 5 s.Progressive to lossless compression methods are used on the full images to reduce image size, while retaining full fidelity and allowing for partial image downloads.Image evaluation is performed to measure contrast, hue and histogram distribution; these metrics can be used to discard images during the capture process as well as to aid the satellite operator in image selection for download.On-board algorithms are used to combine multiple 10 bit images at different exposures into a single 16 bit high dynamic range image.
For end-mass and tether imaging, basic isolated bright area detection is performed and the coordinates of possible end-mass locations are stored for retrieval.Either the average or median of multiple images is calculated to mitigate the effects of random noise during the detection process.Lossless compression is used to reduce the on-board storage requirements.Flexible bit-depth (between 1 to 10 bit) full-frame video Fig. 3. General overview of ESTCube-1 on-board camera firmware, where darker blocks indicate software shared with the on-board computer system [38].
capture is used during the early phases of the tether deployment, to record any possible problems.Due to the current hardware and software configuration, video capture frame rates are limited by available SRAM and transfer speeds to the command and data handling system (CDHS).
Since the system features a capable processor, which is not in use for long periods of time, the camera subsystem can be used to support data processing for other subsystems.On ESTCube-1, the camera system also provides lossless data compression capabilities for telemetry data, collected by the CDHS [38].
To control the imaging, configuration data, and initial set-up commands are sent to the camera over the communication interface.After this, the camera waits for either a trigger command or an interrupt on a dedicated hardware input.Giving the camera time to stabilize prior to use and triggering with the hardware interrupt enables an external system to have precise control over capture timing.
Software updates can be performed in-orbit using a custom bootloader.The bootloader features 3 independent slots for firmware images, with one slot designated as a fallback in case a newer version of the firmware fails.Commands for the bootloader are stored in external FRAM.As a contingency system, it is possible to load the built-in bootloader of the STM32 MCU by holding the hardware trigger line in an active state during startup.The built-in bootloader allows full access to the internal flash memory.
To detect and recover from failures, the software features custom exception handling and an independent watchdog timer.A software controlled heartbeat signal is sent to the electrical power subsystem (EPS) [32] to indicate that the main task scheduling of the FreeRTOS operating system is working.As an additional check, the EPS sends out periodic ping commands to all subsystems to verify the integrity of the communication link.

Relative spectral responsivity
The relative spectral responsivity of the fully assembled imaging system was measured from 400 to 900nm with 1 nm step using a calibrated SDL1 monochromator with a 1000 W FEL lamp.At each measured wavelength, the values of each Bayer filter array channel were averaged over an area of 100 pixels of uniform irradiance.The relative spectral responsivity curves are shown in Fig. 4.

Power consumption
Power consumption has been characterized by measuring voltage drop across a shunt resistor using an oscilloscope with an average expanded uncertainty of 15 mW (coverage factor k = 2).The results (Fig. 5) show an average power consumption of 115 mW when idle with all peripherals switched on (stage 5) and a peak power consumption of 280 mW during image capture (stage 7).

Thermal noise characteristics of the CMOS sensor
Image sensor thermal noise tests have been performed in a temperature chamber in the range of -30 to +70 • C. The sensor was covered to block all light.Ten images, with an exposure of 1 s, were taken at 10 ± 1 • C intervals after a stabilization period of 15 min.A time delay of 15 s was used between capturing each image, to avoid the effects of residual signals.The results of these tests are summarized in Fig. 6.The results measured at temperatures below -10 • C are not shown in the figure, since the thermal noise signal was not distinguishable from the read-out noise around the bias level of 42 analogue to digital units (ADU).The dark frames from these tests can be used during image processing to remove bias level and hot pixels.

Tether end-mass imaging
Tether end-mass imaging tests have been done in laboratory conditions by suspending the aluminium endmass from a piece of tether at various distances from the  camera.The tether and end-mass were lit from the side by a light source that provided Sun-equivalent irradiance.
The results (Fig. 7) show that the camera is able to clearly see not only the end-mass, but also the tether.test the custom bootloader, thousands of firmware images were uploaded to the camera.No problems were detected during the tests.Vibration tests at a maximum of 22.5 g and shock tests at a maximum of 1410 g have been performed as part of the system level qualification tests [30].The camera subsystem passed the tests without any problems.

DISCUSSION AND CONCLUSIONS
A small and robust imaging system for CubeSat proximity operations has been developed, tested, and characterized for use in the ESTCube-1 mission.
The system uses low power COTS components for a cost-effective and reusable design.By including a 120 MHz processor, it is possible for the camera module to do much of its own image processing and compression.In addition it can provide data processing and compression capabilities to external systems, usually the camera itself is not in constant use.Several software features have been included to allow for error handling and firmware upgrades in orbit.This enables in-orbit testing of new algorithms in the future.
The imaging system fully meets the power requirements of ESTCube-1.It would be possible to further lower the power consumption of the system, various levels of standby states can be used, where different peripherals are switched off, to the point where the MCU and external memory are only powered to retain data while the CDHS is not ready to receive it.
The system passed all tests and fulfils all mission requirements for ESTCube-1.As the final test on the Earth, an image of ESTCube-1 satellite was taken in the clean room of Guiana Space Centre in Kourou using a mirror (Fig. 8).
ESTCube-1 was launched on-board the Vega launcher VV02 flight on 7 May 2013.While this article covers the design and pre-launch characterization of the imaging system, the following in-orbit validation and characterization tests are planned for the future.While the main mission of the camera module on ESTCube-1 is to monitor the tether deployment and capture images of Earth, the imaging system might be used to calibrate the attitude determination and control system (ADCS) of ESTCube-1 by taking images of the Sun and Earth horizon with precision timing [39].With the ability to easily replace the optics, image sensor, and on-board image processing algorithms, the camera can be used in a wide range of low Earth orbit applications in the future, such as proximity monitoring, horizon tracking, remote sensing, and star tracking.

Fig. 1 .
Fig. 1.An exploded view of the final camera module design, where SRAM stands for static random-access memory, ND for neutral density, MCU for microcontroller unit, and IR for infrared.

Fig. 4 .
Fig. 4. Relative spectral response of the assembled optical system.

Fig. 5 .
Fig. 5. Power consumption of the ESTCube-1 camera module in different operating stages: (1) reset; (2) processor initialization; (3) peripheral initialization; (4) image sensor initialization; (5) idle; (6) preparation for imaging; (7) image capture; (8) standby.The periodic peaks during stages (3) and (4) are caused by the analogue to digital converters measuring temperatures and current consumption.Stages of camera operation have been delayed in time in order to make the transitions more distinguishable on the graph.

Fig. 6 .
Fig. 6.Image sensor average dark pixel value (solid line) and maximum pixel value (dashed line) during temperature tests.Dark frames taken at 1 s exposure.Results presented in analogue to digital units (ADU).

•
Point light source imaging to test end-mass detection.• Video capture test with Earth to confirm functionality before starting tether deployment.• Flat and dark frame capture to analyse errors.• Evaluation of long term performance of the system in space environment.• Resolution test in orbit.• Software algorithm tests.

Fig. 8 .
Fig.8.A photo of the satellite in the mirror, made by the camera onboard the satellite at the Guiana Space Centre in Kourou.The satellite is connected to the access port device, which is used to communicate with the satellite before launch.