AP0101CS AP0101CS High-Dynamic Range (HDR) Image Signal Processor (ISP) General Description www.onsemi.com The ON Semiconductor AP0101CS is a high-performance, ultra-low power in-line, digital image processor optimized for use with High Dynamic Range (HDR) sensors. The AP0101CS provides full auto-functions support (AWB and AE) and Adaptive Local Tone Mapping (ALTM) to enhance HDR images and advanced noise reduction which enables excellent low-light performance. Table 1. KEY PERFORMANCE PARAMETERS Parameter VFBGA81, 6.5x6.5 CASE 138AG Value Primary camera interface Parallel Primary camera input format RAW12 Linear/Companded Bayer data Output interface Up to 20-bit Parallel (Note 1) Output format YUV422 8-bit, 10-bit, and SMPTE296M 10-, 12-bit tone-mapped Bayer Maximum resolution 1280 x 960 (1.2 Mp) Input clock range (Note 2) 6-30 MHz Maximum frame rate (Note 3) 45 fps at 1.2 Mp, 60 fps at 720 p Maximum output clock frequency Parallel clock up to 84 MHz Supply voltage VDDIO_S 1.8 or 2.8 V nominal VDDIO_H 2.5 or 3.3 V nominal VDD_REG 1.8 V nominal VDDIO_OTPM 2.5 or 3.3 V nominal XXXXXXXXXXX Operating temperature (ambient - TA) -30C to + 70C Typical power consumption (Note 4) 130 mW 1. 20-bit in one pixel clock format is only available in SMPTE mode with the use of 4 GPIOs. 2. With input clock below 10 MHz, the two wire serial interface is supported only up to 100 KHz. 3. Maximum frame rate depends on output interface and data format configuration used. 4. 720 p HDR 60 fps 74.25 MHz YCbCr_422_16 Features * Supports ON Semiconductor sensors with up to * * * * 1.2 Mp (1280 x 960) 45 fps at 1.2 Mp, 60 fps at 720 p Optimized for operation with HDR sensors Color and gamma correction Auto exposure, auto white balance, 50/60 Hz flicker avoidance (c) Semiconductor Components Industries, LLC, 2016 October, 2018 - Rev. 8 MARKING DIAGRAM BALL A1 ID = Laser Marking ORDERING INFORMATION See detailed ordering and shipping information on page 2 of this data sheet. * * * * * * * * * * Adaptive Local Tone Mapping (ALTM) Test Pattern Generator Two-wire serial programming interface Interface to low-cost Flash or EPROM through SPI bus (to configure and load patches) High-level host command interface Standalone operation supported Up to 5 GPIO Fail-safe IO Multi-Camera synchronization support Dual Band IR filter support Applications * SMPTE296 HDCCTV cameras * Surveillance network IP cameras 1 Publication Order Number: AP0101CS/D AP0101CS ORDERING INFORMATION Table 2. AVAILABLE PART NUMBERS Part Number Product Description AP0101CS2L00SPGA0-DR1 1Mp Co-Processor, 100-ball VFBGA AP0101CS2L00SPGAD3-GEVK AP0101CS Demo Kit AP0101CS2L00SPGAH-GEVB AP0101CS Head Board Orderable Product Attribute Description Drypack 5. See the ON Semiconductor Device Nomenclature document (TND310/D) for a full description of the naming convention used for image sensors. For reference documentation, including information on evaluation kits, please visit our web site at www.onsemi.com. FUNCTIONAL OVERVIEW Figure 1 shows the typical configuration of the AP0101CS in a camera system. On the host side, a two-wire serial interface is used to control the operation of the AP0101CS, and image data is transferred using the parallel bus between the AP0101CS and the host. The AP0101CS interface to the sensor also uses a parallel interface. 1.2Mp HDR Sensor 12-bit parallel Two-wire serial I/F (Master) Up to 20-bit parallel Host Two-wire serial IF (Slave) Figure 1. AP0101CS Connectivity SYSTEM INTERFACES Figure 2 shows typical AP0101CS device connections. All power supply rails must be decoupled from ground using capacitors as close as possible to the package. The AP0101CS signals to the sensor and host interfaces can be at different supply voltage levels to optimize power consumption and maximize flexibility. Table 4 on page 4 provides the signal descriptions for the AP0101CS. www.onsemi.com 2 AP0101CS RPULL-UP (Note 2) VDDIO_S 1.8 V (Regulator IP) 1.2 V (Regulator OP) Power up Core and PLL VDD_REG FB_SENSE LDO_OP OTPM Power VDD _PLL VDD M_S CLK M_S DATA (Note 3) EXTCLK_OUT V DD IO_OTPM Sensor IO power Host IO Power VDDIO_H (Note 2) SCLK RESET_BAR SDATA SADDR FRAME_SYNC RESET_BAR_OUT EXTCLK FV_IN LV_IN PIXCLK_IN RPULL-UP Oscillator XTAL STANDBY FV_OUT LV_OUT PIXCLK_OUT DOUT [15:0] DIN [11:0] TRIGGER_OUT SPI_CS_BAR SPI_CLK SPI_SDO SPI_SDI GPIO_1 GPIO_2 GPIO_3 GPIO_4 GPIO_5 GND_REG VDDIO_S (Note 6) GND VDD_REG (Note 4) LDO_OP (Note 4) VDDIO_OTPM TRST_BAR (Note 5) VDDIO_H (Note 6) NOTES: 1. This typical configuration shows only one scenario out of multiple possible variations for this sensor. 2. ON Semiconductor recommends a 1.5 kW resistor value for the two-wire serial interface RPULL-UP; however, greater values may be used for slower transmission speed. 3. RESET_BAR has an internal pull-up resistor and can be left floating if not used. 4. The decoupling capacitors for the regulator input and output should have a value of 1.0 mF. The capacitors should be ceramic and need to have X5R or X7R dielectric. 5. TRST_BAR connects to GND for normal operation. 6. ON Semiconductor recommends that 0.1 mF and 1 mF decoupling capacitors for each power supply are mounted as close as possible to the pin. Actual values and numbers may vary depending on layout and design consideration. Figure 7. Typical Configuration Crystal Usage The following table summarizes the key signals when using the internal regulator. (The internal regulator has to be used for AP0101AT.) As an alternative to using an external oscillator, a crystal may be connected between EXTCLK and XTAL. Two small loading capacitors and a feedback resistor should be added, as shown in Figure 3. Table 3. KEY SIGNALS WHEN USING THE REGULATOR Signal Name Internal Regulator VDD_REG 1.8 V FB_SENSE 1.2 V (input) LDO_OP 1.2 V (output) www.onsemi.com 3 AP0101CS AP0101 C1 EXTCLK Rf=1 MW XTAL C2 NOTE: Rf represents the feedback resistor, an Rf value of 1 MW would be sufficient for AP0101CS. C1 and C2 are decided according to the crystal or resonator CL specification. In the steady state of oscillation, CL is defined as (C1 x C2)/(C1+C2). In fact, the I/O ports, the bond pad, package pin and PCB traces all contribute the parasitic capacitance to C1 and C2. Therefore, CL can be rewritten to be (C1* x C2*)/(C1*+C2*), where C1*=(C1+Cin, stray) and C2*=(C2+Cout, stray). The stray capacitance for the IO ports, bond pad and package pin are known which means the formulas can be rewritten as C1*=(C1+1.5pF+Cin, PCB) and C2*=(C2+1.3pF+Cout, PCB). Figure 8. Using a Crystal Instead of External Oscillator PIN DESCRIPTIONS Table 4. PIN DESCRIPTIONS Name Type Description EXTCLK Input Master input clock, nominally 27 MHz. This can either be a square-wave generated from an oscillator (in which case the XTAL input must be left unconnected) or direct connection to a crystal. XTAL Output RESET_BAR Input/PU SCLK Input Two-wire serial interface clock (host interface). SDATA Input/Output Two-wire serial interface data (host interface). SADDR Input Selects device address for the two-wire slave serial interface. When connected to GND, the device ID is 0x90. When wired to VDDIO_H, a device ID of 0xBA is selected. FRAME_SYNC Input This input can be used to set the output timing of the AP0101CS. This signal should be connected to GND if not used. STANDBY Input Standby mode control, active HIGH. SPI_SCLK Output SPI_SDI Input SPI_SDO Output Data out to SPI flash or EEPROM memory SPI_CS_BAR Output Chip select out to SPI flash or EEPROM memory. FV_OUT Output Host frame valid output (synchronous to PIXCLK_OUT). LV_OUT Output Host line valid output (synchronous to PIXCLK_OUT). PIXCLK_OUT Output Host pixel clock output. DOUT [15:0] Output Host pixel data output (synchronous to PIXCLK_OUT) DOUT[15:0]. Note 20-bit output (SMPTE) also uses GPIO[5:2]. GPIO [5:1] I/O If EXTCLK is connected to one pin of a crystal, this signal is connected to the other pin, otherwise this signal must be left unconnected. Master reset signal, active LOW. This signal has an internal pull up. Clock output for interfacing to an external SPI flash or EEPROM memory Data in from SPI flash or EEPROM memory. When no SPI device is fitted, this signal is used to determine whether the AP0101CS should auto-configure: 0: Do not auto-configure; two-wire interface will be used to configure the device (host-config mode) 1: Auto-configure. This signal has an internal pull-up resistor General purpose digital I/O. Note: 20-bit output (SMPTE) also uses GPIO[5:2] www.onsemi.com 4 AP0101CS Table 4. PIN DESCRIPTIONS (continued) Name Type Description TRST_BAR Input EXT_CLK_BAR Output Clock to external sensor. RESET_BAR_OUT Output Reset signal to external sensor. M_SCLK Output Two-wire serial interface interface clock (Master). M_SDATA I/O Two-wire serial interface interface clock (Master). FV_IN Input Sensor frame valid input. LV_IN Input Sensor line valid input. PIXCLK_IN Input Sensor pixel clock output. DIN [11:0] Input Sensor pixel data input DIN [11:0]. TRIGGER_OUT Output Trigger signal for external sensor. VDDIO_S Supply Sensor I/O power supply. Must be tied to GND in normal operation. GND Supply Ground for sensor IO, host IO, PLL, VDDIO_OTPM, and VDD. VDD_REG Supply Input to on-chip 1.8 V to 1.2 V regulator. LDO_OP Output Output from on-chip 1.8 V to 1.2 V regulator. Note: The regulator on the AP0101CS must be used. FB_SENSE Input GND_REG Supply Ground for on-chip regulator. VDD_PLL Supply PLL supply. VDD Supply Core supply. VDDIO_OTPM Supply OTPM power supply. VDDIO_H Supply Host I/O power Supply. On-chip regulator sense signal. Table 5. PACKAGE PINOUT 1 2 3 4 5 6 7 8 9 A EXTCLK XTAL SCLK SPI_SDO DOUT[15] DOUT[13] DOUT[10] DOUT[9] DOUT[8] B VDD VDDIO_H SDATA SPI_SDI DOUT[14] DOUT[12] DOUT[11] DOUT[7] DOUT[6] C EXT_CLK _OUT VDDIO_S SADDR SPI_CS_BAR GND PIXCLK _OUT FV_OUT DOUT[5] DOUT[4] D RESET_BAR _OUT VDD GND SPI_SCLK GND TRST_BAR LV_OUT DOUT[3] DOUT[2] E DIN[3] DIN[7] GND FB_SENSE GND GND VDD_PLL DOUT[1] DOUT[0] F DIN[11] DIN[2] LDO_OP GND_REG GND GND VDD_PLL VDD_PLL VDDIO _OTPM G DIN[6] DIN[1] DIN[4] VDD_REG VDDIO_S VDD RESET_BAR GPIO[4] GPIO[5] H DIN[10] DIN[0] DIN[8] FV_IN M_SDATA VDDIO_H FRAME _SYNC GPIO[2] GPIO[3] J DIN[5] DIN[9] PIXCLK_IN LV_IN M_SCLK VDD STANDBY TRIGGER _OUT GPIO[1] www.onsemi.com 5 AP0101CS Power-Up and Down Sequence timing requirements are shown in Table 6. The AP0101CS includes a power-on reset feature that initiates a reset upon power up of the AP0101CS. Powering up and down the AP0101CS requires voltages to be applied in a particular order, as seen in Figure 4. The dv/dt VDDIO_H dv/dt t7 t1 dv/dt VDDIO_S, VDDIO_OTPM t6 t2 VDD_REG t5 t3 EXTCLK SCLK t4 SDATA RESET Figure 9. Power-Up and Power-Down Sequence Table 6. POWER-UP AND POWER-DOWN SIGNAL TIMING Symbol Parameter Min Typ Max Unit t1 Delay from VDDIO_H to VDDIO_S, VDDIO_OTPM 0 - 50 ms t2 Delay from VDDIO_H to VDD_REG 0 - 50 ms t3 EXTCLK activation t2+1 - - ms t4 First serial command 100 - - EXTCLK cycles t5 EXTCLK cutoff t6 - - ms t6 Delay from VDD_REG to VDDIO_H 0 - 50 ms t7 Delay from VDDIO_S, VDDIO_OTPM to VDDIO_H 0 - 50 ms Power supply ramp time (slew rate) - - 0.1 V/ms dv/dt 6. If the system cannot support this power supply slew rate, then power supplies must be designed to overcome inrush currents in Table 25, "Inrush Current," on page 26. * An internal power-on reset. Reset The AP0101CS has 3 types of reset available: * A hard reset is issued by toggling the RESET_BAR signal. * A soft reset is issued by writing commands through the two-wire serial interface. Table 7 shows the output states when the part is in various states. Table 7. OUTPUT STATES Hardware States Default State Hard Standby Soft Standby Streaming Idle EXTCLK (clock running or stopped) (clock running) (clock running or stopped) (clock running) (clock running) (clock running) Input XTAL n/a n/a n/a n/a n/a n/a Input RESET_BAR (asserted) (negated) (negated) (negated) (negated) (negated) Input Name Reset State Firmware States www.onsemi.com 6 Notes AP0101CS Table 7. OUTPUT STATES (continued) Hardware States Name Reset State Default State Firmware States Hard Standby Soft Standby Streaming Idle Notes SCLK n/a n/a (clock running or stopped) (clock running or stopped) (clock running or stopped) (clock running or stopped) Input. Must always be driven to a valid logical level. SDATA High- impedance High-impedance High- impedance High- impedance High- impedance High- impedance Input/Output. A valid logic level should be established by pull-up. SADDR n/a n/a n/a n/a n/a n/a Input. Must always be driven to a valid logical level. FRAME_SYNC n/a n/a n/a n/a n/a n/a Input. Must always be driven to a valid logical level. STANDBY n/a (negated) (negated) (negated) (negated) (negated) Input. Must always be driven to a valid logical level. SPI_SCLK High- impedance driven, logic 0 driven, logic 0 driven, logic 0 Output SPI_SDI Internal pull- up enabled Internal pull- up enabled Internal pull- up enabled Internal pull- up enabled Input. Internal pull-up permanently enabled. SPI_SDO High- impedance driven, logic 0 driven, logic 0 driven, logic 0 Output SPI_CS_BAR High- impedance driven, logic 1 driven, logic 1 driven, logic 1 Output EXT_CLK _OUT driven, logic 0 driven, logic 0 driven, logic 0 driven, logic 0 Output RESET_BAR _OUT driven, logic 0 driven, logic 0 driven, logic 1 driven, logic 1 Output. Firmware will release sensor reset. M_SCLK High- impedance High- impedance High- impedance High- impedance Input/Output. A valid logic level should be established by pull-up. M_SDATA High- impedance High- impedance High- impedance High- impedance Input/Output. A valid logic level should be established by pull-up. FV_IN, LV_IN, PIXCLK_IN, DIN [11:0] n/a n/a n/a n/a FV_OUT, LV_OUT, PIXCLK_OUT, DOUT [15:0] High- impedance Varied Driven if used Driven if used TX_CLK, RX_CLK, GTX_CLK High- impedance Varied Driven if used GPIO[5:2] High- impedance Input, then high- impedance GPIO1 High- impedance TRIGGER _OUT TRST_BAR n/a Input. Must always be driven to a valid logical level. Driven if used Driven if used Output. Default state dependent on configuration. Driven if used Driven if used Driven if used Output. Default state dependent on configuration Driven if used Driven if used Driven if used Driven if used Input/Output. After reset these pins are sampled as inputs as part of auto-configuration. High- impedance High- impedance High- impedance High- impedance High- impedance High- impedance High- impedance Driven if used Driven if used Driven if used Driven if used n/a n/a (negated) (negated) (negated) (negated) www.onsemi.com 7 Input. Must always be driven to a valid logic level. AP0101CS Hard Reset The AP0101CS enters the reset state when the external RESET_BAR signal is asserted LOW, as shown in Figure 5. All the output signals will be in High-Z state. t4 t1 t2 t3 EXTCLK RESET_BAR SDATA All Outputs Data Active Data Active Mode Reset Enter streaming mode Internal Initialization Time Figure 10. Hard Reset Operation Table 8. HARD RESET Symbol Parameter Min Typ Max Unit EXTCLK cycles t1 RESET_BAR pulse width 50 - - t2 Active EXTCLK required after RESET_BAR asserted 10 - - t3 Active EXTCLK required before RESET_BAR de-asserted 10 - - t4 First two-wire serial interface communication after RESET_BAR is HIGH 100 - - Entering Standby Mode 1. Assert STANDBY signal HIGH. Soft Reset A soft reset sequence to the AP0101CS can be activated by writing to a register through the two-wire serial interface. Exiting Standby Mode 1. De-assert STANDBY signal LOW. Hard Standby Mode The AP0101CS can enter hard standby mode by using the external STANDBY signal, as shown in Figure 6. t1 t2 STANDBY Asserted STANDBY Mode t3 EXTCLK STANDBY Mode EXTCLK Disabled Figure 11. Hard Standby Operation www.onsemi.com 8 EXTCLK Enabled AP0101CS Table 9. HARD STANDBY SIGNAL TIMING Symbol Min Typ Max Unit t1 Standby entry complete Parameter - - 2 Frames Lines t2 Active EXTCLK required after going into STANDBY mode 10 - - EXTCLKs t3 Active EXTCLK required before STANDBY de-asserted 10 - - EXTCLKs MULTI-CAMERA SYNCHRONIZATION SUPPORT The AP0101CS supports multi-camera synchronization via the FRAME_SYNC pin. The host (or controlling entity) 'broadcasts' a sync-pulse to all cameras within the system that triggers streaming start. The AP0101CS will propagate the signal to the TRIGGER_OUT pin to the sensor's TRIGGER pin. The AP0101CSsupports two different trigger modes. The first mode supported is 'single-shot'; this is when the trigger pulse will cause one frame to be output from the image sensor and AP0101CS (see Figure 7). t_FRAMESYNC FRAME_SYNC t_TRIGGER_ PROP TRIGGER_OUT FV_OUT T_FRAMESYNC_FVH Figure 12. Single-Shot Mode Table 10. TRIGGER TIMING Parameter Name FRAME_SYNC to FV_OUT Conditions Min Typ Max Unit tFRMSYNC_FVH 8 lines + exposure time + sensor delay - - Lines FRAME_SYNC to TRIGGER_OUT tTRIGGER_PROP - - 9 ns tFRAME_SYNC tFRAME_SYNC 3 - - EXTCLK cycles The second mode supported is called `continuous'; this is when a trigger pulse will cause the part to continuously output frames, see Figure 8. This mode would be especially useful for applications which have multiple sensors and need to have their video streams synchronized (for example, surround view or panoramic view applications). www.onsemi.com 9 AP0101CS FRAME_SYNC TRIGGER_OUT FV_OUT NOTE: This diagram is not to scale. Figure 13. Continuous Mode microcontroller will automatically adjust the operating parameters. For normal operation of the AP0101CS, a stream of raw image data from the attached image sensor is fed into the color pipeline. The user also has the option to select a number of test patterns to be input instead of sensor data. The test pattern is fed to the IFP for testing the image pipeline without sensor operation. The test patterns can be selected by programming variables. To select enter test pattern mode, set R0xC88F to 0x02 and issue a Change- Config request; to exit this mode, set R0xC88F to 0x00. When two or more cameras have a signal applied to the FRAME_SYNC input at the same time, the respective FV_OUT signals would be synchronized within 5 PIXCLK_OUT cycles. This assumes that all cameras have the same configuration settings and that the exposure time is the same. IMAGE FLOW PROCESSOR Image and color processing in the AP0101CS is implemented as an image flow processor (IFP) coded in hardware logic. During normal operation, the embedded RAW 12- or 20-bit Bayer AE, FD and ALTM stats 12-bit ALTM Bayer linear or companded data RX decom panding Progressive Test pattern generator Defect correction Noise correction Black level substraction Digital gain control, PGA ALTM Color interpolation Aper- ture Cor- rec- tion Color Cor- rec- tion Crop Gamma RGB- 2YUV Color Kill AW B stats RAW Bayer ALTM Bayer RGB YCbCr Figure 14. Continuous Mode www.onsemi.com 10 YUV filters Scaler Progressive parallel or SMPTE (YcbCr or Bayer) AP0101CS Test Patterns Table 11. TRIGGER TIMING Test Pattern Example FLAT FIELD FIELD_WR= CAM_MODE_SELECT, 0x02 FIELD_WR= CAM_MODE_TEST_PATTERN_SELECT, 0x01 FIELD_WR= CAM_MODE_TEST_PATTERN_RED, 0x000FFFFF FIELD_WR= CAM_MODE_TEST_PATTERN_GREEN, 0x000FFFFF FIELD_WR= CAM_MODE_TEST_PATTERN_BLUE, 0x000FFFFF Load = Change-Config Changing the values in R0xC890-R0xC898 will change the color of the test pattern. 100% Color Bar FIELD_WR= CAM_MODE_SELECT, 0x02 FIELD_WR= CAM_MODE_TEST_PATTERN_SELECT, 0x02 Load = Change-Config Pseudo-Random FIELD_WR= CAM_MODE_SELECT, 0x02 FIELD_WR= CAM_MODE_TEST_PATTERN_SELECT, 0x05 Load = Change-Config Fade-to-Gray FIELD_WR= CAM_MODE_SELECT, 0x02 FIELD_WR= CAM_MODE_TEST_PATTERN_SELECT, 0x08 Load = Change-Config Linear Ramp FIELD_WR= CAM_MODE_SELECT, 0x02 FIELD_WR= CAM_MODE_TEST_PATTERN_SELECT, 0x09 Load = Change-Config Defect Correction AdaCD (Adaptive Color Difference) After data decompanding the image stream processing starts with defect correction. To obtain defect free images, the pixels marked defective during sensor readout and the pixels determined defective by the defect correction algorithms are replaced with values derived from the non-defective neighboring pixels. This image processing technique is called defect correction. Automotive applications require good performance in extremely low light, even at high temperature conditions. In these stringent conditions the image sensor is prone to higher noise levels, and so efficient noise reduction techniques are required to circumvent this sensor limitation and deliver a high quality image to the user. www.onsemi.com 11 AP0101CS While global tone mapping methods provide computationally simple and easy to use solutions, they often cause loss of contrast and detail. A local tone mapping is thus necessary in addition to global tone mapping for the reproduction of visually more appealing images that also reveal scene details that are important for automotive safety and surveillance applications. Local tone mapping methods use a spatially varying mapping function determined by the neighborhood of a pixel, which allows it to increase the local contrast and the visibility of some details of the image. Local methods usually yield more pleasing results because they exploit the fact that human vision is more sensitive to local contrast. ON Semiconductor's ALTM solution significantly improves the performance over global tone mapping. ALTM is directly applied to the Bayer domain to compress the dynamic range from 20-bit to 12-bit. This allows the regular color pipeline to be used for HDR image rendering. The AdaCD Noise Reduction Filter is able to adapt its noise filtering process to local image structure and noise level, removing most objectionable color noise while preserving edge details. Black Level Subtraction and Digital Gain After noise reduction, the pixel data goes through black level subtraction and multiplication of all pixel values by a programmable digital gain. Independent color channel digital gain can be adjusted with registers. Black level subtract (to compensate for sensor data pedestal) is a single value applied to all color channels. If the black level subtraction produces a negative result for a particular pixel, the value of this pixel is set to 0. Positional Gain Adjustments (PGA) Lenses tend to produce images whose brightness is significantly attenuated near the edges. There are also other factors causing fixed pattern signal gradients in images captured by image sensors. The cumulative result of all these factors is known as image shading. The AP0101CS has an embedded shading correction module that can be programmed to counter the shading effects on each individual R, Gb, Gr, and B color signal. Color Interpolation In the raw data stream fed by the sensor core to the IFP, each pixel is represented by a 20- or 12-bit integer number, which can be considered proportional to the pixel's response to a one-color light stimulus, red, green, or blue, depending on the pixel's position under the color filter array. Initial data processing steps, up to and including ALTM, preserve the one-color-per-pixel nature of the data stream, but after ALTM it must be converted to a three-colors-per-pixel stream appropriate for standard color processing. The conversion is done by an edge-sensitive color interpolation module. The module pads the incomplete color information available for each pixel with information extracted from an appropriate set of neighboring pixels. The algorithm used to select this set and extract the information seeks the best compromise between preserving edges and filtering out high frequency noise in flat field areas. The edge threshold can be set through register settings. The correction functions The correction functions can then be applied to each pixel value to equalize the response across the image as follows: P corrected(row, col) + P sensor(row, col) f(row, col) (eq. 1) where P are the pixel values and f is the color dependent correction functions for each color channel. Adaptive Local Tone Mapping (ALTM) Real world scenes often have very high dynamic range (HDR) that far exceeds the electrical dynamic range of the imager. Dynamic range is defined as the luminance ratio between the brightest and the darkest object in a scene. In recent years many technologies have been developed to capture the full dynamic range of real world scenes. For example, the multiple exposure method is a widely adopted method for capturing high dynamic range images, which combines a series of low dynamic range images of the same scene taken under different exposure times into a single HDR image. Even though the new digital imaging technology enables the capture of the full dynamic range, low dynamic range display devices are the limiting factor. Today's typical LCD monitor has contrast ratio around 1,000:1; however, it is not atypical for an HDR image having contrast ratio around 250,000:1. Therefore, in order to reproduce HDR images on a low dynamic range display device, the captured high dynamic range must be compressed to the available range of the display device. This is commonly called tone mapping. Tone mapping methods can be classified into global tone mapping and local tone mapping. Global tone mapping methods apply the same mapping function to all pixels. Color correction and aperture correction To achieve good color fidelity of the IFP output, interpolated RGB values of all pixels are subjected to color correction. The IFP multiplies each vector of three pixel colors by a 3 x 3 color correction matrix. The three components of the resulting color vector are all sums of three 10-bit numbers. The color correction matrix can be either programmed by the user or automatically selected by the auto white balance (AWB) algorithm implemented in the IFP. Color correction should ideally produce output colors that are corrected for the spectral sensitivity and color crosstalk characteristics of the image sensor. The optimal values of the color correction matrix elements depend on those sensor characteristics and on the spectrum of light incident on the sensor. The color correction variables can be adjusted through register settings. To increase image sharpness, a programmable 2D aperture correction (sharpening filter) is applied to www.onsemi.com 12 AP0101CS color-corrected image data. The gain and threshold for 2D correction can be defined through register settings. values of those pixels are attenuated proportionally to the difference between their luminance and the threshold. Gamma Correction YUV Color Filter The gamma correction curve is implemented as a piecewise linear function with 33 knee points, taking 12-bit arguments and mapping them to 10-bit output. The abscissas of the knee points are fixed at 0, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, 448, 512, 640, 768, 896, 1024, 1280, 1536, 1792, 2048, 2560, 3072, 3584, and 4096. The 10-bit ordinates are programmable through variables. The AP0101CS has the ability to calculate the 33-point knee points based on the tuning of cam_ll_gamma and cam_ll_contrast_gradient_bright. The other method is for the host to program the 33 knee point curve themselves. Also included in this block is a Fade-to Black curve which sets all knee points to zero and causes the image to go black in extreme low light conditions. As an optional processing step, noise suppression by one-dimensional low-pass filtering of Y and/or UV signals is possible. A 3- or 5-tap filter can be selected for each signal. CAMERA CONTROL AND AUTO FUNCTIONS Auto Exposure The auto exposure algorithm optimizes scene exposure to minimize clipping and saturation in critical areas of the image. This is achieved by controlling exposure time and analog gains of the sensor core as well as digital gains applied to the image. The auto exposure module analyzes image statistics collected by the exposure measurement engine, makes a decision, and programs the sensor and color pipeline to achieve the desired exposure. The measurement engine subdivides the image into 25 windows organized as a 5 x 5 grid. Color Kill To remove high-or low-light color artifacts, a color kill circuit is included. It affects only pixels whose luminance exceeds a certain preprogrammed threshold. The U and V Figure 15. 5 x 5 Grid AE TRACK DRIVER Other algorithm features include the rejection of fast fluctuations in illumination (time averaging), control of speed of response, and control of the sensitivity to small changes. While the default settings are adequate in most situations, the user can program target brightness, measurement window, and other parameters described above. The driver changes AE parameters (integration time, gains, and so on) to drive scene brightness to the programmable target. To avoid unwanted reaction of AE on small fluctuations of scene brightness or momentary scene changes, the AE track driver uses a temporal filter for luma and a threshold around the AE luma target. The driver changes AE parameters only if the difference between the AE luma target and the filtered luma is larger than the AE target step and pushes the luma beyond the threshold. AUTO WHITE BALANCE The AP0101CS has a built-in AWB algorithm designed to compensate for the effects of changing spectra of the scene illumination on the quality of the color rendition. The algorithm consists of two major parts: a measurement engine performing statistical analysis of the image and a driver performing the selection of the optimal color correction matrix and IFP digital gain. While default settings of these algorithms are adequate in most situations, www.onsemi.com 13 AP0101CS calculate the exposure and gains, the host then copies this to the slaves, and all changes are then applied at the same time. the user can reprogram base color correction matrices, place limits on color channel gains, and control the speed of both matrix and gain adjustments. The AP0101CS AWB displays the current AWB position in color temperature, the range of which will be defined when programming the CCM matrixes. The region of interest can be controlled through the combination of an inclusion window and an exclusion window. Manual Mode Manual mode is intended to allow simple manual exposure and white balance control by the host. The host needs to set the CAM_AET_EXPOSURE_TIME_MS, CAM_AET_EXPOSURE_GAIN and CAM_AWB_CO- LOR_TEMPERATURE controls, the camera will calculate the appropriate integration times and gains. DUAL BAND IRCF For some applications a day/night filter would be switched in/out, this option is an additional cost to the camera system. The AP0101CS supports the use of dual band IRCF, which removes the need for the switching day/night filter. Tuning support is provided for this usage case. Refer to the AP0101CS developer guide for details. Host Controlled The Host Controlled mode is intended to give the host full control over exposure and gains. FLICKER AVOIDANCE Flicker occurs when the integration time is not an integer multiple of the period of the light intensity. The AP0101CS can be programmed to avoid flicker for 50 or 60 Hertz. For integration times less than the light intensity period (10ms for 50 Hz environment), flicker cannot be avoided. The AP0101CS supports an indoor AE mode, that will ensure flicker-free operation. EXPOSURE AND WHITE BALANCE MODES AP0101CS supports auto and manual exposure and white balance modes. In addition, it will operate within synchronized multi-camera systems. In this use case, one camera within the system will be the 'master', and the others 'slaves'. The master is used to calculate the appropriate exposure and white balance. This is then applied to all slaves concurrently under host control. OUTPUT FORMATTING The AP0101CS can output pixel data as an 8 or 10 bit word, over one or two clocks per pixel. AP0101AT supports parallel output & SMPTE modes. Auto Mode In Auto Exposure mode the AE algorithm is responsible for calculating the appropriate exposure to keep the desired scene brightness, and for applying the exposure to the underlying hardware. In Auto White Balance mode the AWB algorithm is responsible for calculating the color temperature of the scene and applying the appropriate red and blue gains. Uncompressed YCbCr Data Ordering The AP0101CS supports swapping YCbCr mode, as illustrated in Table 12. Table 12. YCbCr OUTPUT DATA ORDERING Mode Triggered Auto Mode The Triggered Auto Exposure and Triggered Auto White Balance modes are intended for the multicamera use cases, where a host is controlling the exposure and white balance of a number of cameras. The idea is that one camera is in triggered-auto mode (the master), and the others in host-controlled mode (slaves). The master camera must Data Sequence Default (no swap) Cbi Yi Cri Yi+1 Swapped CrCb Cri Yi Cbi Yi+1 Swapped YC Yi Cbi Yi+1 Cri Swapped CrCb, YC Yi Cri Yi+1 Cbi The data ordering for the YCbCr output modes for AP0101CS are shown in Table 13 and Table 14: Table 13. YCbCr Output Modes (cam_port_parallel_msb_align=0x1, cam_port_parallel_swap_bytes = 0, cam_output_format_yuv_swap_red_blue = 0) Mode YCbCr_422_8_8 YCbCr_422_10_10 YCbCr_422_16 Byte Pixel i Pixel i+1 Odd (DOUT [15:8]) Cbi Cri Even (DOUT [15:8]) Yi Yi+1 Odd (DOUT [15:6]) Cbi Cri Even (DOUT [15:6]) Yi Yi+1 Single (DOUT [15:0]) Cbi_Yi Cri_Yi+1 7. Odd means first cycle; even means second cycle. www.onsemi.com 14 Notes Data range of 0-255 (Y = 16-235 and C = 16-240) Data range of 0-1023 (Y = 64-940 and C = 64-960) Data range of 0-255 (Y = 16-235 and C = 16-240) AP0101CS Table 14. YCbCr Output Modes (cam_port_parallel_msb_align=0x0, cam_port_parallel_swap_bytes = 0, cam_output_format_yuv_swap_red_blue = 0) Mode YCbCr_422_8_8 YCbCr_422_10_10 YCbCr_422_16 Byte Pixel i Pixel i+1 Odd (DOUT [7:0]) Cbi Cri Even (DOUT [7:0]) Yi Yi+1 Odd (DOUT [9:0]) Cbi Cri Even (DOUT [9:0]) Yi Yi+1 Single (DOUT [15:0]) Cbi_Yi Cri_Yi+1 Notes Data range of 0-255 (Y = 16-235 and C = 16-240) Data range of 0-1023 (Y = 64-940 and C = 64-960) Data range of 0-255 (Y = 16-235 and C = 16-240) Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[15:8] 00 Cr Data[7:0] HBlank Y Cb Y Cr Y Cb Y Cr Y Cb Y Cr Image HBlank Y Cb Y Cr Image HBlank Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[15:8] 00 Cr Data[7:0] HBlank Y Cb Y Cr Y Cb Y Cr Y Cb Y Cr Image HBlank Y Cb Y Cr Image HBlank Active Video Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[15:8] Data[7:0] 00 Y Cb Y Cr Image Vblank Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[15:8] 00 Data[7:0] Cr Vblank Vertical Blanking NOTES: cam_port_parallel_msb_align = 0 cam_port_parallel_swap_bytes = 1 cam_output_format_yuv_swap_red_blue = 0 Figure 16. 8-bit YCbCr Output (YCbCr_422_8_8) www.onsemi.com 15 Y Cb Y Cr Image AP0101CS Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid 00 Data[5:0] Cr Data[15:6] Y Cb Y Cr HBlank Y Cb Y Cr Y Cb Y Cr Image HBlank Y Cb Y Cr Image HBlank Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid 00 Data[5:0] Cr Data[15:6] Y Cb Y Cr HBlank Y Cb Y Cr Y Cb Y Cr Image HBlank Y Cb Y Cr Image HBlank Active Video Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[5:0] Data[15:6] 00 Y Cb Y Cr Image Vblank Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[5:0] 00 Data[15:6] Cr Vblank Vertical Blanking NOTES: cam_port_parallel_msb_align = 1 cam_port_parallel_swap_bytes = 1 cam_output_format_yuv_swap_red_blue = 0 Figure 17. 10-bit YCbCr Output (YCbCr_422_10_10) www.onsemi.com 16 Y Cb Y Cr Image AP0101CS Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[7:0] Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Data[15:8] Cr Y Cb Y Cr Y Cb Y Cr Y Cb Y Cr Y Cb Y Cr HBlank Image HBlank Image HBlank Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[7:0] Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Data[15:8] Cr Y Cb Y Cr Y Cb Y Cr Y Cb Y Cr Y Cb Y Cr HBlank Image HBlank Image HBlank Active Video Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[7:0] Y Y Y Y Data[15:8] Cr Cr Cr Cr Image Vblank Pixel Clock Frame Valid Porch - 0-255 cycles Line Valid Data[7:0] Y Y Y Y Y Data[15:8] Cr CbCb CbCb Vblank Image Vertical Blanking NOTES: cam_port_parallel_swap_bytes = 0 cam_output_format_yuv_swap_red_blue = 0 Figure 18. 16-bit YCbCr Output (YCbCr_422_16) SMPTE Output The data ordering for the SMPTE output mode for AP0101AT is shown in Table 15. Table 15. SMPTE OUTPUT MODE Mode SMPTE Byte Single{Dout[15:8],GPIO[5:4]} Cb/Cr {Dout[7:0],GPIO[3:2]} Y Pixel i Pixel i+1 Cbi_Yi Cri_Yi+1 www.onsemi.com 17 Notes Data range of 4-1019 (Y = 64-940 and C = 64-960) AP0101CS Pixel Clock Data[7:0] GPIO3, GPIO2 Data[15:8] GPIO5, GPIO4 040 3FF 000 000 200 Y Y Y Y 200 3FF 000 000 200 Cb Cr Cb Cr Blanking SAV Y Y Y Y 3FF 000 000 274 Cb Cr Cb Cr 3FF 000 000 274 Image EAV 040 3FF 000 000 200 Y Y Y Y 200 3FF 000 000 200 Cb Cr Cb Cr Blanking HBlank SAV Y Y Y Y 3FF 000 000 274 Cb Cr Cb Cr 3FF 000 000 274 Image 040 200 EAV Blanking HBlank HBlank Pixel Clock Data[7:0] GPIO3, GPIO2 Data[15:8] GPIO5, GPIO4 040 3FF 000 000 200 Y Y Y Y 200 3FF 000 000 200 Cb Cr Cb Cr Blanking SAV Y Y 3FF 000 000 274 040 3FF 000 000 2AC 040 3FF 000 000 2D8 Cb Cr Cb Y Y Cr 3FF 000 000 274 200 3FF 000 000 2AC 200 3FF 000 000 2D8 Image EAV Blanking HBlank SAV Image 040 200 EAV Blanking HBlank HBlank Active Video Pixel Clock Data[7:0] GPIO3, GPIO2 Data[15:8] GPIO5, GPIO4 040 3FF 000 000 2AC 200 Blanking 040 3FF 000 000 2D8 3FF 000 000 2AC 200 3FF 000 000 2D8 SAV Blank VBlank EAV Blank 040 3FF 000 000 2AC 040 3FF 000 000 2D8 200 3FF 000 000 2AC 200 3FF 000 000 2D8 SAV Blank VBlank EAV Blank Blanking HBlank HBlank 040 200 Blanking HBlank Pixel Clock Data[7:0] GPIO3, GPIO2 Data[15:8] GPIO5, GPIO4 040 200 Blanking 040 3FF 000 000 2AC 3FF 000 000 2D8 3FF 000 000 2AC 200 3FF 000 000 2D8 SAV Blank VBlank EAV Blank 040 3FF 000 000 200 Y Y 200 3FF 000 000 200 Cb Cr Cb Blanking HBlank SAV Y Y Y Cr Y Y Y 3FF 000 000 274 Cb Cr Cb Cr 3FF 000 000 274 VBlank 040 200 EAV HBlank Blanking HBlank Figure 19. SMPTE296M Output ALTM Bayer Output selected by setting cam_mode_select = 7 (ALTM Bayer 12) or 8 (ALTM Bayer 10). The data ordering for the ALTM Bayer output modes for AP0101CS are shown in Table 16. ALTM Bayer modes are Table 16. ALTM BAYER OUTPUT MODES Mode Byte D15 D14 D13 D12 D11 D10 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0 ALTM_Bayer_10 Single 0 0 0 0 0 0 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0 ALTM_Bayer_12 Single 0 0 0 0 D11 D10 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0 Table 16 and Table 17 show LSB aligned data; it is possible by using a register setting to obtain MSB aligned data. The data ordering for the Bayer output modes for AP0101CS are shown in Table 17. Table 17. BAYER OUTPUT MODES Mode Byte D15 D14 D13 D12 D11 D10 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0 Raw_Bayer_12 Single 0 0 0 0 D11 D10 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0 www.onsemi.com 18 AP0101CS SENSOR EMBEDDED DATA The AP0101CS is capable of passing sensor embedded data in Bayer output mode only. The AP0101CS Statistics are available through the serial interface. Refer to the Developer Guide for details. Slave Address/Data Direction Byte Bits [7:1] of this byte represent the device slave address and bit [0] indicates the data transfer direction. A "0" in bit [0] indicates a write, and a "1" indicates a read. The default slave addresses used by the AP0101CS are 0x90 (write address) and 0x91 (read address). Alternate slave addresses of 0xBA (write address) and 0xBB (read address) can be selected by asserting the SADDR input signal. SLAVE TWO-WIRE SERIAL INTERFACE The two-wire slave serial interface bus enables read/write access to control and status registers within the AP0101CS. The interface protocol uses a master/slave model in which a master controls one or more slave devices. Message Byte Message bytes are used for sending register addresses and register write data to the slave device and for retrieving register read data. The protocol used is outside the scope of the two-wire serial interface specification. PROTOCOL Data transfers on the two-wire serial interface bus are performed by a sequence of low-level protocol elements, as follows: * a start or restart condition * a slave address/data direction byte * a 16-bit register address * an acknowledge or a no-acknowledge bit * data bytes * a stop condition The bus is idle when both SCLK and SDATA are HIGH. Control of the bus is initiated with a start condition, and the bus is released with a stop condition. Only the master can generate the start and stop conditions. The SADDR pin is used to select between two different addresses in case of conflict with another device. If SADDR is LOW, the slave address is 0x90; if SADDR is HIGH, the slave address is 0xBA. See Table 18 below. The user can change the slave address by changing a register value. Acknowledge Bit Each 8-bit data transfer is followed by an acknowledge bit or a no-acknowledge bit in the SCLK clock period following the data transfer. The transmitter (which is the master when writing, or the slave when reading) releases SDATA. The receiver indicates an acknowledge bit by driving SDATA LOW. As for data transfers, SDATA can change when SCLK is LOW and must be stable while SCLK is HIGH. No-Acknowledge Bit The no-acknowledge bit is generated when the receiver does not drive SDATA low during the SCLK clock period following a data transfer. A no-acknowledge bit is used to terminate a read sequence. Stop Condition A stop condition is defined as a LOW-to-HIGH transition on SDATA while SCLK is HIGH. Typical Operation Table 18. TWO-WIRE INTERFACE ID ADDRESS SWITCHING SADDR Two-Wire Interface Address ID 0 0x90 1 0xBA A typical READ or WRITE sequence begins by the master generating a start condition on the bus. After the start condition, the master sends the 8-bit slave address/data direction byte. The last bit indicates whether the request is for a READ or a WRITE, where a "0" indicates a WRITE and a "1" indicates a READ. If the address matches the address of the slave device, the slave device acknowledges receipt of the address by generating an acknowledge bit on the bus. If the request was a WRITE, the master then transfers the 16-bit register address to which a WRITE will take place. This transfer takes place as two 8-bit sequences and the slave sends an acknowledge bit after each sequence to indicate that the byte has been received. The master will then transfer the 8-bit or 16-bit data, as one or two 8-bit sequences and the slave sends an acknowledge bit after each sequence to indicate that the byte has been received. The master stops writing by generating a (re)start or stop condition. If the request was a READ, the master sends the 8-bit write slave address/data direction byte and 16-bit register address, just as in the write request. The master then Start Condition A start condition is defined as a HIGH-to-LOW transition on SDATA while SCLK is HIGH. At the end of a transfer, the master can generate a start condition without previously generating a stop condition; this is known as a "repeated start" or "restart" condition. Data Transfer Data is transferred serially, 8 bits at a time, with the MSB transmitted first. Each byte of data is followed by an acknowledge bit or a no-acknowledge bit. This data transfer mechanism is used for the slave address/data direction byte and for message bytes. One data bit is transferred during each SCLK clock period. SDATA can change when SCLK is low and must be stable while SCLK is HIGH. www.onsemi.com 19 AP0101CS generates a (re)start condition and the 8-bit read slave address/data direction byte, and clocks out the register data, 8 bits at a time. The master generates an acknowledge bit after each 8-bit transfer. The data transfer is stopped when the master sends a no-acknowledge bit. Single READ from random location Figure 15 shows the typical READ cycle of the host to the AP0101CS. The first two bytes sent by the host are an internal 16-bit register address. The following 2-byte READ cycle sends the contents of the registers to host. Previous Reg Address, N S Slave Address 0 A Reg Address [15:8] S = start condition P = stop condition Sr = restart condition A = acknowledge A = no-acknowledge A Reg Address, M Reg Address [7:0] A Sr Slave Address 1 A M+1 Read Data Read Data A [15:8] [7:0] A P slave to master master to slave Figure 20. Single READ from Random Location Single READ from current location Figure 16 shows the single READ cycle without writing the address. The internal address will use the previous address value written to the register. Previous Reg Address, N S Slave Address Read Data [15:8] 1 A Reg Address, N+1 A Read Data [7:0] A P S Slave Address 1 A N+2 Read Data [15:8] A Read Data [7:0] A P Figure 21. Single Read from Current Location has been transferred, the master generates an acknowledge bit and continues to perform byte READs until "L" bytes have been read. Sequential READ, start from random location This sequence (Figure 17) starts in the same way as the single READ from random location (Figure 15). Instead of generating a no-acknowledge bit after the first byte of data Previous Reg Address, N S Slave Address 0 A Reg Address [15:8] M+1 A M+2 Reg Address, M Reg Address [7:0] M+3 A Sr Slave Address M+L-2 Read Data Read Data Read Data Read Data A A [15:8] A [7:0] [15:8] A [7:0] 1 A M+L-1 M+1 Read Data A M+L Read Data Read Data Read Data Read Data A P A [15:8] A [7:0] [15:8] A [7:0] Figure 22. Sequential READ, Start from Random Location has been transferred, the master generates an acknowledge bit and continues to perform byte reads until "L" bytes have been read. Sequential READ, start from current location This sequence (Figure 18) starts in the same way as the single READ from current location (Figure 16). Instead of generating a no-acknowledge bit after the first byte of data Previous Reg Address, N S Slave Address 1 A N+1 N+2 Read Data Read Data Read Data Read Data Read Data Read Data A A A A A A [15:8] [7:0] [15:8] [7:0] [15:8] [7:0] Figure 23. Sequential READ, Start from Current Location www.onsemi.com 20 N+L-1 N+L Read Data Read Data A P A [15:8] [7:0] AP0101CS Single write to random location Previous Reg Address, N S Slave Address 0 A Reg Address [15:8] A M+1 Reg Address, M Reg Address [7:0] A A P A Write Data Figure 24. Single WRITE to Random Location has been transferred, the master generates an acknowledge bit and continues to perform byte writes until "L" bytes have been written. The WRITE is terminated by the master generating a stop condition. Sequential WRITE, start at random location This sequence (Figure 20) starts in the same way as the single WRITE to random location (Figure 19). Instead of generating a no-acknowledge bit after the first byte of data Previous Reg Address, N S Slave Address 0 A Reg Address [15:8] M+1 A M+2 M+1 Reg Address, M Reg Address [7:0] M+3 A Write Data M+L-2 A M+L-1 M+L Read Data Read Data Read Data Read Data A A [15:8] A [7:0] [15:8] A [7:0] A Read Data Read Data Read Data Read Data A A [15:8] A [7:0] [15:8] A [7:0] P Figure 25. Sequential WRITE, Start from Current Location Device Configuration and Usage Modes If SPI_SDI is high, then it enters the Auto-Config mode. If a device is detected, the firmware switches to the Flash-Config mode. After power is applied and the device is out of reset (either the power on reset, hard or soft reset), it will enter a boot sequence to configure its operating mode. There are essentially three configuration modes: Flash/EEPROM Config, Auto Config, and Host Config. The AP0101CS firmware supports a System Configuration phase at start-up. This consists of four sub-phases of execution: Flash detection, then one of: a. Flash Config b. Auto Config c. Host Config The System Configuration phase is entered immediately following power-up or reset. Then the firmware performs Flash Detection. Flash Detection attempts to detect the presence of an SPI Flash or EEPROM device: * If no device is detected, the firmware then samples the SPI_SDI pin state to determine the next mode: If SPI_SDI is low, then it enters the Host-Config mode. * In the Flash-Config mode, the firmware interrogates the device to determine if it contains valid configuration records: * If no records are detected, then the firmware enters the Auto-Config mode. * If records are detected, the firmware processes them. By default, when all Flash records are processed the firmware switches to the Host-Config mode. However, the records encoded into the Flash can optionally be used to instruct the firmware to proceed to auto-config, or to start streaming (via a Change-Config). In the Host-Config mode, the firmware performs no configuration, and remains idle waiting for configuration and commands from the host. The System Configuration phase is effectively complete and the AP0101CS will take no actions until the host issues commands. In the Auto-Config mode, the part will start streaming with the default settings. www.onsemi.com 21 AP0101CS USAGE MODES How a camera based on the AP0101CS will be configured depends on what features are used. In the simplest case, an AP0101AT operating in Auto-Config mode with no customized settings might be sufficient. In the simplest case no EEPROM or Flash memory or mC is required, as shown in Figure 21. AP0101CS + image sensor Auto-Config Mode Digital Out Figure 26. Auto-Config Mode AP0101CS + image sensor Serial EEPROM/Flash SPI Figure 27. Flash Mode AP0101CS + image sensor 8/16 bit mC two-wire System Bus NOTE: Serial EEPROM/Flash SPI In this configuration all settings are communicated to the AP0101CS and sensor through the micro-controller. Figure 28. Host Mode with Flash AP0101CS + image sensor 8/16 bit mC System Bus two-wire Figure 29. Host Mode Supported NVM Devices The AP0101AT supports a variety of SPI NVM devices. Refer to the Flash/EEPROM programming section of the Developer Guide for details. www.onsemi.com 22 AP0101CS ELECTRICAL SPECIFICATIONS Caution: Stresses greater than those listed in Table 19 may cause permanent damage to the device. This is a stress rating only, and functional operation of the device at these or any other conditions above those indicated in the operational sections of this specification is not implied. Exposure to absolute maximum rating conditions for extended periods may affect reliability. HOST COMMAND INTERFACE The AP0101CS has a mechanism to execute higher level commands, the Host Command Interface (HCI). Once a command has been written through the HCI, it will be executed by on-chip firmware and the results are reported back. EEPROM or Flash memory is also available to store commands for later execution. For details on the host command interface and host commands, refer to the Host Command Interface document. Table 19. ABSOLUTE MAXIMUM RATINGS Rating Symbol Parameter Min Max Unit VDD_REG Digital power (1.8 V) -0.3 4.95 V VDDIO_H Host I/O power (2.5 V, 3.3 V) 2.25 5.4 V VDDIO_S Sensor I/O power (1.8 V, 2.8 V) 1.7 5.4 V VDD Digital core power 1.1 2.5 V VDD_PLL PLL power 1.1 2.5 V VDDIO_OTPM OTPM power 2.25 5.4 V VIN DC Input Voltage -0.3 VDDIO_* + 0.3 V VOUT DC Output Voltage -0.3 VDDIO_* + 0.3 V TSTG Storage Temperature -50 150 C Table 20. ELECTRICAL CHARACTERISTICS AND OPERATING CONDITIONS Parameter Min Typ Max Unit Supply input to on-chip regulator (VDD_REG) 1.62 1.8 1.98 V Host IO voltage (VDDIO_H) 2.25 2.5/3.3 3.6 V Sensor IO voltage (VDDIO_S) 1.7 1.8/2.8 3.1 V Core voltage (VDD) 1.08 1.2 1.32 V PLL voltage (VDD_PLL) 1.08 1.2 1.32 V OTPM power supply (VDDIO_OTPM) 2.25 2.5/3.3 3.6 V Functional operating temperature (ambient - TA) -30 70 C Storage Temperature -55 150 C www.onsemi.com 23 AP0101CS Figure 30. Parallel Digital Output I/O Timing Table 21. AC ELECTRICAL CHARACTERISTICS (referring to Figure 25) Default Setup Conditions: fEXTCLK= 27 MHz, fPIXCLK = 74.125 MHz or fPIXCLK = 84 MHz, VDDIO_H = VDD_OTPM = 2.8 V, VDD_REG = VDDIO_S = 1.8 V, TA = 25C unless otherwise stated. Symbol fEXTCLK Parameter Conditions External clock frequency (Note 8) Min Typ 6 Max Unit 30 MHz tR External input clock rise time (Note 9) 10%-90% VDDIO_H - 2 5 ns tF External input clock fall time (Note 9) 90%-10% VDDIO_H - 2 5 ns External input clock duty cycle 40 50 60 % tJITTER External input clock jitter - 500 - ps fPIXCLK Pixel clock frequency (one-clock/pixel) 6 - 74.25 MHz Pixel clock frequency (two-clocks/pixel) 6 - 84 MHz DEXTCLK tRPIXCLK Pixel clock rise time (10-90%) CLOAD = 35 pF - 3 5 ns tFPIXCLK Pixel clock fall time (10-90%) CLOAD = 35 pF - 3 5 ns tPD PIXCLK to data valid - 3 5 ns tPFH PIXCLK to FV HIGH - 3 5 ns tPLH PIXCLK to LV HIGH - 3 5 ns tPFL PIXCLK to FV LOW - 3 5 ns tPLL PIXCLK to LV LOW - 3 5 ns 8. VIH/VIL restrictions apply. 9. This is applicable only a when the PLL is bypassed. When the PLL is being used then the user should ensure that VIH/VIL is met. www.onsemi.com 24 AP0101CS Table 22. DC ELECTRICAL CHARACTERISTICS Symbol Parameter VIH Input HIGH voltage (Note 10) VIL Input LOW voltage (Note 10) IIN Input leakage current (Note 11) VOH Output HIGH voltage VOL Output LOW voltage Conditions Min Max Unit VDDIO_H or VDDIO_S*0.8 V VDDIO_H or VDDIO_S*0.2 V 10 mA VIN = 0 V or VIN = VDDIO_H or VDDIO_S VDDIO_H or VDDIO_S*0.8 V VDDIO_H or VDDIO_S*0.2 V 10. VIL and VIH have min/max limitations specified by absolute ratings. 11. Excludes pins that have internal PU resistors. Table 23. OPERATING CURRENT CONSUMPTION Default Setup Conditions: fEXTCLK= 27 MHz, fPIXCLK = as below, VDD_REG = 1.8 V; VDDIO_H not included in measurement VDDIO_S = 2.8 V, VDDIO_OTPM = 3.3 V, TA = 50C unless otherwise stated Conditions Min Typ Max Unit 1.62 1.8 1.98 V VDDIO_H = 2.5 V 2.25 2.5 2.75 V VDDIO_H = 3.3 V 3 3.3 3.6 V VDDIO_S = 1.8 V 1.7 1.8 1.9 V VDDIO_S = 2.8 V 2.5 2.8 3.1 V VDDIO_OTPM = 2.5 V 2.25 2.5 2.75 V VDDIO_OTPM = 3.3 V 3 3.3 3.6 V Symbol VDD_REG VDDIO_H VDDIO_S VDDIO_OTPM IDD_REG IDDIO_S IDDIO_OTPM Total power consumption 960p HDR 30 fps 37.125 MHz YCbCr_422_16 42 mA 800p HDR 30 fps 84 MHz YCbCr_422_10_10 or YCbCr_422_8_8 36 mA 720p HDR 60 fps 74.25 MHz YCbCr_422_16 64 mA 720p HDR 30 fps 37.125 MHz YCbCr_422_16 33 mA 720p HDR 30 fps 74.25 MHz YCbCr_422_10_10 or YCbCr_422_8_8 33 mA 960p HDR 30 fps 37.125 MHz YCbCr_422_16 4.4 mA 800p HDR 30 fps 84 MHz YCbCr_422_10_10 or YCbCr_422_8_8 4.3 mA 720p HDR 60 fps 74.25 MHz YCbCr_422_16 4.5 mA 720p HDR 30 fps 37.125 MHz YCbCr_422_16 4.3 mA 720p HDR 30 fps 74.25 MHz YCbCr_422_10_10 or YCbCr_422_8_8 4.3 mA 960p HDR 30 fps 37.125 MHz YCbCr_422_16 0.25 mA 800p HDR 30 fps 84 MHz YCbCr_422_10_10 or YCbCr_422_8_8 0.25 mA 720p HDR 60 fps 74.25 MHz YCbCr_422_16 0.25 mA 720p HDR 30 fps 37.125 MHz YCbCr_422_16 0.25 mA 720p HDR 30 fps 74.25 MHz YCbCr_422_10_10 or YCbCr_422_8_8 0.25 mA 960p HDR 30 fps 37.125 MHz YCbCr_422_16 89 mW 800p HDR 30 fps 84 MHz YCbCr_422_10_10 or YCbCr_422_8_8 77 mW 720p HDR 60 fps 74.25 MHz YCbCr_422_16 129 mW 720p HDR 30 fps 37.125 MHz YCbCr_422_16 72 mW 720p HDR 30 fps 74.25 MHz YCbCr_422_10_10 or YCbCr_422_8_8 71 mW www.onsemi.com 25 AP0101CS Table 24. STANDBY CURRENT CONSUMPTION fEXTCLK= 27 MHz, VDD_REG = 1.8 V; VDDIO_S = 1.8 V, VDDIO_OTPM = VDDIO_H = 3.3 V, TA = 50C, excludes VDDIO_H current Symbol Parameter Condition Hard standby Total standby current when asserting the STANDBY signal Typ Standby power Soft standby (clock on) Total standby current fEXTCLK = 27 MHz Standby power Max Unit 1.6 mA 2.9 mW 2.1 mA 3.8 mW Table 25. INRUSH CURRENT Supply Max. Current VDD_REG (1.8 V) 150 mA VDDIO_H (2.5/3.3 V) 80 mA VDDIO_S (2.8/1.8 V) 110 mA VDDIO_OTPM (2.5/3.3 V) 170 mA TWO-WIRE SERIAL REGISTER INTERFACE The electrical characteristics of the two-wire serial register interface (SCLK, SDATA) are shown in Figure 26 and Table 26. SDATA tf tLOW tr tf tSU;DAT tHD;STA tr tBUF SCLK S tHD;STA tHD;DAT tSU;STA tHIGH tSU;STO Sr P S Figure 31. Slave Two Wire Serial Bus Timing Parameters (CCIS) Table 26. SLAVE TWO-WIRE SERIAL BUS CHARACTERISTICS (CCIS) (Default Setup Conditions: fEXTCLK = 27 MHz, VDDIO_H = VDD_OTPM = 2.8 V, VDD_REG = VDDIO_S = 1.8 V, TA = 25C unless otherwise stated) Standard-Mode Fast-Mode Symbol Min Max Min Max Unit fSCL 0 100 0 400 KHz tHD;STA 4.0 - 0.6 - ms LOW period of the SCLK clock tLOW 4.7 - 1.3 - ms HIGH period of the SCLK clock tHIGH 4.0 - 0.6 - ms Set-up time for a repeated START condition tSU;STA 4.7 - 0.6 - ms Data hold time tHD;DAT 0 (Note 13) 3.45 (Note 14) 0 0.9 (Note 14) ms Data set-up time tSU;DAT 250 - 100 - ns tr - 1000 20+0.1Cb (Note15) 300 ns Parameter SCLK Clock Frequency Hold time (repeated) START condition After this period, the first clock pulse is generated Rise time of both SDATA and SCLK signals (10-90%) www.onsemi.com 26 AP0101CS Table 26. SLAVE TWO-WIRE SERIAL BUS CHARACTERISTICS (CCIS) (continued) (Default Setup Conditions: fEXTCLK = 27 MHz, VDDIO_H = VDD_OTPM = 2.8 V, VDD_REG = VDDIO_S = 1.8 V, TA = 25C unless otherwise stated) Standard-Mode Fast-Mode Parameter Symbol Min Max Min Max Unit Fall time of both SDATA and SCLK signals (10-90%) tf - 300 20+0.1Cb (Note15) 300 ns tSU;STO 4.0 - 0.6 - ms tBUF 4.7 - 1.3 - ms Cb - 400 - 400 pF CIN SI - 3.3 - 3.3 pF CLOAD SD - 30 - 30 pF RSD 1.5 4.7 1.5 4.7 KW Set-up time for STOP condition Bus free time between a STOP and START condition Capacitive load for each bus line Serial interface input pin capacitance SDATA max load capacitance SDATA pull-up resistor 12. All values referred to VIHmin = 0.9 VDD and VILmax = 0.1 VDD levels. EXCLK = 27 MHz. 13. A device must internally provide a hold time of at least 300 ns for the SDATA signal to bridge the undefined region of the falling edge of SCLK. 14. The maximum tHD;DAT has only to be met if the device does not stretch the LOW period (tLOW) of the SCLK signal. 15. Cb = total capacitance of one bus line in pF. The electrical characteristics of the two-wire serial register interface (SCLK, SDATA) are shown in Figure 27 and Table 27. SDATA tf tLOW tr tf tSU;DAT tHD;STA tr tBUF SCLK S tHD;STA tHD;DAT tSU;STA tHIGH tSU;STO Sr P S Figure 32. Master Two Wire Serial Bus Timing Parameters (CCIS) Table 27. MASTER TWO-WIRE SERIAL BUS CHARACTERISTICS (CCIM) (Default Setup Conditions: fEXTCLK = 27 MHz, VDDIO_H = VDD_OTPM = 2.8 V, VDD_REG = VDDIO_S = 1.8 V, TA = 25C unless otherwise stated) Standard-Mode Fast-Mode Symbol Min Max Min Max Unit fSCL 0 100 0 400 KHz tHD;STA 4.0 - 0.6 - ms LOW period of the M_SCLK clock tLOW 4.7 - 1.2 - ms HIGH period of the M_SCLK clock tHIGH 4.0 - 0.6 - ms Set-up time for a repeated START condition tSU;STA 4.7 - 0.6 - ms Data hold time tHD;DAT 0 (Note 17) 3.45 (Note 18) 0 0.9 (Note 18) ms Data set-up time tSU;DAT 250 - 100 - ns tr - 1000 20+0.1Cb (Note 19) 300 ns Parameter M_SCLK Clock Frequency Hold time (repeated) START condition After this period, the first clock pulse is generated. Rise time of both M_SDATA and M_SCLK signals (10-90%) www.onsemi.com 27 AP0101CS Table 27. MASTER TWO-WIRE SERIAL BUS CHARACTERISTICS (CCIM) (continued) (Default Setup Conditions: fEXTCLK = 27 MHz, VDDIO_H = VDD_OTPM = 2.8 V, VDD_REG = VDDIO_S = 1.8 V, TA = 25C unless otherwise stated) Standard-Mode Parameter Fall time of both M_SDATA and M_SCLK signals (10-90%) Set-up time for STOP condition Bus free time between a STOP and START condition Capacitive load for each bus line Serial interface input pin capacitance M_SDATA max load capacitance M_SDATA pull-up resistor Fast-Mode Symbol Min Max Min Max Unit tf - 300 20+0.1Cb (Note 19) 300 ns tSU;STO 4.0 - 0.6 - ms tBUF 4.7 - 1.3 - ms Cb - 400 - 400 pF CIN SI - 3.3 - 3.3 pF CLOAD SD - 30 - 30 pF RSD 1.5 4.7 1.5 4.7 KW 16. All values referred to VIHmin = 0.9 VDD and VILmax = 0.1 VDD levels. EXCLK = 27 MHz. 17. A device must internally provide a hold time of at least 300 ns for the M_SDATA signal to bridge the undefined region of the falling edge of M_SCLK. 18. The maximum tHD;DAT has only to be met if the device does not stretch the LOW period (tLOW) of the M_SCLK signal. 19. Cb = total capacitance of one bus line in pF. www.onsemi.com 28 MECHANICAL CASE OUTLINE PACKAGE DIMENSIONS VFBGA81 6.5x6.5 CASE 138AG ISSUE O DATE 30 DEC 2014 SCALE 2:1 DOCUMENT NUMBER: DESCRIPTION: 98AON93699F VFBGA81 6.5x6.5 Electronic versions are uncontrolled except when accessed directly from the Document Repository. Printed versions are uncontrolled except when stamped "CONTROLLED COPY" in red. PAGE 1 OF 1 ON Semiconductor and are trademarks of Semiconductor Components Industries, LLC dba ON Semiconductor or its subsidiaries in the United States and/or other countries. ON Semiconductor reserves the right to make changes without further notice to any products herein. ON Semiconductor makes no warranty, representation or guarantee regarding the suitability of its products for any particular purpose, nor does ON Semiconductor assume any liability arising out of the application or use of any product or circuit, and specifically disclaims any and all liability, including without limitation special, consequential or incidental damages. ON Semiconductor does not convey any license under its patent rights nor the rights of others. (c) Semiconductor Components Industries, LLC, 2019 www.onsemi.com ON Semiconductor and are trademarks of Semiconductor Components Industries, LLC dba ON Semiconductor or its subsidiaries in the United States and/or other countries. ON Semiconductor owns the rights to a number of patents, trademarks, copyrights, trade secrets, and other intellectual property. A listing of ON Semiconductor's product/patent coverage may be accessed at www.onsemi.com/site/pdf/Patent-Marking.pdf. ON Semiconductor reserves the right to make changes without further notice to any products herein. ON Semiconductor makes no warranty, representation or guarantee regarding the suitability of its products for any particular purpose, nor does ON Semiconductor assume any liability arising out of the application or use of any product or circuit, and specifically disclaims any and all liability, including without limitation special, consequential or incidental damages. Buyer is responsible for its products and applications using ON Semiconductor products, including compliance with all laws, regulations and safety requirements or standards, regardless of any support or applications information provided by ON Semiconductor. "Typical" parameters which may be provided in ON Semiconductor data sheets and/or specifications can and do vary in different applications and actual performance may vary over time. All operating parameters, including "Typicals" must be validated for each customer application by customer's technical experts. ON Semiconductor does not convey any license under its patent rights nor the rights of others. ON Semiconductor products are not designed, intended, or authorized for use as a critical component in life support systems or any FDA Class 3 medical devices or medical devices with a same or similar classification in a foreign jurisdiction or any devices intended for implantation in the human body. Should Buyer purchase or use ON Semiconductor products for any such unintended or unauthorized application, Buyer shall indemnify and hold ON Semiconductor and its officers, employees, subsidiaries, affiliates, and distributors harmless against all claims, costs, damages, and expenses, and reasonable attorney fees arising out of, directly or indirectly, any claim of personal injury or death associated with such unintended or unauthorized use, even if such claim alleges that ON Semiconductor was negligent regarding the design or manufacture of the part. ON Semiconductor is an Equal Opportunity/Affirmative Action Employer. This literature is subject to all applicable copyright laws and is not for resale in any manner. PUBLICATION ORDERING INFORMATION LITERATURE FULFILLMENT: Email Requests to: orderlit@onsemi.com ON Semiconductor Website: www.onsemi.com TECHNICAL SUPPORT North American Technical Support: Voice Mail: 1 800-282-9855 Toll Free USA/Canada Phone: 011 421 33 790 2910 www.onsemi.com 1 Europe, Middle East and Africa Technical Support: Phone: 00421 33 790 2910 For additional information, please contact your local Sales Representative