Lattice Mipi d-phy User manual

MIPI D-PHY Bandwidth Matrix Table

User Guide

FPGA-UG-02041 Version 1.1

May 2018

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

2 FPGA-UG-02041-1.1

Contents

Acronyms in This Document .......................................................................................................................................... 4

1. Introduction .......................................................................................................................................................... 5

2. Video Format......................................................................................................................................................... 6

2.1. Video Resolution and Pixel Clock .................................................................................................................. 7

2.2. Color Depth.................................................................................................................................................. 8

3. MIPI CSI-2/DSI Interfaces ....................................................................................................................................... 9

4. Packetizing .......................................................................................................................................................... 13

4.1. xMulti-lane................................................................................................................................................. 14

5. Bandwidth and Data Rate .................................................................................................................................... 15

5.1. Bandwidth and Data Rate Calculation ......................................................................................................... 15

5.1.1. Pixel Clock .............................................................................................................................................. 15

5.1.2. Total Data Rate or Bandwidth................................................................................................................. 15

5.1.3. Data Rate per Lane ................................................................................................................................. 15

5.1.4. Bit Clock ................................................................................................................................................. 15

5.2. Examples.................................................................................................................................................... 15

5.2.1. Example 1: 1920x1080p@60Hz, RAW10, 2-lane...................................................................................... 15

5.2.2. Example 2: 3840x2160@30Hz, RAW8, 4-lane.......................................................................................... 15

6. Device Selection .................................................................................................................................................. 16

6.1. Hardware Features ..................................................................................................................................... 16

7. MIPI Data Rate Calculation .................................................................................................................................. 18

7.1. FPGA Receiver Interface ............................................................................................................................. 18

7.1.1. MachXO2/MachXO3L ............................................................................................................................. 18

7.1.2. LatticeECP3 ............................................................................................................................................ 19

7.1.3. ECP5/ECP5-5G ........................................................................................................................................ 20

7.1.4. CrossLink Soft D-PHY .............................................................................................................................. 21

7.1.5. tSU/tHD Valid Window at Higher Data Rate ............................................................................................... 21

7.2. FPGA Transmitter Interface ........................................................................................................................ 22

7.2.1. MachXO2/MachXO3L ............................................................................................................................. 22

7.2.2. LatticeECP3 ............................................................................................................................................ 23

7.2.3. ECP5/ECP5-5G ........................................................................................................................................ 24

7.2.4. Tskew Window at Higher Data Rate ........................................................................................................ 24

7.3. MIPI D-PHY Lane Number Selection Matrix Table ........................................................................................ 25

Reference ................................................................................................................................................................... 27

Technical Support ....................................................................................................................................................... 27

Revision History .......................................................................................................................................................... 27

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-UG-02041-1.1 3

Figures

Figure 1.1. CMOS Sensor Bridge Model ..........................................................................................................................5

Figure 2.1. Interlaced Mode Video Frame Format ..........................................................................................................6

Figure 2.2. Most Common Display Resolutions...............................................................................................................7

Figure 3.1. Unidirectional Receive HS Mode and Bidirectional LP Mode Interface Implementation .................................9

Figure 3.2. Unidirectional Receive HS Mode Only Implementation ...............................................................................10

Figure 3.3. Unidirectional Transmit HS Mode and Bidirectional LP Mode Interface Implementation .............................11

Figure 3.4. Unidirectional Transmit HS Mode Only Implementation ............................................................................. 12

Figure 4.1. An Example of RAW10 Transmissions on CSI-2 Bus .....................................................................................13

Figure 7.1. MachXO2/MachXO3L Maximum Data Rate................................................................................................. 18

Figure 7.2. LatticeECP3 Maximum Data Rate................................................................................................................19

Figure 7.3. ECP5/ECP5-5G Maximum Data Rate ...........................................................................................................20

Figure 7.4. CrossLink Maximum Data Rate ...................................................................................................................21

Figure 7.5. MachXO2 Maximum Data Rate...................................................................................................................22

Figure 7.6. LatticeECP3 Maximum Data Rate................................................................................................................23

Figure 7.7. ECP5/ECP5-5G Maximum Data Rate ...........................................................................................................24

Tables

Table 2.1. Common Video Format..................................................................................................................................8

Table 4.1. CSI-2 Packet Size Constraints* .....................................................................................................................14

Table 6.1. MIPI Soft D-PHY RX/TX Hardware Comparison .............................................................................................16

Table 7.1. tSU/tHD Window for Higher Data Rate ...........................................................................................................21

Table 7.2. Tskew Window for Higher Data Rate............................................................................................................24

Table 7.3. MIPI D-PHY Interface Lane Number and Line Rate Selection Example Matrix Table....................................... 25

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

4 FPGA-UG-02041-1.1

Acronyms in This Document

A list of acronyms used in this document.

Acronym

Definition

CSI

Camera Serial Interface

Data Enable

DMT

Display Monitor Timing

DSI

Display Serial Interface

EAV

End of Active Video

FPGA

Field-Programmable Gate Array

High Speed; Horizontal Sync

Low Power

MIPI

Mobile Industry Processor Interface

SAV

Start of Active Video

Vertical Sync

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-UG-02041-1.1 5

1. Introduction

As we move from the world of standard-definition to the high-definition and ultra-high-definition, the common parallel

interfaces are difficult to expand, require many interconnects and consume relatively large amounts of power.

Emerging packet-based serial interfaces, such as MIPI CSI-2 and DSI address many of the shortcomings of the parallel

interfaces, while also introducing system complexity. Understanding the mathematics behind the parallel and serial

interface bandwidth estimation can prevent a lot of problems when choosing an FPGA device with the right number of

data lanes supporting the required data transfer rate. This document describes in details the methods of calculating the

bandwidth and data rate of the image sensor’s output of the RGB, YUV, or RAW data over a single or multi-lane MIPI

CSI-2 and DSI interface. The same calculation method can be applied to other video interface such as FPD-Link, HiSPI,

and HDMI.

Figure 1.1 shows a conceptual model of the CMOS Sensor Bridge Design. On the left, a CMOS sensor transfers image

data to the FPGA through 1 to 4 serial data lanes; the FPGA sensor bridge merges the image data from multiple lanes

and converts them into parallel data; on the right, the image data are sent out over the parallel bus in standard video

format. Based on the known video format information, we can calculate the required bandwidth. Because the FPGA

does not buffer the video frames, the peak transfer rate of CMOS sensor input must meet the bandwidth requirement

of the output. With this presumption, we can estimate the maximum data rate and bit clock frequency of the CMOS

sensor interface.

Figure 1.1. CMOS Sensor Bridge Model

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

6 FPGA-UG-02041-1.1

2. Video Format

To estimate the data transfer rate, we need to understand the format of the video data transferred over the sensor

bridge. Video is composed of a series of still images. Each still image is composed of individual lines of pixel data. Figure

2.1 illustrates a conceptual interlaced video frame. The progressive video frame is similar to it except for only one field

per frame.

Figure 2.1. Interlaced Mode Video Frame Format

For digital video, either a separate Horizontal Sync (HS), Vertical Sync (VS) and Data Enable (DE) signals are used to

synchronize the video data transfer, or a special sequence embedded into the video data stream indicating the Start of

Active Video (SAV) or End of Active Video (EAV). For MIPI CSI-2, two packets structures are defined for Low Level

Protocol layer: Long packets to carry payload data, and the Short Packets for Frame Synchronization (that is Frame

Start and Frame End) and Line Synchronization (that is Line Start and Line End).

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-UG-02041-1.1 7

There are many characteristics of the video streams such as frame rate, interlaced vs progressive, aspect ratio,

stereoscopic, color space and color depth. We will review the major parameters that will be used in the calculation of

the data transfer rate in the later sections of the document.

2.1. Video Resolution and Pixel Clock

The video resolution is quoted as Width x Height, with the unit in pixels: for example, 1920x1080 means the horizontal

width is 1920 pixels and the vertical height is 1080 lines. Figure 2.2 shows the chart of the common display resolutions.

Figure 2.2. Most Common Display Resolutions

Source: https://en.wikipedia.org/wiki/File:Vector_Video_Standards4.svg

There are two major types of video format standards: SMPTE/CEA defines video standards for the Television and broadcast;

VESA Display Monitor Timing (DMT) standard defines the video standards for the computer monitors. Table 2.1 lists the

most common video resolutions. Among them, we choose HD (1280x720p), FHD (1920x1080p) and UHD (3840x2160p) as

examples to calculate the bandwidth and data rate in the later sections.

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

8 FPGA-UG-02041-1.1

Table 2.1. Common Video Format

Hactive

Vactive

Htotal

Vtotal

Refresh

Rate (Hz)

Pixel Freq

(MHz)

Television Video Format

EDTV

640x480p @ 59.94 Hz

640

480

800

525

59.94

25.175

720x480p @ 59.94 Hz

720

480

858

525

59.94

720x576p @ 50 Hz

720

576

864

625

HDTV

1280x720p @ 60 Hz

1280

720

1650

750

74.25

Full HD

1920x1080i @ 60 Hz

1920

1080

2200

1125

74.25

1920x1080p @ 60 Hz

1920

1080

2200

1125

148.5

UHDTV

3840x2160p @ 60 Hz

3840

2160

4400

2250

594

4096x2160p @ 60 Hz

4096

2160

4400

2250

594

Computer Monitor Format

XGA

1024x768p @ 60 Hz

1024

768

1344

806

WXGA

1280x800p @ 60 Hz

1280

800

1680

831

59.81

83.5

WXGA+

1440x900p @ 60 Hz

1440

900

1904

934

59.887

106.5

1366x768p @ 60 Hz

1366

768

1792

798

59.79

85.5

HD+

1600x900p @ 60 Hz (RB)

1600

900

1800

1000

108

WUXGA

1920x1200p @ 60 Hz (RB)

1920

1200

2080

1235

59.95

154

WQXGA

2560x1600p @ 60 Hz (RB)

2560

1600

2720

1646

59.97

268.5

Some of the image sensors can output the standard video formats by cropping or binning the pixels, the others may

output non-standard resolutions. The important thing is to obtain the information of the Total Horizontal Samples,

Total Vertical Lines and frame Refresh Rate. We will discuss how to calculate the bandwidth based on the above

information.

2.2. Color Depth

Color depth, also known as bit depth, can be either referred to as bits-per-pixel (bpp) which specifies the number of

bits used for a single pixel, or referred to as bits-per-component (bpc) which specifies the number of bits used to

represent each color component of a single pixel. Deep color supports 30/36/48-bit for three RGB colors. In this

document, the term Pixel Size is equivalent to the color depth in bits-per-pixel. For example, the pixel size of a 30-bit

deep color RGB is defined as 30 bits per pixel, or 10 bits per color component.

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-UG-02041-1.1 9

3. MIPI CSI-2/DSI Interfaces

MIPI Camera Serial Interface 2 (CSI-2) and Display serial Interface (DSI) are two of the serial interface protocols based

on MIPI D-PHY physical interface. MIPI D-PHY supports unidirectional HS (High Speed) mode and Bidirectional LP (Low

Power) mode. For the application of CMOS sensor bridge, we will only need the MIPI D-PHY receiver (RX) on the FPGA

receive interface, which allows the bridge to receive HS data on one clock lane and up to four data lanes. Figure 3.1

shows the block diagram of the Unidirectional Receive HS Mode and Bidirectional LP Mode interface. Figure 3.2 shows

the block diagram of the Unidirectional Receiver HS Mode Only interface. Figure 3.3 shows the block diagram of the

Unidirectional Transmit HS Mode and Bidirectional LP Mode interface. Figure 3.4 shows the block diagram of the

Unidirectional Transmit HS Mode Only interface.

Note: Figure 3.1 and Figure 3.2 do not apply to CrossLink since the CrossLink device uses dedicated MIPI D-PHY Input

buffers for both HS and LP mode to meet MIPI input voltage specfication. Figure 3.3 and Figure 3.4 also do not apply to

CrossLink as CrossLink’s programmable I/Os can only be configured as a MIPI D-PHY receiver (soft MIPI D-PHY). Only

the hardened MIPI D-PHY blocks can be configured as a transmitter.

For details on both Receiver/Transmitter HS and LP modes interface implementation, please refer to Lattice reference

design document MIPI D-PHY Interface IP (RD1182).

Aligner

iDDRx4

I/O Controller

LVDS25

LVCMOS12

50 Ω

DATA3_P

DATA3_N

LP3[1]

D3_p

LP3[0]

D3_n

50 Ω

DATA0_P

DATA0_N

LP0[1]

D0_p

LP0[0]

D0_n

50 Ω

CLOCK_P

CLOCK_N

LPCLK[1]

DCK_p

LPCLK[0]

DCK_n LVDS25

LVCMOS12

LVDS25

LVCMOS12

Figure 3.1. Unidirectional Receive HS Mode and Bidirectional LP Mode Interface Implementation

MIPI D-PHY Bandwidth Matrix Table

User Guide

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

10 FPGA-UG-02041-1.1

MIPI DPHY

TX Device

LATTICE FPGA

DATA3_P

DATA3_N

D3_p

D3_n

100 Ω

LVDS25

DPHY RX Module

I/O Controller