汽車英文文獻講課講稿

1、A high speed tri-vision system for automotive applicationsMarc Anthony Azzopardi & Ivan Grech & Jacques LeconteAbstractPurpose Cameras are excellent ways of non-invasively monitoring the interior and exterior of vehicles. In particular, high speed stereovision and multivision systems are important f

2、or transport applications such as driver eye tracking or collision avoidance. This paper addresses the synchronisation problem which arises when multivision camera systems are used to capture the high speed motion common in such applications. Methods An experimental, high-speed tri-vision camera sys

3、tem intended for real-time driver eye-blink and saccade measurement was designed, developed, implemented and tested using prototype, ultra-high dynamic range, automotive- grade image sensors specifically developed by E2V (formerly Atmel) Grenoble SA as part of the Europea n FP6 project -SENSATION (a

4、dva need sen sor developme nt for atte nti on stress, vigilance and sleep/wakefulnessmonitoring). Results The developed system can sustain frame rates of 59.8 Hz at the full stereovision resolution of 1280 480 but this can reach 750 Hz when a 10 k pixel Region of Interest (ROI) is used, with a maxim

5、um global shutte speed of 1/48000 s and a shutter efficiency of 99.7%. The data can be reliably transmitted uncompressed over standard copper Camera-Link? cables over 5 metres. The synchronisation error between the left and right stereo images is less than 100 ps and this has been verified both elec

6、trically and optically. Synchronisation is auto- matically established at boot-up and maintained during resolution changes. A third camera in the set can be configured independently. The dynamic range of the 10bit sensorsexceeds 123 dB with a spectral sensitivity extending well into the infra-red ra

7、nge. Conclusion The system was subjected to a comprehensive testing protocol, which confirms that the salient require- ments for the driver monitoring application are adequately met and in some respects, exceeded. The synchronization technique presented may also benefit several other auto- motive st

8、ereovision applications including near and far- field obstacle detection and collision avoidance, road condition monitoring and others.Keywords Synchronisation . High-speed automotive multivision . Active safety . Driver monitoring . Sensors1 IntroductionOver the coming years, one of the areas of gr

9、eatest research and development potential will be that of automotive sensor systems and telematics 1, 2. In particular, there is a steeply growing interest in the utilisation of multiple cameras within vehicles to augment vehicle Human-Machine Interfacing (HMI) for safety, comfort and security.For e

10、xternal monitoring applications, cameras are emerging as viable alternatives to systems such Radio, Sound and Light/Laser Detection and Ranging (RADAR, SODAR, LADAR/LIDAR). The latter are typically rather costly and either have poor lateral resolution or require mechanical moving parts.For vehicle c

11、abin applications, camerasoutshine other techniques with their ability to collect large amounts of information in a highly unobtrusive way. Moreover, cameras can be used to satisfy several applications at once by re-processing the same vision data in multiple ways, thereby reducing the total number

12、of sensors required to achieve equivalent functionality. However, automotive vision still faces several open challenges in terms of optoelectronic-performance, size, reliability, power con- sumption, sensitivity, multi-camera synchronisation, inter- facing and cost.In this paper, several of these pr

13、oblems are addressed. As an example, driver head localisation, point of gaze detection and eye blink rate measurement is considered for which the design of a dash-board-mountable automotive stereovision camera system is presented. This was developed as part of a large FP6 Integrated Project - SENSAT

14、ION (Advanced Sensor Development for Attention, Stress, Vigilance and Sleep/Wakefulness Monitoring). The overarching goal of exte ndable to multivisi on systems 5 -8.The camera system is built around a matched set of prototype, ultra-high dynamic range, automotive-grade, image sensors specifically d

15、eveloped and fabricated by E2V Grenoble SA for this application. The sensor which is a novelty in its own right, is the AT76C410ABA CMOS monochrome automotive image sensor. This sensor imple- ments a global shutter to allow distortion-free capture of fast motion. It also incorporates an on- chip Mul

16、ti-ROI feature with up to eight Regions Of Interest (ROI) with pre- programming facility and allows fast switching from one image to another. In this way, several real-time parallel imaging processing tasks can be carried out with one sensor. Each ROI is independently programmable on-the-fly with re

17、spect to integration time, gain, sub-sampling/binning, position, width and height.A fairly comprehensive series of“ bench tests we”re conducted in order to test the validity of the new concepts and to initially verify the reliability of the system across various typical automotive operating conditio

18、ns. Additional rigorous testing would of course be needed to guarantee a mean time before failure (MTBF) and to demonstrate the efficacy of the proposed design techniques over statistically significant production quantities.2 Application backgroundThe set of conceivable automotive camera application

19、s is an ever-growing list with some market research reports claiming over 10 cameras will be required per vehicle 9. The incomplete list includes occupant detection, occupant classification, driver recognition, driver vigilance and drowsiness monitoring 10, road surface condition moni- toring, inter

20、section assistance 11, lane-departure warning 12, blind spot warning, surround view, collision warning, mitigation or avoidance, headlamp control, accident record-ing, vehicle security, parking assistance,traffic sign detection 13, adaptive cruise control and night/synthetic vision (Fig.1).Cost cons

21、iderationsThe automotive sector is a very cost-sensitive one and the monetary cost per subsystem remains an outstanding issue which could very well be the biggest hurdle in the way of full deployment of automotive vision. The supply-chain industry has been actively addressing the cost dilemma by int

22、roducing Field Programmable Gate Array (FPGA) vision processing and by moving towards inexpensive image sensors based on Complementary Metal Oxide Semiconductor (CMOS) technology 14. Much has been borrowed from other very large embedded vision markets which are also highly cost-sensitive: These are

23、mobile telephony and portable computing. However, automotive vision pushes the bar substantially higher in terms of performance requirements. The much wider dynamic range, higher speed, global shuttering, and excellent infra-red sensitivity are just a few of the characteristics that set most automot

24、ive vision applications apart. This added complex- ity increases cost. However, as the production volume picks up, unit cost is expected to drop quite dramatically by leveraging on the excellent economies of scale afforded by the CMOS manufacturing process.Some groups have been actively developing a

25、nd pro- moting ways of reducing the number of cameras required per vehicle. Some of these methods try to combine disparate applications to re-use the same cameras. Other techniques (and products) have emerged that trade-off some accuracy and reliability to en able the use of mono cular visi on in sc

26、e narios which traditi on ally required two or more cameras 10, 15, 16. Dista nee estimati on for 3D obstacle localisatio n is one such example. Such tactics will serve well to con tai n cost in the in terim. However, it is expected that the cost of the imagi ng devices will eve ntually drop to a le

27、vel where it will no Ion ger be the determining factor in the overall cost of automotive vision systems. At this point, we argue thatFig. 1Some automotive visi on applicati onsreliability, performa nee and accuracy con sid- erati ons will aga in reach the forefront.In this paper the cost issue is ad

28、dressed, but in a differe nt way. Rather tha n discardi ng stereo- and multi-vision altogether, a low-cost (but still high-performanee) technique for synchronously comb ining multiple cameras is pre- sen ted. Cabli ng requireme nts are likewise shared, result ing in a reducti on in the corresp ondin

29、g cost and cable harn ess weight sav in gs.The role of high speed visi onA number of automotive vision applications require high frame-rate video capture. External applications involving high relative motion such as traffic sign, oncoming traffic or obstacle detecti on are obvious can didates. The n

30、 eed for high speed visi on is perhaps less obvious in the interior of a vehicle. However, some driver monitoring applications can get quite demanding in this respect. Eye-bli nk and saccade measureme nt, for in sta nee, is one of the tech niq ues that may be employed to measure a driver s state of

31、vigilanee and to detect the onset0? f16e epso happe ns that these are also some of the fastest of all huma n moti on and accurate rate of cha nge measureme nts may require frame rates running up to several hun dred hertz. Other applica- tions such as occupa nt detect ion and classificati on can be a

32、ccommodated with much lower frame rates but the n the same cameras may occasi on ally be required to capture high speed moti on for visual-servoing such as when modulating airbag release or seatbelt tensioning during a crash situation.A continued case for stereovision/multivisionSeveral of the appli

33、cations mentioned, stand to benefit from the use of stereovision or multivision sets of cameras operating in tandem. This may be necessaryto extend the field of view or to increase diversity and ruggedness and also to allow accurate stereoscopic depth estimation11. Then, of course, multivision is in

34、deed one of the most effective ways of counteracting optical occlusions.Monocular methods have established a clear role (alongside stereoscopy) but they rely on assumptions that may not always be true or consistently valid. Assumptions such as uniform parallel road marking, continuity of road textur

35、e, and operational vehicle head or tail lights are somewhat utopian and real world variability serves to diminish reliability. Often, what is easily achievable with stereoscopy can prove to be substantially complex with monocular approaches 17. The converse may also be true, because stereovision dep

36、ends on the ability to unambiguously find corresponding features in multiple views. Stereovision additionally brings a few challenges of its own, such as the need for a large baseline camera separation, sensitivity to relative camera positioning and sensitivity to inter-camera synchronisation.Not su

37、rprisingly, it has indeed been shown that better performance (than any single method) can be obtained by combining the strengths of both techniques 18, 19. As the cost issue fades away, monovision and multivision should therefore be viewed as complimentary rather than competing techniques. This is n

38、othing but yet another example of how vision data can be processedand interpreted in multiple ways to improve reliability and obtain additional information.In this paper, the benefit of combining stereo and monocular methods is demonstrated at the hardware level. A tri-vision camera is presented tha

39、t utilises a synchronized stereovision pair of cameras for 3D head localisation and orientation measurement.Using this information, a third monocular high-speed camera can then be accurately controlled to rapidly track both eyes of the driver using the multi-ROI feature. Such a system greatly econom

40、ises on bandwidth by limiting the high speed capture to very small and specific regions of interest. This compares favourably to the alternative method of running a stereovision system at high frame rate and at full resolution.The importance for high synchronisationOne of the basic tenets of multivi

41、sion systems is the accurate temporal correspondence between frames captured by the different cameras in the set. Even a slight frequency or phase difference between the image sampling processes of the cameras would lead to difficulties during transmission and post processing. Proper operation usual

42、ly rests on the ability to achieve synchronised, low latency video capture between cameras in the same multivision set. Moreover, this requirement extends to the video transport mechanism which must also ensure synchronous delivery to the central processing hubs. The need for synchronization depends

43、 on the speed of the motion to be captured rather than the actual frame rate employed, but in general, applications which require high speed vision will often also require high synchronisation.Interestingly, even preliminary road testing of automo- tive vision systems reveals another sticky problem

44、-camera vibrati on. This is a problem that has already bee n faced many years ago by the first optical systems to en ter mai nstream vehicle use 20he optical track ing mecha ni sms used in car-entertainment CDROM/DVD drives are severely affected by automotive vibration and fairly complex (and fairly

45、 expensive) schemesare required to mitigate these effects 21. The inevitable vibration essentially converts nearly all mobile application scenarios into high speed vision problems becauseeven low amplitude camera motion translates into significant image motion. The problem gets worse as the subject

46、distance and/or optical focal length increases. Mounting the cameras more rigidly helps by reducing the vibration amplitude, but it also automatically increases the vibration frequency which negates some of the gain. Active cancellation of vibration is no new topic 22; however, this usually comes at

47、 a disproportionate cost. Thus, while high frame rates may not be important in all situations, short aperture times and high synchronization remain critically important to circumvent the vibration problem.A small numerical example quickly puts the problem into perspective. Consider a forward look in

48、g camera for in - la ne obstacle mon itori ng based on a ? in ch, 1024 512 image sen sor array with an active area of 5.7 a.9 mm behind a 28 mm (focal length) lens. If such a system is subjected to a modest 10 mrad amplitude, sinusoidal, angular vibration at 100 Hz, simple geometric optics implies a

49、 peak pixel shift rate of around 32,000 pixels/sec.Thus, if the error in correspondencebetween left and right stereo frames is to be limited to a vertical shift comparable to one pixel, a stereovision system would require a frame synchronisation accuracy which is better than 30 microseconds. Then on

50、 the road, the levels of vibration can get significantly worse and this does not yet take into account the additional high speed motion that may be present in the field of view. In summary, synchronization is a problem that has been largely overlooked and will become more important as the industry a

51、nd consumer performance expectations increase. In this paper, a synchronisation technique based on matched cameras sharing a single clock is presented. The system affords a very high degree of synchronisation in fact, much higher than is actually demanded by the driver monitoring application. Synchr

52、onisation difficulties arising during initialisation and camera mode changes are also addressed in this paper using a novel frozen-clock programming technique.High bandwidth interconnect and processingAutomotive vision faces another formidable challenge - bandwidth. Having several cameras running at

53、 high frame rates and at high resolutions quickly pushes such applica- tions into the multi GBit/s domain. This poses new pressureson a sector that is still barely warming up to multi-MBit/s interface speeds.New automotive video interface standardswill be required, and while it makes sense to base t

54、hese on existing and proven interconnects, it may be argued that a completely new standard is needed to properly address the requirements of this peculiar market. The stage is set for a standards-war and in fact, one is currently brewing which should eventually see the evolu- tion of a veritable Aut

55、omotive Video Bus. Such a bus faces a tall order which includes: low cable cost, low interface cost, low specific weight, multi-GBit/s sustained throughput, multiplex-ability, preservation of synchronisation, high integrity, excellent electromagnetic compatibility (EMC) characteristics, low latency,

56、 low jitter, and a minimum 5 m cable span without repeaters 23.There is of course a second repercussion of such high bandwidths. Impressive data rates necessitate equally impressive computational power in order to perform all the associated video processing in real-time. This is fairly problematic c

57、onsidering the limited capabilities of most automotive embedded processors, but this is changing with the entry of FPGAs into the automotive market 23 5. Aside from offering substantial (and sufficient) in-line processing power, FPGAs also serve to reduce cost by combining most of the interface glue

58、-logic into a single chip. Then, FPGAs have the added appeal of re- configurability which allows aftermarket updates through simple firmware cha nges - though this raises several security concerns 25. 3 Video interfacesA survey of currently available interface standards reveals that none of the pres

59、ent offerings are ideally suited to faithfully transport high speed, high resolution, synchron- ised stereovideo over appreciable distances. The following is a comparative discussion of the merits and shortcomings of the various interfaces.Bandwidth considerationsThe Interface throughput is the majo

60、r concern since high resolutions are desirable and the required frame rates can reach into the high hundreds per second. At a moderate 200 frames per sec ond, a 10 bit per pixel, greyscale, 640 480X2, ste 惦ovisi on system gen erates video at 1.229 GBit/s. Even 1536 76XX at 12 bit is not at all farfe