start
====== Differences ====== This shows you the differences between two versions of the page.
|
start [2011/04/01 13:34] tobi |
start [2015/11/29 10:59] (current) tobi [News] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ======Dynamic Vision Sensor (DVS) - asynchronous temporal contrast silicon retina====== | ======Dynamic Vision Sensor (DVS) - asynchronous temporal contrast silicon retina====== | ||
| - | {{dvs128-037.jpg?w=400 |DVS128 prototype cameras}} | + | {{dvs128-037.jpg?w=600 |DVS128 prototype cameras}} |
| - | {{dvssystemchip.jpg?400 |Temporal Contrast Vision sensor and die photo}} | + | |
| =====News===== | =====News===== | ||
| - | * ** See [[userguide]] if you are a user of one of our engineering samples. ** | + | * **See [[http://inilabs.com|inilabs.com]] if you are a user** of one of our **DVS128**, **DVS128_PAER**, **eDVS128**, or **DAVIS** inilabs USB camera engineering samples. |
| + | * ** See [[https://www.youtube.com/playlist?list=PLVtZ8f-q0U5iROmshCSqzzBNlqpUGbSkS|YouTube video collection of DVS and DAVIS recordings and applications]] ** | ||
| + | * [[https://wiki.lsr.ei.tum.de/nst/programming/edvsgettingstarted|Check out the eDVS (embedded dynamic vision sensor).]] | ||
| * **Winner, ISSCC 2006 Solid State Circuits Society Jan Van Vessem Outstanding European Paper award.** {{lichtsteiner_isscc2006_d27_09.pdf|Download the 2-page ISSCC paper}}. {{lichtsteiner_dvs_jssc08.pdf|Download the full 2008 JSSC paper}}. | * **Winner, ISSCC 2006 Solid State Circuits Society Jan Van Vessem Outstanding European Paper award.** {{lichtsteiner_isscc2006_d27_09.pdf|Download the 2-page ISSCC paper}}. {{lichtsteiner_dvs_jssc08.pdf|Download the full 2008 JSSC paper}}. | ||
| *Key [[#specifications]]: **128x128 resolution, 120dB dynamic range, 23mW power consumption, 2.1%-contrast threshold mismatch, 15 us-latency** | *Key [[#specifications]]: **128x128 resolution, 120dB dynamic range, 23mW power consumption, 2.1%-contrast threshold mismatch, 15 us-latency** | ||
| *Our partner's [[http://www.smart-systems.at/products/products_smart_optical_sensors_en.html|Smart Systems SmartEye]] - Austrian Research Centers GmbH smart traffic camera using this sensor | *Our partner's [[http://www.smart-systems.at/products/products_smart_optical_sensors_en.html|Smart Systems SmartEye]] - Austrian Research Centers GmbH smart traffic camera using this sensor | ||
| - | * [[http://cordis.europa.eu/ictresults/index.cfm/section/news/tpl/article/BrowsingType/Features/ID/89794|Vision sensors keep their eye on the ball at Euro 2008]] - article published 10 June 2008 on ICT Results. | + | * {{:ict_results_-_vision_sensors_keep_their_eye_on_the_ball_at_euro_2008.pdf|Vision sensors keep their eye on the ball at Euro 2008}} - article published 10 June 2008 on ICT Results. |
| * {{:delbrucknmefreeingvisionfromframes2006.pdf|Freeing vision from frames}} - article in [[http://ine-web.org/research/newsletters/index.html|The Neuromorphic Engineer]], 2006. | * {{:delbrucknmefreeingvisionfromframes2006.pdf|Freeing vision from frames}} - article in [[http://ine-web.org/research/newsletters/index.html|The Neuromorphic Engineer]], 2006. | ||
| * Jorg Conradt's [[http://www.ini.uzh.ch/~conradt/research/PencilBalancer/|Pencil Balancing Robot]] in regular use for demonstrations, Sept. 2009. | * Jorg Conradt's [[http://www.ini.uzh.ch/~conradt/research/PencilBalancer/|Pencil Balancing Robot]] in regular use for demonstrations, Sept. 2009. | ||
| * 20 new [[userguide|DVS128 systems]] were assembled and tested, March 2009. | * 20 new [[userguide|DVS128 systems]] were assembled and tested, March 2009. | ||
| * INI spins off [[http://www.inilabs.com|inilabs]] to sell INI-developed technology, 2010. | * INI spins off [[http://www.inilabs.com|inilabs]] to sell INI-developed technology, 2010. | ||
| - | * 10 DVS sensors are used in {{:lichtkunst_in_der_einstein-passage_am_bahnhof_aarau_-_aarau_-_aargau_-_aargauer_zeitung.pdf|the permanent Einstein Passage exhibit in the train station Aarau}} which opened March 2011. PhD student Christian Braendli writes the bulk of the DVS signal processing, which is done in [[http://jear.wiki.sourceforge.net|jAER]], and which tracks individuals and groups to trigger some visual effects. | + | * 10 DVS sensors are used in {{:lichtkunst_in_der_einstein-passage_am_bahnhof_aarau_-_aarau_-_aargau_-_aargauer_zeitung.pdf|the permanent Einstein Passage exhibit in the train station Aarau}} which opened March 2011. PhD student Christian Braendli writes the bulk of the DVS signal processing, which is done in [[http://jaerproject.net|jAER]], and which tracks individuals and groups to trigger some visual effects. |
| * [[http://aer-ear.ini.uzh.ch|Shih-Chii Liu's AER-EAR binaural silicon cochlea]]. This event-based silicon cochlea offers a user-friendly USB interface to jAER and allow rapid development of event-based auditory processing algorithms for sound localization and auditory scene analysis. New AER-EAR systems are being built in April 2011. | * [[http://aer-ear.ini.uzh.ch|Shih-Chii Liu's AER-EAR binaural silicon cochlea]]. This event-based silicon cochlea offers a user-friendly USB interface to jAER and allow rapid development of event-based auditory processing algorithms for sound localization and auditory scene analysis. New AER-EAR systems are being built in April 2011. | ||
| - | * DVS128 cameras are being commercially assembled, March 2011. | + | * 200 DVS128 cameras were commercially assembled, June 2011. |
| + | |||
| =====Technology Briefing===== | =====Technology Briefing===== | ||
| Line 48: | Line 50: | ||
| | Requires fast PC | Works with any laptop | Lower costs\\ Lower power consumption | | | Requires fast PC | Works with any laptop | Lower costs\\ Lower power consumption | | ||
| | Extremely large data storage (often several TB)\\ Highly redundant data | Low storage requirements\\ No redundant data | Lower costs\\ More portable\\ Easier and faster data management | | | Extremely large data storage (often several TB)\\ Highly redundant data | Low storage requirements\\ No redundant data | Lower costs\\ More portable\\ Easier and faster data management | | ||
| - | | Custom interface cards | Webcam-sized, USB2.0\\ [[http://jear.wiki.sourceforge.net|Java API]] | More portable\\ Easier programming | | + | | Custom interface cards | Webcam-sized, USB2.0\\ [[http://jaerproject.net|Java API]] | More portable\\ Easier programming | |
| | Batch-mode acquisition\\ Off-line post-processing | Real-time acquisition\\ Extremely low latency | Continuous processing\\ No downtime, lower costs | | | Batch-mode acquisition\\ Off-line post-processing | Real-time acquisition\\ Extremely low latency | Continuous processing\\ No downtime, lower costs | | ||
| | Low dynamic range, ordinary sensitivity\\ Needs special bright lighting (lasers, strobes, etc.) for short exposure times | High sensitivity\\ No special lighting needed | Lower costs\\ Simpler data acquisition | | | Low dynamic range, ordinary sensitivity\\ Needs special bright lighting (lasers, strobes, etc.) for short exposure times | High sensitivity\\ No special lighting needed | Lower costs\\ Simpler data acquisition | | ||
| Line 126: | Line 128: | ||
| =====Technology===== | =====Technology===== | ||
| - | The 4 key innovations in this development are the pixel design, the [[http://jaer.wiki.sourceforge.net/biasgen|on-chip digital bias generators]], the highly-usable USB2 implementation, and the [[http://jaer.wiki.sourceforge.net|jAER processing software]]. | + | The 4 key innovations in this development are the pixel design, the [[http:jaerproject.net/biasgen|on-chip digital bias generators]], the highly-usable USB2 implementation, and the [[http://jaerproject.net|jAER processing software]]. |
| Line 133: | Line 135: | ||
| The pixel uses a //continuous-time front end photoreceptor//,(inspired from the [[http://www.ini.uzh.ch/~tobi/anaprose/recep/index.php|adaptive photoreceptor]]), followed by a //precision self-timed switched-capacitor differentiator //(inspired by the column amplifier used in the [[http://www.ini.uzh.ch/~tobi/bipImager/index.php|pulsed bipolar imager]]). The most novel aspects of this pixel are the idea of self-timing the switch-cap differentiation and self-biasing the photoreceptor. This pixel does a data-driven AD conversion (like biology, but very different than the usual ADC architecture). Local capacitor ratio matching gives the differencing circuit a precisely defined gain for changes in log intensity, thus reducing the effective imprecision of the comparators that detect positive and negative changes in log intensity. | The pixel uses a //continuous-time front end photoreceptor//,(inspired from the [[http://www.ini.uzh.ch/~tobi/anaprose/recep/index.php|adaptive photoreceptor]]), followed by a //precision self-timed switched-capacitor differentiator //(inspired by the column amplifier used in the [[http://www.ini.uzh.ch/~tobi/bipImager/index.php|pulsed bipolar imager]]). The most novel aspects of this pixel are the idea of self-timing the switch-cap differentiation and self-biasing the photoreceptor. This pixel does a data-driven AD conversion (like biology, but very different than the usual ADC architecture). Local capacitor ratio matching gives the differencing circuit a precisely defined gain for changes in log intensity, thus reducing the effective imprecision of the comparators that detect positive and negative changes in log intensity. | ||
| - | The pixel is drawn to use quad mirror symmetry to isolate the analog and digital parts. Most of the pixel area is capacitance. The periphery uses the Boahen lab's AER circuits. The chip includes a [[http://jaer.wiki.sourceforge.net/biasgen|fully programmable bias current generator]] that makes the chip's operation largely independent of temperature and process variations; all dozen chips we have built up into boards behave indistinguishably with identical digital bias settings. | + | The pixel is drawn to use quad mirror symmetry to isolate the analog and digital parts. Most of the pixel area is capacitance. The periphery uses the Boahen lab's AER circuits. The chip includes a [[http://jaerproject.net/biasgen|fully programmable bias current generator]] that makes the chip's operation largely independent of temperature and process variations; all dozen chips we have built up into boards behave indistinguishably with identical digital bias settings. |
| {{pixelprinciple.png?w=400|Pixel circuit principle}} | {{pixelprinciple.png?w=400|Pixel circuit principle}} | ||
| Line 141: | Line 143: | ||
| ====System integration==== | ====System integration==== | ||
| - | The DVS is integrated with a USB2.0 high-speed interface that plugs into any PC or laptop. The host software presently stands at >200 Java classes. The **open source** [[http://jaer.wiki.sourceforge.net|jAER software project]] lets you render events in a variety of formats, capture them, replay them, and most important, process them using events and their precise timing. | + | The DVS is integrated with a USB2.0 high-speed interface that plugs into any PC or laptop. The host software presently stands at >200 Java classes. The **open source** [[http://jaerproject.net|jAER software project]] lets you render events in a variety of formats, capture them, replay them, and most important, process them using events and their precise timing. |
| - | ===== Specifications ===== | + | See the [[userguide]] page for detailed chip and camera specifications. |
| - | + | ||
| - | | Functionality|Asynchronous temporal contrast | | + | |
| - | | Pixel size um (lambda)|40x40 (200x200) | | + | |
| - | | Fill factor (%)|9% (PD area 151μm<sup>2</sup>) | | + | |
| - | | Fabrication process|4M 2P 0.35um standard CMOS | | + | |
| - | | Pixel complexity|26 transistors (14 analog), 3 capacitors | | + | |
| - | | Array size|128x128 (higher resolutions coming soon) | | + | |
| - | | Die size mm2|6.0 x 6.3 | | + | |
| - | | Chip interface|15-bit word-parallel AER\\ active low Req and Ack 4-phase handshake| | + | |
| - | | Computer interface|USB 2.0, Windows XP driver\\ [[http://jaer.wiki.sourceforge.net|Java API]] & Matlab output file format | | + | |
| - | | Power consumption|Chip: 23mW @ 3.3V\\ 1.5mA core\\ 0.3mA logic\\ 5.5mA biases\\ USB System: approx. 70mA | | + | |
| - | | Dynamic range|120dB\\ 2 lux to > 100 klux scene illumination with f/1.2 lens with normal contrast objects\\ Moonlight (<0.1 lux) with high contrast scene | | + | |
| - | | Photodiode dark current at room temperature|4fA (~10nA/cm<sup>2</sup>)\\ Nwell photodiode | | + | |
| - | | Response latency|15μs @ 1 klux chip illumination | | + | |
| - | | Max events/sec|~1M events/sec | | + | |
| - | | FPN, matching|2.1% contrast\\ (The event threshold 1-sigma mismatch is 2.1% contrast) | | + | |
| - | | Optics|Standard CS-mount lenses\\ Other custom mounts available | | + | |
| - | + | ||
| Line 193: | Line 176: | ||
| ====== Publications ====== | ====== Publications ====== | ||
| - | * [[Publications]] | + | * [[http://sensors.ini.uzh.ch/publications.html|Sensors group publications (since 2010)]] |
| + | * [[https://www.ini.uzh.ch/~tobi/wiki/doku.php?id=publications|Delbruck publications (back to 1990's)]] | ||
| ======User guide====== | ======User guide====== | ||
| - | See the [[userguide]] page for more information if you are a user of one of the engineering prototype systems. | + | See the [[http://inilabs.com/support/|userguide]] page at inilabs for more information if you are a user of one of the engineering prototype systems. |
| ====== Links to related work ====== | ====== Links to related work ====== | ||
| Line 210: | Line 193: | ||
| ======Contact====== | ======Contact====== | ||
| Tobi Delbruck <tobi@ini.phys.ethz.ch>\\ | Tobi Delbruck <tobi@ini.phys.ethz.ch>\\ | ||
| - | Institute of Neuroinformatics\\ Winterthurerstr. 190\\ 8057 Zürich\\ Switzerland\\ Tel. +41-1-635 3051\\ Fax +41-1-635 3053\\ [[http://www.ini.uzh.ch]] | + | Institute of Neuroinformatics\\ Winterthurerstr. 190\\ 8057 Zürich\\ Switzerland [[http://www.ini.uzh.ch]] |
| + | [[http://inilabs.com|inilabs.com]] for R&D prototype availability and support. | ||
Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /home/httpd/vhosts/siliconretina.ini.uzh.ch/httpdocs/wiki/inc/auth.php on line 656
start.1301657663.txt.gz · Last modified: 2011/04/01 13:34 by tobi
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
