Machine Perception

This course provides an overview of machine perception techniques in robotics, with a focus on intelligent vehicles:

• Machine perception in robotics

• Course organization

• Sensor Overview (camera, radar, LiDAR, tactile)

• 3D Machine Vision

• Perspective camera model

• Extrinsic and intrinsic camera transformations

• Stereo vision

• Radar Sensing and Processing

• Principles of FMCW Radar

• Distance, velocity and angle measurements

• Use in object detection

• Lidar Sensing and Processing

• Point cloud representations

• Use in object detection

• Tactile Sensing and Processing

• Piezoresistive, capacitive, piezoelectric and optical sensing

• Contact and slippage

• Shape extraction and elasticity

• Visual Object Detection and Classification

• Detection vs. classification

• Object proposals

• Handcrafted features (e.g. HOG) en classification (e.g. linear SVM)

• End-to-end learning: neural networks, deep learning

• Performance metrics: confusion matrices, precision vs. recall, ROC curves

• State Estimation

• Bayesian Filtering

• Kalman Filtering

• Particle Filtering

• Object Tracking

• Data Association

• Track Management

• Self-Localization en Sensor Fusion

• Absolute vs. relative localization

• Ego-motion compensation (e.g. odometry, ICP algorithm)

• Extrinsic sensor calibration

• Environment representations (grids, voxels)

The course has a substantial practicum component (about 65% of course work load), where learned concepts are put into practice by means of programming assignments (Python, using Jupyter notebooks).

Interested students outside ME (e.g. EEMCS, Civil Engineering and Aerospace Engineering faculties) with the proper background (see Prerequisites below) are encouraged to attend.

Final practicum assignment (50%)

- Individual assignment

- Late submission: max. one day late for -1.0 grade point

- Students having an initial grade of < 5.8 will be offered an opportunity for a remake to obtain a (maximum) grade of 5.8 for this component in the Q3 quarter.

Final written exam (50%)

- Written exam on campus.

Knock-out criteria

- At most one of the Pass/Fail practicum assignments is Fail

- The minimum final practicum assignment grade is 5

- The minimum final written exam grade is 5

- The minimum final course grade is 6

The final practicum assignment will be evaluated on the computers in the designated lab rooms. If a student chooses to work using own computing equipment, the student needs to test the software on the designated computers before handing-in.

Both final practicum assignment and final written exam results remain valid for one year. Each result can separately be transferred to the next academic year, upon request.