# 3D Vision

### Second semester 2020

Andrés Marrugo, PhD

*Universidad Tecnológica de Bolívar*

## Aims and Scope

This course covers the main techniques of 3D data acquisition, both passive (stereoscopic vision and multiple views) and active (active triangulation by structured light). The camera calibration process and the use of the geometric constraints of stereo systems are studied in detail in order to obtain adequate correspondences. Finally, the post-processing stages of three-dimensional vision systems are discussed: the representation of 3D images (e.g. through meshes, depth maps etc.), the extraction of characteristics and the recognition of 3D objects.

In this course the student will obtain proficiency in the fundamentals of stereo vision systems, the calibration of cameras, the geometric restrictions of stereoscopic systems (epipolar line, fundamental matrix, etc.) and carrying out the software implementation in simple applications. In addition, the student will learn the main active techniques of 3D image acquisition (active triangulation by structured light, and the basics of kinect, among others). Finally, different forms of representation of 3D images and their use in problems of computer vision will be presented.

Prior knowledge of this course includes probability, linear algebra, and calculus. Programming experience in MATLAB is desirable, but not required.

## Useful Resources

### Tutorials, review materials

- MATLAB tutorial
- More MATLAB tutorials: basic operations, programming, working with images
- Linear algebra review
- Random variables review

### MATLAB reference

## Outline

This is a new course, this website will be updated as we go along.

### Lecture 1: Introduction and Linear Algebra Resfresher

We will be discussing the main aspects and motivation for 3D computer vision. Also a brief overview of the Linear Algebra involved in the course.

### Assignment 1

A summary of Linear Algebra and Probability exercises. **Due date:** 2020-08-21.

**Send by email**.

### Lecture 2: Sensors and Image Formation

We will be studying the main aspects about image formation, the pinhole camera model and image sensors.

#### Reading

### Assignment 2

Matlab basics and image formation. **Due date:** 2020-08-28.

- Assignment 2
**Send by email**.

### Lecture 3: 2D-2D Coordinate transforms

In this lecture we introduce 2D coordinate transforms.

### Lecture 4: 3D-3D Coordinate transforms

In this lecture we introduce 3D coordinate transforms.

#### Code

#### Reading

### Quiz

**TBD** Quiz on Coordinate transforms.

### Assignment 3

Coordinate transforms. **Due date:** TBD.

### Lecture 5: 3D-2D Coordinate Transforms

In this lecture we recall perspective projection, this time from a matrix multiplication perspective.

### Lecture 6: Image Filtering

Linear filters, correlation, convolution kernel, smoothing and edge detection via the gradient.

#### Reading

### Lecture 7: Binary Images

Manipulating Binary or logical images.

### Assignment 4

Coordinate transforms. **Due date:** TBD.

- Assignment 4
**Send by email**.

### Lecture 7: 2D Image transforms

2D Image transformations and least squares fitting.

### Lecture 8: Alignment using Nonlinear Least Squares

Alignment using Newton’s method.

### Lecture 9: Pose estimation

#### Reading

### Assignment 5

Correspondence problem. **Due date:** TBD.

- robot1.jpg
- robot2.jpg
- Assignment 5
**Send by email**.

### Lecture 11: Corner detection

We now describe how to detect features or salient points from an image.

#### Reading

### Lecture 12: SIFT

In this lecture we will discuss Scale-Invariant Keypoints.

#### Reading

- Read “Distinctive Image Features from Scale-Invariant Keypoints” by D. Lowe.

### Lecture 13: Linear Pose Estimation

In this lecture we will discuss the direct estimation of a camera pose using matrix algebra.

#### Reading

### Projects

In this course you are required to complete a short project, similar to the assignments, but you are free to choose the approach and the implementation. You will work in teams of two and you will deliver a project report in the IEEE paper format and a 15 minute presentation.

**The project is due December 15th.**

### Assignment 6

Correspondence problem. **Due date:** TBD.

### Lecture 14: Stereo

In this lecture we will discuss stereo vision.

- Lecture 14 slides
- pentagonLeft.png
- pentagonRight.png
- left.png
- right.png
- stereo_ball.m
- Lecture 13 examples

### Lecture 15: Stereo

In this lecture we will discuss stereo vision.

### Lecture 16: Structure from Motion

In this lecture we will discuss structure from motion.

### Lecture 17: Camera calibration

In this lecture we will discuss how to calculate the camera intrinsic parameters.

#### Reading

### Lecture 18: Structured Light

In this lecture we will discuss structured light systems.

### Final exam

The exam is 3 hours and you may use any resources, (MATLAB, Books, slides, etc.). However, provide full answers. These may be done by hand and scanned. Please make everything as clear as possible.