I am away

from 28 Oct to 2 Nov (sorry, ICCV)

First GIF movie for challenge 2

courtesy of Bolton Bailey and Duc Phan Huy

Movies from class projects

Homework

• Homework 1: (easy get started stuff) Do ex 17.3 and 17.4 in AML. Now do ex 18.5 in AML. Due 10 Sep. Do this in teams of at most 3, and hand in one page at most explaining results of 18.5, by email, in PDF
• Homework 2: Do spatial priors improve semantic segmentation? (answers in teams of no more than five, due end October).
• Challenge 1: Cause the vehicle to brake after detecting a person (answers in teams of no more than five, due mid Nov; can be done!
• Challenge 2: Build a map of high bay using lidar and ros mapping software. Use a particle filter to localize in this map, using either lidar or vision (answers in teams of no more than five, due end Nov; You may share maps!).

SAFETY TRAINING!

• All please take the online course here, put together by UIUC Department of Research Safety: go to: DRS and look under training; you should find a link for laboratory safety training. Finish the two simple online courses, then email John M Hart with your course certificate, netid, UIN with “Requesting access to IRL” as the subject. This should get you access to IRL and High Bay. Email address: jmhart3@illinois.edu
• First block of training for safety lookout is 15h45, Thur 19 Sep, at High Bay. You reach the High Bay by going to the iHotel, round the conference center side, at the back parking lot you'll see a building with orange "Engineering" sign on it; that's the one, there'll be people there.
• Second block of training for safety lookout is 15h45, Thur 26 Sep, at High Bay. You reach the High Bay by going to the iHotel, round the conference center side, at the back parking lot you'll see a building with orange "Engineering" sign on it; that's the one, there'll be people there.
• Safety driver rules
• Safety lookout rules
• General safety rules

Vehicle resources

• Sending messages to the GEM and setting up camera
• Setting up ROS workspaces on the vehicle
• Autonomoustuff list of messages
• Some possible help when vehicle is misbehaving
• Notes on controlling vehicle from Ehsan and Saba
• Bag of LIDAR data of high bay HERE
• Controlling the car speed: some sample code from Ehsan Saleh; there is an error in this code, the integral term is wrong
• Controlling the car speed: a Jupyter notebook from Ehsan Saleh, showing what happened when we tried PID
• Controlling the car speed: a Jupyter notebook from Ehsan Saleh, showing a good set of simulation parameters for the GEM

My Slides (keynote)

• Context
• Intro stuff
• Basic points about networks
• Detection
• Segmenting with grab cut
• Simple semantic segmentation
• Fully Connected CRFs
• ROS Intro
• Basic ideas in Tracking
• Particle Filters

My Slides (PDF)

• Context
• Intro stuff
• Basic points about networks
• Detection
• Segmenting with grab cut
• Simple semantic segmentation
• Fully Connected CRFs
• ROS Intro
• Basic ideas in Tracking
• Particle Filters

Topics

Basic Neural Networks

Resources:

• Chapter 16, 17, Applied Machine Learning D.A. Forsyth; get the e-book from the library here.
• Very nice Stanford course notes from CS231N
• Tensorflow download install
• Tensorflow tutorials
• PyTorch

Image classification

• Chapter 17, 18, Applied Machine Learning D.A. Forsyth; get the e-book from the library here.
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe, Christian Szegedy
• Dropout: A Simple Way to Prevent Neural Networks from Overfitting Nitish Srivastava Geoffrey Hinton Alex Krizhevsky Ilya Sutskever Ruslan Salakhutdinov
• keitakurita wrote a nice blog post explaining various normalization methods here
• Swapout: Learning an ensemble of deep architectures Saurabh Singh, Derek Hoiem, David Forsyth
• ImageNet: ImageNet Large Scale Visual Recognition Challenge, by Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei in International Journal of Computer Vision December 2015, Volume 115, Issue 3, pp. 211–252.
• Pascal: The Pascal Visual Object Classes (VOC) Challenge, by Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zisserman, International Journal of Computer Vision, June 2010, Volume 88, Issue 2, pp. 303–338.
• VGGNet: Very Deep Convolutional Networks for Large-Scale Image Recognition by Karen Simonyan and Andrew Zisserman, Proc. Int. Conf. Learned Representations, 2015. You can find a version of this here
• Inception: Going Deeper with Convolutions, by Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, Proc Computer Vision and Pattern Recognition, 2015. You can find a version of this here
• ResNets: Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Proc Computer Vision and Pattern Recognition, 2015. You can find a version of this here

Object detection

• Chapter 18, Applied Machine Learning D.A. Forsyth; get the e-book from the library here.
• Selective search: Selective Search for Object Recognition by J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders, International Journal of Computer Vision September 2013, Volume 104, Issue 2, pp. 154–171.
• R-CNN: Rich feature hierarchies for accurate object detection and semantic segmentation, by R. Girshick, J. Donahue, T. Darrell, and J. Malik, IEEE Conf. on Computer Vision and Pattern Recognition, 2014. You can find a version of this here
• Fast R-CNN: Fast R-CNN, by Ross Girshick, IEEE Int. Conf. on Computer Vision (ICCV), 2015, pp. 1440–1448. You can find a version of this here
• Faster R-CNN: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, by Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun, Advances in Neural Information Processing Systems 28 (NIPS 2015). You can find a version of this here
• YOLO: You Only Look Once: Unified, Real-Time Object Detection, by Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi, Proc Computer Vision and Pattern Recognition, 2016. You can find a version of this at here. There’s a home page here
• Grabcut: Interactive foreground extraction using iterated graph cuts C Rother, V Kolmogorov, A Blake - ACM transactions on graphics (TOG), 2004 - dl.acm.org
• GREIG, D., PORTEOUS, B., AND SEHEULT, A. 1989. Exact MAP estimation for binary images. J. Roy. Stat. Soc. B. 51, 271–279.

Segmentation

• "Fully Convolutional Networks for Semantic Segmentation" Jonathan Long∗ Evan Shelhamer∗ Trevor Darrell, 2014
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Philipp Krähenbühl, Vladlen Koltun
• Fast high-dimensional filtering using the permutohedral lattice.A. Adams, J. Baek, and M. A. Davis. Computer Graphics Forum, 29(2), 2010. 2, 5

ROS

Some resources:

• ROS home page This has install instructions, tutorials, etc
• Detailed slides, with examples, etc.from Roi Yehoshua at Bar-Ilan U.
• Course, with movies from ETH Zurich
• ROS tutorial page (in case you couldn't find it from link above!)

Autoencoding

Tracking

• Notes on simple tracking and Kalman filters
• Extra notes on Kalman filter with 1D derivation
• Tracking Emerges by Colorizing Videos Carl Vondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, Kevin Murphy, ECCV 2018
• Particle filter slides by Pieter Abbeel
• Particle filter slides by Cyrill Stachniss
• Particle filter notes by Drew Bagnell, scribes Greg Seyfarth, Zachary Batts, David Fouhey
• Bag of LIDAR data of high bay HERE

Simple Mapping

• My notes on least squares, ICP, and IRLS
• Weighted least squares solutions for rotation translation; very nice notes from Olga Sorkine-Hornung and Michael Rabinovitch
• Slides on ICP from Burgard, Stachniss, Bennewitz, Arras at TU Freiburg
• Notes on robustness and iteratively reweighted least squares
• Robust registration of point sets using iteratively reweighted least squares P. Bergstrom and A.Edlund, Computational Optimization and Applications July 2014, Volume 58, Issue 3, pp 543–561

SLAM

• Basic slam notes
• Slam with an extended Kalman filter from Stachniss, TU Freiburg; does most details, most of which I omitted
• Notes on movement models for mobile robotsby Andrew Davison
• My notes on simple feature based visual slam
• LSD-Slamis a direct visual slam method; note GITHUB page
• Some notes on 3D reconstruction and perspective cameras
• Tutorial on visual slam at CVPR2014

  I found the following particularly helpful

  • Visual odometry Stephan Weiss
  • Stereo Visual odometryChris Beall
  • Dense odometryRichard Newcombe
• Supplementary notes for tutorial slides
• Homogeneous coordinates

Simple control

• PID control: the basics (My notes)
• Controlling the car speed: some sample code from Ehsan Saleh
• Controlling the car speed: a Jupyter notebook from Ehsan Saleh

Reinforcement learning

• Rough notes on the linear-quadratic regulator
• Very nice introduction to Markov decision processes, byCsaba Szepesvari Algorithms for reinforcement learning
• Rough notes on Markov decision processes
• Notes on policy gradient methods
• Very nice set of notes and resources from OpenAI

Motion Planning

• Notes on sampling based motion planning, chapter 5 of Lavalle's book "Planning Algorithms"
• Notes on planning in the presence of dynamics , chapter 14 of Lavalle's book "Planning Algorithms"
• Dubins, L.E. (July 1957). "On Curves of Minimal Length with a Constraint on Average Curvature, and with Prescribed Initial and Terminal Positions and Tangents". American Journal of Mathematics. 79 (3): 497–516.
• J. A. Reeds and L. A. Shepp. Optimal paths for a car that goes both forwards and backwards. Pacific Journal of Mathematics, 145(2):367-393, 1990.
• The open mpl library and app
• My planning notes
• My planning notes, dynamics case
• Motion graphs L. Kovar, M. Gleicher, F. Pighin, SIGGRAPH 02
• Interactive motion generation from examplesO. Arikan and D.A. Forsyth, SIGGRAPH 02
• Interactive control of avatars animated with human motion data J Lee, J Chai, PSA Reitsma, JK Hodgins, NS Pollard, SIGGRAPH 02
• Motion synthesis from annotations O Arikan, DA Forsyth, JF O'Brien… - ACM Transactions on …, 2003 - dl.acm.org

People: detecting pedestrians, etc.

People: body pose and prediction

3D reconstruction from motion

3D reconstruction from single images

Sonar sensors

Lidar sensors

Using detection to localize

Data:

• Some data from Autonomoustuff; 5G or so from a recent drive. This data is in a ROS bag (link should help sort things out)
• A small tarfile of likely useful code for reading the bag
• A bag of mobileye data from Autonomoustuff
• A bag of general data collected on a long run from Autonomoustuff (just over 6G)
• Another bag of general data collected on a long run from Autonomoustuff all sensors, rather big
• A bag of data collected in a highway construction region from Autonomoustuff all sensors, huge
• A bag of data collected in a run from Morton to Washington (?) at dusk on a long run from Autonomoustuff all sensors, huge
• A bag of data collected on a run down Green street from Autonomoustuff all sensors, huge

Papers