Deep Learning Self Driving Car - MNY Team

MNYs Deep Learning Self Driving Car

2019-07-08T00:00:00+02:00

For the code please visit our Github page

Some Context

This is our bachelor project at the Chair for Autonomous Intelligent Systems (AIS) at the University of Freiburg.

It was on a self-driving car based on the paper “End-to-End Driving via Conditional Imitation Learning” by Codevilla et al.

Here we implement the architecture given in the paper and attempted to create variations and improvements,

including a version which first attempts to first perform semantic segmentation on the camera image first.

Video Demo

Variations & Improvements

We made several different variations to the original paper.
The first was to use only one camera instead of the proposed three.
This was a necessity, due to hardware limitations of the tools provided.
We later managed to expand this to two cameras by rewriting some of the code written by the supervisors.

Network	Description
standard	Uses only one forward facing camera to drive
segmented	Uses only the ground truth segmentation of a forward facing camera
seg and normal	Uses both the image and ground truth segmentation of a forward facing camera
last image too	Uses both the current and previous image from a forward facing camera
two cams	Uses a forward facing and a camera angled towards the right
self segmentation	Uses the forward facing camera and segments the image with a network first

Results

Standard

Single camera and otherwise the implementation from the paper.
Works pretty well and is shown on the video.
This is on the testing dataset / unknown map and only with a small amount of data.

Segmented

Has the ground truth segmentation of a single camera output.
As expected the results of this artificial neural network are extremely good and it handles even difficult problems with ease.
Obtaining such a perfect segmentation in reality is of course very hard if not impossible.

Had some problems with lanes coming from the opposite direction, as they had the same coloring
(eg. left lane one color right lane another) and the left lanes from both sides had the same one.

Seg And Normal

Had both the original image of a camera as well as its ground truth segmentation.
It did not perform better in most situations, as it carried with it some errors of the image only,
like thinking multiple parking spaces in a row are another lane.
It performed better than the segmentation only on streets with two lanes for each direction,
this was likely becuase it now saw the line separating the two directions. In general it was perfoming on the same level as seg. only.

Last Image Too

This one was a bit tough and brought some problems,
as we doubled the input space it saved something similar to a state machine.
It tended to overcorrect, as it thought it was still very bad,
when it saw the last picture as the same as the new one before.
Though this was probably only because we trained it with the normal dataset.
In the end we were constrained by the end of the project.

Two Cams

A second image from the same position facing a bit to the left,
though in hindsight we should've done it to the right,
to get the outer side of the lanes in better view.
This overcorrected to the left, as it had the same problem as last img. too.
With a bigger dataset both would've probably just ignored the other image.

Self Segmentation

Here we implemented our own segmentation network to segment street,
lines indicating lanes and unimportant stuff.
Sadly on the computers provided this could barely run in real time and
lagged terribly, even though it worked quite well, if slow on our GPU.

For more info on the structure of the neural networks look into our slides

Environment

The whole simulation took place on a test map similar to the Audi Cup inside Unreal Engine.

Welcome to Jekyll!

2019-07-07T00:12:31+02:00

You’ll find this post in your _posts directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different ways, but the most common way is to run jekyll serve, which launches a web server and auto-regenerates your site when a file is updated.

To add new posts, simply add a file in the _posts directory that follows the convention YYYY-MM-DD-name-of-post.ext and includes the necessary front matter. Take a look at the source for this post to get an idea about how it works.

Jekyll also offers powerful support for code snippets:

def print_hi(name)
  puts "Hi, #{name}"
end
print_hi('Tom')
#=> prints 'Hi, Tom' to STDOUT.

Check out the Jekyll docs for more info on how to get the most out of Jekyll. File all bugs/feature requests at Jekyll’s GitHub repo. If you have questions, you can ask them on Jekyll Talk.

Welcome to Jekylllllllll!

2019-07-06T00:12:31+02:00

Jekyll also offers powerful support for code snippets:

def print_hi(name)
  puts "Hi, #{name}"
end
print_hi('Tom')
#=> prints 'Hi, Tom' to STDOUT.

Check out the Jekyll docs for more info on how to get the most out of Jekyll. File all bugs/feature requests at Jekyll’s GitHub repo. If you have questions, you can ask them on Jekyll Talk.

Edge Case: Nested and Mixed Lists

2009-05-15T00:00:00+02:00

Nested and mixed lists are an interesting beast. It’s a corner case to make sure that

Lists within lists do not break the ordered list numbering order
Your list styles go deep enough.

Ordered – Unordered – Ordered

ordered item
ordered item
- unordered
- unordered
1. ordered item
2. ordered item
ordered item
ordered item

Ordered – Unordered – Unordered

ordered item
ordered item
- unordered
- unordered
- unordered item
- unordered item
ordered item
ordered item

Unordered – Ordered – Unordered

unordered item
unordered item
1. ordered
2. ordered
  - unordered item
  - unordered item
unordered item
unordered item

Unordered – Unordered – Ordered

unordered item
unordered item
- unordered
- unordered
  1. ordered item
  2. ordered item
unordered item
unordered item

Task Lists

Finish my changes
Push my commits to GitHub
Open a pull request