Skip to content

Cars learn how to drive using evolutionary algorithms and neural networks.

License

Notifications You must be signed in to change notification settings

TheFebrin/Evolutionary-cars

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Evolutionary-cars

Cars learn how to drive using evolutionary algorithms and neural networks.

Table of contents

General info

Cars:

  1. Each car has k sensors.
  2. Car's position is calculated using its current velocity and angle.
  3. Each sensor detects the distance from the car to the race track barrier.
  4. Those measurements are inputs of the neural network that predicts the action.
  5. The neural network learns using an evolutionary algorithm.
  6. Car's velocity and angle are changed according to the network's output.

Tracks:

  • Track 1 - easy
  • Track 2 - hard
  • Track 3 - very hard - waiting for introduction

Miscellaneous:

  1. Program saves the best cars it has seen.
  2. Program can save all sensors measurements during the laps.

Neural network

I used a simple network with 2 hidden layers. Network

  • Input Layer: (k sensors measurements, actual speed, actual angle)

  • Output Layer: (angle change, acceleration change)

Objective function

As it's quite hard to define an objective function I created checkpoints.

  • When a car reaches a checkpoint it earns points.
  • It is punished for the distance to the next checkpoint.

Genetic algorithm

  1. Parent Selection

    From the current population, we choose parents such that the probability of choosing a parent p is equal to its fitness value f(x), where:

    F(x) is an objective function value.

  2. Crossovers

    Parents chosen in the previous step pair up to produce offspring.

    • Simple crossover

      Each pair produces two children. We draw n such pairs, where n is a population size
      and do as follows:

      1. Iterate through genotype.
      2. With 50% probability, child_1 will get a gene from parent_1, then child_2 gets a gene from parent_2.
      3. Otherwise child_1 will get a gene from parent_2 and child_2 from parent_1.
    • Neural networks crossover

      Each pair produces one child. We draw n such pairs, where n is a population size.

      We have two parent neural networks N1 and N2. The goal is to produce an offspring network N3. We do as follows:

      1. For each car we record the sensors measurements and car's decision at every frame.
      2. From each parent we draw k samples of those measurements and corresponding predicts.
      3. We feed 2k samples to the child network N3.
      4. We hope that N3 learned something and drives not worst than it's parents.
  3. Mutations

    • Simple mutation

      Add a Gaussian Noise to the network.

    • ES mutation

      Use ES(λ + μ) algorithm for mutation. This is quite powerful algorithm, which gives us a possibility to run it alone without a crossover.

Screenshots

  • Sensor

    GIF

  • Track 1

    GIF

  • Track2

    GIF

  • Track 3

    image

Technologies

  • Python - version 3.7.3

Libraries

  • numpy
  • pygame
  • Image
  • pytorch
  • json

Results

  • Cars have no problems with learning how to deal with the first track.
  • For now they cannot finish 2nd track, because they speed up to much near the end.
  • Neural network crossover works quite well, but it is slow.
  • Simple and fast ES is the best way to go here.

Status

Project is: in_the_middle

Credits

Created by @TheFebrin
Racing tracks Designed by Freepik

About

Cars learn how to drive using evolutionary algorithms and neural networks.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages