Deepdrive docs
  • Deepdrive
  • vendor
    • openai
      • README
        • PPO2
        • A2C
    • tensorflow
      • Surgically extracted MobileNetV2 from tensorflow/models @ 84da970ee43c04fbd53a1db3c824ea32cec8936b
        • TensorFlow Research Models
          • slim
            • nets
              • README
          • Running the TensorFlow Official ResNet with TensorRT
  • Deepdrive
  • docs
    • tutorial
      • cloud
        • Cloud setup
      • uepy
        • UnrealEnginePython Tutorial
      • leaderboard
        • Submitting to the Deepdrive leaderboard
    • Miniconda
  • README
Powered by GitBook
On this page
  • Overview
  • Step 1: Fork botleague
  • Step 2: Clone your fork
  • Step 3: Create a bot.json
  • Step 4: Commit and push your bot.json
  • Step 5: Click Pull Request on your repo's page
  • Step 6: Create your pull request
  • Step 7: Confirm that botleague has started your evaluation
  • Step 8: Wait for your bots evaluation to complete
  • Step 9: Check that your pull request is merged
  • Step 10: Go to the leaderboard!
  • Next steps
  • Local development of your bot

Was this helpful?

  1. docs
  2. tutorial
  3. leaderboard

Submitting to the Deepdrive leaderboard

PreviousleaderboardNextMiniconda

Last updated 5 years ago

Was this helpful?

Overview

First-off we'll get you on the leaderboard with the default bot, so you can get familiar with the submission process. The Deepdrive leaderboard uses to evaluate submissions. To place an agent on the leaderboards, you just need to submit a bot.json file to the league repo via pull request.

Est. time (5-15 minutes depending on your internet connection)

Step 1: Fork botleague

Login to your GitHub account and fork the repo with the fork button on the top right.

fork botleague

Step 2: Clone your fork

git clone https://github.com/<YOUR-GITHUB-NAME>/botleague
cd botleague

Step 3: Create a bot.json

Create a bots/<YOUR-GITHUB-NAME>/forward-agent/bot.json in your fork with the following JSON.

{ 
  "docker_tag": "crizcraig/forward-agent",
  "problems": ["deepdrive/unprotected_left"] 
}

NOTE: Here crizcraig/forward-agent is the default docker image for the forward-agent bot. Later on, when you modify your bot, you will replace this docker tag with a repo you have push access to.

Step 4: Commit and push your bot.json

git commit -am 'forward-agent'
git push origin master

Step 5: Click Pull Request on your repo's page

Step 6: Create your pull request

Step 7: Confirm that botleague has started your evaluation

Your pull request status should update to something similar to the following

Step 8: Wait for your bots evaluation to complete

Grab a coffee! This will take 5-10 minutes.

Step 9: Check that your pull request is merged

Once your evaluation is complete, it will be automatically merged as displayed in the image below. You may need to refresh the page. If you see an error, contact craig@voyage.auto, or file a GitHub issue. We are automatically notified of these errors as well.

Step 10: Go to the leaderboard!

Next steps

Local development of your bot

Finally, to submit your bot, create a pull request as we did above, pointing to your bot's docker image. If the image is the same, you can just add whitespace, or change some comment text to allow for the pull request.

click pull request
create the pull request yay
pull request status update
click

Finally, check the to see your bot's score and video ranked among the others.

To test your bot locally, it's ideal to run the sim and agent on your local machine as in our . You can see what your bot scores locally by passing the to main.py as we do on the evaluation server excluding the --server parameter.

Next you can run the sim in server mode locally with those again, but this time, keeping --server in the params passed to main.py.

Now make sure your bot runs as a docker container against the official scenario container. For the case of unprotected_left, for example, the docker image would be deepdriveio/deepdrive:problem_unprotected_left. You can see how our scenario problem images are built and run .

Then to build your bot container, refer to how our baseline agent bot containers are built .

leaderboards
examples
same parameters
same parameters
here
here
Botleague
botleague