flatland.evaluators package

Submodules

• flatland.evaluators.aicrowd_helpers module
  • generate_movie_from_frames()
  • get_boto_client()
  • get_submission_id()
  • is_aws_configured()
  • is_grading()
  • make_subprocess_call()
  • upload_folder_to_s3()
  • upload_random_frame_to_s3()
  • upload_to_s3()
• flatland.evaluators.client module
  • FlatlandRemoteClient
    • FlatlandRemoteClient.env_create()
    • FlatlandRemoteClient.env_step()
    • FlatlandRemoteClient.get_redis_connection()
    • FlatlandRemoteClient.ping_pong()
    • FlatlandRemoteClient.submit()
    • FlatlandRemoteClient.update_running_stats()
  • TimeoutException
• flatland.evaluators.messages module
  • FLATLAND_RL
    • FLATLAND_RL.ENV_CREATE
    • FLATLAND_RL.ENV_CREATE_RESPONSE
    • FLATLAND_RL.ENV_RESET
    • FLATLAND_RL.ENV_RESET_RESPONSE
    • FLATLAND_RL.ENV_RESET_TIMEOUT
    • FLATLAND_RL.ENV_STEP
    • FLATLAND_RL.ENV_STEP_RESPONSE
    • FLATLAND_RL.ENV_STEP_TIMEOUT
    • FLATLAND_RL.ENV_SUBMIT
    • FLATLAND_RL.ENV_SUBMIT_RESPONSE
    • FLATLAND_RL.ERROR
    • FLATLAND_RL.PING
    • FLATLAND_RL.PONG
• flatland.evaluators.service module
  • FlatlandRemoteEvaluationService
    • FlatlandRemoteEvaluationService.collect_analysis_data()
    • FlatlandRemoteEvaluationService.compute_mean_scores()
    • FlatlandRemoteEvaluationService.delete_key_in_running_stats()
    • FlatlandRemoteEvaluationService.get_env_filepaths()
    • FlatlandRemoteEvaluationService.get_env_test_and_level()
    • FlatlandRemoteEvaluationService.get_next_command()
    • FlatlandRemoteEvaluationService.get_redis_connection()
    • FlatlandRemoteEvaluationService.handle_aicrowd_error_event()
    • FlatlandRemoteEvaluationService.handle_aicrowd_info_event()
    • FlatlandRemoteEvaluationService.handle_aicrowd_success_event()
    • FlatlandRemoteEvaluationService.handle_env_create()
    • FlatlandRemoteEvaluationService.handle_env_step()
    • FlatlandRemoteEvaluationService.handle_env_submit()
    • FlatlandRemoteEvaluationService.handle_ping()
    • FlatlandRemoteEvaluationService.instantiate_evaluation_metadata()
    • FlatlandRemoteEvaluationService.instantiate_redis_connection_pool()
    • FlatlandRemoteEvaluationService.report_error()
    • FlatlandRemoteEvaluationService.run()
    • FlatlandRemoteEvaluationService.save_actions()
    • FlatlandRemoteEvaluationService.save_analysis_data()
    • FlatlandRemoteEvaluationService.save_episode()
    • FlatlandRemoteEvaluationService.save_merged_env()
    • FlatlandRemoteEvaluationService.send_error()
    • FlatlandRemoteEvaluationService.send_response()
    • FlatlandRemoteEvaluationService.update_evaluation_metadata()
    • FlatlandRemoteEvaluationService.update_running_stats()

Module contents

Top-level package for flatland.