flatland.evaluators.evaluator_callback module

flatland.evaluators.evaluator_callback module#

class flatland.evaluators.evaluator_callback.FlatlandEvaluatorCallbacks[source]#

Bases: FlatlandCallbacks

Implements Flatland evaluation similar to FlatlandRemoteEvaluationService for just one scenario and in offline mode.

The result dict is similar to its evaluation_state.

The following features are not implemented as they concern the evaluation of a full test with several trajectories in interactive mode: - INTIAL_PLANNING_TIMEOUT - PER_STEP_TIMEOUT - OVERALL_TIMEOUT - DEFAULT_COMMAND_TIMEOUT

get_evaluation() → dict[source]#

Evaluation for the trajectory.

Returns#

rewardfloat

cumulative reward of all agents.

normalized rewardfloat

The normalized rewards normalize the reward for an episode by dividing the whole reward by max-time-steps allowed in that episode, and the number of agents present in that episode.

termination_causeOptional[str]

if timeout occurs.

percentage_completefloat

ratio of agents done.

on_episode_end(*, env: RailEnv | None = None, data_dir: Path | None = None, **kwargs) → None[source]#

Called when an episode is done (after terminated/truncated have been logged).

The exact time of the call of this callback is after env.step([action])

Parameters#

envEnvironment

the env

data_dirPath

trajectory data dir

kwargs:

Forward compatibility placeholder.

Contents