Setting up Training Environment...
Creating Liquid PPO Agent...
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Starting Training (This may take a while)...
----------------------------------
| rollout/           |           |
|    ep_len_mean     | 1e+03     |
|    ep_rew_mean     | -2.56e+04 |
| time/              |           |
|    fps             | 608       |
|    iterations      | 1         |
|    time_elapsed    | 3         |
|    total_timesteps | 2048      |
----------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.23e+04    |
| time/                   |              |
|    fps                  | 177          |
|    iterations           | 2            |
|    time_elapsed         | 23           |
|    total_timesteps      | 4096         |
| train/                  |              |
|    approx_kl            | 0.0049500195 |
|    clip_fraction        | 0.0341       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.67        |
|    explained_variance   | -9.31e-05    |
|    learning_rate        | 0.0003       |
|    loss                 | 1.15e+05     |
|    n_updates            | 10           |
|    policy_gradient_loss | -0.00313     |
|    std                  | 0.999        |
|    value_loss           | 1.99e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.19e+04   |
| time/                   |             |
|    fps                  | 129         |
|    iterations           | 3           |
|    time_elapsed         | 47          |
|    total_timesteps      | 6144        |
| train/                  |             |
|    approx_kl            | 0.002073763 |
|    clip_fraction        | 0.00542     |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.67       |
|    explained_variance   | 2.62e-05    |
|    learning_rate        | 0.0003      |
|    loss                 | 5.71e+04    |
|    n_updates            | 20          |
|    policy_gradient_loss | -0.000381   |
|    std                  | 0.996       |
|    value_loss           | 1.15e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.28e+04    |
| time/                   |              |
|    fps                  | 116          |
|    iterations           | 4            |
|    time_elapsed         | 70           |
|    total_timesteps      | 8192         |
| train/                  |              |
|    approx_kl            | 0.0047560623 |
|    clip_fraction        | 0.0275       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.64        |
|    explained_variance   | 8.34e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 7.31e+04     |
|    n_updates            | 30           |
|    policy_gradient_loss | -0.00268     |
|    std                  | 0.988        |
|    value_loss           | 1.42e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.4e+04    |
| time/                   |             |
|    fps                  | 93          |
|    iterations           | 5           |
|    time_elapsed         | 109         |
|    total_timesteps      | 10240       |
| train/                  |             |
|    approx_kl            | 0.004183922 |
|    clip_fraction        | 0.0234      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.62       |
|    explained_variance   | -4.77e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 1.35e+05    |
|    n_updates            | 40          |
|    policy_gradient_loss | -0.003      |
|    std                  | 0.985       |
|    value_loss           | 2.21e+05    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.28e+04   |
| time/                   |             |
|    fps                  | 98          |
|    iterations           | 6           |
|    time_elapsed         | 125         |
|    total_timesteps      | 12288       |
| train/                  |             |
|    approx_kl            | 0.005293761 |
|    clip_fraction        | 0.0418      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.61       |
|    explained_variance   | -3.58e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 1.32e+05    |
|    n_updates            | 50          |
|    policy_gradient_loss | -0.00347    |
|    std                  | 0.985       |
|    value_loss           | 2.86e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.41e+04    |
| time/                   |              |
|    fps                  | 97           |
|    iterations           | 7            |
|    time_elapsed         | 146          |
|    total_timesteps      | 14336        |
| train/                  |              |
|    approx_kl            | 0.0050999783 |
|    clip_fraction        | 0.0295       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.59        |
|    explained_variance   | -7.15e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 4.66e+04     |
|    n_updates            | 60           |
|    policy_gradient_loss | -0.00355     |
|    std                  | 0.974        |
|    value_loss           | 8.14e+04     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.6e+04     |
| time/                   |              |
|    fps                  | 103          |
|    iterations           | 8            |
|    time_elapsed         | 158          |
|    total_timesteps      | 16384        |
| train/                  |              |
|    approx_kl            | 0.0042739166 |
|    clip_fraction        | 0.0147       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.57        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.64e+05     |
|    n_updates            | 70           |
|    policy_gradient_loss | -0.0019      |
|    std                  | 0.972        |
|    value_loss           | 3.31e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.7e+04     |
| time/                   |              |
|    fps                  | 108          |
|    iterations           | 9            |
|    time_elapsed         | 170          |
|    total_timesteps      | 18432        |
| train/                  |              |
|    approx_kl            | 0.0053871158 |
|    clip_fraction        | 0.0297       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.56        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.43e+05     |
|    n_updates            | 80           |
|    policy_gradient_loss | -0.00304     |
|    std                  | 0.972        |
|    value_loss           | 5.33e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.81e+04    |
| time/                   |              |
|    fps                  | 112          |
|    iterations           | 10           |
|    time_elapsed         | 181          |
|    total_timesteps      | 20480        |
| train/                  |              |
|    approx_kl            | 0.0035741455 |
|    clip_fraction        | 0.0138       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.56        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 1.74e+05     |
|    n_updates            | 90           |
|    policy_gradient_loss | -0.00138     |
|    std                  | 0.971        |
|    value_loss           | 3.49e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.79e+04   |
| time/                   |             |
|    fps                  | 116         |
|    iterations           | 11          |
|    time_elapsed         | 193         |
|    total_timesteps      | 22528       |
| train/                  |             |
|    approx_kl            | 0.004108442 |
|    clip_fraction        | 0.0245      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.56       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 2.53e+05    |
|    n_updates            | 100         |
|    policy_gradient_loss | -0.00274    |
|    std                  | 0.971       |
|    value_loss           | 5.84e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.84e+04    |
| time/                   |              |
|    fps                  | 119          |
|    iterations           | 12           |
|    time_elapsed         | 205          |
|    total_timesteps      | 24576        |
| train/                  |              |
|    approx_kl            | 0.0057261223 |
|    clip_fraction        | 0.0375       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.53        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 8.12e+04     |
|    n_updates            | 110          |
|    policy_gradient_loss | -0.003       |
|    std                  | 0.96         |
|    value_loss           | 1.72e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.82e+04    |
| time/                   |              |
|    fps                  | 122          |
|    iterations           | 13           |
|    time_elapsed         | 217          |
|    total_timesteps      | 26624        |
| train/                  |              |
|    approx_kl            | 0.0051155365 |
|    clip_fraction        | 0.0225       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.5         |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.4e+05      |
|    n_updates            | 120          |
|    policy_gradient_loss | -0.00331     |
|    std                  | 0.955        |
|    value_loss           | 4.7e+05      |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.83e+04   |
| time/                   |             |
|    fps                  | 125         |
|    iterations           | 14          |
|    time_elapsed         | 228         |
|    total_timesteps      | 28672       |
| train/                  |             |
|    approx_kl            | 0.005322621 |
|    clip_fraction        | 0.042       |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.48       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.33e+05    |
|    n_updates            | 130         |
|    policy_gradient_loss | -0.00404    |
|    std                  | 0.952       |
|    value_loss           | 2.58e+05    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.78e+04   |
| time/                   |             |
|    fps                  | 127         |
|    iterations           | 15          |
|    time_elapsed         | 240         |
|    total_timesteps      | 30720       |
| train/                  |             |
|    approx_kl            | 0.006120109 |
|    clip_fraction        | 0.0588      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.46       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.32e+05    |
|    n_updates            | 140         |
|    policy_gradient_loss | -0.00445    |
|    std                  | 0.942       |
|    value_loss           | 2.73e+05    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.79e+04   |
| time/                   |             |
|    fps                  | 129         |
|    iterations           | 16          |
|    time_elapsed         | 253         |
|    total_timesteps      | 32768       |
| train/                  |             |
|    approx_kl            | 0.004814163 |
|    clip_fraction        | 0.0178      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.43       |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 5.79e+04    |
|    n_updates            | 150         |
|    policy_gradient_loss | -0.000932   |
|    std                  | 0.939       |
|    value_loss           | 1.42e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.82e+04    |
| time/                   |              |
|    fps                  | 130          |
|    iterations           | 17           |
|    time_elapsed         | 266          |
|    total_timesteps      | 34816        |
| train/                  |              |
|    approx_kl            | 0.0027875581 |
|    clip_fraction        | 0.0163       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.43        |
|    explained_variance   | 1.19e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.62e+05     |
|    n_updates            | 160          |
|    policy_gradient_loss | -0.00144     |
|    std                  | 0.94         |
|    value_loss           | 3.25e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.84e+04    |
| time/                   |              |
|    fps                  | 132          |
|    iterations           | 18           |
|    time_elapsed         | 277          |
|    total_timesteps      | 36864        |
| train/                  |              |
|    approx_kl            | 0.0035902325 |
|    clip_fraction        | 0.0154       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.43        |
|    explained_variance   | 1.19e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.59e+05     |
|    n_updates            | 170          |
|    policy_gradient_loss | -0.00172     |
|    std                  | 0.942        |
|    value_loss           | 3.88e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.83e+04    |
| time/                   |              |
|    fps                  | 134          |
|    iterations           | 19           |
|    time_elapsed         | 289          |
|    total_timesteps      | 38912        |
| train/                  |              |
|    approx_kl            | 0.0044348813 |
|    clip_fraction        | 0.025        |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.44        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 1.36e+05     |
|    n_updates            | 180          |
|    policy_gradient_loss | -0.00187     |
|    std                  | 0.943        |
|    value_loss           | 2.4e+05      |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.77e+04   |
| time/                   |             |
|    fps                  | 136         |
|    iterations           | 20          |
|    time_elapsed         | 300         |
|    total_timesteps      | 40960       |
| train/                  |             |
|    approx_kl            | 0.003115609 |
|    clip_fraction        | 0.0162      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.42       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.02e+05    |
|    n_updates            | 190         |
|    policy_gradient_loss | -0.00152    |
|    std                  | 0.936       |
|    value_loss           | 2.02e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.72e+04    |
| time/                   |              |
|    fps                  | 136          |
|    iterations           | 21           |
|    time_elapsed         | 314          |
|    total_timesteps      | 43008        |
| train/                  |              |
|    approx_kl            | 0.0044121114 |
|    clip_fraction        | 0.0311       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.42        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 4.9e+04      |
|    n_updates            | 200          |
|    policy_gradient_loss | -0.00261     |
|    std                  | 0.941        |
|    value_loss           | 1.09e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.69e+04    |
| time/                   |              |
|    fps                  | 138          |
|    iterations           | 22           |
|    time_elapsed         | 326          |
|    total_timesteps      | 45056        |
| train/                  |              |
|    approx_kl            | 0.0050966754 |
|    clip_fraction        | 0.0294       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.41        |
|    explained_variance   | 1.19e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 5.13e+04     |
|    n_updates            | 210          |
|    policy_gradient_loss | -0.00221     |
|    std                  | 0.933        |
|    value_loss           | 1.11e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.71e+04    |
| time/                   |              |
|    fps                  | 139          |
|    iterations           | 23           |
|    time_elapsed         | 337          |
|    total_timesteps      | 47104        |
| train/                  |              |
|    approx_kl            | 0.0042023044 |
|    clip_fraction        | 0.0154       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.4         |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 5.53e+04     |
|    n_updates            | 220          |
|    policy_gradient_loss | -0.000932    |
|    std                  | 0.934        |
|    value_loss           | 1.32e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.74e+04    |
| time/                   |              |
|    fps                  | 140          |
|    iterations           | 24           |
|    time_elapsed         | 348          |
|    total_timesteps      | 49152        |
| train/                  |              |
|    approx_kl            | 0.0060270163 |
|    clip_fraction        | 0.0548       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.4         |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 1.27e+05     |
|    n_updates            | 230          |
|    policy_gradient_loss | -0.00514     |
|    std                  | 0.932        |
|    value_loss           | 2.93e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.77e+04   |
| time/                   |             |
|    fps                  | 141         |
|    iterations           | 25          |
|    time_elapsed         | 361         |
|    total_timesteps      | 51200       |
| train/                  |             |
|    approx_kl            | 0.003641401 |
|    clip_fraction        | 0.0161      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.4        |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.7e+05     |
|    n_updates            | 240         |
|    policy_gradient_loss | -0.00216    |
|    std                  | 0.937       |
|    value_loss           | 3.48e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.79e+04    |
| time/                   |              |
|    fps                  | 142          |
|    iterations           | 26           |
|    time_elapsed         | 372          |
|    total_timesteps      | 53248        |
| train/                  |              |
|    approx_kl            | 0.0040730843 |
|    clip_fraction        | 0.0225       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.41        |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 2.05e+05     |
|    n_updates            | 250          |
|    policy_gradient_loss | -0.00147     |
|    std                  | 0.934        |
|    value_loss           | 4.28e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.8e+04    |
| time/                   |             |
|    fps                  | 143         |
|    iterations           | 27          |
|    time_elapsed         | 384         |
|    total_timesteps      | 55296       |
| train/                  |             |
|    approx_kl            | 0.003144626 |
|    clip_fraction        | 0.00791     |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.42       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.71e+05    |
|    n_updates            | 260         |
|    policy_gradient_loss | -0.00195    |
|    std                  | 0.94        |
|    value_loss           | 3.93e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.83e+04    |
| time/                   |              |
|    fps                  | 144          |
|    iterations           | 28           |
|    time_elapsed         | 397          |
|    total_timesteps      | 57344        |
| train/                  |              |
|    approx_kl            | 0.0052720373 |
|    clip_fraction        | 0.0272       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.42        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.75e+05     |
|    n_updates            | 270          |
|    policy_gradient_loss | -0.00242     |
|    std                  | 0.935        |
|    value_loss           | 3.07e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.79e+04    |
| time/                   |              |
|    fps                  | 145          |
|    iterations           | 29           |
|    time_elapsed         | 409          |
|    total_timesteps      | 59392        |
| train/                  |              |
|    approx_kl            | 0.0041839215 |
|    clip_fraction        | 0.0244       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.4         |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.79e+05     |
|    n_updates            | 280          |
|    policy_gradient_loss | -0.00283     |
|    std                  | 0.933        |
|    value_loss           | 3.86e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.8e+04     |
| time/                   |              |
|    fps                  | 145          |
|    iterations           | 30           |
|    time_elapsed         | 421          |
|    total_timesteps      | 61440        |
| train/                  |              |
|    approx_kl            | 0.0053371564 |
|    clip_fraction        | 0.0308       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.35        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 3.19e+04     |
|    n_updates            | 290          |
|    policy_gradient_loss | -0.00282     |
|    std                  | 0.915        |
|    value_loss           | 6.26e+04     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.78e+04    |
| time/                   |              |
|    fps                  | 146          |
|    iterations           | 31           |
|    time_elapsed         | 433          |
|    total_timesteps      | 63488        |
| train/                  |              |
|    approx_kl            | 0.0045930664 |
|    clip_fraction        | 0.0416       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.31        |
|    explained_variance   | 1.19e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.66e+05     |
|    n_updates            | 300          |
|    policy_gradient_loss | -0.00376     |
|    std                  | 0.913        |
|    value_loss           | 3.15e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.77e+04   |
| time/                   |             |
|    fps                  | 147         |
|    iterations           | 32          |
|    time_elapsed         | 445         |
|    total_timesteps      | 65536       |
| train/                  |             |
|    approx_kl            | 0.006433362 |
|    clip_fraction        | 0.0423      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.29       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 7.14e+04    |
|    n_updates            | 310         |
|    policy_gradient_loss | -0.00386    |
|    std                  | 0.906       |
|    value_loss           | 1.45e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.74e+04    |
| time/                   |              |
|    fps                  | 147          |
|    iterations           | 33           |
|    time_elapsed         | 457          |
|    total_timesteps      | 67584        |
| train/                  |              |
|    approx_kl            | 0.0060111308 |
|    clip_fraction        | 0.0567       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.27        |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 8.6e+04      |
|    n_updates            | 320          |
|    policy_gradient_loss | -0.004       |
|    std                  | 0.904        |
|    value_loss           | 1.9e+05      |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.71e+04   |
| time/                   |             |
|    fps                  | 148         |
|    iterations           | 34          |
|    time_elapsed         | 469         |
|    total_timesteps      | 69632       |
| train/                  |             |
|    approx_kl            | 0.002752479 |
|    clip_fraction        | 0.0267      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.28       |
|    explained_variance   | -1.19e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 3.72e+04    |
|    n_updates            | 330         |
|    policy_gradient_loss | -0.000776   |
|    std                  | 0.909       |
|    value_loss           | 5.9e+04     |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.7e+04    |
| time/                   |             |
|    fps                  | 149         |
|    iterations           | 35          |
|    time_elapsed         | 480         |
|    total_timesteps      | 71680       |
| train/                  |             |
|    approx_kl            | 0.004144692 |
|    clip_fraction        | 0.0243      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.29       |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 4.48e+04    |
|    n_updates            | 340         |
|    policy_gradient_loss | -0.00131    |
|    std                  | 0.907       |
|    value_loss           | 1.06e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.69e+04    |
| time/                   |              |
|    fps                  | 149          |
|    iterations           | 36           |
|    time_elapsed         | 492          |
|    total_timesteps      | 73728        |
| train/                  |              |
|    approx_kl            | 0.0060227686 |
|    clip_fraction        | 0.043        |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.28        |
|    explained_variance   | 1.19e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 9.28e+04     |
|    n_updates            | 350          |
|    policy_gradient_loss | -0.00293     |
|    std                  | 0.903        |
|    value_loss           | 1.91e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.68e+04   |
| time/                   |             |
|    fps                  | 150         |
|    iterations           | 37          |
|    time_elapsed         | 504         |
|    total_timesteps      | 75776       |
| train/                  |             |
|    approx_kl            | 0.003745494 |
|    clip_fraction        | 0.0147      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.29       |
|    explained_variance   | -1.19e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 6.81e+04    |
|    n_updates            | 360         |
|    policy_gradient_loss | -0.00168    |
|    std                  | 0.91        |
|    value_loss           | 1.31e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.71e+04    |
| time/                   |              |
|    fps                  | 150          |
|    iterations           | 38           |
|    time_elapsed         | 516          |
|    total_timesteps      | 77824        |
| train/                  |              |
|    approx_kl            | 0.0039524576 |
|    clip_fraction        | 0.0286       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.3         |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.13e+05     |
|    n_updates            | 370          |
|    policy_gradient_loss | -0.00305     |
|    std                  | 0.909        |
|    value_loss           | 2.44e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.67e+04   |
| time/                   |             |
|    fps                  | 150         |
|    iterations           | 39          |
|    time_elapsed         | 529         |
|    total_timesteps      | 79872       |
| train/                  |             |
|    approx_kl            | 0.005160669 |
|    clip_fraction        | 0.0254      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.29       |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 1.6e+05     |
|    n_updates            | 380         |
|    policy_gradient_loss | -0.00292    |
|    std                  | 0.907       |
|    value_loss           | 2.55e+05    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.66e+04    |
| time/                   |              |
|    fps                  | 151          |
|    iterations           | 40           |
|    time_elapsed         | 540          |
|    total_timesteps      | 81920        |
| train/                  |              |
|    approx_kl            | 0.0046265204 |
|    clip_fraction        | 0.0285       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.27        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.09e+04     |
|    n_updates            | 390          |
|    policy_gradient_loss | -0.00145     |
|    std                  | 0.902        |
|    value_loss           | 3.81e+04     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.67e+04    |
| time/                   |              |
|    fps                  | 151          |
|    iterations           | 41           |
|    time_elapsed         | 553          |
|    total_timesteps      | 83968        |
| train/                  |              |
|    approx_kl            | 0.0042863134 |
|    clip_fraction        | 0.0239       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.26        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 9.31e+04     |
|    n_updates            | 400          |
|    policy_gradient_loss | -0.00143     |
|    std                  | 0.9          |
|    value_loss           | 1.85e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.68e+04   |
| time/                   |             |
|    fps                  | 151         |
|    iterations           | 42          |
|    time_elapsed         | 566         |
|    total_timesteps      | 86016       |
| train/                  |             |
|    approx_kl            | 0.005065168 |
|    clip_fraction        | 0.0253      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.24       |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 1.1e+05     |
|    n_updates            | 410         |
|    policy_gradient_loss | -0.0026     |
|    std                  | 0.894       |
|    value_loss           | 2.2e+05     |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.67e+04    |
| time/                   |              |
|    fps                  | 152          |
|    iterations           | 43           |
|    time_elapsed         | 577          |
|    total_timesteps      | 88064        |
| train/                  |              |
|    approx_kl            | 0.0030657728 |
|    clip_fraction        | 0.0121       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.22        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.52e+05     |
|    n_updates            | 420          |
|    policy_gradient_loss | -0.00173     |
|    std                  | 0.892        |
|    value_loss           | 3.03e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.73e+04    |
| time/                   |              |
|    fps                  | 153          |
|    iterations           | 44           |
|    time_elapsed         | 588          |
|    total_timesteps      | 90112        |
| train/                  |              |
|    approx_kl            | 0.0051104454 |
|    clip_fraction        | 0.0357       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.22        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 8.25e+04     |
|    n_updates            | 430          |
|    policy_gradient_loss | -0.00201     |
|    std                  | 0.893        |
|    value_loss           | 1.81e+05     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -2.8e+04     |
| time/                   |              |
|    fps                  | 153          |
|    iterations           | 45           |
|    time_elapsed         | 600          |
|    total_timesteps      | 92160        |
| train/                  |              |
|    approx_kl            | 0.0051720007 |
|    clip_fraction        | 0.033        |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.23        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 3.47e+05     |
|    n_updates            | 440          |
|    policy_gradient_loss | -0.00428     |
|    std                  | 0.896        |
|    value_loss           | 7.59e+05     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.9e+04    |
| time/                   |             |
|    fps                  | 153         |
|    iterations           | 46          |
|    time_elapsed         | 612         |
|    total_timesteps      | 94208       |
| train/                  |             |
|    approx_kl            | 0.004487371 |
|    clip_fraction        | 0.0192      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.23       |
|    explained_variance   | 5.96e-08    |
|    learning_rate        | 0.0003      |
|    loss                 | 6.84e+05    |
|    n_updates            | 450         |
|    policy_gradient_loss | -0.00245    |
|    std                  | 0.895       |
|    value_loss           | 1.3e+06     |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -2.98e+04   |
| time/                   |             |
|    fps                  | 153         |
|    iterations           | 47          |
|    time_elapsed         | 625         |
|    total_timesteps      | 96256       |
| train/                  |             |
|    approx_kl            | 0.005325151 |
|    clip_fraction        | 0.0271      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.22       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 9.17e+05    |
|    n_updates            | 460         |
|    policy_gradient_loss | -0.00341    |
|    std                  | 0.892       |
|    value_loss           | 1.81e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -3.08e+04   |
| time/                   |             |
|    fps                  | 154         |
|    iterations           | 48          |
|    time_elapsed         | 637         |
|    total_timesteps      | 98304       |
| train/                  |             |
|    approx_kl            | 0.004731435 |
|    clip_fraction        | 0.0245      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.21       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 8.34e+05    |
|    n_updates            | 470         |
|    policy_gradient_loss | -0.00319    |
|    std                  | 0.888       |
|    value_loss           | 1.53e+06    |
-----------------------------------------
----------------------------------------
| rollout/                |            |
|    ep_len_mean          | 1e+03      |
|    ep_rew_mean          | -3.21e+04  |
| time/                   |            |
|    fps                  | 154        |
|    iterations           | 49         |
|    time_elapsed         | 648        |
|    total_timesteps      | 100352     |
| train/                  |            |
|    approx_kl            | 0.00386829 |
|    clip_fraction        | 0.00859    |
|    clip_range           | 0.2        |
|    entropy_loss         | -5.2       |
|    explained_variance   | -1.19e-07  |
|    learning_rate        | 0.0003     |
|    loss                 | 1.05e+06   |
|    n_updates            | 480        |
|    policy_gradient_loss | -0.00151   |
|    std                  | 0.887      |
|    value_loss           | 2.04e+06   |
----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -3.34e+04   |
| time/                   |             |
|    fps                  | 154         |
|    iterations           | 50          |
|    time_elapsed         | 660         |
|    total_timesteps      | 102400      |
| train/                  |             |
|    approx_kl            | 0.005242249 |
|    clip_fraction        | 0.0372      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | -1.19e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 1.95e+06    |
|    n_updates            | 490         |
|    policy_gradient_loss | -0.00506    |
|    std                  | 0.889       |
|    value_loss           | 3.17e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -3.5e+04    |
| time/                   |             |
|    fps                  | 155         |
|    iterations           | 51          |
|    time_elapsed         | 673         |
|    total_timesteps      | 104448      |
| train/                  |             |
|    approx_kl            | 0.003204999 |
|    clip_fraction        | 0.00566     |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.21       |
|    explained_variance   | 5.96e-08    |
|    learning_rate        | 0.0003      |
|    loss                 | 1.29e+06    |
|    n_updates            | 500         |
|    policy_gradient_loss | -0.000996   |
|    std                  | 0.89        |
|    value_loss           | 3.12e+06    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -3.71e+04    |
| time/                   |              |
|    fps                  | 155          |
|    iterations           | 52           |
|    time_elapsed         | 685          |
|    total_timesteps      | 106496       |
| train/                  |              |
|    approx_kl            | 0.0037713286 |
|    clip_fraction        | 0.0106       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.2         |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.73e+06     |
|    n_updates            | 510          |
|    policy_gradient_loss | -0.00205     |
|    std                  | 0.889        |
|    value_loss           | 3.39e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -3.89e+04   |
| time/                   |             |
|    fps                  | 155         |
|    iterations           | 53          |
|    time_elapsed         | 699         |
|    total_timesteps      | 108544      |
| train/                  |             |
|    approx_kl            | 0.003621605 |
|    clip_fraction        | 0.00576     |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 2.11e+06    |
|    n_updates            | 520         |
|    policy_gradient_loss | -0.00108    |
|    std                  | 0.889       |
|    value_loss           | 5.23e+06    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -4.07e+04    |
| time/                   |              |
|    fps                  | 155          |
|    iterations           | 54           |
|    time_elapsed         | 710          |
|    total_timesteps      | 110592       |
| train/                  |              |
|    approx_kl            | 0.0037987605 |
|    clip_fraction        | 0.0108       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.21        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.02e+06     |
|    n_updates            | 530          |
|    policy_gradient_loss | -0.00193     |
|    std                  | 0.891        |
|    value_loss           | 4.78e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -4.31e+04    |
| time/                   |              |
|    fps                  | 155          |
|    iterations           | 55           |
|    time_elapsed         | 722          |
|    total_timesteps      | 112640       |
| train/                  |              |
|    approx_kl            | 0.0041659893 |
|    clip_fraction        | 0.00801      |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.21        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.27e+06     |
|    n_updates            | 540          |
|    policy_gradient_loss | -0.00111     |
|    std                  | 0.89         |
|    value_loss           | 5e+06        |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -4.52e+04    |
| time/                   |              |
|    fps                  | 156          |
|    iterations           | 56           |
|    time_elapsed         | 734          |
|    total_timesteps      | 114688       |
| train/                  |              |
|    approx_kl            | 0.0052787326 |
|    clip_fraction        | 0.0296       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.21        |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 3.7e+06      |
|    n_updates            | 550          |
|    policy_gradient_loss | -0.00406     |
|    std                  | 0.889        |
|    value_loss           | 6.79e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -4.67e+04   |
| time/                   |             |
|    fps                  | 156         |
|    iterations           | 57          |
|    time_elapsed         | 746         |
|    total_timesteps      | 116736      |
| train/                  |             |
|    approx_kl            | 0.004433933 |
|    clip_fraction        | 0.0184      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | 5.96e-08    |
|    learning_rate        | 0.0003      |
|    loss                 | 2.93e+06    |
|    n_updates            | 560         |
|    policy_gradient_loss | -0.00222    |
|    std                  | 0.888       |
|    value_loss           | 6.15e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -4.83e+04   |
| time/                   |             |
|    fps                  | 156         |
|    iterations           | 58          |
|    time_elapsed         | 758         |
|    total_timesteps      | 118784      |
| train/                  |             |
|    approx_kl            | 0.004643922 |
|    clip_fraction        | 0.025       |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 2.54e+06    |
|    n_updates            | 570         |
|    policy_gradient_loss | -0.00334    |
|    std                  | 0.888       |
|    value_loss           | 4.8e+06     |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -4.99e+04   |
| time/                   |             |
|    fps                  | 156         |
|    iterations           | 59          |
|    time_elapsed         | 769         |
|    total_timesteps      | 120832      |
| train/                  |             |
|    approx_kl            | 0.004279623 |
|    clip_fraction        | 0.00815     |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 2.58e+06    |
|    n_updates            | 580         |
|    policy_gradient_loss | -0.00133    |
|    std                  | 0.89        |
|    value_loss           | 4.7e+06     |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -5.18e+04   |
| time/                   |             |
|    fps                  | 157         |
|    iterations           | 60          |
|    time_elapsed         | 782         |
|    total_timesteps      | 122880      |
| train/                  |             |
|    approx_kl            | 0.004928913 |
|    clip_fraction        | 0.0252      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 2.06e+06    |
|    n_updates            | 590         |
|    policy_gradient_loss | -0.00314    |
|    std                  | 0.886       |
|    value_loss           | 4.64e+06    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -5.37e+04    |
| time/                   |              |
|    fps                  | 157          |
|    iterations           | 61           |
|    time_elapsed         | 793          |
|    total_timesteps      | 124928       |
| train/                  |              |
|    approx_kl            | 0.0044577485 |
|    clip_fraction        | 0.0167       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.19        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.75e+06     |
|    n_updates            | 600          |
|    policy_gradient_loss | -0.00216     |
|    std                  | 0.884        |
|    value_loss           | 5.37e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -5.54e+04    |
| time/                   |              |
|    fps                  | 157          |
|    iterations           | 62           |
|    time_elapsed         | 805          |
|    total_timesteps      | 126976       |
| train/                  |              |
|    approx_kl            | 0.0033779903 |
|    clip_fraction        | 0.00854      |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.19        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.75e+06     |
|    n_updates            | 610          |
|    policy_gradient_loss | -0.000881    |
|    std                  | 0.887        |
|    value_loss           | 5.11e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -5.84e+04    |
| time/                   |              |
|    fps                  | 157          |
|    iterations           | 63           |
|    time_elapsed         | 817          |
|    total_timesteps      | 129024       |
| train/                  |              |
|    approx_kl            | 0.0036522774 |
|    clip_fraction        | 0.00669      |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.2         |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.36e+06     |
|    n_updates            | 620          |
|    policy_gradient_loss | -0.00121     |
|    std                  | 0.889        |
|    value_loss           | 5.49e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -6.02e+04   |
| time/                   |             |
|    fps                  | 158         |
|    iterations           | 64          |
|    time_elapsed         | 829         |
|    total_timesteps      | 131072      |
| train/                  |             |
|    approx_kl            | 0.005089692 |
|    clip_fraction        | 0.0314      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | -1.19e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 2.64e+06    |
|    n_updates            | 630         |
|    policy_gradient_loss | -0.00405    |
|    std                  | 0.887       |
|    value_loss           | 5.26e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -6.17e+04   |
| time/                   |             |
|    fps                  | 158         |
|    iterations           | 65          |
|    time_elapsed         | 841         |
|    total_timesteps      | 133120      |
| train/                  |             |
|    approx_kl            | 0.004889611 |
|    clip_fraction        | 0.0176      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.2        |
|    explained_variance   | 1.19e-07    |
|    learning_rate        | 0.0003      |
|    loss                 | 1.92e+06    |
|    n_updates            | 640         |
|    policy_gradient_loss | -0.00247    |
|    std                  | 0.888       |
|    value_loss           | 4.65e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -6.28e+04   |
| time/                   |             |
|    fps                  | 158         |
|    iterations           | 66          |
|    time_elapsed         | 853         |
|    total_timesteps      | 135168      |
| train/                  |             |
|    approx_kl            | 0.004658374 |
|    clip_fraction        | 0.021       |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.19       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.67e+06    |
|    n_updates            | 650         |
|    policy_gradient_loss | -0.00222    |
|    std                  | 0.886       |
|    value_loss           | 3.85e+06    |
-----------------------------------------
----------------------------------------
| rollout/                |            |
|    ep_len_mean          | 1e+03      |
|    ep_rew_mean          | -6.38e+04  |
| time/                   |            |
|    fps                  | 158        |
|    iterations           | 67         |
|    time_elapsed         | 866        |
|    total_timesteps      | 137216     |
| train/                  |            |
|    approx_kl            | 0.00569404 |
|    clip_fraction        | 0.0394     |
|    clip_range           | 0.2        |
|    entropy_loss         | -5.19      |
|    explained_variance   | 5.96e-08   |
|    learning_rate        | 0.0003     |
|    loss                 | 1.44e+06   |
|    n_updates            | 660        |
|    policy_gradient_loss | -0.00414   |
|    std                  | 0.887      |
|    value_loss           | 2.78e+06   |
----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -6.47e+04   |
| time/                   |             |
|    fps                  | 157         |
|    iterations           | 68          |
|    time_elapsed         | 887         |
|    total_timesteps      | 139264      |
| train/                  |             |
|    approx_kl            | 0.004979359 |
|    clip_fraction        | 0.0314      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.19       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 7.57e+05    |
|    n_updates            | 670         |
|    policy_gradient_loss | -0.00347    |
|    std                  | 0.883       |
|    value_loss           | 1.69e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -6.6e+04    |
| time/                   |             |
|    fps                  | 155         |
|    iterations           | 69          |
|    time_elapsed         | 911         |
|    total_timesteps      | 141312      |
| train/                  |             |
|    approx_kl            | 0.003934146 |
|    clip_fraction        | 0.0181      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.17       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 8.59e+05    |
|    n_updates            | 680         |
|    policy_gradient_loss | -0.002      |
|    std                  | 0.882       |
|    value_loss           | 1.66e+06    |
-----------------------------------------
----------------------------------------
| rollout/                |            |
|    ep_len_mean          | 1e+03      |
|    ep_rew_mean          | -6.73e+04  |
| time/                   |            |
|    fps                  | 154        |
|    iterations           | 70         |
|    time_elapsed         | 929        |
|    total_timesteps      | 143360     |
| train/                  |            |
|    approx_kl            | 0.00488944 |
|    clip_fraction        | 0.0386     |
|    clip_range           | 0.2        |
|    entropy_loss         | -5.17      |
|    explained_variance   | 0          |
|    learning_rate        | 0.0003     |
|    loss                 | 1.07e+06   |
|    n_updates            | 690        |
|    policy_gradient_loss | -0.00419   |
|    std                  | 0.879      |
|    value_loss           | 2.26e+06   |
----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -6.88e+04    |
| time/                   |              |
|    fps                  | 151          |
|    iterations           | 71           |
|    time_elapsed         | 956          |
|    total_timesteps      | 145408       |
| train/                  |              |
|    approx_kl            | 0.0039507896 |
|    clip_fraction        | 0.026        |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.16        |
|    explained_variance   | -2.38e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 1.23e+06     |
|    n_updates            | 700          |
|    policy_gradient_loss | -0.00263     |
|    std                  | 0.879        |
|    value_loss           | 2.33e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -7e+04       |
| time/                   |              |
|    fps                  | 151          |
|    iterations           | 72           |
|    time_elapsed         | 974          |
|    total_timesteps      | 147456       |
| train/                  |              |
|    approx_kl            | 0.0048819017 |
|    clip_fraction        | 0.0321       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.16        |
|    explained_variance   | -1.19e-07    |
|    learning_rate        | 0.0003       |
|    loss                 | 1.66e+06     |
|    n_updates            | 710          |
|    policy_gradient_loss | -0.00324     |
|    std                  | 0.877        |
|    value_loss           | 3.4e+06      |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -7.08e+04    |
| time/                   |              |
|    fps                  | 150          |
|    iterations           | 73           |
|    time_elapsed         | 994          |
|    total_timesteps      | 149504       |
| train/                  |              |
|    approx_kl            | 0.0051534167 |
|    clip_fraction        | 0.0276       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.15        |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.15e+06     |
|    n_updates            | 720          |
|    policy_gradient_loss | -0.0034      |
|    std                  | 0.877        |
|    value_loss           | 2.73e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -7.17e+04    |
| time/                   |              |
|    fps                  | 150          |
|    iterations           | 74           |
|    time_elapsed         | 1009         |
|    total_timesteps      | 151552       |
| train/                  |              |
|    approx_kl            | 0.0039522136 |
|    clip_fraction        | 0.0215       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.17        |
|    explained_variance   | 5.96e-08     |
|    learning_rate        | 0.0003       |
|    loss                 | 8.39e+05     |
|    n_updates            | 730          |
|    policy_gradient_loss | -0.00278     |
|    std                  | 0.885        |
|    value_loss           | 1.82e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -7.25e+04    |
| time/                   |              |
|    fps                  | 150          |
|    iterations           | 75           |
|    time_elapsed         | 1023         |
|    total_timesteps      | 153600       |
| train/                  |              |
|    approx_kl            | 0.0037896927 |
|    clip_fraction        | 0.0131       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.18        |
|    explained_variance   | 1.19e-07     |
|    learning_rate        | 0.0003       |
|    loss                 | 1.18e+06     |
|    n_updates            | 740          |
|    policy_gradient_loss | -0.00183     |
|    std                  | 0.883        |
|    value_loss           | 2.15e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -7.33e+04   |
| time/                   |             |
|    fps                  | 149         |
|    iterations           | 76          |
|    time_elapsed         | 1038        |
|    total_timesteps      | 155648      |
| train/                  |             |
|    approx_kl            | 0.005035511 |
|    clip_fraction        | 0.0261      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.16       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 6.91e+05    |
|    n_updates            | 750         |
|    policy_gradient_loss | -0.00347    |
|    std                  | 0.877       |
|    value_loss           | 1.49e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -7.45e+04   |
| time/                   |             |
|    fps                  | 149         |
|    iterations           | 77          |
|    time_elapsed         | 1052        |
|    total_timesteps      | 157696      |
| train/                  |             |
|    approx_kl            | 0.005323178 |
|    clip_fraction        | 0.0373      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.15       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 8.95e+05    |
|    n_updates            | 760         |
|    policy_gradient_loss | -0.00419    |
|    std                  | 0.876       |
|    value_loss           | 1.9e+06     |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -7.61e+04   |
| time/                   |             |
|    fps                  | 149         |
|    iterations           | 78          |
|    time_elapsed         | 1071        |
|    total_timesteps      | 159744      |
| train/                  |             |
|    approx_kl            | 0.005339088 |
|    clip_fraction        | 0.0279      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.13       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.56e+06    |
|    n_updates            | 770         |
|    policy_gradient_loss | -0.00348    |
|    std                  | 0.871       |
|    value_loss           | 3.12e+06    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -7.72e+04    |
| time/                   |              |
|    fps                  | 148          |
|    iterations           | 79           |
|    time_elapsed         | 1088         |
|    total_timesteps      | 161792       |
| train/                  |              |
|    approx_kl            | 0.0021434259 |
|    clip_fraction        | 0.00132      |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.13        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.39e+06     |
|    n_updates            | 780          |
|    policy_gradient_loss | -0.000295    |
|    std                  | 0.874        |
|    value_loss           | 3.14e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -7.88e+04   |
| time/                   |             |
|    fps                  | 148         |
|    iterations           | 80          |
|    time_elapsed         | 1106        |
|    total_timesteps      | 163840      |
| train/                  |             |
|    approx_kl            | 0.004515986 |
|    clip_fraction        | 0.0185      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.13       |
|    explained_variance   | -2.38e-07   |
|    learning_rate        | 0.0003      |
|    loss                 | 1.26e+06    |
|    n_updates            | 790         |
|    policy_gradient_loss | -0.00181    |
|    std                  | 0.872       |
|    value_loss           | 2.6e+06     |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -8.02e+04   |
| time/                   |             |
|    fps                  | 147         |
|    iterations           | 81          |
|    time_elapsed         | 1125        |
|    total_timesteps      | 165888      |
| train/                  |             |
|    approx_kl            | 0.004079287 |
|    clip_fraction        | 0.0139      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.13       |
|    explained_variance   | 5.96e-08    |
|    learning_rate        | 0.0003      |
|    loss                 | 1.86e+06    |
|    n_updates            | 800         |
|    policy_gradient_loss | -0.00189    |
|    std                  | 0.873       |
|    value_loss           | 3.26e+06    |
-----------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -8.2e+04    |
| time/                   |             |
|    fps                  | 146         |
|    iterations           | 82          |
|    time_elapsed         | 1147        |
|    total_timesteps      | 167936      |
| train/                  |             |
|    approx_kl            | 0.004386483 |
|    clip_fraction        | 0.0127      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.13       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 1.87e+06    |
|    n_updates            | 810         |
|    policy_gradient_loss | -0.00205    |
|    std                  | 0.873       |
|    value_loss           | 3.56e+06    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -8.37e+04    |
| time/                   |              |
|    fps                  | 145          |
|    iterations           | 83           |
|    time_elapsed         | 1167         |
|    total_timesteps      | 169984       |
| train/                  |              |
|    approx_kl            | 0.0041227336 |
|    clip_fraction        | 0.0245       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.14        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.95e+06     |
|    n_updates            | 820          |
|    policy_gradient_loss | -0.00324     |
|    std                  | 0.873        |
|    value_loss           | 3.58e+06     |
------------------------------------------
----------------------------------------
| rollout/                |            |
|    ep_len_mean          | 1e+03      |
|    ep_rew_mean          | -8.62e+04  |
| time/                   |            |
|    fps                  | 145        |
|    iterations           | 84         |
|    time_elapsed         | 1180       |
|    total_timesteps      | 172032     |
| train/                  |            |
|    approx_kl            | 0.00430945 |
|    clip_fraction        | 0.0171     |
|    clip_range           | 0.2        |
|    entropy_loss         | -5.13      |
|    explained_variance   | 0          |
|    learning_rate        | 0.0003     |
|    loss                 | 2.18e+06   |
|    n_updates            | 830        |
|    policy_gradient_loss | -0.0021    |
|    std                  | 0.872      |
|    value_loss           | 3.97e+06   |
----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -8.81e+04    |
| time/                   |              |
|    fps                  | 145          |
|    iterations           | 85           |
|    time_elapsed         | 1198         |
|    total_timesteps      | 174080       |
| train/                  |              |
|    approx_kl            | 0.0027071913 |
|    clip_fraction        | 0.0043       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.14        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.6e+06      |
|    n_updates            | 840          |
|    policy_gradient_loss | -0.000301    |
|    std                  | 0.876        |
|    value_loss           | 3.94e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -8.97e+04   |
| time/                   |             |
|    fps                  | 144         |
|    iterations           | 86          |
|    time_elapsed         | 1218        |
|    total_timesteps      | 176128      |
| train/                  |             |
|    approx_kl            | 0.003243664 |
|    clip_fraction        | 0.0133      |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.15       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 2.15e+06    |
|    n_updates            | 850         |
|    policy_gradient_loss | -0.00204    |
|    std                  | 0.878       |
|    value_loss           | 4.5e+06     |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -9.15e+04    |
| time/                   |              |
|    fps                  | 143          |
|    iterations           | 87           |
|    time_elapsed         | 1239         |
|    total_timesteps      | 178176       |
| train/                  |              |
|    approx_kl            | 0.0029459428 |
|    clip_fraction        | 0.00308      |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.16        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.44e+06     |
|    n_updates            | 860          |
|    policy_gradient_loss | -0.000751    |
|    std                  | 0.88         |
|    value_loss           | 4.24e+06     |
------------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -9.36e+04    |
| time/                   |              |
|    fps                  | 143          |
|    iterations           | 88           |
|    time_elapsed         | 1258         |
|    total_timesteps      | 180224       |
| train/                  |              |
|    approx_kl            | 0.0043680565 |
|    clip_fraction        | 0.0208       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.17        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 2.09e+06     |
|    n_updates            | 870          |
|    policy_gradient_loss | -0.00237     |
|    std                  | 0.881        |
|    value_loss           | 4.39e+06     |
------------------------------------------
-----------------------------------------
| rollout/                |             |
|    ep_len_mean          | 1e+03       |
|    ep_rew_mean          | -9.52e+04   |
| time/                   |             |
|    fps                  | 142         |
|    iterations           | 89          |
|    time_elapsed         | 1283        |
|    total_timesteps      | 182272      |
| train/                  |             |
|    approx_kl            | 0.004063189 |
|    clip_fraction        | 0.012       |
|    clip_range           | 0.2         |
|    entropy_loss         | -5.16       |
|    explained_variance   | 0           |
|    learning_rate        | 0.0003      |
|    loss                 | 2.77e+06    |
|    n_updates            | 880         |
|    policy_gradient_loss | -0.00207    |
|    std                  | 0.877       |
|    value_loss           | 4.73e+06    |
-----------------------------------------
------------------------------------------
| rollout/                |              |
|    ep_len_mean          | 1e+03        |
|    ep_rew_mean          | -9.69e+04    |
| time/                   |              |
|    fps                  | 141          |
|    iterations           | 90           |
|    time_elapsed         | 1306         |
|    total_timesteps      | 184320       |
| train/                  |              |
|    approx_kl            | 0.0049707354 |
|    clip_fraction        | 0.0279       |
|    clip_range           | 0.2          |
|    entropy_loss         | -5.13        |
|    explained_variance   | 0            |
|    learning_rate        | 0.0003       |
|    loss                 | 1.65e+06     |
|    n_updates            | 890          |
|    policy_gradient_loss | -0.00372     |
|    std                  | 0.871        |
|    value_loss           | 3.68e+06     |
------------------------------------------