Standard RL environ for physical donkeycar?

Does there exist a physical env object for the donkeycar as is used in most RL experiments? Something that supports code like:
obs, rewards, dones, info = env.step(action) where obs would be the (camera image, throttle) and action would be (steering, throttle)
Tawn’s sim code has the usual RL framework like above but I can only find an environment for the simulator, not a connection to the real hardware.

I’m new here , so sorry if I missed the Doc somewhere

Did you look into any branches( he names them based on the purpose ) in autorope repo or his personal repo?

I know he tried RL on a physical vehicle but found the human activation of the rewarding was too slow so proper rewarding wasn’t possible.

Do you have some way planned or implemented which will trigger the rewarding system?

Yes, the Car definitely needs some automatic way to get rewards and do retries.

I’ve added three sensors to the Car : “training wheels” IR sensors at the two front corners to detect the white lane boundaries, a sonar in the front to detect obstacles and a wheel odometer (also IR) to get distance/speed. I will add a routine to the Car to stop when it reaches a lane line and end the episode with a negative reward. Then, before restarting the training, the Car will back up slightly, turn its wheels sharply toward the center, and move forward slightly. Then I’ll restart the training.Hopefully, that should enable automated training. I plan to use the SAC RL algorithm which should train fairly quickly based on some JetBot experiments.

I built the sensors using ATTiny Arduino’s which will be connected to the second I2C bus. I;ll add them as parts and or write a blog post if anyone is interested.

2 Likes

I have not messed with it but there should be some RL code in the repo(s).

Humm… Thanks for the suggestion. I found gym_real.py in donkey which connects the sim code to the real car via MQTT(?) I’ll DM Tawn and ask why he took that roundabout path.

@mercury,

Very interested.

Have you seen this Lidar for the Donkey Car?

Regards,
TCIII