Deleting tubs in mycar/data

Hi All,
I have asked this question before, but I never received a satisfactory answer as to how to delete unwanted tub files in the mycar/data directory.
When I arbitrarily deleted all of the tub files in the mycar/data directory, manage.py drive would not start properly.
When I used donkey tubclean mycar/data, I was able to eliminate all of the frame and jason data records except for meta.jason in each tub file, however the tub files, even though they are empty except for meta.jason, are still visible in the mycar/data directory.
Therefore to remove the existing empty tub files, how should I proceed without compromising the manage.py drive program?
Should I use rm -rf or what?
Regards,
TCIII

just delete the unwanted tub_xxxx directories. If manage.py won’t run then you’re doing the deleting incorrectly. And delete the directories before running/starting manage.py.

I use the command rm -rf to remove unwanted tubs and it works without any issues.
(Just be careful to delete only the ones you don’t want)

I found that I had to delete all of the records and the meta.json file in a tub directory before deleting the tub directory. If I just used rm -rf to delete the tub directory before deleting the records and the meta.json file "manage.py drive would not run correctly.

Regards,
TCIII

Then you’re doing something wrong. look up what rm -rf does, it just deletes the files in the directory and the directory itself. I’ve used it dozens of times if not more in the past 2 years of doing donkeycar.

You just have to NOT be running any donkey car stuff, delete the directories(tubs) and then start your donkey car software.

@DougL,

There was no donkey car app running at the time I deleted the tubs using rm -rf which resulted in the manage.py drive program to not run correctly as it could not create a new tub.

As above, I first deleted all of the records and the meta.json file in the tub and then when it was empty, deleted the tub directory.

I am using donkey car 3.1.1 where the tubs are stored in the data directory. The older donkey car 2.5.1 does not have a data directory where the tubs are stored.

Regards,
TCIII

The order in which you delete files and directores does not matter if in the end the same files and didrectories are gone when donkey is started.

And if in version 3.1.1, where the json data is all in one file and you only want to delete some training data then you have to ONLY delete the corresponding json data from the meta file so they match what image data you removed. Or run a tools or script which results in the images having matching meta data.

@DougL,

The json steering and throttle data is not stored in the meta.json file in my 3.1.1 tubs. They are stored as individual files just like the video jpg snapshots are stored as individual files.

Regards,
TCIII

As stated previously, standard operating proceedures for the last 2 years has been to delete the tub directory(s) you don’t want and move on.

I just took a machine, loaded donkey 3.1.1, collected data, deleted the tub and was able to collect data without any problems.

234 git clone http://github.com/autorope/donkey donkeycar311
235 cd donkeycar311/
236 pip3 install -e .[pi]
237 pip install --upgrade pip
238 pip3 install --upgrade pip
239 donkey createcar --path ~/d3_11
240 cd ~/d3_11/
241 diff …/d3_1/myconfig.py myconfig.py
242 vi …/d3_1/myconfig.py myconfig.py
243 diff …/d3_1/myconfig.py myconfig.py
244 python ./manage.py drive
245 sudo halt
246 cd d3_11/
247 python ./manage.py drive
248 ls
249 cd data/
250 ls
251 ll
252 ls
253 cd tub_1_20-02-24/
254 ls
255 vi meta.json
256 cd …
257 rm -rf tub_1_20-02-24/
258 cd …
259 python ./manage.py drive
260 history

OOPS, I forgot to change to the master branch so I did that and reran the test. I got the same results. tubs have two file data points and deleting the tub and restarting the drive command worked perfectly.

BTW, there is a version of donkey where the tub json files are no longer separate but instead the data is written to one JSON file. Supposedly it improves writing(recording) performance. The standard setup with tub directories containing both jpg and json files for each data point has been used for a couple years.