Once you collect some new data on the Stick, you need to process it into a dataset before you can train policies on it. This step will help you get started on that.
Last updated
On your machine, in a new conda/virtual environment
For extracting a single environment:
Compress video taken from the Record3D app:
Get the files on your machine.
Using Google drive:
[Only once] Generate Google Service Account API key to download from private folders on Google Drive. There are some instructions on how to do so in this Stackoverflow link
[Only once] Rename the .json file to client_secret.json and put it in the same directory as gdrive_downloader.py
Upload .zip file into its own folder on Google Drive, and copy folder_id from URL to put it in the GDRIVE_FOLDER_ID in the ./do-all.sh file.
Manually:
Comment out the GDRIVE_FOLDER_ID line from ./do-all.sh and create the following hierarchy locally
The .zip files should contain .r3d files exported from the Record3D app in the previous step.
Modify required variables in do-all.sh.
TASK_NO task id, see gdrive_downloader.py for more information.
HOME name or ID of the home.
ROOT_FOLDER folder where the data is stored after downloading.
EXPORT_FOLDER folder where the dataset is stored after processing. Should be different from ROOT_FOLDER.
ENV_NO current environment number in the same home and task set.
GRIPPER_MODEL_PATH path to the gripper model. It should be in the github repo already, and can be downloaded from .
Change current working directory to local repository root folder and run
Split the extracted data to include a validation set for each environment. The data should follow the following hierarchy: (Be sure change the corresponding paths in r3d_files.txt to include “_val”)\
