File rights in HDFS

If you write files with t. HDFS the files will, by default, not be editable for other projects. However, problems arise if multiple people are using a notebook; for instance to write parquet files. By applying the following command (in a terminal on the Project Machine e.g. SSH or the terminal window in RStudio) you are able to change the rights to the file:

 hdfs dfs -chmod -R g+w <file path>

Where ‘file path’ could be the root of the project e.g. “/projects/p003/”. The -R argument ensures that the rights are changed in all subfolders. If this is not desired then a specific folder should be inserted in ‘file path’.