How do I make other users able to see my running Spark job?

When you start your Spark job in RStudio, use this bit of code:

conf <- spark_config()
conf$spark.ui.view.acls.groups <- "p002"
sc <- spark_connect(master = 'yarn-client', config = conf)

This gives all users in the “p002”-group access to see what you’re doing in your current Spark session.