Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

README.md

Spark-on-SLURM prerequisites

1. Download spark binaries

mkdir -p /fsx/bigcode/spark/
chmod 777 /fsx/bigcode/spark/
cd /fsx/bigcode/spark/
wget https://dlcdn.apache.org/spark/spark-3.5.0/spark-3.5.0-bin-hadoop3.tgz
tar -xvzf spark-3.5.0-bin-hadoop3.tgz