Spark on Yarn within Docker Containers


The open source project “spark-on-yarn”

GitHub:

https://github.com/madaibaba/spark-on-yarn

Docker Hub:

https://hub.docker.com/r/madaibaba/spark-on-yarn

Bigmap of “hadoop-on-docker” archtect

How to use it:

1. Clone Github Repository

git clone https://github.com/madaibaba/spark-on-yarn

2. Pull Docker Image

sudo docker pull madaibaba/spark-on-yarn:1.0

3. Start Docker Container

3.1 Start Three Container for default (one master and two slaves)

cd spark-on-yarn

sudo ./start-container.sh

3.2 Start six Container as below (one master and five slaves)

cd hadoop-on-docker

sudo ./start-container.sh 6

Another open source project “hadoop-on-docker” without spark install, as below:

GitHub:

https://github.com/madaibaba/hadoop-on-docker

Docker Hub:

https://hub.docker.com/r/madaibaba/haoop-on-docker

, , ,
Trackback

no comment untill now

Add your comment now

切换到手机版