Steps

Create Spark with Jupyter Application on BlueData App Workbench

Step 1 - Preparing the Environment

We have already installed BlueData Epic Work Bench, for more information, please click on this link.here

For checking the version of BlueData App-Workbench, execute the following command.

bdwb --version

Step 2 - Getting Started.

To begin creating the Spark Application Image follow the steps mentioned below:
Create a directory on the system named as Spark.
This directory will house all the files and components for the Spark Application Image.To do this, execute the following command:

mkdir ~/Spark

Change to your new source directory:

cd ~/Spark

Use the bdwb command to create a skeleton file structure with the appropriate directories for creating the Application .bin file:

bdwb --init

Click on the below command to view the files and directories created in Spark folder.

ls -R

Create a folder named Centos and a sub folder spark inside the image directory. To do that, execute the following command:

mkdir ~/Spark/image/centos

Step 3 - Create the Dockerfile.

Let's create Dockerfile now..

Create a Dockerfile inside centos folder. For reference, we already have a Dockerfile in ~/test directory.
We can just copy the reference Dockerfile to Centos folder.cp ~/test/Dockerfile ~/Spark/image/centos

To view the content of Dockerfile please execute the below command.cat ~/Spark/image/centos/Dockerfile

Step 4 - Creating few more file which are used to run Spark services.

In this step we would be creating some more files that are required for Spark Application Image.
We already have reference files, we can just copy them to appropriate location.

Debugging Scenarios

Help

Katacoda offerings an Interactive Learning Environment for Developers. This course uses a command line and a pre-configured sandboxed environment for you to use. Below are useful commands when working with the environment.

cd <directory>

Change directory

ls

List directory

echo 'contents' > <file>

Write contents to a file

cat <file>

Output contents of file

Vim

In the case of certain exercises you will be required to edit files or text. The best approach is with Vim. Vim has two different modes, one for entering commands (Command Mode) and the other for entering text (Insert Mode). You need to switch between these two modes based on what you want to do. The basic commands are: