Skip to content

PadoGrid on OCI Compute

Jump to bottom
javapark1 edited this page Oct 26, 2023 · 7 revisions

◀️ Platform Kubernetes 🔗 Pado CLASSPATH ▶️ 🔗 Geode CLASSPATH ▶️ 🔗 Hazelcast CLASSPATH ▶️ 🔗 SnappyData CLASSPATH ▶️ 🔗 Coherence CLASSPATH ▶️ 🔗 Hadoop CLASSPATH ▶️ 🔗 Kafka CLASSPATH ▶️ 🔗 Mosquitto CLASSPATH ▶️ 🔗 Redis CLASSPATH ▶️ 🔗 Spark CLASSPATH ▶️

OCI Cloud Free Tier provides the Always Free cloud services that include up to 4 ARM Ampere A1 Compute VM.Standard.A1.Flex Shape instances and up to a total of 24 GB of memory. This is enough to run JupyterLab and other PadoGrid components for creating a development environment.

This chapter describes how to setup PadoGrid in OCI Cloud Free Tier.

1. Provision Compute instances

The following table shows some of provisioning options.

Option Shape OCPU Memory Boot Volume Volume Count
A ARM Ampere A1 Compute VM.Standard.A1.Flex 1 6GB 50GB N/A 4
B ARM Ampere A1 Compute VM.Standard.A1.Flex 2 12GB 50GB 50GB 2
C ARM Ampere A1 Compute VM.Standard.A1.Flex 2 12GB 100GB N/A 2
D ARM Ampere A1 Compute VM.Standard.A1.Flex 4 24GB 100GB 100GB 1
F ARM Ampere A1 Compute VM.Standard.A1.Flex 4 24GB 200GB N/A 1

We will use Option A in our example.

2. Collect host names of VMs

After VMs are provisioned (Option A), collect the host names (or private IPs) and public IPs of all four (4) VMs. We will use the following list for our example.

Host Name (Private) Public IPs
p-1 150.136.180.153
p-2 150.214.165.122
p-3 150.215.160.200
p-4 132.143.213.132

3. SSH into one of the VMs.

# Copy your ssh private key to one of the VMs
scp -i private.key private.key opc@150.136.180.153:

# Login to the VM
ssh -i private.key opc@150.136.180.153

4. Install Required Software

sudo yum install -y git maven java-1.8.0-openjdk-devel

# Optional: Store git credential for private repos (timeout default: 900 or 15 min)
git config --global credential.helper cache "cache --timeout=30000"

5. Install PadoGrid

# Specify the product to install along with PadoGrid
PRODUCT=hazelcast-oss
#PRODUCT=hazelcast-enterprise
#PRODUCT=geode
curl -fsSL https://raw.githubusercontent.com/padogrid/padogrid/develop/padogrid-deployment/src/main/resources/common/bin_sh/install_padogrid | /bin/bash -s -- -no-stty -quiet -product $PRODUCT

# Initialize PadoGrid
.  ~/Padogrid/workspaces/myrwe/initenv.sh -quiet

# Create VM workspace 
create_workspace -name ws-oci \
-vm p1,p2,p3,p4 \
-vm-public 150.136.180.153,150.214.165.122,150.215.160.200,132.143.213.132 \
-name ws-oci -vm-java /usr/lib/jvm/java-1.8.0 \
-vm-key ~/private.key \
-vm-java /usr/lib/jvm/java-1.8.0 \
-quiet

# Switch to the newly created workspace
switch_workspace ws-oci

# Disable firewall in all VMs
vm_exec sudo systemctl stop firewalld

# Install PadoGrid and required software in all VMs
vm_sync
vm_exec sudo yum install -y git maven java-1.8.0-openjdk-devel

# Check status of all VMs
vm_test

6. Install JupyterLab

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh
bash Miniconda3-latest-Linux-aarch64.sh -b -p $(dirname $PADOGRID_HOME)/miniconda3 && rm -f Miniconda3-latest-Linux-aarch64.sh
$(dirname $PADOGRID_HOME)/miniconda3/bin/conda init
. ~/.bashrc
conda install -y pip
conda update -y --all
pip install jupyterlab

7. Initialize JupyterLab

#Create a self-signed certificate to be used for SSL.
switch_workspace ws-oci
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout .mykey.key -out .mycert.pem
chmod 400 .mycert.pem

# Set password
jupyter server password

# Generate and update a Jupyter server config file.
jupyter server --generate-config

cat <<EOF >>  ~/.jupyter/jupyter_server_config.py
c = get_config()
c.ServerApp.certfile = u'/home/opc/Padogrid/workspaces/myrwe/ws-oci/.mycert.pem'
c.ServerApp.keyfile = u'/home/opc/Padogrid/workspaces/myrwe/ws-oci/.mykey.key'
c.ServerApp.ip = '*'
EOF

# Override markdown editor with preview
SETTINGS_DIR="$(jupyter lab path | grep "Application" | sed 's/^.*://' | xargs)/settings"
OVERRIDES_FILE="$SETTINGS_DIR/overrides.json"
if [ ! -d "$SETTINGS_DIR" ]; then
   mkdir $SETTINGS_DIR
fi
if [ ! -f "$OVERRIDES_FILE" ]; then
cat <<EOF > "$OVERRIDES_FILE"
{
  "@jupyterlab/docmanager-extension:plugin": {
     "defaultViewers": {
       "markdown": "Markdown Preview"
     }
   }
}
EOF
fi

8. Start JupyterLab

switch_workspace ws-oci
start_jupyter -default -dashboard

Use the public IP address of the ARM Ampere A1 instance to login from your browser. For our example, the public IP address is the IP address we used to ssh into the OCI instance, i.e., 150.136.180.153. Both of the following open the same workspace dashboard.

Add port 8888 to the public zone in the firewall.

sudo  firewall-cmd --zone=public --add-port=8888/tcp

References

  1. Oracle® Linux 8 Configuring the Firewall, https://docs.oracle.com/cd/F22978_01/8/firewall/F20786.pdf

◀️ Platform Kubernetes 🔗 Pado CLASSPATH ▶️ 🔗 Geode CLASSPATH ▶️ 🔗 Hazelcast CLASSPATH ▶️ 🔗 SnappyData CLASSPATH ▶️ 🔗 Coherence CLASSPATH ▶️ 🔗 Hadoop CLASSPATH ▶️ 🔗 Kafka CLASSPATH ▶️ 🔗 Mosquitto CLASSPATH ▶️ 🔗 Redis CLASSPATH ▶️ 🔗 Spark CLASSPATH ▶️

PadoGrid Manual

Overview

Operations

Tools

Platforms

Clouds

Pado

Geode/GemFire

Hazelcast/Jet

ComputeDB/SnappyData

Coherence

Hadoop

Kafka/Confluent

Mosquitto

Redis

Spark

Clone this wiki locally