Skip to content

Geode Workspaces on VMs

Jump to bottom
padogrid edited this page Jun 10, 2022 · 19 revisions

◀️ Geode WAN Example 🔗 Geode-on-AWS-EC2 ▶️

Geode/GemFire Workspaces on VMs

PadoGrid provides a simple way to manage Geode/GemFire clusters on laptop, PadoGrid pods, and VMs. This article describes how to create and run a Geode/GemFire cluster on multiple VMs and/or PMs (Physical Machines). The instructions provided apply to any VMs and PMs including AWS, Azure, GCP, physical machines, VirtualBox, vSphere, PadoGrid pods (VMs), etc.

With PadoGrid, you can have a complete control over Geode/GemFire running on VMs from your local machine. PadoGrid preserves the state of VM configurations and deployments on your local machine so that you can conveniently activate or deactivate the VM environment at any time. After you are done running VMs, you can tear them down and reinstate the very same environment later with a single command. This saves you time and reduces VM costs when you are working in a cloud environment, for example.

Installation Steps

There are four (4) main steps to installing PadoGrid on VMs. Once these steps are performed, you can then synchronize your local machine with the remote VMs by simply executing the vm_sync command, which enables the local control of VMs.

  1. Setup password-less SSH login
  2. Download required software
  3. Create VM workspace
  4. Synchronize VM Workspace

Password-less SSH Login

The first step is to check to make sure you are able to login to the VMs that you want to cluster without the password. For AWS EC2, for example, this is already done for you so you can skip this section.

If you need to manually setup password-less SSH login, then follow the instructions here.

❗ Make sure you can ssh from any of the VMs including your local machine without the password.

Download Software

PadoGrid requires the following software products. If they are not installed on VMs, then download their tarball or zip distributions.

For our demo, we'll assume they are downloaded in the following directory. Note that they must be tarball or zip distributions.

/home/dpark/Downloads
├── jdk1.8.0_212.tar.gz
└── apache-geode-1.12.0.tar.tgz

Create VM Workspace

Create a VM workspace on your local machine, e.g., your laptop.

create_workspace -vm -name ws-vm

The above command interactively prompts for software installation paths information. The -vm option enables VMs and -name option names the workspace. The following shows my laptop environment. The environment variables that begin with VM_ are specific to the remote VMs. All other environment variables without the prefix VM_ pertain to your laptop environment. For example, VM_PADOGRID_HOME is the PadoGrid installation path in the VM hosts you specified.

Please answer the prompts that appear below. If you are not able to complete
the prompts at this time then use the '-quiet' option to bypass the prompts.
You can complete the requested values later in the generated 'setenv.sh' file
You can abort this command at any time by entering 'Ctrl-C'.

Enter Java home path.
[/home/dpark/Padogrid/products/jdk1.8.0_212]:

Enter the local product home directory path. Choose one
from the defaults listed below or enter another.
   /home/dpark/Work/linux/hazelcast-enterprise-3.12.6
   /home/dpark/Work/linux/hazelcast-jet-enterprise-3.2.2
   /home/dpark/Padogrid/products/pivotal-gemfire-9.10.0
[]:
/home/dpark/Padogrid/products/apache-geode-1.12.0
Enter workspace name.
[ws-vm]:
ws-vm
Enter default cluster name.
[mygeode]:

Enable VM? Enter 'true' or 'false' [true]: 
Enter VM JDK home path.
[/home/dpark/Padogrid/products/jdk1.8.0_212]:
/home/dpark/Geode/jdk1.8.0_212
Enter VM product home path.
[/home/dpark/Padogrid/products/apache-geode-1.12.0]:
/home/dpark/Geode/apache-geode-1.12.0
Enter VM padogrid home path.
[/home/dpark/Padogrid/products/padogrid_0.9.7]:
/home/dpark/Geode/padogrid_0.9.7
Enter VM workspaces path.
[/home/dpark/Padogrid/workspaces/myrwe]:
/home/dpark/Geode/workspaces/myrwe
Enter VM host names or IP addresses separated by comma.
[]:
ubuntu1,ubuntu2,ubuntu3,ubuntu4
Enter VM user name.
[]: dpark
Enter VM SSH private key path. If you don't have the key file (.pem) then
leave it blank for now. You can place the file in the workspace directory or
set the path in the workspace 'setenv.sh' file later.
[]:


You have entered the following.
                     JAVA_HOME: /home/dpark/Padogrid/products/jdk1.8.0_212
                  PRODUCT_HOME: /home/dpark/Padogrid/products/apache-geode-1.12.0
            PADOGRID_WORKSPACE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm
               Default Cluster: mygeode
                    VM_ENABLED: true
                  VM_JAVA_HOME: /home/dpark/Geode/jdk1.8.0_212
               VM_PRODUCT_HOME: /home/dpark/Geode/apache-geode-1.12.0
              VM_PADOGRID_HOME: /home/dpark/Geode/padogrid_0.9.7
   VM_PADOGRID_WORKSPACES_HOME: /home/dpark/Geode/workspaces/myrwe
         VM_PADOGRID_WORKSPACE: /home/dpark/Geode/workspaces/myrwe/ws-vm
                      VM_HOSTS: ubuntu1,ubuntu2,ubuntu3,ubuntu4
                       VM_USER: dpark
           VM_PRIVATE_KEY_FILE:
Enter 'c' to continue, 'r' to re-enter, 'q' to quit: c
----------------------------------------------------------------
                  WORKSPACE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm
            Cluster Created: mygeode
               CLUSTER_TYPE: geode
                 Deployment: VM
                CLUSTER_DIR: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode
                 CONFIG_DIR: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/etc
                CONFIG_FILE: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/etc/cache.xml
                    LOG_DIR: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/log
         LOCATOR_START_PORT: 10334
          MEMBER_START_PORT: 40404
     MEMBER_HTTP_START_PORT: 7080
           DEBUG_START_PORT: 9101
             JMX_START_PORT: 12001
JMX_MANAGER_HTTP_START_PORT: 7070
     JMX_MANAGER_START_PORT: 9051
      PROMETHEUS_START_PORT: 8091

The default config.xml file has been placed in the CONFIG_DIR directory.
Modify it or replace it with the appropriate file.

One (1) member has been configured (added) to run in this cluster as follows:

      Member: mygeode-member-padogrid-01
 Working Dir: /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/run/mygeode-member-padogrid-01

1. Make sure to replace or update cache.xml:
      /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/etc/cache.xml
2. Place your application jar files in one of the 'plugins' directories:
      /home/dpark/Padogrid/workspaces/myrwe/ws-vm/plugins/
      /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/plugins/
3. Configure cluster properties in:
      /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode/etc/cluster.properties
4. You must configure password-less ssh login to all VMs. On AWS, this
   is already configured but you will need to enter the correct private key
   file path in the cluster.properties file.
5. To change VM host names, list the VM host names in the cluster.properties file
   for locators and members.
      vm.locator.hosts=host1,host2,...
      vm.hosts=host1,host2,...
6. Deploy the cluster configuration changes you made to all the VM hosts listed in the
   previous step by running 'vm_copy' or 'vm_sync'.
      vm_copy /home/dpark/Padogrid/workspaces/myrwe/ws-vm/clusters/mygeode
7. To start the cluster:
      start_cluster -cluster mygeode
8. To verify the cluster state:
      show_cluster -cluster mygeode
9. To view log file:
      show_log -cluster mygeode
----------------------------------------------------------------

Workspace created:
   /home/dpark/Padogrid/workspaces/myrwe/ws-vm

The workspace directory also contains the following files.

   initenv.sh -  Workspace initialization file. Source this file in to
                 enter the workspace.

   setenv.sh -  You can set your workspace-wide environment variables in
                this file.

   .addonenv.sh - A hidden file containing the workspace environment variables.
                  You can overwrite them in setenv.sh as needed.

IMPORTANT:
----------
To use this workspace, execute the 'switch_workspace' command as follows:
   switch_workspace ws-vm

You have created a VM workspace. To add or remove VM hosts, set VM_HOSTS in
the workspace 'setenv.sh' file. Note that VM_HOSTS applies to the entire workspace
whereas the 'vm.hosts' property in the cluster 'etc/cluster.properties' file applies
only to the defining cluster. By default, 'vm.hosts' is initially set to VM_HOSTS.
   /home/dpark/Padogrid/workspaces/myrwe/ws-vm/setenv.sh

The above example shows that the installation paths in the VMs are different from the local installation paths. We choose to install all the software components in the /home/dpark/Geode for the VMs.

If you specify the private key file, i.e., ecs.pem in our example, then it is automatically copied to the newly created workspace directory. It is also later deployed to the VMs when you execute the vm_sync command.

You can also run the create_workspace in the non-interactive mode by specifying the -quiet option. The following command produces the same workspace configurations as the above interactive example.

create_workspace -quiet \
-name ws-vm \
-cluster mygeode \
-java /home/dpark/Work/linux/jdk1.8.0_212 \
-product /home/dpark/Padogrid/products/apache-geode-1.12.0 \
-vm ubuntu1,ubuntu2,ubuntu3,ubuntu4 \
-vm-java /home/dpark/Geode/jdk1.8.0_212 \
-vm-product /home/dpark/Geode/apache-geode-1.12.0 \
-vm-padogrid /home/dpark/Geode/padogrid_0.9.7 \
-vm-workspaces /home/dpark/Geode/workspaces/myrwe \
-vm-user dpark

SSH Private Key

If a private key file is required and you have not specified it in the previous example, then you can place it in the workspace directory. PadoGrid automatically picks up the first .pem file found in the workspace directory if the VM_PRIVATE_KEY_FILE environment variable is not set in the $PADOGRID_WORKSPACE/setenv.sh file. The following shows the contents of the ws-vm workspace directory we created.

# If a private key file (.pem) is required then place it in in the
# ws-vm workspace directory.
switch_workspace ws-vm
tree -L 1 .
.
├── apps
├── bundles
├── clusters
├── docker
├── initenv.sh
├── k8s
├── lib
├── plugins
├── pods
└── setenv.sh

Test VM Workspace

Before you sync the local workspace with VMs - which may initially take some time to complete - if you need to install software products, we should first test the environment to make sure SSH works properly. This is done by executing vm_test.

vm_test

vm_test outputs the following:

------------------------------------------------------------------------------------------
Workspace: ws-vm

Environment:
                  VM_ENABLED=true
                    VM_HOSTS=ubuntu1,ubuntu2,ubuntu3,ubuntu4
                     VM_USER=dpark
         VM_PRIVATE_KEY_FILE=
                VM_JAVA_HOME=/home/dpark/Geode/jdk1.8.0_212
           VM_GEODE_HOME=/home/dpark/Geode/apache-geode-1.12.0
     VM_PADOGRID_HOME=/home/dpark/Geode/padogrid_0.9.7
VM_PADOGRID_WORKSPACE=/home/dpark/Geode/workspaces/myrwe/ws-vm
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
Network Test:
   ubuntu1 - OK
   ubuntu2 - OK
   ubuntu3 - OK
   ubuntu4 - OK
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
WARNING:
/home/dpark/Geode/jdk1.8.0_212
   Java is not installed on the following VMs. The workspace will not be operational
   until you install Java on these VMs.
       ubuntu1 ubuntu2 ubuntu3 ubuntu4

   To install Java on the above VMs, download the correct version of Java and execute 'vm_install'.

   Example:
      vm_install -java jdk1.8.0_212.tar.gz
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
WARNING:
/home/dpark/Geode/apache-geode-1.12.0
   Geode is not installed on the following VMs. The workspace will not be operational
   until you install Geode on these VMs.
       ubuntu1 ubuntu2 ubuntu3 ubuntu4

   To install Geode on the above VMs, download the correct version of Geode and
   execute 'vm_install'.

   Example:
      vm_install -padogrid apache-geode-1.12.0.tar.gz
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
/home/dpark/Geode/padogrid_0.9.7
   padogrid is not installed on the following VMs. It will be automatically installed
   when you execute 'vm_sync'.
       ubuntu1 ubuntu2 ubuntu3 ubuntu4
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
Summary:
   One or more VM issues found. Please correct them before executing 'vm_sync'.

Workspace Issues:
   None.
Network Issues:
   None.
Software Issues:
   Java missing from the following VMs. Install Java with 'vm_install -java'.
      ubuntu1 ubuntu2 ubuntu3 ubuntu4
   Geode missing from the following VMs. Install Geode with 'vm_install -product'.
      ubuntu1 ubuntu2 ubuntu3 ubuntu4
------------------------------------------------------------------------------------------

vm_test scans all the VMs and reports any issues it encounters. For example, the abovevm_test report shows that SSH sessions are working fine but Java and Geode are not installed on all of the VMs. To install Java and Geode on the VMs, we execute the vm_install command as follows.

vm_install -java /home/dpark/Downloads/jdk-8u212-linux-x64.tar.gz \
           -product /home/dpark/Downloads/apache-geode-1.12.0.tgz

vm_install outputs the following:

Java installation
    Local Source: /home/dpark/Downloads/jdk-8u212-linux-x64.tar.gz
   Remote Target: /home/dpark/Geode/jdk1.8.0_212
   Installing ubuntu1...
   Installing ubuntu2...
   Installing ubuntu3...
   Installing ubuntu4...
Java installation complete.

Geode installation
    Local Source: /home/dpark/Downloads/apache-geode-1.12.0.tgz
   Remote Target: /home/dpark/Geode/apache-geode-1.12.0
   Installing ubuntu1...
   Installing ubuntu2...
   Installing ubuntu3...
   Installing ubuntu4...
Geode installation complete.

Run 'vm_test' to check installation and configuration status.

If you run vm_test after installing Java and Geode, you should see no issues in the VM workspace. You can now proceed to sync the VM workspace.

Synchronize VM Workspace

In the previous section, we have locally created and tested a VM workspace. Before we can use it on the remote VMs, we need to synchronize it with the VMs. To do so, we execute the vm_sync command, which synchronizes the workspace you just created and automatically installs PadoGrid on all the VMs if it is not already installed.

vm_sync

vm_sync outputs the following:

Deploying padogrid_0.9.7 to ubuntu1...
Deploying padogrid_0.9.7 to ubuntu2...
Deploying padogrid_0.9.7 to ubuntu3...
Deploying padogrid_0.9.7 to ubuntu4...

Workspace sync: ws-vm
   Synchronizing ubuntu1...
   Synchronizing ubuntu2...
   Synchronizing ubuntu3...
   Synchronizing ubuntu4...
Workspace sync complete.

Configure Geode Cluster

In our example, we have created the default cluster named, mygeode. Let's switch into the mygeode cluster.

switch_cluster mygeode

The cluster directory has the following files.

mygeode/
├── bin_sh
│   └── setenv.sh
├── etc
│   ├── cache.xml
│   ├── cluster.properties
│   ├── gemfire.properties
│   ├── hibernate.cfg-mysql.xml
│   ├── hibernate.cfg-postgresql.xml
│   ├── log4j2.properties
│   └── prometheus.yml
├── lib
├── log
├── plugins
└── run

etc/cluster.properties

In the mygeode directory, you will find etc/cluster.properties which defines cluster-level properties. In this file, by default, the vm.locator.hosts property is set to the first VM host you listed for the workspace and the vm.hosts property is set to the entire host list. Let's make some changes to see how the cluster gets affected. Edit this file and change the vm.locator.hosts to ubuntu4 and change the vm.hosts property to ubuntu1,ubuntu2,ubuntu3. The these changes, the locator will run on ubuntu4 and the cache servers will run on ubuntu1,ubuntu2,ubuntu3.

# Browse the cluster properties and change them as needed. 
# Pay attention to the following two (2) properties:
#   vm.locator.hosts=
#   vm.privateKeyFile=
#   vm.hosts=
vi etc/cluster.properties

# A comma separated list of host names or addresses. IMPORANT: No spaces allowed.
vm.locator.hosts=ubuntu4

# Set the private key file path. If this property is set then it overwrites the
# VM_PRIVATE_KEY_FILE  environment variable set in the workspace's setenv.sh file. 
#vm.privateKeyFile=~/Geode/Workspaces/ecs.pem

# Include additional VM host names. Host names (or IP addresses) must be
# comma separated with no spaces. Spaces are not supported.
vm.hosts=ubuntu1,ubuntu2,ubuntu3

etc/cache.xml

You can also update the cache.xml file at this time as needed.

vi etc/cache.xml

Sync Changes

Any changes you made can be easily deployed to all the VMs specified by the VM_HOSTS environment variable in the $PADOGRID_WORKSPACE/setenv.sh by running the vm_sync command, which copies the entire workspace directory to the VMs. Be aware that this can take some time to complete if you have many VMs and large binary files in the workspace.

After the VMs have been synchronized, if you make changes to only a few files, then instead of executing vm_sync again, you can execute vm_copy which copies only the specified file or directory to the VMs.

switch_cluster mygeode

# To copy a single file
vm_copy etc/cluster.properties

# To copy the entire directory
vm_copy .

vm_sync vs vm_copy

The main difference between vm_sync and vm_copy is that vm_sync copies the entire workspace and vm_copy copies only the specified file or directory. Both commands allow you to apply the changes you made to all of the VMs. As you might have guessed, vm_sync will take longer to complete since it copies the entire workspace, but it is more convenient to use than vm_copy since you don't have to remember the files you modified. These commands will quickly become your favorite commands if you often reconfigure Geode clusters or update application library files.

❗ Note that vm_copy by default only copies the files that are in the workspace directory hierarchy. If you try to copy a file that is outside of the workspace directory then it will fail and output an error message. If you need to copy non-workspace files, specify the -mirror option, which copies the absolute file path to the same absolute file path in the remote VMs.

Start Cluster

You are now ready to start the mygeode cluster.

start_cluster

Monitor Cluster

To monitor the cluster:

show_cluster

View Log

To view logs

# ubuntu1
show_log

# ubuntu2
show_log -num 2

Stop/Kill Cluster

To stop or kill the cluster:

stop_cluster
kill_cluster

Running from VMs

You can also run VM workspaces from any of the VMs. As with the local environment, you must first setup the PadoGrid environment on the VM in which you want to run workspaces by sourcing in the workspaces initenv.sh file.

# SSH into one of the VMs
ssh ubuntu1

# Source in intienv.sh
. ~/Geode/workspaces/myrwe/initenv.sh

# Optionally add the above line in .bashrc (or .bash_profile) so that PadoGrid is
# automatically initialized when you login next time. Make sure to include the '-quiet' option.
echo ". ~/Geode/workspaces/myrwe/initenv.sh -quiet" >> ~/.bash_rc

# Once you have initenv.sh sourced in you can then execute any of the PadoGrid
# commands as before. The following starts the `mygeode` cluster.
switch_workspace ws-vm
swtich_cluster mygeode
start_cluster

Teardown

If you want to remove the cluster from all the VMs, then you must first stop the cluster and execute the remove_cluster command. The workspace can be removed using the remove_workspace command.

# Stop cluster including members and locators
stop_cluster -all

# Remove cluster. Unlike other commands, this command requires the '-cluster' option.
remove_cluster -cluster mygeode

# Simulate removing workspace from all VMs. Displays removal steps but does not
# actually remove the workspace.
remove_workspace -workspace ws-vm -simulate

# Remove workspace from all VMs. Runs in interactive mode.
remove_workspace -workspace ws-vm

References

  1. Geode/GemFire on AWS EC2 Instances
  2. Reactivating Geode/GemFire Workspaces on AWS EC2

◀️ Geode WAN Example 🔗 Geode-on-AWS-EC2 ▶️

PadoGrid Manual

Overview

Operations

Tools

Platforms

Clouds

Pado

Geode/GemFire

Hazelcast/Jet

ComputeDB/SnappyData

Coherence

Hadoop

Kafka/Confluent

Mosquitto

Redis

Spark

Clone this wiki locally