Standalone Receiver Setup for Cisco WLC

• Installation
• Upgrade

The data receiver can be set up in standalone mode to work as a proxy. In addition it can be configured as a High Availability/Scalable N+1 configuration using zookeeper for group coordination and leader election. These instructions only cover single server/non high available installation.

These instructions are for a single node non-HA receiver.

If required, the receiver can be set up in either an Active/Active (using 3 or more hardware independent nodes) or Active/Passive (using two nodes). Please ask if you need to implement these configurations.

Configuration

Please ensure that the server configuration is completed first - talk to Kloudspot support to get this done.

Proxy VM requirements

OS/Resources:

The VM running the proxy should have the following specification:

• RAM: 8GB
• Disk :50 GB
• 4 core
• OS: Ubuntu 18.04

Firewall:

The following routes should be enabled:

• From VM to WLC : port 16113
• From VM to Kloudspot Analytics Platform : port 9094

Proxy Receiver Setup.

Set up and update/upgrade a clean installation of Ubuntu 18.04 LTS.

When installing from scratch, make sure to install the open SSH server to allow remote access.

Test connectivity to the WLC and the Kloudspot Analytics Server from the VM:

$ nc -w2 -vz <WLC IP> 16113
$ nc -w2 -vz <Kloudspot server IP> 9094

Add Kloudspot’s official GPG public key:

 $ curl -fsSL https://registry.kloudspot.com/repository/files/kloudspot.gpg.key | sudo apt-key add -

Verify that you now have the key with the fingerprint 7DD9 F762 BBDB FBC9 3103 4270 0B15 B423 21FA FC35, by searching for the last 8 characters of the fingerprint.

$ sudo apt-key fingerprint 21FAFC35
pub   rsa2048 2019-12-02 [SC] [expires: 2021-12-01]
      7DD9 F762 BBDB FBC9 3103  4270 0B15 B423 21FA FC35
uid           [ unknown] Steve Jerman <steve@kloudspot.com>
sub   rsa2048 2019-12-02 [E] [expires: 2021-12-01]

Use the following command to set up the repository.

$ sudo add-apt-repository \
  "deb [arch=amd64] https://registry.kloudspot.com/repository/kloudspot-apt/  bionic main"

Install the receiver and its required components java, zookeeper

$ sudo apt-get update
$ sudo apt-get install kloudspot-receiver

Start zookeeper

$ sudo service zookeeper start
$ sudo systemctl enable zookeeper

Kloudspot Support will provide a client.truststore.pkcs file and password, copy this to /etc/kloudspot and edit the /etc/kloudspot/receiver.yml configuration to set the password and server address:

kafka:
  servers: <kloudspot server IP>:9094
  ssl: true
  truststore-location: /etc/kloudspot/client.truststore.pkcs
  truststore-password: replace-me

Kloudspot Internal Note: See here for generation instructions

Run receiver to see the connection command.

$ sudo -H -u kloudspot /usr/local/kloudspot/receiver/run.sh

You can stop the script (Cntl C) as soon as you see this:

**************************

Run this command on the WLC
  config auth-list add sha256-lbs-ssc <MAC Address> <SHA256>

********************

Run the specific command shown in program log from the above step on the WLC:

Then edit the /etc/kloudspot/receiver.yml and add the WLC host IP:

    standalone:      
        enabled: true
        connections:
        - type: wlc
          host: <WLC IP>

At this point installation should be complete. You can start up the receiver as a service with the following commands:

$ sudo service kloudspot-receiver start
$ sudo systemctl enable kloudspot-receiver

You can see the log using:

$ sudo journalctl -u kloudspot-receiver -f
$ sudo journalctl -u kloudspot-receiver --since "10min ago"

Generating the WLC SSL Connection File

The connection to the WLC is authorized using an MAC address and an SSL file. In order to regenerate this file following the following steps:

1. Edit the /etc/kloudspot/receiver.yml file. Remove the current MAC Address and set the keystore to an empty writeable location, also change the password if desired:

push:  
..  
   nmsp:  
      inputBufferSize: 48768
      macAddress: '50:D3:7B:5B:70:F8'
      keystore:
         password: **erHSbFfpKWLf**`  
         file: file:/tmp/wlc-keystore.pks``

2. Run receiver to see the connection command.

    $ sudo -H -u kloudspot /usr/local/kloudspot/receiver/run.sh
    ...
    **************************
    Add this values to the config file (push.nmsp.macAddress) :
    MAC: 50:D3:7B:5B:70:F8

    Run this command on the WLC
      config auth-list add sha256-lbs-ssc 50:D3:7B:5B:70:F8 fecb74538bb6be79f33b4dc23951552cd86523c0e563b5ac13070bf4205e0538
    ********************

Stop the receiver as soon as you see the connection command.

3. Copy the generated keystore (/tmp/wlc-keystore.pks) to /etc/kloudspot and edit /etc/kloudspot/receiver.yml as follows:

push:
...
    nmsp:
...
        macAddress: '50:D3:7B:5B:70:F8'
        keystore:
           password: erHSbFfpKWLf
           file: file:/etc/kloudspot/wlc-keystore.pks

Upgrade

The following procedure should be followed to upgrade the proxy receiver. Note that downgrade is not suupported.

Preparation

Prior to doing the upgrade make sure to backup your system.

1. Backup VM:
   • Ideally, have a snapshot available to restore in case of issues.
2. File Backup:
   • Take a copy of all files located in /etc/Kloudspot.

Upgrade Process

1. Stop Services:`

   sudo service kloudspot-receiver stop
   sudo service zookeeper stop

2. Update the KloudInsights instance that will be receiving data. This will likely need liason with Kloudspot operations.

3. Update Receiver:

   sudo apt-get update
   sudo apt-get upgrade
   <reboot>

The upgrade process is now complete.

Post-Upgrade Checks

Perform the following checks to ensure the successful completion of the upgrade:

1. Check Java Version:

   java  -version
   The version should be 17.

2. Check Receiver Logs:

   sudo journalctl  -u kloudspot-receiver  -f

There should be no errors displayed.

Additional Considerations

• Ubuntu Version:

  • The server should be running Ubuntu 18.04.5 LTS, which is still in support.

• Snapshot Backup:

  • Data will be lost for the duration the receiver is down.

• Checking Component Versions:

  • Use the following command to list available versions:

      apt list kloudspot-receiver  -a

• Checking Specific Component Version:

  • To install a specific version, use:

     sudo apt-get install  -y kloudspot-receiver=<version>

Questions and Answers

1. Can we use the following commands instead?

    apt-get install  -y openjdk-17-jdk
    apt-get install  -y kloudspot-receiver

Because our VPN software might be included in the package list and we don’t want to upgrade any packages except regarding Kloudspot-receiver.

Ans. Yes, that approach is acceptable.

2. What version should kloudspot-receiver be upgraded to? Also, how can we confirm the version after upgrading? Will “kloudspot-receiver -version” work?

Ans. Ask Kloudspot for the appropriate version to upgrade to. To confirm the version, use the following command:

apt list kloudspot-receiver  -a

Using a custom StorageClass

Most standard installatations on MicroK8S use OpenEBS for storage and are configured automatically. However, some installations might need a custom configuration - for example to use NAS storage.

It is possible to configure the system to use custom Kubernetes StorageClasses as follows:

You can use the ‘kloudspot init –custom-storage’ command to create a configuration file for your new system. It will ask a few questions and then create a ‘values.yaml’ file with the necessary configuration.

sjerman@k8s-single:~$ kloudspot init --custom-storage
?                                  Storage Configurations : Use custom storage class.
Use custom storage class.
?                            StorageClass for RWO storage : Name: microk8s-hostpath, Provisioner: microk8s.io/hostpath
?                            StorageClass for RWX storage : Name: microk8s-hostpath, Provisioner: microk8s.io/hostpath
Now using custom StorageClass for configuration
Initialize Kloudspot System Configuration

By default, the ingress controller will
use a self signed certificate. It is much better to
use a ‘proper’ SSL certificate.

Do you want to add ssl certificate? y/n
? Enter ssl key filepath: server.key
? Enter ssl ssl cert filepath: server.cert

Initialize Kloudspot System Configuration

First basic system information...
? DNS Hostname dibble.net
? Customer Reference steve

What features should be enabled
? Enable KloudManage No
? Enable KloudInsights No
? Enable Kloudhybrid Yes
Using ubuntu-vg for unshared volumes
Available:  Shared 32 GB, Unshared 49 GB

How much storage do you want to assign to each volume
? Stream processing elasticsearch (GB) 10
? Kloudinsights database (GB) 10
Required: Shared 0 GB, Unshared 20 GB
'/etc/kloudspot/values.yaml' created sucessfully.

Command Line Interface Overview

The Command Line Interface (CLI) in Ubuntu is a powerful text-based tool that offers efficiency, automation, and control. Accessed through the Terminal, it allows users to execute commands for file manipulation, system management, networking, and more. CLI’s advantages include resource efficiency, remote server management, and deeper system understanding. It complements the graphical interface, empowering users to customize their Ubuntu experience, perform complex tasks, and boost productivity. With scripting capabilities, the CLI becomes indispensable for system administrators, developers, and experienced users, providing unparalleled flexibility and insights into the Ubuntu operating system.

Configuration File Reference

kloudspot

A Helm chart for Kloudspot KloudInsights & KloudManage Applications

Requirements

Values

Autogenerated from chart metadata using helm-docs v1.11.0

SSH Shared Key Authentication

These instructions apply to MacOS

We recommend the use of SSH Shared Keys to secure CLI access to the system. The following steps can be used to set it up.

Create a Key Pair

1. On a local system (e.g. your laptop) create a key pair using the following command:

ssh-keygen

accept the default location.

2. It will ask for a passphrase. You can either set the passphrase to blank or use ssh-agent to cache the pass phrases.

3. The utility will create:

   • A private key: id_rsa This is the private key that needs to be used by everyone who will log in.
   • A public key: id_rsa.pub. This is added to the system you want to login to.

Install on Remote systems

1. Run the following command to copy the public key to a remote system.

ssh-copy-id <system>

2. You should now be able to log in to the remote host …

ssh <system>

(Optional) Disable Password login on remote host

1. Edit /etc/ssh/sshd_config and set following parameter to no:

PasswordAuthentication no

2. Then restart ssh server:

sudo service ssh restart

Securing MicroK8S

See:

• Configure host interfaces
• Ports
• Kubernetes

To list ciphers on port:

sudo snap install nmap
nmap --script ssl-enum-ciphers -p 16443 192.168.1.97

Ports

Update Configuration

Edit:

• /var/snap/microk8s/current/args/kube-apiserver
• /var/snap/microk8s/current/args/kube-scheduler
• /var/snap/microk8s/current/args/kube-controller-manager
• /var/snap/microk8s/current/args/kubelet

Add:

--tls-min-version=VersionTLS12
--tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA

You cannot directly edit cipher suites for cluster-agent. So either turn it off (microk8s disable ha-cluster) or ..

Edit /var/snap/microk8s/current/args/cluster-agent and add:

--min-tls-version=tls13

Restart Microk8s

Run:

sudo snap restart microk8s

You can then check the port usage, using nmap as described above.

Monitoring

For a HA on-prem installation it is a good idea to set up monitoring and alerting so that you can monitor the state of the cluster and get alerts for issues such as memory limits exceeded or low disk space.

Once the kloudspot platform is installed, the necessary files will be installed in /opt/kloudspot/monitoring:

• values.yaml : Helm chart configuration.
• dashboard-config.yaml : Loader for Kloudspot specific dashboard.
• monitors.yaml : Custom POD and Service monitor configurations to gather prometheus information from Kloudspot components.

Installation

Install the following Helm repo:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts   
helm repo update

Modify the /etc/monitoring/values.yaml file to suit. Typically this will only mean setting the Grafana URL:

  grafana.ini:
    server:
      root_url: https://localhost/grafana

Install the helm chart:

cd /opt/kloudspot/monitoring
helm install mtr  -f values.yaml --create-namespace -n mtr prometheus-community/kube-prometheus-stack

Once started, you can login to the Grafana instance with the following credentials:

• URL: https:\<server>/grafana/
• Username: admin
• Password: prom-operator

You can also access the Prometheus UI using port forwarding:

kubectl port-forward -n mtr service/prometheus-operated 9090:9090

Load Kloudspot Configuration

Next install some POD Monitors and Service monitors specific to the Kloudspot Platform

sjerman@steve-nuc:/opt/kloudspot/monitoring$ kubectl apply -f monitors.yaml 
servicemonitor.monitoring.coreos.com/kloudspot-flink-job-metrics created
podmonitor.monitoring.coreos.com/kloudspot-flink-tm-metrics created
servicemonitor.monitoring.coreos.com/kloudspot-web-ui-metrics created

Add a custom dashboard for the Kloudspot Platform:

sjerman@steve-nuc:/opt/kloudspot/monitoring$ kubectl apply -f dashboard-configmap.yaml 
configmap/kloudspot-grafana-dashboard created

Enable Kafka Monitoring. Edit /etc/kloudspot/values.yaml:

kafka:
...
    metrics:
       kafka:
          enabled: true
       serviceMonitor:
          enabled: true
          labels:
            release: mtr

Then restart the Kloudspot services:

kloudspot update -u

You can also use the Grafana administration interface to create custome alerts and dashboards as required.

Once you have the monitors and dashboard loaded you will be able to see some data:

Reference

• Kubernetes Operator
• Operator Helm Docs
• Prometheus
• Grafana