Skip to main content
Version: NG-3.1

Deployment and Installation using vuLauncher

Introduction​

vuLauncher is a standalone application designed to facilitate the deployment of vuSmartMaps™. The application operates via a command-line interface (CLI) and supports installation in two distinct environments:

  1. Virtual Machines: vuSmartMaps can be installed on virtual machines running any of the supported operating systems. The vuLauncher installs the base Kubernetes platform using the kubeadm provider, followed by the deployment of the vuSmartMaps application.
  2. Managed Kubernetes: vuSmartMaps can also be deployed on managed Kubernetes infrastructures, such as Amazon EKS, Azure AKS, Google GKE, OpenShift, and more. Refer the Managed k8s Deployment guide for more details

Deployment prerequisites document can be found here Do not start the deployment until all the pre-requisites are in place. The vuLauncher installer will run pre-flights in the beginning to validate the readiness of the nodes for installation. If one or more nodes are not ready, the installation will not start.

Deployment Specification (D-spec)​

The d-spec is unique to each environment and client installation. It provides control over resource limits for services, service grouping across nodes, and additional configurations.

D-Specs can be generated from the D-Spec utility of the vuSizer tool available here.

Downloading the binary​

  1. Create a folder in the home directory and download the NG installation binary using the below command
  • wget https://download.vunetsystems.com/blob/3.1/vuSmartMaps_offline_NG-3.1.tar.gz - - user=<username> - - password=<password> - - no-check-certificate
note

If you don’t have access to the download server, download the binaries directly from this URL.

  • Please check with the VuNet support team to get the credentials for the Download server and the d-spec.
  • d-spec is unique for an environment, update the d-spec details according to the deployment environment. Ensure that the Dspec file - should be in this format - sizing-vm.yaml

Starting vuLauncher​

Verify the build version, before the start of the deployment - ./build/launcher_linux_x86 --version

  • This will initiate the deployment for vuSmartMaps. It’ll auto detect username, private key and the sizing file.
  • Now, start the deployment using - ./build/launcher_linux_x86 --debug

To get all the possible arguments that can be passed to the launcher binary can be found via “./build/launcher_linux_x86 - - help”

CLI Arguments​

vuLauncher CLI arguments :

  • --init — Generates install-config.yaml with auto-detected values. Run this first before anything else. ./build/launcher_linux_x86 - -init
  • --validate-config — Validates the sizing file for syntax and required fields. No installation is performed. ./build/launcher_linux_x86 --dspec sizing-vm-3.0.yaml --validate-config
  • --dry-run — Validates all inputs without starting installation. Use as a final check before deploying. ./build/launcher_linux_x86 --dry-run
  • --debug - Initiates the deployment of vuSmartMaps
  • --view installation — Prints all installation stages with their current status. ./build/launcher_linux_x86 --view installation
  • --view services — Prints all installed Helm chart services and their status. Use to verify post-install health. ./build/launcher_linux_x86 --view services.
  • --status — Shows the current installation progress and exits. Can be run at any time during or after installation to check which stages have completed.
  • --log-level — Controls the verbosity of terminal log output. Accepted values: debug (shows all), info (default — shows INFO + WARN + ERROR), warn (shows WARN + ERROR only), error (shows ERROR only).

Launcher Workflow​

When we start the launcher, we will validate all the configuration files (launcher config, d-spec). The entire workflow of the launcher is divided into the following stages

Copying Kubernetes Package​

  • This stage copies the kubernetes.tar.gz to all the available nodes in the d-spec.
  • This tar file contains all the Kubernetes-related binaries, kubeadm scripts, and docker images.
  • This will be copied to the data tier that we specified in the d-spec. The preference of the data tier is vusmart > default > hot > warm > cold > default-replica.

Extract Kubernetes Package​

  • This extracts the kubernetes.tar.gz on all the nodes.
  • Here launcher executes the bash command (tar -xvzf kubernetes.tar.gz) over an ssh connection
  • This stage creates the soft link for containerd and kubelet.
  • The launcher will choose the mount point-based preference: vusmart > default > hot > warm > cold > default-replica.
  • Assuming the mount point chosen is /data, then:
  • /var/lib/containerd -> /data/vunet/containerd
  • /var/lib/kubelet -> /data/vunet/kubelet
note

Please ensure that no softlinks are present before the installation.

Master Node Setup​

  • This stage installs Kubernetes services in the master node. The master node will be identified in the d-spec, marked with is_master: true.
  • For a multi-master set-up, provide is_master: true to the required nodes in the Dspec.

Worker Node Setup​

  • This stage will install Kubernetes in all the worker nodes. This installation will happen in parallel.
  • If any of the worker nodes fail to get installed, then we will mark this as a failure.

Kubernetes Pre-Setup​

  • This stage contains the pre-installation tasks required before installing the actual helm charts. These tasks include:
  • Creating node-labels and taints
  • Creating a secret to pull docker images
  • Executes preK8sManifest

Kubernetes cluster Validation​

  • Once the Kubernetes cluster is successfully installed, we will validate it again to see that all the services are running correctly

Helm Chart Updates​

  • This stage will update all the helm-charts with the contents(resources & limits, PV Mapping details and any other manual config overrides) in the d-spec.

Service Installation​

  • This stage will install all the helm-charts for the services listed in the d-spec.
  • Each helm-chart will have its dependent helm-charts specified in the values.yaml file, so the launcher will identify the correct order to install helm-chart.
  • If any service helm-chart fails to get installed with an error, then all its parent helm-charts will be skipped.

Template Upload​

  • This stage handles uploading all the agent binaries, vublock, vustream templates, and other static-files to MinIO UI.

Post-deployment steps​

  1. Update the Kafka Host & Port for omniagent/values.yaml and deploy omniagent service.
  • KAFKA_EXTERNAL_HOST: <hostname> (Should point to broker DNS FQDNs )
  • KAFKA_EXTERNAL_PORT: "443"
  • PLATFORM_HOST: "x.x.x.x" (Public IP of the VM)
note

(Note datahub services(brokers) are usually deployed in dnsname mode, however for some reason, if its deployed in hostIP mode, the external host should point to any of the broker IPs and port should be 9094(secure broker port) )

  1. For internal setups where datahub is deployed in dnsname mode however there is no real DNS available, we can add the below entry (Public IP of the brokers) under /etc/hosts of target nodes sending data to vusmartmaps (ex: omniagent nodes).
  • x.x.x.x (Public IP of any of the VMs) vunet-datahub.prod.mybank.com
  • x.x.x.x (Public IP of any of the VMs) vunet-datahub-0.prod.mybank.com
note

In case of multinode, add the other node entry details (IP & DNS) as well.

  1. Run the below command under post-jobs helm -chart for omniagent package preparation and discovery probes.

  • kubectl -n vsmaps delete job omniagent-package-prep --ignore-not-found

  • helm template post-job . \

    --namespace vsmaps \

    --set omniagentPackagePrep.enabled=true \

    --show-only templates/omniagent-package-prep-rbac.yaml \

    | kubectl apply -n vsmaps -f -

  • helm template post-job . \

    --show-only templates/omniagent-package-prep-job.yaml \

    --set omniagentPackagePrep.enabled=true \

    -n vsmaps | kubectl apply -n vsmaps -f -

Conclusion​

This user guide outlines the systematic process for deploying vuSmartMaps NG-3.1, providing clear instructions to ensure a smooth and successful system deployment.

Key Takeaways​

  • Streamlined Installation of vuSmartMaps: vuLauncher is a dedicated application designed to simplify the installation of vuSmartMaps within VM-based environments. The user-friendly UI allows users to input essential details for efficient deployment.
  • Comprehensive Deployment: This application encompasses key functionalities, including installing vuSmartMaps, controlling pipelines via Data Stream Orchestration, and deploying O11ySources using O11ySources Orchestration, providing a complete solution for system deployment.
  • Prerequisites and Configuration: Ensure VMs meet prerequisites such as proper configuration, connectivity to SiteManager, and necessary command execution. Expose essential ports, set environment variables, and have uniform credentials across VMs.
  • Step-by-Step Installation: The installation process involves starting SiteManager, uploading licenses, selecting the target VM platform, configuring VM SSH credentials, confirming mapping, customizing port configurations, and initiating the installation process.
  • Efficient Mapping and Deployment: vuLauncher employs a resource-driven backtracking algorithm to optimally allocate services to available VMs. This ensures effective utilization of resources, with mapping confirmed and services deployed after smart allocation.

The streamlined process ensures smooth deployment of vuSmartMaps while optimizing resource utilization, paving the way for effective system operation.

FAQs​

What happens if my vuLauncher installation fails midway? Can I resume, or do I need to start over?

If the installation fails at any stage, vuLauncher allows you to resume from where it stopped. Use the same command without the --sizing argument to avoid overwriting the previous state.

Refer to: Service Installation and CLI Arguments > --start-from

How can I validate if my d-spec file is correctly configured before initiating a full installation?

You can run vuLauncher in dry-run mode using the --dry-run flag. This checks the validity of the d-spec file and highlights issues before actual deployment begins.

Refer to: CLI Arguments > --dry-run

Is it mandatory to expose all listed ports if my setup is on a single-node deployment?

No, for single-node deployments, the required ports must be open internally on the same node, not across multiple nodes.

How does vuLauncher decide where to create containerd and kubelet directories during VM installation?

vuLauncher uses a preference order for mount points to place containerd and kubelet data. If directories like /var/lib/containerd already exist, this stage fails.

Refer to: Launcher Workflow > Create Soft Link

Can I install vuSmartMaps on VMs that already have Kubernetes or Docker installed?

No, the documentation explicitly recommends using fresh VMs. Existing Kubernetes or Docker installations may conflict with vuLauncher’s process.

What should I do if my organization has strict internet access policies? Will vuLauncher still work?

Yes, but in such cases, VuNet support will help install missing Linux dependencies and may ship required images instead of pulling them online.

How does vuLauncher manage service-to-VM mapping for optimized deployment?

The tags in the d-spec file group VMs, allowing services to be mapped to optimized node groups using resource-driven logic during deployment.

Refer to: Deployment Specification (D-spec) > category and tags sections

What are the critical services that need external access, and what are their corresponding port requirements?

Key external communications include downloading installation packages, accessing the GitHub Container Registry, and telemetry data ingestion. Ports like 443, 9092, 4317, 161, etc., must be opened accordingly.

How do I decide whether to set env_name to “vm” or “cloud” in the d-spec file?

Set env_name: "vm" if you're installing on VMs without an existing Kubernetes setup. Use env_name: "cloud" if deploying on an existing managed Kubernetes cluster (e.g., AKS, EKS, GKE).

Refer to: Deployment Specification > env_name parameter

If one of the worker nodes fails during Kubernetes setup, will the whole installation stop?

Yes. vuLauncher will mark the installation as failed if even one worker node setup fails. You'll need to resolve the issue and restart from the worker node setup stage.

Refer to: Launcher Workflow > Worker Node Setup