Skip to main content
Version: NG-3.1

Data Modelling Workspace

Overview​

In today's data-driven world, the ability to effectively manage and interpret vast amounts of information is crucial. Data Models play a pivotal role in this process by providing a structured and user-friendly framework for querying and refining data.

Users can create and manipulate data models that encapsulate complex business logic, making it easier to derive insights and make data-driven decisions. By simplifying database queries, Data Models enable users to focus on defining golden signals and their dimensions, applying transformations, and enriching data with domain-specific knowledge. This empowers users to build comprehensive data pipelines, enhance data with contextual information, and ultimately streamline the analysis process, ensuring that critical insights are readily accessible and actionable. It powers the functionality of Data Visualization, Alerts, Reports, and Insights, ensuring a comprehensive and intuitive data-handling experience.

Additionally, vuSmartMaps enhances the query-writing experience with an intelligent SQL Autocompletion capability. This feature assists users while writing queries by providing real-time suggestions, templates, and schema-aware recommendations, reducing manual effort and improving query accuracy and consistency.

Getting Started with Data Modelling Workspace​

  1. Accessing Data Modelling Workspace: Navigating your Data Modelling Workspace is a breeze. vuSmartMaps Data Modelling can be accessed from the left navigation menu by clicking Data Modelling Workspace under Configure Observability. Once you're there, you'll find a list of all the data models. You can easily manage your Data Models by performing actions such as creating, viewing, editing, and deleting them. For enhanced control, you can customize permissions for different roles—whether it's granting viewing rights or permitting modifications.
  2. Data Store: A data store is a repository for persistently storing and managing collections of data. Accessing and handling the data stores is simple and can be found at the top-left corner of the Data Models listing page. vuSmartMaps supports a variety of backend databases, making it highly adaptable to your needs. You can effortlessly query these Data Stores using a standard Query Editor or tailor your queries to the specific capabilities of each Data Store. Currently, Data Stores officially supports PostgreSQL, MySQL, Elasticsearch, and Hyperscale DB.
  3. New Data Model Creation: Creating a new Data Model is a breeze with our intuitive user interface. The process consists of four straightforward steps: Start by providing the basic information, set up your data query, apply any necessary transformations, and finally, define the properties of your Data Model.
    • Basic Information: You are required to provide a unique and intuitive name and a brief description of the data model.
    • Prepare Data Query: This involves crafting queries to fetch data from your databases. The process allows you to choose between building queries using a user-friendly Query Builder or directly specifying a Raw query, giving you flexibility in how you model your data.
    • Do Transform: 3. These transformations enable data manipulation to suit your specific needs.These include:
      • Join
      • Concatenate
      • Arithmetic operations
      • Custom expressions While not mandatory for single-query data models, transformations become essential when working with multiple queries.
    • Additional Attributes: you can select the metrics and buckets you want to expose through this data model. In the 'Set Properties' step, you can choose, and define properties for the metrics and buckets to be accessible to Data Model users.
  4. Support for Dynamic Filters in Dashboards VuNet's dashboard lets you use dynamic filters for data viewing. The addition of the data model to the dashboard occurs indirectly via Insights. Configure your queries with custom filters to accurately filter the data on the dashboard.

Why This Feature Is Useful​

  • The Data Modelling Workspace simplifies the process of transforming raw data into meaningful business insights. It enables users to create reusable data models that encapsulate business logic, apply data transformations, and expose only the required metrics and dimensions for analysis.
  • By providing a centralized workspace for query creation, data enrichment, and data preparation, the Data Modelling Workspace reduces the complexity of working directly with underlying databases. The resulting data models can be consumed across dashboards, insights, alerts, and reports, ensuring consistency and reusability across the platform.

Example Scenario​

An operations team wants to monitor application response times stored in PostgreSQL and visualize the data on multiple dashboards. Instead of writing separate queries for every dashboard, the administrator creates a reusable data model in the Data Modelling Workspace, validates the query, applies the required transformations, and exposes the processed data. The same data model can then be reused across dashboards, alerts, and reports, ensuring consistency while reducing duplicate query creation.

When to Use This Feature​

Use the Data Modelling Workspace when you need to:

  • Create reusable datasets for dashboards, reports, alerts, or insights.
  • Combine data from one or more supported data stores.
  • Apply business-specific logic and transformations to raw data.
  • Build metrics, dimensions, and golden signals for monitoring and analysis.
  • Filter, aggregate, and enrich data before exposing it to end users.
  • Standardize data access across multiple teams and use cases.
  • Create data models that support dynamic filtering and interactive visualizations.

Step-by-step instructions​

Follow these high-level steps to start working with Data Models:

  1. Open Observability Studios > Data Modelling Workspace.
  2. Review the existing Data Models available in the workspace.
  3. Configure or verify the required Data Store connection.
  4. Create a new Data Model using the + button.
  5. Provide basic information such as the Data Model name and type.
  6. Configure one or more queries using the Query Builder or Write Query option.
  7. Apply transformations, if required.
  8. Define metrics, dimensions, and properties that should be exposed through the Data Model.
  9. Validate the configuration and save the Data Model.
  10. Use the Data Model in dashboards, insights, alerts, or reports.

Page / Screen Overview​

The Data Modelling Workspace provides a centralized interface for creating, managing, and maintaining data models. From this workspace, users can manage data stores, configure permissions, build queries, apply transformations, preview results, and publish reusable data models. The workspace contains the following major sections:

  • Data Models List – Displays all available data models and provides options to create, edit, clone, or delete them.
  • Data Store Management – Allows users to configure and manage supported data stores.
  • Permission Settings – Enables administrators to control access to individual data models.
  • Query Configuration – Supports both Query Builder and SQL Editor for data retrieval.
  • Transformation Workspace – Applies data transformations before publishing the model.
  • Preview and Validation – Helps verify query results and validate the model before saving.
  • Properties Configuration – Defines the final properties exposed by the data model.

Accessing Data Modelling Workspace​

vuSmartMaps Data Modelling Workspace can be accessed from the left navigation menu (Observability Studios > Data Modelling Workspace). Upon clicking 'Data Modelling Workspace', you are taken to the 'Landing Page’.

Landing Page​

All the Data Models created in the system are visible here. Utilize features in the listing table, such as search and filters, to locate specific Data Models.

The listing gives information on the following:

  • Name: The data model's name. Click it to view the data model in non-edit mode.
  • Data Store Name: The name of the Data Store used by this Data Model.
  • Type: Indicates if it's a data model that comprises metrics or events.
  • Created By: The user who created the data model.
  • Created At: The date and time when the data model was created.
  • Modified By: The user who recently modified the data model.
  • Modified At: The date and time of the most recent modification.
  • Actions: Edit or Delete your Data Models using the buttons under this column.
  • + button: It is used to create a new Data Model. You will learn how to create a Data Model under the New Data Model Creation Section

Actions

The Data Modelling Workspace's home page can be used for various basic operations such as, Edit and Delete Data Models.

Edit- Click on this icon to edit the respective Data Model.

Delete: You can delete a Data Model by clicking the delete option under the Action column. Please note that the Data Models can't be deleted if Insights Alerts or Reports are using it.

Deleting Multiple Data Models: Select the multiple Data Models by checking the boxes on the extreme left and clicking the Delete button on the top right.

note

Data Models cannot be deleted if they are used in Insights, Alerts, or Reports.

Setting Permissions for a Data Model​

We use RBAC (Role Based Access Control) to let you grant access to Object Level Permissions which are made available in 'Edit' Mode. This ensures that only authorized users can access specific functions, resources, or information.

  • Click on Edit for the data model you want to manage.
  • You'll be directed to the Set Properties page.
  • Click on Permissions to access and manage Object Level Permissions.

The screen will look like this, For each role, you can assign three types of permissions

  • None: There are no permissions given.
  • View: The selected user can only view the Data Modelling Workspace.
  • Modify: The selected can also modify and make changes to the Data Modelling Workspace.
Need to manage your Data Models? Head to the Actions column to edit, delete, or bulk delete—plus, use RBAC to control who can view or modify them securely!

Data Store​

Data Store refers to the SQL and NoSQL databases available in vuSmartMaps. vuSmartMaps supports multiple databases.

Supported Data Stores​

The following Data Stores are officially supported:

  • PostgreSQL 14.7 and above
  • MySQL - 14.14 and above
  • Elasticsearch 7.0 and above
  • Hyperscale DB 23.8+ and above

Data Store Management​

vuSmartMaps comes with the default configurations of NoSQL and Hyperscale DB. You can easily add other supported data stores to the available data stores. Click on the Data Store button at the top right corner to see the listing of already configured data stores and also manage them.

Setting up your Data Store? Choose from PostgreSQL, MySQL, Elasticsearch, or Hyperscale DB—just follow the steps, test the connection, and you're all set!

Add New Data Store​

  1. Click Data Store on the right top corner of the landing page.

  1. The Data Store landing page shows a list of previously configured Data Stores. Now click on the + icon on the top right to create a new Data Store.

    • Name: User-defined unique name for the Data Store.
    • Data Store Type: Type of the selected Data Store (e.g., PostgreSQL, Elasticsearch).
    • Created By: Name of the user who created the Data Store.
    • Created At: Date and time when the Data Store was created.
    • Modified By: Name of the user who made the most recent modifications to the Data Store's configuration.
    • Modified At: Date and time of the most recent configuration changes.
    • Actions: Each Data Store has two buttons:
      • Edit: Click to make changes to the Data Store's configuration.
      • Delete: Click to delete the Data Store (further details are in the next section).

  2. You can see the list of supported Data Stores. You can search for any specific supported Data Store in the search bar. Click on the Data Store of your choice.

  1. Follow the Data Store's instructions for configuration, then click the Save & Test button.

    • PSQL Data Store: All the fields are mandatory. The platform engineer or an admin should know the access details. _ Data Store Name: Enter the name of the Data Store.
      _ Description: Give a brief description.
      _ Database: Admin to enter the Database name.
      _ Host: Enter the Database Host IP address.
      _ User: Enter the user name of the Database
      _ Password: Enter the Database password.
      _ Port: Enter the Port number.
      _ Time Zone: You can manually choose the Time Zone or click on the Detect Time Zone to automatically select the timezone.
      _ Selecting a Time Zone at the Data Store level ensures correct time-range data is fetched from the Data Store.
      _ isSSL: Enable it if you have SSL (Secure Sockets Layer) certified and enter the CA certificate details, Client Certificate, and vuSmartMaps.dataModels.labels.client_key.

    • Elasticsearch Data Store: All the fields are mandatory. The platform engineer or an admin should know the access details. _ Data Store Name: Enter the name of the Data Store.
      _ Description: Give a brief description.
      _ Host: Enter the Database Host IP address.
      _ Port: Enter the Port number.
      _ ES Version: Choose the Elasticsearch version from the dropdown.
      _ Time Zone: You can manually choose the Time Zone or click on the Detect Time Zone to automatically select the timezone. It is a mandatory field.

      • Hyperscale Data Store: All the fields are mandatory. The platform engineer or an admin should know the access details.

        • Data Store Name: Enter the name of the Data Store.
        • Description: Give a brief description.
        • Database: Admin to enter the Database name.
        • Host: Enter the Database Host IP address.
        • User: Enter the user name of the Database
        • Password: Enter the Database password.
        • Port: Enter the Port number.
        • Time Zone: You can manually choose the Time Zone or click on the Detect Time Zone to automatically select the timezone.
        • isSSL: Enable it if you have SSL (Secure Sockets Layer) certified and enter the CA certificate details, Client Certificate, and vuSmartMaps.dataModels.labels.client_key.

  2. If the Data Store connection is successful, you'll be directed to the listing page, and your newly added Data Store should be visible.

note

If the hostname, user, and password are incorrect then the Data Store will not get created and immediately throws an error message.

Delete

Use this option to delete the Data Stores that are no longer required.

  • Click the Delete button under the “Actions" column for the specific Data Store you want to delete.
  • A confirmation window will appear to ensure you want to proceed with the deletion.
  • Click Delete to confirm and permanently remove the Data Store.
  • If you change your mind, you can click Cancel to abort the deletion process.

Please note that deleting a Data Store is irreversible, so be certain of your decision before confirming the deletion.

note

You can’t delete the Data Store if it is being used by any of the Data Models. The system will immediately throw an error.

With that, you have successfully created a new Data Store.

New Data Model Creation​

To add a new Data Model, simply click the + icon on the Data Models listing page.

Data Model creation is a 4 step process. These steps are as follows:

  1. Basic Information: Fill in the basic details.
  2. Prepare Data Query: Configure the data query.
  3. Do Transform: Apply transformations as needed.
  4. Additional Attributes: Define the Additional Attributes of your Data Model.

FAQs​

What are Data Stores in vuSmartMaps?

Data Stores in vuSmartMaps are repositories for persistent data storage and management. Key features include:

  • Supported Databases: vuSmartMaps supports PostgreSQL, MySQL, Elasticsearch, and Hyperscale DB.
  • Data Handling: You can manage and access these Data Stores through the Data Modelling Workspace, enabling you to store and retrieve large volumes of data efficiently.
  • Configuration: Set up Data Stores by specifying connection details, database type, and authentication settings.
Can I customize my data models to suit my specific needs?

Yes! You can customize your data models by defining properties, applying transformations, and adding new fields. You can also use arithmetic operations, expressions, or custom values to modify data models based on your specific requirements.

What should I do if I encounter issues while connecting to a Data Store?
  • Check Credentials: Ensure that the hostname, username, password, and port are correctly entered.
  • Network Configuration: Verify that network settings and firewall rules allow connections to the Data Store.
  • SSL Configuration: If using SSL, confirm that the CA certificate, client certificate, and client key are properly configured.
What happens if a Data Store is deleted but still in use?

If you try to delete a Data Store that is currently linked to any existing Data Models, the system will throw an error and block the deletion. You must first disassociate or delete the dependent models.

Can I set access permissions for specific users in the Data Modelling Workspace?

Yes. vuSmartMaps supports Role-Based Access Control (RBAC). You can assign permissions at the object level, specifying whether a user role can view, modify, or have no access to a particular Data Model.