Tables & Data Storage

Overview

Table & Data Storage in vuSmartMaps helps administrators manage data stored in the HyperScale Data Store. It provides visibility into how data is organized across storage tiers and allows archived data to be restored when required.

The HyperScale Data Store organizes data into three storage tiers:

1. Hot Data Storage: Stores recently accessed and critical data. It provides high-speed access for immediate retrieval. Data in Hot storage is maintained based on the policy defined in Data Retention settings.
2. Warm Data Storage: Stores data that is accessed less frequently but is still relevant. It provides moderate-speed access.
3. Archived Data Storage: Stores historical or rarely accessed data for long-term archival needs. Archived data cannot be queried directly. It must be restored before users can access it again.

Table & Data Storage helps administrators view storage usage, check available tables, monitor retention periods, view system tables when required, and restore archived data back to the main table.

Why This Feature Is Useful

In banking and payment environments, large volumes of operational data are generated from systems such as core banking, UPI, switches, gateways, middleware, APIs, and infrastructure components.

Table & Data Storage is useful because it helps teams:

• Understand how data is distributed across Hot, Warm, and Archived storage.
• Check which tables are available in each storage tier.
• Monitor storage utilization for each tier.
• Identify when storage usage reaches warning or critical levels.
• Keep recent data accessible for faster analysis.
• Store older data in lower-access tiers for long-term retention.
• Restore archived data when historical analysis is required.
• Ensure archived data becomes queryable again after restoration.

Example Scenario

A support engineer needs to investigate an issue reported five days ago. The required data is no longer available in the active queryable table because it has moved to Archived storage based on the configured retention policy.

The engineer opens Data Studio > Table & Data Storage, goes to the Archive tab, searches for the required archived table, and clicks Restore from the Actions column. After confirming the restore action, the archived data is restored to the original main table and becomes queryable again.

The restored data remains available for the configured restore retention period. By default, this period is 7 days, and it can be configured in the Data Retention module.

When to Use This Feature

Use Table & Data Storage when:

• Administrators need to view Hot, Warm, or Archived data tables.
• Storage utilization needs to be checked.
• Data retention periods must be reviewed for available tables.
• System tables need to be displayed along with regular data tables.
• Archived data must be restored for analysis.
• Multiple archived tables need to be restored together.
• Users need to verify whether archived data is already present in Hot or Warm storage.
• Users need to understand whether a restore operation is blocked due to storage restrictions.
• Teams need to confirm whether Hot, Warm, or Archived storage is enabled.

Use Table & Data Storage when you need to review Hot, Warm, or Archived tables, check storage usage, show system tables, or restore archived data for analysis.

Comprehensive Understanding

The Table & Data Storage page gives administrators a single view of how data is stored across the HyperScale Data Store. It helps users check available tables, view storage usage, search for tables, display system tables when required, and restore archived data.

Navigation Path

To open the Table & Data Storage page, go to: Data Studio > Table & Data Storage After opening the page, the HyperScale Data Store view is displayed. This view is organized into storage tabs for different data states.

Storage Tabs

The page has three storage tabs:

1. Hot
2. Warm
3. Archive

Each tab represents a different storage tier.

• Hot: storage contains recently accessed and critical data. This data is available for fast access.
• Warm: storage contains data that is accessed less frequently but is still useful.
• Archive: storage contains historical or rarely accessed data. Archived data cannot be queried directly. It must be restored before it can be accessed again.

Table List Area

Each storage tab displays a list of tables available in that storage tier.

In the Hot and Warm tabs, the table list shows:

1. Name of the table.
2. Size of the table.
3. Modified At time.
4. Data Retention Period.

The Data Retention Period shows how long the data stays in the current tier. After this period is completed, the data is either removed or moved to another storage tier based on the configured retention policy.

In the Archive tab, the table list shows archived data. The archived table view includes:

1. Name of the archived table.
2. Actual Size.
3. Compressed Size.
4. Modified At time.
5. Data Retention Period.
6. Actions.

The Actions column in the Archive tab provides the Restore option.

Storage Utilization Indicator

At the top of each storage tab, the page shows storage utilization.

This indicator shows:

1. How much storage is currently used.
2. What percentage of total available storage is used.

The utilization status is color-coded so administrators can quickly understand whether storage usage is normal, near warning level, or critical.

For Hot and Warm storage:

1. Warning threshold is 50% of total memory.
2. Critical threshold is 80% of total memory.

For Archived storage:

1. Warning threshold is 80% of total memory.
2. Critical threshold is 90% of total memory.

These thresholds help administrators monitor storage capacity and take action before storage becomes a problem.

Search Tables

The Search Tables field helps users quickly find a table in the selected storage tab. Use this field when the table list is long and the user wants to locate a specific table by name.

Show System Tables

The Show System Tables radio button controls whether internal platform tables are displayed. By default, system tables are hidden because they are used internally by the platform. When Show System Tables is selected, system tables are shown along with regular data tables. This gives administrators a complete view of the data stored in the selected storage tier.

Restore Option in Archive Tab

The Restore option is available only for archived data.

Users can restore archived data in two ways:

1. Restore a single table
   Use the Restore button in the Actions column for the required archived table.
2. Restore multiple tables
   Select multiple rows in the Archive tab and click the Restore button at the top of the table.

When the restore process is started, archived data is moved back to the Hot or Warm disk from which it was originally archived. The restored data is added to the main table and becomes queryable again.

Restore Confirmation Pop-up

After clicking Restore, a confirmation pop-up appears. In this pop-up, users must click Initiate Restore to start the restore process.

The restore action brings archived data back from archival storage to the original Hot or Warm disk. The restored data remains available for the configured restore retention period. By default, this period is 7 days, and it can be changed in the Data Retention module.

Archived data cannot be queried directly. Restore it from the Archive tab before users can access it again.

Step-by-Step Instructions

View Tables in Hot, Warm, or Archived Storage

1. Open the Table & Data Storage page.
2. Select the required tab:
   • Hot
   • Warm
   • Archive
3. Review the table list shown in the selected tab.
4. Check the table name, size, modified time, and data retention period.
5. In the Archive tab, also review archive-specific size details such as actual size and compressed size where shown.
6. Use the data retention period to understand how long data remains in that storage tier.

Check Storage Utilization

1. Open the required storage tab.
2. Check the storage utilization message shown at the top of the page.
3. Review the actual memory used and the percentage of total storage used.
4. Use the color-coded indicator to identify whether usage is normal, warning, or critical.
5. For Hot and Warm storage, treat 50% as the warning threshold and 80% as the critical threshold.
6. For Archived storage, treat 80% as the warning threshold and 90% as the critical threshold.

Search for a Table

1. Open the required storage tab.
2. Use the Search Tables field.
3. Enter the table name or part of the table name.
4. Review the filtered table list.

Show System Tables

1. Open the required storage tab.
2. Locate the Show System Tables radio button.
3. Select Show System Tables.
4. The system displays system tables along with regular data tables.

By default, system tables are hidden because they are used internally by the platform.

Restore a Single Archived Table

1. Open the Archive tab.
2. Find the archived table that needs to be restored.
3. In the Actions column, click the Restore button for that row.
4. A pop-up window opens.
5. Click Initiate Restore.
6. The system restores the data from archival storage to the Hot or Warm disk from which it was originally archived.
7. The restored data is moved to the main table only.
8. After restoration, the data becomes part of the original dataset and can be queried again.

Restore Multiple Archived Tables

1. Open the Archive tab.
2. Select the rows that need to be restored.
3. Click the Restore button located at the top of the table.
4. A restore pop-up opens.
5. Click Initiate Restore.
6. The selected archived data is restored to the Hot or Warm disk from which it was originally archived.

Understand Restored Data Retention

After archived data is restored:

1. The data is placed immediately on the original Hot or Warm disk from which it was archived.
2. The data is restored to the main tables only.
3. The restored data becomes part of the original dataset.
4. The restored data remains in the main table for the restore retention period.
5. The default restore retention period is 7 days.
6. This restore retention period is configurable in the Data Retention module.
7. After the restore retention period ends, the restored data is permanently deleted.

As part of the schedule, retention settings for the corresponding table are updated. This ensures restored data is retained from the point of restoration for the defined restore retention period.

Restored data is placed on the original Hot or Warm disk, restored to the main table only, and retained for the configured restore retention period.

Understand Restore Restrictions

Users cannot initiate restore in the following cases:

1. Restore Already in Progress: A restore operation is already in progress according to system information.
2. Data Already Present in Hot or Warm Storage: The data is already present in Hot or Warm storage. This is determined by checking whether a table with the same name exists in Hot or Warm storage. It also applies if the data is within the retention period specified in Data Retention settings and is already stored in Hot or Warm storage.
3. Insufficient Hot or Warm Storage Space: Restore will not proceed if the operation would leave less than 5% of Hot or Warm storage space free after restoration.

What Happens After the Steps

After users open the Table & Data Storage page, vuSmartMaps displays tables across the selected storage tier.

Users can:

• View Hot, Warm, and Archived data tables.
• Search for specific tables.
• Check table size and modification time.
• Review data retention periods.
• View storage utilization and threshold status.
• Show or hide system tables.
• Restore archived data when needed.

After restoring archived data:

• The data is placed on the Hot or Warm disk from which it was originally archived.
• The restored data is moved to the main table only.
• The data becomes queryable again.
• The restored data remains available for the restore retention period.
• The default restore retention period is 7 days.
• The restored data is permanently deleted after the restore retention period ends.
• The restore retention value can be changed in the Data Retention module.

Before restoring archived data, check whether a restore is already running, whether data is already present in Hot or Warm storage, and whether enough storage space is available.

Troubleshooting

1. Issue: Archived data cannot be queried

   • Possible cause: Archived data is not directly queryable.
   • Solution: Restore the archived data first. After restoration, it is moved to the main table and becomes queryable again.

2. Issue: Warm or Archived storage does not show data

   • Possible cause: Warm or Archived storage may not be enabled.
   • Solution: If the storage is not enabled, the system displays: "Disk is disabled in this system."

3. Issue: Data is inaccessible even though storage is enabled

   • Possible cause: Storage is enabled, but the data cannot be accessed.
   • Solution: The actual error message is displayed by the system. Review the error message for more details.

4. Issue: Restore button cannot be used

   • Possible cause: A restore operation may already be in progress.
   • Solution: Wait for the current restore operation to complete before initiating another restore.

5. Issue: Restore does not proceed because data is already available

   • Possible cause: The same data is already present in Hot or Warm storage, or the data is still within the retention period configured in Data Retention settings.
   • Solution: Check the Hot and Warm tabs for the same table name before restoring archived data.

6. Issue: Restore does not proceed due to storage space

   • Possible cause: The restore operation would leave less than 5% free space in Hot or Warm storage.
   • Solution: Free up storage space or review storage capacity before attempting restore again.

7. Issue: Restored data disappeared after some time

   • Possible cause: The restored data is retained only for the restore retention period.
   • Solution: The default restore retention period is 7 days. Update the restore retention setting in the Data Retention module if a longer period is required.

8. Issue: System tables are not visible

   • Possible cause: System tables are hidden by default.
   • Solution: Select Show System Tables to display them.

FAQs

What is Table & Data Storage in vuSmartMaps?

Table & Data Storage helps administrators view and manage data stored in the HyperScale Data Store. It shows how data is organized across Hot, Warm, and Archived storage tiers.

What are Hot, Warm, and Archived storage?

• Hot Storage stores recent and critical data for fast access.
• Warm Storage stores less frequently accessed data that is still relevant.
• Archived Storage stores historical or rarely accessed data for long-term storage. Archived data cannot be queried directly and must be restored first.

Where can I access the Table & Data Storage page?

Go to: Data Management > Table & Data Storage This opens the Table & Data Storage page with separate tabs for Hot, Warm, and Archive.

What information is shown for each table?

The table list shows details such as the table name, size, modified time, and data retention period. In the Archive tab, archived data can also show Actual Size, Compressed Size, and a Restore action.

What does the Data Retention Period mean?

The Data Retention Period shows how long data remains in the current storage tier. After this period, data is removed or moved to another tier based on the configured retention policy.

Can archived data be queried directly?

No. Archived data cannot be queried directly. It must first be restored using the Restore option in the Archive tab.

What happens after archived data is restored?

The restored data is placed back on the original Hot or Warm disk from which it was archived. It is restored to the main table only, becomes part of the original dataset, and can be queried again.

How long does restored data remain available?

By default, restored data remains available for 7 days. This restore retention period can be changed in the Data Retention module. After the restore retention period ends, the restored data is permanently deleted.

When can restore not be initiated?

Restore cannot be initiated if a restore is already in progress, if the data is already present in Hot or Warm storage, or if restoring the data would leave less than 5% free space in Hot or Warm storage.

What is the purpose of Show System Tables?

Show System Tables displays internal platform tables along with regular data tables. By default, system tables are hidden because they are used internally by the platform.