Overview
A Mango system left unmanaged will eventually consume all available disk space. Four categories account for the vast majority of storage growth: NoSQL time-series data, backup archives, event messages, and log files. This guide explains how to monitor each consumer, configure automated purge settings, and perform manual cleanup when needed.
The Four Disk Consumers
| Consumer |
Default Location |
Growth Pattern |
| NoSQL time-series data |
databases/mangoTSDB/ |
Largest \u2014 scales with point count and logging frequency |
| Backup files |
backup/ |
Grows with each scheduled or manual backup |
| Event messages |
SQL database (events table) |
Grows with alarm frequency and detector count |
| Log files |
logs/ |
Grows with log verbosity; rotates but old files accumulate |
Approximate Sizing for Time-Series Data
As a planning guide, 200 million point values occupy approximately 4 GB of NoSQL storage. Actual size varies with data type and value distribution.
| Points |
Logging Interval |
Values per Day |
Approx. Monthly Storage |
| 100 |
10 seconds |
864,000 |
~70 MB |
| 1,000 |
30 seconds |
2,880,000 |
~700 MB |
| 5,000 |
10 seconds |
43,200,000 |
~3.5 GB |
| 10,000 |
5 seconds |
172,800,000 |
~14 GB |
WARNING:
These are estimates. Binary and multistate points consume less storage than numeric points. High-precision floating-point values consume more.
Configuring Purge Settings
Purge settings control how long historical point values are retained before automatic deletion. They can be set at three levels, with the most specific level taking priority.
System-Level Purge (Default for All Points)
- Go to Administration > System Settings > Purge
- Set the Default Purge Period (e.g., 1 Year)
- Configure the Purge Run Time \u2014 typically overnight during low-activity hours (e.g., 3:00 AM)
Mango runs the purge task daily at the configured time and removes point values older than the retention period.
Data Source-Level Purge (Override per Source)
Override the system default for all points belonging to a specific data source:
- Open the data source's Edit page
- Enable Purge Override
- Set a custom period (e.g., 6 Months for high-frequency diagnostic data sources)
All data points in this source inherit the override unless they have their own point-level setting.
Data Point-Level Purge (Override per Point)
The most granular control \u2014 set retention on individual data points:
- Open the data point's Properties or Logging tab
- Under Purge Settings, enable the override
- Set the desired retention period
TIP:
Use a tiered strategy: set a conservative system default (e.g., 1 year), extend retention on critical points individually (e.g., energy meters to 5 years), and shorten retention on high-frequency diagnostic points (e.g., 30 days).
NoSQL Shard File Structure
The NoSQL time-series database organizes data into shard files, each covering a fixed time period. The shard directory typically looks like:
databases/mangoTSDB/
data_2023-01.dat
data_2023-02.dat
...
data_2024-11.dat
data_2024-12.dat
Each file contains time-series data for all points during that time window. Older files can be identified by their date-based filenames.
Manual Shard Deletion by Date
When disk space is critically low and you need to free space immediately, you can manually delete old shard files:
- Stop Mango \u2014 never delete shard files while Mango is running
sudo systemctl stop mango
- Identify and remove shards older than your desired cutoff:
# Remove all 2022 and older shards
rm /opt/mango/databases/mangoTSDB/data_2021-*.dat
rm /opt/mango/databases/mangoTSDB/data_2022-*.dat
- Restart Mango:
sudo systemctl start mango
DANGER:
Manual shard deletion is irreversible. The historical data in deleted files cannot be recovered. Double-check file dates before deleting and ensure you have a backup if the data may be needed later.
Built-In Disk Space Monitoring
Mango includes internal data points that monitor host system resources. Use them to set up proactive alerts before the disk fills.
- Locate the Internal Metrics or System Information data source (created automatically)
- Enable the Partition Free Space or Disk Free Bytes data point for the Mango partition
- Create an Event Detector with a low-limit threshold (e.g., alarm when free space drops below 2 GB)
- Attach an Event Handler (email, Slack, or other) to get notified
This gives your team early warning and time to act before a disk-full condition causes downtime.
Event Message Purge
Event messages (alarms, system events, audit trail entries) accumulate in the SQL database events table and can grow to millions of rows on busy systems.
Configure Event Purge
- Go to Administration > System Settings > Purge
- Set the Event Purge Period (e.g., 90 days)
- Optionally configure different retention for different alarm levels
Counting Events with the SQL Console
To check how many events are stored:
- Go to Administration > SQL Console (superadmin access required)
- Run:
SELECT COUNT(*) FROM events;
For a breakdown by alarm level:
SELECT alarmLevel, COUNT(*) AS total
FROM events
GROUP BY alarmLevel
ORDER BY total DESC;
TIP:
If the event count is in the millions, consider reducing the event purge period or tuning noisy event detectors to reduce alarm volume at the source.
Log File Management
Mango rotates log files automatically via Log4j, but old rotated files can accumulate over time.
# Check total log directory size
du -sh /opt/mango/logs/
# Remove rotated logs older than 30 days
find /opt/mango/logs/ -name "*.log.*" -mtime +30 -delete
If logs are growing too fast, review the logging configuration in classes/log4j2.xml. Avoid running DEBUG or TRACE level logging in production \u2014 INFO is appropriate for most deployments.
Troubleshooting
Disk full \u2014 Mango will not start
- Free space immediately by deleting old backup files from
backup/ or old rotated logs from logs/ (these are safe to remove without stopping Mango)
- If NoSQL shards must be deleted, stop Mango first, then remove the oldest shard files
Purge task does not seem to run
- Verify the purge schedule in System Settings and confirm the configured time has passed
- Check
ma.log for purge-related messages or errors
Disk usage growing despite purge settings
- Data points set to "All data" logging at high frequency (e.g., every second) generate enormous data volumes
- Switch high-frequency points to "Interval" logging with a longer period (e.g., log one value per minute)
- Review whether all points truly need to be logged \u2014 some diagnostic points can be set to "Do not log"
