Skip to main content

Monitoring

Comprehensive monitoring is essential for maintaining Data Formulator in production environments. The system provides built-in metrics collection, alerting capabilities, and integration with popular monitoring tools.

Built-in Monitoring​

Data Formulator includes a web-based monitoring dashboard accessible at http://localhost:8080/monitor. The dashboard displays real-time metrics including:

  • Processing throughput and latency

  • System resource utilization

  • Error rates and failed job counts

  • Queue depths and processing backlogs

Log Management​

Logs are written to /var/log/dataformulator/ and include application logs, access logs, and error logs. Configure log levels in the main configuration file:


Bash
sudo tail -f /var/log/dataformulator/application.log

External Monitoring Integration​

For Ubuntu systems, integrate with Prometheus:


Bash
sudo apt install -y prometheus-node-exporter

For RHEL8, use the DNF package manager:​


Bash

sudo dnf install -y golang-github-prometheus-node-exporter

Configure Grafana dashboards for visualization and set up alerting rules based on your operational requirements.