Basics
Monitoring and Analytics
Use Monitoring and Analytics to understand how your AI agent performs in real-world conversations.
Dashboards and reports help you track performance, detect issues early, and continuously improve your conversational experience.
Real-Time Monitoring
Keep track of what’s happening right now.
- Active Sessions – See all ongoing conversations in real time.
- Response Time – Measure how fast your agent replies to customers.
- Error Alerts – Get instant notifications about technical or logical errors.
- Conversation Volume – Monitor message traffic and identify usage peaks.
Tip: Use real-time dashboards during deployments or updates to confirm your agent is performing as expected.
Performance Metrics
Track key indicators that show how well your agent is working.
| Metric | Description | Why It Matters |
|---|---|---|
| Average Response Time | Time between customer message and agent reply | Indicates responsiveness |
| Session Duration | Typical length of a conversation | Helps spot overly long or stuck sessions |
| Resolution Rate | Percentage of conversations successfully completed | Reflects effectiveness |
| Customer Satisfaction | Inferred from sentiment and feedback | Measures perceived quality |
| Escalation Rate | How often sessions are handed to humans | Helps balance automation and support |
Reporting and Dashboards
The platform features a powerful analytical engine built on Apache Superset. This allows for sophisticated data visualization, high-speed reporting, and complete data isolation between different organizations.
Built-in Performance Dashboards
The default Dashboards section under Analytics provides standardized reports to help you assess the value of your AI operations. These dashboards are automatically filtered by your tenant_id, ensuring you only see your own data.
The section is divided into two main tabs:
- Agents: Tracks the performance, message volume, and resolution rates of your AI agents.
- Campaigns: Evaluates the effectiveness, reach, and business outcomes of your outbound campaigns.
Using Dashboard Filters
Standard dashboards include built-in filters to refine your view:
- Agent: Filter by one or multiple specific AI agents.
- Lookup Period: Define the date range for the data displayed (for example, Last quarter, Last month).
- Timegrain: Adjust the time grouping for trend charts (for example, Hourly, Day, Week, Month).
Don't forget to click Apply after changing the filters to update the charts.
💡 Tip: These dashboards are re-implemented in Superset for better performance and allow for seamless data exploration within the portal.
Conversation Quality Insights
Understand what customers feel and where your agent can improve.
- Sentiment Analysis – Detect positive, neutral, or negative tones.
- Goal Completion – Track if conversations meet defined objectives.
- Common Issues – Identify repeating customer problems or agent misunderstandings.
- Agent Comparison – Compare results across agent versions or configurations.
Reporting and Dashboards
The platform features a powerful analytical engine built on Apache Superset. This allows for sophisticated data visualization, high-speed reporting, and complete data isolation between different organizations.
Built-in Performance Dashboards
The default Dashboards section under Analytics provides standardized reports to help you assess the value of your AI operations. These dashboards are automatically filtered by your tenant_id, ensuring you only see your own data.
The section is divided into two main tabs:
- Agents: Tracks the performance, message volume, and resolution rates of your AI agents.
- Campaigns: Evaluates the effectiveness, reach, and business outcomes of your outbound campaigns.
Using Dashboard Filters
Filtering is handled directly within the embedded dashboard interface. You can adjust the following parameters at the top of the screen to refine your view:
- Lookup Period: Define the date range for the data displayed (for example, Last quarter, Last month).
- Timegrain: Adjust the time grouping for trend charts (for example, Day, Week, Month).
- Agent Name / Campaign Name: Choose specific agents or campaigns to analyze from the dropdown list.
Don't forget to click Apply after changing the filters to update the charts.
💡 Tip: Use these dashboards to support quarterly reviews, calculate ROI, and measure the real business value of your AI implementation.
Custom Dashboards (Embedded)
If standard reports cover general needs, Custom Dashboards allow you to integrate specialized analytical resources (such as Flametree Superset BI) directly into your workspace.
Managing Custom Dashboards
- Go to the Analytics section.
- Switch to the Custom tab.
Note: Standard filters (such as date range or agent selection) are disabled in this tab because it displays external resources that manage their own filtering.
- Click Add (+) to connect a new dashboard.
- Fill in the fields:
- Name: A display name for the dashboard (for example, "Q3 Financials").
- URL: The direct link to the external dashboard.
- Click Save.
The added dashboard will appear in the list as a clickable link, allowing quick access to custom BI reports without leaving the platform context.
Integration with Other Platform Features
Monitoring connects naturally with other parts of the Flametree platform.
Knowledge Base Analytics
- Search Trends – See which topics customers ask about most.
- Content Gaps – Identify missing or unclear information.
- Usage Patterns – Track which FAQ articles drive successful answers.
Workflow Optimization
- State Flow Analysis – Visualize how customers move between conversation states.
- Bottleneck Detection – Find points where conversations often stop.
- Flow Adjustments – Use insights to simplify transitions and improve flow logic.
Agent Configuration Improvement
- Behavior Analysis – Refine tone, empathy, or style based on customer reactions.
- Response Tuning – Update prompts or templates that lead to weak results.
- Performance Calibration – Adjust parameters to reduce latency or escalation.
Best Practices
Regular Monitoring
| Frequency | Focus |
|---|---|
| Daily | Check live sessions and critical alerts |
| Weekly | Review performance metrics and error rates |
| Monthly | Evaluate trends and identify workflow improvements |
| Quarterly | Perform full configuration and strategy review |
Quality Assurance
- Review random conversation samples for accuracy and tone.
- Analyze errors to find training or setup issues.
- Collect and act on customer feedback regularly.
Data Management
- Define data retention policies for compliance.
- Ensure access control to sensitive conversation logs.
- Maintain secure data backups to prevent loss.
Continuous Optimization
- Monitor response times and error rates closely.
- Use session logs to find weak spots in agent logic.
- Update the knowledge base when the agent frequently escalates to humans.
- Apply data-driven improvements for training and configuration.