Products Solutions Templates Pricing Resources

Anomaly detection: automatically spotting unusual changes in your data

Home / Everything About / Everything About Analytics / Anomaly detection: automatically spotting unusual changes in your data

Your conversion rate drops 15% on Tuesday. You don't notice until Thursday when you check your dashboard. Two days of lost revenue, and you still don't know why. An anomaly detection system would have alerted you Wednesday morning: "Conversion down 15% yesterday, mobile checkout affected." Now you investigate immediately. You find a deploy broke mobile payments. You fix it that afternoon. Anomaly detection is the difference between finding problems days later and finding them hours later.

Anomaly detection is a system that watches your metrics continuously and alerts you when something unusual happens. It doesn't wait for you to check the dashboard. It knows your baseline (what "normal" looks like), compares every day to that baseline, and flags deviations. A traffic spike, a conversion drop, engagement declining—these get caught automatically and surfaced before they become disasters.

What does anomaly detection actually do?

Take any metric and you'll find the same pattern: it fluctuates naturally. Conversion rate is never exactly the same every day. Traffic varies by day of week. Mobile converts differently than desktop. If you set an alert at "conversion below 2.5%", you'll get false alarms because conversion naturally dips to 2.4% some days. Anomaly detection solves this by learning what normal variation looks like, then alerting only when something deviates beyond that natural range.

The system learns your baseline from historical data. It answers: for this metric, on this day of week, with this traffic volume, what's the expected range? Wednesday conversions are typically between 2.1% and 2.9%. Mobile conversion is typically between 1.8% and 2.4%. Email traffic typically converts between 4% and 6%. The system builds a model of normal behavior for each combination of conditions.

When a new data point comes in, the system compares it to the expected range. Is today's conversion within the normal Wednesday range? Yes, it's 2.5%, totally normal. Is tomorrow's conversion within the normal Thursday range? No, it's 1.2%, that's an anomaly. Alert the team. They investigate and find the mobile checkout broke. Without anomaly detection, they discover this Friday afternoon. With it, they know Wednesday morning.

Different types of anomalies trigger different alerts. A sudden spike (traffic 200% above normal) is flagged. A sudden drop (conversion 50% below normal) is flagged. A gradual decline (conversion slowly dropping over a week) is flagged. A recurring pattern that suddenly breaks (mobile always converts 20% better than desktop, but this week it's equal) is flagged. The system watches for all of these.

Why does catching problems early matter so much?

Look at how most brands discover problems and you'll see the pattern: reactive discovery. A customer emails complaining checkout is broken. You check your dashboard and confirm it. Or you notice Thursday that Monday was weird and try to figure out what happened. This is hours or days of lost opportunity.

At scale, the cost of delay compounds. An e-commerce site with $1 million daily revenue losing 15% conversion to a bug costs $150,000 per day. Two days of delay costs $300,000. If anomaly detection catches it 6 hours earlier, you save $37,500. A SaaS site with 5,000 users seeing 2% additional churn costs 100 users. Two weeks of delay means 1,400 lost users. Catching it immediately means 100. The difference is thousands of dollars in ARR.

Speed also affects reputation. A customer service team wants to know about problems before customers report them. With anomaly detection, you can email customers: "We detected an issue on your account Wednesday morning and fixed it by afternoon." Without it, customers complain Friday, you say "we'll look into it", and they've already switched to a competitor. The psychology of "we spotted it" vs. "you reported it" changes customer perception.

Speed enables experimentation. If you deploy a change and can see the impact within hours, you can iterate faster. Without anomaly detection, you run an experiment for a week, check results, then iterate. With it, you can see if a change helped or hurt within 24 hours. That speed compounds over a year.

What kinds of anomalies should you actually catch?

Not every anomaly is worth catching. Some variations are expected. But some matter for business decisions. The key is setting up anomaly detection for metrics that, if they change unexpectedly, require immediate action.

Revenue-impacting anomalies are priority one. Conversion rate dropping, average order value declining, checkout abandonment spiking—these directly affect revenue. Alert on all of them. If conversion drops 10%, you want to know immediately, not after the weekend.

User engagement anomalies matter because they predict problems. Session duration dropping, feature adoption declining, login frequency decreasing—these are early warnings of churn. A user engagement drop this week predicts a retention drop next month. Catch it now and you have time to intervene.

Traffic anomalies matter because they reveal problems with reach or acquisition. Organic traffic spiking means your SEO is working (good). Organic traffic dropping means something broke in search (bad). Paid traffic declining means your ad campaigns are underperforming (bad). Alert when traffic sources change unexpectedly.

Quality anomalies matter because they're silent killers. Bounce rate increasing means your site is becoming less relevant. Pages per session declining means users find what they need faster (good) or can't find what they want (bad). Session errors spiking means site is breaking. These are invisible if you're not watching closely.

How do you set up anomaly detection without false alarms?

Ask any analytics team about false alarms and you'll hear the same complaint: alert fatigue. If every small fluctuation triggers an alert, the team stops reading them. They get desensitized. Real problems get lost in the noise. The key to good anomaly detection is setting thresholds that catch real problems while ignoring natural variation.

Start with the metrics that matter most to your business. Not every metric needs anomaly detection. Focus on revenue metrics (conversion, order value, churn), user health metrics (engagement, retention, feature adoption), and system health metrics (errors, latency, uptime). These are the ones that require immediate action if they change.

Set sensitivity based on volatility. Some metrics are naturally volatile. Traffic bounces around day to day. Set anomaly thresholds wide enough to absorb this natural variation. For conversion rate, a 20% deviation might be normal; flag only if it deviates 30% or more. For revenue, a 5% deviation might be normal. For engagement, a 15% deviation might be normal. The thresholds depend on your historical data.

Build in context. Day of week matters. Saturday usually has different patterns than Wednesday. Holidays have different patterns than regular days. Anomaly detection should account for this. "Conversion below 1.5% on a Sunday" might be normal. "Conversion below 1.5% on a Wednesday" is an anomaly. Good systems learn these patterns automatically.

Start conservative and tighten over time. If you're unsure about thresholds, set them wide. Better to miss some anomalies than flood the team with false alarms. As you build confidence in the system and the team gets used to reading alerts, tighten the thresholds and catch smaller anomalies.

What are the limits of anomaly detection?

Anomaly detection finds deviations from normal. But "normal" is whatever your historical data says is normal. If your data is wrong or biased, the system will miss real problems or flag false positives. If you had a bug for three months that artificially inflated your conversion rate, the system learned that inflated rate as "normal." When you fix the bug, conversion drops to true-normal levels, and the system flags it as an anomaly.

Anomaly detection doesn't explain causation. It tells you something changed. It doesn't tell you why. If conversion dropped, you still need to investigate. Is it a technical problem? A market change? A competitor's move? Competition from a new entrant? Anomaly detection surfaces the problem. Diagnosis is still human work.

Anomaly detection doesn't work well at very small scale. If you have 100 visitors per day, random variation is huge. Every day looks anomalous compared to the others. Anomaly detection works best at larger scale where patterns are stable. A SaaS with 50,000 monthly users has clean, stable patterns. A SaaS with 500 monthly users has too much noise.

Gradual changes can hide from anomaly detection. If conversion is slowly declining (down 0.5% per week), the system might not flag it because each week only deviates slightly from the previous week. You might not notice until you're down 10% total. To catch gradual degradation, you need trend analysis in addition to point-in-time anomaly detection.