Store: Effective Monitoring and Alerting

by Slawek Ligus


Read now


ISBN: 9781449333485

Published: Nov 26

Publisher: O'Reilly Media


Effective Monitoring and Alerting 2 >
SPECIAL OFFER: Upgrade this ebook with O’Reilly 3 >
Preface 4 >
Who Should Read This Book 5 >
Conventions Used in This Book 5 >
Using Code Examples 6 >
Safari® Books Online 6 >
How to Contact Us 7 >
Acknowledgements 8 >
1. Introduction 9 >
Monitoring, Alerting, and What They Can Do for You 9 >
Monitoring and Alerting in a Nutshell 13 >
The Challenges 19 >
Important Terms 21 >
2. Monitoring 23 >
The Building Blocks 23 >
Drawing Conclusions from Timeseries Plots 48 >
3. Alerting 61 >
The Challenge 61 >
Prerequisites 62 >
Understanding Failure and Its Impact 63 >
Anatomy of an Alarm 69 >
Case Study: A Data Pipeline 77 >
Types of Alerts 79 >
Setting Up Alarms 81 >
Alerting Suggestions 93 >
4. At Scale 95 >
Implications of Scale 95 >
Composition of Large-Scale Systems 97 >
Commonalities of Large-Scale Alerting Configurations 99 >
Monitoring Coverage 100 >
Managing Large Alerting Configurations 103 >
5. Monitoring in System Automation 121 >
Choosing Appropriate Maintenance Times Automatically 122 >
Controlling the Rate of Upgrade 123 >
Recovery-Oriented Admission Control 125 >
Automated Deployment and Rollback 128 >
6. The Work Environment 130 >
Keeping an Audit Trail 130 >
Working with Tickets 131 >
Dealing with Anomalies 137 >
Learning from Outages 137 >
Using Checklists 138 >
Creating Dashboards 139 >
Service-Level Agreements 140 >
Preventing the Ironies of Automation 140 >
Culture 141 >
7. Measuring Success 142 >
The Feedback Loop 142 >
Ticket Reporting 146 >
Measuring Detectability 149 >
Transition to Automated Alarms 153 >
Maintenance Overhead 153 >
How (Not) to Measure 154 >
8. The Principles 155 >
Get in the Habit of Measuring 155 >
Draw Conclusions Reliably 155 >
Monitor Extensively 156 >
Alarm Selectively 156 >
Work Smart, Not Hard 157 >
A. Setting Up OpenTSDB 161 >
The Software 161 >
First Steps 162 >
Gathering Data System-Wide 170 >
Timeseries Plots 173 >
Get Involved 174 >
About the Author 175 >
SPECIAL OFFER: Upgrade this ebook with O’Reilly 176 >
Copyright 177 >