| name | detecting-data-anomalies |
|---|---|
| description | Process identify anomalies and outliers in datasets using machine learning algorithms. Use when analyzing data for unusual patterns, outliers, or unexpected deviations from normal behavior. Trigger with phrases like "detect anomalies", "find outliers", or "identify unusual patterns". |
| allowed-tools | Read, Bash(python:*), Grep, Glob |
| version | 1.0.0 |
| author | Jeremy Longshore <[email protected]> |
| license | MIT |
Detecting Data Anomalies
Overview
This skill provides automated assistance for the described functionality.
Prerequisites
Before using this skill, ensure you have: - Dataset in accessible format (CSV, JSON, or database) - Python environment with scikit-learn or similar ML libraries - Understanding of data distribution and expected patterns - Sufficient data volume for statistical significance - Knowledge of domain-specific normal behavior - Data preprocessing capabilities for cleaning and scaling
Instructions
- Load dataset using Read tool
- Inspect data structure and identify relevant features
- Clean data by handling missing values and inconsistencies
- Normalize or scale features as appropriate for algorithm
- Split temporal data if time-series analysis is needed
- Apply selected algorithm using Bash tool
- Generate anomaly scores for each data point
- Classify points as normal or anomalous based on threshold
- Extract characteristics of identified anomalies
See {baseDir}/references/implementation.md for detailed implementation guide.
Output
- Total data points analyzed
- Number of anomalies detected
- Contamination rate (percentage of anomalies)
- Algorithm used and configuration parameters
- Confidence scores for detected anomalies
- Record identifier and timestamp (if applicable)
Error Handling
See {baseDir}/references/errors.md for comprehensive error handling.
Examples
See {baseDir}/references/examples.md for detailed examples.
Resources
- Isolation Forest documentation and implementation examples
- One-Class SVM for novelty detection
- Local Outlier Factor (LOF) for density-based detection
- Autoencoder-based anomaly detection for deep learning approaches
- scikit-learn anomaly detection module