One-Class Support Vector Machine
Updated: September 20, 2017
Creates a one class Support Vector Machine model for anomaly detection
Category: Anomaly Detection
This article describes how to use the One-Class Support Vector Model module in Azure Machine Learning to create an anomaly detection model.
This module is particularly useful in scenarios where you have a lot of "normal" data and not many cases of the anomalies you are trying to detect. For example, if you need to detect fraudulent transactions, you might not have many examples of fraud that you could use to train a typicaly classification model, but you might have many examples of good transactions.
You use the One-Class Support Vector Model module to create the model, and then train the model using the Train Anomaly Detection Model. The datset that you use for training can contain all or mostly normal cases.
You can then apply different metrics to identify potential anomalies. For example, you might use a large dataset of good transactions to identify cases that that possibly represent fraudulent transactions.
The One-Class Support Vector Model module creates a kernel-SVM model, which means that it is not very scalable. If training time is limited, or you have too much data, you can use other methods for anomaly detectors, such as PCA-Based Anomaly Detection. |
Understanding One-Class SVM
Support vector machines (SVMs) are supervised learning models that analyze data and recognize patterns, and that can be used for both classification and regression tasks.
Typically, the SVM algorithm is given a set of training examples labeled as belonging to one of two classes. The SVM algorithm represents the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to one category or another, based on which side of the gap they fall on.
Sometimes oversampling is used to replicate the existing samples so that you can create a two-class model, but it is impossible to predict all the new patterns of fraud or system faults from limited examples. Moreover, collection of even limited examples can be expensive.
Therefore, in one-class SVM, the support vector model is trained on data that has only one class, which is the “normal” class. It infers the properties of normal cases and from these properties can predict which examples are unlike the normal examples. This is useful for anomaly detection because the scarcity of training examples is what defines anomalies: that is, typically there are very few examples of the network intrusion, fraud, or other anomalous behavior.
For more information, including links to basic research, see the Technical Notes section.
Add the One-Class Support Vector Model module to your experiment. YOu can find the module in Azure Machine Learning Studio under Machine Learning - Initialize, and then Anomaly Detection.
In the Properties pane of the One-Class Support Vector Model module, select an option from the Create trainer mode list.
This is an important option that specifies how the model should be trained, and must be set before any other parameters.
Single Parameter
If you know how you want to configure the model, you can provide a specific set of values as arguments. You might have learned these values by experimentation or received them as guidance.
Parameter Range
If you are not sure of the best parameters, you can find the optimal parameters by specifying multiple values and using a parameter sweep to find the optimal configuration.
Tune Model Hyperparameters will iterate over all possible combinations of the settings you provided and determine the combination of settings that produces the optimal results.
For η, type a value that represents the upper bound on the fraction of outliers.
This parameter, which corresponds to the nu-property described in this paper, lets you control the trade-off between outliers and normal cases.
For ε (epsilon), type a value to use as the stopping tolerance.
The stopping tolerance, affects the number of iterations used when optimizing the model, and depends on the stopping criterion value. When the value is exceeded, the models stops iterating on the solution.
Connect one of the modules used for training, together with a tagged dataset.
- If you set the Create trainer mode option to Single Parameter, train the model by using the Train Model module.
- If you set the Create trainer mode option to Parameter Range, train the model using the Tune Model Hyperparameters module.
See the Technical Notes section for more information about the default ranges for each of these options.
Run the experiment.
Results
The module returns a trained anomaly detection model. You can either save the model in your workspace, or you can connect the Score Model module and use the trained model to detect possible anomalies.
If you trained the model using a parameter sweep, make a note of the optimal parameter settings to use when configuring a model for use in production.
For examples of how this module is used in anomaly detection, see this sample experiment in the Model Gallery:
- The Anomaly Detection: Credit Risk sample illustrates how to find outliers in data. This example uses a parameter sweep to find the optimal model. It then applies that model to new data to identify risky transactions that might represent fraud, comparing two different anomaly detection models.
Predictions from the One-Class SVM are uncalibrated scores that may be possibly unbounded. As the example in the Cortana Intelligence Gallery demonstrates, be sure to normalize scores if you are comparing models based on different algorithms.
Research
This implementation wraps the library for support vector machines named libsvm. The general theory on which libsvm is based, and the approach towards one-class support vector machines, is described in these papers by B. Schӧlkopf et al.:
Configuration Tips
- If you pass a parameter range to Train Model, it will use only the first value in the parameter range list.
- If you pass a single set of parameter values to the Tune Model Hyperparameters module, when it expects a range of settings for each parameter, it ignores the values and using the default values for the learner.
- If you select the Parameter Range option and enter a single value for any parameter, that single value you specified will be used throughout the sweep, even if other parameters change across a range of values.
| Name | Type | Range | Optional | Description | Default |
|---|---|---|---|---|---|
| Create trainer mode | Create Trainer Mode | List:Single Parameter|Parameter Range | Required | Single Parameter | Specify learner options. Use the SingleParameter option to manually specify all values. Use the ParameterRange option to sweep over tunable parameters. |
| nu | Float | >=double.Epsilon | mode:Single Parameter | 0.1 | This parameter (represented by the Greek letter nu) determines the trade-off between the fraction of outliers and the number of support vectors. |
| epsilon | Float | >=double.Epsilon | mode:Single Parameter | 0.001 | Specifies the stopping tolerance. |
| psnu | ParameterRangeSettings | [0.001;1.0] | mode:Parameter Range | 0.001; 0.01; 0.1 | Specifies the range for the trade-off between the fraction of outliers and the number of support vectors. |
| psEpsilon | ParameterRangeSettings | [1e-6;1.0] | mode:Parameter Range | 0.001; 0.01; 0.1 | Specifies the range for stopping tolerance. |
| Name | Type | Description |
|---|---|---|
| Untrained model | ILearner interface | An untrained anomaly detection model |