1. Home
  2. ADASYN: Adaptive Synthetic Sampling Approach

ADASYN: Adaptive Synthetic Sampling Approach

ADASYN, an extension of the SMOTE technique, is also used in handling imbalanced datasets. ADASYN focuses on local densities of minority classes. It finds out the regions where the imbalance is very severe and applies the strategy to generate synthetic samples there. It generates more samples where the density is high and fewer samples where the density is low. This approach is highly useful in scenarios where class distribution varies across the feature space.

Working Procedure of ADASYN

  • Class Imbalance Ratios: The initial step is ADASYN is to calculate the ratio of minority class which is obtained by dividing the number of majority class samples by the number of minority class samples.
  • Finding density distribution: For every minority instance, we find its k-nearest neighbors. Then we find the distance between them using metrics like Manhattan distance or Euclidean distance. If the instances are surrounded by more nearby neighbors, then we consider the density to be higher else the density is considered to be low.
  • Sample generation ratio: Once both class imbalance ratio and density distribution are calculated, we compute the sample generation ratio. It finds out how many samples are to be generated for each minority class instance. For Higher densities and larger imbalanced instances, more synthetic samples are generated.
  • Generating synthetic samples: By combining the minority instances with their nearest neighbors, new samples are generated.
  • Balanced dataset creation: By combining the new synthetic samples with the original minority instances, the frequency of the minority classes increases. This makes the dataset balanced and helps the model to learn more accurately.

Python Implementation For ADASYN

Python 
from imblearn.over_sampling import ADASYN

# Applying ADASYN
adasyn = ADASYN(sampling_strategy='minority')
x_resampled, y_resampled = adasyn.fit_resample(x, y)
# Count outcome values after applying ADASYN
y_resampled.value_counts()

Output:

Outcome
1    500
0    500
Name: count, dtype: int64

SMOTE for Imbalanced Classification with Python

Imbalanced datasets impact the performance of the machine learning models and the Synthetic Minority Over-sampling Technique (SMOTE) addresses the class imbalance problem by generating synthetic samples for the minority class. The article aims to explore the SMOTE, its working procedure, and various extensions to enhance its capability. The article provides Python implementations for SMOTE and its extensions, offering a comprehensive guide to tackle the problem of Imbalanced datasets in Python.

Table of Content

  • Data Imbalance in Classification Problem
  • SMOTE : Synthetic Minority Over-Sampling Technique
  • Extensions of SMOTE
  • ADASYN: Adaptive Synthetic Sampling Approach
  • Borderline SMOTE
  • SMOTE-ENN (Edited Nearest Neighbors)
  • SMOTE- TOMEK Links
  • SMOTE-NC (Nominal Continuous)
  • SMOTE for Imbalanced Classification: When to Use

Similar Reads

Data Imbalance in Classification Problem

Data imbalance in classification refers to skewed class distribution, hindering machine learning models’ performance. Majority classes dominate while minority classes are underrepresented. This challenge arises when one category vastly outnumbers others. Techniques like Oversampling, Undersampling, Threshold moving, and SMOTE help address this issue. Handling imbalanced datasets is crucial to prevent biased model outputs, especially in multi-classification problems....

Synthetic Minority Over-Sampling Technique

The Synthetic Minority Over-Sampling Technique (SMOTE) is a powerful method used to handle class imbalance in datasets. SMOTE handles this issue by generating samples of minority classes to make the class distribution balanced. SMOTE works by generating synthetic examples in the feature space of the minority class....

Implementing SMOTE for Imbalanced Classification in Python

In this section, we’ll use Pima Indian Diabetes Dataset. In the following code snippet, we load the dataset and plot the class distribution....

Extensions of SMOTE Models

SMOTE effectively addresses data imbalance by generating synthetic samples, enriching the minority class and refining decision boundaries. Despite its benefits, SMOTE’s computational demands can escalate with larger datasets and high-dimensional feature spaces....

ADASYN: Adaptive Synthetic Sampling Approach

ADASYN, an extension of the SMOTE technique, is also used in handling imbalanced datasets. ADASYN focuses on local densities of minority classes. It finds out the regions where the imbalance is very severe and applies the strategy to generate synthetic samples there. It generates more samples where the density is high and fewer samples where the density is low. This approach is highly useful in scenarios where class distribution varies across the feature space....

Borderline SMOTE

Borderline SMOTE is designed to better address the issue of misclassification of minority class samples that are near the borderline between classes. These samples are often the hardest to classify and are more likely to be mislabeled by classifiers. Borderline SMOTE focuses on generating synthetic samples near the decision boundary between the minority and majority classes. It targets instances that are more challenging to classify, aiming to improve the generalization performance of classifiers....

SMOTE-ENN (Edited Nearest Neighbors)

SMOTE-ENN combines the SMOTE method with the Edited Nearest Neighbors (ENN) rule. ENN is used to clean the data by removing any samples that are misclassified by their nearest neighbors. This combination helps in cleaning up the synthetic samples, improving the overall quality of the dataset. The objective of ENN is to remove noisy or ambiguous samples, which may include both minority and majority class instances....

SMOTE- TOMEK Links

SMOTE+TOMEK links combine the SMOTE technique with TOMEK links, which are pairs of very close instances, but from opposite classes. By removing TOMEK links, instances that are close to each other but belong to different classes may be eliminated, which can help in reducing overlap between classes and improving the separability of the classes....

SMOTE-NC (Nominal Continuous)

SMOTE-NC is a variant of SMOTE that is suitable for datasets containing a mix of nominal (categorical) and continuous features. It modifies the SMOTE algorithm to correctly handle categorical data. The traditional SMOTE algorithm excels in generating synthetic samples to address class imbalance in datasets with only numerical features. However, when categorical features are present, applying SMOTE directly can be problematic. This is because SMOTE operates in the feature space, interpolating between instances based on their numerical attributes. Interpolating between categorical features is not meaningful and can lead to synthetic samples that do not accurately represent the original data....

SMOTE for Imbalanced Classification: When to Use

AlgorithmBest Use CaseStrengthsWhen to UseTraditional SMOTEGeneral imbalanced datasets where minority class enhancement is needed.Increases the number of minority class samples through interpolation, improving the generalization ability of classifiers.Use when your dataset is imbalanced but doesn’t have extreme noise or overlapping class issues. Suitable for straightforward augmentation needs.ADASYN (Adaptive Synthetic Sampling)Datasets where imbalance varies significantly across the feature space.Focuses on generating samples next to the original samples that are harder to learn, adapting to varying degrees of class imbalance.Use when certain areas of the feature space are more imbalanced than others, requiring adaptive density estimation.Borderline SMOTEDatasets where minority class examples are close to the decision boundary.Enhances classification near the borderline where misclassification risk is high.Use when data points from different classes overlap and are prone to misclassification, particularly in binary classification problems.SMOTE-NC (Nominal Continuous)Datasets that include a combination of nominal (categorical) and continuous features.Handles mixed data types without distorting the categorical feature space.Use when your dataset includes both categorical and continuous inputs, ensuring that synthetic samples respect the nature of both data types.SMOTE-ENN (Edited Nearest Neighbors)Datasets with potential noise and mislabeled examples.Combines over-sampling with cleaning to remove noisy and misclassified instances.Use when the dataset is noisy or contains outliers, and you want to refine the class boundary further after over-sampling.SMOTE+TOMEKBest for reducing overlap between classes after applying SMOTE.Cleans the data by removing Tomek links, which can help in enhancing the classifier’s performance.Use when you need a cleaner dataset with less overlap between classes, suitable for situations where class separation is a priority....

Conclusion

To sum up, SMOTE is an effective technique to handle imbalanced datasets. It finds the minority class in the dataset and generates synthetic samples for them. It thus helps in balancing data which makes the machine learning model better learn. It is widely used in classification problems. However, it is essential to carefully analyze the problem before applying the method, as sometimes it might lead to trade-offs. Overall, SMOTE plays a vital role in handling imbalance datasets....

FAQs on SMOTE for Imbalanced Classification

What is SMOTE?...

Categories

HTML CSS3 Bootstrap Foundation Sass JavaScript TypeScript JQuery AngularJS React NodeJS Highcharts PHP Python Python3 Django Linux Docker Ruby Java C C ++ Perl Servlet JSP Lua Scala Go ASP C # SQL MySQL SQLite MongoDB Redis Android Swift ionic Kotlin XML tutorial

Contact US