Concept Drifts
Under non-stationary environments, machine learning models must be constantly updated, and changes must be detected to make the models compatible with the current environment conditions. These changes that force the model’s updates are known as Concept Drifts. A model that cannot cope with such changes will have its performance degraded over time.
For example, a model must be constantly updated when dealing with fraud detection since attackers are constantly creating new fraud approaches.
In these scenarios, both model updates and predictions must be made in real-time due to time constraints and the fast arrival of samples.
Current Results
The Dynse Framework.
The Dynse Framework is able to adapt dynamic selection methods to deal with non-stationary environments containing concept drifts. It achieves state-of-the-art results in well-known benchmarks, and it is publicly available at https://github.com/paulorla/dynse
Publications about the Dynse
Distance Function and Normalization Under Stream Scenarios
Measuring the distance when no normalization is possible is a challenge since we do not know the exact range of the feature space. For data streams, if we normalize the data, we should consider strategies that we can reply in real environments, such as using the statistics of a recent batch, or to use a sample of the dataset. Furthermore, maybe the normalization is not mandatory for data streams. Giving information to our Machine Learning models that we are not sure of — i.e., the range of the data — is not the best approach.
One example is the temperature, which varies a lot throughout the year. Using the statistics of the summer to normalize data from the winter is not valid. In the paper Distance Function and Normalization Under Stream Scenarios, we conclude that the non-normalization of data leads is a good strategy under concept drifting streams.
We also conclude that the Canberra and the Manhattan Distance functions were the best performing ones when the range of the features varied with time.
