Weighted Concept Drift SVM

Bibliography

1. Concept Drift description on Wikipedia
2. A. Tsymbal, The problem of concept drift: definitions and related work local
3. G. Forman, Tackling Concept Drift by Temporal Inductive Transfer local
4. M. Masud, Mining Concept-Drifting Data Stream to Detect Peer-to-Peer Botnet Traffic local
5. EDDM-IWKDDS-2006, M. Garcia, Early Drift Detection Method in The International Workshop on Knowledge Discovery from Data Stream, code and datasets
6. Weka, Single Classifier Drift
7. J. Gama, A Survey on Concept Drift Adaptation local
8. R. Polikar,Guest Editorial Learning in Nonstationary and Evolving Environments local content
9. I. Zliobaite et al., Active Learning with Drifting Streaming Data local
10. M. Wozniak et al., Active Learning Classification With Drifted Streaming Data local
11. S. Janardan et al., Concept drift in Streaming Data Classification: Algorithms, Platforms and Issues, overview
12. S. Zubin et al., Concept Drift Detection and Adaptation with Hierarchical Hypothesis Testing

Concept drift papers with code / benchmarks

1. A. Saffari et al., On-line Random Forests

• source code that works, with data
• presentation
• continuation, 2010, does not build

2. A.P. Cassidy, Calculating Feature Importance in Data Streams with Concept Drift using Online Random Forest

• got to it by this stackexchange topic

3. Datasets for concept drift

• J. Gama's workgroup, explore around for papers

Roadmap

Code

• Classic IDSVM
  • solution for two opposite patterns
  • insert/remove a pattern from the solution
  • compare g() for polynomial kernel
  • insert 500 USPS patterns (kernel)
  • remove 480 USPS patterns (from now on, kernel only)
  • adapt code to be device-independent (CUDA is faster)
  • reduce C from 5.0 to 1.0 and observe error vectors, repeating learn/unlearn 500 patterns
• Weighted IDSVM
  • make C individual for each pattern, adapt Migration and extend its unit test
  • also extend Migration unit test for lambda
  • vary C for a specific pattern on basic 2-vector solution, probing for SVs, EVs and RVs
  • define rectangular shift kernel and probe for a window of 20-40 patterns
  • define a linear shift kernel (timing info is needed here, check bibliography)
  • define an exponential shift kernel (same as above)
  • comparison with classic SVM
  • time comparison for Forest Covertype

Article

• add bibliography
• add latex template with todonotes package
• define main structure
• add theoretical considerations on incr/decr SVM
• refine for weighted SVM section
• final draft