Data stream analysis and evaluation under the presence of verification latency

• Grant number: 15/01701-8
• Support Opportunities: Scholarships abroad - Research Internship - Master's degree
• Start date: May 26, 2015
• End date: October 25, 2015
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Gustavo Enrique de Almeida Prado Alves Batista
• Grantee: Denis Moreira dos Reis
• Supervisor: Peter A. Flach
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Institution abroad: University of Bristol, England
• Associated to the scholarship: 14/12333-7 - Data stream classification with concept drift and extreme verification latency, BP.MS

Abstract

Machine Learning has proven its importance by its rapidly growth in number of published works and consequently maturity. However, although there is a crescent concern about correctly assessing new propositions, the currently used evaluation methodology can be not enough to compare data stream classifiers for real world scenarios. Evaluation, nowadays, is limited to the case where, once the assessed algorithm classified an example, it immediately obtains the information of its true class. The exclusive analysis with such constraint, hardly achieved by real world applications, hides severe behavioral changes that are caused by small delays in the obtainment of the true classes, also called verification latencies. This work proposes an extended study about the implications of verification latency in the state-of-the-art classifiers and the need of new evaluation considerations for future works. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

DOS REIS, DENIS; FLACH, PETER; MATWIN, STAN; BATISTA, GUSTAVO; ASSOC COMP MACHINERY. Fast Unsupervised Online Drift Detection Using Incremental Kolmogorov-Smirnov Test. KDD'16: PROCEEDINGS OF THE 22ND ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, v. N/A, p. 10-pg., 2016-01-01. (15/01701-8, 13/50379-6, 14/12333-7)