In this thesis, background theory about the online kernel-based algorithms and their use for online learning is presented. The analysis of the state-ofthe- art methods highlights an important drawback in many kernel online learning algorithms. This is the large memory storage needed due to the amount of support vectors generated. We study the SCA approach for reducing support vectors in the batch learning case and propose its adaptation to the online scenario. POLSCA is the algorithm proposed for solving the addressed problems that online learning presents. The proposed algorithm is constructed by merging the concepts of Primal formulation of the optimization problem, online learning with stochastic subgradient descent solver(PEGASOS) and the support vector reduction method SCA.