New proposals and applications of echo state networks

Author(s)

Levy Boccato

• Document type: Doctoral Thesis
• Press: Campinas, SP.
• Institution: Universidade Estadual de Campinas (UNICAMP). Faculdade de Engenharia Elétrica e de Computação
• Defense date: 2013-04-07

Examining board members

Romis Ribeiro de Faissol Attux; Guilherme de Alencar Barreto; Ricardo Suyama; Fernando José Von Zuben; Leandro Nunes de Castro Silva

• Advisor: Romis Ribeiro de Faissol Attux

Abstract

Echo state networks (ESNs) are recurrent structures capable of allying dynamic processing to a relatively simple training process, which amounts to adapting the coefficients of the linear combiner at the output in the minimum mean squared error (MSE) sense, while the connection weights in the dynamical reservoir are adjusted in advance and remain fixed. The present thesis deals with the main elements that characterize ESNs and proposes: (i ) a unification between reservoir computing approaches, such as ESNs and liquid state machines (LSMs), and extreme learning machines (ELMs), under the general term of unorganized machines, which establishes a connection with the pioneering connectionist ideas of Alan Mathison Turing; (ii ) a novel ESN architecture whose output layer is composed of a Volterra filter and of a compression stage based on Principal Component Analysis (PCA); (iii ) the use of information-theoretic learning criteria and those based on Lp norms instead of the MSE criterion for the adaptation of the parameters of the ESN output layer; and (iv) an unsupervised strategy for designing the recurrent layer of ESNs based on lateral interactions, modeled according to the mexican hat function, and on the self-organization of the input weights. The proposals developed in this work are analyzed through simulations in the context of different information processing problems, such as channel equalization, source separation and time series prediction (AU)