No (296)
Used filters
(4)
Number of found documents:
296
Published from
to
A Hybrid Method for Nonlinear Least Squares that Uses Quasi-Newton Updates Applied to an Approximation of the Jacobian Matrix
Lukšan, Ladislav; Vlček, Jan
2019 - English
In this contribution, we propose a new hybrid method for minimization of nonlinear least squares. This method is based on quasi-Newton updates, applied to an approximation A of the Jacobian matrix J, such that AT f = JT f. This property allows us to solve a linear least squares problem, minimizing ∥Ad+f∥ instead of solving the normal equation ATAd+JT f = 0, where d ∈ Rn is the required direction vector. Computational experiments confirm the efficiency of the new method.
Keywords:
nonlinear least squares; hybrid methods; trust-region methods; quasi-Newton methods; numerical algorithms; numerical experiments
Available at various institutes of the ASCR
A Hybrid Method for Nonlinear Least Squares that Uses Quasi-Newton Updates Applied to an Approximation of the Jacobian Matrix
In this contribution, we propose a new hybrid method for minimization of nonlinear least squares. This method is based on quasi-Newton updates, applied to an approximation A of the Jacobian matrix J, ...
Application of the Infinitely Many Times Repeated BNS Update and Conjugate Directions to Limited-Memory Optimization Methods
Vlček, Jan; Lukšan, Ladislav
2019 - English
To improve the performance of the L-BFGS method for large scale unconstrained optimization, repeating of some BFGS updates was proposed. Since this can be time consuming, the extra updates need to be selected carefully. We show that groups of these updates can be repeated infinitely many times under some conditions, without a noticeable increase of the computational time. The limit update is a block BFGS update. It can be obtained by solving of some Lyapunov matrix equation whose order can be decreased by application of vector corrections for conjugacy. Global convergence of the proposed algorithm is established for convex and sufficiently smooth functions. Numerical results indicate the efficiency of the new method.
Keywords:
unconstrained minimization; limited-memory variable metric methods; the repeated Byrd-Nocedal-Schnabel update; the Lyapunov matrix equation; the conjugate directions; global convergence; numerical results
Available at various institutes of the ASCR
Application of the Infinitely Many Times Repeated BNS Update and Conjugate Directions to Limited-Memory Optimization Methods
To improve the performance of the L-BFGS method for large scale unconstrained optimization, repeating of some BFGS updates was proposed. Since this can be time consuming, the extra updates need to be ...
How to down-weight observations in robust regression: A metalearning study
Kalina, Jan; Pitra, Zbyněk
2018 - English
Metalearning is becoming an increasingly important methodology for extracting knowledge from a data base of available training data sets to a new (independent) data set. The concept of metalearning is becoming popular in statistical learning and there is an increasing number of metalearning applications also in the analysis of economic data sets. Still, not much attention has been paid to its limitations and disadvantages. For this purpose, we use various linear regression estimators (including highly robust ones) over a set of 30 data sets with economic background and perform a metalearning study over them as well as over the same data sets after an artificial contamination. We focus on comparing the prediction performance of the least weighted squares estimator with various weighting schemes. A broader spectrum of classification methods is applied and a support vector machine turns out to yield the best results. While results of a leave-1-out cross validation are very different from results of autovalidation, we realize that metalearning is highly unstable and its results should be interpreted with care. We also focus on discussing all possible limitations of the metalearning methodology in general.
Keywords:
metalearning; robust statistics; linear regression; outliers
Available on request at various institutes of the ASCR
How to down-weight observations in robust regression: A metalearning study
Metalearning is becoming an increasingly important methodology for extracting knowledge from a data base of available training data sets to a new (independent) data set. The concept of metalearning is ...
The Computational Power of Neural Networks and Representations of Numbers in Non-Integer Bases.
Šíma, Jiří
2017 - English
We briefly survey the basic concepts and results concerning the computational power of neural networks which basically depends on the information content of weight parameters. In particular, recurrent neural networks with integer, rational, and arbitrary real weights are classified within the Chomsky and finer complexity hierarchies. Then we refine the analysis between integer and rational weights by investigating an intermediate model of integer-weight neural networks with an extra analog rational-weight neuron (1ANN). We show a representation theorem which characterizes the classification problems solvable by 1ANNs, by using so-called cut languages. Our analysis reveals an interesting link to an active research field on non-standard positional numeral systems with non-integer bases. Within this framework, we introduce a new concept of quasi-periodic numbers which is used to classify the computational power of 1ANNs within the Chomsky hierarchy.
Keywords:
neural network; Chomsky hierarchy; beta-expansion; cut language
Available at various institutes of the ASCR
The Computational Power of Neural Networks and Representations of Numbers in Non-Integer Bases.
We briefly survey the basic concepts and results concerning the computational power of neural networks which basically depends on the information content of weight parameters. In particular, recurrent ...
The Computational Power of Neural Networks and Representations of Numbers in Non-Integer Bases
Šíma, Jiří
2017 - English
We briefly survey the basic concepts and results concerning the computational power of neural networks which basically depends on the information content of weight parameters. In particular, recurrent neural networks with integer, rational, and arbitrary real weights are classified within the Chomsky and finer complexity hierarchies. Then we refine the analysis between integer and rational weights by investigating an intermediate model of integer-weight neural networks with an extra analog rational-weight neuron (1ANN). We show a representation theorem which characterizes the classification problems solvable by 1ANNs, by using so-called cut languages. Our analysis reveals an interesting link to an active research field on non-standard positional numeral systems with non-integer bases. Within this framework, we introduce a new concept of quasi-periodic numbers which is used to classify the computational power of 1ANNs within the Chomsky hierarchy.
Keywords:
neural network; Chomsky hierarchy; beta-expansion; cut language
Available at various institutes of the ASCR
The Computational Power of Neural Networks and Representations of Numbers in Non-Integer Bases
We briefly survey the basic concepts and results concerning the computational power of neural networks which basically depends on the information content of weight parameters. In particular, recurrent ...
Robust Regression Estimators: A Comparison of Prediction Performance
Kalina, Jan; Peštová, Barbora
2017 - English
Regression represents an important methodology for solving numerous tasks of applied econometrics. This paper is devoted to robust estimators of parameters of a linear regression model, which are preferable whenever the data contain or are believed to contain outlying measurements (outliers). While various robust regression estimators are nowadays available in standard statistical packages, the question remains how to choose the most suitable regression method for a particular data set. This paper aims at comparing various regression methods on various data sets. First, the prediction performance of common robust regression estimators are compared on a set of 24 real data sets from public repositories. Further, the results are used as input for a metalearning study over 9 selected features of individual data sets. On the whole, the least trimmed squares turns out to be superior to the least squares or M-estimators in the majority of the data sets, while the process of metalearning does not succeed in a reliable prediction of the most suitable estimator for a given data set.
Keywords:
robust estimation; linear regression; prediction; outliers; metalearning
Available at various institutes of the ASCR
Robust Regression Estimators: A Comparison of Prediction Performance
Regression represents an important methodology for solving numerous tasks of applied econometrics. This paper is devoted to robust estimators of parameters of a linear regression model, which are ...
Various Approaches to Szroeter’s Test for Regression Quantiles
Kalina, Jan; Peštová, Barbora
2017 - English
Regression quantiles represent an important tool for regression analysis popular in econometric applications, for example for the task of detecting heteroscedasticity in the data. Nevertheless, they need to be accompanied by diagnostic tools for verifying their assumptions. The paper is devoted to heteroscedasticity testing for regression quantiles, while their most important special case is commonly denoted as the regression median. Szroeter’s test, which is one of available heteroscedasticity tests for the least squares, is modified here for the regression median in three different ways: (1) asymptotic test based on the asymptotic representation for regression quantiles, (2) permutation test based on residuals, and (3) exact approximate test, which has a permutation character and represents an approximation to an exact test. All three approaches can be computed in a straightforward way and their principles can be extended also to other heteroscedasticity tests. The theoretical results are expected to be extended to other regression quantiles and mainly to multivariate quantiles.
Keywords:
Heteroscedasticity; Regression median; Diagnostic tools; Asymptotics
Available on request at various institutes of the ASCR
Various Approaches to Szroeter’s Test for Regression Quantiles
Regression quantiles represent an important tool for regression analysis popular in econometric applications, for example for the task of detecting heteroscedasticity in the data. Nevertheless, they ...
Some Robust Distances for Multivariate Data
Kalina, Jan; Peštová, Barbora
2016 - English
Numerous methods of multivariate statistics and data mining suffer from the presence of outlying measurements in the data. This paper presents new distance measures suitable for continuous data. First, we consider a Mahalanobis distance suitable for high-dimensional data with the number of variables (largely) exceeding the number of observations. We propose its doubly regularized version, which combines a regularization of the covariance matrix with replacing the means of multivariate data by their regularized counterparts. We formulate explicit expressions for some versions of the regularization of the means, which can be interpreted as a denoising (i.e. robust version) of standard means. Further, we propose a robust cosine similarity measure, which is based on implicit weighting of individual observations. We derive properties of the newly proposed robust cosine similarity, which includes a proof of the high robustness in terms of the breakdown point.
Keywords:
multivariate data; distance measures; regularization; robustness; high dimension
Available on request at various institutes of the ASCR
Some Robust Distances for Multivariate Data
Numerous methods of multivariate statistics and data mining suffer from the presence of outlying measurements in the data. This paper presents new distance measures suitable for continuous data. ...
The thermal regime of ice pits of the Borec hill
Türkott, L.; Martinčíková, Eva; Potop, V.
2014 - English
The ecological stability of the sites with stenoec organisms is important factor for maintaining them at given location. Phonolite system of the Borec hill creates a unique labyrinth of vents. Thermal anomalies occur during the year in the fissure system and create specific microclimate. Flow direction is given by the temperature gradient inside and outside of the system. The lower part of fissure system is located in the debris fields, while the upper part on top of the hill. Phonolite rocks are cooled down by air streaming from the debris fields during the winter. Direction of the air flow changes in the spring and summer. The cold air is exhaled from these vents on the lower parts of system and creates ice pits with the typical vegetation.
Keywords:
ice pit; air temperature; Boreč hill; ventalore
Available on request at various institutes of the ASCR
The thermal regime of ice pits of the Borec hill
The ecological stability of the sites with stenoec organisms is important factor for maintaining them at given location. Phonolite system of the Borec hill creates a unique labyrinth of vents. Thermal ...
Inconspicuous Appeal of Amorphous Computing Systems
Wiedermann, Jiří
2014 - English
Amorphous computing systems typically consist of myriads of tiny simple processors that are randomly distributed at fixed positions or move randomly in a confined volume. The processors are “embodied” meaning that each of them has its own source of energy, has a “body” equipped with various sensors and communication means and has a computational control part. Initially, the processors have no identifiers and from the technological reasons, in the interest of their maximal simplicity, their computational, communication, sensory and locomotion (if any) parts are reduced to an absolute minimum. The processors communicate wirelessly, e.g., in an airborne medium they communicate via a short-range radio, acoustically or optically and in a waterborne medium via molecular communication. In the extreme cases the computational part of the processors can be simplified down to probabilistic finite state automata or even combinatorial circuits and the system as a whole can still be made universally programmable. From the theoretical point of view the structure and the properties of the amorphous systems qualify them among the simplest (non-uniform) universal computational devices. From the practical viewpoint, once technology will enable a mass production of the required processors a host of new applications so far inaccessible to classical approaches to computing will follow.
Keywords:
amorphous computing; computational universality; computational complexity
Available on request at various institutes of the ASCR
Inconspicuous Appeal of Amorphous Computing Systems
Amorphous computing systems typically consist of myriads of tiny simple processors that are randomly distributed at fixed positions or move randomly in a confined volume. The processors are “embodied” ...
NRGL provides central access to information on grey literature produced in the Czech Republic in the fields of science, research and education. You can find more information about grey literature and NRGL at service web
Send your suggestions and comments to nusl@techlib.cz
Provider
Other bases