A Comparative Study of Nearest Neighbor Regression and Nadaraya Watson Regression
DOI:
https://doi.org/10.25007/ajnu.v10n2a505Keywords:
Nadaraya Watson regression, nearest neighbor regression, Monte Carlo simulationAbstract
Two non-parametric statistical methods are studied in this work. These are the nearest neighbor regression and the Nadaraya Watson kernel smoothing technique. We have proven that under a precise circumstance, the nearest neighborhood estimator and the Nadaraya Watson smoothing produce a smoothed data with a same error level, which means they have the same performance. Another result of the paper is that nearest neighborhood estimator performs better locally, but it graphically shows a weakness point when a large data set is considered on a global scale.
Downloads
References
[2] Silver, D., & Wang, X. (1998, October). Tracking scalar features in unstructured data sets. In Proceedings Visualization'98 (Cat. No. 98CB36276) (pp. 79-86). IEEE.
[3] Bell, G., Hey, T., & Szalay, A. (2009). Beyond the data deluge. Science, 323(5919), 12971298.
[4] Durinck, S., Moreau, Y., Kasprzyk, A., Davis, S., De Moor, B., Brazma, A., & Huber, W. (2005). BioMart and Bioconductor: a powerful link between biological databases and microarray data analysis. Bioinformatics, 21(16), 3439-3440.
[5] Koop, G. (2006). Analysis of financial data. Chichester; Hoboken, NJ: John Wiley & Sons Inc..
[6] Michie, D., Spiegelhalter, D. J., & Taylor, C. C. (1994). Machine learning. Neural and Statistical Classification, 13.
[7] Seber, G. A., & Lee, A. J. (2012). Linear regression analysis (Vol. 329). John Wiley & Sons.
[8] Weisberg, S. (2005). Applied linear regression (Vol. 528). John Wiley & Sons.
[9] Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied linear statistical models (Vol. 5). Boston: McGraw-Hill Irwin.
[10] Wright, R. E. (1995). Logistic regression.
[11] Kleinbaum, D. G., Dietz, K., Gail, M., Klein, M., & Klein, M. (2002). Logistic regression. New York: Springer-Verlag.
[12] Menard, S. (2002). Applied logistic regression analysis (Vol. 106). Sage.
[13] Pregibon, D. (1981). Logistic regression diagnostics. The Annals of Statistics, 9(4), 705-724.
[14] Höskuldsson, A. (1992). Quadratic PLS regression. Journal of Chemometrics, 6(6), 307334.
[15] Cudeck, R., & Du Toit, S. H. (2002). A version of quadratic regression with interpretable parameters. Multivariate Behavioral Research, 37(4), 501-519.
[16] Stinchcombe, J. R., Agrawal, A. F., Hohenlohe, P. A., Arnold, S. J., & Blows, M. W. (2008). Estimating nonlinear selection gradients using quadratic regression coefficients: double or nothing?. Evolution: International Journal of Organic Evolution, 62(9), 2435-2440.
[17] Altman, N. S. (1992). An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician, 46(3), 175-185.
[18] Stute, W. (1984). Asymptotic normality of nearest neighbor regression function estimates. The Annals of Statistics, 12(3), 917-926.
[19] Maltamo, M., & Kangas, A. (1998). Methods based on k-nearest neighbor regression in the prediction of basal area diameter distribution. Canadian Journal of Forest Research, 28(8), 1107-1115.
[20] Cai, Z. (2001). Weighted nadaraya–watson regression estimation. Statistics & probability letters, 51(3), 307-318.
[21] Devroye, L. P. (1978). The uniform convergence of the nadaraya‐watson regression function estimate. Canadian Journal of Statistics, 6(2), 179-191.
[22] Wand, M. P., & Jones, M. C. (1994). Kernel smoothing. Chapman and Hall/CRC.
[23] Daouia, A., Gardes, L., & Girard, S. (2013). On kernel smoothing for extremal quantile regression. Bernoulli, 19(5B), 2557-2589.
[24] Altman, N. S. (1990). Kernel smoothing of data with correlated errors. Journal of the American Statistical Association, 85(411), 749-759.
[25] Kramer, O. (2011, December). Dimensionality reduction by unsupervised k-nearest neighbor regression. In 2011 10th International Conference on Machine Learning and Applications and Workshops (Vol. 1, pp. 275-278). IEEE.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Sarwar A. Hamad, Kawa S. Mohamed Ali
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors retain copyright
The use of a Creative Commons License enables authors/editors to retain copyright to their work. Publications can be reused and redistributed as long as the original author is correctly attributed.
- Copyright
- The researcher(s), whether a single or joint research paper, must sell and transfer to the publisher (the Academic Journal of Nawroz University) through all the duration of the publication which starts from the date of entering this Agreement into force, the exclusive rights of the research paper/article. These rights include the translation, reuse of papers/articles, transmit or distribute, or use the material or parts(s) contained therein to be published in scientific, academic, technical, professional journals or any other periodicals including any other works derived from them, all over the world, in English and Arabic, whether in print or in electronic edition of such journals and periodicals in all types of media or formats now or that may exist in the future. Rights also include giving license (or granting permission) to a third party to use the materials and any other works derived from them and publish them in such journals and periodicals all over the world. Transfer right under this Agreement includes the right to modify such materials to be used with computer systems and software, or to reproduce or publish it in e-formats and also to incorporate them into retrieval systems.
- Reproduction, reference, transmission, distribution or any other use of the content, or any parts of the subjects included in that content in any manner permitted by this Agreement, must be accompanied by mentioning the source which is (the Academic Journal of Nawroz University) and the publisher in addition to the title of the article, the name of the author (or co-authors), journal’s name, volume or issue, publisher's copyright, and publication year.
- The Academic Journal of Nawroz University reserves all rights to publish research papers/articles issued under a “Creative Commons License (CC BY-NC-ND 4.0) which permits unrestricted use, distribution, and reproduction of the paper/article by any means, provided that the original work is correctly cited.
- Reservation of Rights
The researcher(s) preserves all intellectual property rights (except for the one transferred to the publisher under this Agreement).
- Researcher’s guarantee
The researcher(s) hereby guarantees that the content of the paper/article is original. It has been submitted only to the Academic Journal of Nawroz University and has not been previously published by any other party.
In the event that the paper/article is written jointly with other researchers, the researcher guarantees that he/she has informed the other co-authors about the terms of this agreement, as well as obtaining their signature or written permission to sign on their behalf.
The author further guarantees:
- The research paper/article does not contain any defamatory statements or illegal comments.
- The research paper/article does not violate other's rights (including but not limited to copyright, patent, and trademark rights).
This research paper/article does not contain any facts or instructions that could cause damages or harm to others, and publishing it does not lead to disclosure of any confidential information.
