A Comparative Study of Nearest Neighbor Regression and Nadaraya Watson Regression
Keywords:Nadaraya Watson regression, nearest neighbor regression, Monte Carlo simulation
Two non-parametric statistical methods are studied in this work. These are the nearest neighbor regression and the Nadaraya Watson kernel smoothing technique. We have proven that under a precise circumstance, the nearest neighborhood estimator and the Nadaraya Watson smoothing produce a smoothed data with a same error level, which means they have the same performance. Another result of the paper is that nearest neighborhood estimator performs better locally, but it graphically shows a weakness point when a large data set is considered on a global scale.
 Silver, D., & Wang, X. (1998, October). Tracking scalar features in unstructured data sets. In Proceedings Visualization'98 (Cat. No. 98CB36276) (pp. 79-86). IEEE.
 Bell, G., Hey, T., & Szalay, A. (2009). Beyond the data deluge. Science, 323(5919), 12971298.
 Durinck, S., Moreau, Y., Kasprzyk, A., Davis, S., De Moor, B., Brazma, A., & Huber, W. (2005). BioMart and Bioconductor: a powerful link between biological databases and microarray data analysis. Bioinformatics, 21(16), 3439-3440.
 Koop, G. (2006). Analysis of financial data. Chichester; Hoboken, NJ: John Wiley & Sons Inc..
 Michie, D., Spiegelhalter, D. J., & Taylor, C. C. (1994). Machine learning. Neural and Statistical Classification, 13.
 Seber, G. A., & Lee, A. J. (2012). Linear regression analysis (Vol. 329). John Wiley & Sons.
 Weisberg, S. (2005). Applied linear regression (Vol. 528). John Wiley & Sons.
 Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied linear statistical models (Vol. 5). Boston: McGraw-Hill Irwin.
 Wright, R. E. (1995). Logistic regression.
 Kleinbaum, D. G., Dietz, K., Gail, M., Klein, M., & Klein, M. (2002). Logistic regression. New York: Springer-Verlag.
 Menard, S. (2002). Applied logistic regression analysis (Vol. 106). Sage.
 Pregibon, D. (1981). Logistic regression diagnostics. The Annals of Statistics, 9(4), 705-724.
 Höskuldsson, A. (1992). Quadratic PLS regression. Journal of Chemometrics, 6(6), 307334.
 Cudeck, R., & Du Toit, S. H. (2002). A version of quadratic regression with interpretable parameters. Multivariate Behavioral Research, 37(4), 501-519.
 Stinchcombe, J. R., Agrawal, A. F., Hohenlohe, P. A., Arnold, S. J., & Blows, M. W. (2008). Estimating nonlinear selection gradients using quadratic regression coefficients: double or nothing?. Evolution: International Journal of Organic Evolution, 62(9), 2435-2440.
 Altman, N. S. (1992). An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician, 46(3), 175-185.
 Stute, W. (1984). Asymptotic normality of nearest neighbor regression function estimates. The Annals of Statistics, 12(3), 917-926.
 Maltamo, M., & Kangas, A. (1998). Methods based on k-nearest neighbor regression in the prediction of basal area diameter distribution. Canadian Journal of Forest Research, 28(8), 1107-1115.
 Cai, Z. (2001). Weighted nadaraya–watson regression estimation. Statistics & probability letters, 51(3), 307-318.
 Devroye, L. P. (1978). The uniform convergence of the nadaraya‐watson regression function estimate. Canadian Journal of Statistics, 6(2), 179-191.
 Wand, M. P., & Jones, M. C. (1994). Kernel smoothing. Chapman and Hall/CRC.
 Daouia, A., Gardes, L., & Girard, S. (2013). On kernel smoothing for extremal quantile regression. Bernoulli, 19(5B), 2557-2589.
 Altman, N. S. (1990). Kernel smoothing of data with correlated errors. Journal of the American Statistical Association, 85(411), 749-759.
 Kramer, O. (2011, December). Dimensionality reduction by unsupervised k-nearest neighbor regression. In 2011 10th International Conference on Machine Learning and Applications and Workshops (Vol. 1, pp. 275-278). IEEE.
How to Cite
Copyright (c) 2021 Sarwar A. Hamad, Kawa S. Mohamed Ali
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-ND 4.0] that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
AJNU is committed to protecting the privacy of the users of this journal website. The names, personal particulars and e-mail addresses entered in this website will be used only for the stated purposes of this journal and will not be made available to third parties without the user's permission or due process. Users consent to receive communication from the AJNU for the stated purposes of the journal. Queries with regard to privacy may be directed to firstname.lastname@example.org.