Statistical Methods
Christer Samuelsson, Sanja Štajner · 2019
Current mainstream natural language processing tasks rely heavily on the use of statistical methods. This chapter presents statistical methods from their fundamentals to more complex methods, including hidden Markov and maximum entropy models applied in language modelling, and the expectation maximization method used in machine translation. It also introduces robust estimation and the methods for calculating inter-annotator agreement and statistical significance, as well as suggestions for further reading and some mathematical details of the presented statistical methods.