PyArabic: A Python package for Arabic text
Taha Zerrouki · The Journal of Open Source Software · 2023
Because text is the most common type of information representation, text processing and manipulation require recurring routines and functions.Every day, massive amounts of text are processed.Indeed, with the advent of artificial intelligence and new machine learning and deep learning enhancements, natural language processing has become a critical domain.PyArabic is a collection of modules that provide basic functionality for manipulating Arabic texts, phrases, words, numbers, and letters.It primarily provides preprocessing tools such as normalization, tokenization, diacritics removal, number conversion, transliteration, and so on.For years, researchers and developers who worked on machine learning algorithms for natural language processing have used the library for Arabic text preprocessing and cleaning.The library becomes more important for machine learning. Statement of needPyArabic is a Natural Language Processing Python package for Arabic text 1 .It is a simple library with basic functions for manipulating Arabic letters and text, such as detecting Arabic letters, Arabic letter groups and characteristics, removing diacritics, and so on.It contains the most basic and useful routines used by developers and researchers working with Arabic texts.Some key features are as follows:• Text tokenization.• Remove diacriticts (Harakat) from words (all, except Shadda, Tatweel, last haraka).• Separate a word into letters and diacritics.• Reduce diactritics of words.• Measure tashkeel similarity (Harakats, fully or partially vocalized similarity with a template).• Letter normalization (ligatures and Hamza).• Numbers to words.• Extract numerical phrases and prevocalize it.• Unshaping texts to handle letter glyphs.• Convert encoding and transliteration.The PyArabic package includes five major submodules:• Araby: Basic tools and routines for manipulating Arabic text and letters, such as tokenization and diacritics removal, are provided.• Number: Contains routines for dealing with numbers and numeric words; allows conversion of numbers to words and words to numbers; detects numeric phrases, and more.• Named: Provides simple tools for extracting named entities from text.1 The library can be found at [PyPi.org index](