Distributed task-based execution engine for support of text-mining processes
Peter Butka, Peter Bednár, František Babič, Karol Furdík, Ján Paralič · 2009
This paper describes design and implementation aspects for extension of our original software system developed in Java for support of information retrieval and text mining with specialized execution engine for different type of tasks. Some of our experiences and specific requirements of the real applications lead us to idea give the system possibility to run the tasks in distributive way. The result of the idea is task-based execution engine, which represents middleware-like transparent layer (mostly for programmers who want to re-use functionality of our package) for running of different tasks in multi-thread environment. The original system is being developed as open source with the intention to provide an easy extensible, modular framework for pre-processing, indexing and further exploration of large text collections. This specialized execution engine has been implemented within APVV project called PoZnaT (Support for knowledge creation processes) in order to extend proposed text-mining platform for educational and experimental purposes. Conceptual architecture of the system is provided as well as details regarding our extension of library like usage of content repository paradigm, representation and encapsulation of tasks, implementation of the engine itself and its role within PoZnaT platform.