Neural Network Metalearning for Parallel Textual Information Retrieval
Lean Yu, Shouyang Wang, Kin Keung Lai · 2008
In this study, a triple-phase neural network metalearning technique is proposed to perform parallel textual information retrieval tasks. In the first phase, the massive textual data collections with the terabyte scale are first partitioned into various relatively small textual data subsets. Then these small data subsets are moved to different computational agents. In the second phase, the single neural network model as the intelligent learning agent is applied to the different textual data subsets so as to retrieve some relevant text documents responding to a query. For a given query, the neural network learning agent, which is sufficiently trained by back-propagation learning algorithm on underlying text documents, can produce a relevance score between 0 and 1 for a certain text document. In the third phase, based on the different relevance scores produced by the previous phase, a neural-network-based metamodel by integrating the relevance results is generated to provide a proactive information extraction model that can be used on unseen query to determine the relevance degree between the query and textual documents (out of many candidates). For illustration and testing purposes, a practical web textual information retrieval experiment is performed to verify the effectiveness and efficiency of the proposed neural-network-based metalearning technique.