Evaluating and Measuring Citator Functionality during Migration from a Search Engine to a Graph Database
A. N. Shah, Mihir Joshi · SSRN Electronic Journal · 2021
Shepard's service is an online service offered within the suite of search and retrieval Service. It allows customer to validate cites, find citing references to cases, statutes, and other legal content. It has several reporting and analyzed information that are tied to regular search and Full doc view. The dynamic nature and relationship of this cites to their citing references and the underlying size of the corpus creates a complex system, which involves several different search techniques and reporting methods. Currently, there is an ongoing effort to migrate Shepard's service from search engine to graph database. The most significant challenge the new system is facing is to be at par with the old Search engine infrastructure not only in terms of data but also for different Shepard's functionalities like the report, preview, case card, and signals. There was a need to come up with a tool that would give measurement and evaluation at every point of development. This system should also cater to all the stakeholders such as the developer, product, and the management to identify the issues and roadblocks and help them keep track of the progress. In this paper, we discuss an API comparison framework that distinguishes two sets of responses coming from two different service calls. The service calls could come from different environment, or same environment with different ABE users configured. Underlying database used could be configured using ABE or some other method transparent to this Tool. Some of the existing tool like STF does an excellent work on showing the relevance, position, and several aspects of the search, however along with STF we need something which does more in depth analysis of data. This tool fills the gap that STF is not covering and evaluates different result that are provided to the client. The tool can look at live production request that have come within a given time frame and replay them in an environment that is configurable within the tool. The current implementation does evaluate and compares the result of two responses, measuring how close or different those service layer responses are from each other, and it gives detail information on where the issue is and what service request was associated with it. Capturing the request allows to reuse it and reproduce the system behavior at a later time. It also keeps track of the transaction id, so that developers can investigate further during execution of the test. This tool saved a lot of Testing time in this project and it processes large set of test cases and data, helping to identify issues earlier in the development cycle. It is also like a regression suite giving confidence on what is getting built. This tool fits in along with several tools and approach we have like (ABE/ dark Launch / STF). Other than migration work described here, this tool can be used for regular functional enhancements or regression.