Collecting High-quality Multi-modal Conversational Search Data for E-Commerce
Marcus Collins, Oleg Rokhlenko, Eugene Agichtein, Shervin Malmasi · 2024
Continued improvement of conversational assistants in knowledge-rich domains like E-Commerce requires large volumes of realistic high-quality conversation data to power increasingly sophisticated LLM chatbots, dialogue managers, response rankers, and recommenders.The problem is worse for multimodal interactions in realistic conversational product search and recommendation.Here, an artificial sales agent must interact intelligently with a customer using both textual and visual information, and incorporate results from external search systems, such as a product catalog.Yet, it remains an open question how to best crowd-source large-scale, naturalistic multi-modal dialogue and action data, required to train such an artificial agent.We describe our crowd-sourced task where one worker (the Buyer) plays the role of the customer, and other (the Seller) plays the role of the sales agent.We identify subtle interactions between one worker's environment and their partner's behavior mediated by workers' word choice.We find that limiting information presented to the Buyer, both in their backstory and by the Seller, improves conversation quality.We also show how conversations are improved through minimal automated Seller "coaching".While typed and spoken messages are slightly different, the differences are not as large as frequently assumed.We plan to release our platform code and the resulting dialogues to advance research on conversational search agents.Example Dialog Customer: I'm looking for a good sized TV for my living room.Agent: Nice!How much are you looking to spend?Customer: 400 dollars Agent: Do you mostly watch TV shows, or maybe you want something for gaming?Customer: I like to watch movies and play video games Agent: How big?What about resolution?Customer: at least 42 inches.what's resolution?Agent: number of pixels per inch.Higher is better.Customer: I don't care Agent: Ok, do you care about refresh rate?Or brand?Customer: I like Sony.It should be at least 120 frames per second.Agent: OK, Sony has 3 TVs that match.Do you have another criteria you care about