How to troubleshoot LLM customer service summary missing key information

The problem encountered Last month, we added an LLM summary layer to our customer service system, hoping that supervisors wouldn't have to browse through the entire ticket to see the refund reason, order number, and next steps. The first week of the gray test seemed to be fine, and the average processing time was also reduced. However, during the quality inspection sampling, a very troublesome…