Automating Reason Code Classification with AI and NLP

Many businesses rely on reason codes to track why a particular action, status change, or transaction occurred.

Reason codes typically are predefined numeric or string identifiers and are commonly used in business systems and databases.

For example, in the cloud and communications services industry, reason codes are commonly used to track:

• Service Cancellation Codes, e.g. “Cost too High”, “Support Issues”, etc.
• Case Reason Codes, e.g.,“01 – Installation”, “02 – Performance”, etc.

Reason codes are used in reporting and auditing and play an important role in decision making (e.g. where to improve products or services, restructure teams, focus investments, etc.).

This post outlines the challenges with traditional reason codes, and how to utilize AI (Artificial Intelligence) and NLP (Natural Language Processing) to provide a better solution that captures the voice of the customer more effectively and is easier to maintain.

Traditional Approach

The traditional approach to implementing reason codes is to add a field in the database that tracks the reason using a set of predefined values (e.g., a picklist in Salesforce or a dictionary in a relational database).

For example, Salesforce offers a Case Reason that users can modify to suit their needs:

At some stage in the process, a reason code is entered by the user. For example, a customer selecting a service cancellation reason when submitting a request through the customer portal.

Based on the data collected, reports can be generated. A basic example might look like this:

While this approach is popular, it presents several challenges:

• Maintenance
  • New reason codes are often needed as new products or services are introduced.
  • Overtime, new codes are added to capture more specific or detailed information.
  • Some codes become obsolete and must are removed or disabled for user selection.
• Accuracy
  • Over time, the list of codes grows too large, and users may select a random option just to proceed.
  • The “Other” category becomes a catch-all, leading to incomplete or unstructured insights.

However, the most critical issue is the assumption that all valid reasons can be known and predefined in advance — when in reality, users may have reasons that go beyond what the provider anticipated.

As a result, the reporting often lacks completeness and accuracy, ultimately leading to misleading insights for decision-makers.

An Alternative — AI Based Classification

An alternative to the traditional approach is to use AI to automatically classify data based on the language (text) provided by customers:

1. Instead of selecting a predefined option like “Cost Too High” from a drop-down list, the user can enter their own comment — for example, “Decided to go with a more cost-effective option.”
2. AI, using Natural Language Processing (NLP), analyzes and classifies the input to derive a category — for example, “Cost”.
3. This derived category is then stored as the Reason Code in the database, ensuring that existing legacy reports continue to work.

In short, instead of maintaining a static set of codes:

• Let AI derive codes dynamically from user input.
• Let AI manage and expand the list as new patterns emerge.

(The existing set of codes can be provided to the AI with instructions to prefer them and create new codes only when necessary).

AI Classification Benefits

• Self-maintaining reason code list — reduces manual upkeep.
• Adaptability — codes evolve naturally as user language changes.
• Access to raw data — original customer text remains available for deeper insights (e.g., AI-driven customer sentiment analysis).
• Backward compatibility — supports existing reports and workflows.

Implementation Steps

Let’s walk through an example implementation for classifying service cancellation (or disconnect) reasons using AI:

1. Data Model Changes.
   Add a Comment textual field alongside the existing Reason Code.
   Note: The actual location of this information will depend on the back-office systems in use — typically, it resides on a service record, an order, or a cancellation request.
2. User Interface (UI) Changes. When creating a cancellation order, the user is required to enter free-text comments instead of choosing from a drop-down list, similar to this:
   
   
   Consideration: Label the field in a user-friendly way (e.g., “Tell us more” or “Comments”) to encourage richer customer input.
3. Data Scrubbing. Depending on the input format (e.g., HTML, RTF, or plain text), consider cleaning the text to remove markup such as HTML tags. This will make it easier for AI to analyze the text.
4. Database Update. Once the request is submitted, the customer comments are stored on the cancellation order (or equivalent record).
5. AI-Analyze Data. A scheduled batch job (e.g., every 24 hours, depending on volume) performs the following:
   1. Retrieve new (unclassified) cancellation requests.
   2. Pass the Comments (and existing classification codes) to the AI.
   3. Receive classification results from the AI.
   4. Update the Reason Code fields accordingly.

Example AI-Generated Reason Codes

For example, Open AI NLP without any model training generates the following classification for the following sample input:

Additional Implementation Considerations

1. Preserving Existing Reason Codes. Matching user inputs to existing codes — and creating new ones only when necessary — is essential. A strategy should be in place to support this.
2. Input Data Quality. NLP models are generally robust and can handle imperfect or unstructured text, helping mitigate the impact of lower-quality input.
3. Security. Although customer-specific data should not be present in the input, further anonymization may be required. It is also important to ensure that the data is not used for training the AI model (for example, OpenAI’s API does not use submitted data to train its models).
4. Code Optimization. Since the classification process may involve transferring significant data between systems and the AI engine, consider using bulk APIs and minimizing payload sizes for efficiency.

Summary

Traditional reason code systems — like drop-down lists in CRMs — are rigid, require ongoing maintenance, and often fail to capture the true voice of the customer. Users may choose inaccurate or generic options, leading to incomplete and misleading reports for decision-makers.

An AI-based alternative uses Natural Language Processing (NLP) to classify free-form customer input (e.g., comments or emails) into meaningful categories like “Cost” or “Service Quality.” This approach keeps reports compatible with existing systems, while improving accuracy, reducing maintenance, and allowing the reason code list to evolve automatically based on real user language.

Nextian offers software products and services helping cloud & communications providers improve their Quote-to-Cash to drive revenue growth and customer retention. Our offerings include advanced AI solutions in the areas of customer support (generative AI, ticket routing), and analytics (sentiment scoring, churn prediction, skills-based matching), especially in conjunction with CRMs and customer support systems.

Contact us today to find out how we can help you!