We have an identified Problem wherein the users assign wrong categories to SRs and Incidents.

A Problem ticket was raised to address this and after an RCA we concluded that the category structure was confusing and that very limited effort had been made in training the users to select appropriate categories.
Now in order to resolve this, action items like fine-tuning of categories, setting criteria for category selection, provide examples, etc were undertaken.

After all this, the latest reports show a considerable reduction in miss-classifications. However, we still find this happening on a few occasions.

Now the question here is: when should we say that the problem has been resolved?
A) We can treat this as resolved, since the number of miss-classifications is reducing
B) We should set up a benchmark for max number of miss-classified tickets and consider the problem to be resolved, if the stats are within its scope
C) Such problems cannot be put in a ‘Resolved’ state as their recurrence is unavoidable

Most of us here think that approach B should be followed.
I welcome your opinion/comments on this…

Mis-classification was not the problem. It was the outcome/symptom of the problem. You identified two problems: a) confusing category structure (design issue) and b) inadequate training of users (training issue).

The problems are resolved when you achieve your quality objectives for design of the category structure and training of users respectively.

Presumably, the reason you treated this as a problem initially will have been because the frequency/volume of mis-classifications either caused significant cost or created substantial risk (or a combination of the two - [dybeach - song reference]).

Because these are people driven actions you cannot eliminate errors completely. but if they have not subsided to the level of trivial nuisance, then you still have a problem. Unless you can think of another possible cause, you will either have to accept it as a cost/risk of the system or accept hat it is inappropriate to have such a system and stop asking users to submit classification codes when they raise requests and incidents.

In a manner of speaking the above amounts to your choice of B, but I have described it in such a way as to separate out the elements of design, training and incidents so that you can better consider each of them according to what is practical._________________"Method goes far to prevent trouble in business: for it makes the task easy, hinders confusion, saves abundance of time, and instructs those that have business depending, both what to do and what to hope."
William Penn 1644-1718

Also completely agree with Diarmid. Manual errors cannot be a problem. A problem has to ideally drill down to a process, technology, system, etc. Manual errors are the outcome that results in Incidents.

Mis-classification was not the problem. It was the outcome/symptom of the problem. You identified two problems: a) confusing category structure (design issue) and b) inadequate training of users (training issue).

I would argue that mis-classification was the problem. When investigating, two root causes were identified: a) poorly designed category structure and b) inadequate training of users. I can imagine that there may actually be a third root cause: lack of incentive/enforcement around proper classification. And worst case, you may have a fourth root cause: the people who need to use the categorization structure may not be able to perform at the level needed to do this job. Each of these root causes can then be addressed separately. Obviously you want to address them in a certain order: fix the design, make sure you have the right kind of people, train the users, stimulate/enforce proper use.

It is indeed important to define your success criteria that once met will allow you to consider the problem solved. With problems where human behavior plays a role it is very difficult to reach zero failure. Your failure tolerance depends on the risk associated with failure. For the classification of incidents 5% failure may be good enough. For the maintenance of a nuclear power plant a single failure might be too much._________________Manager of Problem Management
Fortune 100 Company
ITIL Certified

Mis-classification was an incident every time it happened._________________"Method goes far to prevent trouble in business: for it makes the task easy, hinders confusion, saves abundance of time, and instructs those that have business depending, both what to do and what to hope."
William Penn 1644-1718

Known Error: A Problem that is successfully diagnosed and for which a Work-around has been identified.

ITIL V3 (Service Operation book, pages 236 and 240):

Problem: A cause of one or more Incidents. The cause is not usually known at the time a Problem Record is created, and the Problem Management Process is responsible for further investigation.

Known Error: A Problem that has a documented Root Cause and a Workaround. Known Errors are created and managed throughout their Lifecycle by Problem Management. Known Errors may also be identified by Development or Suppliers._________________John Hardesty
ITSM Manager's Certificate (Red Badge)

That is true. Every incident was an occurrence of the mis-classification problem.

I think I understand your point though: if a problem is the unknown underlying cause of the incidents, then they can't be the same. In practicality I believe that when you have a number of these incidents and you determine that you have a problem that needs to be investigated, your problem would be described by the symptoms of the incidents. In this case: mis-classified records.

Similar with say an application that has poor performance. You start out with a bunch of poor performance incidents and then would identify a 'poor performance' problem that requires investigation._________________Manager of Problem Management
Fortune 100 Company
ITIL Certified