Best method or Addon to deal with duplicates across modules? Version 7.5 Enterprise

What is the best approach to minimise duplicate records across modules? Where an overly enthusiastic sales rep enters a new lead and the record might already be an existing Account? I have seen the suggestion here http://support.sugarcrm.com/04_Knowle....

As well as a number of add-ons.

But could really use some suggestions as to the most pragmatic\efficient approach.

I'm not sure about all the different best practices that people use, though I am curious to hear what people have to say.

Lead data does tend to be dirty which is part of the reason why we keep Accounts and Contacts separate from Leads. The Lead qualification or conversion process is one way to manage that sanitization process where you keep dirty data separate from clean data.

I did want to point out that in Sugar 7.x we do have a configurable and extensible DuplicateCheck API that gets used when users create new Records via the UI. It is configurable because the out of the box implementation allows you to setup the Filter criteria you want to use in a Module's vardefs. It is extensible because you can define your own custom DuplicateCheckStrategy via custom code for your module.

For example, here is some example Bugs dupecheck metadata. Since it is very common for the same Bugs to creep up again and again, people are more concerned with new Bugs that are duplicates of other open Bugs. This filter criteria reflects this. It only looks for name matches for other Bugs that are not Closed.

This method does not let you search other modules (like your example where somebody is creating a Lead that is a duplicate of an existing Account). That is where you could write some custom code to handle different kinds of dupe check processes in Sugar 7.

Detect duplicates with a universal search dash let visible in the edit screen of any module.This would work as follows;User decides to add a record, so goes to their chosen module and clicks ADD.• Starts typing the Name.• A dashlet would be visible to the right of the edit form• It would be displaying Elastic Search results for the data typed into selected edit screens fields.• If at any point the user decided to click on a search result, the Add record edit would be discontinued and the details screen of the search result displayed.

Perhaps a level of configuration might be needed to say what edit fields should be used in the universal search for each module• Search fields to consider• Any Name field• Phone number field• Email fields• Domain name references (whether from an email address or website URL)

A typical and massive business case regarding Leads and conversion has these features:

User wants to search for duplicates by unique IDs: "Email account", "Document number" (passport, DNI or NIE in Spain, for example). First and last name are not good enough for this.

Admin User should have control ober this process: I envision an option in "Admin" area with a basic configuration: "Allow duplicates" (yes/no: in some companies duplicates are not a problem because they want to keep track of how many times a Lead has contacted us by a website, for example). "Unique ID fields" for controlling duplicates (you can't save a new record that is duplicated according to this/these field/s). "Next action" -> the user should choose to block creation (a new record can't be created if duplicated) or choose a merge: merge Lead data with Account/Contact data that was already created (this process would manually enrich current data).

this validation would be done for manual creation of record and web-to-lead forms (maybe we should think about this process carefully, let me know your opinion).

I think your idea of using global search API to check for duplicates is very doable. Our global search (which uses Elasticsearch) is driven via a Search API that you could easily call from a dashlet. All that configurability could be built into the dashlet as well. It could make for a very portable and flexible duplicate detection mechanism. We are also looking at enhancing our global search support over the next couple releases which would make that API more powerful.

@David

Of course, handling administration and web-to-lead forms makes this a bit more than a dashlet. This might be a good project for a workshop!