Master Entity Profile mapping into the Analytics Module

  • 13 April 2022
  • 6 replies
  • 90 views

Userlevel 3

Despite our efforts to build robust validation criteria on the front end, we often struggle with duplicate master entity profiles. Of particular note, our master person profiles are a mess. 

 

Some of this is due to our inability to impose validation criteria at the entry point of person profiles in disparate systems that we source profiles for Mark43 ingestion into person profiles. Further, the contextual data elements for these profiles can often be incomplete or contradictory. 

 

I have structured a few different data lake queries and an analytics dashboard to tease out merge candidates (we have more than 10,000 candidates based on complete match on first name, last name, and DOB). The ability to do this work within the analytics module is limited as the profile attributes (i.e., name, DOB, SSN, etc.) are often provided as contextual elements and require 1:1 complete parity of the merge criteria to identify them as candidates. Since the data for person profiles is often structured within the different analytic views (i.e., offense/incident persons, use of force subjects,  I typically do this based upon the same first name, last name, and date of birth. However, if I have two profiles that should be merged (1. John Smith 01/01/2000 & 2. John Smyth 01/01/2000), I will not be able to detect their candidacy due to the different spellings of the last names. This is further exacerbated when you consider hyphenated last names and every way personnel can enter/ingest names.

 

What I have found is that mandatory data elements (i.e., person race) are often recorded as “unknown,” despite previous contacts with the subjects providing a known race. This is not exclusive to the race field, but it is an example. When I stumble upon duplicate master person profiles, I do what I can to merge up all the duplicates (sometimes there are 3, 4, or more duplicates). While this helps resolve the master person profile attributes, it does not resolve contextual data elements. To be clear, I agree with this logic and do not advocate for an update on the contextual information. 

 

Where this presents an issue is within the analytics module and how the data is structured within the analytics views. The person attributes that are provided are context data elements derived from the associated reports, but we know this information is often incomplete or unreliable. Since they are contexted, they are related to an individual report_id within disparate analytic views (i.e., offense/incident, use of force, arrest, field contact, etc.). This means that in order to have a full accounting, a complex merge query needs to be created connecting all these views based upon a match on first, last and DOB (however this is unreliable and requires a lot of processing power to render results). This also does not provide orphan profiles (profiles that exist in the tenant, but have no association to a report) because the views are constrained to persons associated to reports only.

 

In order to create a workaround, I would like to propose that Mark43 create analytic views for ALL MASTER ENTITIES that provide the relevant data attributes that are conducive to analysis. That way when a master profile is updated (whether through a direct edit, a profile merge, or a subsequent report association), the information displayed for the profile can be homogenous. If the data is mapped into the module as discrete views providing the appropriate key fields (i.e., master_person_id, master_item_id, etc.), then we will have the ability to leverage the merge feature to join these consistent attributes into different looks. It would also be nice to configure the master_id fields to hyperlink directly to the profile details page so we don’t have to create a custom hyperlink dimension to enable consumers to easily navigate to an item. 

 

Examples may include:

Master Person Profiles (Master ID, Names, DOB, Race, Ethnicity,  Cautions, Current Home Address)

 

Master Item Profiles (Master ID, Current Item Status - Stolen/Recovered/etc., most recent description, serial number, color, etc.)

 

Master Firearm Profiles (master ID, make, model, caliber, finish, serial number, etc.)

 

Master Vehicle Profiles (master IS, VIN, License Plate State, Plate, colors, description,  current status, etc.)

 

 

Does anyone else struggle with duplicate entity profiles? What work arounds do you impose to improve your ability to discern information? 


This topic has been closed for comments

6 replies

Userlevel 3
Badge

@Jaycee.Elliott Thank you for the detailed write up and explanation of the pain point. We recently had a conversation very similar to this after it was brought up internally. Your idea has been heard. Please give us some time to investigate this and follow up with you.

Userlevel 3

Thanks @DavidSchwindt 

 

I’m hoping other agencies working through similar issues may be able to offer some insights as to how they have overcome these challenges. 

I know many agencies struggle with duplicate profiles as well. One additional aspect of this is when profiles are created by different agencies that all share Mark 43 reading permissions. For instance, where I work we have read permission for several agencies that use Mark 43 in the area. When an officer or deputy writes a report and has their search permissions selected only for their department, they often create a duplicate profile unintentionally.

I have often arrested someone only to look them up in Mark 43 and locate several duplicate profiles created by officers or deputies in various jurisdictions. Our records department is able to combine our duplicate profiles but not the others created by other departments. Our records department generally asks the officer reporting the duplicate entry to select the physical attributes when combining duplicate profiles. 

I am also curious how other agencies deal with duplicate profiles. 

 

Userlevel 3

@Mgoode 

 

Interesting… We are not in a consortium at the moment. We have discussed it, but the workflow you have discussed above with the duplication of a person profile from another agency’s person profile is one of our biggest hesitations to go toward a consortium. Well that and having parity among our attribute configurations (which is near impossible for agencies along state borders that require different offense tables and reporting requirements). 

 

I believe the duplication of the person profile as you have described is intended behavior for the software, which in my opinion is fundamentally flawed. Attributes entered from another agency’s contacts with the person are copied from the other agency tenant into your agency’s tenant when your officer selects the profile from the other agency. All that information is adopted into a new master person profile that is owned by your agency. Both of those profiles are maintained separately  by each agency and though related, are not joined/merged. We do not like this approach because we don’t want to source and store data regarding a person within our own record of the person unless our personnel intentionally sourced the information. It is problematic for us for public disclosure requests as well if we are releasing information regarding a person, but we are not able to detail where and how we associated the information to a person. 

I should have stated earlier though, as a frontline patrol officer, consortiums do have their benefits as a investigative tool. In our area, there are several city departments that use Mark 43 within the same county. The consortium saves our frontline officers a ton of time and is a huge investigative resource. Many area agencies upload their traffic citations and field contact cards to Mark 43 which is a great investigative tool..

There have been many crimes with vehicle descriptions with or without plates where suspects were located due to another agency’s citation upload to Mark 43. This can be especially helpful if a vehicle has changed hands several times without proper registration. 

 

Just food for thought on consortiums benefits. 

Matt

 

Userlevel 3

@Mgoode 

I completely agree with the concepts of consortium and the efficiency that is offers; especially to folks on the road. This is especially true when neighboring agencies all roll out Mark43 simultaneously and can agree upon common configurations. Our situation is unique in that we did not roll our Mark43 as a consortium among neighboring agencies and it is further complicated that the agencies considered for consortium reside in two different states along the border. We agree that read only access to neighboring tenants is a huge benefit, but wading through the front end searching capabilities of Mark43 has proven a bit cumbersome (especially with data quality issues) and we are concerned that entering a consortium will only compound this issue. Ideally, the advanced search pages would have increased functionality to allow end users to either show or hide attributes that are enabled or disabled. Similarly in a consortium, end users could have the ability to search by attributes configured in their tenant only. The latter is less of an issue for consortiums with participating agencies configuring a single homogenous list of shared attributes, but again, that is not the case for us and one of the primary reasons we departed from our previous RMS and consortium.