Inviting Epiphany

Last time I talked a bit about the context of my experience with Machine Learning products and the high level issues we had getting customers to switch to what was clearly a better product. This time I get into some technical examples from these checklists and try to demonstrate the conflict.

The first and most important distinction between our product and our competitors is we did “Entity Resolution” while they did mostly “Name Matching with Rules”. What does this look like in practice?

When we built our system our competitors were using name matching systems with rules layered on top. These systems would first create matches based on the textual similarity of the name (based on some fast text matching data structure like a suffix tree). Then, this huge volume of matches would be slimmed down with rules like “only hit if the last name starts with the same letter and the dates of birth are the same”. A typical customer would have hundreds of these rules to manage and would have a large team dedicated to changing them and running simulations of those changes and reviewing huge numbers of false positives for potential matches.

Our Entity Resolution system would take into account many different facets of a person at once: name, date of birth, address, phone number etc, and decide based on the the total evidence available, including frequency information, if they were likely to be the same person. For example, a match on a very common name like “John Smith” does not merit as much evidence as a rare one like “Richard Minerich”. Similarly Dick vs Richard can indicate a match but it’s not as strong as “Richard vs Richard”. There are variations on this theme for every data element. Once two records were considered to have enough evidence of similarity they could be linked and considered to represent the same entity. The highest probability records would be linked with no human intervention, while in the middle some amount of human review would be required. The resulting impact to the customer is there are many times fewer false positives to look at and, as there are fewer knobs to turn, fewer simulations required.

In both cases “is this match relevant” is the next most important question which maybe I’ll get into down the road. It doesn’t matter if a match is true if no one cares, but no one knows if tomorrow it might suddenly become relevant. This dimension of relevancy is worth talking about as it’s missed by far too many projects.

Now that you have the background let’s get on to some examples from those checklists. I hope you’ll understand I’ve only included examples from long enough ago to not conflict with my time at my previous employer.

Can your product match all of the following names (see attached list of name pairs)?

Invariably this would be a list of several hundred names someone typed into Excel by hand that they made up on the fly. Almost nothing in here would resemble the issues we’d see in real data but that didn’t matter. We always had to try our best to match these, and then later we would show the customer what the cost was in terms of false positive increases. Often we won these battles but not always. In the end a configurable matching system combined with an ad-hoc data cleaning system was invaluable for dealing with cases like this.

Does your product allow scans to be rerun with different settings?

This was an interesting one that we received many variations of, often it was an implicit expectation. In fact our platform depended on model configuration being consistent in production for both efficiency and correctness.

Efficiency was important and the consistency of the configuration enabled us to only screen changes to the data. In practice this is basically the difference between millions and thousands of records daily per client in terms of computational cost. We were then able to spend those cycles on fancier Entity Resolution techniques dramatically reducing false positives.

The process depended on its history. If the settings were to change, so would what matched previously and all the saved scores, different records would be above or below different thresholds triggering different downstream workflows and remediation queues. Changes would make the history you are comparing to suspect and so all of the records would need to be rescreened and for a big customer. Rescreening an entire database took a lot of compute and potentially several days. That’s not to say we never changed settings and rescreened, but it was a planned event we’d coordinate with a customer and scale up hardware for.

Correctness was very important as well, our database was the “system of record”, we strove to have the data always model what actually happened on a given day because we needed to be able to answer ad-hoc regulatory queries into what happened when and why. Multiple screening events with different settings would call into question the validity of those screening events and make it very hard to give clear answers.

So to help we would often start with what we considered very low thresholds and many many examples of what might match or might not to build comfort. In some cases we would even end up setting special test environments for bigger customers and give them ability to roll back the data there on request. This turned out to be a deeper issue and later we addressed it with additional product specific features.

Can we control who can see which matches?

This one caught us off guard early on and it goes a lot deeper than you might expect. While we did have data controls from day one we expected that data would be relatively open for review inside compliance departments, silly to think that in retrospect given the risk involved in some of these matches. In actuality our customers were often carved up into different teams with different levels of access within the bigger institutions. Often they didn’t talk to each other at all.

Workflows that determined which team’s queue the matches were put into would take into account many upstream factors that could change including it’s likelihood score. Often the more probable or riskier matches were routed directly to “level 2” analysts. Sometimes certain facets of records would cause matches to be routed to different teams as well, for example sanctions.

Consider the case where a record is updated with some new evidence and now the match belongs to an entirely new department. Even worse, what happens when the evidence goes down, do you take the record away from the level 2 reviewers? We could leave the old match in the old queue and put the new one in the new queue but this would cause cross-team confusion about who would take an action.

Additionally, one of the most important views in our industry is the “alert” view which is simply a roll up of all of the matches for a given customer record. The customer record is the unit they act on (i.e. file reports on), and typically at most one of the matches on a customer record can be true. It’s easy to see why they often do review in terms of their own customer records. If one is deemed clearly true the rest can usually be discarded as false.

But what happens now when this is spread across teams that can’t see matches in each others queues? You have remediated matches in the system but can’t see the decision or why, and so can’t consider the full set of potential matches to judge if a record is true. On top of all that, as records bounced between teams, certain elements of that record’s history would have to be omitted depending on who remediated it and who was looking at it.

Different kinds of customers wanted to solve this differently and so our solutions to this were myriad, we ended up building a very configurable system. Some customers allowed ownership of the entire history to move between teams. Others wanted to show that the data was there but omitted. Some just opened things up as long as it wasn’t a particularly risky data segment.

I hope this peek into some of the pitfalls I ran into as a technical leader for a machine learning based product line was interesting. Please let me know if you’re finding this series interesting by leaving a comment or giving me a shoutout on twitter and I’ll keep at it. Thanks!

First some context. My direct experience over the last decade+ has been mostly B2B selling enterprise grade software to large corporates and so my perspective is skewed this way. I have had many conversations with folks working in the small business and consumer spaces and know those can be very different worlds.

When you sell software in a risk-averse corporate context checklists always come into play. Many many checklists. Feature checklists of course but also many flavors of contractual, security and even internal role/responsibility based checklists. Most of these checklists have to be approved before a sale can be completed, much less a corporate integration.

These checklists presume the business, much less the software, will work in specific ways. Ways that try to make it as easy as possible to line up and compare vendors within a space. Ways that a Machine Learning driven approach come into direct conflict with. Ways that make it very hard to participate in the market as a startup. When you go about trying to do anything in a different way it is inevitable that you come into direct conflict with these checklists.

Now, it is possible to bypass many of these checklists if the customer is sufficiently motivated. This is how we got our first big customers at Safe Banking Systems: They were often under heavy pressure from a government, sometimes even a cease and desist. The current products in the market couldn’t do anything to save them on the timelines they needed. With our AI we could come in and do in a couple of weeks what would take other vendors half a year or more.

Repeated successes led the market start to bend to us. After a few years our reputation allowed us to skip some parts of these checklists. Later we even saw the checklists starting to conform to us, but this took a solid decade and a lot of outreach work.

For example, I personally spent over half a year of my life working on a rather large document for practitioners explaining how our AI worked, why it worked as it did, the process for using that AI, how to interpret it, and why certain processes that were previously necessary were unnecessary and vice versa. Our model choices were heavily constrained by how explainable they were to compliance department bankers.

Also sometimes constraining were the internal structures our customers businesses. After we launched our end to end platform we found one area of friction was that our software required a tiny percent of the number of operators to review results. On the surface this sounds like a big win right? It’s so much cheaper. However, some decision makers measured their power in terms of the number of employees under them and so our software became a threat. Of course, you can always review more records that have a very high probability of being false positives, there could be something in there.

Of course, this is all on top of the typical technical challenges of integrating with large corporates on a data focused project: Data being combined from many systems upstream by an unrelated department. Different departments with different conflicting requirements. Impossible product change requests. Surprisingly onerous clauses slipped into contracts such as “no downtime”. I could spend years writing down the stories.

In the end I believe the key was that our domain was a “small world” and the higher level executives all talked to each other, and as we won them over, us. They were all very risk averse, it was their job to control risk, but it eventually became clear we were the ones to call if you needed to make this very hard problem easy.