-
Notifications
You must be signed in to change notification settings - Fork 544
Open
Description
Which problem is this feature request solving?
I have workflows that pickup data that goes into a postgres database (using the destination plugin).
But I currently don't have a way to know what rows have changed. The postgres plugin updates the "cq-sync" time whether the actual values changed or not. So I have to process EVERY row against downstream data sources to see if there is a delta. It's very wasteful and slow.
Describe the solution you'd like
A way to know the true "last updated" time for a row. Not the last time a sync was run.
This could take the form of upsert logic doesn't execute an insert if the values haven't changed (obviously excluding things like cq-sync), or an additional field like "cq-updated".
It might also be nice to configure this on a table by table basis for fine tuning.
Pull request (optional)
- I can submit a pull request