Skip to content

Conversation

@roman-vynar
Copy link
Contributor

@roman-vynar roman-vynar commented Apr 9, 2020

The existing replication_group_member_stats collector was written for MySQL 5.7.
With MySQL 8.x there are twice more columns.
Especially interesting is COUNT_TRANSACTIONS_REMOTE_IN_APPLIER_QUEUE which is basically the replication delay (number of transactions) for a secondary member of Group Replication.

So what I did:

  • support additional columns with MySQL 8.0
  • backward compatible with MySQL 5.7
  • if new columns are added in the future, the only change to be is to add one to the list
  • no weird variable namings, compact code
  • FIX: metric "transaction_in_queue" was Counter instead of Gauge
  • BREAKING: renamed 3 old metrics starting with mysql_perf_schema_transaction_ to start with mysql_perf_schema_transactions_ to be consistent with column names
  • BREAKING: exposing only server's own stats (matching MEMBER_ID with @@server_uuid), it does not make sense to expose other server stats to introduce confusing and you will be monitoring other servers separately anyway. Thus, member_id label is dropped.
  • Add a test

@nirgilboa
Copy link
Contributor

LGTM
Also ran it against my staging cluster and seems to be working correctly

@SuperQ
Copy link
Member

SuperQ commented Apr 12, 2020

Thanks. Mostly LGTM.

I would prefer to drop the _count_ redundancy. Yes, it's in the column names, but we try and fix bad style in Prometheus. We prefer making them easier to understand for Prometheus users.

@roman-vynar
Copy link
Contributor Author

Ok, done. Updated PR description with the following:

BREAKING: renamed 3 old metrics starting with mysql_perf_schema_transaction_ to start with mysql_perf_schema_transactions_ to be consistent with column names.

Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also update the CHANGELOG with the new metrics.

@roman-vynar
Copy link
Contributor Author

roman-vynar commented Apr 13, 2020

Done.

Also if you can take a look into this other PR related to Group Replication #459

Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks!

@SuperQ SuperQ merged commit d9d5543 into prometheus:master Apr 13, 2020
SuperQ added a commit that referenced this pull request Apr 23, 2021
Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

* [CHANGE] Switch to go-kit for logs. #433
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517
* [ENHANCEMENT] Support heartbeats in UTC #471
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539

Signed-off-by: Ben Kochie <[email protected]>
@SuperQ SuperQ mentioned this pull request Apr 23, 2021
SuperQ added a commit that referenced this pull request Apr 25, 2021
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [ENHANCEMENT] Support heartbeats in UTC #471
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <[email protected]>
SuperQ added a commit that referenced this pull request Apr 25, 2021
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [ENHANCEMENT] Support heartbeats in UTC #471
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <[email protected]>
SuperQ added a commit that referenced this pull request May 18, 2021
BREAKING CHANGES:

Changes related to `replication_group_member_stats` collector:
* metric "transaction_in_queue" was Counter instead of Gauge
* renamed 3 metrics starting with `mysql_perf_schema_transaction_` to start with `mysql_perf_schema_transactions_` to be consistent with column names
* exposing only server's own stats by matching MEMBER_ID with @@server_uuid resulting "member_id" label to be dropped.

Changes:

* [CHANGE] Switch to go-kit for logs. #433
* [FEATURE] Add `tls.insecure-skip-verify` flag to ignore tls verification errors #417
* [FEATURE] Add collector for AWS Aurora information_schema.replica_host_status #435
* [FEATURE] Add collector for `replication_group_members` #459
* [FEATURE] Add new metrics to `replication_group_member_stats` collector to support MySQL 8.x. #462
* [FEATURE] Add collector for `performance_schema.memory_summary_global_by_event_name` #515
* [FEATURE] Support authenticating using mTLS client cert and no password #539
* [FEATURE] Add TLS and basic authentication #522
* [ENHANCEMENT] Support heartbeats in UTC #471
* [ENHANCEMENT] Improve parsing of boolean strings #548
* [BUGFIX] Fix binlog metrics on mysql 8.x #419
* [BUGFIX] Fix output value of wsrep_cluster_status #473
* [BUGFIX] Fix collect.info_schema.innodb_metrics for new field names (mariadb 10.5+) #494
* [BUGFIX] Fix log output of collect[] params #505
* [BUGFIX] Fix collect.info_schema.innodb_tablespaces for new table names #516
* [BUGFIX] Fix innodb_metrics for mariadb 10.5+ #523
* [BUGFIX] Allow perf_schema.memory summary current_bytes to be negative #517

Signed-off-by: Ben Kochie <[email protected]>
@SuperQ SuperQ mentioned this pull request May 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants