Skip to content

Incorrect setting of updated_at in library.json prevents device updates #2985

Closed
@mintar

Description

System Health details

System Information

version core-2025.1.2
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.13.1
os_name Linux
os_version 6.6.66-haos
arch x86_64
timezone Europe/Berlin
config_dir /config
Home Assistant Community Store
GitHub API ok
GitHub Content ok
GitHub Web ok
HACS Data ok
GitHub API Calls Remaining 4994
Installed Version 2.0.3
Stage running
Available Repositories 1503
Downloaded Repositories 10
Home Assistant Cloud
logged_in false
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 14.1
update_channel stable
supervisor_version supervisor-2024.12.3
agent_version 1.6.0
docker_version 27.2.0
disk_total 30.8 GB
disk_used 11.1 GB
healthy true
supported true
host_connectivity true
supervisor_connectivity true
ntp_synchronized true
virtualization kvm
board ova
supervisor_api ok
version_api ok
installed_addons Advanced SSH & Web Terminal (20.0.0), Mosquitto broker (6.5.0), Studio Code Server (5.18.0), Home Assistant Google Drive Backup (0.112.1), MQTT Explorer (browser-1.0.3)
Dashboards
dashboards 2
resources 0
views 0
mode storage
Recorder
oldest_recorder_run 7. Januar 2025 um 15:01
current_recorder_run 20. Januar 2025 um 16:18
estimated_db_size 519.95 MiB
database_engine sqlite
database_version 3.47.1

Checklist

  • I have read the FAQ to see if there is a known solution to my problem.
    I have enabled debug logging for my installation.
    I have filled out the issue template to the best of my ability.
    This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).
    This issue is not a duplicate issue of currently previous issues..

Describe the issue

PowerCalc version: v1.17.2

When adding an entry for the HmIP-BDT smart dimmer (see #2782), I expected two things to happen:

None of these things happened (see reproduction steps below).

Reproduction steps

I removed the manual entry that I had and then added it back using the new library entry. The screenshots below show the full configuration steps (sorry that they are in German).


Image


Image


Image


Image


Image


Image


Debug logs

https://pastebin.com/iy5hnZAV

Diagnostics dump or YAML config

No response

Activity

bramstroker

bramstroker commented on Jan 20, 2025

@bramstroker
Owner

Very strange I updated to v1.17.2 also on my production instance.

  • Choose Virtual power (library)
  • Selected some light entity
  • Unchecked confirm model (because I don't have the eq3 device)
  • Selected eq3
  • Selected HmIP-BDT

And correctly got redirected to the linear config dialog.

Image

Are you sure you don't have this same model in custom directory in config/powercalc/profiles?

Also you are confident to have restarted after upgrading to v1.17.2?

Otherwise might help maybe to call update library action: https://docs.powercalc.nl/library/library/#updating-the-library

Btw the discovery remarks will currently only be shown when the device is actually discovered.
When you choose to setup manually from library this won't show.

mintar

mintar commented on Jan 22, 2025

@mintar
ContributorAuthor

Also you are confident to have restarted after upgrading to v1.17.2?

Yes, several times.

Are you sure you don't have this same model in custom directory in config/powercalc/profiles?

I don't have the directory config/powercalc/profiles. However, I've used ack to find all occurrences of HmIP-BDT, and this is what I've found:

config/.storage/powercalc_profiles/library.json:
[...]
{"id":"HmIP-BDT","name":"HmIP-BDT","device_type":"smart_dimmer","updated_at":"2024-12-09T15:48:40"}
[...]

config/.storage/powercalc_profiles/eq-3/HmIP-BDT/model.json:
{
  "measure_description": "Manually measured",
  "measure_method": "manual",
  "measure_device": "From manufacturer specifications",
  "name": "HmIP-BDT",
  "standby_power": 0.4,
  "standby_power_on": 0.4,
  "sensor_config": {
    "power_sensor_naming": "{} Device Power",
    "energy_sensor_naming": "{} Device Energy"
  },
  "device_type": "light",
  "calculation_strategy": "fixed",
  "fixed_config": {
    "power": 0
  },
  "created_at": "2024-09-06T18:37:40",
  "author": "CV",
  "description": "HomematicIP smart trailing edge dimmer for outlets. This model is configured only for standby power usage of the smart device itself. The connected device(s) power usage must be configured by the user since it is possible to connect anything to it."
}

This file is indeed outdated. As far as I remember, I have not created or modified it manually. The up-to-date file at the time of writing is https://github.com/bramstroker/homeassistant-powercalc/blob/3cf050be4229e3937a71114e5716c344cbfaa9e4/profile_library/eq-3/HmIP-BDT/model.json.

I've tried moving it away in the hope that it will be re-downloaded, but that gives me the following error:

Image

So I've moved the outdated file back.

Otherwise might help maybe to call update library action: https://docs.powercalc.nl/library/library/#updating-the-library

I've called that action, but it doesn't change the file.

The log (when calling the update library action) says:

[...]
2025-01-22 15:32:03.846 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] Loading library.json from github
2025-01-22 15:32:03.980 DEBUG (MainThread) [custom_components.powercalc.discovery] Start auto discovery
2025-01-22 15:32:03.981 DEBUG (MainThread) [custom_components.powercalc.discovery] Start entity discovery
[...]
2025-01-22 15:32:04.271 DEBUG (MainThread) [custom_components.powercalc.discovery] Done auto discovery

I guess I could just manually copy the new file contents into the old file, but I'm afraid my library doesn't update for whatever reason, so if you can spare the time I'd like to get to the bottom of this.

bramstroker

bramstroker commented on Jan 22, 2025

@bramstroker
Owner

Interesting. Could you please check contents of file config/.storage/powercalc_profiles/.last_update?

Logic to determine if new version of model.json should be downloaded is here btw: https://github.com/bramstroker/homeassistant-powercalc/blob/master/custom_components/powercalc/power_profile/loader/remote.py#L174

Removing the file and restarting or call the service to upload library should both download latest version of model.json.

mintar

mintar commented on Jan 27, 2025

@mintar
ContributorAuthor

Could you please check contents of file config/.storage/powercalc_profiles/.last_update?

The content of that file is 1734282394.4677932 (which corresponds to Sun Dec 15 2024 17:06:34 GMT+0000) - in other words, the model.jsons have not been updated in over 6 weeks.

Logic to determine if new version of model.json should be downloaded is here btw

Thanks! I've added a bunch of log output to that function and tried digging through the code, and I found the bug!

Here's the output of my debug messages:

2025-01-27 13:36:24.947 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] _needs_update({'id': 'HmIP-BDT', 'name': 'HmIP-BDT', 'device_type': 'smart_dimmer', 'updated_at': '2024-12-09T15:48:40'}, /config/.storage/powercalc_profiles/eq-3/HmIP-BDT/model.json, False)
2025-01-27 13:36:24.947 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] path_exists = True
2025-01-27 13:36:24.948 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] remote_modification_time = 1733755720.0
2025-01-27 13:36:24.948 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] self.last_update_time = 1734282394.4677932

TL;DR: The bug is that the updated_at field in library.json is the time stamp when the commit in the pull request was created, not when it was merged!

This is what happened:

  • The current library.json contains the string:

    {"id": "HmIP-BDT", "name": "HmIP-BDT", "device_type": "smart_dimmer", "updated_at": "2024-12-09T15:48:40"}
    
  • My local copy of library.json is up to date; calling powercalc.update_library has no effect.

Timeline:

Today, when Powercalc checks whether it needs to re-download HmIP-BDT's model.json, it checks if 2024-12-15 < 2024-12-09 = False, so it doesn't re-download and I am stuck with the pre-2025-01-12 version.

(BTW, this is on Powercalc v1.17.5.)

changed the title Discovery remarks and fixed config not showing for HmIP-BDT Incorrect setting of updated_at in library.json prevents device updates on Jan 27, 2025
mintar

mintar commented on Jan 27, 2025

@mintar
ContributorAuthor

A good fix would be:

  • Store the last_updated time stamp per device, not globally.
  • Upon saving the last_updated stamp, don't store the current date, but instead store the current updated_at time stamp. This would ensure that whenever the updated_at stamp in the remote library changes to something newer, the device gets updated.
bramstroker

bramstroker commented on Jan 28, 2025

@bramstroker
Owner

@mintar thanks for providing this info, and debugging. Yes that would be the actual cause.
Current logic to determine last_updated_at is here btw: https://github.com/bramstroker/homeassistant-powercalc/blob/master/.github/scripts/profile_library/update-library.py#L155C26-L155C46, which will run after a profile get merged into powercalc master branch.

I agree with you keeping track of updated_at per profile is more robuust.

bramstroker

bramstroker commented on Feb 22, 2025

@bramstroker
Owner

I have implemented a new mechanism with #3114.
Essentially it keeps a hash of all the model information per model.
Whenever the hash changes it will redownload.

Will be in next release.

mintar

mintar commented on Feb 24, 2025

@mintar
ContributorAuthor

Great, this worked! I didn't even have to remove and re-add the device, but it has been updated in the background, so now I have the "Linear options" dialog for the device. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Incorrect setting of updated_at in library.json prevents device updates · Issue #2985 · bramstroker/homeassistant-powercalc