Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scrutiny - cannot use user defined COLLECTOR_API_ENDPOINT as already defined by app owner #1244

Open
Agent6-6-6 opened this issue Dec 26, 2024 · 1 comment
Open

Scrutiny - cannot use user defined COLLECTOR_API_ENDPOINT as already defined by app owner #1244

Agent6-6-6 opened this issue Dec 26, 2024 · 1 comment

Comments

@Agent6-6-6
Copy link

Hi

A feature of Scrutiny is being able to define an endpoint on a remote computer for remote monitoring of smart data.

In the docker-compose.yaml file it has been hardcoded to the localhost which means it cannot be altered to send the smart data to a remote host.
https://github.com/truenas/apps/blob/f886be723b410ad8c60af75f6f43088f10bc003c/ix-dev/community/scrutiny/templates/docker-compose.yaml#L35C31-L35C53

If you add an additional environment variable within truenas for the COLLECTOR_API_ENDPOINT it does not override the value given in the docker-compose file, it causes the following error instead:-
image

  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 556, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1367, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 287, in do_update
    app = self.update_internal(job, app, data, trigger_compose=app['state'] != 'STOPPED')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 317, in update_internal
    update_app_config(app_name, app['version'], new_values, custom_app=app['custom_app'])
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/ix_apps/lifecycle.py", line 59, in update_app_config
    render_compose_templates(
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/ix_apps/lifecycle.py", line 50, in render_compose_templates
    raise CallError(f'Failed to render compose templates: {cp.stderr}')
middlewared.service_exception.CallError: [EFAULT] Failed to render compose templates: Traceback (most recent call last):
  File "/usr/bin/apps_render_app", line 33, in <module>
    sys.exit(load_entry_point('apps-validation==0.1', 'console_scripts', 'apps_render_app')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/catalog_templating/scripts/render_compose.py", line 47, in main
    render_templates_from_path(args.path, args.values)
  File "/usr/lib/python3/dist-packages/catalog_templating/scripts/render_compose.py", line 19, in render_templates_from_path
    rendered_data = render_templates(
                    ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/catalog_templating/render.py", line 36, in render_templates
    ).render({'ix_lib': template_libs, 'values': test_values})
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/mnt/.ix-apps/app_configs/scrutiny/versions/1.0.10/templates/docker-compose.yaml", line 42, in top-level template code
    {{ tpl.render() | tojson }}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/.ix-apps/app_configs/scrutiny/versions/1.0.10/templates/library/base_v2_1_6/render.py", line 69, in render
    "services": {c._name: c.render() for c in self._containers.values()},
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/.ix-apps/app_configs/scrutiny/versions/1.0.10/templates/library/base_v2_1_6/render.py", line 69, in <dictcomp>
    "services": {c._name: c.render() for c in self._containers.values()},
                          ^^^^^^^^^^
  File "/mnt/.ix-apps/app_configs/scrutiny/versions/1.0.10/templates/library/base_v2_1_6/container.py", line 327, in render
    result["environment"] = self.environment.render()
                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/.ix-apps/app_configs/scrutiny/versions/1.0.10/templates/library/base_v2_1_6/environment.py", line 109, in render
    raise RenderError(f"Environment variable [{k}] is already defined from the application developer.")
base_v2_1_6.error.RenderError: Environment variable [COLLECTOR_API_ENDPOINT] is already defined from the application developer.

Solution - allow for this variable to be overridden by a user environment variable so a hub/spoke model use case for Scrutiny can be realised?
Use default as currently provided, but override if a user-defined value is provided.
Potentially bringing the field into the app edit screen with a default value of the localhost is presumably easy, allowing user to define their own IP or URL for the endpoint?
@Agent6-6-6
Copy link
Author

Note as a workaround you can setup a cron job on Truenas in the UI to run the collector with the desired remote endpoint until this is addressed.

image

Full command in this case would be in this format:-

sudo docker exec -w /bin [container_name] /bin/sh -c 'scrutiny-collector-metrics run --api-endpoint "http://[IP]:[port]"'

Modify as required for your situation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Agent6-6-6