Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bind mounted config.yaml does not automatically reload #151

Closed
roycyt opened this issue Aug 11, 2021 · 10 comments
Closed

Bind mounted config.yaml does not automatically reload #151

roycyt opened this issue Aug 11, 2021 · 10 comments
Labels
bug Something isn't working

Comments

@roycyt
Copy link

roycyt commented Aug 11, 2021

Step to reproduce:

$ cat config/config.yaml
services:
  - name: example
    url: "https://example.org/"
    interval: 30s
    conditions:
      - "[STATUS] == 200"
$ docker run \
    --rm \
    -p 8080:8080 \
    -v "$(pwd)"/config/config.yaml:/config/config.yaml \
    twinproduction/gatus

Change the name field in the config/config.yaml.

Observation:

In the HasLoadedConfigurationFileBeenModified function, the result of the comparison of config.lastFileModTime.Unix() != fileInfo.ModTime().Unix() is always false.

@TwiN
Copy link
Owner

TwiN commented Aug 11, 2021

I've tried it with the exact steps you've outlined, and I wasn't able to reproduce the issue.

Note that the file changes aren't picked up instantly - there's a job for changes every 30 seconds.

@TwiN TwiN added the cannot reproduce This doesn't seem right label Aug 11, 2021
@roycyt
Copy link
Author

roycyt commented Aug 12, 2021

Thanks for the confirmation. Will double check and update later.

@roycyt
Copy link
Author

roycyt commented Aug 12, 2021

I can reproduce at my side but I am not sure what is going wrong.
Since I can not provide more information.
I will re-open it when new information comes to light.

@roycyt roycyt closed this as completed Aug 12, 2021
@1player
Copy link

1player commented Sep 3, 2021

Not sure why was this closed, I'm having the same issue on a freshly-installed gatus instance.

FWIW the mount point is a glusterfs mount, but file modification time is updating every time I change the config file, so I have no idea why gatus isn't picking up the changes.

In fact, after multiple tests, only one time it has printed [main][listenToConfigurationFileChanges] Configuration file has been modified, reloaded itself and never again.

@1player
Copy link

1player commented Sep 3, 2021

I found a way to reproduce. tl;dr: this software doesn't play well with vim.

Steps to reproduce:

  1. Start gatus
  2. touch config.yaml, notice the service has picked up the changes
  3. nano config.yaml, notice the service has picked up the changes
  4. vim config.yaml, service doesn't notice the changes (this is a stock Ubuntu vim installation, important)
  5. Service stops picking up any change even with touch

Here's the reason: https://unix.stackexchange.com/a/188954

nano and touch update the file in place, which trigger gatus' configuration update mechanism, while vim by default writes on a temporary file and then overwrites the original file with it, and this breaks gatus. The biggest problem is once gatus reload system has been broken by vim, it will stop noticing any configuration change until it's restarted.

@TwiN TwiN reopened this Sep 4, 2021
@TwiN
Copy link
Owner

TwiN commented Sep 4, 2021

@1player Thanks for the heads up.

To answer your question, the person who created the issue is the one who closed it -- but it was somewhat reasonable at the time given that nobody else had reported this issue before.

In any case, I tried to reproduce it with my main machine, which runs Windows, to no avail:

  • Running Gatus locally, the changes were picked up
  • Running Gatus on Docker, the changes were picked up when modified from PS/CMD
  • Running Gatus on Docker, the changes were picked up when modified from bash (WSL)

But since this is the second person that brings it up, I gave it a shot my laptop which runs Manjaro, and:

  • Running Gatus locally, the changes were picked up
  • Running Gatus on Docker, the changes were not picked up

This is a pretty weird issue...

@TwiN TwiN added bug Something isn't working and removed cannot reproduce This doesn't seem right labels Sep 4, 2021
@1player
Copy link

1player commented Sep 4, 2021

I found the issue: https://stackoverflow.com/a/53548078

My Docker instance of gatus used a bind mount for the config.yaml file, as suggested by the README, as such: -v ./config/config.yaml:/config/config.yaml.

As that Stack Overflow link above suggests, bind mounts are based on inodes and when the file is deleted and recreated then the bind-mount is broken, so when vim replaces the original file with the updated one, the inode changes, and the bind mount breaks.

If you mount the whole parent directory instead of just the config file (-v ./config:/config), the app will reload as expected even when using vim.

EDIT: see also moby/moby#6011

@TwiN
Copy link
Owner

TwiN commented Sep 4, 2021

@1player Great catch!

Would you be interested in updating the following examples?:

Basically, it's just adding a config folder, moving the config.yaml file in it and updating the docker-compose.yml files to bind the folder instead of the file.

(P.S. If you don't want to, no worries - I'll update them. I'm just offering it since you're the one who found the root cause)

@1player
Copy link

1player commented Sep 5, 2021

The first one to do it wins! :-)

A bit busy atm, if I find a moment to do it, I will push a PR, but if you're in front of the code, go ahead and thanks for taking care of updating the examples. Also perhaps we should update the README, and adding a note why binding the file itself is not the best.

@TwiN
Copy link
Owner

TwiN commented Sep 5, 2021

I added a small note in the README.md for now (see #166), but I'll let you take care of the examples if you don't mind 😄

@TwiN TwiN closed this as completed Dec 12, 2021
appleboy added a commit to appleboy/gatus that referenced this issue Mar 19, 2022
ref: TwiN#151 (comment)

update all exmaple in docker-compose file.

Signed-off-by: Bo-Yi Wu <[email protected]>
TwiN pushed a commit that referenced this issue Mar 20, 2022
ref: #151 (comment)

update all exmaple in docker-compose file.

Signed-off-by: Bo-Yi Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants