-
-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
Describe the bug
Rules with uppercase keywords find no results when those rules are defined in a TOML that is extended.
When the TOML is consumed into the Config object, the keywords for each rule in the TOML are converted to lowercase and then added to a summary of keywords on the Config object.
Line 97 in 48a2e0e
| keywords[strings.ToLower(k)] = struct{}{} |
Then, during the detect phase, these lowercase keywords are used to detect secrets in the the raw fragment which has also been converted to lowercase.
This works as intended for simple configurations, but when one configuration extends from another, the keywords of the base configuration are not converted to lowercase.
Take for example the configuration
title = "Extend Config"
[extend]
path="base.toml"
[[rules]]
id = "aws-secret-key-again-again"
description = "A new description"# base.toml
title = "Base Configuration"
[[rules]]
id = "aws-secret-key-again-again"
description = "AWS Secret Key"
regex = '''(?i)aws_(.{0,20})?=?.[\'\"0-9a-zA-Z\/+]{40}'''
tags = ["key", "AWS"]
keywords = ["AWS"]
[[rules.allowlists]]
stopwords = ["fake"]Once parsed, the above configuration results in the keyword AWS, rather than aws which is required for detect to work as expected.
To Reproduce
Clone this test project.
Navigate to the cloned folder and scan for secrets using
docker run -it --rm -w /myapp -v ${PWD}:/myapp zricethezav/gitleaks:latest detect -v -l trace --report-path /myapp/gitleaks.json --report-format json --source /myapp/src --config src/rule_extentions.toml --no-git
The results are:
○
│╲
│ ○
○ ░
░ gitleaks
6:09AM DBG using gitleaks config src/rule_extentions.toml from `--config`
6:09AM DBG extending config with /myapp/base_rules.toml
6:09AM TRC Scanning path: {/myapp/src/rule_extentions.toml }
6:09AM TRC Scanning path: {/myapp/src/secrets.txt }
6:09AM TRC Skipping finding due to global allowlist regex finding="glpat-Ibbbbbbbbbbbbbbbbbbb " path=/myapp/src/secrets.txt rule-id=gitlab_personal_access_token
6:09AM TRC Skipping finding due to global allowlist regex finding=glpat-Ibbbbbbbbbbbbbbbbbbb' path=/myapp/src/rule_extentions.toml rule-id=gitlab_personal_access_token
Finding: glpat-Dbbbbbbbbbbbbbbbbbbb //should be detect...
Secret: glpat-Dbbbbbbbbbbbbbbbbbbb
RuleID: gitlab_personal_access_token
Entropy: 1.596185
Tags: [gitlab revocation_type gitlab_blocking]
File: /myapp/src/secrets.txt
Line: 1
Fingerprint: /myapp/src/secrets.txt:gitlab_personal_access_token:1
however, https://gitlab.com/craigmsmith/test_extend_494511/-/blob/f9ac067ccc2d4cc5c38c807f8cc20b4dd1a65191/src/secrets.txt#L3 should also have been detected.
Expected behavior
https://gitlab.com/craigmsmith/test_extend_494511/-/blob/f9ac067ccc2d4cc5c38c807f8cc20b4dd1a65191/src/secrets.txt#L3 should also have been detected.
Screenshots
If applicable, add screenshots to help explain your problem.
Basic Info (please complete the following information):
- OS:
- Gitleaks Version:
Additional context
Add any other context about the problem here.
cc @zricethezav