Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to test_logs against kwargs. #47536

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

NHDaly
Copy link
Member

@NHDaly NHDaly commented Nov 10, 2022

Currently, you cannot use @test_logs to test a log message's keyword arguments. This extends @test_logs to let you do this.

Additionally, while I was here, I also changed @test_logs itself to let you specify patterns using keyword args / NamedTuples, so that you can more easily specify parts of the LogRecord that come at the end (like the kwargs).

Here is an example of a log message with kwargs:

julia> @info "hi" x=2
┌ Info: hi
└   x = 2

Which produces this LogRecord:

LogRecord(Info, "hi", Main, Symbol("REPL[5]"), :Main_c95d07ae, "REPL[5]", 2, Base.Pairs(:x => 2))

After this PR, you can test for this kind of log message by matching the :kwargs field to @test_logs:

@test_logs (:info, "hi", Test.Ignored(), Test.Ignored(), Test.Ignored(), Test.Ignored(), Test.Ignored(), (;y=3)) @info("hi", x=2)

But obviously the above is way verbose, so as I described in the second paragraph, this PR also adds the following shorter syntax:

@test_logs (level=:info, message="hi", kwargs=(;x=2)) @info("hi", x=2)

Also updated the docstring.

@NHDaly NHDaly added test This change adds or pertains to unit tests logging The logging framework labels Nov 10, 2022
@@ -285,12 +306,32 @@ logfield_contains(a, r::Regex) = occursin(r, a)
logfield_contains(a::Symbol, r::Regex) = occursin(r, String(a))
logfield_contains(a::LogLevel, b::Symbol) = a == parse_level(b)
logfield_contains(a, b::Ignored) = true
logfield_contains(a::NamedTuple, b) = logfield_contains(pairs(a), b)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting to see a Dict overload here as well; is it somehow covered by other methods? I see you have a test for passing kwargs as a Dict, so I also assume it's working somehow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 yeah, note that i'm matching this on a, not b, which is the LogRecord field, not the user-supplied Pattern.

I did that because i wanted to be more generic in what types i accepted in the Pattern, and the LogRecord will always have a fixed type field.

Does that seem reasonable to you as well?

🤔 Although, now that i think about it, i guess this could cause method ambiguities? Like if you try to use Ingored on the kwargs, i think it would give a method ambiguity. Maybe that's the only one i need to worry about though. Maybe i should just add an overload just for that case? I'll do that now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

K, done: i added logfield_contains(a::NamedTuple, b::Ignored) = true # method amibguity resolution

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:/ this is still causing a method ambiguities test failure though:

Error in testset ambiguous:
Test Failed at /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/test/ambiguous.jl:187
  Expression: isempty(detect_ambiguities(modules...; recursive = true, allowed_undefineds))
   Evaluated: isempty(Tuple{Method, Method}[(logfield_contains(a, b::Test.Ignored) @ Test /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/stdlib/v1.9/Test/src/logging.jl:308, logfield_contains(a::Base.Pairs, pattern) @ Test /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/stdlib/v1.9/Test/src/logging.jl:311), (logfield_contains(a, r::Regex) @ Test /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/stdlib/v1.9/Test/src/logging.jl:305, logfield_contains(a::Base.Pairs, pattern) @ Test /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/stdlib/v1.9/Test/src/logging.jl:311), (logfield_contains(a, r::Regex) @ Test /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/stdlib/v1.9/Test/src/logging.jl:305, logfield_contains(a::NamedTuple, b) @ Test /cache/build/default-amdci5-4/julialang/julia-master/julia-5647ec6a75/share/julia/stdlib/v1.9/Test/src/logging.jl:309)])

It's unhappy about this:

[logfield_contains(a, b::Test.Ignored),
 logfield_contains(a::Base.Pairs, pattern),
 logfield_contains(a, r::Regex),
 logfield_contains(a::Base.Pairs, pattern),
 logfield_contains(a, r::Regex),
 logfield_contains(a::NamedTuple, b)]

so i guess i'll probably need to resolve the ambiguities after all

NHDaly and others added 2 commits November 13, 2022 14:16
Currently, you cannot use `@test_logs` to test a log message's keyword
arguments. This extends `@test_logs` to let you do this.

Additionally, while I was here, I also changed `@test_logs` itself to
let you specify _patterns_ using keyword args / NamedTuples, so that you
can more easily specify parts of the LogRecord that come at the end
(like the `kwargs`).

Here is an example of a log message with kwargs:
```julia
julia> @info "hi" x=2
┌ Info: hi
└   x = 2
```
Which produces this LogRecord:
```julia
LogRecord(Info, "hi", Main, Symbol("REPL[5]"), :Main_c95d07ae, "REPL[5]", 2, Base.Pairs(:x => 2))
```

After this PR, you can test for this kind of log message by matching the
`:kwargs` field to `@test_logs`:
```julia
@test_logs (:info, "hi", Test.Ignored(), Test.Ignored(), Test.Ignored(), Test.Ignored(), Test.Ignored(), (;y=3)) @info("hi", x=2)
```

But obviously the above is way verbose, so as I described in the second
paragraph, this PR also adds the following shorter syntax:
```julia
@test_logs (level=:info, message="hi", kwargs=(;x=2)) @info("hi", x=2)
```

Also updated the docstring.
logfield_contains(a::NamedTuple, b::Ignored) = true  # method amibguity resolution
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logging The logging framework test This change adds or pertains to unit tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants