Bug (xUnit v2.x): Collection fixture is being disposed even if all its tests are skipped. #3066
Comments
|
As an update: my GitLab pipeline succeeded after this change: It means I suspect having method |
|
Hey @voroninp, you might be interested in testing the new (unreleased yet) |
|
A preview version of Testcontainers.Xunit has been released on NuGet. Testing with xUnit.net should get you started. Please try it and report any issue on the testcontainers-dotnet repository. |
|
@0xced , I'll take a look, thanks. Yet it does not cancel the fact that current xUnit's behaviour is at least strange. |
Indeed! I looked at your |
|
Here is my simple repro: using System.Threading.Tasks;
using Xunit;
using Xunit.Abstractions;
using Xunit.Sdk;
namespace xunit3066;
public class Fixture(IMessageSink messageSink) : IAsyncLifetime
{
public Task DisposeAsync()
{
messageSink.OnMessage(new DiagnosticMessage("DisposeAsync"));
return Task.CompletedTask;
}
public Task InitializeAsync()
{
messageSink.OnMessage(new DiagnosticMessage("InitializeAsync"));
return Task.CompletedTask;
}
}
[CollectionDefinition("UnitTest1Collection")]
public class UnitTest1Collection : ICollectionFixture<Fixture>
{ }
[Collection("UnitTest1Collection")]
public class UnitTest1
{
[Fact(Skip = "Skipped Test")]
public void Test1()
{ }
}When running the test, you can see that both
This is what I would expect. It does appear that the exception in
You're in a strange edge case where all of your tests are skipped, and we prioritize the skipping of the test over failing due to startup exceptions, so you're never seeing your exception happen. It's still happening, you're just not seeing it get reported, because we have no vehicle to report it with. It is extremely unusual to have a fixture created and every test it's assigned to is skipped. There is not much we can do here, and I consider this to be (strangely) working as designed. As a helpful tip to avoid situations like this in the future, IMO you should find any usage of the public sealed class PostgresSqlContainerFixture : IAsyncLifetime
{
private static readonly PostgreSqlBuilder Builder = new();
private PostgreSqlContainer? container;
public PostgreSqlContainer Container =>
container ?? throw new InvalidOperationException("Container was not initialized");
public async Task DisposeAsync()
{
if (container is not null)
await container.StopAsync();
}
public async Task InitializeAsync()
{
container = Builder.Build();
await container.StartAsync();
}
} |
I get the following error when my tests are run in GitLab:
Our current GitLab setting causes tests which use Docker to fail, so I temporarily skip all the tests which use Test Containers.
Here's the code of the fixture:
I cannot find an exception coming from
InitializeAsyncin GitLab log, but I assume it is called, and exception is swallowed/ignored.Maybe it's not called at all. I do not know for sure.
I changed
InitializeAsynctoAnd I started seeing these errors when run tests locally:
However, my project to reproduce the issue does not indicate
InitializeAsyncgets called at all:So what we have:
It's either only
DisposeAsyncbeing always called, or bothInitializeAsyncandDisposeAsyncbeing called and both failing.The text was updated successfully, but these errors were encountered: