Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build dependency libraries instead of bundling pre-built binaries. #3141

Merged
merged 1 commit into from
Dec 4, 2024

Conversation

binary1248
Copy link
Member

Title.

In order to:

  • Improve toolchain compatibility (e.g. building with Emscripten)
  • Reduce maintenance effort of having to build and ship binaries
  • Reduce the size of the source archive
  • Allow more interesting dependencies (e.g. HarfBuzz) to be easily integrated in the future
  • Provide a better debugging experience by providing debug libraries and symbols of dependencies
  • Allow user compiler/linker flags to propagate down to the dependencies as well

This change gets rid of the packaged pre-built dependencies and builds them alongside the SFML modules.

It would be nice to get this into SFML 3 so that we have a clean environment to start adding some bigger features in SFML 4.

Currently the focus was on getting this to build successfully on CI, however more widespread testing will be necessary especially on more exotic platforms that we don't currently cover in our CI. The goal is to support as many platforms as is feasible as long as they are still considered somewhat up-to-date.

@binary1248 binary1248 self-assigned this Jun 29, 2024
@binary1248 binary1248 force-pushed the feature/build_dependencies branch from c6e9d96 to 6ea310f Compare June 29, 2024 15:38
@eXpl0it3r eXpl0it3r added this to the 3.0 milestone Jun 29, 2024
CMakeLists.txt Outdated Show resolved Hide resolved
@binary1248 binary1248 force-pushed the feature/build_dependencies branch 2 times, most recently from 8a7cd44 to 33399f3 Compare July 1, 2024 19:05
vittorioromeo added a commit to vittorioromeo/VRSFML that referenced this pull request Jul 23, 2024
src/SFML/Audio/CMakeLists.txt Outdated Show resolved Hide resolved
@vittorioromeo
Copy link
Member

vittorioromeo commented Aug 6, 2024

Another issue I am getting is that, when building imgui-sfml using

cmake .. -DIMGUI_SFML_FIND_SFML=1 -DSFML_ROOT="C:/OHWorkspace/SFML/build/install" -DIMGUI_DIR="C:/OHWorkspace/imgui" -DSFML_DIR="C:/OHWorkspace/SFML/build/install"

I get the following error

Call Stack (most recent call first):
  CMakeLists.txt:28 (find_package)


CMake Error at CMakeLists.txt:28 (find_package):
  Found package configuration file:

    C:/OHWorkspace/SFML/build/install/lib/cmake/SFML/SFMLConfig.cmake

  but it set SFML_FOUND to FALSE so package "SFML" is considered to be NOT
  FOUND.  Reason given by package:

  The following imported targets are referenced, but are missing:
  Vorbis::vorbis Vorbis::vorbisfile Vorbis::vorbisenc FLAC::FLAC

Not sure what the right way of solving this one is.

@eXpl0it3r eXpl0it3r force-pushed the feature/build_dependencies branch 4 times, most recently from 562fb6d to 5b6adc9 Compare September 12, 2024 21:42
@ChrisThrasher
Copy link
Member

I get the following error

Likewise. Not sure what the fix is either.

@ChrisThrasher
Copy link
Member

Oddly enough, I can fix that ImGui-SFML configuration error by adding the Audio component to the find_package(SFML call. Perhaps something about package component handling got broken.

@ChrisThrasher
Copy link
Member

Have you looked into CPM.cmake and whether it would offer a simpler implementation? For example, CPM.cmake will let you specify options for fetched projects that you do stuff like build a dependency as a static library while not affecting the value of BUILD_SHARED_LIBS in the consuming project. This removes the need to cache the value of that variable then careful restore it later.

@binary1248
Copy link
Member Author

The initial version of this change used CPM.cmake and as much as I would have liked to keep using it it very quickly falls apart when trying to add dependencies that can't be bothered to adhere to standard CMake practices. The fact that we have to patch many dependencies should already be a red flag that CPM.cmake isn't going to have a good time with them.

@eXpl0it3r
Copy link
Member

eXpl0it3r commented Nov 13, 2024

@binary1248 if have a bit of time, could you look at what's causing the error that is mentioned by Vittorio?

I really like to include this for the next RC, so that we have more people than just the SFML Team to try out this change.

In my opinion this is on the critical path. For one we've decided to move in this direction many years ago, but more importantly this will unlock a lot of new possibilities, such as HarfBuzz or Opus support.

I don't think this should be held back for SFML 3.1 or similar, as it's quite a fundamental change in the dependency handling and will cause more issues, if users have to migrate between minor versions.

@binary1248
Copy link
Member Author

If someone could add the appropriate tests for this scenario to CI it would be easier to ensure this change or any future changes don't break this functionality. It would also make it easier for me to fix the outstanding issue/issues without having to make sure my local test environment matches what is expected.

@ChrisThrasher
Copy link
Member

ChrisThrasher commented Nov 13, 2024

If someone could add the appropriate tests for this scenario to CI it would be easier to ensure this change or any future changes don't break this functionality. It would also make it easier for me to fix the outstanding issue/issues without having to make sure my local test environment matches what is expected.

I added those tests two years ago. #2206

Oddly enough, I can fix that ImGui-SFML configuration error by adding the Audio component to the find_package(SFML call. Perhaps something about package component handling got broken.

The problem is that this error only occurs if you omit the Audio component when calling find_package. Our install interface tests requests all components thus sidestepping this bug.

find_package(SFML 3.0.0 EXACT CONFIG REQUIRED COMPONENTS Graphics Network Audio)

@ChrisThrasher
Copy link
Member

I don't think this should be held back for SFML 3.1 or similar, as it's quite a fundamental change in the dependency handling and will cause more issues, if users have to migrate between minor versions.

Is it? Notice how all our CI build jobs still work without modification. What about users' workflows will change if we merge this?

@eXpl0it3r
Copy link
Member

eXpl0it3r commented Nov 13, 2024

I narrowed it down to the difference between using SFML_DIR and SFML_ROOT, the latter we use in the install tests, thus it doesn't fail in CI.

Simple repo:

  1. Build & install SFML to some directory:
    • cmake -S . -B build -DCMAKE_INSTALL_PREFIX=C:/Temp/SFML
    • cmake --build build --target install
  2. Create the two files from below
  3. Run CMake for the project: cmake -S . -B build -DSFML_DIR=C:/Temp/SFML/lib/cmake/SFML
  4. Fail

It works if you run instead cmake -S . -B build -DSFML_ROOT=C:/Temp/SFML

CMakeLists.txt

cmake_minimum_required(VERSION 3.28)
project(CMakeSFMLProject LANGUAGES CXX)

set(SFML_STATIC_LIBRARIES ON)
find_package(SFML REQUIRED COMPONENTS Graphics Audio)

add_executable(main main.cpp)
target_link_libraries(main PRIVATE SFML::Graphics SFML::Audio)

main.cpp

#include <SFML/Graphics.hpp>

int main()
{
    auto window = sf::RenderWindow({1920u, 1080u}, "CMake SFML Project");
    window.setFramerateLimit(144);

    while (window.isOpen())
    {
        for (auto event = sf::Event(); window.pollEvent(event);)
        {
            if (event.type == sf::Event::Closed)
            {
                window.close();
            }
        }

        window.clear();
        window.display();
    }
}

@eXpl0it3r
Copy link
Member

I don't know if there is a solution to allow SFML_DIR.

Personally, I'm also fine to merge this as is and discourage usage of SFML_DIR in favor of SFML_ROOT.

@eXpl0it3r eXpl0it3r mentioned this pull request Nov 19, 2024
54 tasks
@ChrisThrasher ChrisThrasher force-pushed the feature/build_dependencies branch from 5b6adc9 to 444873d Compare November 19, 2024 20:51
@ChrisThrasher
Copy link
Member

@eXpl0it3r
Copy link
Member

@binary1248 binary1248 force-pushed the feature/build_dependencies branch 10 times, most recently from 007c481 to 6827f7c Compare December 1, 2024 03:45
Copy link
Member

@eXpl0it3r eXpl0it3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good. Some small comments and I still need to retest it.

cmake/Macros.cmake Outdated Show resolved Hide resolved
cmake/Modules/FindFreetype.cmake Show resolved Hide resolved
src/SFML/Graphics/CMakeLists.txt Show resolved Hide resolved
src/SFML/Graphics/CMakeLists.txt Show resolved Hide resolved
@eXpl0it3r
Copy link
Member

Tested it successfully locally. It's awesome to know that we no longer depend on pre-built binaries. The build time also doesn't seem to drastically increase for me.

We may want to consider updating some of the dependencies after this PR.

@binary1248 binary1248 force-pushed the feature/build_dependencies branch from 6827f7c to bbad388 Compare December 3, 2024 23:53
@eXpl0it3r eXpl0it3r merged commit 248fd6d into master Dec 4, 2024
220 checks passed
@eXpl0it3r eXpl0it3r deleted the feature/build_dependencies branch December 4, 2024 22:52
@eXpl0it3r
Copy link
Member

Remember when Bromeon opened an internal proposal for this in September 2015? 😄

Let's see what we broke with this change, but I'm excited to see all the possibilities ahead!

@ChrisThrasher
Copy link
Member

I want to mention a few observations as I update some projects to the latest state of master.

  1. project(my_app CXX) no longer works. SFML requires a C compiler to build and for whatever reason that requirement isn't automatically inherited by projects that consume SFML via FetchContent. This generates a rather obtuse warning that I think many users will struggle to understand. Users must use project(my_app C CXX) or more simply project(my_app) to deal with this. I'm not aware of any action we can take to in the library itself fix this.

    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
    Missing variable is:
    CMAKE_C_COMPILE_OBJECT
    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
    Missing variable is:
    CMAKE_C_COMPILE_OBJECT
    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
    Missing variable is:
    CMAKE_C_COMPILE_OBJECT
    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
    Missing variable is:
    CMAKE_C_COMPILE_OBJECT
    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
    Missing variable is:
    CMAKE_C_COMPILE_OBJECT
    -- Generating done (0.1s)
    CMake Generate step failed.  Build files cannot be regenerated correctly.
    
  2. Building SFML use to be very clean to nearly any warning you may apply. It's bad practice to build 3rd party code with warnings enabled but sometimes users do it accidentally. Now that we build lots of 3rd party code, enabling even basic warnings causes a lot of console output about SFML's dependencies. Again, no real action we can take but another thing users may notice in the transition.

  3. I get a warning in 3rd party code and am not sure how to silence it. This is not a technical problem, but it looks a bit sloppy that building the library produces a warning. Maybe building our C deps with a newer C standard will fix this.

    /Users/thrasher/Projects/mandelbrot/build/_deps/flac-src/microbench/util.c:79:11: warning: mixing declarations and code is incompatible with standards before C99 [-Wdeclaration-after-statement]
       79 |         uint64_t duration = *end - *start;
          |                  ^
    1 warning generated.
    

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

6 participants