-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert Atom_RPI.Public and Atom_RPI.Edit to shared libraries #18427
Convert Atom_RPI.Public and Atom_RPI.Edit to shared libraries #18427
Conversation
509ea40
to
1031b67
Compare
2f7131a
to
777f76e
Compare
I'm still reading thru this (there's some minor spelling stuff so far and I'm trying to think of a way to get around the Id/NonId thing... but I'm okay with disabling that warning, for non-exported modules. Doesn't it show up later as a linker error anyway if its something non-exported but actually used? As for the complaint about notification bus complaining about the destructor - I'm not sure. Many of the other busses do not have a destructor and don't need one, but the scene one doesn't have one, and you say it still complains. Note that the destructor is not being forward declared, so ... would that not mess with externing somehow? I'm not sure - we're supposed to be externing the ebus, not the interface... so it shouldn't matter. I don't mind the macros which extern them, but I'm concerned about things which make us have to extern them. Its supposed to be a bit of an optimization, not somehting which actually is required to function, and we were planning to use it for any busses that showed up with a lot of frequency in the binary analysis. I'm not completely against it if it solves a problem to do with linking of course. extern and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really see a problem with all this. we'll have to try it out a lot before it gets deployed for weird 3rd party gem side effects, but moving to better use of the shared library space we have is going to help a lot in compile time and workflow.
If I was a better Template Magician I'm sure I could figure out a way to reduce those macros, but alas, I'm not a Template Magician and in fact, really dislike template metaprogramming and the symbol explosion it causes. I can't think of a way to squish those defs down, and might be better not to, to reduce instantiations.
that being said,
before we actually pull the trigger on this one, I would like to see otehr input on this, especially renderer folks or core folks and get approval for the blanket warning removal.
That, and I'm interested in what other testing you might need, in order to feel "safe" about pushing this...
One thing I can think of is building and installer and trying it out (build o3de with cmake configure options -ULY_PROJECTS
and build the target install
in both profile (and release? AFAIK each time you build install
it overlays that configuration into the install
subfolder and makes it more capable.)
Then use the subfolder install
to test, making a project or whatever.
}; | ||
|
||
// This functino needs to be defined outside of the class definition such that it can get explicitly instantiated correctly. This is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spelling functino
(sorry for the nit!)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wish I knew more about the internals of ebus - I've always wanted to ask the question, what is the callstack storage for anyway? is it necessary to function or just a debug tool? if the latter, could we remove it except in debug or something? NotE: NOT NECESSARY to change this for this PR, just curious as to whether you have any insight for future. It costs a bit of overhead and constantly shows up when I use a binary analysis tool to find whats inside a binary.
Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
…andler and intrusive_refcount outside of class declaration Signed-off-by: Markus Prettner <[email protected]>
Signed-off-by: Markus Prettner <[email protected]>
… Linux Signed-off-by: Markus Prettner <[email protected]>
…e boundaries Signed-off-by: Markus Prettner <[email protected]>
…re it Signed-off-by: Markus Prettner <[email protected]>
…s as extern Signed-off-by: Markus Prettner <[email protected]>
…tribute having nonstatic members without dllexport attribute Signed-off-by: Markus Prettner <[email protected]>
777f76e
to
7a3af85
Compare
Signed-off-by: Markus Prettner <[email protected]>
7a3af85
to
1a0bb7a
Compare
After some PRs (18501, 18525, multiplayersample 493) the following tested scenarios work ("MyGem" is an internal gem which interacts with Atom a lot):
So the bottom line is that the things that worked before on my machine still work, but I am not sure why the release build is this problematic. I cannot rule out the possibility that other linker errors will occur when using different parts of the engine from an external gem with the SDK, but at least for MyGem it works without problem. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm all for these kind of improvements. But let me try first with Android (Quest3) to give you feedback if it breaks mobile or not.
@galibzon Have you had a chance to try this on Android? |
I'm testing now |
It works. I compiled AutomatedTesting for Android. Deployed to a Google Pixel 7 Pro. Loaded the "DefaultLevel". Looks great. |
What does this PR do?
TLDR: Convert
Atom_RPI.Public
andAtom_RPI.Edit
to shared libraries and fix all upstream (mostly in AzCore) problems. Feedback needed on whether the MSVC warning 4251 could be disabled globally. In comparison to the reverted PR18315, this PR does not export all symbols but only those which are explicitly marked for export.Detailed changes
Configuration.h
which contain the export macro definitionsATOM_RPI_PUBLIC_API
,ATOM_RPI_REFLECT_API
,ATOM_RPI_EDIT_API
) to class declarations in Atom_RPIAZ_DLL_EXPORT
macros to explicit template specializations of exported template functionsGetBuildTimestamp()
in ShaderVariant/ShaderVariantAsset because the implementation in ShaderVariantAsset was missingResourcePoolSourceData::Reflect
to new cpp file (was in ResourcePoolBuilder.cpp, which is a different cmake target)AZ_DLL_EXPORT_EXTERN
andAZ_DLL_IMPORT_EXTERN
macros since Clang on Linux requires an additional dllexport when declaring an extern template, but Clang on Windows and MSVC dont allow thisAZ::Internal::IdHandler
/AZ::Internal::NonIdHandler
/AZ::Internal::MultiHandler
ouside of the class definition such that externing them works correctly (this is mostly due to linker errors (*IdHandler), and in one instance about crashes when an EBus context is accessed from multiple modules, eg. when the codepath from an EBus broadcast is handled in different modules): This works for the most part, the downside is thatAZ::Internal::NonIdHandler
and::IdHandler
also need to be externed with aextern template class Internal::IdHandler...
in addition to the EBus itself, and sinceusing
directives cannot be used forextern template class
(extern template class EBus<SystemTickEvents>::BusesContainer::Handler
is not allowed (it works in MSVC but is not allowed according to the standard, and doesnt work with Clang on Windows), as well as eg.AZStd::conditional<...>
which usesusing
as well), there are now two separate macros required for declaring and instanciating an extern EBus (one for Id, one for NonId). I didnt find a way to get the id/nonid information from the class with template magic, since not being allowed to use ausing
directive limits template metaprogramming significantly.Feedback needed
'type' : class 'type1' needs to have dll-interface to be used by clients of class 'type2'
": This PR only compiles when disabling this warning globally (it would be possible to use theAZ_PUSH_DISABLE_DLL_EXPORT_MEMBER_WARNING
/AZ_POP_DISABLE_DLL_EXPORT_MEMBER_WARNING
macros to disable this warning more locally for the affected classes (of which there are many) if this is not an option). I dont really see the point of keeping this warning enabled, since it is disabled a total of ~350 times in the codebase; mostly due to Qt classes, but it would also needed to be used in any Atom::RPI class which contains AZStd::* members or members from AzCore, such as intrusive_ptr (until AzCore is a shared library itself). So I would suggest to disable this warning globally.DECLARE_EBUS_EXTERN_DLL_SINGLE_ADDRESS
and similar ones are not really that pretty, but it does not seem to be possible to get the correct members to declare asextern template
due to theusing
limitation of the extern template syntax. Maybe someone could think of a better way to handle this?RPI::SceneNotificationBus
andRPI::ShaderReloadNotificationBus
: It works fine with the unity build, but lead to linker errors (IdHandler::~IdHandler already defined in CommonFeaturesAtom.Editor.Static
) when using a nounity build. The workaround is to declare the bus as extern with the DECLARE_EBUS_EXTERN_DLL_MULTI_ADDRESS macro, but I dont know why this is necessary (could be any of: template instantiations are not deduplicated across dll boundries, a derived dllexported class exports the handler, the functions are not implicitlyinline
, the functions are not exactly the same in all translation units due to changes in compile options, or something completely different; it is very difficult for me to get a complete picture of the EBus and related classes to see where the problem could be, and since it is all template code it should just work™, but obviously it does not). This being necessary only for the SceneNotificationBus with nounity-build suggests that some specific constellation of defining handlers and dispatching events leads to problems, so it could be a problem for other RPI buses which are used in external gems.Future work
Based on the large number of
AZ_PUSH_DISABLE_DLL_EXPORT_BASECLASS_WARNING
/AZ_POP_DISABLE_DLL_EXPORT_BASECLASS_WARNING
that needed to be added, and some linker errors which are likely due to staticAzCore
being linked into multiple dlls, it is probably advisable to convert the core libraries (AzCore
/AzFramework
) first and then move outwards to other targets and gems for future work (which is also suggested in the "Improve Developer iteration workflow RFC").How was this PR tested?
Run the Editor and do some interactions, run tests locally on Linux, run the AtomSampleViewer and look at a few samples