Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AuthBackendRPCImplementation Crashes (Firebase v11.2) #13650

Closed
ondrejkorol opened this issue Sep 16, 2024 · 24 comments · Fixed by #13658
Closed

AuthBackendRPCImplementation Crashes (Firebase v11.2) #13650

ondrejkorol opened this issue Sep 16, 2024 · 24 comments · Fixed by #13658
Assignees
Milestone

Comments

@ondrejkorol
Copy link

Description

Hi Firebase team,

I’ve encountered crash in my app after updating to Firebase 11.2.0 and GoogleSignIn 8.0.0. The crash is reported by Crashlytics and has occurred on user devices running iOS 17.6.1. I’m using Xcode 15.4 for development and publishing the app. I can't reproduce it locally.

We discussed this crash in this GitHub thread but I was advised to open a new issue here.

There are two "different crashes" so far:
Crash Details:
• Exception Type: EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000000000000003
• Thread: com.apple.root.user-initiated-qos.cooperative
• Affected File: AuthBackend.swift - Line 143
• Function: specialized AuthBackendRPCImplementation.callInternal(with:) + 143

The first crash stack trace:
app_issue_466ae93783cfc5e3010951fc92f6e334_crash_session_c89332271f904df98a543d8daf437162_DNE_0_v2_stacktrace.txt
second.crash_issue_466ae93783cfc5e3010951fc92f6e334_crash_session_449ef354ab5e4450b4e53a2fde399be3_DNE_0_v2_stacktrace.txt

A different stack trace:
crash.type2.txt

I'd appreciate any guidance on how to mitigate this crash. Please let me know if you need any additional information to help diagnose this issue.

Thank you for your assistance.

Best regards,

Ondrej

Reproducing the issue

No response

Firebase SDK Version

11.2

Xcode Version

15.4

Installation Method

Swift Package Manager

Firebase Product(s)

Authentication, Crashlytics, Firestore, Messaging

Targeted Platforms

iOS

Relevant Log Output

No response

If using Swift Package Manager, the project's Package.resolved

No response

If using CocoaPods, the project's Podfile.lock

No response

@google-oss-bot
Copy link

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

@ncooke3
Copy link
Member

ncooke3 commented Sep 16, 2024

Hi @ondrejkorol, thanks for opening the issue! I just merged #13647 will should hopefully address the problem. I wasn't able to reproduce it, but my rationale from looking at the stack trace is that the shared concurrency queue (com.apple.root.user-initiated-qos.cooperative) is being blocked by a call to [FIRHeartbeatLogger headerValue] which makes a synchronous dispatch to a different queue. Swift concurrency's shared co-op queue should not be blocked (Swift forums), so the SDK has been changed to execute that synchronous call onto another thread of the queue.

I suspect the second kind of crash is related since it does have the same Crashed: stack, but it is interesting so I will investigate it a bit more.

Crashed: com.apple.root.user-initiated-qos.cooperative
0  libswiftCore.dylib             0x4043a4 swift_unknownObjectRetain + 48
...

@ondrejkorol
Copy link
Author

That sounds great! I really appreciate your fast response and the whole team’s support. So I guess I should wait for a new Firebase version, and then we can try that.

Yeah, it's tricky as it's not easily reproducible, right? Let me know if I can help somehow.
Cheers!

@ncooke3
Copy link
Member

ncooke3 commented Sep 17, 2024

So I guess I should wait for a new Firebase version, and then we can try that.

Yes, @ondrejkorol. Though in the meantime, I do have some tips that may help reproduce it:

  • All of the three traces are doing something with Metal on the main thread, and Firebase Auth is either trying to sign in or refresh a token. It could be a coincidence but it seems suspicious. Hopefully this is a clue to the part of your app causing the crash (e.g. maybe Metal is used for an intricate animation while the user authenticates). If Metal isn't being used at the time where a user logs in, than it it likely due to Auth happening to refresh the auth token in the background while Metal is also doing work. If this sounds like more of a possibility, than you could try to recreate the situation by calling the func getIDToken(forcingRefresh forceRefresh: Bool = false) async throws -> String with forceRefresh = true around the time that Metal is also doing work.
  • EXC_BAD_ACCESS KERN_INVALID_ADDRESS indicates a memory issue, so testing in release mode should give a higher chance of exposing the bad state than debug mode would (https://stackoverflow.com/a/44571456/9331576).
Screenshot 2024-09-17 at 9 06 14 AM - Enable ASan profiler Screenshot 2024-09-17 at 9 54 21 AM

@ncooke3
Copy link
Member

ncooke3 commented Sep 17, 2024

Between #13647 and #13658, this issue should be addressed in the next Firebase 11.3 release.

@fanwgwg
Copy link

fanwgwg commented Sep 18, 2024

@ncooke3 +1 Thank you for your information. This is now starting to impact our users in production and causing crashes. When will be a new release containing these fixes?

@ajunjunandtmac
Copy link

@ncooke3 +1 Thank you for your information. This is now starting to impact our users in production and causing crashes. When will be a new release containing these fixes?

same issue, and cause a lot of crashes.

@paulb777
Copy link
Member

We plan to release 11.3.0 the week of September 30th

@eeirinberg
Copy link

@ncooke3 +1 Thank you for your information. This is now starting to impact our users in production and causing crashes. When will be a new release containing these fixes?

same issue, and cause a lot of crashes.

+1 we are receiving 100s of crashes in production from this issue. Fixing this ASAP would be much appreciated.

@thomasdao
Copy link

We plan to release 11.3.0 the week of September 30th

@paulb777 For serious issues that crash the app in production, I strongly urge Firebase to always release the fix as soon as possible instead of waiting for the normal release cycle. Every time the app crashes, the user is very likely to give the app 1 star, and waiting for the Firebase release cycle would easily reduce the rating from 4.8 to 4 or below.

@paulb777
Copy link
Member

11.3.0 is now published.

@efstathiosntonas
Copy link

@mikehardy hi, can you please update rn-firebase with version 11.3.0? thanks brother

@mikehardy
Copy link
Contributor

Hey @efstathiosntonas - I posted a PR up there invertase/react-native-firebase#8042 but please note that while I'm helping as needed, Invertase is training up internally on react-native-firebase so this will work through others before release, please be patient :-)

@efstathiosntonas
Copy link

@paulb777 just updated to 11.3.0 on a react-native project and it seems the issue is still there:

Screenshot 2024-10-01 at 08 36 59

@mikehardy
Copy link
Contributor

@efstathiosntonas it appears like you have a good reproduction scenario? That may help a lot in nailing this down

@efstathiosntonas
Copy link

efstathiosntonas commented Oct 1, 2024

@mikehardy it’s kinda random, had to build on 5-6 devices before it shows up. It crashes on app launch only.

Let me know guys if you need something

@ncooke3
Copy link
Member

ncooke3 commented Oct 1, 2024

@efstathiosntonas, are you able to provide a complete crash report? The trace should look a little different after upgrading to 11.3, so I'd like to see what is happening across all threads. Also, which Xcode version are you reproducing it with? Thanks!

@paulb777
Copy link
Member

paulb777 commented Oct 1, 2024

@efstathiosntonas And please create a new issue with the crash report and issue template updated.

@efstathiosntonas
Copy link

@paulb777, @ncooke3 been trying to reproduce for like 15 builds now with no luck, got the issue template ready and I keep spamming the builds 😅

Probably a long shot but is there any chance a small internet interruption of 1 or < 1second to cause this instability on the rpc? I'm on Starlink and these small disconnects can happen from time to time.

@ncooke3
Copy link
Member

ncooke3 commented Oct 1, 2024

Thanks, @efstathiosntonas!

Probably a long shot but is there any chance a small internet interruption of 1 or < 1second to cause this instability on the rpc? I'm on Starlink and these small disconnects can happen from time to time.

I'm not sure, but it's a good thought. I believe there is a networking tool in Xcode that may help us simulate this.


First launches are a little different from successive launches as that's when Firebase is configured for the first time. Does there seem to be any difference in the repro rate when deleting the app between launches?

@efstathiosntonas
Copy link

efstathiosntonas commented Oct 1, 2024

I always delete the app before launches, will keep trying till I reproduce it, thank god mac studio spits fire lol.

About the internet disconnects, on 10.29.0 the crash never occurred locally or on production, (i don't have a prod version with 11.x)

@efstathiosntonas
Copy link

efstathiosntonas commented Oct 1, 2024

@ncooke3 @paulb777

edit: @ncooke3 checked Starlink app for disconnects when this happened and there weren't any, I guess we can rule this out.

@Druzrka
Copy link

Druzrka commented Oct 3, 2024

Hello @ncooke3, @paulb777, and others,

We're still experiencing crashes in production after updating to version 11.3.0. The crash events are occurring within the first second of the user's session.

Below is the stack trace (with some data, such as names, redacted for privacy reasons):

app-name-issue_b0e914da2c805c483404a82fb92d8df0_crash_session_e8a29db84fc54b75815895053c813e37_DNE_0_v2_stacktrace.txt

also crash report:

Crashed: com.apple.root.user-initiated-qos.cooperative
0 app-name 0x15ed0 __swift_project_boxed_opaque_existential_0 + 4299726544 (:4299726544)
1 app-name 0xd3c420 specialized AuthBackendRPCImplementation.callInternal(with:) + 271 (AuthBackend.swift:271)
2 libswift_Concurrency.dylib 0x610e4 + 252
3 libswift_Concurrency.dylib 0x624f4 + 144
4 libdispatch.dylib 0x15d8c _dispatch_root_queue_drain + 392
5 libdispatch.dylib 0x16590 _dispatch_worker_thread2 + 156
6 libsystem_pthread.dylib 0x4c40 _pthread_wqthread + 228
7 libsystem_pthread.dylib 0x1488 start_wqthread + 8

Any help or insights would be greatly appreciated. Thank you!

@paulb777
Copy link
Member

paulb777 commented Oct 3, 2024

I'm going to lock this for comments in favor of #13761. Please add any additional comments, backtraces or repros there.

@firebase firebase locked as resolved and limited conversation to collaborators Oct 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.