@mikelehen I have not been able to reproduce the issue myself, but generally users on Equinor/Statoil corporate network experience the issue, also some users with adblocker (VPN based i think) have reported this.

I never deployed the setting to production because of the performance issues, so I am unable to confirm if it helps or not, but I have some users that can help testing when/if performance is improved.

I did end up adding an option for the user to enable this manually, and have asked a few users to try it out, and it did indeed help with the issue, and they where able to use the application.

So I hope this option is not removed, as it will help even if performance is lower. Not all my users have huge amount of documents.

One user had issues on his Mac, same network as his phone (where it works). gist.

One user is on a "high security" corporate network, gist.

Behaviour without experimentalForceLongPolling, is no data is retrieved or being sent to firestore, but works when enabled.

Hope it helps.

Maybe it could be possible for firebase-js to detect when this is needed as a fallback?

@tgangso Thanks very much for the data. We'll plan to keep the option around unless the problem stops happening or we devices a better solution (like automatic fallback).

Hi. I cannot connect to firestore without experimentalForceLongPolling.

1. Windows, on a corporate network with McAfee endpoint security.

2. On first load I get "Could not reach Cloud Firestore backend. Backend didn't respond within 10 seconds.", but it works if I reload the page.

3. experimentalForceLongPolling seems to completely resolve the issue.

4. https://gist.github.com/dubvi5/1731f17940531d94451aeded761d08ad

Thanks.

Hello, we have a Firestore connection issue with one of our computers at work. The error message says "Firestore (6.3.0): Could not reach Cloud Firestore backend. Backend didn't respond within 10 seconds..."

This issue doesn't occur on the other computers within the same network. Enabling ExperimentalForceLongPolling seems to resolve the issue.

The problematic one is a Windows PC with "Symantec Endpoint Protection" installed. The others are all Mac computers without antivirus.

Here's the gist: https://gist.github.com/dcc82/270cc8eac2f0fe312aa4ac8befad7925

Thanks.

Hi,

• Firebase Auth works in all environments (with and without long polling).
• Firebase Firestore works in localhost but fails completely in corporate environment with Win10/WinDefender/Bluecoat layers. The corporate environment is a very standard.

With experimentalForceLongPolling set to true, the corporate environment works beautifully with excellent performance and responsiveness. The gist is here: https://gist.github.com/abrice/de6984ffb7361cf4c071d574e49eb552.

Note, we only use the web (no mobile).

So, the issue then becomes, is 'experimental' going to become 'production'? It seems very risky to proceed with a solution that may or may not be withdrawn at any time.

The Firebase Auth/Firestore solution is truely impressive and ForceLongPolling works! Please move it to production status.

@abrice Thanks for the feedback!

The high-level answer is:

1. We aren't currently planning to make experimentalForceLongPolling the default behavior because it has some performance drawbacks and it should not be necessary for the vast majority of users (but we're using this github issue to collect / monitor feedback in that regard).
2. That said, we have no intention to remove experimentalForceLongPolling until / unless we have a better option for any users that are encountering the connection issues it currently solves.

For your specific case, can you elaborate on "Bluecoat layers"? I guess Blue Coat Systems was acquired by Symantec and perhaps this is the current version of their proxy software: https://www.symantec.com/products/proxy-sg-and-advanced-secure-gateway ?

It looks like it is known to cause problems do to SSL interception / proxying, e.g. with Windows Update and Office 365. So I'm not at all surprised it causes issues for Firestore as well, although interestingly this is the first report I've seen.

I don't suppose you or your IT department would be interested in engaging with Symantec on this issue and see if it can be resolved? I'd be happy to participate in the discussion and provide technical details, etc. If you're interested, feel free to reach out / CC me at [email protected]. Thanks!

Any update on if it is possible to make it fall back automatically to forceLongPolling?

I agree there are performance drawbacks, and should not be the default option.

I get a few questions every week from users experiencing problems (mostly on corporate networks), and enabling this in the settings of my app always resolve it.

Hi @mikelehen ,

The corporate runs O365 and has dealt with the challenges of Windows Updates. The network side is complex as there are a few other parties involved that provision various levels of proxy servers, anti-virus, load balancing, etc as-a-service. There's quite a lot of complex traffic moving about so, unfortunately, I can't help any further on pursuing a more in-depth technical examination.

But it's excellent news about your approach to experimentalForceLongPolling. Thank you!

@tgangso Adding a fallback mechanism is still on our radar but not something we're actively pursuing. Unfortunately it would be pretty complicated to add since it's hard to differentiate this case from being offline or even just on a slow network, so it's not obvious when to employ the fallback, etc. Right now we're still gathering feedback on the scope of the problem. Thanks!

Hi @mikelehen

I have been seeking help from Firebase Support, and after one and a half week we came up with changing the firestore settings to experimentalForceLongPolling:true.
With this setting, I was able to reduce loading times (when reading through .get()) from 50-60 seconds to 137 milliseconds. Before enabling forceLongPolling I would get a error when trying to add anything due to timeout.

When using any website requesting data from Cloud Firestore, I am unable to use the site. On my own however, I enable the setting simply for me to be able to actually use it.
I have been working on this from two different computers, and tried on to different networks as well.
Furthermore, I have tried disabling anything from proxies to firewalls - even Windows Defender Firewall. Nothing helped except enabling forceLongPolling.

@DJ-Shady02 Thank you for the information! Usually this issue only manifests in the face of specific corporate proxy or antivirus software, etc., so the fact that you're seeing it on two different computers and two different networks, even with disabling proxies / firewalls is unusual. I would still guess that there's some common factor that's causing it to happen, but it's hard to know what it might be. If by chance you're able to collect any more information that helps isolate the root cause (by trying from yet more devices including phones or tablets) or more networks (coffee shop, etc.), let us know. Thanks!

@mikelehen Finally, I have an update!

I have bought myself a new computer. Used Firestore for a month without issues. I install Bullguard, bam! Same issue as before. Being assured the issue was created by Bullguard, I contacted their support. It appears their "Safe Surfing" (The function name is translated from my native language, so it may vary), is causing the issue. Disabling it will trade a warning notification on sketchy websites with fully functional firestore without using forceLongPolling - worth it!

While this is not directly helpful to you on the topic of the experimentalForceLongPolling option, it can help others avoid it.

@DJ-Shady02 Thanks for the report!

We're fairly certain that this problem isn't going away and are working on building logic into the SDK that will be able to automatically fall back on long polling.

gist: https://gist.github.com/Great62/335089807ad609ad6a5c0cd578bc9f63 Basically failing at everything, do you guys have any idea how I could fix this? Thanks. Any bit helps, really desperate here.

@Great62 Hi! I'm sorry to hear about your frustrations!

In your log, I'm seeing that you have FAILED from 1-6, and SUCCEEDED from 7-9.

The 1-6 failure was likely due to our testing server problem. I've just restarted the server and it should be working now! (sorry for the confusion!)

The success from 7-9 is indicating that Firestore listen should be working with or without both flags specified at all.

So it's quite odd that things are not working for you.

Could you try comparing the request/response from debug-my.firebaseapp.com v.s. your application, in Chrome networking tab?

Hope that helps!

Hello, I'm having issues with a client using Zscaler. Their Firebase test is as follows:

All tests done.
1: webchannel.googleapis.com with default options: FAILED (75043ms)
2: webchannel.googleapis.com with detectBufferingProxy: SUCCEEDED (1142ms)
3: webchannel.googleapis.com with forceLongPolling: SUCCEEDED (1015ms)
4: webchannel.sandbox.google.com with default options: FAILED (75029ms)
5: webchannel.sandbox.google.com with detectBufferingProxy: SUCCEEDED (1540ms)
6: webchannel.sandbox.google.com with forceLongPolling: SUCCEEDED (1643ms)
7: Firestore listen test with default options: FAILED (75039ms)
8: Firestore listen test with detectBufferingProxy: SUCCEEDED (699ms)
9: Firestore listen test with forceLongPolling: SUCCEEDED (319ms)

Is the solution more or less to forceLongPolling? How much of a degradation in performance can be expected for data sets in the tens or hundreds of thousands?

@sparkts-shaun looking through my notes from when this happened with one of my customers behind a Zscalar:

From another SaaS vendor:

If you have Zscaler, we have reports that adding an SSL bypass rule for the firestore.googleapis.com domain will resolve blocking issues. Here is a link for more information: https://help.zscaler.com/zia/controlling-access-google-consumer-apps

Also helpful is configuring SSL Certificate Pinning for Google Shared Services as outlined here:

https://help.zscaler.com/zia/certificate-pinning-and-ssl-inspection

There may also have been an issue with the Zscaler requiring the strict-origin-when-cross-origin policy.

Ultimately, the customer had to reach out to Zscalar for support, and I'm not sure I received a full explanation other than they had to update their configuration/rules.

@bgetsug Thank you for the quick response!

The customer is hesitant to implement a bypass rule for as broad a domain as firestore.googleapis.com, which was my initial suggestion to them.

You may not be the person to ask, but do you have any concept what degree of performance impact the "experimentalLongPolling" flag introduces / at what scale it is noticeable? We would have to roll this out to all of our customers given our current infrastructure, and I'm hesitant to just flip that on.

Is the solution more or less to forceLongPolling? How much of a degradation in performance can be expected for data sets in the tens or hundreds of thousands?

@sparkts-shaun Hi! Yes, it does looks like from your results that both "longpolling" flags would be able to resolve the issue for you :)

Also, as per my understanding, Auto-detect long polling is already on by default since SDK 9.22:
https://firebase.google.com/support/release-notes/js

So far we haven't gotten any feedbacks about it having a negative impact on performance or reliability.

So i think it should be rather safe if you're still on an earlier SDK and doesn't have it turned on yet :)

(If you're on 9.22 onwards, there might be other issues at hand, like @bgetsug has indicated.)

@sparkts-shaun, I understand your hesitance. I don't have experience enabling that flag, and I chose to avoid enabling it given it's "experimental," and the same customer did not have connectivity issues using our mobile app (with underlying native SDKs) on their network.

In my case, I had to have a couple of conference calls with this particular customer's IT department and encourage them to get help with their Zscalar. I don't think they had to add an exception for the domain entirely, but they did have to "tune" their Zscalar configuration to not "mangle" the packets at hand.

You could also foster more understanding about Firestore to help them better assess the perceived risk. I'd argue that while firestore.googleapis.com is a public endpoint, I wouldn't consider it a "broad domain." It's a single web service supported by secure web protocols, per-tenant security rules, and an infrastructure certified by many security compliance frameworks.

gist: https://gist.github.com/Great62/335089807ad609ad6a5c0cd578bc9f63 Basically failing at everything, do you guys have any idea how I could fix this? Thanks. Any bit helps, really desperate here.

@Great62 Hi! I'm sorry to hear about your frustrations!

In your log, I'm seeing that you have FAILED from 1-6, and SUCCEEDED from 7-9.

The 1-6 failure was likely due to our testing server problem. I've just restarted the server and it should be working now! (sorry for the confusion!)

The success from 7-9 is indicating that Firestore listen should be working with or without both flags specified at all.

So it's quite odd that things are not working for you.

Could you try comparing the request/response from debug-my.firebaseapp.com v.s. your application, in Chrome networking tab?

Hope that helps!

Hey! Thanks. Yes, every now and then I get the message in the console telling me the backend wasn't reached in time. However, in the network tab, there are only requests with a 200 status, except for one: 'https://firestore.googleapis.com/google.firestore.v1.Firestore/Listen/channel?VER=8&database=projects%...' (TYPE: terminate). Other than that, I have only successful requests even though the backend wasn't reached... Any thoughts on this? What request would you like to see exactly?

Thank you so much for your help!
Have a great day.

Hey! Thanks. Yes, every now and then I get the message in the console telling me the backend wasn't reached in time. However, in the network tab, there are only requests with a 200 status, except for one: 'https://firestore.googleapis.com/google.firestore.v1.Firestore/Listen/channel?VER=8&database=projects%...' (TYPE: terminate). Other than that, I have only successful requests even though the backend wasn't reached... Any thoughts on this? What request would you like to see exactly?

Thanks a lot for the additional details!

So you're saying that the "backend not reached" error is not consistent on your client, but rather sporadic?

If so, it seems to indicates an issue that is NOT related to a buffering proxy, which normally remains a constant issue when there is one. And if so, the flag mentioned in this issue might not be able to help you.

In terms of the network tab, i was originally thinking that you'd get a screenshot of all the requests related to the "Listen/channel" path as you've mentioned. That looks kinda like this (although the paths here are different):

And in this case since you get a failed request, if you can gather all the HTTP Headers of the failed requests (either via screenshot or github gist), it could help with understanding the issue too.

Thank you so much for your help! Have a great day.

You're most welcome and hope you have a great day too :)

@bgetsug @sampajano For posterity, we have enabled the flag and haven't run into negative consequences so far, will update here if something goes awry.

Hey! Thanks. Yes, every now and then I get the message in the console telling me the backend wasn't reached in time. However, in the network tab, there are only requests with a 200 status, except for one: 'https://firestore.googleapis.com/google.firestore.v1.Firestore/Listen/channel?VER=8&database=projects%...' (TYPE: terminate). Other than that, I have only successful requests even though the backend wasn't reached... Any thoughts on this? What request would you like to see exactly?

Thanks a lot for the additional details!

So you're saying that the "backend not reached" error is not consistent on your client, but rather sporadic?

If so, it seems to indicates an issue that is NOT related to a buffering proxy, which normally remains a constant issue when there is one. And if so, the flag mentioned in this issue might not be able to help you.

In terms of the network tab, i was originally thinking that you'd get a screenshot of all the requests related to the "Listen/channel" path as you've mentioned. That looks kinda like this (although the paths here are different):

And in this case since you get a failed request, if you can gather all the **HTTP Headers** of the failed requests (either via screenshot or github gist), it could help with understanding the issue too.

Thank you so much for your help! Have a great day.

You're most welcome and hope you have a great day too :)

Hey @sampajano @cherylEnkidu, thanks for your feedback.
I have a screenshot and a url to provide.

I'm starting to think there's something wrong with my app instead, because I have 2 apps running on next that use different versions of NextJs. On the first app (that I don't use anymore and can't use for production), I never get the issue, but on the second app, that has a more recent version of Next and firebase, I sometimes get the issue.
App 1 (deprecated but never has the issue): https://audio-consulting.vercel.app/products
App 2 (production app, sometimes has the issue): https://audio-consulting-next.vercel.app/products

I'm starting to think it's probably due to the version of firebase I'm using, but I can't go back because the old versions don't work for some reason.
App 1: "firebase": "^9.21.0", "next": "13.3.0", "react": "18.2.0", "@chakra-ui/next-js": "^2.1.5", "@chakra-ui/react": "^2.5.5"
App 2: "firebase": "^10.7.0", "next": "13.5.4", "react": "^18", "@chakra-ui/next-js": "^2.1.5", "@chakra-ui/react": "^2.8.1"

There isn't much difference between the 2 apps. Both are aimed at the same backend, I don't know if that might be the problem? Anyway, the problem still persists, and I can't understand why, even more so now that I know it works on the other deprecated app...

Any ideas?

Finally after getting other occurrences where it didn't work, there's no failed request.
Screenshots of network tab:

Thank you so much again for your support honestly.
Have a great day.
Adam.

@Great62 Thanks again for the detailed feedback and providing the links to your apps! It's very helpful!

I'm starting to think it's probably due to the version of firebase I'm using, but I can't go back because the old versions don't work for some reason.
App 1: "firebase": "^9.21.0", "next": "13.3.0", "react": "18.2.0", "@chakra-ui/next-js": "^2.1.5", "@chakra-ui/react": "^2.5.5"
App 2: "firebase": "^10.7.0", "next": "13.5.4", "react": "^18", "@chakra-ui/next-js": "^2.1.5", "@chakra-ui/react": "^2.8.1"

This is very important information.. And rather telling.. Because experimentalAutoDetectLongPolling was turned on at 9.22.0 (according to release notes, so your App 1 would be using the version with that flag off, whereas App 2 would be using the flag on.

Although, when i inspected the network traffic (on your App 2), i didn't see experimentalAutoDetectLongPolling taking effect, so my guess is it's possibly unrelated.. But in any case, it might be something good to verify..

Can you try the following things? It would be best if you could try both, but even just doing one of them would provide good evidences:

1. In your App 1, updated firebase to 9.22.0, and see if the issue disappears.

2. In your App 2, set the FirestoreSettings.experimentalForceLongPolling flag to false, and see if the issue disappears.

My guess is, if your problem is indeed related to experimentalAutoDetectLongPolling, then both above things will make the issue disappear.

If neither is making a difference, then it means your issue might be hiding somewhere unrelated.. :)

Hope that helps!!

Thank you so much again for your support honestly.
Have a great day.

You're very welcome. Hope you have a great day and holidays too! :) (I'll be on vacation for 3 weeks so i won't be replying in this time :))

I need to turn on experimentalForceLongPolling or experimentalAutoDetectLongPolling for one of my collection. Other collections run ok without this.

@vietstone-ng Hi Thanks for informing us!

May i ask what is your Firebase version? From what i understood, experimentalAutoDetectLongPolling should have been enabled since 9.22.0, so you shouldn't have to set it manually. Was it not the case for you?

Thanks!

@sampajano Hi, I use Firebase 10.9.0

@sampajano Hi, I use Firebase 10.9.0

@vietstone-ng Oh Thanks for confirming!!

Could you confirm again that turning experimentalAutoDetectLongPolling on manually fixes the issue you had? This is quite interesting because it's supposed on by default since 9.22.0..

It would indicate some flag configuration issues if it's not actually on by default..

Thanks a lot!

@sampajano I'm not sure if I wrote the correct code, because I'm a newbie with JS.
I created a Vite React project, with package.json, firebaseConfig.js and my component FireStoreTest.jsx as below:
package.json

# package.json

{
  "name": "vite_firestore",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "lint": "eslint . --ext js,jsx --report-unused-disable-directives --max-warnings 0",
    "preview": "vite preview"
  },
  "dependencies": {
    "firebase": "^10.9.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0"
  },
  "devDependencies": {
    "@types/react": "^18.2.66",
    "@types/react-dom": "^18.2.22",
    "@vitejs/plugin-react": "^4.2.1",
    "eslint": "^8.57.0",
    "eslint-plugin-react": "^7.34.1",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.6",
    "vite": "^5.2.0"
  }
}

// firebaseConfig.js

import firebase from 'firebase/compat/app';
import 'firebase/compat/firestore';

const firebaseConfig = {
    ...
  };

firebase.initializeApp(firebaseConfig);

const firestore = firebase.firestore();
firebase.firestore().settings({ experimentalAutoDetectLongPolling: true});
firebase.firestore.setLogLevel('debug');

export default firestore;

// FireStoreTest.jsx

import { useEffect } from 'react'
import firestore from './firebaseConfig'

function FireStoreTest() {
  useEffect(() => {
    const fetchData = async () => {
      const collectionRef = firestore.collection('la')

      try {
        const snapshot = await collectionRef.get()
        snapshot.forEach((doc) => {
          console.log(doc.id, '=>', doc.data())
        })
      } catch (err) {
        console.log('Error getting documents', err)
      }
    }

    fetchData()
  }, [])

  return <div>{/* Your component content */}</div>
}

export default FireStoreTest

In firebaseConfig.js I need to turn on experimentalAutoDetectLongPolling for my app to work.

Below is the error log when I turned it off:

[2024-04-10T04:21:49.907Z]  @firebase/firestore: Firestore (10.9.0): Could not reach Cloud Firestore backend. Backend didn't respond within 10 seconds.
This typically indicates that your device does not have a healthy Internet connection at the moment. The client will operate in offline mode until it is able to successfully connect to the backend.

@vietstone-ng Thanks for confirming!!

This is slightly concerning to me because experimentalAutoDetectLongPolling is supposed to be on by default since Firebase 9, yet you have to turn it on manually in 10.9.0.

@wu-hui @dconeybe Hi Hui and Denver, could you help take a look and see whether the user is configuring his app correctly?

Any theory on why experimentalAutoDetectLongPolling had to be manually turned on?

Thanks!

@vietstone-ng I see that your code is using firebase/compat/firestore The "compat" libraries are mostly provided for migration to the new APIs adding in v9.0.0 of the Firebase JS SDK. If you are writing a new project, please just use the non-compat firebase/app and firebase/firestore libraries. Also, if you're able, please provide a github link to the application that you use to reproduce this problem so that we can try to reproduce for ourselves and/or look at your code to understand your usage pattern.

@vietstone-ng I see that your code is using firebase/compat/firestore The "compat" libraries are mostly provided for migration to the new APIs adding in v9.0.0 of the Firebase JS SDK. If you are writing a new project, please just use the non-compat firebase/app and firebase/firestore libraries. Also, if you're able, please provide a github link to the application that you use to reproduce this problem so that we can try to reproduce for ourselves and/or look at your code to understand your usage pattern.

@dconeybe Does this mean if you use the compact libraries that you are hard-coded to Firebase JS version 9.0.0 even if you upgrade it in your package.json?

Does this mean if you use the compact libraries that you are hard-coded to Firebase JS version 9.0.0 even if you upgrade it in your package.json?

No. The latest version of the firebase-compat libraries use the latest version of the non-compat libraries under the hood. The compat libraries do, however, add a layer of indirection making debugging more difficult and, also, they are intended for a migration from the old API to the new API rather than something to use out of the gate.

Also, I confirmed that using the "compat" libraries should have experimentalAutoDetectLongPolling=true by default and explicitly specifying it in the settings should not make any difference.

@vietstone-ng If you'd like us to continue investigation, please log a new issue and we will continue investigation there. What you are reporting is definitely not expected behavior.

@dconeybe I've just created a new issue to describe the problem with compat lib, as well as another problem with normal modular lib. Please help to check. Thank you very much.
#8176

Is the debugging website still working as expected? I get a lot of failed tests.

******************************************************
All tests done.
1: webchannel.googleapis.com with default options: FAILED (39793ms)
2: webchannel.googleapis.com with detectBufferingProxy: FAILED (39790ms)
3: webchannel.googleapis.com with forceLongPolling: FAILED (39689ms)
4: webchannel.sandbox.google.com with default options: FAILED (39826ms)
5: webchannel.sandbox.google.com with detectBufferingProxy: FAILED (19915ms)
6: webchannel.sandbox.google.com with forceLongPolling: FAILED (149889ms)
7: Firestore listen test with default options: SUCCEEDED (555ms)
8: Firestore listen test with detectBufferingProxy: SUCCEEDED (336ms)
9: Firestore listen test with forceLongPolling: SUCCEEDED (1005ms)
10: Firestore listen test with FetchXmlHttpFactory: SUCCEEDED (379ms)
11: Firestore listen test with XMLHttpRequest: SUCCEEDED (445ms)

**************************************

Is the debugging website still working as expected? I get a lot of failed tests.

hmm... Let me revisit it and make sure it's up-to-date.