Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

firestore:delete not working against emulator since version 12.6.2 / fails in 12.7+ #6537

Closed
mikehardy opened this issue Nov 22, 2023 · 9 comments · Fixed by #6772
Closed

Comments

@mikehardy
Copy link

mikehardy commented Nov 22, 2023

[REQUIRED] Environment info

firebase-tools: 12.9.1

Platform: any platform, mostly testing macOS

[REQUIRED] Test case

Note that this will work against firestore in the cloud if not careful please do not destroy data accidentally!

It does not delete data against the emulator but against the cloud it works just fine!

  • Start the firestore emulator
  • Create a firestore collection in your emulator
  • run FIRESTORE_EMULATOR_HOST=localhost:8080 yarn firebase firestore:delete --all-collections --project <your project name> --account <your acct> --force

(or, at least I run it with yarn around it, but I'm sure you see how to run it - project name / account are whatever you need in your local env against your emulator)

[REQUIRED] Steps to reproduce

The test case above is it, if you call this using firebase-tools 12.6.2, all the collections will be removed. From 12.7.0 onwards the collections will not be removed.

The emulator server side seems version-insensitive, the behavior varies by version on the calling side

[REQUIRED] Expected behavior

All the collections should be removed, for a fresh data load (I test data loads + migrations...)

[REQUIRED] Actual behavior

The collections stay in place, no data is deleted.

I bisected the versions to see which version began failing because I knew it used to work, but it took me a while to find time to pinpoint this for this repo (I'm normally swamped, as the react-native-firebase maintainer 👋 😆 )


firebase-tools 10.2.2 calling to 12.9.1 works (use node v16 to work)
firebase-tools ^10 (10.9.2) calling to 12.9.1 works (use node v16 to work)
firebase-tools ^11 (11.30.0) callnig to 12.9.1 works (use node v16 to work)
Firebase-tools ^12 (12.9.1) calling to 12.9.1 does not work (v16, v18 or v20 node all fail)
callnig with 12.0.0 to 12.9.1 works
~12.5 (12.5.4) works
~12.7 (12.7.0) does not work
12.6.2 works

diff between 12.6.2 and 12.7.0 is the problem --> v12.6.2...v12.7.0

@google-oss-bot
Copy link
Contributor

This issue does not have all the information required by the template. Looks like you forgot to fill out some sections. Please update the issue with more information.

@mikehardy
Copy link
Author

They both appear like they are working based on debug logs, I just did the deletes against a trimmed down version of my data set (just domain data really, not our transaction data) and I observed the behavior I describe despite the debug logs both looking pretty similar to my untrained eye

12.6.2-debug.log
12.9.1-debug.log

@aalej
Copy link
Contributor

aalej commented Nov 23, 2023

Hey @mikehardy, thanks for the detailed report, this is a lot of info! I was able to reproduce the behavior you described. As you mentioned, it does look like the issue started occurring on v12.7.0. Let me raise this to our engineering team so that they can take a look and investigate the issue.

@mikehardy
Copy link
Author

I spend most of my time in react-native-firebase trying to reproduce issues users send me, and then trying to fix them, so I'm very sympathetic to reproduction challenges and sensitive to not having enough info to reproduce. It's hard maintaining software!

Glad you were able to see it with info provided, and I'll be happy to collaborate to get a fix going or verify one if needed. Cheers @aalej

@Pickachu
Copy link

Pickachu commented Dec 14, 2023

Just deleted my production users collection after running my local tests, 😢 .
For my luck i had last month's backup.

Changing from firestoreOrigin to firestoreOriginOrEmulator as a parameter for the gcp firestore cliente seems to fix the exact issue for me:

import { firestoreOrigin } from "../api";
import { Client } from "../apiv2";
import { logger } from "../logger";
const apiClient = new Client({
auth: true,
apiVersion: "v1",
urlPrefix: firestoreOrigin,
});

It appears that firestoreOriginOrEmulator as a valid export from the API:

export const firestoreOriginOrEmulator = utils.envOverride(
Constants.FIRESTORE_EMULATOR_HOST,
utils.envOverride("FIRESTORE_URL", "https://firestore.googleapis.com"),
(val) => {
if (val.startsWith("http")) {
return val;
}
return `http://${val}`;
}
);
export const firestoreOrigin = utils.envOverride(
"FIRESTORE_URL",
"https://firestore.googleapis.com"
);

I don't know why this choice was made though. So just letting info here in hopes that help the team to fix this issue.

Edit:

As a quick workaround setting the FIRESTORE_URL environment variable will override firestoreOrigin and yield the correct results.

@phillipshaong
Copy link

I am getting the same behavior in v13.0.2. Even though I have set FIRESTORE_EMULATOR_HOST=127.0.0.1:8080, any collection or document I delete using firebase firestore:delete will delete in the production Firestore database, and not the Firestore Emulator.

(project-id) user@user-mbp project-id-main-app % firebase firestore:delete /business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv --debug   
i  You have set FIRESTORE_EMULATOR_HOST=127.0.0.1:8080, this command will execute against the Firestore Emulator running at that address. 
[2023-12-21T22:35:20.994Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2023-12-21T22:35:20.994Z] > authorizing via signed-in user ([email protected])
[2023-12-21T22:35:20.994Z] [iam] checking project project-id-d1935 for permissions ["datastore.entities.delete","datastore.entities.list","firebase.projects.get"]
[2023-12-21T22:35:20.995Z] >>> [apiv2][query] POST https://cloudresourcemanager.googleapis.com/v1/projects/project-id-d1935:testIamPermissions [none]
[2023-12-21T22:35:20.995Z] >>> [apiv2][(partial)header] POST https://cloudresourcemanager.googleapis.com/v1/projects/project-id-d1935:testIamPermissions x-goog-quota-user=projects/project-id-d1935
[2023-12-21T22:35:20.995Z] >>> [apiv2][body] POST https://cloudresourcemanager.googleapis.com/v1/projects/project-id-d1935:testIamPermissions {"permissions":["datastore.entities.delete","datastore.entities.list","firebase.projects.get"]}
[2023-12-21T22:35:21.307Z] <<< [apiv2][status] POST https://cloudresourcemanager.googleapis.com/v1/projects/project-id-d1935:testIamPermissions 200
[2023-12-21T22:35:21.308Z] <<< [apiv2][body] POST https://cloudresourcemanager.googleapis.com/v1/projects/project-id-d1935:testIamPermissions {"permissions":["datastore.entities.delete","datastore.entities.list","firebase.projects.get"]}
? You are about to delete the document at business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv for projects/project-id-d1935/databases/(default)/documents. Are you sure? Yes
[2023-12-21T22:35:34.818Z] >>> [apiv2][query] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery [none]
[2023-12-21T22:35:34.819Z] >>> [apiv2][body] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery {"structuredQuery":{"limit":1,"from":[{"allDescendants":true}],"select":{"fields":[{"fieldPath":"__name__"}]},"orderBy":[{"field":{"fieldPath":"__name__"}}]}}
[2023-12-21T22:35:34.837Z] <<< [apiv2][status] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery 200
[2023-12-21T22:35:34.837Z] <<< [apiv2][body] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery [{"readTime":"2023-12-21T22:35:34.831979Z","done":true}]
[2023-12-21T22:35:34.838Z] >>> [apiv2][query] DELETE https://firestore.googleapis.com/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv [none]
[2023-12-21T22:35:35.057Z] <<< [apiv2][status] DELETE https://firestore.googleapis.com/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv 200
[2023-12-21T22:35:35.058Z] <<< [apiv2][body] DELETE https://firestore.googleapis.com/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv {}
[2023-12-21T22:35:35.061Z] >>> [apiv2][query] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery [none]
[2023-12-21T22:35:35.062Z] >>> [apiv2][body] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery {"structuredQuery":{"limit":7500,"from":[{"allDescendants":false}],"select":{"fields":[{"fieldPath":"__name__"}]},"orderBy":[{"field":{"fieldPath":"__name__"}}]}}
[2023-12-21T22:35:35.070Z] <<< [apiv2][status] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery 200
[2023-12-21T22:35:35.070Z] <<< [apiv2][body] POST http://127.0.0.1:8080/v1/projects/project-id-d1935/databases/(default)/documents/business/Iy4CvVsmQkAdEe8s7LB1/location/LUuoVsBHYBvDqa0N5wjv:runQuery [{"readTime":"2023-12-21T22:35:35.067257Z","done":true}]

@phillipshaong
Copy link

Taking @Pickachu suggestion, I was able to delete the data in the emulator by setting: export FIRESTORE_URL="http://127.0.0.1:8080".

@mikehardy
Copy link
Author

Nice - confirmed for me that if I set FIRESTORE_URL in my script I successfully get collection deletion in my local emulator

including an example in case it maybe helps someone

  // options parsing etc etc, determine my script should use emulator, set these things:
  process.env.FIRESTORE_URL = "http://localhost:8080";  // that's new
  process.env.FIRESTORE_EMULATOR_HOST = 'localhost:8080';
  process.env.FIREBASE_AUTH_EMULATOR_HOST = 'localhost:9099';
  process.env.GCLOUD_PROJECT = getFirebaseProjectName();

  // ...do all your stuff, including maybe awaiting a call to this function and it'll work as child inherits parent env
  function deleteAllFirestoreCollections() {
    const child = spawnSync(
      'yarn',
      [
        'firebase',
        'firestore:delete',
        '--all-collections',
        '--project',
        getFirebaseProjectName(),
        '--account',
        '[email protected]',
        '--force',
      ],
      { shell: process.platform === 'win32' },
    );

@velocd
Copy link

velocd commented Feb 12, 2024

Was confused 🤔 on this until I found this post. This is still an issue with 13.2.1, and as previously mentioned setting FIRESTORE_URL is the fix. I don't see FIRESTORE_URL documented anywhere in the official docs.

Bad:

export FIRESTORE_EMULATOR_HOST="127.0.0.1:8080"

firestore:delete --debug prints [apiv2][body] POST https://firestore.googleapis.com/....

Good:

export FIRESTORE_EMULATOR_HOST="127.0.0.1:8080"
export FIRESTORE_URL="http://127.0.0.1:8080"

firestore:delete --debug prints [apiv2][status] POST http://127.0.0.1:8080/....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants