Skip to content

Conversation

@GarrettBurroughs
Copy link
Contributor

Support for cloud tasks within the emulator

defaultUri: string;

// The configurations below this point do not currently have any effect on how the task queues are handled within the emulator
secrets?: string[]; // TODO(gburroughs): look into how we can handle this
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secrets should be handled by the functions emulator at startup, but it would be good to confirm this.

Comment on lines +596 to +602
const tasksAddr = legacyGetFirstAddr(Emulators.TASKS);
const tasksEmulator = new TasksEmulator({
host: tasksAddr.host,
port: tasksAddr.port,
});

await startEmulator(tasksEmulator);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: does this mean we always run the task emulator whenever we run the functions emulator?

I think this is fine for now, but I think an ideal solution would run the task emulator only if a task queue function is defined.

I don't know whether something like that is possible, but if you have time would love for you to see if that's even feasible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the task queue emulator is run whenever the functions emulator is run.

There is the code here:

if (!this.controller.isRunning()) {

which only starts the listening logic of the task queue emulator once an actual task has been enqueued to avoid doing unnecessary updates while the user isn't doing anything with task queues in their functions code. The express server is still spun up every time the functions emulator starts though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I wonder if we should instead have Task Queue Emulators be explicitly started, just like the Pub/sub emulator.

@blidd-google thoughts?

@GarrettBurroughs GarrettBurroughs marked this pull request as ready for review August 14, 2024 20:40
@blidd-google blidd-google enabled auto-merge (squash) August 27, 2024 17:32
@joehan joehan disabled auto-merge August 27, 2024 20:40
@joehan joehan merged commit 2b1a314 into master Aug 27, 2024
@joehan
Copy link
Member

joehan commented Aug 27, 2024

Overriding the CLA check here as Garret worked on this during his time at Google.

@joehan joehan deleted the gburroughs-taskqueue-emulator branch August 27, 2024 21:02
hlshen pushed a commit that referenced this pull request Sep 3, 2024
* register cloud task triggers

* Create and register task queue emulator

* Add basic features of task queue emulator

* Implement a queue to improve performance of tasks emulator

* Remove unused import

* Register task queues from functions emulator

* change task queue endpoint to accept IDs in body instead of query params

* set enviornment variable when task queue emulator is running

* Remove bug where tasks were getting dispatched too fast

* Rework task enqueueing API and support task deletion

* properly support task deletion

* Refactor Task Dispatching Algorithm

* Add task deduplication for all tasks that have ever been added

* Refactor Task Queue

* Add tests for task queue emulator

* respond to PR review

* update firebase-config.json

* use node-fetch

* fix tests

* Make tasks emulator start up whenever functions emulator starts

* fix error message

* Add statistics endpoint

* Add in global task deduplication

* Add headers to request

* Review Changes

* Remove console logs

* Update backoff calculations

* add changelog

---------

Co-authored-by: blidd-google <[email protected]>
Co-authored-by: joehan <[email protected]>
Co-authored-by: Brian Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants