Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage emulator: Calling file.save() two times appends instead of replacing the original file #3406

Closed
jpreynat opened this issue May 26, 2021 · 2 comments

Comments

@jpreynat
Copy link
Contributor

[REQUIRED] Environment info

firebase-tools: 9.11.0

Platform: macOS, Ubuntu

[REQUIRED] Test case

import { Storage } from '@google-cloud/storage';

const gcs = new Storage({
    projectId: // ...,
    credentials: // ...
});
const bucket = gcs.bucket('my-bucket');

async function run() {
    const path = 'some/file';
    const content1 = 'some-content';

    const file = bucket.file(path);
    await file.save(Buffer.from(content1), {
        resumable: false,
        validation: false
     });

    const [readContent1] = await file.download({
        validation: false,
    });

    readContent1.toString('utf8') === content1; // true

    const content2 = 'some-other-content';
    await file.save(Buffer.from(content2), {
        resumable: false,
        validation: false
     });

    const [readContent2] = await file.download({
        // This is necessary to check the content of the second download, otherwise it throws with:
        // "The downloaded data did not match the data from the server. To be sure the content is the same, you should download the file again."
        validation: false
    });

    readContent2.toString('utf8') === content2; // false
    // readContent2 === 'some-contentsome-o'
}

run();

[REQUIRED] Steps to reproduce

Simply run the above example with the storage emulator properly configured.

[REQUIRED] Expected behavior

The content of the newly uploaded file should replace the existing one, so readContent2 === 'some-other-content' in the test case.

[REQUIRED] Actual behavior

The new content is appended to the previously uploaded one, instead of completely replacing it, which makes the storage emulator useless if a file is modified at least once.

It seems that the gcloud storage layer currently calls oneShotUpload() that appends content to the existing one, instead of replacing it in this case.

@samtstern
Copy link
Contributor

@jpreynat thank you for the very clear bug report!

@abeisgoat
Copy link
Contributor

Good catch will try to get a fix out today!

abeisgoat added a commit that referenced this issue May 26, 2021
@joehan joehan closed this as completed in 2cc2293 May 26, 2021
devpeerapong pushed a commit to devpeerapong/firebase-tools that referenced this issue Dec 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants