Skip to content

fix: Update minimum protobuf version to >=6.30.2#5458

Open
misohu wants to merge 1 commit intofeast-dev:masterfrom
misohu:fix-protobuf-version
Open

fix: Update minimum protobuf version to >=6.30.2#5458
misohu wants to merge 1 commit intofeast-dev:masterfrom
misohu:fix-protobuf-version

Conversation

@misohu
Copy link
Copy Markdown

@misohu misohu commented Jun 17, 2025

What this PR does / why we need it:

This PR updates the minimum required protobuf version in pyproject.toml to >=6.30.2.

The current version range (>=3.20.3,<7) allows older versions like 4.25.6 or 5.26.0 that do not include google.protobuf.runtime_version, which is used by Feast internally. This causes an ImportError when importing modules like FeatureStore.

Updating the minimum ensures that users install a compatible version of protobuf and avoids runtime failures.

Which issue(s) this PR fixes:

Fixes #5457

@ntkathole
Copy link
Copy Markdown
Member

ntkathole commented Jun 17, 2025

@misohu Thanks for the report and fix, please modify https://github.com/feast-dev/feast/blob/master/setup.py#L36 and run make lock-python-dependencies-all to compile the requirements.

@ntkathole ntkathole changed the title fix: update minimum protobuf version to >=6.30.2 fix: Update minimum protobuf version to >=6.30.2 Jun 17, 2025
@misohu
Copy link
Copy Markdown
Author

misohu commented Jun 17, 2025

Hey @ntkathole thanks for your message. Looks like the resolution fails with

× No solution found when resolving dependencies:
  ╰─▶ Because grpcio-tools>=1.56.2,<=1.62.3 depends on protobuf>=4.21.6,<5.0.dev0 and only the following versions of grpcio-tools
      are available:
          grpcio-tools<=1.56.2
          grpcio-tools==1.57.0
          grpcio-tools==1.58.0
          grpcio-tools==1.58.3
          grpcio-tools==1.59.0
          grpcio-tools==1.59.2
          grpcio-tools==1.59.3
          grpcio-tools==1.59.5
          grpcio-tools==1.60.0
          grpcio-tools==1.60.1
          grpcio-tools==1.60.2
          grpcio-tools==1.61.3
          grpcio-tools==1.62.0
          grpcio-tools==1.62.1
          grpcio-tools==1.62.2
          grpcio-tools>=1.62.3
      we can conclude that grpcio-tools>=1.56.2,<=1.62.3 depends on protobuf>=4.21.6,<5.0.dev0.
      And because feast depends on protobuf>=6.30.2 and grpcio-tools>=1.56.2,<=1.62.3, we can conclude that your requirements are
      unsatisfiable.

      hint: Pre-releases are available for `grpcio-tools` in the requested range (e.g., 1.62.0rc1), but pre-releases weren't enabled
      (try: `--prerelease=allow`)
make: *** [Makefile:101: lock-python-dependencies-all] Error 1

@ntkathole
Copy link
Copy Markdown
Member

ntkathole commented Jun 17, 2025

grpcio-tools

You might need to pin grpcio-* to upgraded versions

@misohu
Copy link
Copy Markdown
Author

misohu commented Jun 17, 2025

Hey @ntkathole ok found some version ... now having problems with the make lock-python-dependencies-all getting

...
# The following packages were excluded from the output:
# milvus-lite
Using Python 3.10.17 environment at: infra/scripts/pixi/.pixi/envs/py310
Audited 2 packages in 1ms
Traceback (most recent call last):
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/bin/pybuild-deps", line 10, in <module>
    sys.exit(cli())
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/click/core.py", line 1442, in __call__
    return self.main(*args, **kwargs)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/click/core.py", line 1363, in main
    rv = self.invoke(ctx)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/click/core.py", line 1226, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/pybuild_deps/scripts/compile.py", line 114, in compile
    results = compiler.resolve(dependencies)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/pybuild_deps/compile_build_dependencies.py", line 60, in resolve
    build_dependencies = self._resolve_build_deps_for_ireq(
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/pybuild_deps/compile_build_dependencies.py", line 92, in _resolve_build_deps_for_ireq
    build_ireqs = set(self._find_build_dependencies(ireq))
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/pybuild_deps/compile_build_dependencies.py", line 169, in _find_build_dependencies
    for build_dep in find_build_dependencies(
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/site-packages/pybuild_deps/cache.py", line 27, in wrapper
    with shelve.open(str(CACHE_PATH / cache_name)) as cache:  # noqa: S301
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/dbm/__init__.py", line 91, in open
    raise error[0]("db type is {0}, but the module is not "
dbm.error: db type is dbm.gnu, but the module is not available
make: *** [Makefile:101: lock-python-dependencies-all] Error 1

I have this installed

sudo apt update
sudo apt install libgdbm-dev

The pixi env has some problems

infra/scripts/pixi/.pixi/envs/py310/bin/python3 -c "import dbm.gnu; print('gdbm ok')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/pocik/Documents/code/upstream/feast/infra/scripts/pixi/.pixi/envs/py310/lib/python3.10/dbm/gnu.py", line 3, in <module>
    from _gdbm import *
ModuleNotFoundError: No module named '_gdbm'

I am running the make with uv env activated

@HaoXuAI
Copy link
Copy Markdown
Collaborator

HaoXuAI commented Jun 17, 2025

Hey, thanks for contributing. Some of the companies are still using 4.25.x. So before bumping to 5.x we need to be really careful about it. Instead. I think a temporary fix should be lower the upper bound for protobuf version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

do-not-merge/hold

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bug Report: Incorrect Protobuf Version Compatibility

3 participants

@misohu @ntkathole @HaoXuAI