Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a way to easily load multiple schema files #364

Open
mvdan opened this issue Feb 18, 2022 · 3 comments
Open

add a way to easily load multiple schema files #364

mvdan opened this issue Feb 18, 2022 · 3 comments
Assignees
Labels
P2 Medium: Good to have, but can wait until someone steps up

Comments

@mvdan
Copy link
Contributor

mvdan commented Feb 18, 2022

We currently have these three top-level APIs:

func LoadSchema(name string, r io.Reader) (*schema.TypeSystem, error)
func LoadSchemaBytes(src []byte) (*schema.TypeSystem, error)
func LoadSchemaFile(path string) (*schema.TypeSystem, error)

It would be nice to support loading multiple schemas into a single TypeSystem, such as loading the selector schema alongside a custom schema that uses the selector type.

One can do this directly via the schema/dsl and schema/dmt packages, but it would require finding those APIs and figuring out how to string them together, so it's not very nice.

Another option is to concatenate the schema files and load them via LoadSchemaBytes. Certainly doable, but it's still hacky, and means that position information will be mangled.

Going with the flow of go:embed and io/fs, I think the following would be nice:

func LoadSchemaFS(fsys fs.FS) (*schema.TypeSystem, error)

Then, for instance:

//go:embed schemas/*.ipldsch
var schemaFS fs.FS

[...] = LoadSchemaFS(schemaFS)

It would walk the entire filesystem, so it would work with directories too, e.g. //go:embed schemas. One could still only parse subsets of directory trees via APIs like https://pkg.go.dev/io/fs#Sub or https://github.com/josharian/filterfs.

cc @rvagg @hannahhoward per recent graphsync discussions

@mvdan
Copy link
Contributor Author

mvdan commented Feb 18, 2022

A slight alternative to this proposed API is to instead tweak LoadSchemaFile to also accept directories, to then walk them and load all schema files within. But that's significantly less powerful: it can't be used with embedded directories nor any other form of in-memory filesystem, nor can one choose what files to walk via e.g. filterfs.

@rvagg
Copy link
Member

rvagg commented Feb 18, 2022

Another alternative is to make it func LoadSchemaFile(path ...string) (*schema.TypeSystem, error)

@mvdan
Copy link
Contributor Author

mvdan commented Feb 18, 2022

Certainly an option as well :) I think my only minor reservation, beyond that it still doesn't support go:embed, is that the API name would remain singular even though it can take multiple arguments.

@BigLep BigLep added the P2 Medium: Good to have, but can wait until someone steps up label Apr 8, 2022
@BigLep BigLep moved this to 🗄️ Backlog in IPLD team's weekly tracker Jul 19, 2022
@rvagg rvagg moved this from 🗄️ Backlog to 🥞 Todo in IPLD team's weekly tracker Aug 2, 2022
@rvagg rvagg self-assigned this Aug 2, 2022
@rvagg rvagg moved this from 🥞 Todo to 🗄️ Backlog in IPLD team's weekly tracker Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 Medium: Good to have, but can wait until someone steps up
Projects
Status: 🗄️ Backlog
Development

No branches or pull requests

3 participants