Last week we announced a major milestone in the OpenTofu journey, with the OpenTofu v1.6.0-beta release.
The release introduces several bug fixes, security improvements, and updates to documentation, the details of which you can find in our change log. Most importantly, it marks the introduction of our new OpenTofu public registry.
In this post, I will dive into the solution we zeroed in on, an open-source registry inspired by Homebrew.
The Challenge
We needed a centralized repository that would act as an index file or a catalog for all providers and modules used by OpenTofu. In essence, an open-source variant of the Terraform registry, which is no longer open to use for non-Terraform projects, following their license change.
Outside of the core functionality, we had many other requirements in mind, primarily:
- The registry had to be as self-sufficient as possible, so as not to bog us down with maintenance.
- Anticipating growing demand, the registry had to be stable and perform well at scale.
- It needed to take the MVP (Minimum Viable Product) approach, delivering all of the core functionality without unnecessary bells and whistles.
- The solution should allow for a seamless transition from HashiCorp's registry, to support our ‘drop-in replacement’ vision.
- Whatever we went with, it had to be open source.
Enter Homebrew.
Homebrewing Together
For those unfamiliar, Homebrew is a popular open-source package manager for MacOS (what APK or RPM is to Linux or Chocolatey is to Windows). You can go as far as calling it a de facto choice for package manager for MacOS.
Homebrew's simple nature and capabilities made it the perfect candidate for the MVP, something we could rely on to quickly launch a registry with all of the required capabilities.
Its popularity also made us confident in its ability to operate well at scale. More so, since provider and module data could be served as static files from an R2 bucket on Cloudflare, which would ensure high performance and availability via global CDN and caching.
A chime-in from Homebrew’s project lead Mike McQuaid drove this point home, confirming our approach while providing us with tips for performance improvements. That included advice on sharding data so that packages themselves did not have too many files or folders weighing them down.
For me, this was a great example of the collaborative nature of this project and yet another reminder of the ‘better together’ benefit of open source.
How It Works
Once we had the solution outlined, we hydrated the registry via GitHub for all providers and modules, then aggregated each individual provider or module’s details and meta-data into its own JSON file.
All of those JSON files are hosted at registry.opentofu.org, with each file acting as a source of truth for the provider and module and their upstream APIs. Finally, we added a GitHub action to periodically look for updated versions of all indexed providers and modules.
For the next steps, we will look to add a frontend with UI and documentation. This is an important bit, but nothing we felt should delay the release.
Also, there is already an RFC in place, accepted for implementation, to use IssueOps flow for adding and validating new providers.
Combined with automated validation, this means that submissions can be auto-processed and added to the registry, with Issues providing context, as well as opening a door for public audit.
Ready for Prime Time
Finalizing the registry implementation and verifying its stability, as well as its ability to withstand high loads, was the last piece of the OpenTofu GA puzzle. Now, with that behind us, we are full steam ahead towards a release.
In the meantime, you can already take OpenTofu for a spin to test the new registry for yourself. We are still ironing out some last bits, but the solution is already extremely stable, meeting even our the-World-is-watching extremely high standards.
Our platform has full support for OpenTofu since the alpha release two months ago. Getting started takes 3 clicks, 60 seconds, and zero code changes.
To learn more, check out the video below.
Last week we announced a major milestone in the OpenTofu journey, with the OpenTofu v1.6.0-beta release.
The release introduces several bug fixes, security improvements, and updates to documentation, the details of which you can find in our change log. Most importantly, it marks the introduction of our new OpenTofu public registry.
In this post, I will dive into the solution we zeroed in on, an open-source registry inspired by Homebrew.
The Challenge
We needed a centralized repository that would act as an index file or a catalog for all providers and modules used by OpenTofu. In essence, an open-source variant of the Terraform registry, which is no longer open to use for non-Terraform projects, following their license change.
Outside of the core functionality, we had many other requirements in mind, primarily:
- The registry had to be as self-sufficient as possible, so as not to bog us down with maintenance.
- Anticipating growing demand, the registry had to be stable and perform well at scale.
- It needed to take the MVP (Minimum Viable Product) approach, delivering all of the core functionality without unnecessary bells and whistles.
- The solution should allow for a seamless transition from HashiCorp's registry, to support our ‘drop-in replacement’ vision.
- Whatever we went with, it had to be open source.
Enter Homebrew.
Homebrewing Together
For those unfamiliar, Homebrew is a popular open-source package manager for MacOS (what APK or RPM is to Linux or Chocolatey is to Windows). You can go as far as calling it a de facto choice for package manager for MacOS.
Homebrew's simple nature and capabilities made it the perfect candidate for the MVP, something we could rely on to quickly launch a registry with all of the required capabilities.
Its popularity also made us confident in its ability to operate well at scale. More so, since provider and module data could be served as static files from an R2 bucket on Cloudflare, which would ensure high performance and availability via global CDN and caching.
A chime-in from Homebrew’s project lead Mike McQuaid drove this point home, confirming our approach while providing us with tips for performance improvements. That included advice on sharding data so that packages themselves did not have too many files or folders weighing them down.
For me, this was a great example of the collaborative nature of this project and yet another reminder of the ‘better together’ benefit of open source.
How It Works
Once we had the solution outlined, we hydrated the registry via GitHub for all providers and modules, then aggregated each individual provider or module’s details and meta-data into its own JSON file.
All of those JSON files are hosted at registry.opentofu.org, with each file acting as a source of truth for the provider and module and their upstream APIs. Finally, we added a GitHub action to periodically look for updated versions of all indexed providers and modules.
For the next steps, we will look to add a frontend with UI and documentation. This is an important bit, but nothing we felt should delay the release.
Also, there is already an RFC in place, accepted for implementation, to use IssueOps flow for adding and validating new providers.
Combined with automated validation, this means that submissions can be auto-processed and added to the registry, with Issues providing context, as well as opening a door for public audit.
Ready for Prime Time
Finalizing the registry implementation and verifying its stability, as well as its ability to withstand high loads, was the last piece of the OpenTofu GA puzzle. Now, with that behind us, we are full steam ahead towards a release.
In the meantime, you can already take OpenTofu for a spin to test the new registry for yourself. We are still ironing out some last bits, but the solution is already extremely stable, meeting even our the-World-is-watching extremely high standards.
Our platform has full support for OpenTofu since the alpha release two months ago. Getting started takes 3 clicks, 60 seconds, and zero code changes.
To learn more, check out the video below.
.png)
.png)
