A side note: the past has shown that it's not really feasible to write all documentation during implementation, often because different people are writing docs than the code. I'm unsure, but would it be worthwhile to add a list of non-coding tasks to the RFC explicitly?

Since my scope here was focused mainly on the tracking issues rather than the RFC content I described non-coding tasks as something to be listed in the tracking RFC, but I think you're right that it could be helpful for the RFC to include the initial list of non-coding tasks (and possibly also coding tasks?) that would get copied directly into the tracking issue when it's first created.

Does that match what you were imagining, or did I misunderstand the idea? 🤔

That would be helpful I believe, yes.

If we can, we may want to also add username tags to the list to quickly see who is working on each of these issues. It's a bit of an administrative overhead, but it makes triages a lot easier.

I would also add here that the feature issues should, themselves, contain as much information about what that part specifically is, so even uninitiated can get a picture of what's going on. Specifically, if a subissue deals with implementing a specific syntax part, the issue should contain an example what it does.

I had elsewhere in the doc called for assigning the tracking issue to the folks working on it, but your comment here made me realize that I left a gap: assignees give a list of people but they don't represent what each person is responsible for.

At the very least it seems important for the tracking issue to include the username of the "project owner", since they have some specific responsibilities that other stakeholders rely on. I feel less sure about whether it's worth getting into details like who is writing code for each feature, since that's more of a project-internal concern than something that needs to be broadly known and in practice folks' relationships to projects tend to be more complicated than just "I'm spending 100% of my time working on this". 🤔

Would you be comfortable with the compromise of requiring the tracking issue to mention the name of the project owner, but using the standard GitHub "assignees" for everything else for now? If we find that we're lacking some context by doing that then I'd be happy to revise that plan, but I feel that right now as a small team our relationships to projects tend to be a bit "freeform" and so I'd prefer to focus only on the one new role of "project owner" than to try to standardize everything all in one step.

I wonder if labels are the right tool for this, especially since some issues may span multiple repos. I wonder if rather creating a public project would be more helpful. It can be one board per RFC, or one board for all. It would make communicating the status of features a lot easier too.

The other projects I was taking inspiration from (mainly Rust and Go) use labels, though I must admit I don't know why they use labels. I might guess that it's because it's the most prominent filtering mechanism for GitHub issues and so it's the most accessible to folks who aren't familiar with the fine details of a project's workflow.

With that said, I was intending to suggest that the labels be in addition to a more structured project management tool like GitHub's Projects. My other section below about "roadmapping" talks a little about that, but I kept it light and unspecific for now: since our current representation of that is not public I expect that's something we'll want to evolve, but exactly how to evolve it seems like a much bigger discussion than how to represent the status of individual RFCs and so I'd prefer to leave that for a later RFC.

I like this a lot. I think that having an owner / "project manager" for each RFC who is responsible for keeping issues updated and well documented is going to improve our public communication by a lot.

I would also add (somewhere) that we should link issues together to make navigation easier, such as in the issues here (parent-child relationships clearly linked).

Totally agreed about linking other issues. I must admit I hadn't really considered the idea of using issues as "subtasks" -- my earlier writing talked about explicitly listing out all of the pull requests because I imagined that as being the primary way to represent specific subtasks.

But I can imagine situations where it could be helpful to create an issue representing a subtask that is itself complex enough to warrant multiple pull requests, in which case linking those would be important too.

Are you okay with letting this RFC remain a little vague on this for now, and then learning from what happens in practice on real projects we run in the near future? I was intentionally hesitating on being too prescriptive about exactly how the individual work items ought to be represented and trying to focus on the general idea that the tracking issue must link to everything else regardless of what the "everything else" might be for a particular project, since I expect projects of different sizes and shapes would have different needs.

I wonder if we should expand the owner / "project manager" responsibility also to be the technical lead and product lead of the RFC.
As RFCs change over time, and while working on them, people might hit blocks that require changing the original RFC. These blocks might be actual technical blocks that we missed in the RFC or just blocks that will be too costly, and we can decide to change the technical implementation or the user-facing behavior to avoid facing them.
So, as the technical / product lead, the owner will be responsible for keeping the RFC up to date, and he'll challenge any possible alterations (technical or user-facing) during the implementation.
He doesn't have to update the RFC on his own, but he should discuss the intended changes, provide input, and oversee that the agreed-upon changes are merged back into the RFC.
It's essential to have this persona, as he took the time to investigate and understand all the trade-offs of his suggested RFC, while the people working on the RFC might not be that versed in the details and all the different considerations.

I saw you addressed most of the operative part of what I mentioned above in the "RFC Amendments". But it'll still be interesting to hear your opinion about the RFC owner being the technical lead / product lead of the RFC implementation.

I had intended this RFC to focus primarily on the communication about the project rather than on other project management details, but I do agree with you that the person who is responsible for keeping the tracking issue updated is probably also the main person responsible for ensuring the RFC stays updated and that the final implementation actually does match the RFC.

I'd hesitate to grow the scope of this RFC to talk about project management in general; my intention had only been to prescribe a way to communicate about a project in a way that's compatible with asynchronous and decentralized work and external observers. But I do think that we could separately agree within the core team that by default we have a single lead that's responsible for all of these roles, unless a particular project decides to split that work up differently for some special reason.

It makes sense to me; I agree that we shouldn't include it in the scope of this RFC.

I think we should add a little text here that asks us (maybe at the discretion of the project owner?) to create a feature branch and a main PR where everyone can build the latest feature version of OpenTofu from. This would help with early testing.

In the situation I was intending to describe here, the "partial solution" would already be merged to main, and so could be built from there.

The kind of "partial solution" I'm thinking about here is, for example, if we decided to respond to #1329 by writing a separate RFC for each lifecycle argument, since all of them have different constraints and concerns and so it'd presumably be less risky to approach it gradually. If we took that approach then there would be several RFCs and several tracking issues all linking back to the same feature request, but the feature request as a whole would be closed only once all of those RFC tracking issues are closed.

I think you are considering a slightly different situation where a single RFC gets implemented over multiple PRs and where in some situations it might not be appropriate to merge the partial work to main until the rest is completed. If so, I agree that it's important to make it clear exactly which branch the intermediate work is being merged into, and I agree with your earlier comment about ensuring that branch and/or PR are clearly linked from the RFC tracking issue, along with everything else.

I wonder if each RFC should translate to either a new doc in the docs folder or an updated doc in the same. This would be a lot more work, but give a point-in-time state of each of the topics discussed in these RFCs.

Personally, I'm not a fan of having these docs in the main repo because we have multiple version branches and it would be easier to manage them in a separate repo, but for visibility it is certainly helpful to have them here.

I think the question of how to organize docs is probably better saved for another RFC; my goal here was only to give some examples of locations where we already keep documentation to try to clarify what kinds of documentation this section was talking about.

As the person who originally put this docs directory here I can say that my intent with it was specifically that it is tethered to particular version branches and particular commits, so that if you're working on something for an old release you can more easily find the docs about the codebase as it existed in that release. Of course, that's only true if we actually remember to update the docs when we change something, and so I suppose this section doubles as a reminder to actually do that. 😀

However, I understand that there are different tradeoffs here and would be happy to discuss other ways to organize the developer-oriented documentation in a separate RFC focused on that topic!

The CONTRIBUTING.md is more policy-oriented (don't use Copilot, etc) and doesn't really contain that much technical detail on contributing. I wonder how much this is documentation in the traditional sense.

Indeed, these particular files would presumably not be updated by a typical feature-driven RFC.

They seem like the most appropriate place (of the places we currently have) to document the outcome of a development process RFC like this one, though. I think it's important to have the current active process documented in a form that doesn't require the reader to understand the full history of how the policy was decided, which I think would effectively be the case if the development process RFCs were the only place this information was recorded.

(For example, I had intended that if this RFC is accepted then part of its "implementation" would be to update rfcs/README.md to describe what tracking issues are and how we use them, so that a newcomer to the project can learn that without having to first find this specific RFC document.)

I wonder if we should mention a practice we lately started in some places, where we add an explicit README.md to packages containing the developer-facing docs for that specific package, for example here. I know it isn't super popular in the codebase (yet), but it's an in-context way to help people out.

Yeah, the decision between package-level Go doc commands vs. a separate README.md in the directory has always been a little unclear to me. I don't have a strong opinion either way and have done both at different times.

I guess I've typically preferred to use Go doc comments and used a separate readme only when the docs got particularly long or benefitted from Markdown formatting, but it is true that the Go doc comments are not so easy to find for someone who doesn't already know how to construct a pkg.go.dev URL for one of our internal packages, whereas the README.md is rendered prominently in the GitHub UI.

Again, I think this would be a good topic for a separate RFC focused on documentation. As with the others, I included this only as an example of what we might currently do. I will update this to mention that a README.md in the package directory is a valid alternative though, since that is describing something we already do rather than a new proposal.

I would go a step further. Even once the RFC is merged, there's work to be done on the public relations side:

• Feature preview video
• Beta blog post
• Release blog post

How should we track these?

I was alluding to this sort of work in various places where I broadly referred to "non-coding tasks" rather than specifically "writing documentation".

I think I'd like to avoid getting into the weeds of what exactly the "non-coding tasks" are, who is responsible for doing them, and how we track the need to do them in this specific RFC just because it's already quite long, but I do fully agree that this is an important question to answer: I'd just prefer to get there iteratively as we learn from running this process in real life, rather than trying to get everything nailed down all in one step.

In particular this set of things seems to overlap with what I wrote under "Future Considerations" about whether our release planning model is like "it's not done until all of these features are complete" or like "choose an approx. freeze date, and include anything that was completed in time".

An key part of that tradeoff, I think, is whether we focus on "release blog posts" or "per-feature blog posts" (and other media, like video). If we decide that each feature gets its own separate blog post, etc then it could be reasonable for the RFC tracking issue to include the tasks of creating those, but if we treat it more like a "release rollup" then that work would presumably sometimes happen quite some time after a particular feature is complete, because not all features will necessarily complete development at the same point in a development period.

Again, I do completely agree that this is important stuff to decide, but I'd prefer not to get too into the specifics in this RFC because our process for PR/marketing seems considerably less settled and thus far more likely to change quickly than our process for writing new code and documenting it. If you feel strongly about including it then I can be flexible, though.