AIP-191

File and directory structure

A consistent file and directory structure, while making minimal difference technically, makes API surface definitions easier for users and reviewers to read.

Guidance

Note: The following guidance applies to APIs defined in protocol buffers, such as those used throughout Google. While the spirit of this guidance applies to APIs defined using other specification languages or formats, some of the particular recommendations might be irrelevant.

Syntax

APIs defined in protocol buffers must use proto3 syntax.

Single package

APIs defined in protocol buffers must define each individual API in a single package, which must end in a version component. For example:

syntax = "proto3";

package google.cloud.translation.v3;

Google APIs must reside in a directory that matches the protocol buffer package directive. For example, the package above dictates that the directory be google/cloud/translation/v3.

File names

It is often useful to divide API definitions into multiple files. File names must use snake_case.

APIs should have an obvious "entry" file, generally named after the API itself. An API with a small number of discrete services (Google Cloud Pub/Sub's Publisher and Subscriber is a good example) may have a separate entry file per service.

APIs with only one file should use a filename corresponding to the name of the API.

API service definitions and associated RPC request and response message definitions should be defined in the same file.

Bear in mind that the file names often become module names in client libraries, and customers use them in import or use statements. Therefore, choosing a descriptive and language keyword-free filename does matter. For example, a file called import.proto may be problematic in Python.

Note: The version must not be used as a filename, because this creates bizarre imports in client libraries. Filenames such as v3.proto or v1beta1.proto are prohibited.

File layout

Individual files should place higher level and more important definitions before lower level and less important definitions.

In a proto file, components should be in the following order, and each of these should be separated by a blank line:

  • Copyright and license notice (if applicable).
  • The proto syntax statement.
  • The proto package statement.
  • Any import statements, in alphabetical order.
  • Any file-level option statements.
  • Any service definitions.
    • Methods should be grouped by the resource they impact, and standard methods should precede custom methods.
  • Resource message definitions. A parent resource must be defined before its child resources.
  • The RPC request and response message definitions, in the same order of the corresponding methods. Each request message must precede its corresponding response message (if any).
  • Any remaining message definitions.
  • Any top-level enum definitions.

Packaging annotations

Protocol buffers ships with annotations to declare the package or namespace (depending on the vocabulary of the target language) of the generated files. For example, setting go_package or csharp_namespace will override the inferred package name.

When defining APIs, the following rules apply:

  • Java
    • The java_package annotation must be set. The correct value is usually the proto package with the appropriate TLD prefixed. Example: com.google.example.v1.
    • The java_multiple_files annotation must be set to true.
    • The java_outer_classname annotation must be set, and should be set to the name of the proto filename, in PascalCase, with Proto appended. Example: LibraryProto.
  • Other languages
    • Package or namespace directives for other languages must be set either in every file in the proto package, or none of them. If they are set, the values must be identical in every file.
    • If any part of the protobuf package is a compound name (such as accessapproval), C#, Ruby and PHP options must be specified in order to take account of the word breaks using PascalCase (UpperCamelCase). Example: 
      option csharp_namespace = "Google.Cloud.AccessApproval.V1";
option php_namespace = "Google\\Cloud\\AccessApproval\\V1";
option ruby_package = "Google::Cloud::AccessApproval::V1";
    • The go_package value depends directly on how the Go code is managed i.e. if the module name is based on the VCS provider or using a remote import path, but often has a consistent structure.
      • The module may differ based on product area e.g. google.cloud.accessapproval.v1 would be in module cloud.google.com/go/accessapproval.
      • The package import path should be derived from the proto package.
      • An API version in the proto package should be prefixed with api e.g. the proto package segment v1 becomes apiv1.
      • The terminal import path segment should be based on the product name found within the proto package and must be suffixed with pb e.g. accessapproval becomes accessapprovalpb.
      • This value should be left to the team owning the generated code to decide on.

All packaging annotations should be specified in alphabetical order of name. Refer to the Protobuf documentation for more about language package options.

Important: While languages other than Java have sensible defaults for APIs which don't include compound names, be aware that adding this annotation (with a value not equivalent to the default) constitutes a breaking change in that language. When releasing protos, be sure that omissions are intentional.

Rationale

Java packaging options

Set the option, java_multiple_files, to true to get a cleaner file structure. Doing so instructs protoc to create one output file per Protobuf type, which allows for more fine-grained imports. The option, java_outer_classname, is required in combination with java_multiple_files. It instructs protoc to wrap each compiled Protobuf type in a Java class whose name is the value of the option. This prevents potential naming collisions between generated types.

Go packaging option

The Go packaging option needs to be decided by the team that owns the generated code, because it is directly tied to the source code management practices of the team. Allowing every proto package to decide on their own Go package creates inconsistencies and friction in management of the code. Within that owning team, having a consistent structure in the Go package naming is critical to a consistent end user experience.

Changelog

  • 2024-06-13: Added guidance for Go packaging annotation.
  • 2024-06-05: Added rationale for Java packaging options.
  • 2023-02-24: Added guidance on protobuf syntax.
  • 2022-10-18: Added guidance on Ruby/PHP/C# options.
  • 2019-11-18: Added guidance on the packaging annotations.