FOIA.gov Draft RESTful HTTPS API Spec

This is a draft spec for integrating the FOIA.gov portal with existing FOIA case management systems operated by individual agencies in the federal government. This work stems from the interviews and research that led to our FOIA Portal Discovery Recommendations.

Once an agency’s case management system supports this specification, it can receive FOIA requests directly from the FOIA.gov portal, rather than having the request data sent to the agency via e-mail.

To minimize agency effort, we’ve designed this spec so that some of the tedious bits of implementing an API can be handled by a service like api.data.gov, which provides a free API management service for federal agencies.

Change Log

Version 1.1.0

• Add the new field “pdf”, to contain an object with file data for a PDF version of the request
• Add the new field “testing”, to indicate when a request came from a non-production environment

Receive a FOIA Request

Notes

This draft does not address:

• versioning
• size or rate limits
• error message/status code related to exceeded the rate limit
• any subsequent calls to the internal FOIA.gov API (to capture info needed to subsequently retrieve status, for example)
• detailed security controls

Security controls

Once we have confidence in our data models and the touch points for interoperability, we will be working with the security team at DOJ to ensure the Portal meets the federal requirements for security controls. For now, we include only some preliminary ideas based on the agency feedback we’ve received.

Each agency will be responsible for implementing security controls on their own end as per their agency regulations and authority to operate. This may include configuration of a web application firewall, anti-virus scanning of file attachments, and validation of HTTP headers.

HTTPS

Agency endpoints should be restricted to HTTPS only with valid TLS certificates. The Portal will validate your certificate as part of the HTTPS request.

Authentication

To ensure that your API and case management system aren’t publicly writable, we recommend restricting your API access to the FOIA.gov Portal. This can be done via a shared secret HTTP header token. You will provide this secret token to the Portal though configuration. Every request from the Portal will include this token in the HTTP header FOIA-API-SECRET, and your API should validate that it is the correct token.

Services like api.data.gov provide this authentication for you and give you additional options.

URL

There are no required parameters or format for your API URL. You may choose any pathname you wish. If your system handles requests for multiple agency components (common for decentralized agencies), we recommend using a URL structure that explicitly identifies the agency component receiving the FOIA request. Your URL should not contain any query parameters.

Recommended URL format for decentralized agencies:

/components/:id/requests/

Where id is a unique identifier for a component within your agency.

For example:

/components/88/requests/

But not:

/requests?component=88

In addition, we recommend hosting the API on a dedicated sub-domain like foia-api.agency.gov. Using this kind of pathname hierarchy allows us to add additional API endpoints for future development and features.

Method:

POST

URL parameters

Required:

id=[integer], where id is the unique identifier of the agency component that should receive the request.

Data parameters

JSON payload that contains the form fields.

Content-Type: application/json

Request fields

Sample payload

{
    "version": "1.1.0",
    "request_id": 1534,
    "address_city": "Mount Vernon",
    "address_country": "United States",
    "address_line1": "1800 F Street",
    "address_line2": "Suite 400",
    "address_state_province": "Virginia",
    "address_zip_postal_code": "98273",
    "agency_component_name": "Office of Information Policy",
    "agency": "Department of Justice",
    "attachments_supporting_documentation": [{"filename": "letter.pdf", "content_type": "application/pdf", "filesize": 27556, "filedata": "YSBiYXNlNjQgZW5jb2RlZCBmaWxlCg=="}],
    "company_organization": "Newspaper Inc",
    "email": "[email protected]",
    "expedited_processing_explanation": "The request should be given expedited processing because…",
    "expedited_processing": "no",
    "fax_number": "+15551234589",
    "fee_amount_willing": "25",
    "fee_waiver_explanation": "As a journalist organization, I am requesting these records on behalf of the public and intend to make these records accesible to the public.",
    "fee_waiver": "no",
    "name_first": "George",
    "name_last": "Washington",
    "phone_number": "+15551234567",
    "request_category": "individual",
    "request_description": "I am seeking records pertaining to ..."
}

Agency component specific form fields

Your agency component might have additional fields that are unique to your agency and are also captured in this request payload. If you have ever received any submissions via email, then you can consult these emails to see the machine-names of each field. If you have not yet received any of these emails, simply perform a test submission on the portal and consult the email that arrives.

Responses

Responses should be in application/json format and include an appropriate HTTP status code.

Success Response

Error Response

Sample request

$ curl -X POST -H "Content-Type: application/json" -d @- https://foia-api.agency.gov/components/234/requests <<EOF
{
    "version": "1.1.0",
    "request_id": 1534,
    "address_city": "Mount Vernon",
    "address_country": "United States",
    "address_line1": "1800 F Street",
    "address_line2": "Suite 400",
    "address_state_province": "Virginia",
    "address_zip_postal_code": "98273",
    "agency_component_name": "Office of Information Policy",
    "agency": "Department of Justice",
    "attachments_supporting_documentation": [{"filename": "letter.pdf", "content_type": "application/pdf", "filesize": 27556, "filedata": "YSBiYXNlNjQgZW5jb2RlZCBmaWxlCg=="}],
    "company_organization": "Newspaper Inc",
    "email": "[email protected]",
    "expedited_processing_explanation": "The request should be given expedited processing because…",
    "expedited_processing": "no",
    "fax_number": "+15551234589",
    "fee_amount_willing": "25",
    "fee_waiver_explanation": "As a journalist organization, I am requesting these records on behalf of the public and intend to make these records accesible to the public.",
    "fee_waiver": "no",
    "name_first": "George",
    "name_last": "Washington",
    "phone_number": "+15551234567",
    "request_category": "individual",
    "request_description": "I am seeking records pertaining to ..."
}

EOF

Troubleshooting request failures

Various factors may cause a request submitted on FOIA.gov to an agency via API to fail. In these instances, the National FOIA Portal Administrator will inform the agency/component of the failure, and will pass along the error message. Agencies/components will coordinate with their tracking system vendor to analyze this error message and address the problem to prevent similar failures in the future. The following list contains common error messages that may be helpful in this analysis.

Additional info on CURL errors can be found here: libcurl - Error Codes.