Building a Code Climate Plugin

Whether you've started working on a plugin or are just curious what goes into making a new analysis tool work on the Code Climate Platform, this document is for you.

What is a Code Climate Plugin?

In order to understand how to build a Code Climate plugin, we thought it would be helpful to start with a working definition:

A Code Climate Plugin is a Docker Image that invokes a program which parses a config file and analyzes files, potentially producing formatted output representing issues in those files.

Within that definition are some bolded words which could use some elaboration.

Before we dig in, this is a good time to state that that the design, packaging, and operation of an plugin is guided by the Code Climate Spec, and Quality Assurance steps are outlined in the Plugin QA spreadsheet.

Docker Image - all Code Climate plugins are packaged as Docker Images. If you're not familiar with Docker and how it works, we recommend checking out their website - there's lots of great information there. We use Docker because it allows us to let people use a variety of implementations for their engines and still package them in a repeatable, specification compliant way. Notably, when a Docker Image is created for a Code Climate Plugin, we run the same image wherever we offer analysis - locally on your computer in the CLI, in our cloud on codeclimate.com, or behind the firewalls of enterprises with Code Climate Enterprise. Pretty neat.
invokes a program - Inside your Docker Image, your plugin will invoke a program (often just a simple script) which will do all of the work necessary to be specification compliant.
parses a config file - The first thing your program will do is parse a config file which is detailed in the Code Climate Spec and will contain information that is necessary to run the plugin properly. The configuration a user provides with their .codeclimate.yml file ends up in this config file, and includes data like which files to analyze, how to configure the plugin, etc.
analyze files - Once the config file is parsed and the plugin has a record of which files to analyze, it can iterate over them and do it's job. Your plugin can either wrap an existing tool or be its own novel tool, it doesn't really matter - as long as you comply with the Spec, your plugin should function properly.
formatted output - The output produced when an issue is found in a file to be analyzed is strictly controlled both in how the issues are formatted and how they are sent to STDOUT for consumption by various Code Climate services. Check the Spec for more information.
issues - Issues found with analyzed files are to be assigned categories, and given short and long descriptions which are described in the Code Climate spec. Issues are what users see when they run Code Climate, so they are subject to a high quality bar for content and accuracy.

How to Get Started

Once you've read the "How to build an plugin" blog post, looked at the Spec, seen the QA spreadsheet, and maybe even browsed around a few example plugins, you still may not know what a good approach is to getting started.

Here are some tips that we've given to a number of plugin authors. We think they're good inspiration for getting started with what can seem like a daunting task. If they help, let us know! If they don't, hey, let us know too! We'd love to make this list as complete as possible.

Concerning Docker

Docker is an essential aspect of our platform - the whole thing basically runs on it. To that end, you need to have a fully functioning Docker environment running locally, and you need to understand the basics of making a Docker Image to correctly build a Code Climate Plugin. That being said, it can wait.

You can get started with the basics of your plugin, like making a simple script, studying plugins, etc., before you worry about making your own Dockerfile, and besides, in all likelihood you'll be able to copy an existing plugin's implementation for most of what you need.

So if the Docker work is scaring you off, don't let it. Support for Docker is becoming better and better on all widely deployed Desktop environments, educational material is abundant, and much of the work ahead you has been done for you.

Start with a simple script

Once you're ready to build your plugin, we recommend as a first step that you build a simple script that can accomplish the following:

Accept two parameters - config_file_path and workspace_path
Use the config_file_path parameter to find and parse a file named config.json, and store its keys and values in a data structure
Create a list of files to analyze using the include_paths key from the parsed config.json file and the workspace_path parameter
Invoke the tool your plugin is wrapping with the list of files to analyze
Output issues that have been found in JSON format to STDOUT

Parsing a config, executing the tool, and formatting the output comprise the majority of the functionality of a Code Climate engine. Once you have a script which can accomplish these things, you're most of the way there!

Study some other plugins, and copy what you can

All Code Climate plugins are Open Source. Feel free to fork any of the plugins available in our list of plugins and use it as the basis for your own work.

If you're building plugins in pretty much any major programming language, we have examples for you to be inspired by. There are even libraries in certain languages for making writing plugins even easier.

When you're ready to test with the CLI

The Code Climate CLI ships with a few handy tricks for making plugin development easier:

The --dev flag - when you want to test your plugin with the CLI, add it to the .codeclimate.yml file of a project, and run codeclimate analyze --dev. The --dev flag will allow the CLI to check locally for plugin Docker Images, letting you specify plugins that aren't generally available.
The -f json flag - if your issues are breaking the standard output or you want to see parts that aren't represented there, you can make the CLI output JSON, which shows all available fields
CODECLIMATE_DEBUG=1 codeclimate analyze The CLI looks for the presence of an environment variable called CODECLIMATE_DEBUG and outputs extra useful debugging information if it finds it. This can save you lots of time!

If you've come across any other good tips, let us know!

When you want to see how your issues will render on codeclimate.com

The Code Climate CLI has an HTML output flag that will show you rendered markdown for issue descriptions and content. To use the HTML formatter, run codeclimate analyze -f html > out.html and inspecting out.html in a browser.

Use the community

Join our Code Climate Developer Slack Channel! You should get an invitation automatically when you join the Developer Program.