What is Neptune?

Neptune is an experiment tracker. It enables researchers to monitor their model training, visualize and compare model metadata, and collaborate on AI/ML projects within a team.

What's different about Neptune Scale?

Neptune Scale is the next major version of Neptune. It's built on an entirely new architecture for ingesting and rendering data, with a focus on responsiveness and accuracy at scale.

Neptune Scale supports forked experiments, with built-in mechanics for retaining run ancestry. This way, you can focus on analyzing the latest runs, but also visualize the full history of your experiments.

pip install neptune-scale

1. Log in to your Neptune Scale workspace.

2. Get your API token from your user menu in the bottom left corner.

   If you're a workspace admin, you can also set up a service account. This way, multiple people or machines can share the same API token. To get started, access the workspace settings via the user menu.

3. In the environment where neptune-scale is installed, save your API token to the NEPTUNE_API_TOKEN environment variable:

   export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6...Y2MifQ=="
   

4. Create a project, or find an existing project you want to send the run metadata to.

   To create a project via API:

   from neptune_scale.projects import create_project
   
   create_project(
       name="project-x",
       workspace="team-alpha",
   )

5. (optional) In the environment where neptune-scale is installed, save your full project path to the NEPTUNE_PROJECT environment variable:

   export NEPTUNE_PROJECT="team-alpha/project-x"

   If you skip this step, you need to pass the project name as an argument each time you start a run.

You're ready to start using Neptune Scale.

For more help with setup, see Get started in the Neptune documentation.

Create an experiment:

from neptune_scale import Run

run = Run(
    experiment_name="ExperimentName",
    run_id="SomeUniqueRunIdentifier",
)

Then, call logging methods on the run and pass the metadata as a dictionary.

Log configuration or other simple values with log_configs():

run.log_configs(
    {
        "learning_rate": 0.001,
        "batch_size": 64,
    }
)

Inside a training loop or other iteration, use log_metrics() to append metric values:

# inside a loop
for step in range(100):
    run.log_metrics(
        data={"acc": 0.89, "loss": 0.17},
        step=step,
    )

To help identify and group runs, you can apply tags:

run.add_tags(tags=["tag1", "tag2"])

The run is stopped when exiting the context or the script finishes execution, but you can use close() to stop it once logging is no longer needed:

run.close()

To explore your experiment, open the project in Neptune and navigate to Runs. For an example, see the demo project →

For more instructions, see the Neptune documentation:

• Quickstart
• Create an experiment
• Log metadata

Representation of experiment tracking metadata logged with Neptune Scale.

Initialize with the class constructor:

from neptune_scale import Run

run = Run(...)

or using a context manager:

from neptune_scale import Run

with Run(...) as run:
    ...

Parameters

Examples

Create a new run:

from neptune_scale import Run

with Run(
    project="team-alpha/project-x",
    api_token="h0dHBzOi8aHR0cHM6...Y2MifQ==",
    run_id="likable-barracuda",
) as run:
    ...

For help, see Create an experiment in the Neptune docs.

To restart an experiment, create a forked run:

with Run(
    run_id="adventurous-barracuda",
    experiment_name="swim-further",
    fork_run_id="likable-barracuda",
    fork_step=102,
) as run:
    ...

Continue a run:

with Run(
    run_id="likable-barracuda",  # a Neptune run with this ID already exists
    resume=True,
) as run:
    ...

The regular way to end a run. Waits for all locally queued data to be processed by Neptune (see wait_for_processing()) and closes the run.

This is a blocking operation. Call the function at the end of your script, after your model training is completed.

Examples

from neptune_scale import Run

run = Run(...)

# logging and training code

run.close()

If using a context manager, Neptune automatically closes the run upon exiting the context:

with Run(...) as run:
    ...

# run is closed at the end of the context

Logs the specified metadata to a Neptune run.

You can log configurations or other single values. Pass the metadata as a dictionary {key: value} with

• key: path to where the metadata should be stored in the run.
• value: the piece of metadata to log.

For example, {"parameters/learning_rate": 0.001}. In the attribute path, each forward slash / nests the attribute under a namespace. Use namespaces to structure the metadata into meaningful categories.

Any datetime values that don't have the tzinfo attribute set are assumed to be in the local timezone.

Parameters

Examples

Create a run and log metadata:

from neptune_scale import Run

with Run(...) as run:
    run.log_configs(
        data={
            "parameters/learning_rate": 0.001,
            "parameters/batch_size": 64,
        },
    )

Logs the specified metrics to a Neptune run.

You can log metrics representing a series of numeric values. Pass the metadata as a dictionary {key: value} with

• key: path to where the metadata should be stored in the run.
• value: the piece of metadata to log.

For example, {"metrics/accuracy": 0.89}. In the attribute path, each forward slash / nests the attribute under a namespace. Use namespaces to structure the metadata into meaningful categories.

Parameters

Examples

Create a run and log metrics:

from neptune_scale import Run

with Run(...) as run:
    run.log_metrics(
        data={"loss": 0.14, "acc": 0.78},
        step=1.2,
    )

Note: To correlate logged values, make sure to send all metadata related to a step in a single log_metrics() call, or specify the step explicitly.

When the run is forked off an existing one, the step can't be smaller than the step value of the fork point.

Adds the list of tags to the run.

Parameters

Example

with Run(...) as run:
    run.add_tags(tags=["tag1", "tag2", "tag3"])

Removes the specified tags from the run.

Parameters

Example

with Run(...) as run:
    run.remove_tags(tags=["tag2", "tag3"])

Waits until all metadata is submitted to Neptune for processing.

When submitted, the data is not yet saved in Neptune (see wait_for_processing()).

Parameters

Example

from neptune_scale import Run

with Run(...) as run:
    run.log_configs(...)
    ...
    run.wait_for_submission()
    run.log_metrics(...)  # called once queued Neptune operations have been submitted

Waits until all metadata is processed by Neptune.

Once the call is complete, the data is saved in Neptune.

Parameters

Example

from neptune_scale import Run

with Run(...) as run:
    run.log_configs(...)
    ...
    run.wait_for_processing()
    run.log_metrics(...)  # called once submitted data has been processed

In case an unrecoverable error is encountered, you can terminate the failed run in your error callback.

Note: This effectively disables processing in-flight operations as well as logging new data. However, the training process isn't interrupted.

Example

from neptune_scale import Run

def my_error_callback(exc):
    run.terminate()


run = Run(..., on_error_callback=my_error_callback)

For help, contact [email protected].