Tales from Nixpkgs - PR

This is the first in a new series of posts called “Tales from Nixpkgs” where we examine Nixpkgs commits to understand how this ecosystem works and provide a bit of visibility into the process. This also gives us a chance to show appreciation for the maintainers and perhaps also uncover interesting stories along the way.

Let’s take a look at a randomly picked commit from the last year of Nixpkgs:

git log --pretty=oneline --since 1y | shuf | head -n1 | cut -f1 -d' ' | xargs git show
commit 785bafc33818503172c7eecb60af711d794195b3
Merge: e7d00dfbd39 1e69e5c4280
Author: Fabian Affolter <[email protected]h>
Date:   Fri Mar 3 15:05:51 2023 +0100

    Merge pull request #219037 from fabaff/openai-bump

    python310Packages.openai: 0.26.5 -> 0.27.0

The original PR for this is #219037. The motivation is to update the version of the openai Python package. It looks like along the way there is also a related update to shell-genie and some improvements to the metadata in Nixpkgs. A simple version bump is usually simpler, but this example is good because it will help us explain a few things along the way.

The entails taking a look at the build recipe and updating a few things:

git diff 785bafc33818503172c7eecb60af711d794195b3~1 785bafc33818503172c7eecb60af711d794195b3
pkgs/development/python-modules/openai/default.nix
diff --git a/pkgs/development/python-modules/openai/default.nix b/pkgs/development/python-modules/openai/default.nix
index ede1a7a3d84..92f2e327eb3 100644
--- a/pkgs/development/python-modules/openai/default.nix
+++ b/pkgs/development/python-modules/openai/default.nix
@@ -23,7 +23,7 @@

 buildPythonPackage rec {
   pname = "openai";
-  version = "0.26.5";
+  version = "0.27.0";
   format = "setuptools";

   disabled = pythonOlder "3.7.1";
@@ -31,8 +31,8 @@ buildPythonPackage rec {
   src = fetchFromGitHub {
     owner = "openai";
     repo = "openai-python";
-    rev = "v${version}";
-    hash = "sha256-eKU+WRFf7f1yH63vcoQ9dVeqhJXBqMJGpk/9AoEgR0M=";
+    rev = "refs/tags/v${version}";
+    hash = "sha256-pXttGvnApYuwkWU7kCDNjw0rjHD5AyUvujfvpDVjgxM=";
   };

   propagatedBuildInputs = \[
@@ -91,6 +91,7 @@ buildPythonPackage rec {
   meta = with lib; {
     description = "Python client library for the OpenAI API";
     homepage = "https://github.com/openai/openai-python";
+    changelog = "https://github.com/openai/openai-python/releases/tag/v${version}";
     license = licenses.mit;
     maintainers = with maintainers; \[ malo \];
   };

First off, we see a bump in version from 0.26.5 to 0.27.0. Next we see the source is also updated with its hash providing an integrity check. (Note: these two steps are often performed by automation, stay tuned for future episodes). Lastly we see an improvement to the metadata pointing at a consistent location for the changelog of the OpenAI project.

Next we see changes to shell-genie. This is an application that uses openai to provide helpful command-line suggestions and it uses openai as a dependency.

git diff 785bafc33818503172c7eecb60af711d794195b3~1 785bafc33818503172c7eecb60af711d794195b3
pkgs/applications/misc/shell-genie/default.nix
diff --git a/pkgs/applications/misc/shell-genie/default.nix b/pkgs/applications/misc/shell-genie/default.nix
index 1429aabcd80..7225eb4c3fc 100644
--- a/pkgs/applications/misc/shell-genie/default.nix
+++ b/pkgs/applications/misc/shell-genie/default.nix
@@ -8,18 +8,16 @@ with python3.pkgs;

 buildPythonPackage rec {
   pname = "shell-genie";
-  version = "unstable-2023-01-27";
+  version = "0.2.6";
   format = "pyproject";

-  src = fetchFromGitHub {
-    owner = "dylanjcastillo";
-    repo = pname;
-    rev = "d6da42a4426e6058a0b5ae07837d8c003cd1239e";
-    hash = "sha256-MGhQaTcl3KjAJXorOmMRec07LxH02T81rNbV2mYEpRA=";
+  src = fetchPypi {
+    pname = "shell\_genie";
+    inherit version;
+    hash = "sha256-MgQFHsBXrihfWBB/cz45ITf8oJG2gSenf1wzdbrAbjw=";
   };

   nativeBuildInputs = \[
-    poetry
     poetry-core
   \];

@@ -35,9 +33,14 @@ buildPythonPackage rec {
   # No tests available
   doCheck = false;

+  pythonImportsCheck = \[
+    "shell\_genie"
+  \];
+
   meta = with lib; {
     description = "Describe your shell commands in natural language";
     homepage = "https://github.com/dylanjcastillo/shell-genie";
+    # https://github.com/dylanjcastillo/shell-genie/issues/3
     license = licenses.unfree;
     maintainers = with maintainers; \[ onny \];
   };

We see a set of very similar changes, a version bump, update and improvement to how we fetch the sources, additional checks performed, as well as an identification of an issue - there is no declared license for the shell-genie project. This brings us to an important part of the work of packaging. It is not only the technical work to shepherd updates along and fix bugs, but to communicate and coordinate across the open source world. @fabaff submitted an issue to the upstream project which resulted in this addition.

It is through many small changes and fixes like this that the overall ecosystem improves and these improvements have benefits not just for the Nix ecosystem, but outside of it as well. A common theme we will see in these tales is that our very strict approach to packaging has trickle-down benefits for upstream projects and other package managers.

So we have a fairly simple update, but also improvements to metadata and collaboration with other communities. Next in the PR we see some testing of the new recipe across various architectures and an approval from @malob. With the approval and all checks and tests in the green, the PR was merged. Sounds like success!

Thank you @fabaff for your contribution!