Fix CI for Windows (DLTcollab#18)

* Update windows-x86_64.jsonl * Update azure-pipelines.yml * Delete .appveyor_generate_make.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Debug windows build * Experiment * Try adding conda-forge scripts * Try using build from blis conda-forge * Fix yaml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update build.sh * Update build.sh * Update azure-pipelines.yml * Update azure-pipelines.yml * Update build.sh * Update azure-pipelines.yml * Update build.sh * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update setup.py * Update setup.py * Update azure-pipelines.yml * Update setup.py * Update setup.py * Update setup.py * Update setup.py * Update azure-pipelines.yml * Update setup.py * Update windows-x86_64.jsonl * Update azure-pipelines.yml * Update windows-x86_64.jsonl * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update setup.py * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Update azure-pipelines.yml * Reorganize old CI files * Update azure-pipelines.yml * Update setup.py * Update azure-pipelines.yml * Update setup.py * Update setup.py * Update azure-pipelines.yml * Update README.md [ci skip] * Update azure-pipelines.yml Co-authored-by: Matthew Honnibal <[email protected]>
sebpop · Nov 14, 2019 · 02ac486 · 02ac486
1 parent 4530245
commit 02ac486
Show file tree

Hide file tree

Showing 11 changed files with 53 additions and 80 deletions.
diff --git a/.appveyor_generate_make.yml b/.appveyor_generate_make.yml
diff --git a/README.md b/README.md
@@ -7,7 +7,7 @@ routines as a self-contained Python C-extension.
 
 Currently, we only supports single-threaded execution, as this is actually best for our workloads (ML inference).
 
-[![Appveyor](https://img.shields.io/appveyor/ci/explosion/cython-blis/master.svg?style=flat-square&logo=appveyor)](https://ci.appveyor.com/project/explosion/cython-blis)
+[![Azure Pipelines](https://img.shields.io/azure-devops/build/explosion-ai/public/6/master.svg?logo=azure-pipelines&style=flat-square)](https://dev.azure.com/explosion-ai/public/_build?definitionId=6)
 [![pypi Version](https://img.shields.io/pypi/v/blis.svg?style=flat-square)](https://pypi.python.org/pypi/blis)
 [![conda](https://img.shields.io/conda/vn/conda-forge/cython-blis.svg?style=flat-square)](https://anaconda.org/conda-forge/cython-blis)
 [![Python wheels](https://img.shields.io/badge/wheels-%E2%9C%93-4c1.svg?longCache=true&style=flat-square&logo=python&logoColor=white)](https://github.com/explosion/wheelwright/releases)

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
@@ -5,43 +5,45 @@ trigger:
     - '*'
 
 jobs:
-
 - job: 'Test'
   strategy:
     matrix:
-      Python27Linux:
-        imageName: 'ubuntu-16.04'
-        python.version: '2.7'
-      Python27Mac:
-        imageName: 'macos-10.13'
-        python.version: '2.7'
       Python35Linux:
         imageName: 'ubuntu-16.04'
         python.version: '3.5'
       Python35Mac:
         imageName: 'macos-10.13'
         python.version: '3.5'
+      Python35Windows:
+        imageName: 'vs2017-win2016'
+        python.version: '3.5'
       Python36Linux:
         imageName: 'ubuntu-16.04'
         python.version: '3.6'
       Python36Mac:
         imageName: 'macos-10.13'
         python.version: '3.6'
+      Python36Windows:
+        imageName: 'vs2017-win2016'
+        python.version: '3.6'
       Python37Linux:
         imageName: 'ubuntu-16.04'
         python.version: '3.7'
       Python37Mac:
         imageName: 'macos-10.13'
         python.version: '3.7'
+      Python37Windows:
+        imageName: 'vs2017-win2016'
+        python.version: '3.7'
       Python38Linux:
         imageName: 'ubuntu-16.04'
         python.version: '3.8'
-      # Python38Windows:
-      #   imageName: 'vs2017-win2016'
-      #   python.version: '3.8'
       Python38Mac:
         imageName: 'macos-10.13'
         python.version: '3.8'
+      Python38Windows:
+        imageName: 'vs2017-win2016'
+        python.version: '3.8'
     maxParallel: 4
   pool:
     vmImage: $(imageName)
@@ -52,33 +54,47 @@ jobs:
       versionSpec: '$(python.version)'
       architecture: 'x64'
 
-  - script: |
-      source ./travis/before_install_linux.sh
-      before_install
-    condition: eq( variables['Agent.OS'], 'Linux')
-    displayName: 'Preinstall (Linux)'
+  - script: choco install llvm
+    condition: eq( variables['Agent.OS'], 'Windows_NT')
+    displayName: 'Preinstall (Windows)'
 
   - script: |
       python -m pip install --upgrade pip wheel
       pip install -r requirements.txt
     displayName: 'Install dependencies'
 
-  - script: python setup.py bdist_wheel
-    displayName: 'Build wheel'
-
+  # Set the correct paths and includes. Only the env variables set here are
+  # used, not the ones defined in the .jsonl.
   - script: |
-      choco install llvm
-      cd ..
-      bash -c "cp -r $(Agent.BuildDirectory) /c/build"
-      bash -lc "python -m pip install /c/build/dist/*.whl"
+      set "PATH=C:\Program Files\LLVM\bin;%PATH%"
+      set "INCLUDE=%$VC_INCLUDEPATH%"
+      clang --version
+      python setup.py bdist_wheel
     condition: eq( variables['Agent.OS'], 'Windows_NT')
-    displayName: 'Install wheel (Windows)'
+    displayName: 'Build wheel (Windows)'
 
-  - script: |
+  - script: python setup.py bdist_wheel
+    condition: or(eq(variables['Agent.OS'], 'Linux'), eq(variables['Agent.OS'], 'Darwin'))
+    displayName: 'Build wheel (Linux / Mac)'
+
+  - task: PythonScript@0
+    inputs:
+      scriptSource: inline
+      script: |
+        # https://github.com/pypa/pip/issues/6951
+        import os
+        wheel_file = os.listdir('./dist')[0]
+        wheel_path = os.path.join('./dist', wheel_file)
+        os.rename(wheel_path, wheel_path.replace("cp38m-win", "cp38-win"))
+        print(os.listdir('./dist'))
+      failOnStderr: true
+    condition: and(eq(variables['Agent.OS'], 'Windows_NT'), eq(variables['python.version'], '3.8'))
+    displayName: Fix wheel name (Windows, Python 3.8)
+
+  - bash: |
       rm -rf blis*
       python -m pip install dist/*.whl
-    condition: in( variables['Agent.OS'], 'Linux', 'Darwin')
-    displayName: 'Install wheel (Linux, Mac)'
+    displayName: 'Install wheel'
 
   - script: |
       python -m pytest --pyargs blis

diff --git a/.appveyor_compile_jsonl.yml → bin/.appveyor_compile_jsonl.yml b/.appveyor_compile_jsonl.yml → bin/.appveyor_compile_jsonl.yml
diff --git a/.appveyor_run_tests.yml → bin/.appveyor_run_tests.yml b/.appveyor_run_tests.yml → bin/.appveyor_run_tests.yml
diff --git a/.update_blis_travis.yml → bin/.update_blis_travis.yml b/.update_blis_travis.yml → bin/.update_blis_travis.yml
diff --git a/travis/before_install_linux.sh → bin/travis/before_install_linux.sh b/travis/before_install_linux.sh → bin/travis/before_install_linux.sh
diff --git a/travis/before_install_osx.sh → bin/travis/before_install_osx.sh b/travis/before_install_osx.sh → bin/travis/before_install_osx.sh
diff --git a/travis/common_utils.sh → bin/travis/common_utils.sh b/travis/common_utils.sh → bin/travis/common_utils.sh
diff --git a/blis/_src/make/windows-x86_64.jsonl b/blis/_src/make/windows-x86_64.jsonl
diff --git a/setup.py b/setup.py
@@ -6,7 +6,7 @@
 # but we need to tell OSX to build for 10.7.
 # Otherwise, wheels don't work. We can't use 10.6,
 # it doesn't compile.
-#if "MACOSX_DEPLOYMENT_TARGET" not in os.environ:
+# if "MACOSX_DEPLOYMENT_TARGET" not in os.environ:
 #    os.environ["MACOSX_DEPLOYMENT_TARGET"] = "10.7"
 
 import contextlib
@@ -197,8 +197,14 @@ def build_object(self, compiler, source, target, flags, macros, include, env=Non
         command.extend(flags)
         command.extend(macros)
         command.extend(include)
-        print(" ".join(command))
-        subprocess.check_call(command, cwd=BLIS_DIR)
+        print("[COMMAND]", " ".join(command))
+        try:
+            kwargs = {"cwd": BLIS_DIR, "check": True}
+            if sys.version_info[0] == 3 and sys.version_info[1] > 6:
+                kwargs["capture_output"] = True
+            subprocess.run(command, **kwargs)
+        except Exception as err:
+            raise ValueError(getattr(err, "stderr", err))
         return target