Ietf (Chia-Network#119)

* wip schemes * wip: all schemes tests pass * del relic in prep for submodule add * add relic submodule, major wip commit with basic functionality * manual merge hdkey, better logic for g1/g2 leading bits * fix compresspoint g2 * g1/g2 serialization correct * Get IETF branch building under MacOS and Linux (Chia-Network#120) * remove travis * bring the ci forward 2 months * bring setup.py forward * add flake8 ini * add mypi.ini * first attempt at FetchContent chia-relic-src * Final cmake fixes * Add lgtm.yml * explicit order of ops * clean up code for mypy/flake8 - black * enough to start looking at plotter * lgtm assignment operator * add 0x80 compression bit for 0 and 48 byte * reverted bit changes and commented out G2 tests * switch repos for relic * Remove Unknown distribution option: 'build_requires' warning * msvc fixes * Add libsodium to all platform wheels. README nit * Fix ARM, use nuget on Windows * Try libsodium-devel, bump verbosity to confirm sodium * ARM install epel first - Fix find sodium macos * no need for floor * add fingerprint, derive_child * fix __bytes__ * add n of n sigs in AugScheme, use MPL suffix * standardizing around __bytes__ * libsodium on linux (x86_64 & ARM) * allow agg sig of 0 msgs * alias g1/g2 element * Try find gmp from chiavdf * Build gmp and sodium from scratch * Fix ARM sodium and gmp - start debugging macos * delete dist dir on ARM shr's * Move all but Windows to build verbosity 0 * Changes required for chia-blockchain (deepcopy) * ADD hd keys tests * added gmp include folder to python-bindings cmake * pyecc tests * Move ci's to pytest * Add test vectors ietf * g2 elements start with 0b000 now * Use FromSeed * Revert bad test vectors Co-authored-by: Alex Wice <[email protected]> Co-authored-by: Gene Hoffman <[email protected]> Co-authored-by: Bill Blanke <[email protected]> Co-authored-by: Gene Hoffman <[email protected]>
sarvex · Jul 14, 2020 · c6010c8 · c6010c8
1 parent f4ffccd
commit c6010c8
Show file tree

Hide file tree

Showing 677 changed files with 4,651 additions and 212,176 deletions.
diff --git a/.clang-format b/.clang-format
@@ -0,0 +1,17 @@
+BasedOnStyle: Google
+UseTab: Never
+ColumnLimit: 80
+IndentWidth: 4
+TabWidth: 4
+AllowShortIfStatementsOnASingleLine: false
+IndentCaseLabels: false
+AccessModifierOffset: -4
+BinPackArguments: false
+BinPackParameters: false
+AlignAfterOpenBracket: AlwaysBreak
+IndentCaseLabels: true
+AllowAllParametersOfDeclarationOnNextLine: false
+BreakBeforeBraces: Custom
+BraceWrapping:
+    AfterFunction: true
+PenaltyReturnTypeOnItsOwnLine: 1000
diff --git a/.flake8 b/.flake8
@@ -1,5 +1,4 @@
 [flake8]
 max-line-length = 120
-exclude = ./typings/**/*
+exclude = ./typings/**/* python-impl/fields.py
 ignore = E203,W503
-
diff --git a/.github/workflows/build-aarch64.yml b/.github/workflows/build-aarch64.yml
@@ -18,7 +18,7 @@ jobs:
         access_token: ${{ github.token }}
 
     - name: Checkout code
-      uses: actions/checkout@v1
+      uses: actions/checkout@v2
       with:
         fetch-depth: 0
         # we need fetch-depth 0 so setuptools_scm can resolve tags
@@ -47,13 +47,13 @@ jobs:
       run: |
         source venv/bin/activate
         pip install flake8
-        flake8 src setup.py
+        flake8 src setup.py python-bindings python-impl
 
     - name: Lint source with mypy
       run: |
         source venv/bin/activate
         pip install mypy
-        mypy --config-file mypi.ini python-impl python-bindings
+        mypy --config-file mypi.ini python-bindings python-impl
 
     - name: Build wheel and test
       run: |
@@ -64,16 +64,22 @@ jobs:
         CIBW_BUILD: cp38-*
         CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014
         # we need boost
+        CIBW_BUILD_VERBOSITY_LINUX: 0
         CIBW_BEFORE_BUILD_LINUX: >
           yum -y install epel-release
-          && yum -y install cmake3
+          && yum -y install cmake3 lzip
           && ln -s /usr/bin/cmake3 /usr/local/bin/cmake
           && python -m pip install --upgrade pip
           && cmake --version
           && uname -a
-        CIBW_BUILD_VERBOSITY_LINUX: 0
+          && curl -L https://gmplib.org/download/gmp/gmp-6.2.0.tar.lz | lzip -dc | tar x
+          && cd gmp-6.2.0 && ./configure --enable-fat
+          && make && make install && cd .. && rm -rf gmp-6.2.0
+          && curl -L https://download.libsodium.org/libsodium/releases/libsodium-1.0.18-stable.tar.gz | tar xz
+          && cd libsodium-stable && ./configure
+          && make && make install && cd .. && rm -rf libsodium-stable
         CIBW_TEST_REQUIRES: pytest
-        CIBW_TEST_COMMAND: python {project}/python-bindings/test.py
+        CIBW_TEST_COMMAND: pytest -v {project}/python-bindings/test.py
 
     - name: Upload artifacts
       uses: actions/upload-artifact@v1
@@ -107,6 +113,7 @@ jobs:
         source venv/bin/activate
         twine upload --non-interactive --skip-existing --verbose 'dist/*'
 
-    - name: Clean up venv
+    - name: Clean up
       run: |
         rm -rf venv
+        rm -rf dist
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -17,7 +17,8 @@ jobs:
       with:
         access_token: ${{ github.token }}
 
-    - uses: actions/checkout@v1
+    - name: Checkout code
+      uses: actions/checkout@v2
       with:
         fetch-depth: 0
         # we need fetch-depth 0 so setuptools_scm can resolve tags
@@ -45,12 +46,12 @@ jobs:
     - name: Lint source with flake8
       run: |
         pip install flake8
-        flake8 src setup.py
+        flake8 src setup.py python-bindings python-impl
 
     - name: Lint source with mypy
       run: |
         pip install mypy
-        mypy --config-file mypi.ini python-impl python-bindings
+        mypy --config-file mypi.ini python-bindings python-impl
 
     - name: Build source distribution with MacOS
       if: startsWith(matrix.os, 'mac')
@@ -62,27 +63,42 @@ jobs:
       run: |
         python -m cibuildwheel --output-dir dist
       env:
+        CIBW_BUILD_VERBOSITY_MACOS: 0
+        CIBW_BUILD_VERBOSITY_LINUX: 0
+        CIBW_BUILD_VERBOSITY_WINDOWS: 1
         # Python 3.7 and 3.8
         CIBW_BUILD: cp37-* cp38-*
         # don't build i686 targets, can't seem to find cmake for these
         CIBW_SKIP: '*-manylinux_i686 *-win32'
         # we need boost
         CIBW_TEST_REQUIRES: pytest
         CIBW_MANYLINUX_X86_64_IMAGE: manylinux2010
+        CIBW_ENVIRONMENT_LINUX: "PATH=/project/cmake-3.17.3-Linux-`uname -m`/bin:$PATH"
         CIBW_BEFORE_BUILD_LINUX: >
-          curl -L https://github.com/Kitware/CMake/releases/download/v3.17.0/cmake-3.17.0-Linux-`uname -m`.sh > cmake.sh
+          yum -y install lzip
+          && curl -L https://github.com/Kitware/CMake/releases/download/v3.17.3/cmake-3.17.3-Linux-`uname -m`.sh > cmake.sh
           && yes | sh cmake.sh | cat
-          && rm -f /usr/bin/cmake
+          && rm -f /usr/bin/cmake && hash -r
           && python -m pip install --upgrade pip
           && which cmake
           && cmake --version
-        CIBW_BEFORE_BUILD_MACOS: python -m pip install --upgrade pip
+          && curl -L https://gmplib.org/download/gmp/gmp-6.2.0.tar.lz | lzip -dc | tar x
+          && cd gmp-6.2.0 && ./configure --enable-fat
+          && make && make install && cd .. && rm -rf gmp-6.2.0
+          && curl -L https://download.libsodium.org/libsodium/releases/libsodium-1.0.18-stable.tar.gz | tar xz
+          && cd libsodium-stable && ./configure
+          && make && make install && cd .. && rm -rf libsodium-stable
+        # GH hosted runner has gmp and libsodium natively
+        CIBW_BEFORE_BUILD_MACOS: >
+          python -m pip install --upgrade pip
         CIBW_ENVIRONMENT_MACOS: MACOSX_DEPLOYMENT_TARGET=10.14
-        CIBW_BUILD_VERBOSITY_MACOS: 0
         CIBW_BEFORE_BUILD_WINDOWS: >
-          python -m pip install --upgrade pip && pwd
-          && git clone https://github.com/Chia-Network/relic_gmp_64.git
-          && ls -l relic_gmp_64
+          python -m pip install --upgrade pip
+          && pwd
+          && nuget install libsodium -Version 1.0.18
+          && ls -l libsodium.1.0.18/runtimes/win-x64/native/
+          && git clone https://github.com/Chia-Network/relic_ietf_64.git
+          && ls -l relic_ietf_64
           && git clone https://github.com/Chia-Network/mpir_gc_x64.git
           && ls -l mpir_gc_x64
         CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: >
@@ -98,12 +114,11 @@ jobs:
           && delocate-wheel -v -i mpir_gc_x64/mpir_skylake.dll {wheel}
           && delocate-wheel -v -i mpir_gc_x64/mpir_skylake_avx.dll {wheel}
           && cp {wheel} {dest_dir} && ls -l {dest_dir}
-          && rm -fr relic_gmp_64
+          && rm -fr relic_ietf_64
           && rm -fr mpir_gc_x64
         # The pure python implementation, is not used anywhere and will be replaced
         # pytest {project}/python-impl/tests.py
-        CIBW_TEST_COMMAND: python {project}/python-bindings/test.py
-        CIBW_ENVIRONMENT_LINUX: "PATH=/project/cmake-3.17.0-Linux-`uname -m`/bin:$PATH"
+        CIBW_TEST_COMMAND: pytest -v {project}/python-bindings/test.py
         CIBW_REPAIR_WHEEL_COMMAND_MACOS: >
           pip uninstall -y delocate
           && pip install git+https://github.com/Chia-Network/delocate.git
@@ -114,9 +129,10 @@ jobs:
     - name: Mac OS build C++ and test
       if: startsWith(matrix.os, 'macos')
       run: |
-        brew install pkg-config
-
+        ls -l
+        export MACOSX_DEPLOYMENT_TARGET=10.14
         mkdir -p build
+        ls -l build
         cd build
         cmake ../
         cmake --build . -- -j 6
@@ -128,6 +144,11 @@ jobs:
       run: |
         sudo apt-get update
         sudo apt-get install valgrind libsodium-dev -y
+        sudo apt-get install snap -y
+        sudo apt-get remove --purge cmake -y
+        sudo snap install cmake --classic
+        hash -r
+        cmake --version
         mkdir -p build
         cd build
         cmake ../

diff --git a/.gitignore b/.gitignore
@@ -53,3 +53,6 @@ blsbench.*
 *.pb.h
 
 **/.DS_Store
+
+*.whl
+venv
diff --git a/.gitmodules b/.gitmodules
@@ -1,3 +1,6 @@
 [submodule "contrib/pybind11"]
 	path = contrib/pybind11
 	url = https://github.com/pybind/pybind11.git
+[submodule "contrib/relic"]
+	path = contrib/relic
+	url = https://github.com/Chia-Network/relic
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -1,5 +1,4 @@
-# add_compile_definitions() was introduced in cmake 3.12 but this compiles on 3.10
-CMAKE_MINIMUM_REQUIRED(VERSION 3.10.0 FATAL_ERROR)
+CMAKE_MINIMUM_REQUIRED(VERSION 3.11.0 FATAL_ERROR)
 set(CMAKE_CXX_STANDARD 11)
 set(CMAKE_POSITION_INDEPENDENT_CODE ON)
 
@@ -49,6 +48,7 @@ set(VERBS "off" CACHE STRING "")
 set(ALLOC "AUTO" CACHE STRING "")
 set(SHLIB "OFF" CACHE STRING "")
 set(MULTI "PTHREAD" CACHE STRING "")
+set(DOCUM "off" CACHE STRING "")
 
 set(FP_PRIME 381 CACHE INTEGER "")
 
@@ -78,12 +78,13 @@ set(QUIET 1 CACHE INTEGER "")
 set(PP_EXT "LAZYR" CACHE STRING "")
 set(PP_METHD "LAZYR;OATEP" CACHE STRING "")
 
-add_subdirectory(contrib/relic)
+#add_subdirectory(contrib/relic)
 add_subdirectory(src)
 
 if (EMSCRIPTEN)
   add_subdirectory(js-bindings)
 else()
   # emscripten can't build python bindings, it produces only javascript
+  # add_subdirectory(contrib/pybind11)
   add_subdirectory(python-bindings)
 endif()
diff --git a/README.md b/README.md
@@ -17,11 +17,10 @@ Implements BLS signatures with aggregation as in
 , using [relic toolkit](https://github.com/relic-toolkit/relic)
 for cryptographic primitives (pairings, EC, hashing).
 The [BLS12-381](https://github.com/zkcrypto/pairing/tree/master/src/bls12_381)
-curve is used. The spec is
+curve is used. The original spec is
 [here](https://github.com/Chia-Network/bls-signatures/tree/master/SPEC.md).
-This library will be migrating to the
-[IETF BLS RFC](https://datatracker.ietf.org/doc/draft-irtf-cfrg-bls-signature/)
-shortly.
+This library now implements
+[IETF BLS RFC](https://datatracker.ietf.org/doc/draft-irtf-cfrg-bls-signature/).
 
 Features:
 * Non-interactive signature aggregation on identical or distinct messages
@@ -266,15 +265,12 @@ function. Note: relic is used with the Apache 2.0 license.
 Libsodium and GMP are optional dependencies: libsodium gives secure memory
 allocation, and GMP speeds up the library by ~ 3x. To install them, either
 download them from github and follow the instructions for each repo, or use
-a package manager like APT or brew.
+a package manager like APT or brew. You can follow the recipe used to build
+python wheels for multiple platforms in `.github/workflows/`
 
 ### Discussion
-Discussion about this library and other Chia related development is on Keybase.
-Install Keybase, and run the following to join the Chia public channels:
-
-```bash
-keybase team request-access chia_network.public
-```
+Discussion about this library and other Chia related development is in Chia's
+[public Keybase channels](https://keybase.io/team/chia_network.public).
 
 ### Code style
 * Always use uint8_t for bytes
@@ -296,10 +292,10 @@ build binary wheels for MacOS, Linux (x64 and aarch64), and Windows and publish
 them with a source wheel on PyPi. See `.github/workflows/build.yml`. CMake uses
 [FetchContent](https://cmake.org/cmake/help/latest/module/FetchContent.html)
 to download [pybind11](https://github.com/pybind/pybind11) for the Python
-bindings. Building is then managed by
-[cibuildwheel](https://github.com/joerick/cibuildwheel). Further installation
-is then available via `pip install blspy` e.g. The ci builds include GMP and
-libsoduium.
+bindings and relic from a chia relic forked reporitory. Building is then
+managed by [cibuildwheel](https://github.com/joerick/cibuildwheel).
+Further installation is then available via `pip install blspy` e.g. The ci
+builds include GMP and soduium.
 
 ### Contributing and workflow
 Contributions are welcome and more details are available in chia-blockchain's
-Original file line number
+Diff line change
@@ Expand Up / @@ -53,3 +53,6 @@ blsbench.* @@
     *.pb.h
     **/.DS_Store
+    *.whl
+    venv