Merge branch 'master' into fft-example

t-fine · Jun 5, 2023 · c003b4a · c003b4a
2 parents 9cabf7b + 53c47c1
commit c003b4a
Show file tree

Hide file tree

Showing 234 changed files with 3,508 additions and 35,510 deletions.
diff --git a/.github/ISSUE_TEMPLATE/Other.md b/.github/ISSUE_TEMPLATE/Other.md
@@ -0,0 +1,6 @@
+---
+name: Other
+about: Something else
+
+---
+
diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md
@@ -0,0 +1,32 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Operating Environment:**
+ - Desktop, Cluster, Cloud, other 
+ - OS, technology stack
+ - other
+
+**Additional context**
+Add any other context about the problem here.
diff --git a/.github/ISSUE_TEMPLATE/documentation-update.yml b/.github/ISSUE_TEMPLATE/documentation-update.yml
@@ -0,0 +1,30 @@
+name: Documentation Update📄
+description: Suggest how we can improve our documentation
+title: "Documentation📄: "
+labels: [documentation]
+
+
+body:
+  - type: markdown
+    attributes:
+      value: Thank you for taking the time to file a Documentation update report
+  - type: textarea
+    attributes:
+      label: What is the current documentation state?
+      description: "Copy and paste the text currently in the documentation. If you want to improve graphs or photos, please provide a screenshot of the original."
+  - type: textarea
+    attributes:
+      label: Where is this stated?
+      description: "Provide additional information about the location where you have copied the above statement from."
+  - type: textarea
+    attributes:
+      label: Why do you want to improve the statement?
+      description: "Please explain why the statement needs to be updated. This can be because it is confusing, incorrect, spelling/grammatical errors, etc."
+  - type: textarea
+    attributes:
+      label: Proposed Statement
+      description: "Please propose how you would rewrite the statement. If you have a precise proposal, please make the changes and open a pull request."
+  - type: textarea
+    attributes:
+      label: Additional context.
+      description: "Add any other context or useful information about the documentation update here."
diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -0,0 +1,25 @@
+name: Feature Request
+description: Suggest an idea for this project
+title: "Feature Request: "
+labels: [enhancement]
+
+body:
+  - type: markdown
+    attributes:
+      value: Thank you for taking the time to file a Feature Request report.
+  - type: textarea
+    attributes:
+      label: Is your feature request related to a problem? Please describe.
+      description: 'A clear and concise description of what the problem is. Ex. I am always frustrated when [...]'
+  - type: textarea
+    attributes:
+      label: Describe the solution you'd like.
+      description: 'A clear and concise description of what you want to happen.'
+  - type: textarea
+    attributes:
+      label: Describe alternatives you've considered
+      description: 'A clear and concise description of any alternative solutions or features you have considered.'
+  - type: textarea
+    attributes:
+      label: Additional context.
+      description: 'Add any other context or screenshots about the feature request here.'
diff --git a/.github/config.yml b/.github/config.yml
@@ -0,0 +1,41 @@
+# Configuration for welcome - https://github.com/behaviorbot/welcome
+
+# Configuration for new-issue-welcome - https://github.com/behaviorbot/new-issue-welcome
+
+# Comment to be posted to on first time issues
+newIssueWelcomeComment: |
+  Hello there!👋 Welcome to Open-Horizon-Examples!💖
+  Thank you and congratulations 🎉 for opening your very first issue in this project.
+  Please checkout if any of the already existing issue template fits for your isssue or you may open a blank issue as well ✨
+  We will try to get back to you as soon as we can.👀
+
+# Configuration for new-pr-welcome - https://github.com/behaviorbot/new-pr-welcome
+
+# Comment to be posted to on PRs from first time contributors in your repository
+newPRWelcomeComment: |
+  Hello there!👋 Welcome to Open-Horizon-Examples!💖
+  Thank you and congrats 🎉 for opening your first PR on this project.✨
+  We will review it soon! Till then you can checkout the `README.md` for more details on it.
+# Configuration for first-pr-merge - https://github.com/behaviorbot/first-pr-merge
+
+# Comment to be posted to on pull requests merged by a first time user
+firstPRMergeComment: |
+  Congratulations on merging your first contribution to Open-Horizon-Examples!✨
+# *Required* Comment to reply with
+requestInfoReplyComment: >
+  Hi,
+  Thanks for raising an issue/pull request! We at Open-Horizon appreciate your concerns and views to
+  help us improve. 
+# *OPTIONAL* default titles to check against for lack of descriptiveness
+# MUST BE ALL LOWERCASE
+requestInfoDefaultTitles:
+  - update readme.md
+  - updates
+  - add files via upload
+  - readme.md
+  - changes made
+
+# *OPTIONAL* Label to be added to Issues and Pull Requests with insufficient information given
+requestInfoLabelToAdd: needs-more-info
+
+# It is recommended to include as many gifs and emojis as possible!
diff --git a/.github/workflows/pr-test.yml b/.github/workflows/pr-test.yml
@@ -0,0 +1,79 @@
+# This pipeline is to build anax binaries
+
+name: pr-test
+
+on: 
+  pull_request:
+    branches:
+      - master
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    env:
+      HZN_EXCHANGE_URL: https://stg.edge-fabric.com/v1 
+      EVTSTREAMS_TOPIC: cpu2evtstreams 
+      HZN_DEVICE_ID: travis-test
+      HZN_EXCHANGE_USER_AUTH: iamapikey:blah2
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+
+      # prepare the environment
+      - name: Prep Env
+        run: |
+          sudo apt-get update
+          sudo apt-get install librdkafka-dev libyajl-dev kafkacat qemu qemu-user-static binfmt-support dpkg-cross
+      
+      # Install the hzn cli
+      - name: Install hzn
+        run: |
+          wget https://github.com/open-horizon/anax/releases/latest/download/horizon-agent-linux-deb-amd64.tar.gz -O /tmp/horizon-agent-linux-deb-amd64.tar.gz
+          tar -zxf /tmp/horizon-agent-linux-deb-amd64.tar.gz -C /tmp/
+          sudo apt-get install -y /tmp/horizon-cli*.deb
+
+      # Build and test the example services that were changed in the PR for all arches 
+      # Note: tools/travis-find determines if the specified "service" was modified, so we know if we should test it.
+      # The three docker commands are to simulate the arm and arm64 arches for cross-compiling 
+      # cpu2evtstreams is the only service that is checked end to end, making sure data is delivered to IBM Event Streams
+      - name: Build and Test
+        shell: bash {0}
+        run: |
+          # test services
+          # set -x
+          # Note that several services are omitted due to the unavailability of a publicly accessible Exchange server which used to be stg.edge-fabric.com
+          # services=( edge/services/helloworld edge/services/cpu_percent edge/services/gps edge/services/sdr edge/services/helloMMS edge/services/mqtt_broker edge/services/mqtt2kafka edge/services/hotword_detection edge/services/stopword_removal edge/services/audio2text edge/services/text2speech edge/services/voice2audio edge/services/processtext edge/evtstreams/cpu2evtstreams edge/evtstreams/sdr2evtstreams edge/evtstreams/watson_speech2text )
+          services=( edge/services/helloworld edge/services/cpu_percent edge/services/gps edge/services/sdr edge/services/helloMMS edge/services/mqtt_broker edge/services/stopword_removal )
+          docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+          docker run --rm -t arm32v6/alpine uname -m
+          docker run --rm -t arm64v8/ubuntu uname -m
+
+          for service in "${services[@]}"; do
+            echo "*********************************** Testing example service: $service *********************************** "
+            if [[ $service == *"cpu2evtstreams"* ]]; then 
+              echo "Monitoring IBM Event Streams"
+              kafkacat -C -c 1 -q -o end -f "%t/%p/%o/%k: %s\n" -b $EVTSTREAMS_BROKER_URL -X api.version.request=true -X security.protocol=sasl_ssl -X sasl.mechanisms=PLAIN -X sasl.username=token -X sasl.password=$EVTSTREAMS_API_KEY -t $EVTSTREAMS_TOPIC > $GITHUB_WORKSPACE/output.txt 2>&1 &
+            fi
+            cd $GITHUB_WORKSPACE/$service
+            make test-all-arches
+            if [ $? -eq 0 ]; then
+              echo "Make exited 0"
+              if [[ $service == *"cpu2evtstreams"* ]]; then
+                if grep -Fq "$HZN_DEVICE_ID" $GITHUB_WORKSPACE/output.txt; then
+                  cat $GITHUB_WORKSPACE/output.txt
+                  passed=( "${passed[@]}" "$service" )
+                fi
+              else
+                passed=( "${passed[@]}" "$service" )
+              fi
+            else
+              echo "***********************************"
+              echo "*********************************** Make exited with non-zero result *********************************** "
+              echo "***********************************"
+              exit 1
+            fi
+          done
+          cd $GITHUB_WORKSPACE
+ 
diff --git a/.travis.yml b/.travis.yml
@@ -37,14 +37,10 @@ env:
   - HZN_EXCHANGE_URL=https://stg.edge-fabric.com/v1 EVTSTREAMS_TOPIC=cpu2evtstreams HZN_DEVICE_ID=travis-test
 
 # Install the hzn command
-addons:
-  apt:
-    update: true
-    sources:
-    - sourceline: deb [arch=amd64,armhf,arm64] http://pkg.bluehorizon.network/linux/ubuntu bionic-updates main
-      key_url: 'http://pkg.bluehorizon.network/bluehorizon.network-public.key'
-    packages:
-    - horizon-cli
+install:
+  - wget https://github.com/open-horizon/anax/releases/latest/download/horizon-agent-linux-deb-amd64.tar.gz -O /tmp/horizon-agent-linux-deb-amd64.tar.gz
+  - tar -zxf /tmp/horizon-agent-linux-deb-amd64.tar.gz -C /tmp/
+  - sudo apt-get install -y /tmp/horizon-cli*.deb
 
 # Build and test the example services that were changed in the PR for all arches 
 # Note: tools/travis-find determines if the specified "service" was modified, so we know if we should test it.

diff --git a/README.md b/README.md
@@ -1,13 +1,70 @@
-# examples
-Code examples you can use with Horizon.
+![open-horizon-logo](image/open-horizon-color.png)
 
-Generic edge service examples are found, perhaps unsurpringly, in:
- - edge/services
+# Getting Started
 
-Edge examples specific to the Watson IoT Platform are found in:
- - edge/wiotp
+## Documentation
 
-In general to run the code from one of these examples:
- - cd into the directory
- - run *make*
+**Open Horizon documentation repository coming soon!** For the time being, you can learn more about [Open Horizon here](https://www.ibm.com/support/knowledgecenter/SSFKVV_4.2/kc_welcome_containers.html).
 
+## Management Hub Installation
+
+Before you can publish and use any of the services in this repository, you must first deploy your own Horizon Management Hub. This can be done with one simple command using the `deploy-mgmt-hub.sh` script located in the [devops repository](https://github.com/open-horizon/devops/tree/master/mgmt-hub#horizon-management-hub). This will give you a management hub with several services, policies and patterns published in the exchange.
+
+## Register an Edge Node with your Mangement Hub
+
+In order to deploy a service to an edge node it must first be registered with a management hub. The `agent-install.sh` script is a fast and easy way to register an edge node with a management hub. More information can be found in the [open-horizon/anax](https://github.com/open-horizon/anax/tree/master/agent-install#edge-node-agent-install) repository. Edge nodes can be either a device or a cluster. Open Horizon edge cluster capability helps you manage and deploy workloads from a management hub cluster to remote instances of OpenShift® Container Platform or other Kubernetes-based clusters.
+
+Typically, **edge devices** have a prescriptive purpose, provide (often limited) compute capabilities, and are located near or at the data source. Currently supported edge device OS and architectures:
+
+* x86_64
+  * Linux x86_64 devices or virtual machines that run Ubuntu 20.x (focal), Ubuntu 18.x (bionic), Debian 10 (buster), Debian 9 (stretch)
+  * Red Hat Enterprise Linux® 8.x and 9.x
+  * Fedora Workstation 32 - 37
+  * CentOS 8.x
+  * SuSE 15 SP2
+* ppc64le (support starting Horizon version 2.28)
+  * Red Hat Enterprise Linux® 7.9
+* ARM (32-bit)
+  * Linux on ARM (32-bit), for example Raspberry Pi, running Raspberry Pi OS buster or stretch
+* ARM (64-bit)
+  * Linux on ARM (64-bit), for example NVIDIA Jetson Nano, TX1, or TX2, running Ubuntu 18.x (bionic) or Raspberry Pi OS 64bit
+* Mac
+  * macOS (Intel)
+
+Open Horizon **edge cluster** capability helps you manage and deploy workloads from a management hub cluster to remote instances of OpenShift® Container Platform or other Kubernetes-based clusters. Edge clusters are edge nodes that are Kubernetes clusters. An edge cluster enables use cases at the edge, which require colocation of compute with business operations, or that require more scalability, availability, and compute capability than what can be supported by an edge device. Further, it is not uncommon for edge clusters to provide application services that are needed to support services running on edge devices due to their close proximity to edge devices. Open Horizon deploys edge services to an edge cluster, via a Kubernetes operator, enabling the same autonomous deployment mechanisms used with edge devices. The full power of Kubernetes as a container management platform is available for edge services that are deployed by Open Horizon. Currently supported edge cluster architectures:
+
+* [OCP on x86_64 platforms](https://docs.openshift.com/container-platform/4.5/welcome/index.html)
+* [K3s - Lightweight Kubernetes](https://rancher.com/docs/k3s/latest/en/)
+* [MicroK8s](https://microk8s.io/docs) on Ubuntu 18.04 (for development and test, not recommended for production)
+
+Currently there is only one example service in this repository that is designed to run on an edge cluster and that is the [nginx-operator](edge/services/nginx-operator).
+
+# Example Services
+
+During the management hub installation, several services should have been published into the exchange automatically. The following three command will list the services, patterns, and deployment policies available in your exchange:
+
+```sh
+hzn exchange service list IBM/
+hzn exchange pattern list IBM/
+hzn exchange deployment listpolicy
+```
+
+**Note:** The above commands assume you have the Horizon environment variables `HZN_ORG_ID` and `HZN_EXCHANGE_USER_AUTH` set.
+
+You can find a list of available edge services in this repository located in the [edge/services](edge/services) directory. For the most part, each of the services are broken up into micro-services designed to accomplish one specific task. This makes them easier to incorporate into a wide variety of "top-level" services.
+
+A good example of a "top-level" service is [cpu2evtstreams](edge/evtstreams/cpu2evtstreams), which has two dependent services ([cpu_percent](edge/services/cpu_percent), and [gps](edge/services/gps)). It uses these two micro-services to gather information about the edge node it is running on and sends it to an instance of IBM Event Streams using `kafkacat`.
+
+Edge examples specific to the Watson IoT Platform are found in [edge/wiotp](edge/wiotp). These examples are not being maintained.
+
+The backend infrastructure for the `sdr2evtstreams` service has been deprecated and removed from this repo. If you wish to access the most recent version, you can find it in the [deprecated_cloud_sdr_example](https://github.com/open-horizon/examples/tree/deprecated_cloud_sdr_example) branch or any open-horizon/examples release v2.30.0-934 or older.
+
+# Using Example Services
+
+Each example service in this repo has a [README](edge/services/helloworld/README.md#horizon-hello-world-example-edge-service) that includes steps to run it when it is currently published in your exchange, or a ["Create your own"](edge/services/helloworld/CreateService.md#creating-your-own-hello-world-edge-service) set of instructions that will guide you through the process of publishing your own version to your exchange.
+
+Contributors
+
+<a href="https://github.com/open-horizon/examples/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=open-horizon/examples" />
+</a>
diff --git a/cloud/sdr/.gitignore b/cloud/sdr/.gitignore
diff --git a/cloud/sdr/data-ingest/example-go-clients/.gitignore b/cloud/sdr/data-ingest/example-go-clients/.gitignore
diff --git a/cloud/sdr/data-ingest/example-go-clients/Makefile b/cloud/sdr/data-ingest/example-go-clients/Makefile
diff --git a/cloud/sdr/data-ingest/example-go-clients/README.md b/cloud/sdr/data-ingest/example-go-clients/README.md