Merge branch 'main' into spacy-3-dot-0-support

Dazu-io · Mar 4, 2021 · 3e9815a · 3e9815a
2 parents 29d1a98 + 776406c
commit 3e9815a
Show file tree

Hide file tree

Showing 167 changed files with 1,877 additions and 1,350 deletions.
diff --git a/.github/workflows/continous-integration.yml b/.github/workflows/continous-integration.yml
@@ -194,6 +194,7 @@ jobs:
       matrix:
         os: [ubuntu-latest, windows-latest]
         python-version: [3.6, 3.7, 3.8]
+        test: [test-non-training, test-training]
 
     steps:
     - name: Checkout git repository 🕝
@@ -277,7 +278,7 @@ jobs:
         JOBS: 2
         PYTHONIOENCODING: "utf-8"
       run: |
-        make test
+        make ${{ matrix.test }}
 
     - name: Send Coverage Report 📊
       if: needs.changes.outputs.backend == 'true' && matrix.python-version == 3.6 && matrix.os != 'windows-latest'

diff --git a/CHANGELOG.mdx b/CHANGELOG.mdx
@@ -16,6 +16,41 @@ https://github.com/RasaHQ/rasa/tree/main/changelog/ . -->
 
 <!-- TOWNCRIER -->
 
+## [2.3.4] - 2021-02-26
+
+
+### Bugfixes
+- [#8014](https://github.com/rasahq/rasa/issues/8014): Setting `model_confidence=cosine` in `DIETClassifier`, `ResponseSelector` and `TEDPolicy` is deprecated and will no longer be available. This was introduced in Rasa Open Source version `2.3.0` but post-release experiments suggest that using cosine similarity as model's confidences can change the ranking of predicted labels which is wrong.
+
+  `model_confidence=inner` is deprecated and is replaced by `model_confidence=linear_norm` as the former produced an unbounded range of confidences which broke the logic of assistants in various other places.
+
+  We encourage you to try `model_confidence=linear_norm` which will produce a linearly normalized version of dot product similarities with each value in the range `[0,1]`. This can be done with the following config:
+  ```yaml
+  - name: DIETClassifier
+    model_confidence: linear_norm
+    constrain_similarities: True
+  ```
+  This should ease up [tuning fallback thresholds](./fallback-handoff.mdx#fallbacks) as confidences for wrong predictions are better distributed across the range `[0, 1]`.
+
+  If you trained a model with `model_confidence=cosine` or `model_confidence=inner` setting using previous versions of Rasa Open Source, please re-train by either removing the `model_confidence` option from the configuration or setting it to `linear_norm`.
+
+  `model_confidence=cosine` is removed from the configuration generated by [auto-configuration](model-configuration.mdx#suggested-config).
+
+
+## [2.3.3] - 2021-02-25
+
+
+### Bugfixes
+- [#8001](https://github.com/rasahq/rasa/issues/8001): Fixed bug where the conversation does not lock before handling a reminder event.
+
+
+## [2.3.2] - 2021-02-22
+
+
+### Bugfixes
+- [#7972](https://github.com/rasahq/rasa/issues/7972): Fix a bug where, if a user injects an intent using the HTTP API, slot auto-filling is not performed on the entities provided.
+
+
 ## [2.3.1] - 2021-02-17
 
 
@@ -104,6 +139,9 @@ https://github.com/RasaHQ/rasa/tree/main/changelog/ . -->
   Configuration option `loss_type=softmax` is now deprecated and will be removed in Rasa Open Source 3.0.0 . Use `loss_type=cross_entropy` instead.
 
   The default [auto-configuration](model-configuration.mdx#suggested-config) is changed to use `constrain_similarities=True` and `model_confidence=cosine` in ML components so that new users start with the recommended configuration.
+
+  **EDIT**: Some post-release experiments revealed that using `model_confidence=cosine` is wrong as it can change the order of predicted labels. That's why this option was removed in Rasa Open Source version `2.3.3`. `model_confidence=inner` is deprecated as it produces an unbounded range of confidences which can break the logic of assistants in various other places. Please use `model_confidence=linear_norm` which will produce a linearly normalized version of dot product similarities with each value in the range `[0,1]`. Please read more about this change under the notes for release `2.3.4`.
+
 - [#7817](https://github.com/rasahq/rasa/issues/7817): Use simple random uniform distribution of integers in negative sampling, because
   negative sampling with `tf.while_loop` and random shuffle inside creates a memory leak.
 - [#7848](https://github.com/rasahq/rasa/issues/7848): Added support to configure `exchange_name` for [pika event broker](event-brokers.mdx#pika-event-broker).
@@ -1302,6 +1340,12 @@ https://github.com/RasaHQ/rasa/tree/main/changelog/ . -->
 - [#5784](https://github.com/rasahq/rasa/issues/5784), [#5788](https://github.com/rasahq/rasa/issues/5788), [#6199](https://github.com/rasahq/rasa/issues/6199), [#6403](https://github.com/rasahq/rasa/issues/6403), [#6735](https://github.com/rasahq/rasa/issues/6735)
 
 
+## [1.10.23] - 2021-02-22
+
+### Bugfixes
+- [#7895](https://github.com/rasahq/rasa/issues/7895): Fixed bug where the conversation does not lock before handling a reminder event.
+
+
 ## [1.10.22] - 2021-02-05
 
 ### Bugfixes

diff --git a/Makefile b/Makefile
@@ -180,6 +180,14 @@ else
 	set -o allexport; source tests_deployment/.env && OMP_NUM_THREADS=1 poetry run pytest $(INTEGRATION_TEST_FOLDER) -n $(JOBS) -m $(INTEGRATION_TEST_PYTEST_MARKERS) && set +o allexport
 endif
 
+test-non-training: clean
+	# OMP_NUM_THREADS can improve overall performance using one thread by process (on tensorflow), avoiding overload
+	RAISE_ON_TRAIN=True OMP_NUM_THREADS=1 poetry run pytest tests -n $(JOBS) --cov rasa -m "not trains_model" --ignore $(INTEGRATION_TEST_FOLDER)
+
+test-training: clean
+	# OMP_NUM_THREADS can improve overall performance using one thread by process (on tensorflow), avoiding overload
+	OMP_NUM_THREADS=1 poetry run pytest tests -n $(JOBS) --cov rasa -m "trains_model" --ignore $(INTEGRATION_TEST_FOLDER)
+
 generate-pending-changelog:
 	poetry run python -c "from scripts import release; release.generate_changelog('major.minor.patch')"
 

diff --git a/README.md b/README.md
@@ -185,6 +185,21 @@ JOBS=[n] make test
 
 Where `[n]` is the number of jobs desired. If omitted, `[n]` will be automatically chosen by pytest.
 
+#### Tests that train
+A test that trains a model is defined as any test that explicitly or inadvertently calls any method annotated with `@rasa.shared.utils.common.raise_on_unexpected_train`.
+Currently, this is: `nlu.train`, `core.train`, `Agent.train`, and `Trainer.train`.
+
+We specify tests that train a model using the pytest mark `trains_model`.
+e.g:
+
+    @pytest.mark.trains_model
+    def test_some_training()
+        ...
+
+These test are then run separately in the CI using the make commands `make test-non-training` and `make test-training` respectively.
+
+The command `make test-non-training` will fail if training occurs.
+
 
 ### Running the Integration Tests
 

diff --git a/changelog/7113.bugfix.md b/changelog/7113.bugfix.md
@@ -0,0 +1,3 @@
+Fix for the cli command `rasa data convert config` when migrating Mapping Policy and no rules.
+
+Making `rasa data convert config` migrate correctly the Mapping Policy when no rules are available. It updates the `config.yml` file by removing the `MappingPolicy` and adding the `RulePolicy` instead. Also, it creates the `data/rules.yml` file even if empty in the case of no available rules.
diff --git a/changelog/7737.misc.md b/changelog/7737.misc.md
@@ -0,0 +1 @@
+Replace Markdown training data in tests with YAML training data.
diff --git a/changelog/7952.improvement.md b/changelog/7952.improvement.md
@@ -0,0 +1,2 @@
+Speed up `YAMLStoryReader.is_key_in_yaml` function by making it to check if key is in YAML without 
+actually parsing the text file.
diff --git a/changelog/7953.improvement.md b/changelog/7953.improvement.md
@@ -0,0 +1,2 @@
+Speed up YAML parsing by reusing parsers, making the process of environment variable interpolation optional,
+and by not adding duplicating implicit resolvers and YAML constructors to `ruamel.yaml`
diff --git a/changelog/8080.docs.md b/changelog/8080.docs.md
@@ -0,0 +1 @@
+Update links to Sanic docs in the documentation.
diff --git a/data/examples/rasa/demo-rasa-multi-intent.md b/data/examples/rasa/demo-rasa-multi-intent.md
diff --git a/data/examples/rasa/demo-rasa-multi-intent.yml b/data/examples/rasa/demo-rasa-multi-intent.yml
@@ -0,0 +1,75 @@
+version: "2.0"
+nlu:
+- intent: affirm
+  examples: |
+    - yes
+    - yep
+    - yeah
+    - indeed
+    - that's right
+    - ok
+    - great
+    - right, thank you
+    - correct
+    - great choice
+    - sounds really good
+- intent: goodbye
+  examples: |
+    - bye
+    - goodbye
+    - good bye
+    - stop
+    - end
+    - farewell
+    - Bye bye
+    - have a good one
+- intent: greet
+  examples: |
+    - hey
+    - howdy
+    - hey there
+    - hello
+    - hi
+    - good morning
+    - good evening
+    - dear sir
+- intent: chitchat+ask_name
+  examples: |
+    - What's your name?
+    - What can I call you?
+- intent: chitchat+ask_weather
+  examples: |
+    - How's the weather?
+    - Is it too hot outside?
+- intent: restaurant_search
+  examples: |
+    - i'm looking for a place to eat
+    - I want to grab lunch
+    - I am searching for a dinner spot
+    - i'm looking for a place in the [north](location) of town
+    - show me [chinese](cuisine) restaurants
+    - show me [chines]{"entity": "cuisine", "value": "chinese"} restaurants in the [north](location)
+    - show me a [mexican](cuisine) place in the [centre](location)
+    - i am looking for an [indian](cuisine) spot called olaolaolaolaolaola
+    - search for restaurants
+    - anywhere in the [west](location)
+    - anywhere near [18328](location)
+    - I am looking for [asian fusion](cuisine) food
+    - I am looking a restaurant in [29432](location)
+    - I am looking for [mexican indian fusion](cuisine)
+    - [central](location) [indian](cuisine) restaurant
+- synonym: chinese
+  examples: |
+    - chines
+    - Chines
+    - Chinese
+- synonym: vegetarian
+  examples: |
+    - vegg
+    - veggie
+- regex: greet
+  examples: |
+    - hey[^\s]*
+- regex: zipcode
+  examples: |
+    - [0-9]{5}
diff --git a/data/examples/rasa/demo-rasa-responses.md b/data/examples/rasa/demo-rasa-responses.md
diff --git a/data/examples/rasa/demo-rasa-responses.yml b/data/examples/rasa/demo-rasa-responses.yml
@@ -0,0 +1,6 @@
+responses:
+  utter_chitchat/ask_weather:
+    - text: It's sunny where I live
+
+  utter_chitchat/ask_name:
+    - text: I am Mr. Bot
diff --git a/data/examples/rasa/demo-rasa.md b/data/examples/rasa/demo-rasa.md
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		Fix for the cli command `rasa data convert config` when migrating Mapping Policy and no rules.

		Making `rasa data convert config` migrate correctly the Mapping Policy when no rules are available. It updates the `config.yml` file by removing the `MappingPolicy` and adding the `RulePolicy` instead. Also, it creates the `data/rules.yml` file even if empty in the case of no available rules.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Replace Markdown training data in tests with YAML training data.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		Speed up `YAMLStoryReader.is_key_in_yaml` function by making it to check if key is in YAML without
		actually parsing the text file.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		Speed up YAML parsing by reusing parsers, making the process of environment variable interpolation optional,
		and by not adding duplicating implicit resolvers and YAML constructors to `ruamel.yaml`
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Update links to Sanic docs in the documentation.