Elasticsearch version: 5.1.1 - 5.4.1

Plugins installed: [analysis-icu, elasticsearch-position-similarity]

JVM version (java -version):
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (IcedTea 3.4.0) (suse-10.10.3-x86_64)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

OS version (uname -a if on a Unix-like system):
Linux oviken 4.4.36-8-default #1 SMP Fri Dec 9 16:18:38 UTC 2016 (3ec5648) x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:
Upgrading an index using a custom similarity fails in org.elasticsearch.cluster.metadata.MetaDataIndexUpgradeService.checkMappingsCompatibility with the exception "Unknown Similarity type". If the old and the new elasticserver have the same custom similarity installed, upgrading should work.

Looking at the code, it is no surprise that this does not work, as checkMappingsCompatibility creates a SimilarityService with an empty list of additional similarities. It seems something akin to the fake analyzerMap is necessary for custom similarities?

This is a follow up to https://discuss.elastic.co/t/upgrading-index-with-custom-similarity-not-working/76678 as by now I'm quite sure this is a bug with elasticsearch an not with my similarity plugin.

Steps to reproduce:

1. Start elasticsearch version 5.1.1 with https://github.com/sdauletau/elasticsearch-position-similarity installed (I've attached the exact zip I build and used)
   elasticsearch-position-similarity-5.1.1.zip

2. Create an index using this similarity:

curl -s -XPUT "http://localhost:9200/test_index" -d '
{
  "settings": {
    "similarity": {
      "positionSimilarity": {
        "type": "position-similarity"
      }
    }
  }
}'

3. Stop elasticsearch, upgrade to 5.1.2, install the same plugin for this version (I again attached the zip I used)
   elasticsearch-position-similarity-5.1.2.zip

4. Start elasticsearch again

Provide logs (if relevant):

elasticsearch.txt

https://gist.github.com/xabbu42/66dd46bd3cc15244cede8ceadf884fd8