Ah is that the case, good to know. Anyway balanced also seems to work. At least I'm getting an answer and I can see at the ollama "api/ps" endpoint that the embedding model was loaded, but not sure how to check if it was actually used?

But I've also tested a bit more with documents now and it was not as "simple" as I though. In fact both huggingface and local models fail sometimes just with a bit different frequencies, so even using huggingface does not always work. I tried with 4 different pdf files and the success rate was as follows:

• Huggingface/BGE-small 6/9 worked
• Ollama/qwen-embedding-8B 3/9
• Ollama/bge-large 4/8

In all tests I just add the paper and when it is imported I ask: "summarize the attached paper". I added qwen to the mix to see if context size mattered, but that does not seem to be matter. Haven't detected a pattern here, the same document seems to work or fail randomly.

BTW if it matters the local setup uses two servers, ollama for embedding and lemonade for inference, sort of a workaround to avoid reloading models since neither server seems to support multiple simultaneous models.