Please complete the details below then remove this line:
Domino/Notes Version: Domino 14.5
Add-on Product (if appropriate, e.g. Verse / Traveler / Nomad / Domino REST API): Domino IQ
Its Version: The latest Domino IQ - 7 Aug 2025 release
Operating System: Linux
Client (Notes, Nomad Web, Nomad Mobile, Android/iOS, browser version): Notes Client
Problem/Query:
Recently we did test Domino IQ with local LLM(Llama 3.2 1B) especially regarding the summarizing email contents comparing it to Ollama with the same LLM. And we found the quality of result from Domino IQ is significantly lower than the quality from Ollama. You can see the below image with test scenario and summary of my test results.
We conducted tests using not only the single email shown above but also several other Korean emails. The results consistently showed that the values obtained through Domino IQ were significantly lower in quality.
Why does it produce such different quality results?
How can we improve the quality of summary results for Korean email content in Domino IQ perspective?
======================================================
I would like to share the more detailed information and test scenario as like below.
Domino IQ TEST Result"
- In the notes client, they request to summarize this email(Korean Contents) with the default prompt in Domino IQ configuration.
- Domino IQ shows the result like this. The quality is so bad.
Direct request to Ollama:
- They prepared the web page including mail body contents, prompt.
- They input the same mail body contents(Korean contents)and also input the same prompt which was provided in the Domino IQ configuration database.
- And then they request to summarize this email to Ollama directly
- The result is like this. It looks good. This is better than the Domino IQ result.
