Elasticsearch index upgrade - how to?

Hello,
I have upgraded my 1-node small-deployment of Connections 6.5 CR1 => 7.
In addition I am running a standalone Elasticsearch 5.5.x node.
Based on system requirements I would need Elasticsearch 7.6 - but since only Kubernetes-deployment is covered in documentation (and upgrade whitepaper) it seems like I need to find my own way here....& obviously require your assistance. My actions, plan & thoughts so far:

[done] 1. upgrade ES from 5.5 => 5.6 (including mapper & searchguard, without indices)
[done] 2. upgrade ES from 5.6 => 6.8 (including mapper & searchguard, without indices, 5.x still supported)
[not yet done] 3. upgrade indices from 5.x to 6.x to be ready for "upgrade ES 6.8 => 7.6"
I did find "Reindex inplace" but I have no idea how this would look like in reality with my Quickresult & Metrics index (and I assume also the interal/security-indices). Elastic documentation also mentions X-Pack Reindex Helper and Kibana-related tools.
[not yet done] 4. upgrade ES 6.8 => 7.6 (including mapper & searchguard)

a) Is the 4-step approach described above correct ?
b) How to propally upgrade the index ? Does anybody have a reallife example for me ?
Reading through the elastic documentation I still do not have the right plan how to update all indices (internal indices, quickresults,metrics)

Thanks in advance !

Kind regards,
Jan

Hello,

Is there a reason why you would be using a stand-alone Elasticsearch?

Installing Connections 7 Component pack would include Elasticsearch.

Component Pack offerings (hcltechsw.com)

Starting with Component Pack 7, Elasticsearch is the default and only backend for Metrics, Search, and Orient Me.

You may want to review the installation from the documentation:

Sample steps to install or upgrade to Component Pack 7 (hcltechsw.com)

regards,

Ariel

Hi,

why would somebody not run a simple ES instance instead of running a k8s envoirnment ?
It is just beyond scope & requirements. ES standalone is exactly what most customers need - low cost, low failure rates.

Question is still the same: How would I migrate from ES5 to K8s ES7 ?

Regards,
Jan

I managed to update the envoirnment as described in my initial post.
Metrics is working fine now. But Quickresults do not work. The ES error that pops up as soon as Connections trys to generate data:

Rejecting mapping update to [quickresults] as the final mapping would have more than 1 type: [_doc, history]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:272) ~[elasticsearch-7.6.1.jar:7.6.1]

I have found many important notes in the Elasticsearch documentation regarding deprecation of mapping types.
In general it seems like "_doc" is now default & only type. For Metrics this does not cause any problems.

But Quickresults-integration tries to POST with type "history" thus causing "would have more than 1 type: [_doc, history]" I assume.
Based on ES documentation you could still set ""index.mapping.single_type": false" in ES7 to re-allow multiple mapping types.
But I do not know how to apply this setting as it needs to be set during index creation which is actually done by running wsadmin-script "SearchService.createES7QuickResultsIndex()" as far as I understand the problem.

I have asked HCL support on how to change the mapping or re-allow multiple types.


Regards,
Jan

Answer HCL: ElasticSearch 7.6.1 Standalone is not supported for Connections 7.
Before v7 you should move from Standalone ElasticSearch 5.5.1 => Kubernetes Component Pack v6.5 with ElasticSearch 5.5.1. Afterwards update to v7.