Elasticsearch Metrics - Returning Bad requests error 400

Hi,

I have setup a new Connections 7 environment (medium deployment) with Elasticsearch for Metrics.

Data capturing in Metrics is not working. In AppsCluster_server1, I see the following errors:

com.ibm.connections.metrics.elasticsearch.ElasticsearchClient httpRequest ElasticSearch returned - 400 : {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Types cannot be provided in put mapping requests, unless the include_type_name parameter is set to true."}],"type":"illegal_argument_exception","reason":"Types cannot be provided in put mapping requests, unless the include_type_name parameter is set to true."},"status":400}

com.ibm.connections.metrics.elasticsearch.ElasticsearchClient httpRequest ElasticSearch returned - 400 : {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"}],"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"},"status":400}

From Elasticsearch documentation, it looks like that the index is still using / created with something from Elasticsearch v6, which is no longer supported in v7: https://www.elastic.co/guide/en/elasticsearch/reference/7.x/removal-of-types.html

Connections Highway settings include two

c2.export.elasticsearch.baseurl: https://connections-elasticsearch.mycomp.com:9200

c2.export.elasticsearch.baseurl7: https://connections-elasticsearch.mycomp.com:9200

In this environment, Elasticsearch is configured to be accessible on port 9200, not 30098 as in documentation

In Connections itself, I used these wsadmin commands to configure Elasticsearch for Metrics:

> execfile('esSecurityAdmin.py')
> enableSslForMetrics('/opt/HCL/Elasticsearch/elasticsearch-metrics.p12', 'password', '/opt/HCL/Elasticsearch/chain-ca.pem', '9200')
> execfile('metricsEventCapture.py')
> switchMetricsToElasticSearch()

Any similar experiences?

Greetings

Maik

Hi.

for me it looks like Connections is not talking ES7 with the server. Can you review the property '<genericProperty name="elasticsearch.eSmajorVersion">7</genericProperty>' in the LC-config.xml file.

Hello Maik,
Hope you are doing great!

Could you please check the below setting inLotusConnections-config.xml: -

<genericProperty name="elasticsearch.eSmajorVersion">5</genericProperty>


Please confirm if it has "5" ?


Thank you!
Regards,
Pratyush

Hi, just checked LotusConnections-config.xml and it is set to

<genericProperty name="elasticsearch.eSmajorVersion">5</genericProperty>

Should it be 5 or 7 ?

Hello Maik,
Hope you are doing great!



It should be 7 . Like below-

<genericProperty name="elasticsearch.eSmajorVersion">7</genericProperty>

Thank you!
Regards,
Pratyush


The parameter should match your ES version.

According to the error message you have ES 7, so the parameter should be 7.

Martin

I see ... is this expected to be changed manually or should it be already covered by some scripts? Because I do not find anything in the documentation on this.

Hi,

I always change it manually.

According to the documentation and quality ..... It has not improved since IBM. (My personal opinion.).

Martin

Hello Maik,
Hope you are doing good!

The Doc Update request has been created and its being worked upon.

Thank you!

Regards,
Pratyush