IAM Service not starting on Linux

HCL Domino Domino Forum
,
1

I have followed the instructions in https://doc.cwpcollaboration.com/appdevpack/docs/en/setup-guide-iamsrv1.html and when running the code to test the connection:

echo x | openssl s_client -connect domino11.life-etg.com:3003 -cert config/certs/iamapp.crt
 -key config/certs/iamapp.key -CAfile config/certs/ca/ca.crt -quiet > nul

And enter the password, it works correctly:

depth=1 O = KelrosCA, CN = KelrosCA
verify return:1
depth=0 O = Kelros, CN = Domino11
verify return:1
read:errno=0

However, when completing the iam-service.nsf setup, when issuing npm start,
the IAM Service fails to load, giving:

[16:15:10][error][DBConnector][worker-1]: Failed to obtain Domino database instance: Entry not found in index.
[16:15:10][error][iamStorage][worker-1]: Error connecting to DB. Error occurs: Entry not found in index
[16:15:10][error][initServices][worker-1]: DBError occur, please check your DB settings.
[16:15:10][error][initServices][worker-1]: Exiting.. Error: Entry not found in index
[16:15:10][info][initServices][worker-1]: IAM service is shutdown
[16:15:10][info][cluster][master]: Worker 1 exits with code: 1
[16:15:11][error][DBConnector][worker-2]: Failed to obtain Domino database instance: Entry not found in index.
[16:15:11][error][iamStorage][worker-2]: Error connecting to DB. Error occurs: Entry not found in index
[16:15:11][error][initServices][worker-2]: DBError occur, please check your DB settings.
[16:15:11][error][initServices][worker-2]: Exiting.. Error: Entry not found in index

Turing on the debug option does not give any meaningful clues as to what is wrong.

The iam-service.nsf is there, i can access it via HTTPS.

I'm really starting to hate the AppDev Pack.
2

Entry not found in index is coming from Domino.

Can you turn on Proton tracing and look for any errors like that on the console?

With tracing on we should be able to see what it's doing.

Also, do you have a support case for this problem? I've seen some traffic from our support team with your name.

3

Dan,

Support case is CS0235392. Mary should be having a remote session with me at 3pm UK time today.

4

I had a post disappear on me... I was going to say

PROTON_TRACE_ALL=1

and then restart the proton task.

------------------------------

But before you do that you might want to make double sure that the iam-service.nsf was signed by the server (so the agents can run) and run

run updall -e iam-service.nsf

to make sure that the DQL index is up to date.

We have a work item in the backlog for proton to do the signing automatically, but it has not been addressed yet.

5

Dan

Turned on PROTON_TRACE_ALL and restarted the IAM Service. Error from IAM the same, Protn dumped this on the Domino Console.

03/06/2021 10:21:04 PROTON: Session::init_via_server_context: enter (psession.cc:61)
03/06/2021 10:21:04 PROTON: Session::init_via_server_context2: enter (psession.cc:77)
03/06/2021 10:21:04 PROTON: Certificate::init: enter (pcert.cc:222)
03/06/2021 10:21:04 PROTON: Certificate::extract_b64_cert: enter (pcert.cc:137)
03/06/2021 10:21:04 PROTON: ReturnCode = 65536
03/06/2021 10:21:04 PROTON: ReturnCode = 0 (was 65536)
03/06/2021 10:21:04 PROTON: Certificate::extract_b64_cert: exit
03/06/2021 10:21:04 PROTON: Certificate::init: subject: CN=iamapp/O=KelrosCA (pcert.cc:270)
03/06/2021 10:21:04 PROTON: Certificate::init: exit
03/06/2021 10:21:04 PROTON: SessionCache::find_session_entry: enter (pcache.cc:189)
03/06/2021 10:21:04 PROTON: SessionCache::find_session_entry: certificate does not exist in cache: 9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01 (pcache.cc:243)
03/06/2021 10:21:04 PROTON: SessionCache::find_session_entry: exit
03/06/2021 10:21:04 PROTON: Certificate::lookup: enter (pcert.cc:53)
03/06/2021 10:21:04 PROTON: ReturnCode = 65553
03/06/2021 10:21:04 PROTON: ReturnCode = 0 (was 65553)
03/06/2021 10:21:04 PROTON: Session::init_via_server_context: enter (psession.cc:61)

03/06/2021 10:21:04 PROTON: Session::init_via_server_context2: enter (psession.cc:77)
03/06/2021 10:21:04 PROTON: Certificate::init: enter (pcert.cc:222)
03/06/2021 10:21:04 PROTON: Certificate::extract_b64_cert: enter (pcert.cc:137)
03/06/2021 10:21:04 PROTON: ReturnCode = 65536
03/06/2021 10:21:04 PROTON: ReturnCode = 0 (was 65536)
03/06/2021 10:21:04 PROTON: Certificate::extract_b64_cert: exit
03/06/2021 10:21:04 PROTON: Certificate::init: subject: CN=iamapp/O=KelrosCA (pcert.cc:270)
03/06/2021 10:21:04 PROTON: Certificate::init: exit
03/06/2021 10:21:04 PROTON: SessionCache::find_session_entry: enter (pcache.cc:189)
03/06/2021 10:21:04 PROTON: SessionCache::find_session_entry: certificate does not exist in cache: 9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01 (pcache.cc:243)
03/06/2021 10:21:04 PROTON: SessionCache::find_session_entry: exit
03/06/2021 10:21:04 PROTON: Certificate::lookup: enter (pcert.cc:53)
03/06/2021 10:21:04 PROTON: ReturnCode = 65553
03/06/2021 10:21:04 PROTON: ReturnCode = 0 (was 65553)
03/06/2021 10:21:04 PROTON: Certificate::lookup: lookup found: CN=iam application/O=Kelros (pcert.cc:119)
03/06/2021 10:21:04 PROTON: Certificate::lookup: exit
03/06/2021 10:21:04 PROTON: Certificate::lookup: lookup found: CN=iam application/O=Kelros (pcert.cc:119)
03/06/2021 10:21:04 PROTON: Certificate::lookup: exit
03/06/2021 10:21:04 PROTON: SessionCache::save_session_entry: enter (pcache.cc:254)
03/06/2021 10:21:04 PROTON: SessionCacheEntry::SessionCacheEntry: enter (pcache.cc:388)
03/06/2021 10:21:04 PROTON: SessionCacheEntry::SessionCacheEntry: {9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01} @0x7f3374017ec8 valid_for_secs: 300 (pcache.cc:392)
03/06/2021 10:21:04 PROTON: SessionCacheEntry::SessionCacheEntry: exit
03/06/2021 10:21:04 PROTON: SessionCache::save_session_entry: certificate digest does not exist, adding: 9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01 (pcache.cc:324)
03/06/2021 10:21:04 PROTON: SessionCache::save_session_entry: exit
03/06/2021 10:21:04 PROTON: RequestHistory::is_application_jailed: enter (pcache.cc:1018)
03/06/2021 10:21:04 PROTON: RequestHistory::is_application_jailed: application not in jail: CN=iam application/O=Kelros (pcache.cc:1064)
03/06/2021 10:21:04 PROTON: RequestHistory::is_application_jailed: exit
03/06/2021 10:21:04 PROTON: Session::get_password_from_request: enter (psession.cc:479)
03/06/2021 10:21:04 PROTON: Session::get_password_from_request: exit
03/06/2021 10:21:04 PROTON: Session::init_hkey_from_cache_or_vault: enter (psession.cc:542)
03/06/2021 10:21:04 PROTON: SessionCache::get_idfile: enter (pcache.cc:537)
03/06/2021 10:21:04 PROTON: SessionCache::get_idfile: get_idfile @0x7f3374017ec8 rc=65564 req_timeout_ms=0 elapsed_time_ms=0 (pcache.cc:615)
03/06/2021 10:21:04 PROTON: SessionCache::get_idfile: exit
03/06/2021 10:21:04 PROTON: ReturnCode = 65564 (was 0)
03/06/2021 10:21:04 PROTON: ReturnCode = 0 (was 65564)
03/06/2021 10:21:04 PROTON: Session::init_hkey_from_vault: enter (psession.cc:389)
03/06/2021 10:21:05 PROTON: SessionCache::save_session_entry: enter (pcache.cc:254)
03/06/2021 10:21:05 PROTON: SessionCacheEntry::SessionCacheEntry: enter (pcache.cc:388)
03/06/2021 10:21:05 PROTON: SessionCacheEntry::SessionCacheEntry: {9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01} @0x7f337c01b128 valid_for_secs: 300 (pcache.cc:392)
03/06/2021 10:21:05 PROTON: SessionCacheEntry::SessionCacheEntry: exit
03/06/2021 10:21:05 PROTON: SessionCache::save_session_entry: certificate digest already exists: 9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01 (pcache.cc:324)
03/06/2021 10:21:05 PROTON: SessionCacheEntry::~SessionCacheEntry: enter (pcache.cc:403)
03/06/2021 10:21:05 PROTON: SessionCacheEntry::~SessionCacheEntry: del {9DC26F9974045DFEB1E20C7FE646F51FF8D571DB01} @0x7f337c01b128 valid_for_secs: 300 (pcache.cc:404)
03/06/2021 10:21:05 PROTON: SessionCacheEntry::~SessionCacheEntry: exit
03/06/2021 10:21:05 PROTON: SessionCache::save_session_entry: exit
03/06/2021 10:21:05 PROTON: RequestHistory::is_application_jailed: enter (pcache.cc:1018)
03/06/2021 10:21:05 PROTON: RequestHistory::is_application_jailed: application not in jail: CN=iam application/O=Kelros (pcache.cc:1064)
03/06/2021 10:21:05 PROTON: RequestHistory::is_application_jailed: exit
03/06/2021 10:21:05 PROTON: Session::get_password_from_request: enter (psession.cc:479)
03/06/2021 10:21:05 PROTON: Session::get_password_from_request: exit
03/06/2021 10:21:05 PROTON: Session::init_hkey_from_cache_or_vault: enter (psession.cc:542)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: enter (pcache.cc:537)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: get_idfile @0x7f3374017ec8 rc=65564 req_timeout_ms=0 elapsed_time_ms=0 (pcache.cc:615)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 65564 (was 0)
03/06/2021 10:21:05 PROTON: ReturnCode = 0 (was 65564)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: enter (pcache.cc:537)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: wait_for_id_file timeout 10000ms @0x7f3374017ec8 (pcache.cc:569)
03/06/2021 10:21:05 Server Domino11/Kelros reported the following problem causing authentication to fail: Entry not found in index
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: Session::init_hkey_from_vault: rc=17412 SECidfGet (psession.cc:429)
03/06/2021 10:21:05 PROTON: Session::init_hkey_from_vault: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: SessionCache::set_idfile: enter (pcache.cc:483)
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: SessionCache::set_idfile: set_idfile @0x7f3374017ec8 rc=0 (pcache.cc:531)
03/06/2021 10:21:05 PROTON: SessionCache::set_idfile: exit
03/06/2021 10:21:05 PROTON: Session::init_hkey_from_cache_or_vault: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: Session::init_via_server_context2: rc=17412 init_hkey_from_cache_or_vault (psession.cc:279)
03/06/2021 10:21:05 PROTON: Session::init_via_server_context2: exit
03/06/2021 10:21:05 PROTON: Session::init_via_server_context: rc=17412 init (psession.cc:71)
03/06/2021 10:21:05 PROTON: Session::init_via_server_context: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: Req:BLKNT#-1 FindBy:#-1 RetCode:1028 NoteCount:0 TimeMS:124
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: wait_for_id_file done, no_timeout, @0x7f3374017ec8 (pcache.cc:575)
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: get_idfile @0x7f3374017ec8 rc=17412 req_timeout_ms=9996 elapsed_time_ms=4 (pcache.cc:615)
03/06/2021 10:21:05 PROTON: SessionCache::get_idfile: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: Session::init_hkey_from_cache_or_vault: rc=17412 cached_entry.get_idfile (psession.cc:654)
03/06/2021 10:21:05 PROTON: Session::init_hkey_from_cache_or_vault: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: Session::init_via_server_context2: rc=17412 init_hkey_from_cache_or_vault (psession.cc:279)
03/06/2021 10:21:05 PROTON: Session::init_via_server_context2: exit
03/06/2021 10:21:05 PROTON: Session::init_via_server_context: rc=17412 init (psession.cc:71)
03/06/2021 10:21:05 PROTON: Session::init_via_server_context: exit
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: ReturnCode = 17412 (was 0)
03/06/2021 10:21:05 PROTON: Req:BLKNT#-1 FindBy:#-1 RetCode:1028 NoteCount:0 TimeMS:107

I notice quite a few

6

IAM Service now fixed and running correctly. In the end I had to delete and recreate my ID Vault and re-regsiter the iam application user. Validating it was in the new ID Vault and pretty much that was it.

Many thanks Dan, and to Mary on the support team for your help.

Onto the Act-as-User stuff now!