Fully managed cloud. 30-day FREE full featured trial. Start Now
cancel
Showing results for 
Search instead for 
Did you mean: 

AWS New Cluster - One date node does not join cluster

krockhold
Contributor

Created a new cluster in AWS using the Setup Wizard and Cloudformation. Everything seemed ok, but the management interface doesn't come up and the cluster is not usable. Logged into the managment/license server and ran cosps -N:

 

ID	ROOT NODE	STATE
10:	license.c0001.exacluster.local	online
11:	n0011.c0001.exacluster.local	online
12:	n0012.c0001.exacluster.local	online
13:	n0013.c0001.exacluster.local	online
14:	n0014.c0001.exacluster.local	offline

ID	OWNER	GROUP	PARENT	FLAGS	NODES           OFFLINE         COMMAND
1	    0	    0	     0	 RA--	10,11,12,13     -               logd
2	    0	    0	     0	 RA--	10,11,12,13     -               lockd
3	    0	    0	     0	 RA--	10,11,12,13     -               dwad
4	    0	    0	     0	 RA--	10,11,12,13     -               appserverd

 

 SSH to the "offline" node and ran the same:

 

cluster1 [root@n0014 ~]# cosps -N
ID	ROOT NODE	STATE
14:	n0014.c0001.exacluster.local	online

ID	OWNER	GROUP	PARENT	FLAGS	NODES           OFFLINE         COMMAND

 

 
So, without access to the GUI, is there anyway I can get this server into the expected cluster?

NOTE: This is the second cluster on the same subnet if that matters.

4 REPLIES 4

mwellbro
Xpert

Hi @krockhold ,

I just tested this with 2 slightly smaller clusters ( 2x 2-Nodes ) and it did work, both clusters are operational, even it the second took a bit longer until it was "in working shape".

mwellbro_0-1642973980871.png

Got different curves for the first and second cluster in the same subnet ( never done 2 cluster in 1 subnet before ) and my
second cluster halted quite a while here:

mwellbro_1-1642974064020.png

Both have been created with the same cloud formation template in the AWS-Frankfurt-region, the
only difference being that I had to untick a few boxes regarding EC2-Endpoint creation , etc.

I´ll have to admit that I´m using a 7.0.8 AMI , so I have two maybe relevant differences to your setup ( just had the CF-template already on hand so I tested with what I had ).

For the next iteration I´ll switch to the currently generated CF-template from the cloudtools-page and go for 2x 4Nodes - hopefully my AWS-Account will allow for that kind of quota 😆

Cheers,
Malte

mwellbro
Xpert

just a quick question for @krockhold which might benefit others as well: how do you "root" into your AWS clusters ?
The ssh key we configure into the cloudformation stack is used for the ec2-user , is it not ? ( which is why I currently can´t figure out how to "cosps -N" on the node 😉 ).

krockhold
Contributor

Yes, ssh with the ec2-user into the management/license node with the specified key fro CloudFormation. From the management license node you can log into any of the data nodes using the internal naming scheme:

$ ssh n0012




krockhold
Contributor

As to the original problem, restarting the cluster eventually got the orphan node into the cluster but the entire setup was unstable. Trying to restart the data nodes to get them in running status caused the management interface to fail. Eventually I gave up and rebuilt the cluster, this one came up correctly