Team Exasol
Team Exasol

Background

As EXASolution comes with its own operating system and users do rarely have access to that system, testing network connectivity can be quite hard. This makes debugging Import/Export/UDF that fail with timeouts or connection refused a lengthy task.

How to test HTTP connections using python UDF

The attached SQL file contains a SCALAR PYTHON SCRIPT that does minimalistic network testing using the following steps:

  1. Hostname lookup using DNS (Domain Name Service)
  2. TCP connect to the resulting address
  3. HTTP 1.1 request

The script will output some information, including the cluster node it ran on and an elapsed time (in seconds).

Example usage

1 - Testing an internal web server on a random node

select webtest_py('192.168.103.212', 80, '/')
order by 1,2;
NODE_IP SECONDS MESSAGE
n0016.c0001.exacluster.local 0.009 Checking DNS for 192.168.103.212
n0016.c0001.exacluster.local 0.009 Name resolves to IPs ['192.168.103.212']
n0016.c0001.exacluster.local 0.009 Trying to connect to 192.168.103.212, port 80
n0016.c0001.exacluster.local 0.010 Connected.
n0016.c0001.exacluster.local 0.010 HTTP GET request sent
n0016.c0001.exacluster.local 0.011 HTTP/1.1 200 OK
n0016.c0001.exacluster.local 0.011 ...Date: Mon, 12 Oct 2015 14:37:19 GMT
n0016.c0001.exacluster.local 0.011 ...Server: Apache/2.2.9 (Debian) mod_auth_kerb/5.3 DAV/2 SVN/1.5.1 PHP/5.2.6-1+lenny16 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g
n0016.c0001.exacluster.local 0.011 ...Last-Modified: Mon, 16 Jun 2008 08:57:00 GMT
n0016.c0001.exacluster.local 0.011 ...ETag: "50800c-128-44fc4cf7b6f00"
n0016.c0001.exacluster.local 0.011 ...Accept-Ranges: bytes
n0016.c0001.exacluster.local 0.011 ...Content-Length: 296
n0016.c0001.exacluster.local 0.011 ...Vary: Accept-Encoding
n0016.c0001.exacluster.local 0.011 ...Content-Type: text/html
n0016.c0001.exacluster.local 0.011 End of headers

2 - Invalid page

select webtest_py('192.168.103.212', 80, '/xxx.html')
order by 1,2;
NODE_IP SECONDS MESSAGE
n0016.c0001.exacluster.local 0.002 Checking DNS for 192.168.103.212
n0016.c0001.exacluster.local 0.002 Name resolves to IPs ['192.168.103.212']
n0016.c0001.exacluster.local 0.003 Trying to connect to 192.168.103.212, port 80
n0016.c0001.exacluster.local 0.003 Connected.
n0016.c0001.exacluster.local 0.003 HTTP GET request sent
n0016.c0001.exacluster.local 0.004 HTTP/1.1 404 Not Found
n0016.c0001.exacluster.local 0.004 ...Date: Mon, 12 Oct 2015 14:50:31 GMT
n0016.c0001.exacluster.local 0.005 ...Server: Apache/2.2.9 (Debian) mod_auth_kerb/5.3 DAV/2 SVN/1.5.1 PHP/5.2.6-1+lenny16 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g
n0016.c0001.exacluster.local 0.005 ...Vary: Accept-Encoding
n0016.c0001.exacluster.local 0.005 ...Content-Length: 388
n0016.c0001.exacluster.local 0.005 ...Content-Type: text/html; charset=iso-8859-1
n0016.c0001.exacluster.local 0.005 End of headers

3 - Trying wrong port:

select webtest_py('192.168.103.212', 20, '/')
order by 1,2;
NODE_IP SECONDS MESSAGE
n0016.c0001.exacluster.local 0.013 Checking DNS for 192.168.103.212
n0016.c0001.exacluster.local 0.013 Name resolves to IPs ['192.168.103.212']
n0016.c0001.exacluster.local 0.014 Trying to connect to 192.168.103.212, port 20
n0016.c0001.exacluster.local 0.014 Failed: [Errno 111] Connection refused

4 - Firewall blocks external connection from all nodes:

select webtest_py('www.heise.de', 80, '/')
from exa_loadavg
order by 1,2;
NODE_IP SECONDS MESSAGE
n0016.c0001.exacluster.local 0.002 Checking DNS for www.heise.de
n0016.c0001.exacluster.local 0.003 Name resolves to IPs ['193.99.144.85']
n0016.c0001.exacluster.local 0.004 Trying to connect to 193.99.144.85, port 80
n0016.c0001.exacluster.local 63.004 Failed: [Errno 110] Connection timed out
n0017.c0001.exacluster.local 0.009 Checking DNS for www.heise.de
n0017.c0001.exacluster.local 0.011 Name resolves to IPs ['193.99.144.85']
n0017.c0001.exacluster.local 0.011 Trying to connect to 193.99.144.85, port 80
n0017.c0001.exacluster.local 63.011 Failed: [Errno 110] Connection timed out
n0019.c0001.exacluster.local 0.009 Checking DNS for www.heise.de
n0019.c0001.exacluster.local 0.011 Name resolves to IPs ['193.99.144.85']
n0019.c0001.exacluster.local 0.011 Trying to connect to 193.99.144.85, port 80
n0019.c0001.exacluster.local 63.011 Failed: [Errno 110] Connection timed out
n0020.c0001.exacluster.local 0.002 Checking DNS for www.heise.de
n0020.c0001.exacluster.local 0.003 Name resolves to IPs ['193.99.144.85']
n0020.c0001.exacluster.local 0.004 Trying to connect to 193.99.144.85, port 80
n0020.c0001.exacluster.local 63.003 Failed: [Errno 110] Connection timed out
Comments
Contributor

Where do i find this python script?

webtest_py 

 

Team Exasol
Team Exasol

Thank you for asking. I have attached the file.