Exasol DockerDB - Apply EXAConf changes on startup

ikutle
Padawan

Hello everyone,

I am trying to "manipulate" exasol docker image to start a container with enabled auditing, expanded storage and ram, also different nameserver list, etc. I've tried to create a specific entrypoint.sh as it was suggested here. Main problem is that it runs ok, but one has to restart the container in order for changes to take place. Is there easier way to enable and change parameters from EXAConf and use them on a first startup? Another suggestion in mentioned post was exa/etc/rc.local, can somebody give me some instructions on how that works?

Thank you in advance! 🙂

1 ACCEPTED SOLUTION

exa-Oleksandr
Team Exasol
Team Exasol

Hello! Probably the most elegant way is to use rc.local with python language. I have attached an example file. To use it, you need to copy it to a directory, which you use for the /exa directory inside the container, like this:

 

mkdir -p /data/n11/etc
cp rc_local.py /data/n11/etc/rc.local
chmod 755 /data/n11/etc/rc.local
docker run -it --privileged -v /data/n11:/exa exasol/docker-db:latest

 

That way you can set up your container at the start without your own entry point.  This script works with 7.0, to use it with 6.2 you need to use python2 in rc.local. Several things are to mention here:

  • When you mount /exa like this, then your EXAConf will not be completely initialized and you need to do that in the rc.local, especially the storage configuration needs to be done there.
  • This script is supposed to work on single-node systems, on multi-node systems it should work as well, but there you usually set up the EXAConf and the environment beforehand anyway.

Generally with rc.local you can hook into any stage of the boot process and do things.

Best regards,
Oleksandr

 

 

 

#!/usr/bin/env python3

import sys, os
sys.path.insert(0, '{}/lib'.format(os.environ["COS_DIRECTORY"]))

if len(sys.argv) > 1: # manuall execution without arguments is allowed
    if sys.argv[1] != 'stage0': # execute only on stage 0
        sys.exit(0)
    if os.path.exists('/exa/etc/custom_init_done'): # not the first run
        sys.exit(0)

import time
from libconfd.EXAConf import EXAConf, config
from libconfd.common.util import units2bytes

current_nid = os.environ["EXA_NODE_ID"]
exaconf = EXAConf()
changed = False

# setup nameserver
if exaconf.get_nameservers() != ['1.1.1.1']:
    exaconf.config["Global"]["NameServers"] = "1.1.1.1"
    changed = True

# configure node storage
for nid, node in exaconf.get_nodes().items():
    if 'disk1' not in node.disks:
        disk1 = config()
        disk1.component = "exastorage"
        disk1.devices = ["dev.1"]
        disk1.mapping = [('dev.1', '/exa/data/storage')]
        node.disks.disk1 = disk1
        exaconf.set_node_conf(node, nid, commit = False)
        changed = True
        if current_nid == nid:
            with open('/exa/data/storage/dev.1', 'w') as fd:
                # make our storage disk 5 GiB large
                fd.seek(units2bytes('5 GiB'))
                fd.write(' ')

# configure data volume
datavol1 = exaconf.get_volumes()["DataVolume1"]
if datavol1.size == 0:
    datavol1.size = '2 GiB'
    datavol1.disk = 'disk1'
    exaconf.set_volume_conf(datavol1, 'DataVolume1', commit = False)
    changed = True

# commit exaconf changes to the disk
if changed:
    print("EXAConf was changed, commit.")
    exaconf.commit()

# mark this system as initialized
with open('/exa/etc/custom_init_done', 'w') as fd:
    fd.write('%s\n' % time.asctime())

 

 

 

 

View solution in original post

1 REPLY 1

exa-Oleksandr
Team Exasol
Team Exasol

Hello! Probably the most elegant way is to use rc.local with python language. I have attached an example file. To use it, you need to copy it to a directory, which you use for the /exa directory inside the container, like this:

 

mkdir -p /data/n11/etc
cp rc_local.py /data/n11/etc/rc.local
chmod 755 /data/n11/etc/rc.local
docker run -it --privileged -v /data/n11:/exa exasol/docker-db:latest

 

That way you can set up your container at the start without your own entry point.  This script works with 7.0, to use it with 6.2 you need to use python2 in rc.local. Several things are to mention here:

  • When you mount /exa like this, then your EXAConf will not be completely initialized and you need to do that in the rc.local, especially the storage configuration needs to be done there.
  • This script is supposed to work on single-node systems, on multi-node systems it should work as well, but there you usually set up the EXAConf and the environment beforehand anyway.

Generally with rc.local you can hook into any stage of the boot process and do things.

Best regards,
Oleksandr

 

 

 

#!/usr/bin/env python3

import sys, os
sys.path.insert(0, '{}/lib'.format(os.environ["COS_DIRECTORY"]))

if len(sys.argv) > 1: # manuall execution without arguments is allowed
    if sys.argv[1] != 'stage0': # execute only on stage 0
        sys.exit(0)
    if os.path.exists('/exa/etc/custom_init_done'): # not the first run
        sys.exit(0)

import time
from libconfd.EXAConf import EXAConf, config
from libconfd.common.util import units2bytes

current_nid = os.environ["EXA_NODE_ID"]
exaconf = EXAConf()
changed = False

# setup nameserver
if exaconf.get_nameservers() != ['1.1.1.1']:
    exaconf.config["Global"]["NameServers"] = "1.1.1.1"
    changed = True

# configure node storage
for nid, node in exaconf.get_nodes().items():
    if 'disk1' not in node.disks:
        disk1 = config()
        disk1.component = "exastorage"
        disk1.devices = ["dev.1"]
        disk1.mapping = [('dev.1', '/exa/data/storage')]
        node.disks.disk1 = disk1
        exaconf.set_node_conf(node, nid, commit = False)
        changed = True
        if current_nid == nid:
            with open('/exa/data/storage/dev.1', 'w') as fd:
                # make our storage disk 5 GiB large
                fd.seek(units2bytes('5 GiB'))
                fd.write(' ')

# configure data volume
datavol1 = exaconf.get_volumes()["DataVolume1"]
if datavol1.size == 0:
    datavol1.size = '2 GiB'
    datavol1.disk = 'disk1'
    exaconf.set_volume_conf(datavol1, 'DataVolume1', commit = False)
    changed = True

# commit exaconf changes to the disk
if changed:
    print("EXAConf was changed, commit.")
    exaconf.commit()

# mark this system as initialized
with open('/exa/etc/custom_init_done', 'w') as fd:
    fd.write('%s\n' % time.asctime())

 

 

 

 

View solution in original post