17-11-2020 01:57 PM
Hi all,
I was wondering: what is your preferred way of scheduling scripts on exasol without resorting to an external scheduler ?
I´m thinking along the lines "scripts that normally should run periodically while the DB instance is up", like the session_watchdog for example or any script that dumps "snapshoted" tables to somewhere ( while EXA_DB_SIZE_HOURLY is nice, a "per object analysis" does pack a lot more "argumentative punch" ).
We could of course use any external capabilities like a scheduler or a cron job on some other machine but that does create a dependency where there shouldn´t be any...
Do any of you do something along these lines ? Also: how would you/ would we bootstrap something like that "native to/on exa" ?
Perhaps there is a parameter which would allow us to start a JAR from BucketFS or anything like that ?
Looking forward to your input 🙂
20-11-2020 05:05 PM
Hi @mwellbro
Generally, the management node is not designed for executing jobs periodically (on a high frequency). This node is usually undersized for this purpose. In most situations, it will work out when executing a job on a lower frequency. Unfortunately, due to the undersizing of this machine, we do not recommend to schedule jobs on a high frequency, since this will cause the machine to have a higher load of RAM or disk usage. This might cause the clean executing of these jobs and furthermore when connecting to the DB also block the DB (using an active session slot).
Recommended way to schedule and execute these jobs from outside of the cluster. You will find a CLI Exaplus version within our download portal.
This might not be, what you want to hear within this thread. 😁 Sorry about that!
20-11-2020 05:27 PM
Thanks for your input, @exa-ThomasM !
What would you consider high frequency ? I guess once per minute is already be too much ?
The background for this thread is actually that we are scheduling everything outside our cluster, but once the scheduling system goes into "maintenance mode" ( oversimplified ) then there no longer is a watchdog "protecting" the DB instance.
It would "feel" much better to just know once the DB is running such things are taken care of ( or, for that matter, there will always be daily dumps of the exa_dba_object_sizes without the need to configure that on an external scheduler 😊 ).
Well, I´ll look forward to any further comments here, even though so far you all just tell me off....sniff...😉
23-11-2020 12:05 PM
Once per minute is too much 🙈
I think that was the frequency that caused the problems I alluded to...
24-11-2020 11:15 AM
"Once per minute is too much" for the fastest database around ? <pirateVoice>ARRRRR</pirateVoice> 😉
30-11-2020 11:33 AM
20-11-2020 02:41 PM
Maybe not what you want to hear @mwellbro but I have just always used exaplus + cron (like described here).
I'm not aware of any special parameters like that.... I think there has to be a client to trigger a Lua script or UDF (even if these sessions are simply permanently running and use a combination of sleep() and a schedule table to do jobs). So there is still that dependency...
20-11-2020 03:09 PM
Hi @exa-Nico ,
yeah, not what I wanted to hear 😉
But as per your link: "(...)One method is to install Exaplus CLI on a Linux environment(...)" - why not use the good old licence server which doesn´t have that much to do anyways ? We cover linux, I´d think it´s more than capable of running an exaplus and we´ve got cron-tab - all bases covered, I´d say 😉
We´d have created a self-sustained exa-scheduler 😊
Of course not intended for ETL/ELT or any shenanigans like that, but routine maintenance scripts ?
20-11-2020 03:45 PM
Wellll.... there have been cases where heavy cron job use on the license server have caused some problems, which also impacted the database.
A small, separate server is better 😊
Basically, use at your own risk...
Maybe @exa-ThomasM can add something here 😄
Then it's time to become part of a unique family! Discover helpful tips and support other Community members with your knowledge.
Sign In