Fully managed cloud. 30-day FREE full featured trial. Start Now
cancel
Showing results for 
Search instead for 
Did you mean: 

UDF Python flavours: aligning libraries

ellieb
Padawan

Hi all,

I'm afraid I have another question on UDFs and how to get language packages to work!

We are trying to bring a Python library called Statsmodels into a UDF to be able to run a predictive data model via Exasol, but we cannot get the libraries we are bringing in to align. We understand that using flavours should be the preferred method to do this, but we can't get ours to work correctly. I'll explain the specific steps we have taken and the error we have:

Step 1 - Download a pre-packaged flavour that we believe will be suitable for us. Load this into BucketFS.

Step 2 - Set Script_Languages session parameter to include the flavour package from BucketFS.

Step 3 - Write a script in the new language (Python_update)

Step 4 - Attempt to import Statsmodels library and access a specific section of the library.

When we try to complete step 4 we get the attached error, which appears to be a conflict in the library versions we are using (scipy is trying to use a functionality of numpy that no longer exists). To get round this we think we need to edit the library versions that are in the flavour (we have tried importing a newer version of scipy and get more version error messages, so we believe editing the flavour will be the simplest solution).

However, the way we have found to do this flavour editing online seems to be to use Docker, which we don't have available as a tool we can use. Is there any way we can compile our own flavours in Exasol? Or edit existing flavours without using the Docker tool? Or any other solution that might be feasible?

On the other solutions we have tried a variety of different combinations of flavours, libraries and Python versions (as you can probably see by the commented out sections of the error screenshot!). Options we've investigated include a more up-to-date Python version with newer versions of Statsmodels and scipy, older versions of scipy and statsmodels, and other combinations but these have all resulted in version conflicts (scipy and Python version conflicts, Statsmodels and scipy conflicts, etc.!).

Any help would be much appreciated as we are going round in circles with different libraries and Python versions! 

Thanks, Ellie

ellieb_0-1663080428812.png

 

1 REPLY 1

exa-Aleksandr
Team Exasol
Team Exasol

Hi @ellieb ,

Yes, customizing a Language container should be the way to go.

It isn't too hard, actually.

You've, probably, seen the exhaustive tutorial in Using the Script-Language Container.

In a nutshell you need just to:

  1. Install prerequisites. You need a Linux machine to do so. I do it on a VirtualBox virtual machine.
  2. Pick a flavor that requires the least of changes for your needs in Exasol Script Language Flavors overview
  3. Edit the respective package list file, like python3_pip_packages
  4. Run "exaslct" command to build the container .tar.gz file.
  5. Upload it to BucketFS via CURL command.
  6. Run "exaslct" command to generate ALTER SESSION  / ALTER SYSTEM statement text to activate the new container.

Maybe you can give it a try and we try to help you if you are stuck somewhere?