Reading CSV from Python UDF script

Welcome

Hi

I have been trying to load a CSV file using Python UDF script and do computations using Pandas. When I try to read the csv, I am getting File Not Exists error. My code is as follows

CREATE OR REPLACE PYTHON SCALAR SCRIPT TEST_LOAD ()
RETURNS VARCHAR(100) AS
from pandas import pandas
def run(ctx):
data = pandas.read_csv('F://myfile.csv', error_bad_lines = False)
return 'Success'
/
;

I cannot Import the data into table bcoz I need to do some data cleansing. Please help me with this.

Thanks in advance.

 

1 REPLY 1

Contributor

Hi,

you get the error message, as the Python docker container does not have access to your local filesystem.

 

As a best practise I would advice to first load the uncleaned data into a table via IMPORT statement.

(https://docs.exasol.com/sql/import.htm). Just put everything in VARCHAR columns, so that there should be no problem

importing the data.

After that you can do the cleaning of the data via a Python UDF or just a normal SQL statement, depending

on the kind of adaptions you want to do. Therefor I would create a new target table.

 

 

Hope that helps. :)