python - Pentaho PDI: Metadata related Null Pointer Exception in Scripting task -


using pentaho pdi 6, with:

a) csv input on .csv (4 row .csv ibm), ascii file encoding (preview rows works fine)

connected to

b) cpython script executor, installable tools -> marketplace. assumes python, pandas, numpy installed. script settings:

configure, input frames: (previous step), df
python script, manual python script: df.replace(to_replace= "\[|\]|'|\"", value='', regex=true, inplace=true)
output fields, output fields: (column names, string type)

throws

2016/07/25 10:45:21 - cpython script executor.0 - error (version 6.1.0.1-196, build 1 2016-04-07 12.08.49 buildguy) : unexpected error  2016/07/25 10:45:21 - cpython script executor.0 - error (version 6.1.0.1-196, build 1 2016-04-07 12.08.49 buildguy) : java.lang.nullpointerexception  2016/07/25 10:45:21 - cpython script executor.0 - 	at org.pentaho.python.pythonsession.rowstopythondataframe(pythonsession.java:389)  2016/07/25 10:45:21 - cpython script executor.0 - 	at org.pentaho.di.trans.steps.cpythonscriptexecutor.cpythonscriptexecutor.rowstopydataframe(cpythonscriptexecutor.java:458)  2016/07/25 10:45:21 - cpython script executor.0 - 	at org.pentaho.di.trans.steps.cpythonscriptexecutor.cpythonscriptexecutor.processbatch(cpythonscriptexecutor.java:276)  2016/07/25 10:45:21 - cpython script executor.0 - 	at org.pentaho.di.trans.steps.cpythonscriptexecutor.cpythonscriptexecutor.processrow(cpythonscriptexecutor.java:243)  2016/07/25 10:45:21 - cpython script executor.0 - 	at org.pentaho.di.trans.step.runthread.run(runthread.java:62)  2016/07/25 10:45:21 - cpython script executor.0 - 	at java.lang.thread.run(unknown source)  2016/07/25 10:45:21 - cpython script executor.0 - finished processing (i=0, o=0, r=3, w=0, u=0, e=1)

previous debugging suggests processrow might not able determine metadata type error doesn't indicate this.

question: what's proper way set scripting task read in .csv w/o throwing nullpointerexceptions?

edit - error reproduced source materials well. see: mark hall, cpython scripting , example .zip file

edit 1 - python in command prompt gives

c:\users\*****>python

python 3.5.2 (v3.5.2:4def2a2901a5, jun 25 2016, 22:01:18) [msc v.1900 32 bit (intel)] on win32 type "help", "copyright", "credits" or "license" more information.

i not running anaconda (too heavy weight) , version of python .1 ahead, might impacting things hope plugin python version agnostic unless python binary programming interface changed or something.

edit 2 - can't attach kettle file example files mark hall above reproduce same issue encountered.


Comments

Post a Comment

Popular posts from this blog

jOOQ update returning clause with Oracle -

according http://www.jooq.org/doc/3.8/manual/sql-building/sql-statements/update-statement/ jooq doesn't support returning clause in update statements databases except firebird , postgres. does know, still correct oracle jooq? according oracle's documentation, oracle db supports returning clase delete, insert, or update statements. incorrect manual you're right - manual wrong , should fixed: https://github.com/jooq/jooq/issues/5470 single row update returning jooq 3.8 added support single row update .. returning statements. internally, uses callablestatement , pl/sql block of form: begin update "my_table" set "my_table"."column" = 'xyz' "my_table"."id" = 1 returning "my_table"."id", "my_table"."column" ?, ?; ? := sql%rowcount; end; the relevant issue is: https://github.com/jooq/jooq/issues/5190 multi row update returning a...
Read more

java - Warning equals/hashCode on @Data annotation lombok with inheritance -

i have entity inherits other. on other hand, i'm using lombok project reduce boilerplate code, put @data annotation. annotation @data inheritance produces next warning: generating equals/hashcode implementation without call superclass, though class not extend java.lang.object. if intentional, add '@equalsandhashcode(callsuper=false)' type. is advisable add annotation @equalsandhashcode (callsuper = true) or @equalsandhashcode (callsuper = false) ? if not added, 1 callsuper=false or callsuper=true ? the default value false . 1 if don't specify , ignore warning. yes, recommended add @equalsandhashcode annotation on @data annotated classes extend else object. cannot tell if need true or false , depends on class hierarchy, , need examined on case-by-case basis. however, project or package, can configure in lombok.config call super methods if not direct subclass of object. lombok.equalsandhashcode.callsuper = call for configuration system ...
Read more

java - BasicPathUsageException: Cannot join to attribute of basic type -

i using java 8 jpa (hibernate 5.2.1). have clause works perfectly, until introduce clause make use foreign key table values too. i getting following error: basicpathusageexception: cannot join attribute of basic type i think problem related fact have join table, , not sure how create join using join table. employee - employee_category - category model (employee.java): @manytomany(cascade = cascadetype.all, fetch=fetchtype.eager) @jointable ( name="employee_category", joincolumns={ @joincolumn(name="emp_id", referencedcolumnname="id") }, inversejoincolumns={ @joincolumn(name="cat_id", referencedcolumnname="id") } ) private set<category> categories; model (category.java): @id private string id; jpa when introduce following code, fails error above (i think problem new piece of code): join<t, category> category = from.join("id"); predicates.add(criteriabuilder.like(category....
Read more