JNA concurrency issues -


i'm trying access native (fortran) library (mylib.so) loaded jna. library accessed in parallal spark-job. far have not synchronized method calls (nor library) call shared library bottleneck in computations , must run in parallel.

i following error:

# fatal error has been detected java runtime environment: # #  sigsegv (0xb) @ pc=0x00007ffbcb5f8dcd, pid=58569, tid=140708155152128 # # jre version: java(tm) se runtime environment (8.0_60-b27) (build 1.8.0_60-b27) *** error in `/usr/java/jdk1.8.0_60/jre/bin/java': double free or corruption (!prev): 0x0000000001b756d0 *** *** error in `/usr/java/jdk1.8.0_60/jre/bin/java': free(): corrupted unsorted chunks: 0x0000000001b75010 *** *** error in `/usr/java/jdk1.8.0_60/jre/bin/java': double free or corruption (!prev): 0x0000000001b756d0 *** # java vm: java hotspot(tm) 64-bit server vm (25.60-b23 mixed mode linux-amd64 compressed oops) # problematic frame: # c  [libc.so.6+0x38dcd]======= backtrace: ========= ======= backtrace: ========= ======= backtrace: ========= /lib64/libc.so.6(+0x7d053)[0x7ffbcb63d053] /lib64/libc.so.6(+0x7d053)[0x7ffbcb63d053] /lib64/libc.so.6(+0x7d053)[0x7ffbcb63d053] /lib64/libc.so.6(+0x38e90)[0x7ffbcb5f8e90] /usr/java/jdk1.8.0_60/jre/lib/amd64/server/libjvm.so(+0x5d43f9)[0x7ffbcabba3f9] /lib64/libc.so.6(+0x38e90)[0x7ffbcb5f8e90] /lib64/libc.so.6(+0x38eb5)[0x7ffbcb5f8eb5] /lib64/libc.so.6(+0x38e69)[0x7ffbcb5f8e69] /lib64/libc.so.6(+0x38eb5)[0x7ffbcb5f8eb5]   __run_exit_handlers+0x3d # # failed write core dump. core dumps have been disabled. enable core dumping, try "ulimit -c unlimited" before starting java again # /opt/usr/local/lib/mylib.so(for_exit+0x19)[0x7ff9285b0fa9] /lib64/libc.so.6(+0x38eb5)[0x7ffbcb5f8eb5] /opt/usr/local/lib/mylib.so(for_exit+0x19)[0x7ff9285b0fa9] /opt/usr/local/lib/mylib.so(for__once_private+0x27)[0x7ff9285660b7] /opt/usr/local/lib/mylib.so(for__once_private+0x27)[0x7ff9285660b7] /opt/usr/local/lib/mylib.so(for__acquire_lun+0x814)[0x7ff92855ea04] /opt/usr/local/lib/mylib.so(for__acquire_lun+0x814)[0x7ff92855ea04] /opt/usr/local/lib/mylib.so(for_write_int_fmt+0x9c)[0x7ff92857f83c] /opt/usr/local/lib/mylib.so(for__acquire_lun+0x814)[0x7ff92855ea04] /opt/usr/local/lib/mylib.so(for_write_int_fmt+0x9c)[0x7ff92857f83c] /opt/usr/local/lib/mylib.so(seterr_+0x1c8)[0x7ff928515328] /opt/usr/local/lib/mylib.so(for_write_int_fmt+0x9c)[0x7ff92857f83c] /opt/usr/local/lib/mylib.so(seterr_+0x1c8)[0x7ff928515328] /opt/usr/local/lib/mylib.so(ddl2sf_+0x322)[0x7ff92850eb02] /opt/usr/local/lib/mylib.so(seterr_+0x1c8)[0x7ff928515328] /opt/usr/local/lib/mylib.so(entsrc_+0x80)[0x7ff92850edb0] /opt/usr/local/lib/mylib.so(bspline_+0x52c)[0x7ff92850d66c] /opt/usr/local/lib/mylib.so(ddl2sf_+0x2e7)[0x7ff92850eac7] /opt/usr/local/lib/mylib.so(enter_+0x55)[0x7ff92850ecd5] /opt/usr/local/lib/mylib.so(rspbsp_+0x4a)[0x7ff928523cda] /opt/usr/local/lib/mylib.so(bspline_+0x52c)[0x7ff92850d66c] /opt/usr/local/lib/mylib.so(ddl2sf_+0x16d)[0x7ff92850e94d] /tmp/jna--1845237309/jna4302334124297214663.tmp(ffi_call_unix64+0x4c)[0x7ff92909465c] /tmp/jna--1845237309/jna4302334124297214663.tmp(ffi_call+0x1d4)[0x7ff929094164] /opt/usr/local/lib/mylib.so(bspline_+0x52c)[0x7ff92850d66c] /opt/usr/local/lib/mylib.so(rspbsp_+0x4a)[0x7ff928523cda] /tmp/jna--1845237309/jna4302334124297214663.tmp(+0x5870)[0x7ff929087870] /tmp/jna--1845237309/jna4302334124297214663.tmp(java_com_sun_jna_native_invokevoid+0x22)[0x7ff92908a462] [0x7ffbb5015994]

afaik has native library not thread-safe? stacktracke seems me actual problem libc.so, or own library mylib.so ?

case problem in own library, possible overcome problem making multiple physical copies of shared object, 1 each thread example?

this indicate issue in shared library or use. if did not design library thread-safe not. single process cannot load native library multiple times, there no easy solution in way. there couple of avenues:

  • allocate multiple objects native library , different ones each java thread. may not possible if native library internally using static data structures.
  • synchronize access native library via classes/methods expose it. in way single java thread access them @ time. however, depending on library , use case may not performance -- action still in 1 place.
  • make library thread-safe. can lot more difficult in native languages java. additionally algorithms not suited parallelization.

Comments

Post a Comment

Popular posts from this blog

jOOQ update returning clause with Oracle -

according http://www.jooq.org/doc/3.8/manual/sql-building/sql-statements/update-statement/ jooq doesn't support returning clause in update statements databases except firebird , postgres. does know, still correct oracle jooq? according oracle's documentation, oracle db supports returning clase delete, insert, or update statements. incorrect manual you're right - manual wrong , should fixed: https://github.com/jooq/jooq/issues/5470 single row update returning jooq 3.8 added support single row update .. returning statements. internally, uses callablestatement , pl/sql block of form: begin update "my_table" set "my_table"."column" = 'xyz' "my_table"."id" = 1 returning "my_table"."id", "my_table"."column" ?, ?; ? := sql%rowcount; end; the relevant issue is: https://github.com/jooq/jooq/issues/5190 multi row update returning a...
Read more

java - Warning equals/hashCode on @Data annotation lombok with inheritance -

i have entity inherits other. on other hand, i'm using lombok project reduce boilerplate code, put @data annotation. annotation @data inheritance produces next warning: generating equals/hashcode implementation without call superclass, though class not extend java.lang.object. if intentional, add '@equalsandhashcode(callsuper=false)' type. is advisable add annotation @equalsandhashcode (callsuper = true) or @equalsandhashcode (callsuper = false) ? if not added, 1 callsuper=false or callsuper=true ? the default value false . 1 if don't specify , ignore warning. yes, recommended add @equalsandhashcode annotation on @data annotated classes extend else object. cannot tell if need true or false , depends on class hierarchy, , need examined on case-by-case basis. however, project or package, can configure in lombok.config call super methods if not direct subclass of object. lombok.equalsandhashcode.callsuper = call for configuration system ...
Read more

java - BasicPathUsageException: Cannot join to attribute of basic type -

i using java 8 jpa (hibernate 5.2.1). have clause works perfectly, until introduce clause make use foreign key table values too. i getting following error: basicpathusageexception: cannot join attribute of basic type i think problem related fact have join table, , not sure how create join using join table. employee - employee_category - category model (employee.java): @manytomany(cascade = cascadetype.all, fetch=fetchtype.eager) @jointable ( name="employee_category", joincolumns={ @joincolumn(name="emp_id", referencedcolumnname="id") }, inversejoincolumns={ @joincolumn(name="cat_id", referencedcolumnname="id") } ) private set<category> categories; model (category.java): @id private string id; jpa when introduce following code, fails error above (i think problem new piece of code): join<t, category> category = from.join("id"); predicates.add(criteriabuilder.like(category....
Read more