regex - Logstash Ruby Filter to match email addresses -


this question has answer here:

i have ruby filter match email address in log message, remove it, , pass through anonymization filter, this...

  ruby {    code =>     "     begin       if !event['log_message'].nil?         if match = event['log_message'].match(/(\b[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b)/i)            event['user_email'] = match[1]         end       else         puts 'oddity parsing message: log_message nil'         puts event.to_yaml       end     rescue exception => e       puts 'exception parsing user email:'       puts e.message     end     " } if [user_email] {   anonymize {       algorithm => "sha1"     fields => ["user_email"]     key => "mysupersecretpassword"   }   ruby {     code =>       "       begin         event['message'].gsub!(/\b[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b/i, event['user_email'])         event['log_message'].gsub!(/\b[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b/i, event['user_email'])       rescue exception => e         puts 'exception replacing user-email in log:'         puts e.message       end       "       remove_field => ["user_email"]                 } }

as of now, regex isn't catching of anything. tried replacing , got error (which "oddity parsing message" branch of code).

does know how this? don't need crazy over-the-top regex, 1 catch 99% of email addresses. regex tried use 

if match = event['log_message'].match(/(\b[a-za-z0-9_.+=:-]+@[0-9a-za-z][0-9a-za-z-]{0,62}(?:\.(?:[0-9a-za-z][0-‌​9a-za-z-]{0,62}))*\b)/i)

here's log line reference

76817815   11/jun/2016 00:04:28 +0000  info  [eventlistener-3] messagingsvc logdefault    > dosend - sending email... from: "test" <do-not-reply@test.com>

note if can done easier / in more sane way using grok, i'm open removing ruby bit.

this html5 spec 

 [a-za-z0-9.!#$%&'*+/=?^_\`{|}~-]+@[a-za-z0-9](?:[a-za-z0-9-]{0,61}[a-za-z0-9])?(?:\.[a-za-z0-9](?:[a-za-z0-9-]{0,61}[a-za-z0-9])?)*

expanded 

 [a-za-z0-9.!#$%&'*+/=?^_\`{|}~-]+   @  [a-za-z0-9]   (?: [a-za-z0-9-]{0,61} [a-za-z0-9] )?  (?:       \. [a-za-z0-9]        (?: [a-za-z0-9-]{0,61} [a-za-z0-9] )?  )*

Comments

Post a Comment

Popular posts from this blog

jOOQ update returning clause with Oracle -

according http://www.jooq.org/doc/3.8/manual/sql-building/sql-statements/update-statement/ jooq doesn't support returning clause in update statements databases except firebird , postgres. does know, still correct oracle jooq? according oracle's documentation, oracle db supports returning clase delete, insert, or update statements. incorrect manual you're right - manual wrong , should fixed: https://github.com/jooq/jooq/issues/5470 single row update returning jooq 3.8 added support single row update .. returning statements. internally, uses callablestatement , pl/sql block of form: begin update "my_table" set "my_table"."column" = 'xyz' "my_table"."id" = 1 returning "my_table"."id", "my_table"."column" ?, ?; ? := sql%rowcount; end; the relevant issue is: https://github.com/jooq/jooq/issues/5190 multi row update returning a...
Read more

java - Warning equals/hashCode on @Data annotation lombok with inheritance -

i have entity inherits other. on other hand, i'm using lombok project reduce boilerplate code, put @data annotation. annotation @data inheritance produces next warning: generating equals/hashcode implementation without call superclass, though class not extend java.lang.object. if intentional, add '@equalsandhashcode(callsuper=false)' type. is advisable add annotation @equalsandhashcode (callsuper = true) or @equalsandhashcode (callsuper = false) ? if not added, 1 callsuper=false or callsuper=true ? the default value false . 1 if don't specify , ignore warning. yes, recommended add @equalsandhashcode annotation on @data annotated classes extend else object. cannot tell if need true or false , depends on class hierarchy, , need examined on case-by-case basis. however, project or package, can configure in lombok.config call super methods if not direct subclass of object. lombok.equalsandhashcode.callsuper = call for configuration system ...
Read more

java - BasicPathUsageException: Cannot join to attribute of basic type -

i using java 8 jpa (hibernate 5.2.1). have clause works perfectly, until introduce clause make use foreign key table values too. i getting following error: basicpathusageexception: cannot join attribute of basic type i think problem related fact have join table, , not sure how create join using join table. employee - employee_category - category model (employee.java): @manytomany(cascade = cascadetype.all, fetch=fetchtype.eager) @jointable ( name="employee_category", joincolumns={ @joincolumn(name="emp_id", referencedcolumnname="id") }, inversejoincolumns={ @joincolumn(name="cat_id", referencedcolumnname="id") } ) private set<category> categories; model (category.java): @id private string id; jpa when introduce following code, fails error above (i think problem new piece of code): join<t, category> category = from.join("id"); predicates.add(criteriabuilder.like(category....
Read more