Package com.linkedin.feathr.common.util

Class MvelContextUDFs

java.lang.Object
com.linkedin.feathr.common.util.MvelContextUDFs
public class MvelContextUDFs extends Object
MVEL is an open-source expression language and runtime that makes it easy to write concise statements that operate on structured data objects (such as Avro records), among other things. This class contains all the udfs used in Mvel for both online and offline

  • Method Details

    • registerUDFs

      public static void registerUDFs(Class<?> clazz, org.mvel2.ParserConfiguration parserConfig)

    • get_data_type

      public static String get_data_type(Object input)
      Get the class type of the input object WARNING: This is only used for debug for users.
      Returns:
      Type in String form

    • cast_double

      public static Double cast_double(Object input)
      Cast the input to double. If it's null, null is returned. If it's string, it will try to parse it. For number types, it will do standard conversion. For other types, it will be coerced to double.

    • cast_float

      public static Float cast_float(Object input)
      Cast the input to float. If it's null, null is returned. If it's string, it will try to parse it. For number types, it will do standard conversion. For other types, it will be coerced to float.

    • cast_int

      public static Integer cast_int(Object input)
      Cast the input to Integer. If it's null, null is returned. If it's string, it will try to parse it. For number types, it will do standard conversion. For other types, it will be coerced to Integer.

    • and

      public static boolean and(boolean left, boolean right)

    • or

      public static boolean or(boolean left, boolean right)

    • not

      public static boolean not(boolean input)

    • isnull

      public static boolean isnull(Object input)

    • isnotnull

      public static boolean isnotnull(Object input)

    • concat

      public static String concat(String left, String right)
      Concatenate two strings into one.

    • if_else

      public static String if_else(boolean input, String first, String second)
      Ternary operator. If input is evaluated to true, then first is returned, else second is returned.

    • if_else

      public static Double if_else(boolean input, Double first, Double second)

    • if_else

      public static Float if_else(boolean input, Float first, Float second)

    • if_else

      public static Integer if_else(boolean input, Integer first, Integer second)

    • if_else

      public static boolean if_else(boolean input, boolean first, boolean second)

    • isNonZero

      public static boolean isNonZero(Object vector)
      Return true if a feature has at least 1 term with value not zero. For checking if a feature is null, please use isPresent().

    • isPresent

      public static boolean isPresent(Object featureValue)
      return true if a feature variable is not null.

    • toBoolean

      public static Boolean toBoolean(Object item)
      Converts an object to a boolean

    • toNumeric

      public static Object toNumeric(Object item)
      Convert input to numeric value Example inputs that can be converted - Map("" -> 2.0) returns 2.0 - Map("876" -> 1.0) returns 876

    • toCategorical

      public static Object toCategorical(Object item)
      Convert input to categorical feature Example inputs that can be converted - Map("a" -> 2.0) return Map("a" -> 2.0) - Map("" -> 2.5) returns ("2.5", 1.0)

    • getTerms

      public static Collection<String> getTerms(Object item)

    • getTopKTerms

      public static List<String> getTopKTerms(Object item, int k)
      return an ordered list of terms, based on descending order of corresponding values
      Parameters:
      item - Object that can be converted to Map of string to float
      k - integer, if k<0, do reverse order selection, e.g. -1: select bottom first
      Returns:
      List of string, ordered

    • getTopTerm

      public static String getTopTerm(Object item)
      get terms with the highest value (duplicate will resolved randomly)
      Parameters:
      item - Object that can be converted to Map of string to float
      Returns:
      String

    • distinct

      public static Collection<Object> distinct(Collection<Object> collection)

    • flatten

      public static Collection<Object> flatten(Collection<? extends Collection<Object>> collection)

    • cosineSimilarity

      public static Float cosineSimilarity(Object obj1, Object obj2)

    • dotProduct

      public static Double dotProduct(Object obj1, Object obj2)
      Returns a standard dotProduct of two vector objects. Use cosineSimilarity(Object, Object) for normalized dot-product.

    • toLowerCase

      public static String toLowerCase(String input)
      convert input to lower case string
      Parameters:
      input - input string
      Returns:
      lower case input

    • toUpperCase

      public static String toUpperCase(String input)
      convert input to upper case string
      Parameters:
      input - input string
      Returns:
      upper case input

    • time_duration

      public static long time_duration(Object startTime, Object endTime, String outputGranularity)

    • dayofweek

      public static int dayofweek(Object input)

    • dayofmonth

      public static int dayofmonth(Object input)

    • hourofday

      public static int hourofday(Object input)

    • extract_term_value_from_array

      public static Map<String,Float> extract_term_value_from_array(ArrayList<org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema> array, String termFieldName, String valueFieldName)

    • extract_term_value_from_array

      public static Map<String,Float> extract_term_value_from_array(ArrayList<org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema> array, String termFieldName, String valueFieldName, String filterExpr)