Normalizer Filter Functions

The filter functions are used to filter out messages from the stream.

All the filters has a common property that is called __MATCH. The __MATCH property allows us to filter out the messages that either matches the filter or does not.

__MATCH: true The filter filters out the messages that satisfy the filter.
__MATCH: false The filter filters out the messages that don’t satisfy the filter.

FieldFilter

The FieldFilter is a filter that allows us to filter if a specific dimension contains a specific value match.

{
  "name": "myFieldFilter",
  "className": "io.wizzie.normalizer.funcs.impl.FieldFilter",
  "properties": {
    "dimension": "FILTER_DIMENSION",
    "value": "FILTER_VALUE"
  }
}

The FieldFilter has two properties that are called dimension and value:

dimension: The dimension to get the value.
value: Value to compare with obtained value. If we have the next JSON message:

{
  "DIM-A":"VALUE-A",
  "DIM-B":10,
  "FILTER_DIMENSION": "FILTER_VAL"
}

The FieldFilter doesn’t match and return false value. But if we have the next message:

{
  "DIM-A": "VALUE-A",
  "DIM-B": 10,
  "FILTER_DIMENSION": "FILTER_VALUE"
}

The FieldFilter match and return true value

If you use __KEY dimension the filter checks the message key.

MultiValueFieldFilter

The MultiValueFieldFilter is a filter that allows us to filter if a specific dimension contains some value from a list.

{
  "name": "myFieldFilter",
  "className": "io.wizzie.normalizer.funcs.impl.MultiValueFieldfilter",
  "properties": {
    "dimension": "FILTER_DIMENSION",
    "values": ["FILTER_VALUE", "FILTER_VALUE_1"]
  }
}

The FieldFilter has two properties that are called dimension and value:

dimension: The dimension to get the value.
values: Values to compare with obtained value. If we have the next JSON message:

{
  "DIM-A":"VALUE-A",
  "DIM-B":10,
  "FILTER_DIMENSION": "FILTER_VAL"
}

The FieldFilter doesn’t match and return false value. But if we have the next message:

{
  "DIM-A": "VALUE-A",
  "DIM-B": 10,
  "FILTER_DIMENSION": "FILTER_VALUE"
}

{
  "DIM-A": "VALUE-A",
  "DIM-B": 10,
  "FILTER_DIMENSION": "FILTER_VALUE_1"
}

The FieldFilter match and return true value.

If you use __KEY dimension the filter checks the message key.

ContainsDimensionFilter

The ContainsDimensionFilter is a filter that allows us to filter if a JSON contains specific dimensions defined by us.

{
  "name": "myContainsDimensionFilter",
  "className": "io.wizzie.normalizer.funcs.impl.ContainsDimensionFilter",
  "properties": {
    "dimensions": ["DIM-A", "DIM-B", "DIM-C"]
  }
}

The ContainsDimensionFilter has only one property named dimensions. The property indicates the dimensions that json message must have.

If we have the next JSON message:

{
  "DIM-A": "VALUE-A",
  "DIM-K": "VALUE-K",
  "DIM-C": "VALUE-C"
}

The ContainsDimensionFilter doesn’t match. But if we have the next JSON message:

{
  "DIM-A": "VALUE-A",
  "DIM-B": "VALUE-B",
  "DIM-C": "VALUE-C"
}

The ContainsDimensionFilter match.

StartWithFilter

The StartWithFilter is a filter that allows us to filter if a specific dimension contains a value that start with a specific string.

{
  "inputs":{
    "topic":["stream"]
  },
  "streams":{
    "stream":{
      "funcs":[
        {
          "name":"myKeyStartWithFilter",
          "className":"io.wizzie.normalizer.funcs.impl.StartWithFilter",
          "properties": {
            "dimension": "DIM-B",
            "startWith": "FILTER"
          }
        }
      ],
      "sinks":[
        {"topic":"output"}
      ]
    }
  }
}

The StartWithFilter has two properties called dimension and start_with:

dimension: Dimension of which we get the value to filter.
startWith: String to apply to value. If match then return true else return false.

If we have the next JSON message:

{
  "DIM-A":"NOT-FILTER-A",
  "DIM-B":"NOT-FILTER-B",
  "DIM-C":"NOT-FILTER-C"
}

The StartWithFilter doesn’t match. But if we have the next JSON message:

{
  "DIM-A":"NOT-FILTER-A",
  "DIM-B":"FILTER-B",
  "DIM-C":"NOT-FILTER-C"
}

The StartWithFilter match and return true.

AndFilter

The AndFilter is a filter that allows us to apply as many filters as we want. If all filters match then AndFilter return true else return false.

{
  "name":"myAndFilter",
  "className":"io.wizzie.normalizer.funcs.impl.AndFilter",
  "properties": {
    "filters": [
      {
        "name": "myFieldFilter",
        "className": "io.wizzie.normalizer.funcs.impl.FieldFilter",
        "properties": {
          "dimension": "FILTER-DIMENSION",
          "value": "FILTER-VALUE"
        }
      },
      {
        "name": "myContainsFilter",
        "className": "io.wizzie.normalizer.funcs.impl.ContainsDimensionFilter",
        "properties": {
          "dimensions": ["A", "B", "C"]
        }
      }
    ]
  }
}

The AndFilter has been one property called filters.

filters: Array of filters definition. Each filter have particular properties.

If we have this JSON message:

{
  "dimension":"VALUE-1",
  "FILTER-DIMENSION":"FILTER-VALUE",
  "other":"currentValue"
}

The AndFilter doesn’t match and return false because only contains FILTER-DIMENSION.

However if we have this JSON message:

{
  "dimension":"VALUE-1",
  "FILTER-DIMENSION":"FILTER-VALUE",
  "A":"currentValue"
}

The AndFilter matchs and return true because all defined filters have been matched.

OrFilter

The OrFilter is a filter that allows us to apply as many filters as we want. If any filter matches then OrFilter return true else return false.

{
  "name":"myOrFilter",
  "className":"io.wizzie.normalizer.funcs.impl.OrFilter",
  "properties": {
    "filters": [
      {
        "name": "myFieldFilter",
        "className": "io.wizzie.normalizer.funcs.impl.FieldFilter",
        "properties": {
          "dimension": "FILTER-DIMENSION",
          "value": "FILTER-VALUE"
        }
      },
      {
        "name": "myContainsFilter",
        "className": "io.wizzie.normalizer.funcs.impl.ContainsDimensionFilter",
        "properties": {
          "dimensions": ["A", "B", "C"]
        }
      }
    ]
  }
}

The OrFilter has been one property called filters.

filters: Array of filters definition. Each filter have particular properties.

If we have this JSON message:

{
  "dimension":"VALUE-1",
  "FILTER-DIMENSION":"FILTER-VALUE",
  "other":"currentValue"
}

The OrFilter match and return true because the dimension FILTER-DIMENSION contains value FILTER-VALUE

IsStringFilter

The IsStringFilter is a filter that allows us to filter if a specific dimension is a String class.

{
  "name":"isString",
  "className":"io.wizzie.normalizer.funcs.impl.IsStringFilter",
  "properties": {
    "dimension":"string-dimension"
  }
}

The IsStringFilter has two properties that are called dimension and value:

dimension: The dimension to check the value.

If we have the next JSON message:

{
  "DIM-A":"VALUE-A",
  "DIM-B":10,
  "string-dimension": 2
}

The IsStringFilter doesn’t match and return false value. But if we have the next message:

{
  "DIM-A": "VALUE-A",
  "DIM-B": 10,
  "string-dimension": "FILTER_VALUE"
}

The IsStringFilter match and return true value

IsListFilter

The IsListFilter is a filter that allows us to filter if a specific dimension is a List class.

{
  "name":"isList",
  "className":"io.wizzie.normalizer.funcs.impl.IsListFilter",
  "properties": {
      "dimension":"list-dimension"
  }
}

The IsListFilter has two properties that are called dimension and value:

dimension: The dimension to check the value.

If we have the next JSON message:

{
  "DIM-A":"VALUE-A",
  "DIM-B":10,
  "list-dimension": 2
}

The IsListFilter doesn’t match and return false value. But if we have the next message:

{
  "DIM-A": "VALUE-A",
  "DIM-B": 10,
  "list-dimension": ["FILTER_VALUE", "b"]
}

The IsListFilter match and return true value

NumberComparingFilter

The NumberComparingFilter is a filter that allows us to filter if a specific dimension is greater or lower than a number or other dimension.

{
  "name":"numberComparing",
  "className":"io.wizzie.normalizer.funcs.impl.NumberComparingFilter",
  "properties": {
      "firstComparableDimension": "valueA",
      "secondComparable": "valueB",
      "condition": "GREATER"
  }
}

The NumberComparingFilter has three properties that are called firstComparableDimension, secondComparable and condition:

firstComparableDimension : The dimension to compare.
secondComparable : The dimension or number to compare.
condition : This property have two values GREATER or LOWER. By default is GREATER.

If we have the next defined filters:

{
  "name":"greaterThanNumberFilter",
  "className":"io.wizzie.normalizer.funcs.impl.NumberComparingFilter",
  "properties": {
    "firstComparableDimension": "ValueToCompare",
    "secondComparable": 2.5,
    "condition": "GREATER"
  }
},
{
  "name":"lowerThanNumberFilter",
  "className":"io.wizzie.normalizer.funcs.impl.NumberComparingFilter",
  "properties": {
    "firstComparableDimension": "ValueToCompare",
    "secondComparable": 2.5,
    "condition": "LOWER"
  }
},
{
  "name":"greaterThanDimensionFilter",
  "className":"io.wizzie.normalizer.funcs.impl.NumberComparingFilter",
  "properties": {
    "firstComparableDimension": "ValueToCompare",
    "secondComparable": "MaxValue",
    "condition": "GREATER"
  }
},
{
  "name":"lowerThanDimensionFilter",
  "className":"io.wizzie.normalizer.funcs.impl.NumberComparingFilter",
  "properties": {
    "firstComparableDimension": "ValueToCompare",
    "secondComparable": "MaxValue",
    "condition": "LOWER"
  }
},
{
  "name":"defaultConditionFilter",
  "className":"io.wizzie.normalizer.funcs.impl.NumberComparingFilter",
  "properties": {
    "firstComparableDimension": "ValueToCompare",
    "secondComparable": 2.5
  }
}

And we have the next JSON message:

{
  "ValueToCompare": "3.5",
  "MaxValue": 7.05,
  "required": true
}

The results for each filter are the next:

greaterThanNumberFilter returns true because ValueToCompare is greater than 2.5.
lowerThanNumberFilter returns false because ValueToCompare is greater than 2.5
greaterThanDimensionFilter returns false because ValueToCompare is lower than MaxValue.
lowerThanDimensionFilter returns true because ValueToCompare is lower than MaxValue.
defaultConditionFilter returns false because ValueToCompare is lower than MaxValue. In this filter by default the condition is GREATER.

If we have the next JSON message:

{
  "ValueToCompare": "2.5",
  "MaxValue": 2.5,
  "required": true
}

All the filters return false due to ValueToCompare isn’t greater or lower than 2.5 or MaxValue.

By default if the specified dimension in the property firstComparableDimension or secondComparable doesn’t contain a number value (i.e: string, boolean or null) the NumberComparingFilter always returns false.