Speedup the wildcard query search for variable values and variable string filters

XMLWordPrintable

    • Type: Task
    • Resolution: Unresolved
    • Priority: L3 - Default
    • None
    • Affects Version/s: None
    • Component/s: backend
    • Not defined

      Context:
      To provide autocompletion in the string variable modal and also allow to filter for variable contains/not contains, we use a wildcard query to filter for right variable values (if the given value is larger than 10). However, this has several downsides:

      • the ngram analyzer, which is used for variables with values less than 10 signs, will explode for large variable values since it creates all possible combinations of 10 signs
      • the wildcard query has a prefix wildcard (e.g. *myValue), which is not recommended by Elastic since it slows down the query significantly.

      Instead, we should:

      • decrease the value of the ngram analyzer to 5 to mitigate the explosion problem.
      • have an additional multivalue field which is the reversed version of the current value. Then you can do two a postfix wildcard query (e.g. eulaVym*) for the initial prefix query (e.g. *myValue)

      Hint:

      • We might need to think about how to reduce the ngram value in the upgrade script

            Assignee:
            Unassigned
            Reporter:
            Johannes
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: