Search API

Endpoints

Get Indexes

GET /core/v1/search/indexes

200 OK

{
  "indexes": [
    {
      "slug": "my-first-index",
      "title": "My First Index",
      "status": "created",
      "mapping": <index mapping>
  ]
}

Hints:

Create/Modify Index

PUT /core/v1/search/indexes/<index slug>
{
  "commands": [
    <command description>,
    <command description>,
    ...
  ]
}

200 OK

{
  "indexes": [
    {
      "slug": "my-second-index",
      "title": "My Second Index",
      "status": "created",
      "mapping": <index mapping>
  ]
}

Hints:

Fill Index with Data

POST /core/v1/search/indexes/my-second-index/index-posts
POST /core/v1/search/indexes/my-second-index/index-users
GET /core/v1/search/indexes/my-second-index/index-posts?task_id=38e9eea7-fc5b-4373-90f0-cb2d59109113
GET /core/v1/search/indexes/my-second-index/index-users?task_id=38e9eea7-fc5b-4373-90f0-cb2d59109113

200 OK

{
  "task": {
    "id": "38e9eea7-fc5b-4373-90f0-cb2d59109113",
    "is_ready": true,
    "progress": null,  // or same as "response"
    "response": {
      "entities": {
        "total_count": 12343,
        "indexed_count": 12343
      }
    },
    "exception": null
}

Find records count

POST core/v1/search/indexes/my-second-index/find-count
{
  "filters": [
    <filter>,
    <filter>,
    ...
  ]
}

200 OK

{
  "records_count": 41234
}

Hints:

Find records

POST core/v1/search/indexes/my-second-index/find
{
  "filters": [
    <filter>,
    <filter>,
    ...
  ],
  "chunk": {
    "order": <order>,
    "limit": 10,
    "offset": 0,
    "cursor": <null or cursor>
  }
}

200 OK

{
  "records": [
    {"id": <taxonomy guid>, "cursor": "cursor-1"},
    {"id": <taxonomy guid>, "cursor": "cursor-2"},
    ...
  ]
}

Hints:

Commands

The commands allow to navigate an index through its lifecycle:

  1. Create index
    • Update index title
      {“set-title”: “My Second Index”}
    • Update index mapping:
      {“set-mapping”: <index mapping>}
  2. Start index. It’s not possible to change the mapping after an index is started
    {“start”: {}}
    • Fill the index with data
  3. Activate index. It makes it available for reads
    {“activate”: {}}
    • Use index for searching
  4. Stop index. It’s not possible to read or write from it anymore but the data is still stored
    {“stop”: {}}
  5. Remove index. It removes the index with all its data
    {“remove”: {}}

Hints:

Mappings

{
  "root": <taxonomy entity type>,
  "texts": [
    {
      "weight": 8,
      "analyzer": <text analyzer configuration>,
      "extractor": <text extractor configuration>,
    },
    ...
  ],
  "values":[
    {"type": <value type>, "field": <taxonomy field>},
    {"type": <value type>, "field": <taxonomy field>},
    ...
  ],
  "labels": [
    {"field": <taxonomy field>},
    {"field": <taxonomy field>},
    ...
  ]
}

Hints:

Text Analyzers

Text analysis is the process of converting unstructured text, like the body of an email or a product description, into a structured format that’s optimized for search.

  • For full-text search
    {“unicode”: {}}
  • For full-text search by English words
    {“english”: {}}
  • For exact case-insensitive search by words
    {“keyword”: {“normalizers”: [{“lowercase”: {}}]}}
  • For search by a case-insensitive substring with the length at least of 3 symbols (tri-gram tokens)
    {“ngram”: {“size”: 3, “normalizers”: [{“uppercase”: {}}]}}

Text Extractors

Text extraction is the process of retrieving and combining texts from indexed entities. This text then goes through text analysis process before being stored in the index.

  • Take an entity text as it is

    {“basic”: {“field”: <taxonomy field>}}

  • Combine text from a sequence of sub-extractors
    {
      "composite": {
        "extractors": [
          <text extractor>,
          <text extractor>,
          ...
        ],
      }
    }
    
  • Extract text based on a condition
    {
      "conditioned": {
        "condition": {"taxonomy": {"filters": [
          <taxonomy filter>,
          <taxonomy filter>,
          ...
        ]}},
        "then_extractor": <text extractor or null>,
        "else_extractor": <text extractor or null>
      }
    }
    

Hints:

  • <taxonomy field> (see: Fields)
  • <taxonomy filter> (see: Filters)

Value Types

Value fields used for both sorting results as well as filtering by range. Therefore, it can only support some simple types:

  • Strings
    “str”
  • Numbers
    “int”
  • Numbers with floating point
    “float”

Filters

  • Include only records that match the phrase with default syntax and text weights
    {
      "phrase": {
        "phrase": "Hello World"
      }
    }
    
  • Include only records that match the strict syntax query only in the 2nd text field
    {
      "phrase": {
        "phrase": "\"Hello World\"",
        "syntax": {"strict": {}},
        "weights": [0, 10, 0]
      }
    }
    
  • Include only records that match labels and ranges
    {
      "taxonomy": <taxonomy filter>
    }
    

Hints:

Orders

  • Sort by one of the value fields
    {“value”: {“field”: <taxonomy field>, “reverse”: true}}
  • Sort by relevance to the phrase
    {“relevance”: {}}
  • Sort by relevance to the phrase penalizing “older” records relevance score
    {“relevance”: {“decay”: {“field”: <timestamp taxonomy field>}}}

Hints:

  • <taxonomy field> (see: Fields)