Search API

Endpoints

Get Indexes

GET /core/v1/search/indexes

200 OK

{
  "indexes": [
    {
      "slug": "my-first-index",
      "title": "My First Index",
      "status": "created",
      "mapping": <index mapping>
  ]
}

Hints:

Create/Modify Index

PUT /core/v1/search/indexes/<index slug>

{
  "commands": [
    <command description>,
    <command description>,
    ...
  ]
}

200 OK

{
  "indexes": [
    {
      "slug": "my-second-index",
      "title": "My Second Index",
      "status": "created",
      "mapping": <index mapping>
  ]
}

Hints:

Fill Index with Data

POST /core/v1/search/indexes/my-second-index/index-posts

POST /core/v1/search/indexes/my-second-index/index-users

GET /core/v1/search/indexes/my-second-index/index-posts?task_id=38e9eea7-fc5b-4373-90f0-cb2d59109113

GET /core/v1/search/indexes/my-second-index/index-users?task_id=38e9eea7-fc5b-4373-90f0-cb2d59109113

200 OK

{
  "task": {
    "id": "38e9eea7-fc5b-4373-90f0-cb2d59109113",
    "is_ready": true,
    "progress": null,  // or same as "response"
    "response": {
      "entities": {
        "total_count": 12343,
        "indexed_count": 12343
      }
    },
    "exception": null
}

Find records count

POST core/v1/search/indexes/my-second-index/find-count

{
  "filters": [
    <filter>,
    <filter>,
    ...
  ]
}

200 OK

{
  "records_count": 41234
}

Hints:

Find records

POST core/v1/search/indexes/my-second-index/find

{
  "filters": [
    <filter>,
    <filter>,
    ...
  ],
  "chunk": {
    "order": <order>,
    "limit": 10,
    "offset": 0,
    "cursor": <null or cursor>
  }
}

200 OK

{
  "records": [
    {"id": <taxonomy guid>, "cursor": "cursor-1"},
    {"id": <taxonomy guid>, "cursor": "cursor-2"},
    ...
  ]
}

Hints:

Commands

The commands allow to navigate an index through its lifecycle:

  1. Create index
    • Update index title

      {"set-title": "My Second Index"}

    • Update index mapping:

      {"set-mapping": <index mapping>}

  2. Start index. It’s not possible to change the mapping after an index is started

    {"start": {}}

    • Fill the index with data

  3. Activate index. It makes it available for reads

    {"activate": {}}

    • Use index for searching

  4. Stop index. It’s not possible to read or write from it anymore but the data is still stored

    {"stop": {}}

  5. Remove index. It removes the index with all its data

    {"remove": {}}

Hints:

Mappings

{
  "root": <taxonomy entity type>,
  "texts": [
    {
      "weight": 8,
      "analyzer": <text analyzer configuration>,
      "extractor": <text extractor configuration>,
    },
    ...
  ],
  "values":[
    {"type": <value type>, "field": <taxonomy field>},
    {"type": <value type>, "field": <taxonomy field>},
    ...
  ],
  "labels": [
    {"field": <taxonomy field>},
    {"field": <taxonomy field>},
    ...
  ]
}

Hints:

Text Analyzers

Text analysis is the process of converting unstructured text, like the body of an email or a product description, into a structured format that’s optimized for search.

  • For full-text search

    {"unicode": {}}

  • For full-text search by English words

    {"english": {}}

  • For exact case-insensitive search by words

    {"keyword": {"normalizers": [{"lowercase": {}}]}}

  • For search by a case-insensitive substring with the length at least of 3 symbols (tri-gram tokens)

    {"ngram": {"size": 3, "normalizers": [{"uppercase": {}}]}}

Text Extractors

Text extraction is the process of retrieving and combining texts from indexed entities. This text then goes through text analysis process before being stored in the index.

  • Take an entity text as it is

    {"basic": {"field": <taxonomy field>}}

  • Combine text from a sequence of sub-extractors
    {
  "composite": {
    "extractors": [
      <text extractor>,
      <text extractor>,
      ...
    ],
  }
}
  • Extract text based on a condition
    {
  "conditioned": {
    "condition": {"taxonomy": {"filters": [
      <taxonomy filter>,
      <taxonomy filter>,
      ...
    ]}},
    "then_extractor": <text extractor or null>,
    "else_extractor": <text extractor or null>
  }
}

Hints:

  • <taxonomy field> (see: Fields)

  • <taxonomy filter> (see: Filters)

Value Types

Value fields used for both sorting results as well as filtering by range. Therefore, it can only support some simple types:

  • Strings

    "str"

  • Numbers

    "int"

  • Numbers with floating point

    "float"

Filters

  • Include only records that match the phrase with default syntax and text weights
    {
  "phrase": {
    "phrase": "Hello World"
  }
}
  • Include only records that match the strict syntax query only in the 2nd text field
    {
  "phrase": {
    "phrase": "\"Hello World\"",
    "syntax": {"strict": {}},
    "weights": [0, 10, 0]
  }
}
  • Include only records that match labels and ranges
    {
  "taxonomy": <taxonomy filter>
}

Hints:

Orders

  • Sort by one of the value fields

    {"value": {"field": <taxonomy field>, "reverse": true}}

  • Sort by relevance to the phrase

    {"relevance": {}}

  • Sort by relevance to the phrase penalizing “older” records relevance score

    {"relevance": {"decay": {"field": <timestamp taxonomy field>}}}

Hints:

  • <taxonomy field> (see: Fields)