Streams API ¶

Skeleton ¶

Examples ¶

<stream
    id="some-family-stream"
    bucket="family(648ab40ba34e015fa57ee579, {{page.section.id}}, $$any, $$none, $$empty, None)"
    use_pins="true"
    reverse_order="false"
/>
<posts
    source="stream"
    stream_id="some-family-stream"
    ...
/>

<stream
    id="some-family-stream"
    bucket="family(648ab40ba34e015fa57ee579, {{page.section.id}}, $$any, $$none, $$empty, None)"
    use_pins="false"
    reverse_order="true"
>
    <period type="window" size="7" unit_in_seconds="{{ 86400 }}" />
/>
<posts
    source="stream"
    stream_id="some-family-stream"
    ...
/>

<stream
    id="some-family-stream"
    bucket="family(648ab40ba34e015fa57ee579, {{page.section.id}}, $$any, $$none, $$empty, None)"
>
    <period type="lifetime" />
/>
<posts
    source="stream"
    stream_id="some-family-stream"
    ...
/>

<posts
    source="stream"
    bucket="family(648ab40ba34e015fa57ee579, {{page.section.id}}, $$any, $$none, $$empty, None)"
    use_pins="false"
    ...
/>

Attributes ¶

stream.id - the stream configuration ID which is unique to the layouts
stream.bucket - the stream bucket reference
- family(family_id, arg1, arg2, arg3…) – references a particular bucket in a family
stream.use_pins - whether to use pins or not (default: false)
stream.period.type – the type of period to use for the stream
- window – a window of time
  - stream.period.size – the size of the window in units
  - stream.period.unit_in_seconds – the unit of time in seconds
- lifetime – the lifetime
posts.stream_id – the stream configuration ID to use for the posts
posts.bucket – the stream bucket reference to use for the posts (@see stream.bucket)
posts.use_pins - whether to use pins or not (@see stream.use_pins)

Endpoints ¶

Get Families ¶

GET /core/v1/streams/families

200 OK¶

{
  "families": [
    {
      "id": "648ab3c9522b443bd06774d1",
      "title": null,  // or "Some interesting posts [v1.0]",
      "entity": <entity>,
      "status": <family status>,
      "filter": null,  // or <family filter configuration>
      "scores": null  // or <family scores configuration>
    }
  ]
}

Hints:

<entity> (see: Entities)
<family status> (see: Family Statuses)
<family filter configuration> (see: Family Filter Configuration)
<family scores configuration> (see: Family Scores Configuration)

Create Family ¶

POST /core/v1/streams/families

{
  "entity": <entity>,
  "commands": [
    <family command description>,
    <family command description>,
    ...
  ]
}

200 OK¶

{
  "family": {
    "id": "648ab3c9522b443bd06774d1",
    "title": null,
    "entity": <entity>,
    "status": <family status>,
    "filter": null,  // or <family filter configuration>
    "scores": null  // or <family scores configuration>
  }
}

Hints:

<entity> (see: Entities)
<family command description> (see: Family Commands)
<family status> (see: Family Statuses)
<family filter configuration> (see: Family Filter Configuration)
<family scores configuration> (see: Family Scores Configuration)

Update Family ¶

PUT /core/v1/streams/families/648ab3c9522b443bd06774d1

{
  "commands": [
    <family command description>,
    <family command description>,
    ...
  ]
}

200 OK¶

{
  "family": {
    "id": "648ab3c9522b443bd06774d1",
    "title": null,  // or "Some interesting posts [v1.0]",
    "entity": <entity>,
    "status": <family status>,
    "filter": null,  // or <family filter configuration>
    "scores": null  // or <family scores configuration>
  }
}

Hints:

<entity> (see: Entities)
<family command description> (see: Family Commands)
<family status> (see: Family Statuses)
<family filter configuration> (see: Family Filter Configuration)
<family scores configuration> (see: Family Scores Configuration)

Reindex Family ¶

PUT /core/v1/streams/families/648ab3c9522b443bd06774d1/reindex?should_reset_cache=true

GET /core/v1/streams/families/648ab3c9522b443bd06774d1/reindex?task_id=38e9eea7-fc5b-4373-90f0-cb2d59109113

200 OK¶

{
  "task": {
    "id": "38e9eea7-fc5b-4373-90f0-cb2d59109113",
    "is_ready": true,
    "progress": null,  // or same as "response"
    "response": {
      "entities": {
        "total_count": 12343,
        "indexed_count": 12343
      }
    },
    "exception": null
}

Delete Family ¶

DELETE /core/v1/streams/families/648ab3c9522b443bd06774d1

200 OK¶

{}

Find Stream Records ¶

GET /core/v1/streams/records?bucket=<bucket string>&limit=<limit>[&offset=<offset>[&cursor=<string>[&mode=default|ignore-pins|ignore-pins-and-reverse-order]]]

200 OK¶

{
  "records": [
    {
      "entity": <entity>,
      "cursor": <string>  // or `null` if the entity is pinned in this position
    },
    ...
  ]
}

Hints:

<entity> (see: Entities)
<bucket string> (e.g., family(648ab40ba34e015fa57ee579, <some_section_id>, $$any, $$none, $$empty, None))

Count Stream Records ¶

GET /core/v1/streams/records/count?bucket=<bucket string>

200 OK¶

{
  "count": 592
}

Hints:

<bucket string> (e.g., family(648ab40ba34e015fa57ee579, <some_section_id>, $$any, $$none, $$empty, None))

Update Stream Records ¶

POST /core/v1/streams/records?bucket=<bucket string>

200 OK¶

{
  "commands": [
    <stream command>,
    <stream command>,
    ...
  ]
}

Hints:

<bucket string> (e.g., family(648ab40ba34e015fa57ee579, <some_section_id>, $$any, $$none, $$empty, None))
<stream command> (see: Stream Commands)

Families ¶

Statuses ¶

{"paused": {}} – the family is paused. Allows to configure the family and remove it
{"active": {}} – the family is active. Allows entities to be routed by the family configuration into appropriate buckets in the required order

Commands ¶

Configure family
- Set Title
  {"set-title": "Some less interesting posts [v1.1]"}
- Set Filter
  {"set-filter": <family filter configuration>}
- Set Scores
  {"set-scores": <family scores configuration>}
Manage family status
- Activate
  {"activate": {}}
- Pause
  {"pause": {}}}

Hints:

<family filter configuration> (see: Family Filter Configuration)
<family scores configuration> (see: Family Scores Configuration)

Filters ¶

Taxonomy¶

{"taxonomy": {
    variables: [
        <variable configuration>,
        <variable configuration>,
        ...
    ],
    filters: [
        <filter>,
        <filter>,
        <filter>,
        ...
    ]
}}

Variables¶

{
  "alias": null,  // or "some_alias", not implemented yet
  "scope": <scope>,
  "yields": [
    "<yield>",
    "<yield>",
    ...
  ]
}

Yields:

"$any" – route the entity to buckets where this variable is ignored (e.g. post.sections not empty –> an $any bucket)
"$none" – route the entity to buckets where this variable is either not set or represent an empty lists (e.g. post.sections is empty –> a $none bucket)
"$each" – route the entity to buckets where this variable takes one of the values from the variable scope (e.g. post.sections –> N section_id buckets)

Scopes:

{"post-authors": {}} – yields indexed post author IDs
{"post-communities": {}} – yields indexed post community IDs
{"post-primary-tags": {}} – yields indexed post primary tag slug
{"post-regular-tags": {}} – yields indexed post regular tag slugs
{"post-primary-sections": {}} – yields indexed post primary section ID
{"post-regular-sections": {}} – yields indexed post regular section IDs
{"post-custom-field-values": {"path": "path.to.field"}} – yields indexed post custom field values
{"site-sections": {"filters": [<section filter>, <section filter>, ...]}} – yields indexed post primary topic ID
{"manual": {"state": <variable state>}} – yields manually added values

States:

{"tags": {"slugs": [<tag_slug>, <tag_slug>, ...]}} – yields manually added tag slugs
{"sections": {"ids": [<section_id>, <section_id>, ...]}} – yields manually added section IDs
{"scalars": {"type": "str" | "int" | "float", "values": [<value>, <value>, ...]}} – yields manually added scalar values

Filters¶

Variable filters allow to distribute entities into buckets based on the values of the variables. The following variable filters are supported:

Resolve variable reference to a set of taxonomy filters

{"$switch": {
  "$ref": "$$1", // or "$some_alias" (not implemented yet),
  "$cases": {
    "$any": [ // an entity goes to this bucket no matter the variable value
      <variable taxonomy filter>,
      ...
    ],
    "$none": [ // an entity goes to this bucket if the variable value is not set or is an empty list
      <variable taxonomy filter>,
      ...
    ],
    "$each": [ // an entity goes to a bucket for each value the variable yields
      {"any_label": [
        {"$return": {"value": {
          "field": <taxonomy field>,
          "value": "$$1"  // or "$some_alias" (not implemented yet)
        }}},
        <variable taxonomy label>,
        ...
      ]},
      {"any_range": [
        {"$return": {
          "field": <taxonomy field>,
          "min": "$$1[1]",  // or "$some_alias[1]" (not implemented yet)
          "max": "$$1[2]"  // or "$some_alias[2]" (not implemented yet)
        }},
        <variable taxonomy range>,
        ...
      ]},
      <variable taxonomy filter>,
      ...
    ]
  }
}}

Return a set of taxonomy filters without applying any variables

{"$return": [
  <variable taxonomy filter>,
  <variable taxonomy filter>,
  ...
]}

Hints:

<variable taxonomy filter> (e.g., {“any_label”: {“$return”: [<taxonomy label>, …]}})
<variable taxonomy label> (e.g., {“$return”: <taxonomy label>})
<variable taxonomy range> (e.g., {“$return”: <taxonomy range>})
<taxonomy filter> (see: Filters)
<taxonomy label> (see: Labels)
<taxonomy range> (see: Ranges)
<taxonomy field> (see: Fields)

Scores ¶

Static¶

{
    "static": {
        "mode": <mode>,
        "field": <taxonomy field>,
        "scale": null  // or a float number to scale the score taken from the entity field
    }}
}

Modes:

{"init": {}} – when an entity indexed for the first time calculate the value and never update it
{"track": {}} – when an entity indexed for the first time calculate the value and update it when the entity is updated
{"track-until-moved": {}} – when an entity indexed for the first time calculate the value and update it when the entity is updated. Stop tracking the score for a particular bucket as soon as the entity is manually moved in that bucket

Incremental¶

{
    "incremental": {
        "period": <period>,
        "weights": [
            {"event": <taxonomy-event>, "weight": <weight>}
        ]
    }}
}

Periods:

{"window": {"size": 7, "unit_in_seconds": 86400}} – a window of time, e.g. 7 days
{"lifetime": {}} – the lifetime

Hints:

<taxonomy field> (see: Fields)
<taxonomy event> (see: Events)

Streams ¶

Commands ¶

Pin entity
{"pin": {"entity": <entity>, "position": <position>}}
Unpin entity
{"unpin": {"entity": <entity>}}
Move entity to position
{"move-to": {"entity": <entity>, "position": <position>}}
Move entity after another entity
{"move-after": {"entity": <entity>, "other_entity": <entity>}}

Hints:

<entity> (see: Entities)
<position> – a numeric rank in the stream starting with 0

Streams API ¶

Skeleton ¶

Examples ¶

Attributes ¶

Endpoints ¶

Get Families ¶

200 OK¶

Create Family ¶

200 OK¶

Update Family ¶

200 OK¶

Reindex Family ¶

200 OK¶

Delete Family ¶

200 OK¶

Find Stream Records ¶

200 OK¶

Count Stream Records ¶

200 OK¶

Update Stream Records ¶

200 OK¶

Families ¶

Statuses ¶

Commands ¶

Filters ¶

Taxonomy¶

Variables¶

Filters¶

Scores ¶

Static¶

Incremental¶

Streams ¶

Commands ¶

Navigation

Related Topics