==================
Traffic Group Sync
==================

.. sidebar:: Contents

  .. contents::
      :depth: 2
      :local:


This chapter covers some aspects related to the
:osdx:cfg:`service traffic-group-sync *` tool, which allows you to configure one
or multiple **Traffic Group Sync** agents in OSDx.

The traffic-group-sync service enables the automatic updating of traffic groups by
retrieving and processing remote JSON datasets.


Configuration
=============

Each instance can be linked to a different remote **end-point**. The service
retrieves the dataset over HTTP/S and, if applicable, uses its content to update
the traffic groups configured.

This is the syntax to create a :osdx:cfg:`service traffic-group-sync *` instance:

.. code-block:: none

  set service traffic-group-sync <NAME> [ ... ]

Where ``<NAME>`` represents the name of the instance.

When you configure a new instance, you need to at least specify the remote ``url``.

Additional parameters are available, such as the ``poll-interval``, which
defines how often the service attempts to fetch the remote resource.
When using HTTPS, the ``ssl-allow-insecure`` option can be enabled to skip SSL
certificate validation and hostname verification.

By default, the generated traffic is not VRF-aware (meaning the ``main``
VRF will be used). Instances can also be configured to use a **specific VRF**,
via the ``local-vrf`` field.


Dataset format and server configuration
=======================================

The dataset must be a valid JSON file with the following structure: a top-level
map containing zero or more traffic group types (e.g., address, port,
ipv6-address, etc.). Every type should map a set of traffic group names, each
associated with a list of elements (individual values, sets, or ranges).
Overlapping values are allowed within and across groups.

*Example:*

.. code-block:: json

  {
      "address": {
          "LAN_ADDR": ["192.168.1.0/24"],
          "WAN_ADDR": ["10.0.0.2-10.0.0.20"]
      },
      "port": {
          "SYS_PORT": ["1234", "4321"]
      }
  }


The HTTP/S server should tag files using the **ETag header** to prevent
unnecessary downloads by the client service. This mechanism relies on the
**If-Modified-Since** HTTP request header to determine whether the file has
changed since the last fetch or not.

Programs like ``cURL`` can be used to check if the target server supports this
mechanism:

.. code-block:: bash

  admin@PC:~$ curl -v http://my_server/dataset.json
  *   Trying 192.168.215.1:80...
  * Connected to my_server (192.168.215.1) port 80 (#0)
  > GET /dataset.json HTTP/1.1
  > Host: my_server
  > User-Agent: curl/7.81.0
  > Accept: */*
  >
  * Mark bundle as not supporting multiuse
  < HTTP/1.1 200 OK
  < Date: Mon, 21 Apr 2025 09:52:00 GMT
  < Server: Apache/2.4.57 (Debian)
  < Last-Modified: Fri, 21 Mar 2025 12:25:19 GMT
  < ETag: "ea-630d95907ef87"
  < Accept-Ranges: bytes
  < Content-Length: 234
  < Content-Type: application/json
  <
  {
    "address": {
      "TG_d29957": [
        "10.44.44.12",
        "1.1.1.12/32"
      ],
      "TG_0da67e": [
        "32.32.32.32"
      ]
    },
    "mac-address": {
      "TG_d29957": [
        "00:a0:f9:3d:3e:95"
      ],
      "TG_0da67e": [
      ]
    }
  }


Monitoring traffic groups
=========================


We can run the :osdx:op:`service traffic-group-sync * show data`
command to display the latest data acquired.

*Example:*

.. code-block:: none

  admin@osdx$ service traffic-group-sync ACL_FETCHER show data

  ---------------------------------
    Name     Kind       Elements
  ---------------------------------
  LAN_ADDR  address  192.168.1.0/24
  WAN_ADDR  address
  SYS_PORT   port    1234


The :osdx:op:`service traffic-group-sync * show stats` command can be used to
display the statistics for a specific service instance.

*Example:*

.. code-block:: none

  admin@osdx$ service traffic-group-sync ACL_FETCHER show stats

  -----------------------------------------------
  Requests  Updates  Download Errors  Last Update
  -----------------------------------------------
         6        1                3     2s ago


Additionally, the :osdx:op:``traffic group show`` command can be used to
display current elements of the configured traffic groups.

*Example:*

Elements for address traffic groups

.. code-block:: none

  admin@osdx$ traffic group show

  --------------------------------------------------------
    name       elements                 source
  --------------------------------------------------------
  LAN_ADDR  192.168.1.0/24  traffic-group-sync ACL_FETCHER
  WAN_ADDR  10.0.0.2        traffic-group-sync ACL_FETCHER

  Elements for port traffic groups

  ----------------------------------------------------
    name     elements               source
  ----------------------------------------------------
  SYS_PORT  4321, 1234  traffic-group-sync ACL_FETCHER


Some **traffic-group-sync** examples are available in the
:doc:`/examples/service/traffic-group-sync/index` examples page.


.. osdx:cmdtree:: cfg
  :maxdepth: 5

  service traffic-group-sync

.. osdx:cmdtree:: op
  :maxdepth: 5

  service traffic-group-sync