Cluster Configuration - Storm Streaming Server

However, for the purposes of this guide, we will assume that you have three server instances running on separate machines, with each instance hosting only a single application.

Server Structure

We will have a total of three servers, which we will name Alfa, Beta, and Gamma for easier identification.

The Alfa server will host both the origin application and an active ClusterManager module. The remaining two servers will have this module deactivated.

ClusterManager (Alfa Server)

First, we need to activate the ClusterManager module in the configuration of the Alfa server. This module is located in the main block of the config/preferences.xml file:

config/preferences.xml - ClusterManager on Alfa

xml

Copy

<ClusterManager enabled="true">
    <Identity>Primary</Identity>
    <GlobalStatistics>true</GlobalStatistics>
    <Authentication>
        <IPWhiteList>192.168.10.2, 192.168.10.3, 192.168.10.4</IPWhiteList>
        <Secret>X9fT7mLq2Z!</Secret>
    </Authentication>
    <ClusterTranscoding enabled="true" active="true">
        <MinimalSourceResolution>1280x720</MinimalSourceResolution>
        <StreamGraceTimeout>300</StreamGraceTimeout>
        <StreamEvaluationInterval>30</StreamEvaluationInterval>
        <AliasMode>false</AliasMode>
        <StreamWeightOverrides enabled="false" />
    </ClusterTranscoding>
</ClusterManager>

The <Identity> field helps identify which ClusterManager you are dealing with (e.g. Primary, Secondary). The <Secret> is the password that all applications must present when connecting. The <IPWhiteList> restricts access to only the IPs of your cluster nodes.

For a full description of all ClusterManager parameters, including the weight system and AWS integration, see the ClusterManager Settings guide.

Origin (Alfa Server)

The origin application accepts incoming streams and reports their status to the ClusterManager. The most important block here is <StreamingClusterSettings>, which defines the connection with the ClusterManager.

config/preferences.xml - Origin application on Alfa

xml

Copy

<Application name="origin" type="origin">
    <StreamingClusterSettings>
        <SupervisorList>
            <ClusterManager hostname="192.168.10.2" port="443" isSSL="true">
                <Priority>1</Priority>
                <Secret>X9fT7mLq2Z!</Secret>
            </ClusterManager>
        </SupervisorList>
        <NodeAccessSettings hostname="192.168.10.2" port="1935" isSSL="false" />
        <NodePriority>10</NodePriority>
    </StreamingClusterSettings>
    <!-- other configuration blocks -->
</Application>

The <NodeAccessSettings> block tells the ClusterManager the public address of this node, so that edge and transcode applications know where to connect. The <NodePriority> value influences load-balanced routing when multiple origin nodes are available.

For the complete list of origin application settings, see the Origin Application guide. For details on the <StreamingClusterSettings> block, see Streaming Cluster Settings.

Transcode (Beta Server)

The transcode application connects to the ClusterManager, discovers streams on origin nodes, pulls them, and encodes them into configured quality presets.

config/preferences.xml - Transcode application on Beta

xml

Copy

<Application name="transcode" type="transcode">
    <StreamingClusterSettings>
        <SupervisorList>
            <ClusterManager hostname="192.168.10.2" port="443" isSSL="true">
                <Priority>1</Priority>
                <Secret>X9fT7mLq2Z!</Secret>
            </ClusterManager>
        </SupervisorList>
        <NodeAccessSettings hostname="192.168.10.3" port="1935" isSSL="false" />
        <NodePriority>10</NodePriority>
    </StreamingClusterSettings>
    <LiveTranscoding enabled="true">
        <Preset name="480p" />
        <Preset name="720p" />
    </LiveTranscoding>
    <!-- other configuration blocks -->
</Application>

For the complete list of transcode application settings, see the Transcode Application guide.

Edge (Gamma Server)

The edge application copies streams from origin or transcode nodes and distributes them to viewers. Its cluster configuration is similar to origin, with one key difference — instead of <NodePriority>, it uses <CopyOnPublish>.

config/preferences.xml - Edge application on Gamma

xml

Copy

<Application name="edge" type="edge">
    <StreamingClusterSettings>
        <SupervisorList>
            <ClusterManager hostname="192.168.10.2" port="443" isSSL="true">
                <Priority>1</Priority>
                <Secret>X9fT7mLq2Z!</Secret>
            </ClusterManager>
        </SupervisorList>
        <NodeAccessSettings hostname="192.168.10.4" port="1935" isSSL="false" />
        <CopyOnPublish>true</CopyOnPublish>
    </StreamingClusterSettings>
    <!-- other configuration blocks -->
</Application>

When <CopyOnPublish> is set to true, streams are immediately pulled to this edge when published on the origin, ensuring zero delay for the first viewer. When false, the edge waits until a viewer requests the stream before pulling it.

For the complete list of edge application settings, see the Edge Application guide.

Redundancy

A single server instance can only host one ClusterManager. However, for each application, it is possible to define connections to multiple ClusterManagers located on different servers. This ensures that if the primary ClusterManager fails, the cluster continues to operate without interruption. When the primary regains functionality, all applications reconnect to it.

Multiple ClusterManagers for redundancy

xml

Copy

<StreamingClusterSettings>
    <SupervisorList>
        <ClusterManager hostname="192.168.10.2" port="443" isSSL="true">
            <Priority>1</Priority>
            <Secret>X9fT7mLq2Z!</Secret>
        </ClusterManager>
        <ClusterManager hostname="192.168.10.10" port="443" isSSL="true">
            <Priority>2</Priority>
            <Secret>X9fT7mLq2Z!</Secret>
        </ClusterManager>
    </SupervisorList>
    <CopyOnPublish>true</CopyOnPublish>
</StreamingClusterSettings>

The application connects to the ClusterManager with the lowest priority value first. If that connection fails, it falls back to the next one.