Connecting mosaik and Odysseus

Odysseus is a framework for in-memory data stream management that is designed for online processing of big data. Large volumes of data such as continuously occurring events or sensor data can be processed in real time. In combination with mosaik Odysseus can be used to process, visualise and store the results of mosaik during a simulation.

In this first part of the tutorial we cover the two ways to connect mosaik and Odysseus, the second part is about how to use Odysseus to process, visualize and store simulation data.

Note

Connecting mosaik and Odysseus works mosaik >= 3.0

Note

Connecting mosaik and Odysseus works only with mosaik >= 3.0

You can choose between two different solutions to connect mosaik and Odysseus. Both have their advantages and disadvantages and therefore, the right choice depends on your use case. We recommend to use the SimAPI version for beginners.

No matter which connection we use, we first have to download Odysseus Server and Studio Client. For the first start of Odysseus Studio the default user “System” and password “manager” have to be used, the tenant can be left empty.

Connecting via mosaik protocol handler

The easiest way to connect to mosaik is to use the mosaik protocol handler in Odysseus, which is available as installable feature in Odysseus Studio. It uses the mosaik API through remote procedure calls (RPC) and offers a close coupling of mosaik and Odysseus. With this, a blocked simulation in mosaik or a blocked processing in Odysseus will block the other system as well. If this is a problem in your use case, you should look in the section Connecting via ZeroMQ.

First we have to install the mosaik feature from the incubcation site in odysseus, which can be found in the Odysseus Wrapper Plugins.

After installing the feature we create a new Odysseus project and in the project a new Odysseus script file (more information on Odysseus projects and script files can be found in this tutorial). To use mosaik as source we can use the mosaik operator which contains a standard configuration of mandatory parameters. The script-code in the Odysseus query language PQL looks like this:

#PARSER PQL
#METADATA TimeInterval
#QUERY
mosaikCon := MOSAIK({SOURCE = 'mosaik', type='simapi'})

This is for the standard configuration. If you want to change something, for example to use another port, you need a more detailed configuration:

#PARSER PQL
#METADATA TimeInterval
#QUERY
mosaikCon1 := ACCESS({TRANSPORT = 'TCPServer',
                    PROTOCOL = 'mosaik',
                    SOURCE = 'mosaik',
                    DATAHANDLER = 'KeyValueObject',
                    WRAPPER = 'GenericPush',
                    OPTIONS = [
                      ['port', '5555'],
                      ['mosaikPort', '5554'],
                      ['byteorder', 'LITTLE_ENDIAN']
                    ]})

As we can see the protocol ‘mosaik’ is chosen. When the query is started, the mosaik protocol handler in Odysseus opens a TCP server for receiving data from mosaik.

Before we can receive data, we have to adapt our mosaik scenario. Here we take the mosaik-demo as an example. The Odysseus simulator is treated just like any other component in mosaik. It has to be added to the SIM_CONFIG parameter. For the connection to the simulator the connect command is used and the IP address and port of Odysseus have to be specified:

sim_config = {
    'Odysseus': {
        'connect': '127.0.0.1:5554',
    }

After that, we have to initialize the simulator and connect it to all components whose data we want to revceive in Odysseus. For the mosaik-demo, we have to add the following lines of code to the scenario definition:

    # Start simulators
    odysseusModel = world.start('Odysseus', step_size=60*15)

    # Instantiate models
    odysseus = odysseusModel.Odysseus.create(1)
    ody = odysseus[0]

    # Connect entities
    connect_many_to_one(world, nodes, ody, 'P', 'Vm')
    connect_many_to_one(world, houses, ody, 'P_out')
    connect_many_to_one(world, pvs, ody, 'P')

Now we have set up everything to receive mosaiks data in Odysseus. To begin transfering data we have to start first the query in Odysseus and then the simulation in mosaik.

For more information on how to use Odysseus visit part two.

Connecting via ZeroMQ

In contrast to the close coupling via mosaik protocol handler the coupling via ZeroMQ is more loose. Mosaik sends all data as data stream with ZeroMQ and Odysseus can even be closed and restarted during the simulation without affecting mosaik. This behaviour holds the risk of loosing data so it should only be used if this doesn’t cause problems.

First we have to install the following features for Odysseus from incubation site:

  • Odysseus Wrapper Plugins / Zero MQ

  • Odysseus Wrapper Plugins / mosaik (only if you want to use the mosaik operator)

And from the update site:

  • Odysseus Odysseus_core Plugins / Json Wrapper

After installing the features we create a new Odysseus project and in the project a new Odysseus script file. The messages sent by mosaik are formatted in JSON format and sent via ZeroMQ. So we have to choose the corresponding ZeroMQ transport handler and JSON protocol handler:

#PARSER PQL
#METADATA TimeInterval
#QUERY
mosaikCon3 := ACCESS({TRANSPORT = 'ZeroMQ',
                    PROTOCOL = 'JSON',
                    SOURCE = 'mosaik',
                    DATAHANDLER = 'KeyValueObject',
                    WRAPPER = 'GenericPush',
                    OPTIONS = [
                      ['host', '127.0.0.1'],
                      ['readport', '5558'],
                      ['writeport', '5559'],
                      ['byteorder', 'LITTLE_ENDIAN']
                    ]})

If you use the standard configurtion you can use the short version (feature “wrapper / mosaik” has to be installed):

#PARSER PQL
#METADATA TimeInterval
#QUERY
mosaikCon2 := MOSAIK({SOURCE = 'mosaik', type='zeromq'})

After setting up Odysseus we have to install the mosaik-zmq adapter in our mosaik virtualenv. It is available on GitLab and PyPI. To install it we have to activate our mosaik virtualenv and execute (if there are errors during installation have a look in the readme):

pip install mosaik-zmq

The mosaik-zmq adapter is treated in mosaik like any other component of the simulation. If we use the mosaik demo, we have to add the new simulator to the SIM_CONFIG parameter:

sim_config = {
    'ZMQ': {
        'cmd': 'mosaik-zmq %(addr)s',
    },

Also we have to initialize the ZeroMQ simulator and connect it to other components:

    # Start simulators
    zmqModel = world.start('ZMQ', step_size=15*60, duration=END)

    # Instantiate models
    zmq = zmqModel.Socket(host='tcp://*:', port=5558, socket_type='PUB')

    # Connect entities
    connect_many_to_one(world, nodes, zmq, 'P', 'Vm')
    connect_many_to_one(world, houses, zmq, 'P_out')
    connect_many_to_one(world, pvs, zmq, 'P')

For more information on how to use Odysseus visit part two.