Events and file database for isolated testing

General Development
1

I’m trying to use a file database to run isolated unit tests. This is essentially what GitHub - vxgmichel/pytango-devicetest: Unit testing for TANGO devices (deprecated) does behind the scenes. The documentation from pytango-devicetest claims that:

Tango events are not supported by the --file execution mode. See the Tango
documentation for further information.

However, the Tango documentation says:

D.5.2 For release 8 and above

A new event system has been implemented starting with Tango release 8. With this new event system, the CORBA Notification service is not needed any more. This means that as soon as all clients using events from devices embedded in the device server use Tango 8, it is not required to start any process other than the device server and its clients.

Unfortunately, it seems that in practice the pytango-devicetest readme seems to be accurate in spite of what the tango documentation says. It seems that events don’t work when a file database is used.

I’m not sure if I’m doing anything wrong, so I placed a minimal testcase in this gist: Minimal test of tango events an file database using PyTango · GitHub

build info:

python -c “from PyTango.utils import info; print(info())”

PyTango 9.2.0 (9, 2, 0, ‘b’, 1)
PyTango compiled with:
Python : 2.7.6
Numpy : 0.0.0
Tango : 9.2.2
Boost : 1.55.0

PyTango runtime is:
Python : 2.7.6
Numpy : None
Tango : 9.2.2
Boost : 0.0.0

PyTango running on:
(‘Linux’, ‘tango-test’, ‘4.2.0-30-generic’, ‘#36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016’, ‘x86_64’, ‘x86_64’)

Additional details: Built PyTango from pytango9 branch from github, and used self-made tango .debs that were ported from the current debian packages to build on Ubuntu 14.04.

Using the database we run the server as:

python PolledDS.py  --register test/withdb/polled
python PolledDS.py polled

and client as:

python PolledDS_client.py 'test/withdb/polled'

This results in a regular stream of event updates:

event_type: change val: True time: 1462459184.51437 received_time: 1462459184.88911
event_type: periodic val: True time: 1462459184.51437 received_time: 1462459184.92293
event_type: periodic val: True time: 1462459185.51359 received_time: 1462459185.51384
event_type: periodic val: True time: 1462459186.51388 received_time: 1462459186.51412
event_type: periodic val: True time: 1462459187.51414 received_time: 1462459187.51431
event_type: periodic val: True time: 1462459188.51441 received_time: 1462459188.51461
event_type: periodic val: True time: 1462459189.51402 received_time: 1462459189.51462

Running with a file database, we run the server as:

python PolledDS.py polled -ORBendPoint giop:tcp::12345 -file=tango.db

and client as:

python PolledDS_client.py 'localhost:12345/test/nodb/polled#dbase=no'

Then we get the initial event update, but after that, only:

2016-05-05 14:37:53,806 - root - ERROR - PolledDS_client - PolledDS_client.py : 36 - Exception while handling event, event_data: EventData[
attr_name = ‘tango://localhost:12345/test/nodb/polled/scalarbool#dbase=no’
attr_value = None
device = Polled(test/nodb/polled)
err = True
errors = (DevError(desc = ‘Event channel is not responding anymore, maybe the server or event system is down’, origin = ‘EventConsumer::KeepAliveThread()’, reason = ‘API_EventTimeout’, severity = PyTango._PyTango.ErrSeverity.ERR),)
event = ‘change’
reception_date = TimeVal(tv_nsec = 0, tv_sec = 1462459073, tv_usec = 804448)]
Traceback (most recent call last):
File “PolledDS_client.py”, line 29, in printer
value = attr_value.value
AttributeError: ‘NoneType’ object has no attribute ‘value’
event_type: idl5_change val: True time: 1462459073.60559 received_time: 1462459073.80705
2016-05-05 14:37:53,807 - root - ERROR - PolledDS_client - PolledDS_client.py : 36 - Exception while handling event, event_data: EventData[
attr_name = ‘tango://localhost:12345/test/nodb/polled/scalarbool#dbase=no’
attr_value = None
device = Polled(test/nodb/polled)
err = True
errors = (DevError(desc = ‘Event channel is not responding anymore, maybe the server or event system is down’, origin = ‘EventConsumer::KeepAliveThread()’, reason = ‘API_EventTimeout’, severity = PyTango._PyTango.ErrSeverity.ERR),)
event = ‘periodic’
reception_date = TimeVal(tv_nsec = 0, tv_sec = 1462459073, tv_usec = 807174)]
Traceback (most recent call last):
File “PolledDS_client.py”, line 29, in printer
value = attr_value.value
AttributeError: ‘NoneType’ object has no attribute ‘value’
2016-05-05 14:38:03,813 - root - ERROR - PolledDS_client - PolledDS_client.py : 36 - Exception while handling event, event_data: EventData[
attr_name = ‘tango://localhost:12345/test/nodb/polled/scalarbool#dbase=no’
attr_value = None
device = Polled(test/nodb/polled)
err = True
errors = (DevError(desc = ‘Event channel is not responding anymore, maybe the server or event system is down’, origin = ‘EventConsumer::KeepAliveThread()’, reason = ‘API_EventTimeout’, severity = PyTango._PyTango.ErrSeverity.ERR),)
event = ‘change’
reception_date = TimeVal(tv_nsec = 0, tv_sec = 1462459083, tv_usec = 813407)]
Traceback (most recent call last):
File “PolledDS_client.py”, line 29, in printer
value = attr_value.value
AttributeError: ‘NoneType’ object has no attribute ‘value’
event_type: idl5_change val: True time: 1462459083.60535 received_time: 1462459083.81440
2016-05-05 14:38:03,814 - root - ERROR - PolledDS_client - PolledDS_client.py : 36 - Exception while handling event, event_data: EventData[
attr_name = ‘tango://localhost:12345/test/nodb/polled/scalarbool#dbase=no’
attr_value = None
device = Polled(test/nodb/polled)
err = True
errors = (DevError(desc = ‘Event channel is not responding anymore, maybe the server or event system is down’, origin = ‘EventConsumer::KeepAliveThread()’, reason = ‘API_EventTimeout’, severity = PyTango._PyTango.ErrSeverity.ERR),)
event = ‘periodic’
reception_date = TimeVal(tv_nsec = 0, tv_sec = 1462459083, tv_usec = 814496)]
Traceback (most recent call last):
File “PolledDS_client.py”, line 29, in printer
value = attr_value.value
AttributeError: ‘NoneType’ object has no attribute ‘value’

2

Hi Neilen,

I have tried here with one of my test device server. I am using C++ and Tango 9.2.2 with patch applied.
I start my DS in a similar way than yours after a property file generation from Jive (“Save servver data”)
In my device server, I have one device with the state being polled at 3 sec.

On the client side, I have a small client listening on periodic event. Please note that the device name is
tango://myhost:12345/my/device/name#dbase=no (In your post, you did not mention the “tango://”)
I am receiving periodic events every 3 sec as expected:

I have received an event for attribute tango://pcrogucki.esrf.fr:12345/et/jul/01/state#dbase=no
Tue May 10 12:58:42 2016 (1462877922,85122 sec) : state#dbase=no (dim_x = 1, dim_y = 0, w_dim_x = 0, w_dim_y = 0, Data quality factor = VALID, Data format = SCALAR, Data type = DevState)
ALARM
I have received an event for attribute tango://pcrogucki.esrf.fr:12345/et/jul/01/state#dbase=no
Tue May 10 12:58:45 2016 (1462877925,84493 sec) : state#dbase=no (dim_x = 1, dim_y = 0, w_dim_x = 0, w_dim_y = 0, Data quality factor = VALID, Data format = SCALAR, Data type = DevState)
ALARM
I have received an event for attribute tango://pcrogucki.esrf.fr:12345/et/jul/01/state#dbase=no
Tue May 10 12:58:48 2016 (1462877928,84829 sec) : state#dbase=no (dim_x = 1, dim_y = 0, w_dim_x = 0, w_dim_y = 0, Data quality factor = VALID, Data format = SCALAR, Data type = DevState)
ALARM

Could you tried with a device name starting with “tango://” in your client ?
It could also be a PyTango issue because with C++, it is working (at least my test..).

Cheers

Emmanuel

3

Hi Emmanuel,

Thanks for your response. It convinced me that it should work, which got me trying various things.

[quote=“Manu”]
Could you tried with a device name starting with “tango://” in your client ?
It could also be a PyTango issue because with C++, it is working (at least my test..).

Cheers

Emmanuel[/quote]

It turns out that the tango:// bit makes no difference. What does matter is that the hostname is used when connecting to the device server. pytango-devicetest uses “localhost” instead of “tango-test” (the hostname of the machine I was testing on) when setting up the DeviceProxy. I was aware of the fact that bidirection lookups are required, and indeed (checked using resolveip) localhost ↔ 127.0.0.1 looks up both ways. But it does not matter if the host that the client connects to is resolvable via /etc/hosts or via DNS, nor does it matter if the server is specifically bound to the interface (e.g. -ORBendPoint giop:tcp:localhost:12345), events only work if the client connects to the hosts’ name.

Now that I have events working, I ran into another foible. After calling

td.subscribe_event(
    attr_name, PyTango.EventType.PERIODIC_EVENT, printer)

, the printer function is called with a

PyTango.EventData

structure for every event as expected. On the first event callback,

event_data.attr_value.name

contains name as it is in the server code, ‘ScalarBool’ in this case. On subsequent calls, it contains ‘scalarbool#dbase=no’. It seems to be the last part of the fully qualified attribute name, i.e. ‘tango://tango-test:12345/test/nodb/polled/scalarbool#dbase=no’ Is this a bug, or is it considered standard behaviour?

4

Hi Neilen,

You are right, the name of the attribute in the DeviceAttribute instance in the EventData structure is polluted with the
“#dbase=no” but not the first time the callback is executed. This id due to a different callback execution mechanism for this first call. This is a bug. I will create one entry for it in our bug list. I will also try to fix it ASAP.

Thank’s for the info.

Have a nice day

Emmanuel

5

Hi,

[quote=“Manu”]Hi Neilen,

You are right, the name of the attribute in the DeviceAttribute instance in the EventData structure is polluted with the
“#dbase=no” but not the first time the callback is executed. This id due to a different callback execution mechanism for this first call. This is a bug. I will create one entry for it in our bug list. I will also try to fix it ASAP.
Emmanuel[/quote]

Bug link, for those interested: TANGO Control System / Bugs / #796 Event -> Wrong attribute name in DeviceAttribute for DS using a file as Db

Thanks for looking into that. Another comment: whether the database is used or not, it seems the attribute name is converted to lower case on subsequent calls. In the example I provided the attribute name is ‘DevBoolean’; this is converted to the lower case form ‘devboolean’ after the first callback. Are attributes supposed to be case-insensitive in Tango, or is this also considered to be a bug?