Hello tangoers,
we noticed that ATK Panel launched on some computers crashed after a few hours.
After some investigations, it seems computers which make ATK Panels crash are all installed with debian 8 (jessie) packages (with TANGO 8).
The ones with ubuntu 14.04 (with TANGO 8) or debian unstable with TANGO 9 let them run without any problem.
To check if this behaviour is caused by our devices servers programming, I watched the behaviour after launching TangoTest ATKPanel launching command “top” to scrutate TangoTest process (launched with command “valgrind --tool=massif atkpanel sys/tg_test/1”).
I confirm all TangoTest DS consume low CPU in a stable way.
Here are my observations for memory:
-
debian 8 with TANGO 8: Virtual and Resident memory continuously increases for debian 8/TANGO 8 ATK Panels until crashing :
- when launching: 650080 KiB Virtual memory, 62044 KiB Resident memory size, 15996 KiB Shared memory.
- when crashing (after 2 hours): 2632552 KiB Virtual memory, 1.505g KiB Resident memory size, Shared memory stays around 0 KiB.
-
ubuntu trusty (14.04) with TANGO 8: memory slowly increases :
- at beginning: 583828 KiB Virtual memory, 56452 KiB Resident memory size, 12180 KiB Shared memory.
- after 1 hour: 1116312 KiB Virtual memory, 392792 KiB Resident memory size, Shared memory stays around 2200 KiB.
- after 3 hours: 2033816 KiB Virtual memory, 1,044g KiB Resident memory size, Shared memory stays around 1792 KiB.
-
debian unstable with TANGO 9, memory stays at 718800 KiB Virtual memory, 63468 KiB Resident memory size, and 17472 KiB Shared memory size.
-
tangobox 9 VM (ubuntu trusty 14.04) with TANGO 9: memory stays around 740988 KiB Virtual memory, 41000 KiB Resident memory size, and 0 KiB Shared memory size.
Here is the java version and the list of tango packages found on both computers :
pg@debian8:$> dpkg-query -l | grep openjdk
ii openjdk-7-jre:amd64 7u91-2.6.3-0ubuntu0.14.04.1 amd64 OpenJDK Java runtime, using Hotspot JIT
ii openjdk-7-jre-headless:amd64 7u91-2.6.3-0ubuntu0.14.04.1 amd64 OpenJDK Java runtime, using Hotspot JIT (headless)
pg@debian8:$> dpkg-query -l | grep tango
ii liblog4tango5:amd64 8.1.2c+dfsg-5 amd64 logging for TANGO - shared library
ii liblog4tango5-dev:amd64 8.1.2c+dfsg-5 amd64 logging for TANGO - development library
ii libtango-java 8.1.2-1ubuntu1ppa1~precise1 amd64 TANGO java part - shared library
ii libtango-tools 8.1.2c+dfsg-5 amd64 TANGO distributed control system - common executable files
ii libtango8:amd64 8.1.2c+dfsg-5 amd64 TANGO distributed control system - shared library
ii libtango8-dev:amd64 8.1.2c+dfsg-5 amd64 TANGO distributed control system - development library
ii libtango8-doc 8.1.2c+dfsg-5 all TANGO distributed control system - documentation
ii python-pytango 8.1.5-1 amd64 API for the TANGO control system (Python 2)
ii tango-common 8.1.2c+dfsg-5 all TANGO distributed control system - common files
ii tango-db 8.1.2c+dfsg-5 amd64 TANGO distributed control system - database server
ii tango-starter 8.1.2c+dfsg-5 amd64 TANGO distributed control system - starter server
ii tango-test 8.1.2c+dfsg-5 amd64 TANGO distributed control system - test device
pg@ubuntu1409:$> dpkg-query -l | grep openjdk
ii openjdk-7-jre:amd64 7u111-2.6.7-1~deb8u1 amd64 OpenJDK Java runtime, using Hotspot JIT
ii openjdk-7-jre-headless:amd64 7u111-2.6.7-1~deb8u1 amd64 OpenJDK Java runtime, using Hotspot JIT (headless)
pg@ubuntu1409:$> dpkg-query -l | grep tango
ii liblog4tango5:amd64 8.1.2c+dfsg-3ubuntu0.1 amd64 logging for TANGO - shared library
ii liblog4tango5-dev:amd64 8.1.2c+dfsg-3ubuntu0.1 amd64 logging for TANGO - development library
ii libtango-java 8.1.2-1ubuntu1ppa1~precise1 amd64 TANGO java part - shared library
ii libtango-tools 8.1.2c+dfsg-3ubuntu0.1 amd64 TANGO distributed control system - common executable files
ii libtango8:amd64 8.1.2c+dfsg-3ubuntu0.1 amd64 TANGO distributed control system - shared library
ii libtango8-dev:amd64 8.1.2c+dfsg-3ubuntu0.1 amd64 TANGO distributed control system - development library
ii libtango8-doc 8.1.2c+dfsg-3ubuntu0.1 all TANGO distributed control system - documentation
ii python-pytango 8.1.1-1build3 amd64 API for the TANGO control system (Python 2)
ii tango-common 8.1.2c+dfsg-3ubuntu0.1 all TANGO distributed control system - common files
ii tango-db 8.1.2c+dfsg-3ubuntu0.1 amd64 TANGO distributed control system - database server
ii tango-starter 8.1.2c+dfsg-3ubuntu0.1 amd64 TANGO distributed control system - starter server
ii tango-test 8.1.2c+dfsg-3ubuntu0.1 amd64 TANGO distributed control system - test device
ubuntu 14.09 is installed with 8.1.2c+dfsg-3ubuntu0.1 packages while debian 8 is installed with 8.1.2c+dfsg-5 version.
It seems the version is the same but the package release is slightly different (https://wiki.debian.org/DebianMentorsFaq#What_does_.2BIBw-dfsg.2BIB0_in_the_version_string_mean.3F).
Could this be the source of the problem?
Does any one noticed a similar issue ?
edit : I checked the memory consumed by a DS after stopping the ATKPanel linked to this DS, the memory stays high after stopping the ATKPanel.
I even saw a strange behaviour : 4 DS launched, each of them with an ATKPanel launched, the 3 first have increasing consumed memory, and the last one stays at constant memory.
edit : I added java package version of each distribution. I have also complete installed package list and version if needed.