View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 21 posts ] 
Go to page 1, 2  Next

Print view Previous topic   Next topic  
Author Message
Search for:
 Post subject: Does Xymon 'Cry Wolf' ?
PostPosted: Sat Feb 22, 2014 11:49 am 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
8.1.1 Machine appears to be working normally, making recordings, filling disk, leaving about 6 gb free by auto-expiring:

92 programs, using 447 GB (3 days 20 hrs 41 mins) out of 454 GB (6.2 GB free).

Disk Usage Summary:
Total Disk Space:
Total Space: 465,255 MB
Space Used: 459,441 MB
Space Free: 5,814 MB
Space Available After Auto-expire: 351,215 MB
Space Used by LiveTV: 754 MB
Space Used by Deleted Recordings: 0 MB
Space Used by Auto-expirable Recordings: 344,647 MB
Disk Usage Details:
MythTV Drive #1:
Directories: 811linhes27-121:/data/storage/Maxtor7H500F0_H80PS9AH/media/music, 811linhes27-121:/data/storage/Maxtor7H500F0_H80PS9AH/media/streaming, 811linhes27-121:/data/storage/Maxtor7H500F0_H80PS9AH/media/tv, 811linhes27-121:/data/storage/Maxtor7H500F0_H80PS9AH/media/tv/live
Total Space: 465,255 MB
Space Used: 459,441 MB
Space Free: 5,814 MB

So why is Xymon panicking ?

Sat Feb 22 12:05:03 CST 2014 - Filesystems NOT ok

Code:
red /data/storage/disk0 (99% used) has reached the PANIC level (99%)
red /data/storage/Maxtor7H500F0_H80PS9AH (99% used) has reached the PANIC level (99%)

Filesystem     1024-blocks      Used Available Capacity Mounted on
/dev/sda1          4805760   2254596   2307044      50% /
/dev/sda7        476421852 470434872   5986980      99% /data/storage/disk0
/dev/sda5          2883616    169824   2567312       7% /home
/dev/sda6          1923084    165456   1659940      10% /data/srv/mysql
/dev/sda7        476421852 470434872   5986980      99% /data/storage/Maxtor7H500F0_H80PS9AH



UPDATE:

Recording a movie, auto-expire is gobbling up 'oldest' programs, when a 17 gb program was erased, Xymon dropped back to 'yellow' at 95%, but it's still complaining ... Is there a way to bring the 'threshold of squawk' of Xymon into closer accord with the numbers auto-expire maintains ?

Also .. can Xymon be turned off ? I've run Mythboxes for years without anything needing 'immediate' action. Xymon runs popup windows in the main screen that say something like 'RED Orphan' .... Really ... fixing orphans need immediate attention ? .... :lol:


Last edited by snaproll on Tue Feb 25, 2014 11:18 am, edited 1 time in total.


Top
 Profile  
 
PostPosted: Sat Feb 22, 2014 11:02 pm 
Offline
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1343
Location: Orlando FL
yeah that orphan one drove me nuts until I finally decided I had enough of it's nagging and deleted the backup files from 3 years ago. JEEZ
Thankfully it's not a popup but on the health and maintenance page I've got Memory yelling at me that the swap is nearly full! OH NO!

_________________
My System


Top
 Profile  
 
PostPosted: Sun Feb 23, 2014 7:59 am 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
mattbatt wrote:
yeah that orphan one drove me nuts until I finally decided I had enough of it's nagging and deleted the backup files from 3 years ago. JEEZ
Thankfully it's not a popup but on the health and maintenance page I've got Memory yelling at me that the swap is nearly full! OH NO!


LOL ! ... :lol:

Glad to hear it's not just me ... It did drive me to run 'find_orphans.py' ... which shut it up about that ... I just don't want to do it in the middle of a program ... :lol:

Now about disk usage ... looks like it's complaining about 'too efficient' disk usage ... but I don't want that popping up in the middle of a program ... especially on a 'production' machine used by ... er .. 'users' ... :mrgreen:


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 9:30 am 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
Apparently, (According to: https://bb.man-da.de/hobbit/help/hobbit-config.html ... ) you can edit the red and yellow squawk percentages in this configuration file:

~Xymon/server/etc/hobbit-clients.cfg

Except I get a 'no such' file response. Anyone know where LinHes puts it ?

Finding xymon files (there's a bunch ..

locate /xymon | more

This one looks promising: /home/xymon/client/etc/localclient.cfg

Changed disk percentages to 101 and 101 ... now we'll see if it shuts up !

Code:
# The special DEFAULT section can modify the built-in defaults - this must
# be placed at the end of the file.

DEFAULT
 # These are the built-in defaults.
 UP      1h
 LOAD    5.0 10.0
 DISK    * 90 95
 MEMPHYS 100 101
 MEMSWAP 50 80
 MEMACT  90 97


Bah ... Still reporting red (even after reboot)

Edited it again to say DISK IGNORE (per instructional text in the file) Didn't work either :cry:


Last edited by snaproll on Tue Feb 25, 2014 12:30 pm, edited 6 times in total.


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 10:11 am 
Offline
Site Admin
Joined: Fri Jun 11, 2004 7:58 am
Posts: 507
client.cfg to edit percentages, but really alerting at 99% isn't a bad thing. Running disks at 99% usage is generally a bad thing.

alerts.cfg to configure when/how/which alerts are sent out.


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 10:30 am 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
jams wrote:
client.cfg to edit percentages, but really alerting at 99% isn't a bad thing. Running disks at 99% usage is generally a bad thing.

alerts.cfg to configure when/how/which alerts are sent out.


Yes... except in the case of Mythtv, which has been using autoexpire to keep 5-6 gigs clear with no problems since forever.

Trouble is- this really shows up on large 1.5 - 2 tb drives when 5-6 gb free is 99+++ full .... :lol:


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 10:59 am 
Offline
Joined: Sun Sep 05, 2004 7:06 pm
Posts: 690
The Winter Olympics was my first really tuff test with R8.1 and various shows. My box would fill up, expire, delete shows like crazy. Even though it worked correctly some of my recordings for regular shows started to show signs of trouble like weak signal and were junk. In the end I realized that I do not have enough space and performance is important to me....


Last edited by RacerX on Tue Feb 25, 2014 11:05 am, edited 1 time in total.


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 11:05 am 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
RacerX wrote:
The Winter Olympics was my first really hard test with R8.1 and various shows. My box would fill up, expire, delete shows like crazy. Even though it worked correctly some of my recordings for regular shows started to show signs of trouble like weak signal and were junk. In the end I realized that I do not have enough space and performance is important to me....


Y'sure it was a diskspace problem and not a temporary 'signal fade' problem ?

I'm also finding xymon is causing a lot of cpu useage, too ... ( top )

Code:
 PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM     TIME+ COMMAND
 2155 nobody    20   0  101m  11m 3420 R  41.4  0.5   0:00.23 hbfunc.py
 2157 nobody    20   0  101m  10m 3920 R  31.0  0.5   0:00.18 hobbit_myth_dat
 2167 nobody    20   0 88588 9.8m 3752 R  25.9  0.5   0:00.15 python2


I'm thinking for a production machine, Xymon would be better turned off, and the cpu cycles given back to Mythtv ...


Last edited by snaproll on Tue Feb 25, 2014 2:31 pm, edited 1 time in total.


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 11:17 am 
Offline
Joined: Sun Sep 05, 2004 7:06 pm
Posts: 690
The fade (bad packets) problem happened about two thirds into the Olympics during a couple of days a program with shows that I'm missing. It only affected the show and not the Olympics. So I wiped out all the shows on my box and then the problem vanished. My take is that filling up the box over and over is not really great for the system. Especially with Mythwelcome minimizing the time that the system is up.


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 11:30 am 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
RacerX wrote:
The fade (bad packets) problem happened about two thirds into the Olympics during a couple of days a program with shows that I'm missing. It only affected the show and not the Olympics. So I wiped out all the shows on my box and then the problem vanished. My take is that filling up the box over and over is not really great for the system. Especially with Mythwelcome minimizing the time that the system is up.


Dunno.
I'm currently running 2 Myth .27 boxes (8.1.1) as testbeds with generally excellent results except for the $$^&^ fershluggener Xylon squawking.

I have 5 production boxes (Myth .25) in service running default autoexpire 5 - 6 gig. free with no problems at all.

So I tend to view Xymon as a flashy cycle-stealing toy that I'd like to turn off unless I want to temporarily look at something.

It's not in /etc/init.d so I couldn't take it out.

Went to /home/xymon/client and renamed runclient.sh so it wouldn't get run ... Well After a reboot Xymon was still started up.

Went to /home/xymon/etc/analysis.cfg:



& changed all percents ie: * 95 99 to'IGNORE' for DISK

....& it still work s ! :roll:

Out of ideas at the moment.


Can't comment on Mythwelcome, 'cuz my boxes run all the time, and do 'housekeeping during off hours.


Last edited by snaproll on Tue Feb 25, 2014 2:04 pm, edited 2 times in total.


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 1:46 pm 
Offline
Site Admin
Joined: Fri Jun 11, 2004 7:58 am
Posts: 507
Depending on the system type to disable xymon

remove_service.sh xymon-client
or
remove_service.sh xymon-server


Top
 Profile  
 
PostPosted: Tue Feb 25, 2014 2:02 pm 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
So a standalone system, which one ?

Getting rid of 'server' got rid of it, The Health & Maintenance page initially shows the last page before the server went away, & if you click on anything, shows 'cannot load host configuration' All the other configurations work.

'top' no longer shows large amounts (or any ...) useage by 'hobbit' so those cycles are available to run your Mythbox or process commercials out ... :mrgreen:

If you got rid of 'client', the 'xymon server' would still be gobbling up cycles on the machine it runs on.

NOTE on disabling xymon server ...

So far nothing negative on function of the two boxes, probably running better now they have the stolen cycles back.

A nit about clicking on the 'Health' page: it will display the 'health; at the time the server was unplugged and the date remains on when it was unplugged ... If you like serenity, fix all the squawks back to 'green' first, before taking down the xymon server .... :lol:


Top
 Profile  
 
PostPosted: Thu Feb 27, 2014 10:01 am 
Offline
Joined: Fri Jul 21, 2006 11:12 pm
Posts: 1194
Location: SC
snaproll for a standalone or Master_backend system xymon-server is what you would want.

What are your box specs? I am curious because I have never observed the cpu usage with xymon that you are describing even on the most low end box I manage. That box is a P4 which is one of the oldest and slowest but still 64 bit that LH 8 will run on. Also, the hbfunc,py and hobbit_myth_data process you showed in top should only run periodically. If they are running all the time something is wrong. But again from your top output I don't think that is the case because the TIME+ shows that hbfunc.py has only used 23 hundredths of a second of CPU time. So while in that instant it is using 41% of the CPU over time it isn't using much at all.


Top
 Profile  
 
PostPosted: Thu Feb 27, 2014 12:28 pm 
Offline
Joined: Sun Jul 24, 2005 4:16 pm
Posts: 508
Location: Ft. Worth TX
brfransen wrote:
snaproll for a standalone or Master_backend system xymon-server is what you would want.


I have 6 Mythboxes operating on the same network. 2 production boxes (Myth .25) at each of 2 TV stations, 2- 8.11 (Myth .27) testbeds at single computer stations. All are standalone, each user programs their own 'stuff' there is some redundancy in recorded programming.

I don't use a master backend 'cuz I don't like all my eggs in one basket, and it leaves my network capacity available for streaming Netflix. Also I can use WD media boxes to stream content from any Mythbox on the network to any TV station. XBMC is available for that too ...

So as noted above, I disabled xymon server.

Quote:
What are your box specs? I am curious because I have never observed the cpu usage with xymon that you are describing even on the most low end box I manage. That box is a P4 which is one of the oldest and slowest but still 64 bit that LH 8 will run on.


Code:
[greg@811linhes27-121 ~]$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Pentium(R) 4 CPU 3.40GHz
stepping        : 1
microcode       : 0x9
cpu MHz         : 3391.563
cache size      : 1024 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc pebs bts nopl pni dtes64 monitor ds_cpl cid cx16 xtpr
bogomips        : 6785.75
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Pentium(R) 4 CPU 3.40GHz
stepping        : 1
microcode       : 0x9
cpu MHz         : 3391.563
cache size      : 1024 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 0
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc pebs bts nopl pni dtes64 monitor ds_cpl cid cx16 xtpr
bogomips        : 6785.75
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:


This box will record 2 and play one without hitches, but throw in processing commercials, and it will hitch the picture and sound occasionally ... so commercials are processed overnight. It had been running Myth .25 fixes, and I clean installed LinHes 8.1.1. When I got hitches with the commercial processing off, I went looking for what changed, and I found xymon made a graph that showed a load factor going up to 3 at times.

So I figured xymon was squawking itself.

The newer Mythboxes are IBM 9229 running Intel Core 2 Duo E6600 / 2.40 GHz processors, which, I guess can be considered as 4 processors, and might be better at carrying other tasks than the P4.

Quote:
Also, the hbfunc,py and hobbit_myth_data process you showed in top should only run periodically. If they are running all the time something is wrong. But again from your top output I don't think that is the case because the TIME+ shows that hbfunc.py has only used 23 hundredths of a second of CPU time. So while in that instant it is using 41% of the CPU over time it isn't using much at all.


How much xymon ran, I didn't know, or whether it was crowding out Mythtv functions, but I did get that 'top' in the post above, and figured xymon is a diagnostic tool that only needs to run when you're diagnosing ... and the two Mythboxes running 8.11 were doing fine and didn't need any diagnosing ... :lol:

My observation of the xymon page refreshing every minute or so, gave me the impression that it ran a lot.
The xymon installation was the 'as installed' default from a new installation of 8.1.1.


Top
 Profile  
 
PostPosted: Thu Feb 27, 2014 1:17 pm 
Offline
Joined: Fri Jul 21, 2006 11:12 pm
Posts: 1194
Location: SC
Your P4 and the one I manage are very similar. Mine is a 3.2GHz so it is slightly slower. These are hyper-threaded CPUs so they are physically 1 core that appear as 2 to the OS. This box can record 4 and playback 1 using VDPAU with no problem but add in commflagging and it stumbles. I think this is more IO than CPU though. And it certainly doesn't help that the db is on same drive as the most of the media.

Quote:
So I figured xymon was squawking itself.
Possible. I just haven't seen that on my slower P4.

Quote:
The newer Mythboxes are IBM 9229 running Intel Core 2 Duo E6600 / 2.40 GHz processors, which, I guess can be considered as 4 processors, and might be better at carrying other tasks than the P4.
The E6600 would definitely be much better but is not hyper-threaded just 2 physical cores.

Quote:
How much xymon ran, I didn't know, or whether it was crowding out Mythtv functions, but I did get that 'top' in the post above, and figured xymon is a diagnostic tool that only needs to run when you're diagnosing ... and the two Mythboxes running 8.11 were doing fine and didn't need any diagnosing ... :lol:
xymon is a monitoring tool that can be used for diagnostics but that is only one small part of it.

Quote:
My observation of the xymon page refreshing every minute or so, gave me the impression that it ran a lot.
That is just the webpage refreshing to see if the underlying xymon server has any new data. That load on the box is only there when the page is open.

Quote:
The xymon installation was the 'as installed' default from a new installation of 8.1.1.
Good to know and helpful in trying to determine if we have a bug, configuration issue, or if it is just a function of slower hardware.


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 21 posts ] 
Go to page 1, 2  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 45 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu