View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 16 posts ] 
Go to page 1, 2  Next

Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Mon Aug 27, 2012 3:13 pm 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
I am starting to see symptoms of a memory leak in 7.04. I went from 7.02 to 7.04 so cannot say if it was present in 7.03. Is anybody else seeing any symptoms?

Attachment:
mem.png
mem.png [ 18.77 KiB | Viewed 10961 times ]

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Mon Aug 27, 2012 8:17 pm 
Offline
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1343
Location: Orlando FL
Are those two reboots or what cauesed the memory to drop?

_________________
My System


Top
 Profile  
 
PostPosted: Tue Aug 28, 2012 2:31 pm 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
I don't know what happened there - I was on vacation for 10 days so wasn't here... I do know that no recordings were happening...

The logs say that the system ran out of memory and a reboot occurred.

Here is the one for today (Tuesday).

Attachment:
memory.jpg
memory.jpg [ 91.16 KiB | Viewed 10938 times ]

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 4:35 pm 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
I built another test box the same way my prod box was done - upgraded from 7.2 to 7.4 with no additional added software other than Chromium and samba. I have let the test box run for a week with no activity. I am seing the same memory leak (or memory in use climb) that I see on my prod box. I also see swap file use increasing.

Attachment:
mem.jpg
mem.jpg [ 179.17 KiB | Viewed 10907 times ]


Is anybody else seeing this?

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Wed Sep 19, 2012 7:36 pm 
Offline
Joined: Mon Dec 24, 2007 9:47 am
Posts: 535
Location: Ottawa, Canada
What tool are you using to measure memory usage history? I'll give it a try and see if it is happening on my backend.

Currently my master backend has been running for 17 days and is sitting at approx 1GB usage. It usually runs for months before I have any need to reboot it for one reason or another. I only have 1.5GB allocated for it and I have only ever seen minimal swap usage. Currently I have used 0.015GB (ie. 15MB) of swap.


Top
 Profile  
 
PostPosted: Thu Sep 20, 2012 7:25 am 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
The graphs are from rrd. Seems to me i remember something about a memory leak in some nVidia drivers. I think I will try changing to an older release and see what happens.

Can a few people please check their rrd stats and confirm what they see? Thx.

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Thu Sep 20, 2012 10:24 am 
Offline
Joined: Mon Dec 24, 2007 9:47 am
Posts: 535
Location: Ottawa, Canada
Can you give us the commands/scripts used so we can make an appropriate comparison. By default rrd is not on LinHES so rrdtool needs to be installed.


Top
 Profile  
 
PostPosted: Thu Sep 20, 2012 11:35 am 
Offline
Joined: Tue Sep 12, 2006 6:03 am
Posts: 210
Location: Roseville, MI
Upgraded form 7.02 to 7.04 a couple of weeks ago. Memory usage looks good on my Backend/Frontend box. Pretty stable/consistent.
ImageImageImage

_________________
-Roseville, Michigan USA
LinHES R8: FE/BE, FE (x2)


Top
 Profile  
 
PostPosted: Fri Sep 21, 2012 6:29 am 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
Thanks spideyk21...

crist, rrd should be on your machine - look in system setup, I think under software (I know rrd tools i in there somewhere), un-check it, save it, then re-check it again. Go to /data/srv/httpd/htdocs/rrd and make sure the permissions are set right (I had this problem). RRD is then available from your mythtv home web screen (status/rrdtool)

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Sat Sep 22, 2012 6:55 am 
Offline
Joined: Mon Dec 24, 2007 9:47 am
Posts: 535
Location: Ottawa, Canada
Found it! thanks. I had no idea that was there so was planning to do it manually. I expected to see rrd added to cron somewhere though; I'm not sure where it is kicked off from.

I'll update you once I get substantial stats.

Note my backend has no tuner cards or video cards (it's in a VM) so it should provide a decent mythbackend baseline.


Top
 Profile  
 
PostPosted: Thu Sep 27, 2012 2:58 pm 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
Update. I have drilled into this about as far as I know how... The memory creep is caused by something in the frontend on my MBE. If I stop and restart the frontend the memory usage goes bask to a normal state then, after a few hours, starts to creep back up. I have left the fronend down for a couple of days - memory usage stays normal.

I am going to build a fresh 7.04 (again) without restoring my backup (stay virgin) to see what happens (I want to see if it is hardware related...).

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Thu Sep 27, 2012 9:39 pm 
Offline
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1343
Location: Orlando FL
Good test

_________________
My System


Top
 Profile  
 
PostPosted: Sun Sep 30, 2012 6:05 am 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
Getting closer... Will somebody that has mythexport working please check something for me? Do a normal backup (services menu, backup), then check your logs for mythexport (/var/logs/mythtv/mythexport). Do you see a recurring message - Unable to query mythexport_job_queue?

I am seeing it on both my production machine and my test machine. It occurs after a mysql stop issued in the backup. For that matter, just do 'sudo stop mythbackend, sudo sv stop mysql, wait a few seconds then reverse - sudo sv start mysql, sudo start mythbackend. Then check the logs...

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Mon Oct 01, 2012 5:36 am 
Offline
Joined: Wed Jan 07, 2004 12:14 pm
Posts: 425
Location: Charlotte, NC
I should be clearer... There seems to be a problem with /usr/lib/mythexport-daemon whenever mysql goes away... It starts complaining and doesn't stop - even when mysql comes back. If you have mythexport setup (or if you ever tried to set it up) you have mythexport-daemon running. Please stop mysql and check the logs. The way to clear it up is to restart the daemon (sudo sv restart mythexport). Thanks.

_________________
Backend server - 4.0 TB 3.0ghz dual core 6 gig RAM, nVidia 9400, Gigabyte GA-870A-UD3 MB, 2 HD-5500, 2 HD Homerun dual tuners, 3 frontend machines - LinHES 8.6.1


Top
 Profile  
 
PostPosted: Mon Oct 01, 2012 6:40 pm 
Offline
Site Admin
Joined: Fri Jun 11, 2004 7:58 am
Posts: 507
thanks nbdwt73 this is good information.


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ] 
Go to page 1, 2  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 159 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu