LinHES Forums
http://forums.linhes.org/

Automating db optimization
http://forums.linhes.org/viewtopic.php?f=6&t=17988
Page 1 of 4

Author:  Too Many Secrets [ Sun Mar 02, 2008 11:34 pm ]
Post subject: 

Cecil, I know this is very basic, but I've been running the below script to call the optimize_db.sh script in my SVN box. It won't run if recording, running mythfilldatabase or commflag. Maybe someone can add a sleep or dance it up some. (Again, this doesn't work in F27 as the 1 command isn't recognized until recently.)

Code:
#!/bin/bash

mythshutdown -s 1 -v all | grep 'status returned: 0' >/dev/null

        if [ $? == 0 ]; then
                /usr/local/bin/optimize_db.sh
        else
                echo Could not run script
        fi

Author:  cecil [ Mon Mar 03, 2008 12:15 am ]
Post subject: 

Split to start a new thread for discussion...

Author:  Too Many Secrets [ Mon Mar 03, 2008 10:21 am ]
Post subject: 

Thanks for spinning this off Cecil, wasn't sure but it did seem a little OT for the R6 thread.

Just to give a little context, the above script is an attempt to include a 'wrapper script' to allow the optimize_db.sh (or other maintenance) script(s) to run from cron or be called from command line and not interrupt recordings or other important happenings on your Myth-box.

Possible Problems/Improvements:
Seems a sleep command might make this script more functional. Maybe logging? Also don't know the effects of remote SBE tuners after running the optimize script. They might be unavailable, requiring a restart of the SBE BE.

Author:  Girkers [ Tue Mar 04, 2008 9:51 pm ]
Post subject: 

Another thing to consider is when is the next recording etc due, as we don't want the BE down when something is due in the next 5 minutes.

Author:  Too Many Secrets [ Wed Mar 05, 2008 8:49 am ]
Post subject: 

Girkers wrote:
Another thing to consider is when is the next recording etc due, as we don't want the BE down when something is due in the next 5 minutes.



I 'believe' this is handled with the -s option in mythshutdown as it looks for a number of things. From the help page (This is the older version as 8 & 32 are now used for further feedback such as recording).

Code:
-s/--status         (returns a code indicating the current status)
         0 - Idle
         1 - Transcoding
         2 - Commercial Flagging
         4 - Grabbing EPG data
         8 - Not used
        16 - Locked
        32 - Not used
        64 - In a daily wakeup/shutdown period
       128 - Less than 15 minutes to next wakeup period


The above wrapper script (above) shouldn't run unless the status returned is 0.

Author:  cecil [ Wed Mar 05, 2008 5:27 pm ]
Post subject: 

:D Work it out!

Author:  tjc [ Wed Mar 05, 2008 7:35 pm ]
Post subject: 

The method above seems to be going about it the hard way. I believe that the following much simpler form should work just as well:
Code:
mythshutdown -s && mythbackup >/var/log/backup.log 2>&1

Testing this is easy. Schedule a dummy recording for 10 minutes in the future and then try to run this command. It should do nothing. Also try while watching Live TV, or while manually running mythfilldatabase, all should do nothing.

I gave this a shot and it works for transcoding but not for live recording. It also doesn't detect upcoming recordings. I'd call it an unsuccessful method.

Author:  tjc [ Wed Mar 05, 2008 10:33 pm ]
Post subject: 

Here is my first cut at a script to determine if it's safe to run an automated backup. Since mythshutdown doesn't seem to check for upcoming recordings or the like.
Code:
root@black2:~/scripts/backup# cat ./idle.sh
#!/bin/bash
echo "...REMOVED TO AVOID CONFUSION..."

The intended usage pattern would be something like:
Code:
idle.sh && mythbackup >/var/log/backup.log 2>&1

Author:  Too Many Secrets [ Wed Mar 05, 2008 11:10 pm ]
Post subject: 

tjc wrote:
The method above seems to be going about it the hard way. I believe that the following much simpler form should work just as well:
Code:
mythshutdown -s && mythbackup >/var/log/backup.log 2>&1

Testing this is easy. Schedule a dummy recording for 10 minutes in the future and then try to run this command. It should do nothing. Also try while watching Live TV, or while manually running mythfilldatabase, all should do nothing.

I gave this a shot and it works for transcoding but not for live recording. It also doesn't detect upcoming recordings. I'd call it an unsuccessful method.


I agree the first script relying only on mythshutdown doesn't seem to account for upcoming recordings. (I must be misunderstanding the 32 - Jobs running or pending flag. However I can't test LiveTV on my test box.

Author:  Too Many Secrets [ Wed Mar 05, 2008 11:13 pm ]
Post subject: 

tjc wrote:
Here is my first cut at a script to determine if it's safe to run an automated backup. Since mythshutdown doesn't seem to check for upcoming recordings or the like.
Code:
root@black2:~/scripts/backup# cat ./idle.sh
#!/bin/bash

mythshutdown -s
busy="$?"
echo "mythshutdown returned $busy"

SCHEMALOCK=$(mysql mythconverg -sBe '
select count(*)
from schemalock
')
echo "schemalock $SCHEMALOCK"

JOBS=$(mysql mythconverg -sBe '
select count(*)
from jobqueue
where status = 4
')
echo "jobs $JOBS"

INUSE=$(mysql mythconverg -sBe '
select count(*)
from inuseprograms;
')
echo "inuse $INUSE"

UPCOMING=$(mysql mythconverg -sBe '
select count(*)
from recordmatch as rm, program as p
where rm.chanid = p.chanid
and rm.starttime = p.starttime
and rm.starttime < now() + interval 20 minute
and now() < p.endtime + interval 5 minute;
')
echo "upcoming $UPCOMING"

factors=$(expr "$busy" + "$SCHEMALOCK" + "$JOBS" + "$INUSE" + "$UPCOMING")
echo
if [ "$factors" -eq 0 ] ; then
    echo "System is idle"
    exit 0
else
    echo "Blocking factors previously noted"
    exit 1
fi

The intended usage pattern would be something like:
Code:
idle.sh && mythbackup >/var/log/backup.log 2>&1


So much for simpler... :shock: But this does the trick as I can tell. Accounts for upcoming recordings. Again I can't really test LiveTV. still need to test a mythfilldatabase.

Author:  Too Many Secrets [ Thu Mar 06, 2008 12:08 pm ]
Post subject: 

tjc, I seem to be getting a 'false flag' for 'upcoming' when a recording is Scheduled (viewed from Upcoming Recordings in MythWeb) but not Active (seen under Schedule in Backend Status in MythWeb). This in on my F27 machine.

Don't think this is a show stopper, but could be an annoying 'feature'.

Author:  tjc [ Thu Mar 06, 2008 6:44 pm ]
Post subject: 

Too Many Secrets wrote:
tjc, I seem to be getting a 'false flag' for 'upcoming' when a recording is Scheduled (viewed from Upcoming Recordings in MythWeb) but not Active (seen under Schedule in Backend Status in MythWeb).

Yep. It'll do that. Mostly because I didn't try to check if the match was a real recording or just a potential one. That's a lot harder.

Author:  tjc [ Thu Mar 06, 2008 9:29 pm ]
Post subject: 

OK, try this version. It should be smarter about whether a potential recording is active or not. If this tests well I'll make the upcoming gap which is now hard coded to 20 minutes a parameter and submit it to Cecil for the next go 'round.

The idle.sh script (Don't use this version. See the link posted at the end of the thread for the latest idle.sh):
Code:
#!/bin/bash

mythshutdown -s
busy="$?"
echo "mythshutdown returned $busy"

SCHEMALOCK=$(mysql -u root mythconverg -sBe '
select count(*)
from schemalock
')
echo "schemalock $SCHEMALOCK"

JOBS=$(mysql -u root mythconverg -sBe '
select count(*)
from jobqueue
where status = 4
')
echo "running jobs $JOBS"

INUSE=$(mysql -u root mythconverg -sBe '
select count(*)
from inuseprograms;
')
echo "inuse programs $INUSE"

UPCOMING=$(mysql -u root mythconverg -sBe '
select count(*)
from recordmatch as rm, program as p
where rm.chanid = p.chanid
and rm.starttime = p.starttime
and rm.starttime < now() + interval 20 minute
and now() < p.endtime + interval 5 minute;
')
echo "possibly upcoming $UPCOMING"
# Check to see if any of the upcoming recordings are real...
UPCOMING=$(mythbackend --printsched 2>&1 |
  awk -v upcoming=$UPCOMING '
    BEGIN {item=-1;real=0}
    /--- print list start ---/,/---  print list end  ---/ {
        if (item>0 && item<=upcoming && substr($0,71,1) ~ "[0-9]") real+=1;
        item += 1;
    }
    END {print real}
')
echo "really upcoming $UPCOMING"

factors=$(expr "$busy" + "$SCHEMALOCK" + "$JOBS" + "$INUSE" + "$UPCOMING")
echo
if [ "$factors" -eq 0 ] ; then
    echo "System is idle"
    exit 0
else
    echo "System is busy"
    exit 1
fi


3/8/2008 - Edited to fix the DB login issue.
3/8/2008 - Edited to point people to the newest version.

Author:  Too Many Secrets [ Fri Mar 07, 2008 2:08 pm ]
Post subject: 

This looks good! As long as the 'possibly upcoming' and 'really upcoming' dont get people confused. :lol:

Then adding this to cron and configuring for a nightly 'backup' and 'optimize' would be just the needed maintenance. I say nightly as one might be recording/commflag/transcoding something even at 4:29am occasionally. Keeping an eye on logs should show when the scripts were actually allowed to run.

Also to note the idle.sh script needs to run as root for me.

Author:  opel70 [ Fri Mar 07, 2008 3:17 pm ]
Post subject: 

Thanks for the script. I have added it to my cron job that I use for my backup. Though I highly doubt that I will be recording at 3:20a, when I have my backups scheduled, it will be nice to have this as added insurance.

Page 1 of 4 All times are UTC - 6 hours
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/