View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 19 posts ] 
Go to page 1, 2  Next

Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Mon Apr 13, 2009 10:47 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
My machine still suffers from the occasional freeze but I find I can minimise them if I regularly run optimize_db.sh. What would be ideal if this type of maintenance could be run automatically. Reading posts in the forum I see some people do this as a cron job however others do not recommend it as it may happen at an inopportune time. I run my machine (F/E + B/E) only as required so that it starts-up by ACPI for recording and shuts-down from Mythwelcome when not recording. The problem with a cron job for me is the machine has to be up and running at the specified time for it to work. Could the Mythwelcome shutdown procedure be used to initiate optimize_db.sh? Mythwelcome has done all the work of finding put whether the machine is needed and will only proceed in shutting down if safe to do so.

I would imagine this might involve creating a script like

optimize_db.sh
/sbin/poweroff

and changing the "Command to Shutdown" in mythwelcome --setup from /sbin/poweroff with the new script name. There would probably be some permissions issues so could I just add optimize_db.sh to the sudoers list?

I know I could just go and try this but with my limited knowledge of linux I would be interested to know if I am barking up the wrong tree or if there is there a better way?

Going one step further, could this approach be applied to automating backup which will also repair the db? The problem here I see is that backup would occur every time the machine is used which is probably too frequent. Perhaps the shutdown script could be modified to run backup say every 5 shutdowns.

_________________
Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 13, 2009 11:09 pm 
Offline
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location: Perth, Australia
This is a very good idea, something i have been thinking about, but never had time to look into... getting the shutdown command to execute the optimize script strikes me as the best way, as it shuts down the sql and mythtv backend..

I might try it tonight if i get the chance...

Create a script that does
1) The optimise, then
2) the shutdown

Shouldnt be too hard i would think...

If you get there first, post what you did, and i'll do likewise if i extract my digit before you..

Nathan

_________________
LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector


Top
 Profile  
 
PostPosted: Tue Apr 14, 2009 2:21 am 
Offline
Site Admin
Joined: Fri Sep 19, 2003 6:37 pm
Posts: 2659
Location: Whittier, Ca
nicom wrote:
and changing the "Command to Shutdown" in mythwelcome --setup from /sbin/poweroff with the new script name. There would probably be some permissions issues so could I just add optimize_db.sh to the sudoers list?
Write your script and put that script in sudoers... It would be a good idea to include a counter and have it kick off like every 10th time...


Top
 Profile  
 
 Post subject:
PostPosted: Sun Apr 19, 2009 12:56 am 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
Well I have had some success with my idea but I believe there could be some improvements. Firstly I created a script /home/mythtv/sd-opt/shutdown_opt.sh as follows
Code:
#!/bin/bash
#
# Script to run the optimize_db.sh script regularly
# Called from Mythwelcome at shutdown
# Counter included to run optimize_db.sh on every fifth shutdown
#
# Extract shutdown number from file and increment
counter=`cat /home/mythtv/sd-opt/sdcount`
max=5
let "counter += 1"

# At fifth shutdown run optimize_bd script and reset counter
if [ $counter -ge $max ]; then
   counter=0
   bash optimize_db.sh
   echo optimize_db.sh was last run at `date` > /home/mythtv/sd-opt/optimize_db.log
fi

# Write new count number back to file
echo $counter >/home/mythtv/sd-opt/sdcount
#
# Shutdown as per normal
/sbin/poweroff

I located it and the associated files in a sub-directory of /home/mythtv so that it would survive upgrades but I am not sure if it is recommended. In Mythwelcome --setup I changed the "Command for Shutdown" from /sbin/poweroff to /home/mythtv/sd-opt/shutdown_opt.sh. It counts 5 shutdowns before running optimize_db.sh as per Cecil's suggestion. No changes to permissions or sudoers were required. It creates a log file (optimize_db.log) telling when optimize_db.sh was last run at shutdown.

When I run it works but when it performs the optimize_db.sh on shutdown I get a message in Mythwelcome saying it cannot connect to the backend but it does shutdown after another 15 seconds or so.

Improvements I think that could be made:
1. Make it part of the Mythwelcome-set-alarm.sh script. That way it would be upgrade-proof and I think it would be done at a better time in the shutdown sequence. Unfortunately this script is in perl and I struggle enough with bash so I was unable to try it.
2. Change my script to include most of the optimize_db.sh script but exclude the restart database and backend. I tied this briefly but the machine went to frantic hard disk activity and did not shutdown so I gave up and resorted to my normal method.
3. How about using this method to run backup regularly.

Any other thoughts? As I am very new to bash scripting any criticism is welcome.

_________________
Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest


Top
 Profile  
 
 Post subject:
PostPosted: Sun Apr 19, 2009 7:01 pm 
Offline
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location: Perth, Australia
Very cool, looks like you have got your head around the scripting beast! :)

I like you idea about incorporating this into the set-alarm-wakeup script. I use mythwelcome to powerup/powerdown, but in R6 I could not locate the KM/LH script to do it, so i reverted to the method in the wiki (mythtv/knoppmyth i cant remember which one).

Basically, i have a sctipt that has only a few commands in it, but could cut-n-paste your stuff into it, to run at that part of the sequence, rather than at shutdown... not sure if it would work as there may be more "database" things that happen "after" the set-wakeup-alarm script is called..

Nathan

_________________
LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 20, 2009 9:34 am 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Just to toss in a couple good scripting hygiene suggestions...

- You should almost always use double quotes around variable expansions, _especially_ variable expansions in tests. This prevents all sorts of nastiness. Even when you "know" it's not necessary it's a good habit to cultivate.

- I also generally recommend using the new POSIX $() expansion rather than `` since it's nesting behavior and interaction with other types of quoting is so much cleaner.

- You shouldn't need to specifically use bash to invoke optimize_db.sh if it has the right permissions (chmod +x) and #! magic.

Keep up the good work!


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 14, 2010 9:04 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
I finally got around to developing this further. My shutdown script now does three maintenance tasks on a cycle of 10 shutdowns. On the first shutdown it runs optimize_db.sh, on the second it runs mythbackup and on the third it defrags my /myth partition (xfs). All functions seem to work fine and I plan to monitor it for an extended period to make sure the defrag is keeping pace because my script only calls for a single defrag pass to minimise the time it runs.

Now I am wondering do I need to do the optimize_db step at all because mythbackup includes "myisamchk -f *.MYI" in its script. Reading from the MySQL Reference Manual the -f option performs the same function as the --recover option used in the optimize_db.sh. However, optimize_db.sh also includes a step with the --analyze option and a possible --safe-recover option if required, but I don't know how useful they are.

Is it still worth regularly running optimize_db.sh if mythbackup is being run just as frequently?

If anyone is interested I attach the script
Code:
#!/bin/bash
#
# shutdown_maint.sh
#
# Script to perform routine maintenance and housekeeping
# The script is invoked by mythwelcome on shutdown and is listed in mythwelcome -setup
# The script will perform the following on a cycle of 10 shutdowns
# Shutdown number 1 - run the optimize_db.sh script
# Shutdown number 2 - mythbackup
# Shutdown number 3 - defragment the /myth partition (xfs)
# Shutdown number 10 - reset counter to 0
 
# Extract shutdown number from file and increment
counter=`cat /home/mythtv/shutdown_maint/sdcount`
max=10
let "counter += 1"

# At first shutdown run optimize_bd script
if [ $counter -eq 1 ]; then
#   /usr/local/bin/optimize_db.sh
    echo `date` : Started optimize_db >> /home/mythtv/shutdown_maint/sd-maint.log
    echo `/usr/local/bin/optimize_db.sh` > /home/mythtv/shutdown_maint/opt.log
    echo `date` : Completed optimize_db.sh on shutdown >> /home/mythtv/shutdown_maint/sd-maint.log
fi

# At second shutdown run mythbackup
if [ $counter -eq 2 ]; then
#   /usr/local/bin/mythbackup
   echo `date` : Started mythbackup >> /home/mythtv/shutdown_maint/sd-maint.log
   echo `/usr/local/bin/mythbackup` > /home/mythtv/shutdown_maint/backup.log
   echo `date` : Completed mythbackup on shutdown >> /home/mythtv/shutdown_maint/sd-maint.log
fi

# At third shutdown defragment /myth partition
if [ $counter -eq 3 ]; then
#   xfs_fsr -p 1
   echo `date` : Started xfs defrag >> /home/mythtv/shutdown_maint/sd-maint.log
   echo `xfs_fsr -p 1` > /home/mythtv/shutdown_maint/defrag.log
   echo `date` : Completed Defrag on shutdown `xfs_db -c frag -r /dev/sda3` >> /home/mythtv/shutdown_maint/sd-maint.log
fi

# At tenth shutdown reset counter
if [ $counter -ge $max ]; then
   counter=0
fi

# Write new count number back to file
echo $counter >/home/mythtv/shutdown_maint/sdcount
 
# Shutdown as per normal
/sbin/poweroff

The sd-maint.log keeps a running log of the activities and each task writes to its own log file. For some reason the xfs_fsr step is not writing to the log file but it is performing the defrag. More investigation needed.

I am still running R5.5.

_________________
Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 14, 2010 10:21 pm 
Offline
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location: Perth, Australia
Brilliant stuff.... exactly the sort of thing that i required for a mythwelcome-enabled device..

The only thing remaining is the non-xfs disk-check that happens on boot.

Has anyone had any luck in the implementation of that?

Thanks for the script, i'll be implementing it tonight :)

Nathan

_________________
LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 10, 2010 7:44 pm 
Offline
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location: Perth, Australia
Hi All,

I havent implemented this yet, just creating the script to try tonight..

I have modified the script above to my requirements, only have really changed the log file location, and the wakealarm stuff added at the bottom.

Code:
#!/bin/bash
#
# shutdown_maint.sh
#
# Script to perform routine maintenance and housekeeping
# The script is invoked by mythwelcome on shutdown and is listed in mythwelcome -setup
# The script will perform the following on a cycle of 10 shutdowns
# Shutdown number 1 - run the optimize_db.sh script
# Shutdown number 2 - mythbackup
# Shutdown number 3 - defragment the /myth partition (xfs)
# Shutdown number 10 - reset counter to 0

# Extract shutdown number from file and increment
counter=`cat /var/log/mythtv/macdoodles-shutdowncount`
max=10
let "counter += 1"

# At first shutdown run optimize_bd script
if [ $counter -eq 1 ]; then
    echo `date` : Started optimize_db >> /var/log/mythtv/macdoodles-shutdown.log
    echo `/usr/LH/optimize_mythdb.py` > /var/log/mythtv/macdoodles-optimise.log
    echo `date` : Completed optimize_mythdb.py on shutdown >> /var/log/mythtv/macdoodles-shutdown.log
fi

# At second shutdown run mythbackup
if [ $counter -eq 2 ]; then
#   /usr/local/bin/mythbackup
   echo `date` : Started mythbackup >> /var/log/mythtv/macdoodles-shutdown.log
   echo `/usr/LH/mythbackup` > /var/log/mythtv/macdoodles-backup.log
   echo `date` : Completed mythbackup on shutdown >> /var/log/mythtv/macdoodles-shutdown.log
fi

# At third shutdown defragment /myth partition
if [ $counter -eq 3 ]; then
#   xfs_fsr -p 1
   echo `date` : Started xfs defrag >> /var/log/mythtv/macdoodles-shutdown.log
   echo `xfs_fsr -p 1` > /var/log/mythtv/macdoodles-xfsdefrag.log
   echo `date` : Completed Defrag on shutdown `xfs_fsr -p 1` >> /var/log/mythtv/macdoodles-shutdown.log
fi

# At tenth shutdown reset counter
if [ $counter -ge $max ]; then
   counter=0
fi

# Write new count number back to file
echo $counter >/var/log/mythtv/macdoodles-shutdowncount

# Clear the RTC wakealarm
echo 0 > /sys/class/rtc/rtc0/wakealarm      #this clears your alarm.
# Write out the wakeup time into RTC
echo $1 > /sys/class/rtc/rtc0/wakealarm     #this writes your alarm
# Write the RTC wakeup time into the log
echo [`date`] "Set Wakeup Time (`cat /sys/class/rtc/rtc0/wakealarm`)" >> /var/log/mythtv/macdoodles-setwakeup.log

# Shutdown as per normal
#/sbin/poweroff


Additionally, I have located some information allowing the non-xfs disck check to happen on shutdown, rather than on bootup (which can be tedious when your 3yrold want to watch fireman sam!)

http://bbs.archlinux.org/viewtopic.php?id=15496
http://www.xpmediacentre.com.au/community/mythtv-backend-only/41160-ext3-filesystem-check-boot.html

Basically, add a short code block to /etc/rc.shutdown (at the correct spot - after the filesystems have been remounted as RO)
Code:
# CUSTOM STUFF TO DO CHECKDISK
day=`date +"%d"`
runtoday=${day: -1}
if [ "$runtoday" = 2 -o "$runtoday" = 7 ]; then
if [ "$RUNLEVEL" = "0" ]; then
        stat_busy "Checking Filesystems"
        /sbin/fsck -A -T -C -a -- -f
        stat_done
fi
fi


Here is my current file, with the above code in the required spot.

Code:
#!/bin/bash
#
# /etc/rc.shutdown
#

. /etc/rc.conf
. /etc/rc.d/functions

# avoid staircase effect
/bin/stty onlcr

echo " "
printhl "Initiating Shutdown..."
echo " "

# avoid NIS hanging syslog-ng on shutdown by unsetting the domainname
if [ -x /bin/domainname ]; then
        /bin/domainname ""
fi

if [ -x /etc/rc.local.shutdown ]; then
        /etc/rc.local.shutdown
fi

#unmount network filesystems for runit
#if [ -e /var/service/netfs ]
#then
    /etc/rc.d/netfs stop
#fi

if [ "$PREVLEVEL" = "3" -o "$PREVLEVEL" = "5" ]; then
        # Shutdown daemons
        let i=${#DAEMONS[@]}
        while [ $i -ge 0 ]; do
                if [ "${DAEMONS[$i]:0:1}" != '!' ]; then
                        ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@}
                fi
                let i=i-1
        done
        # find any leftover daemons and shut them down in reverse order
        if [ -d /var/run/daemons ]; then
                for daemon in $(/bin/ls -1t /var/run/daemons); do
                        stop_daemon $daemon
                done
        fi
fi


# Terminate all processes
stat_busy "Sending SIGTERM To Processes"
/sbin/killall5 -15 &> /dev/null
/bin/sleep 5
stat_done

stat_busy "Sending SIGKILL To Processes"
/sbin/killall5 -9 &> /dev/null
/bin/sleep 1
stat_done

stat_busy "Saving Random Seed"
/bin/dd if=/dev/urandom of=/var/run/random-seed count=1 bs=512 2> /dev/null
stat_done

stat_busy "Saving System Clock"
if [ "$TIMEZONE" != "" -a -e "/usr/share/zoneinfo/$TIMEZONE" ]; then
        /bin/rm -f /etc/localtime
        /bin/cp "/usr/share/zoneinfo/$TIMEZONE" /etc/localtime
fi

HWCLOCK_PARAMS="--systohc"
if [ "$HARDWARECLOCK" = "UTC" ]; then
        HWCLOCK_PARAMS="$HWCLOCK_PARAMS --utc"
else
        HWCLOCK_PARAMS="$HWCLOCK_PARAMS --localtime"
fi
if [ "$USEDIRECTISA" = "yes" -o "$USEDIRECTISA" = "YES" ]; then
        HWCLOCK_PARAMS="$HWCLOCK_PARAMS --directisa"
fi
/sbin/hwclock $HWCLOCK_PARAMS
stat_done

# removing psmouse module to fix some reboot issues on newer laptops
/sbin/modprobe -r psmouse >/dev/null 2>&1

# Write to wtmp file before unmounting
/sbin/halt -w

stat_busy "Deactivating Swap"
/sbin/swapoff -a
stat_done

stat_busy "Unmounting Filesystems"
/bin/umount -a -r -t noramfs,notmpfs,nosysfs,noproc
stat_done

# Kill non-root encrypted partition mappings
if [ -f /etc/crypttab -a -n "$(/bin/grep -v ^# /etc/crypttab | /bin/grep -v ^$)" ]; then
        stat_busy "Deactivating encrypted volumes:"
        CS=/sbin/cryptsetup.static
        do_uncrypt() {
                if [ $# -ge 3 ]; then
                        stat_append "${1}.."
                        $CS remove $1 >/dev/null 2>&1
                        if [ $? -ne 0 ]; then
                                stat_append "failed "
                        else
                                stat_append "ok "
                        fi
                fi
        }
        while read line; do
                eval do_uncrypt "$line"
        CS=/sbin/cryptsetup.static
        do_uncrypt() {
                if [ $# -ge 3 ]; then
                        stat_append "${1}.."
                        $CS remove $1 >/dev/null 2>&1
                        if [ $? -ne 0 ]; then
                                stat_append "failed "
                        else
                                stat_append "ok "
                        fi
                fi
        }
        while read line; do
                eval do_uncrypt "$line"
        done </etc/crypttab
        stat_done
fi

if [ "$USELVM" = "yes" -o "$USELVM" = "YES" ]; then
        if [ -x /sbin/lvm -a -d /sys/block ]; then
                stat_busy "Deactivating LVM2 groups"
                /sbin/lvm vgchange --ignorelockingfailure -an >/dev/null 2>&1
                stat_done
        fi
fi

stat_busy "Remounting Root Filesystem Read-only"
/bin/mount -n -o remount,ro /
stat_done

# CUSTOM STUFF TO DO CHECKDISK
day=`date +"%d"`
runtoday=${day: -1}
if [ "$runtoday" = 2 -o "$runtoday" = 7 ]; then
if [ "$RUNLEVEL" = "0" ]; then
        stat_busy "Checking Filesystems"
        /sbin/fsck -A -T -C -a -- -f
        stat_done
fi
fi

# Power off or reboot
if [ "$RUNLEVEL" = "0" ]; then
        printsep
        printhl "${C_H2}POWER OFF"
        /sbin/poweroff -d -f -h -i
else
        printsep
        printhl "${C_H2}REBOOTING"
        # if kexec is installed and a kernel is loaded, use it
        [ -x /sbin/kexec ] && /sbin/kexec -e > /dev/null 2>&1
        /sbin/reboot -d -f -i
fi

# End of file
# vim: set ts=2 sw=2 noet:


Also, I have had to add the following line to /usr/LH/bin/mythbackup to enable me to pull the backups to work from my KM at home. since the security is tied down to a single user to log in (mythtv), I need the mythbackup files to be readable by that user (mythtv) when using pscp (putty secure copy). Not sure if this is something that could be rolled into KM, as it is quite specific.

Code:
# If you can't read this you've got no business restoring from it anyway.
$CHOWN root:root $BACKUP_TAR* $BACKUP_SQL*
$CHMOD go-rwx $BACKUP_TAR* $BACKUP_SQL*
$CHMOD o+r $BACKUP_TAR* $BACKUP_SQL*


if this works, it means that i have my mythbox regularily backed up, optimised, disk checked, and mythwelcome-wakeup-enabled. Perfect.

Cheers,

Nathan

_________________
LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector


Last edited by nmcaullay on Mon May 10, 2010 10:22 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 10, 2010 9:40 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
That looks very impressive. I only wish my limited knowledge would allow me to understand more than 5% of it.

I am in the process of planning the upgrade to R6 so your mods to my script are interesting and I have a couple of questions.
1. Is it better to keep logs in the /var/log/mythtv directory or is it just preference? Would it explain why my defrag.log refuses to work?
2. Why is it necessary to clear and re-write the the wake-up time to /sys/class/rtc/rtc0/wakealarm?

For you new script I gather that most of it is involved in safely unmounting the partition to allow you to run fsck. I assume since you included the defrag part of the previous script that your /myth partition is xfs. Why then do you need to force the fsck? The boot time increase for fsck of the / partition is trivial so I leave that to the normal 30 mountings.

By the way, my script has been running successfully for 2 months now and the single pass of defrag keeps the fragmentation around 1%. Maybe I could reduce the frequency a bit but I don't think I will bother.

_________________
Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 10, 2010 9:59 pm 
Offline
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location: Perth, Australia
Its all very nice in theory... i have to try it out tonight :)

Actually, if your script was based on R5.5, i'll need to check that the binaries referenced in the script are still in the same place.

1. I use /var/log/mythtv as a preference. The other myth stuff seems to get logged there. I check every now and then, and delete the odd file, so dont know much about the rotation etc.

2. The clear and rewrite of RTC is something that was suggested somewhere along the way. It works, so i'm not going to try to figure it out :)

I have a second drive that is XFS, so your script will nicely keep that unfragmented. My first drive is a EXT3? which has LH on it, and it is subject to the disk check timesink on boot. So, without using your script (and the rc.shutdown mods) i have the following
* an EXT3 drive has a 1-in30 disk check that always happens at an inappropriate time, and
* an XFS drive that busily gets itself fragmented.

BTW, the "new" script is an existing Arch one (i think), with about 5 lines inserted by me. I dont understand what is going on in there either :)

_________________
LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 10, 2010 10:23 pm 
Offline
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location: Perth, Australia
Updated the script i posted to point to the correct optimise and mythbackup locations. Wouldnt have worked before on a R6.

also, another question.
1) what does "xfs_fsr -p 1" do. doing a man page doesnt show any "-p" options?

Cheers,

Nathan

_________________
LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 10, 2010 11:08 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
You are right that to "-p" option is not mentioned in the man page but I found it in xfs_fsr --help. From my understanding the -p option is for the number of passes over the partition, so my setting of "-p 1" is the minimum of a single pass. For my 500GB disk it takes about 25 minutes if it has some work to do but can be very quick if the disk is not fragmented. I thought that would be better than setting a fixed time.

I would be interested to hear from others if my understanding is correct.

_________________
Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest


Top
 Profile  
 
 Post subject:
PostPosted: Tue May 11, 2010 1:28 am 
Offline
Joined: Mon Jun 21, 2004 5:28 am
Posts: 700
Location: Germany
Just saw this thread and I wanted to add a couple of comments about using cron/mythwelcome.

I recommend a daily wakeup with mythwelcome, even if you don't use the cron method. Otherwise, if you ever shutdown with no scheduled recordings, it will not wakeup again by itself. This can be a problem as no schedules will be updated. If you watch TV every day then this may not be an issue.

As for the cron job possibly colliding with other backend jobs (ex. recording). It can be fixed by checking the status first. For example:
Code:
#!/usr/bin/perl

while (($status=system("/usr/bin/mythshutdown -s")/256) != 64) {
        # Wait until status is 64
        # (Idle, in daily wakeup and not about to record)
        sleep(20);
}

system("/usr/bin/mythshutdown -l");

system("/usr/LH/optimize_mythdb.py");

$day=`date '+%a'`;
chomp($day);
if($day eq "Tue") {
   # do weekly stuff....
}

system("/usr/bin/mythshutdown -u");

_________________
ASUS AT3N7A-I (Atom 330)
TBS 8922 PCI (DVB-S2)


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 28, 2011 9:16 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
Some developments since my previous posts. Earlier this year I had a hard disk failure. The disk failed totally (click of death) and I lost / and /myth including my backups which were all in the /myth partition. My second drive (/myth2) only had some of the recorded programs.

I bought a new drive, reinstalled and repaired the database to what recordings I still had. I also had to reload my music, videos and photos. It was not a small task so I decided to do something smarter this time. I modified my maintenance script and added the facility to create a secondary backup on the /myth2 partition. It backs it up 5 deep which is a bit excessive and I may cut it down a bit. I also included a step to copy videos, music and photos to a backup directory on /myth2. I now have a total of 3 TB storage so I don't even bother compressing. If I have another disk failure (haven't I had my share?) I should only lose some of the recordings. Anything valuable I want to keep I will transcode and put in videos.

I had also modified the defrag steps so that it does one disc at a time and I also went back to a time limit of 20 minutes rather than the passes option.

The script now looks like
Code:
#!/bin/bash
#
# shutdown_maint.sh
#
# Script to perform routine maintenance and housekeeping
# The script is invoked by mythwelcome on shutdown and is listed in mythwelcome -setup
# The script will perform the following on a cycle of 10 shutdowns
# Shutdown number 1 - run the optimize_db.sh script
# Shutdown number 2 - mythbackup
# Shutdown number 3 - defragment the /myth partition (xfs)
# Shutdown number 4 - defragment the /myth2 partition (xfs)
# Shutdown number 5 - backup videos, music and photos
# Shutdown number 10 - reset counter to 0
 
# Extract shutdown number from file and increment
counter=`cat /home/mythtv/sd-maint/sdcount`
max=10
let "counter += 1"

# At first shutdown run optimize_bd script
if [ $counter -eq 1 ]; then
#   /usr/share/mythtv/contrib/maintenance/optimize_mythdb.pl
    echo `date` : Started optimize_db >> /home/mythtv/sd-maint/sd-maint.log
    echo `/usr/share/mythtv/contrib/maintenance/optimize_mythdb.pl` > /home/mythtv/sd-maint/opt.log
    echo `date` : Completed optimize_db.sh on shutdown >> /home/mythtv/sd-maint/sd-maint.log
fi

# At second shutdown run mythbackup
if [ $counter -eq 2 ]; then
#   /usr/LH/bin/mythbackup
   echo `date` : Started mythbackup >> /home/mythtv/sd-maint/sd-maint.log
   echo `/usr/LH/bin/mythbackup` > /home/mythtv/sd-maint/backup.log
   echo `date` : Completed mythbackup >> /home/mythtv/sd-maint/sd-maint.log

#    Rollover secondary (myth2) backup files - 5 deep
   mv -f /myth2/sec-backup/mythconverg.sql.gz.4 /myth2/sec-backup/mythconverg.sql.gz.5
   mv -f /myth2/sec-backup/mythconverg.sql.gz.3 /myth2/sec-backup/mythconverg.sql.gz.4
   mv -f /myth2/sec-backup/mythconverg.sql.gz.2 /myth2/sec-backup/mythconverg.sql.gz.3
   mv -f /myth2/sec-backup/mythconverg.sql.gz.1 /myth2/sec-backup/mythconverg.sql.gz.2
   mv -f /myth2/sec-backup/mythconverg.sql.gz /myth2/sec-backup/mythconverg.sql.gz.1

   mv -f /myth2/sec-backup/savedfiles.tar.gz.4 /myth2/sec-backup/savedfiles.tar.gz.5
   mv -f /myth2/sec-backup/savedfiles.tar.gz.3 /myth2/sec-backup/savedfiles.tar.gz.4
   mv -f /myth2/sec-backup/savedfiles.tar.gz.2 /myth2/sec-backup/savedfiles.tar.gz.3
   mv -f /myth2/sec-backup/savedfiles.tar.gz.1 /myth2/sec-backup/savedfiles.tar.gz.2
   mv -f /myth2/sec-backup/savedfiles.tar.gz /myth2/sec-backup/savedfiles.tar.gz.1

#    Copy latest backup files to secondary backup (myth2)
   sudo cp -f /myth/backup/mythconverg.sql.gz /myth2/sec-backup/mythconverg.sql.gz
   sudo cp -f /myth/backup/savedfiles.tar.gz /myth2/sec-backup/savedfiles.tar.gz

   echo `date` : Completed secondary backup >> /home/mythtv/sd-maint/sd-maint.log
fi

# At third shutdown defragment /myth partition
if [ $counter -eq 3 ]; then
   echo `date` : Started xfs defrag of /myth >> /home/mythtv/sd-maint/sd-maint.log
   echo `sudo xfs_fsr -v -t 1200 -m /home/mythtv/sd-maint/mtab_sda3` > /home/mythtv/sd-maint/defrag.log
   echo `date` : Completed Defrag of sba3 on shutdown `xfs_db -c frag -r /dev/sda3` >> /home/mythtv/sd-maint/sd-maint.log
fi

# At fourth shutdown defragment /myth2 partition
if [ $counter -eq 4 ]; then
   echo `date` : Started xfs defrag of myth2 >> /home/mythtv/sd-maint/sd-maint.log
   echo `sudo xfs_fsr -v -t 1200 -m /home/mythtv/sd-maint/mtab_sdb1` > /home/mythtv/sd-maint/defrag.log
   echo `date` : Completed Defrag of sdb1 on shutdown `xfs_db -c frag -r /dev/sdb1` >> /home/mythtv/sd-maint/sd-maint.log
fi

# At fifth shutdown backup videos, music and photos
if [ $counter -eq 5 ]; then
   echo `date` : Started media backup >> /home/mythtv/sd-maint/sd-maint.log
   echo `date` > /home/mythtv/sd-maint/media-backup.log
   echo `rsync -vrt --delete /myth/video /myth2/media-backup` >> /home/mythtv/sd-maint/media-backup.log
   echo `rsync -vrt --delete /myth/music /myth2/media-backup` >> /home/mythtv/sd-maint/media-backup.log
   echo `rsync -vrt --delete /myth/gallery /myth2/media-backup` >> /home/mythtv/sd-maint/media-backup.log
   echo `date` : Completed media backup >> /home/mythtv/sd-maint/sd-maint.log
fi

# At tenth shutdown reset counter
if [ $counter -ge $max ]; then
   counter=0
fi

# Write new count number back to file
echo $counter >/home/mythtv/sd-maint/sdcount
 
# Shutdown as per normal
/sbin/poweroff

_________________
Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ] 
Go to page 1, 2  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu