View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 51 posts ] 
Go to page Previous  1, 2, 3, 4  Next

Print view Previous topic   Next topic  
Author Message
Search for:
 Post subject: R5D1 updates
PostPosted: Sat Aug 26, 2006 10:36 am 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Upcoming in R5D1, the backup and restore scripts now automatically sanity check the results. The builtin sanity checks are much faster than the original rough version posted here, and can also be run manually. The current versions, which can be found in /usr/local/bin are:

- mythbackup - backup and verify
- mythrestore - restore and verify
- checkbackup - verify that the current backup matches the system state.
- checkrestore - verify that the restored subset matches the system state.

- rollback.sh - A testing utility for discarding the newest backup, and rolling back to the previous one. Note that this does not do a restore. It just does a destructive "pop" on the backup stack. Caveat Utor!!!


Last edited by tjc on Wed Dec 27, 2006 7:46 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject: valid backup.list file?
PostPosted: Sun Sep 03, 2006 6:32 pm 
Offline
Joined: Tue Mar 28, 2006 8:26 pm
Posts: 804
Location: Minneapolis, MN
Could someone post a valid backup.list file?

I've started a backup.list file and I'm not sure if my commands are valid. I'd like to backup all the scripts I've got sitting in /usr/src.
Code:
/usr/src/redoconfig.sh


Here's what my restore.list looks like so far
Code:
/usr/src/redoconfig.sh
/etc/X11/XF86Config-4

Look ok?

In R5D1, what backup file do my additional backup selections go into?

Eric

_________________
KnoppMyth R5.5, Asus A8N-VM CSM (nvidia 6150 onboard video), AMD Athlon 64 dual-core 4200+, two 1GB sticks DDR 400, HD-3000 HDTV card, PVR-150 card, Iguanaworks RS-232 IR receiver/transmitter, Pioneer DVR-110 DVD burner


Top
 Profile  
 
 Post subject:
PostPosted: Mon Sep 04, 2006 10:51 am 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Not to be mean, but you keep asking questions on things that are already called out very clearly in the original documentation posting. You really need to go back and read that posting carefully. Do not skim, do not try to cherry pick, read it from start to finish and if necessary take notes.

All the files go into a single unified archive. This is the very first feature listed for the new backup.

There is a valid example backup.list and restore.list provided in the original posting.

It is explicitly called out that you need to start each entry in the lists with a './' (dot slash) rather than just a '/' (slash), and every directory or file name shown as part of the archive contents includes this to make sure the point is clear.


Top
 Profile  
 
PostPosted: Fri Sep 08, 2006 8:19 am 
Offline
Joined: Tue Mar 28, 2006 8:26 pm
Posts: 804
Location: Minneapolis, MN
TJC,

OK. I now see the"./" line at the end of a paragraph in your forum post. Sorry, I missed that. :oops:

But, my other question is not answered anywhere in your forum posting, so I'd still like it clarified. I'd like to backup all the scripts I've got sitting in /usr/src directory. All your examples call out 1 specific file and not all files of a certain filetype. See what I'm getting at?

So, what would a line in the backup.list file look like that would backup only the "xxxxx.sh" files in that directory? Is it?
Code:
./usr/src/*.sh


Thanks,
Eric

_________________
KnoppMyth R5.5, Asus A8N-VM CSM (nvidia 6150 onboard video), AMD Athlon 64 dual-core 4200+, two 1GB sticks DDR 400, HD-3000 HDTV card, PVR-150 card, Iguanaworks RS-232 IR receiver/transmitter, Pioneer DVR-110 DVD burner


Top
 Profile  
 
 Post subject:
PostPosted: Fri Sep 08, 2006 5:03 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Well, you asked about the scripts yes, but you never asked anything about wildcards until just now.

Frankly, I've never tried using them myself. I would recommend explictly listing the files you're interested in to avoid nasty accidents, however, I believe that the wildcards will work. The best way to find out for sure is probably just to try it.


Top
 Profile  
 
PostPosted: Tue Sep 12, 2006 9:36 am 
Offline
Joined: Tue Mar 28, 2006 8:26 pm
Posts: 804
Location: Minneapolis, MN
TJC, I took your advice and am not using wildcards. I added a few custom backup lines to /myth/backup/backup.list and /myth/backup/restore.list.

Cecil's voice announces that it successfully backsup now, with the addition of the ./ at the start of the lines. :D
To be doublly sure, I also checked the backup file contents with
Code:
tar jtvf /myth/backup/savedfiles.tar.bz2


I'm think I'm at the point of having really easy upgrades now. Ahhhhhhhh. :)

Eric

_________________
KnoppMyth R5.5, Asus A8N-VM CSM (nvidia 6150 onboard video), AMD Athlon 64 dual-core 4200+, two 1GB sticks DDR 400, HD-3000 HDTV card, PVR-150 card, Iguanaworks RS-232 IR receiver/transmitter, Pioneer DVR-110 DVD burner


Last edited by neutron68 on Fri Dec 01, 2006 4:48 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Sep 12, 2006 5:42 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
That's actually part of what the backup checks now do for you. They list the contents of the backup file and compare them against the cuurent directory contents.

Remember not to sit on that backup too long or it will already be out of date when you do the upgrade. I tend to do the backup immediately before rebooting with the CD.

If I get sidetracked for more than a few minutes between doing the backup and rebooting I rerun the check. I've had that save me at least once now.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Sep 22, 2006 6:18 pm 
Offline
Joined: Tue Apr 11, 2006 7:44 am
Posts: 286
Location: Los Angeles, CA
mihanson wrote:
You can get the script rather easily if you have R5C7 on a CD-R...

Quote:
- Boot from the CD as usual, but take option 6 (Exit) from the menu. This wil ask for confirmation then drop you to a command prompt.

- Type mount /dev/hda1 and hit enter. You can now access your root partition on /mnt/hda1. (Note from mihanson: this assumes your installation of R5B7 is on hda1. Substitute sda1 or whatever if you have a non-standard install)


- Move the old script. mv /mnt/hda1/usr/local/bin/mythbackup /mnt/hda1/usr/local/bin/mythbackup.R5B7

- Type cp -a /usr/local/bin/mythbackup /mnt/hda1/usr/local/bin/ and hit enter.

This should copy the new backup script to your existing install. Just pop the CD and reboot.


Hello, I tried doing this, but with an R5D1 CD and wound up with all sorts of errors (which I can post if they will be helpful) when running mythbackup. With the changes in R5D1, does more need to be done to upgrade an R5B7 PC to the new backup scripts?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Sep 22, 2006 7:39 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Yes. The backup/restore/check/rollback scripts now use a common library of variable and function definitions called "backupcommon", however, I don't really recommend taking that route. The R5D1 Upgrade Hints and the backup/restore page on the wiki include a simple command which will add the right tarball to an old style backup to effectively convert it to a new style backup. You can also get a script for checking the validity of the backup from the wiki page. Recommended reading for all your R5D1 upgrade preparations: http://mysettopbox.tv/phpBB2/viewtopic.php?t=11293
http://www.knoppmythwiki.org/index.php?page=MythBackupAndRestoreHowTo


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 30, 2006 11:21 am 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Please note that in R5D1 there is a known issue if you've enabled "logging to the database". The frontend can continue to log messages into the DB while the backup is running, thus changing the record counts the script uses to verify the success of your backup. To avoid this you may want to temporarily disable the logging, or shutdown the front end and run the mythbackup command in a shell session.

Otherwise if you get an error, open a shell window, run the checkbackup script, and if the only difference is the record counts in the "mythlog" DB table you can ignore them.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Nov 17, 2006 8:07 pm 
Offline
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location: Los Angeles
tjc,
I'm having a problem on my R5C7 install and the rollingover of mythconverg dumps.
Code:
mythtv@mythbox:~$ ls -l /myth/backup/mythconverg.*
-rw-------  1 root root 88610429 Nov 17 05:00 /myth/backup/mythconverg.sql
-rw-------  1 root root 27686000 Nov  7 05:00 /myth/backup/mythconverg.sql.bz2
-rw-------  1 root root 27655088 Nov  6 08:25 /myth/backup/mythconverg.sql.bz2.1
-rw-------  1 root root 27654514 Nov  6 05:01 /myth/backup/mythconverg.sql.bz2.2
-rw-------  1 root root 27641938 Nov  5 05:01 /myth/backup/mythconverg.sql.bz2.3
-rw-------  1 root root 27624859 Nov  4 05:01 /myth/backup/mythconverg.sql.bz2.4
-rw-------  1 root root 27625687 Nov  3 05:01 /myth/backup/mythconverg.sql.bz2.5
-rw-------  1 root root 25069190 Nov  2 05:01 /myth/backup/mythconverg.sql.bz2.6
-rw-------  1 root root 25069254 Nov  1 05:01 /myth/backup/mythconverg.sql.bz2.7
-rw-------  1 root root 25067202 Oct 31 05:00 /myth/backup/mythconverg.sql.bz2.8
-rw-------  1 root root 25066816 Oct 30 05:00 /myth/backup/mythconverg.sql.bz2.9


As you can see, once I got to 10 backups, the oldest is not being removed when a new backup is made. I run mythbackup as a root cronjob everyday at 5AM.

Code:
00 05 * * * /usr/local/bin/mythbackup


I do not think I've made any modifications to the script that came with R5C7.

Code:
root@mythbox:~# cat /usr/local/bin/mythbackup
#!/bin/bash

#----------------------------------------------------------------------------

SOUNDS="/usr/share/sounds"
DATABASE="mythconverg"
BACKUP_LIST="./etc ./root ./home"
BACKUP_DIR="/myth/backup"
BACKUP_EXTRAS="$BACKUP_DIR/backup.list"
BACKUP_TAR="$BACKUP_DIR/savedfiles.tar.bz2"
BACKUP_SQL="$BACKUP_DIR/$DATABASE.sql"
PLAYER="aplay"
#----------------------------------------------------------------------------
# Play a sound to let you know I'm starting.
($PLAYER $SOUNDS/beg.wav >& /dev/null)&

#----------------------------------------------------------------------------
# Keep a chain of recent backups,
echo "Starting rollover of old backups, this may take a while..."
[ -f "$BACKUP_SQL" ] && bzip2 $BACKUP_SQL
for f in $BACKUP_SQL.bz2 $BACKUP_TAR ; do
    for i in .9 .8 .7 .6 .5 .4 .3 .2 .1 '' ; do
        [ -f "$f$i" -a -n "$old_f" ] && mv -f $f$i $old_f
        old_f="$f$i"
    done
done
echo "Rollover completed."
#----------------------------------------------------------------------------
# Start with the database backup, first we make sure it's healthy, and
# then we can dump it.

# Doing this while the backend is active would be BAD.
/etc/init.d/mythtv-backend stop

# Stop, check, and fix $DATABASE db to ensure clean copy, then restart it.
/etc/init.d/mysql stop
cd /var/lib/mysql/$DATABASE
myisamchk -f *.MYI
/etc/init.d/mysql start

# Dumps the $DATABASE database
mysqldump -c -u root $DATABASE > $BACKUP_SQL

# Now we can restart the backend.
/etc/init.d/mythtv-backend start

#----------------------------------------------------------------------------
# Now to backup the other files, no fooling around, grab everything in the
# list because you never know what you'll want, and we can always get clever
# about what to restore later...

[ -f "$BACKUP_EXTRAS" ] &&
    BACKUP_LIST="$BACKUP_LIST $(tr '\n' ' ' <$BACKUP_EXTRAS)"

# gather all the things in the list into a nice tidy bundle
cd /
tar jcvf $BACKUP_TAR $BACKUP_LIST

# If you can't read this you've got no business restoring from it anyway.
chmod go-rwx $BACKUP_TAR $BACKUP_SQL

#----------------------------------------------------------------------------
# Play a sound to let you know I'm done.
($PLAYER $SOUNDS/end.wav >& /dev/null)&


Any ideas?

_________________
Mike
My Hardware Profile


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 01, 2006 3:49 pm 
Offline
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location: Los Angeles
Interesting... It seems that if I delete all my mythconverg.sql* files from /myth/backup, but leave my savedfiles.tar.bz2* files, the problem with rollovers I mentioned in my previous post occurs. However, if I delete all my mythconverg.sql* and savedfiles.tar.bz2* files, effectively starting over with all my backup files, everything seems fine. Is there a reason why only deleting the db backups throws off the rollover portion of the script? :?

_________________
Mike
My Hardware Profile


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 01, 2006 10:27 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Memory tells me there _was_ some kind of issue with the older versions of the rollover, but the current code is just fine. In the latest incarnation the rollover function looks like this:
Code:
backup_roller () { # Gets the rollover sequence to use.
    prev_i=$1 ; shift
    for i in "$@" ; do
        for c in .gz .bz2 "" ; do
            for f in $BACKUP_SQL $BACKUP_TAR ; do
                rm -f $f$c$prev_i
                if [ -f "$f$c$i" ] ; then
                    echo "Moving $f$c$i to $f$c$prev_i"
                    mv -f $f$c$i $f$c$prev_i
                fi
            done
        done
        prev_i="$i"
    done
}

And gets called like this:
Code:
backup_roller .9 .8 .7 .6 .5 .4 .3 .2 .1 ''

However I'm a bit too fried out tonight to figure out what I fixed along the way... The modern versions are very different internally.


Top
 Profile  
 
 Post subject: R5E50 updates
PostPosted: Wed Dec 27, 2006 4:15 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Changes in R5E50:

- Default backup list is now:
Code:
./root ./home ./etc ./var/www/mythweb/.htaccess


- Default restore list is now:
Code:
./root ./home ./etc/mythtv/modules ./etc/lirc ./etc/htdigest ./etc/apache2/apache2.conf ./var/www/mythweb/.htaccess


- The scripts are now more tolerant of restore.list and backup.list entries that don't start with "./", and nomalize them to the standard format.

- The default compression used for the backup files is now gzip. Both the saved files and mysql dump archives are compressed. This change was made for the ~20x speedup in backup times over bzip2. The size difference is insignificant compared to the size of other things in the /myth partition, but if you're truly obsessive-compulsive about disk space usage, running the gzip_to_bzip2.sh script will intelligently recompress things for you:
Code:
gzip_to_bzip2.sh /myth/backup/savedfiles.tar* /myth/backup/mythconverg.sql*


- To make the change above transparent, the scripts are now able to automatically detect and handle backup files that are either 1) compressed with gzip, 2) compressed with bzip2, 3) uncompressed. To avoid confusion you must make sure that there is only one form of the /myth/backup/savedfiles.tar* and/or /myth/backup/mythconverg.sql* files. This does not include the version numbered files (i.e. /myth/backup/*.[1-9]).

- The scripts now automatically verify that every backup and restore produced the intended results, and are fairly immune to "crying wolf". Any backup or restore failure messages should be investigated and possibly reported.

- There are now post restore fixups which get run automatically to upgrade certain files. This change simply formalizes and provides a standard location for upgrade edits which were being done elsewhere in the scripts.


Top
 Profile  
 
PostPosted: Thu Jan 11, 2007 10:44 am 
Offline
Joined: Tue Aug 16, 2005 11:50 am
Posts: 181
Location: Douglasville, GA
TJC,

I've done a lot of changes to the mythtv user since I did my first install, many upgrades ago. I get the feeling that by restoring my /home/mythtv I am missing out on some benefits of the newer builds. Even if I am not, I want to see "how the other half lives".

My idea is to
  1. do a backup
  2. create my own tar file of /home/mythtv
  3. remove ./home/mythtv from savedfiles.tar.gz
  4. boot from install CD and upgrade


Can you see any flaws in this? Will I get the same /home/mythtv as a clean install if I do that?

I then plan to restore things from my ~/.[bashrc|fluxbox|mozilla|mplayer|mythtv|screenrc|ssh|vim*] as I encounter a need.

I'm also wondering if I need to use some defaults from /etc also. I think I may pull the ./etc/profile & ./etc/bash.bashrc files from savedfiles.tar.gz. I know that in order to get my ~/.bashrc to load after my upgrade to R5E#, I had to do some serious horking around.

I've not seen a "what you are missing out on by not doing a clean install" thread. Is there a discussion on this? Is it worth having?


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 51 posts ] 
Go to page Previous  1, 2, 3, 4  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu