View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 5 posts ] 
Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Mon Jul 07, 2014 10:51 am 
Offline
Joined: Tue Jul 01, 2014 6:49 am
Posts: 81
Migrating from Mythdora to LinHES or
“What I did for my entire Summer Vacation”

Note,  Since I could not find a specific guide on how to migrate from Mythdora to LinHES, I thought it might be nice to attempt to document the process and save the next poor bastard a ton of frustration and time on research. I’m no Linux expert, there are probably much easier ways to do what I’ve done here, but these are the steps that worked for me, YMMV. Good luck!

I did try and do the various “migrating a backend” steps outlined on mythtv.org, but had no success doing the steps outlined, nor did I have any success with partial database restores or other forms of recommended migration paths. So what you see here is a compilation of steps from many different websites that allowed me to migrate from Mythdora to LinHES, a huge thanks to these “unsung heroes” who took the time to post the snippets I was able to borrow and use successfully!

So with Mythdora being shelved 4 or so years ago and the Mythtv version sitting at 0.24 fixes, I felt it was time to move on to a package that would get me into current versions of Mythtv. Doing some looking around, I read up on LinHES and it fit the bill for what I was looking for.

My setup was 4 Zotac Atom/Ion based boxes with 1 MBE/FE and 3 SBE/FE. My system has been up for around 4 years and I had about 270 shows recorded (about 1.4 TB of video) that I wanted to migrate (mandated by WAF requirements).

Having all of that extra hardware to play with, it was no big deal to take one of the SBE/FE boxes and repurpose it for LinHES.

Luckily I picked one of the boxes that we used very rarely, this was a good news/bad news type of thing, with the bad news being the SATA chipset on the MB was dying. As I struggled with figuring out what was going on, I kept thinking to myself “20 minute setup, my A$$”, I then had to humbly apologize for these thoughts once I figured out it was my hardware and not the distro causing the issues.

Side note, you really can set one of these up in 20 minutes! I did it multiple times while playing with configurations and database migration and finally when I repurposed the final Mythdora boxes to LinHES.

Back to the migration, I needed to replace the failing MB, so I went to the local big box PC store and wanted to find something close to my other machines, what I ended up with was a Asus H81i MB with a pentium CPU system. If you should find yourself in this situation, and you are most familiar with  Atom/Ion boxes hold out until you can get what you want. While I have the Pentium based box working, I spent way too much time (days) trying to get it to work, while my Atom/Ion boxes all came up and were online and configured in ~ an hour each.

So high-level steps on what I did to migrate from Mythdora to LinHES:
1. Create Backup of Mythdora Data Base (DB).
2. Copy Mythdora DB backup and recording files to external backup drive.
3. Repurpose existing Mythdora SBE/FE box to LinHES 8.0 MBE (Load LinHES).
4. Create LinHES backup DB. (Basically an empty database, but with all of the other things LinHES backs up that are not in a Mythdora backup)
5. Restore Mythdora DB into LinHES MBE, let Mythtv update DB from 0.24 fixes to 0.25 fixes schema.
6. Export important MYSQL tables to file. (Mythdora recording/recorded/recording rules etc.)
7. Restore LinHES backup DB created in step 4 above.
8. Import MYSQL tables created in step 6 above into LinHES DB. Create backup of LinHES info updated with Mythdora tables.
9. Upgrade (in my case re-install) LinHES version 8.1.1
10. Restore LinHES version backup created in step 8, let version 8.1.1 update DB schema to 0.27.1 schema
11. Copy recording files to correct storage directory, change permissions on those files
12. Tweak settings so that system operates same as previously, drink heavily….

Sounds easy doesn’t it…So on to the details of what worked for me:

Naming conventions:
New LinHES Master Back End (MBE) system, mean the hardware (CPU, Monitor, Keyboard) of the system you are upgrading
For my intel box, trying to use the standard boot, I would get a black screen and could not go further. To get it working, I had to use the LinHES menu option of Boot with force vesa (special hardware option->force vesa) and add nomodeset to the command line.) 
For my Atom/Ion boxes I just booted as normal, nothing special required.
Separate PC, means any other device with a web browser and USB drive
Mythdora system, means the existing Mythdora Master Back End.
Mythdora SBE/FE other Mythdora system used to create bootable LinHES USB install device.


Steps used:
NOTE: if you are going to use this guide, Please read the text before the code sections, I use that area to try and give a heads up on what is being done, or if something has a variable name and I cannot give specific example that will work.

On the Mythdora system do an optimize db from MBE FE GUI (database options/optimize). This will also create the backup to be used.
Using the Fedora file browser via the GUI, Copy mythconverg.sql.gz (or what ever the backup name actually is) to a thumb drive or other storage media.
Put this backup somewhere safe.

Via the web browser, download the LinHES R8 iso to a Mythdora SBE/FE combo’s (NOT the MBE) so as not to mess up the Master Back End.

Create bootable LinHES R8.0 installation USB
Inserted a flash drive and verified that it had mounted as /dev/sdb (if mounted as something other than /dev/sdb, change /dev/sdb below to reflect actual mount)
Open terminal, login as root
Code:
su

Goto the location of the .iso file (desktop in my case). cd to actual directory where .iso file is located.
Code:
cd ???

Create bootable LinHES USB drive, be sure to change /dev/sdb to the actual mounting location of the usb, be very careful as this will wipe what ever device you enter here:
Code:
dd if=LinHES_R8.0.iso of=/dev/sdb &&sync --verbose

Eject USB drive.
You are done with the Mythdora system for now.

Plug the USB thumb drive into the LinHES MBE machine,  
Reboot the LinHES MBE and entered the Bios setup, set it to boot from USB, then HDD.
Reboot again, system should boot from flash drive
Follow the prompts to install 8.0.
After the install is done and you are prompted to reboot/shutdown, select shut down, once it is done, remove the thumb drive and power it back on. Let it boot to the setup screens and stop. (don’t do anything yet via the setup screens)
On a separate PC, open a web browser and enter the IP address of the LinHES MBE machine, or you can enter host.local, where host is the host name of the LInHES machine.  
This will open up the LinHES web interface, click the health and maintenance option from the menu on the left.
The host name should show up as a hyperlink (it may take a while after initial boot) click the link, then select the backup system option and click go.
This will create a backup with an empty database. (I call this the R8 gold DB) make note of the file name as you will need to reload this file in a later step. This can take a while and it will look like nothing is happening, wait until the screen changes to show the success message, you will see a long list of items that are included in the LinHES backup.
You should be able to see the backup file name using the Download Backup drop down. Record the name of the backup, it should be something like: backup.2014-06-24_00-59.tgz
If you want to verify the backup via terminal enter:
Code:
cd /data/storage/disk0/backup/system_backups

On the same PC, plug in the thumb drive with the Mythdora backup.
Again go to Health and Maintenance, click the backend hyperlink and using the upload file to restore option, Upload the Mythdora backup (mythconverg.sql.gz) file.
The file will be placed at /data/storage/disk0/backup/system_backups.
Now to restore the Mythdora DB to the LinHES MBE. On the LinHES web interface left side menu, click System, then terminal to open a terminal 
login as: mythtv and enter the password you created during setup.
Switch to root:
Code:
su

go to: 
Code:
cd /usr/share/mythtv

To import the Mythdora db into the LinHES MBE db enter:
Code:
./mythconverg_restore.pl --drop-database --create-database  --directory=/data/storage/disk0/backup/system_backups --filename=mythconverg.sql.gz --verbose

To change the host name in the database (change Mythdora_Host and LinHES_Host to the actual host names of the old Mythdora MBE and the new LinHES MBE):
Code:
./mythconverg_restore.pl --change_hostname --old_hostname=Mythdora_Host --new_hostname=LinHES_Host --verbose

You have just uploaded your Mythdora 0.24 db and changed the old host name references in the db to the new hostname.
Using the LinHES MBE system keyboard on the R8 setup screen select General and change DB IP address to the address of the LinHES MBE, then exit setup.
Once the front end is launched hit Alt-h to view the various keyboard control items available, we will want to go back to setup so that the DB Schema is updated.
Hit Alt-s to go back to setup (it will take a while), when prompted to update DB, say yes.

Congrats, at this point your .024fixes database should now be updated to .025fixes, you will now need to save (dump) the important tables to a file (or multiple files) for import into fresh LinHES database. at this point you could do another backup, but there is no need and might only confuse some of the following steps.

Next dump tables to files
dump file:
(on a Separate PC open a terminal via the web-admin or on the LinHES MBE via Alt-x)

Enter root:
Code:
su


Enter:
Code:
mysqldump -u mythtv -p -t mythconverg record recorded oldrecorded oldprogram recordedprogram recordedrating recordedmarkup recordedseek > recordings.sql --verbose


You will be prompted to enter the password for the DB (should be mythtv, assuming you didn’t change it from default)
When it is done you should have a file called recordings.sql which holds the tables dumped above. Depending on how long your system has been in use and how much recoding you have done will determine the file size. Mine was around 60meg after 4 years of use. According to LinHES web interface ~11,000 shows had been recorded.

Using the web interface via the Separate PC, go to Health and Maintenance, click the hyperlink for the LinHES MBE and select the Restore database option. Using the drop down, select the R8 gold backup file (this is the first backup where you recorded the file name). Click go. NOTE: You want to do a full restore.

On the LinHES backend, open a terminal (Alt-x) or via the Separate PC on the left hand menu select System then terminal.
log in as necessary:

Change to root
Code:
su
 

Next you will log in to MYSQL and truncate each of the tables you “dumped” (this wipes any data in that table in the live db):

Enter:
Code:
mysql -u mythtv -p mythconverg


You will be prompted for the DB password:
enter password (should be mythtv, unless you changed it)

At the mysql prompt enter each line below one by one, hit enter to execute the line. (Note \q is used to quit):
Code:
TRUNCATE TABLE record;

Code:
TRUNCATE TABLE recorded;

Code:
TRUNCATE TABLE oldrecorded;

Code:
TRUNCATE TABLE oldprogram;

Code:
TRUNCATE TABLE recordedprogram;

Code:
TRUNCATE TABLE recordedrating;

Code:
TRUNCATE TABLE recordedmarkup;

Code:
TRUNCATE TABLE recordedseek;

Code:
\q

Next you will need to Import Changes

Enter:
Code:
mysql -u mythtv -p mythconverg < recordings.sql --verbose

Create a backup:
On a separate PC, using the LinHES web interface, click the health and maintenance option from the menu on the left. click the LinHES MBE hyperlink, then select the backup system option and click go. 

Once it is done, you should now have a LinHES R8 (mythtv 0.25fixes) database with all of your recording rules, old and current recording info as well as commercial skip and seek tables.

Using the Separate PC go to the web interface, select Health and Maintenance, click the LinHES MBE hyperlink and select Download Backup. Select the latest backup and click go. Store this backup somewhere safe, you will use it to reload all the necessary settings should you do a new install of LinHES 8.1.1.

Upgrade to R8.1.1 as you see fit (new install or upgrade)…
If you reinstall R8.1.1, create a bootable R8.1.1 USB and go through the install steps, but upload and install the last backup saved, reboot and you should be done.
I did not try and do an upgrade via the service menu, but according to the Wiki: Generally to upgrade LinHES to the latest available version just select "Upgrade LinHES" from the Service Menu.

_________________________edit 07/19/14_________________________
After your new install or upgrade, go back into the initial settings screen and verify that the settings are what you want.

Be aware that in the initial setup screens for Mythtv 0.27 (possibly earlier?) includes some demo capabilities, one is in the Video Sources section. When you record a show, the selected Video Source generally provides the first digit of the channel ID, the entire channel ID is used as the first 4 digits of the recording file name (and probably a number of other items). If you add your Video Source without deleting the “DEMO” first all new recording files will begin with “2” instead of 1. The main verified issue I had is that on the old recordings, no channel info is displayed or found in the db (all my new recordings used channel ID’s 2xxx, while the old ones had 1xxx). I believe to fix this I could have deleted the “DEMO” time in Video Sources and then added back my original recording source.

ISSUE: Migrated Recordings take a long time (30 to 60+ seconds) to auto-skip over commercials (new recordings have no issues with auto-skip):
I do not know if the channel ID issue above also caused the seek tables for the migrated recordings to have issues, but I had to rebuild the seek tables and then run commercial detection on each rebuilt file.

The article on rebuilding seektables can be found on the Mythtv wiki:
http://www.mythtv.org/wiki/Repairing_the_Seektable

The pertinent section for my need was:
Automation of mythcommflag repair function

Basically in the initial setup screens, create a user job:
Code:
alt s-> shutdown backend? Y-> General->Job Queue (Job Commands)

Its screen 10/11 in the general section.

Pick a User Job and enter a description:
I used User Job #1 description:
Code:
Fix_Seektable

For the corresponding User Job command enter:
I used User Job #1 command:
Code:
mythcommflag --file %FILE% --rebuild

The name you have selected e.g. "Fix_Seektable" will now appear as a new option below "Job Options" on the "Recording Options" menu.
Code:
Watch Recordings -> select the recording-> m (menu)-> Job Options->Begin Fix_Seektable

Progress on the repair can be seen within the "System Status->Job Queue" menu.

Be aware that this seems to be processor intensive and if you are running on lower powered CPU’s this can make the system very slow to respond. It would be best to queue a bunch of these overnight or at some time when your system is not going to be used. Do them in batches rather than queuing them all at once since on my zotac Atom 330 systems it took about 6 to 10 minutes to rebuild each table.
_________________________edit end 07/19/14_________________________

Copy recordings from external drive to LinHES MBE:

On the Mythdora MBE, use the file browser to locate your recording files, copy everything in that directory over to your LinHES MBE. My files were in a custom directory, so I don’t have the default path info to share. I used an external USB drive to copy to/from, if you have a lot of files, this is going to take a long time. Make sure the USB drive has enough space to hold all the files, or you will need to do this in batches.

On my new LinHES setup, recorded program files were located at /data/storage/disk0/media/tv. So I copied all the files to this directory.

Go to the USB drive holding the files, change ??? with your USB drive info:
NOTE, my USB drive name had spaces in it, so I had to enclose the name in “ “ e.g.:”Seagate Backup Plus”. My files were on the USB drive in a directory called recordings so I had to enter: cd /media/“Seagate Backup Plus”/recordings.

Code:
 cd /media/???


Copy the files from the USB to the LinHES MBE:
Code:
cp *.* /data/storage/disk0/media/tv


Change permissions and owner on recording files once they have been moved to the new HDD

Change to the storage directory:
Code:
cd /data/storage/disk0/media/tv


Change Permissions on all the files:
Code:
chmod 666 *.mpg*

______________________________edit 1/12/15___________________

Change Owner on all the files: (nice catch ntsc525!)
Code:
chown mythtv:mythtv *.mpg*

____________________________end edit 1/12/15__________________

______________________
TWEAKS:
MythCenterWide has disabled wrap around scrolling in the recordings menu:
To enable scrolling you need to edit recordings-ui.xml (for MythCenter-wide v1.7). More info: http://www.mythtv.org/pipermail/mythtv-users/2013-December/357263.html
the file is on my system @: 
Code:
su

Code:
cd /usr/share/mythtv/themes/MythCenter-wide

Code:
nano recordings-ui.xml

using the down arrow scroll through the file
Gist of it is that where you see “captive” you need to change it to “items”
Where you see: <wrapstyle>captive</wrapstyle>
Change to: <wrapstyle> items</wrapstyle>

Write changes and exit Nano:
Cntrl o to write the changes
Cntrl x to exit nano
________________________
HDMI out was not working on my zotac Atom IONs board:

Identify HDMI card and device #:
Code:
su

Code:
aplay -l


This will display a print out similar to:

card 0: NVidia [HDA NVidia], device 0: ALC662 rev1 Analog [ALC662 rev1 Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 1: ALC662 rev1 Digital [ALC662 rev1 Digital]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0

Look for HDMI in the list, in my case the card is 0 and device number for HDMI is 3

To test for sound enter:

Code:
aplay -D plughw:0,3 /usr/share/sounds/alsa/Front_Center.wav


If you get sound then creating the asoundrc file should work for you, if not you will need to search the web for further assistance.

Create the asoundrc file:
Code:
su

Code:
nano ~/.asoundrc


Copy and paste the following into the newly created file NOTE: change card # and device # to actual card/device identified by splay -l above:
__________________________________________

pcm.!default {
type plughw
card 0
device 3
}
ctl.!default {
type plughw
card 0
device 3
}

__________________________________________
Write changes and exit Nano:
Cntrl o to write the changes
Cntrl x to exit nano

Started working as soon as I wrote the file
Change audio settings for both LinHES/mythtv to HDMI card selection

__________________________________________
DVD playback was not working, I missed the checkbox to install libdvdcss-

Repeat on each system you will play DVDs
From the Frontend Main menu
Service Menu ->LinHES Settings->Programs
on screen 2/4 click the checkbox dvdcss
You should now be able to play DVDs via the Optical Disk menu


Again, these are the steps that worked for me, hopefully these will be of use to anyone trying to migrate from Mythdora to LinHES.


Last edited by bgrant3406 on Mon Jan 12, 2015 7:11 am, edited 2 times in total.


Top
 Profile  
 
PostPosted: Tue Jul 08, 2014 9:04 pm 
Offline
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1196
Location: Orlando FL
WOW huge write up!!! Thanks for taking the time to do that.
Just a suggestion for anyone reading this I have found the least stressfull way to do schema upgrades is with virtual machines. They install quickly you can take snapshots to snap back to if necessary and It means you can work from your computer room instead of taking a MythTV box out of service.

_________________
My System


Top
 Profile  
 
PostPosted: Sun Jan 11, 2015 6:05 pm 
Offline
Joined: Sun Jan 11, 2015 5:52 pm
Posts: 2
Much respect to you for taking good notes and writing them up here. If I'd spent an entire summer doing this, I probably would have forgotten most of it by the time I'd finally finished, and wouldn't be able to do it again, let alone write up the entire process.

My last MythTV "migration" went like this:

1: Build a whole new machine. (Mythbuntu with a Happauge tuner, so that took a few days...)

2: Open up two web browsers to MythWeb for each one.

3: Manually copy all of the recording schedules from the 'Dora to the 'Buntu. (If a show wasn't currently airing, tough luck, I had to remember to enter it when it came back on.)

4: Wait until you've watched all of the recorded shows on the old Myth system before retiring it.

That was in 2012. Three years later, and the old MythDora is still running!

Now, the old Myth is starting to make noise, and thanks to you, I got a new platform up and running in a day (most of it spent copying files).

I'm impressed by LinHES so far, mainly because I didn't have to do research and download files to get my Happauge HVR-2250 to work! (Remote still doesn't work, though. The solution on other platforms was to buy a Windows MCE remote with USB receiver.)

Again, thank you for your work and documentation. I'm sure with a little more time, your methods could be applied to other MythTV platforms besides LinHES.


Top
 Profile  
 
PostPosted: Mon Jan 12, 2015 12:10 am 
Offline
Joined: Sun Jan 11, 2015 5:52 pm
Posts: 2
Great write-up, and it worked with LinHES 8.2, though I needed to roll with some minor differences in the setup screens.

One thing I noticed was that root owned all my new copied files. I was still able to watch and delete programs, but for completion, I changed the owner to mythtv to match files recorded by the new system:

If you're not already there, su to root and change to the recordings directory:

Code:
su
cd /data/storage/disk0/media/tv


Then change owners:
Code:
chown mythtv:mythtv *.mpg*


Top
 Profile  
 
PostPosted: Mon Jan 12, 2015 7:13 am 
Offline
Joined: Tue Jul 01, 2014 6:49 am
Posts: 81
Thanks ntsc525, great catch!

I've added the following inline to the writeup:

Then change owners:
Code:
chown mythtv:mythtv *.mpg*



Thanks again for double-checking the work and finding this!

Bruce.


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 


All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu