View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 25 posts ] 
Go to page Previous  1, 2

Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Tue Nov 29, 2016 9:29 am 
Offline
Joined: Sun Apr 30, 2006 7:49 pm
Posts: 37
Location: Ottawa
brfransen wrote:
I wonder if the copy is failing before systemconfig runs. Maybe a bad ISO burn. Did you confirm the md5sum of the burn?

I've checked, the physical disc's MD5 sum matches that of the .iso I downloaded.

brfransen wrote:
In the chroot /data/srv/mysql/ is there any *.err files that give any hint of why mysql isn't starting?

I only imaged sda1, so I didn't copy the mysql mount and the files therein.

I suspect that at least some of the problems arise because of the history of this box, which has been operating for over 10 years now, going back to an early Knoppmyth. I edited files by hand, and it seems that Arch Linux has its own way of doing things. For instance, in the pre-upgrade filesystem, there's a file called /etc/systemconfig. It is entirely unlike the box itself. Different host name, different time zone, no DHCP, no IR settings, and so on. I'll have to learn the Arch way to do things. I doubt, though, that this explains the seriously weird bootup failure.


Top
 Profile  
 
PostPosted: Tue Nov 29, 2016 11:44 am 
Offline
Joined: Fri Jul 21, 2006 11:12 pm
Posts: 1194
Location: SC
neufeld wrote:
For instance, in the pre-upgrade filesystem, there's a file called /etc/systemconfig. It is entirely unlike the box itself. Different host name, different time zone, no DHCP, no IR settings, and so on.
That is a huge red flag. /etc/systemconfig is a text file that should always mirror system settings that are saved in the DB and it is how Upgrade knows what settings to restore to what. It not matching the state of your box would explain the some of the issues you are seeing. Can you compare some of the /etc/systemconfig values to the settings table values that start with host?


Top
 Profile  
 
PostPosted: Tue Nov 29, 2016 1:00 pm 
Offline
Joined: Sun Apr 30, 2006 7:49 pm
Posts: 37
Location: Ottawa
brfransen wrote:
neufeld wrote:
For instance, in the pre-upgrade filesystem, there's a file called /etc/systemconfig. It is entirely unlike the box itself. Different host name, different time zone, no DHCP, no IR settings, and so on.
That is a huge red flag. /etc/systemconfig is a text file that should always mirror system settings that are saved in the DB and it is how Upgrade knows what settings to restore to what. It not matching the state of your box would explain the some of the issues you are seeing. Can you compare some of the /etc/systemconfig values to the settings table values that start with host?

Well, all of the settings for which I could find a parallel were the same between the two. That includes "hostname=diamonds" in systemconfig vs. HostMyhostname="diamonds" in the DB. I've never seen or heard of diamonds before this upgrade, I don't know where that crept in, but the box doesn't seem to know or care about it. The timezone field in the DB is also set to Samoa, but that never kept the system from operating in the correct time zone.

The settings DB does look like it needs a cleaning. There are entries for hostname 'mythtv.i.cneufeld.ca', and others for 'mythtv'. The systemconfig agrees with the former, but not with the latter.


Top
 Profile  
 
PostPosted: Tue Nov 29, 2016 3:49 pm 
Offline
Joined: Fri Jul 21, 2006 11:12 pm
Posts: 1194
Location: SC
Well it is good that systemconfig and the db match. But with them being completely wrong running systemconfig.py on the R8.2 system would probably mess it up. Without good data in the DB Upgrade isn't going to be possible. Your best bet is going to be a clean install and then do a partial restore of the DB as described in the R7 to R8 wiki.


Top
 Profile  
 
PostPosted: Tue Nov 29, 2016 4:23 pm 
Offline
Joined: Sun Apr 30, 2006 7:49 pm
Posts: 37
Location: Ottawa
brfransen wrote:
Well it is good that systemconfig and the db match. But with them being completely wrong running systemconfig.py on the R8.2 system would probably mess it up. Without good data in the DB Upgrade isn't going to be possible. Your best bet is going to be a clean install and then do a partial restore of the DB as described in the R7 to R8 wiki.

That's essentially what I'm working on now. I've installed 8.4.1 on a virtual machine, and I've been tuning it. Then, I'm going to restore the database from a DB backup. The DB backup is generated with
Code:
mysqldump --all-databases > backup

Then the restore is simply
Code:
mysql < backup

After all is complete, I'll send an image of the virtual machine, including the mysql volume, to a directory on the media volumes on the MythTV box. After this, I make a fresh backup of the computer. Then I can boot the DVD, pull up an xterm, and manually transfer the image over. That procedure involves:
    Clean fdisk and format of sda, my 60GB SSD drive
    Adjust the UUIDs of the partitions with tune2fs to match those generated within the VM
    Mount sda1 on /mnt/sda1
    Copy in the new root, /home, and mysql partitions (built on the VM) from the media drive to sda
    Bind mount the live DVD's /proc, /dev, /dev/pts, and /sys under /mnt/sda1
    Chroot into /mnt/sda1
    Run grub-install /dev/sda


Top
 Profile  
 
PostPosted: Tue Nov 29, 2016 4:37 pm 
Offline
Joined: Fri Jul 21, 2006 11:12 pm
Posts: 1194
Location: SC
Doing a full DB restore of a known problematic DB will result in many of the same issues you saw with doing an Upgrade. You will need to do a partial restore using either the LinHES system backup scripts or the MythTV backup/restore scripts.


Top
 Profile  
 
PostPosted: Wed Nov 30, 2016 9:18 am 
Offline
Joined: Sun Apr 30, 2006 7:49 pm
Posts: 37
Location: Ottawa
OK, I have a theory that I believe explains every weirdness I've been seeing. The LinHES backup scripts don't work, and I know why. I never noticed before, since I do a complete backup, filesystem plus media, every night using my own scripts.

The settings database has varchars for setting names and values. The settings aren't keys in the table itself. That means that you can put anything in there, and it will stay there forever unless somebody else removes it. This decade+ old settings database contains a lot of old variables that are no longer used. Further, it seems that some of these settings are written, but never read, just to record interesting things during the installation. But... if your installation was a very long time ago, and you only ever upgraded after that, you might have a system that doesn't have one of these old write-only settings. Now, along comes a system that wants to sniff the database, expecting to find one of these variables, but the variable actually was never set because it dates from a time after the last full install, so it gets back an empty string.

This brings me to a very special variable name. HostSystemType. This is not in my settings table in the DB, probably because it didn't exist at the time the database was created, and nothing ever cared until now. Well, with this variable unset, the /etc/systemconfig also has no value for this key. This empty value breaks the MythVantage universe. Backups don't work correctly (though the script exits with success, the DB is ignored). And... I'll bet during an upgrade, it looks at that variable to decide whether or not it has to install mysqld, and so on. With the field empty, it apparently doesn't install enough packages to build a bootable system.

So, I'm going to have to do some maintenance on the settings table. I've dumped the one on the old system, I'll dump the one in the VM. Then, I'll look at the system settings .xml file in the VM that contains variables and defaults. I'll merge these three things, dropping variable names in the old system that appear unused, and produce a new table that I can use during the upgrade.


Top
 Profile  
 
PostPosted: Wed Nov 30, 2016 10:10 am 
Offline
Joined: Fri Jul 21, 2006 11:12 pm
Posts: 1194
Location: SC
A missing HostSystemType would make sense. It is very odd that it is missing though. It has been around since R6. And even if it was missing for you the R8 upgrade process of doing a partial restore (which is crucial) should have added it. The point of the R8 partial restore was to remove those old unused MythVantage Host settings and to make sure that the new ones were there.

Your way may work but it seems time consuming and could still leave you with a good bit of cruft in the DB. I would either try to set the R8.2 host settings in Service Menu > LinHES Settings > Host and select Master_backend. You many have to enable Change System Type in Access first. Or just get the backup to run and then do a partial restore like in the R7 to R8 wiki. To get the LinHES backup job to run and backup the DB edit /etc/systemconfig with SystemType="Master_backend". Do a backup from the Service Menu or cmd line (lh_system_backup). Verify that the backup has the DB. Then install R8.4.x on the box or VM and do the partial restore using the backup you just made. You will have to setup your Storage Groups but all the MythVantage Host settings should not have been restored over the install. Use the Service Menu to tweak as needed.


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 9:05 pm 
Offline
Joined: Sun Apr 30, 2006 7:49 pm
Posts: 37
Location: Ottawa
OK, I only just got a chance to get back to this. I'll describe where I am so far.
I realized that one of my media drives (/dev/sdb) actually had a partition for an OS, back before I bought a dedicated SSD to hold the OS. I never repartitioned that drive because of the awkwardness of doing that to a drive that has 2TB of recordings you want to keep. So, I copied my VM to that partition and booted into sdb from the BIOS. The new system came up, but wasn't set up for NVidia, since the VM didn't have that. I ran xconfig.sh, but there were conflicts and errors, it couldn't install the nvidia packages because of a conflict with mesa. So, I decided to use the 8.4.1 install disc to "upgrade" this 8.4.1, single partition installation. That mostly worked, though of course it lost the mysql data files that ought to have been in a separate partition during the mkfs. That's something I can take care of.
So, I now have a partly ready 8.4.1 system on /dev/sdb1, and my old system is still on /dev/sda1. I will now re-personalize the /dev/sdb1 image, put its mysql files in place, and see if I can get it up and running. I will then have to merge the old database in. I've decided to merge only program-related tables, specifically:
credits, keyword, oldprogram, oldrecorded, people, powerpriority, record, recorded, recordedcredits, recordedmarkup, recordedprogram, recordedrating, recordedseek, storagegroup.
I've also kept a copy of settings, so I can fix any settings that are missing.
So, I'm on my way again, with a solution in sight.


Top
 Profile  
 
PostPosted: Sun Jan 01, 2017 9:56 pm 
Offline
Joined: Sun Apr 30, 2006 7:49 pm
Posts: 37
Location: Ottawa
OK, I have successfully completed my upgrade. Once the upgrade was complete, things went fairly smoothly. One problem that turned up was in a custom priority I wrote, to avoid using a certain encoder between 2AM and 2:15AM because my cable supplier does a firmware install and reboot of the cable boxes almost every single night at that time! I traced the misbehaviour to the mysql database not being loaded with time zone information, it didn't recognize 'UTC' as a valid time zone identifier.

The zone information can be loaded by running
Code:
$ mysql_tzinfo_to_sql /usr/share/zoneinfo > zonedata.sql

and then going into zonedata.sql and commenting out the 'Factory' zones, as they are not set up and produce output that exceeds the column length of the zone data.
You can then issue:
Code:
$ mysql -u root mysql < zonedata.sql

I noticed that this procedure also blanked my system zone info, so I reset that with
Code:
$ mysql -u root mysql
mysql> set global time_zone = 'EST5EDT';

Now, my custom priority rule works again (the reason it didn't work was that I had to convert the start/endtimes from UTC to the local time zone, since the 2AM issue happens at local time, and without the zone tables, the time zone conversion function was returning NULL instead of the converted times).

Another strange change was that fuser no longer reports /dev/videoX (HDPVR) as being in use during an HD-PVR recording. My scripts that power cycle the PVR to fix flakiness were turning off the machine in the middle of recordings. I just changed my scripts to use the backend API to query the card status.


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 25 posts ] 
Go to page Previous  1, 2



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 18 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu