View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 41 posts ] 
Go to page Previous  1, 2, 3  Next

Print view Previous topic   Next topic  
Author Message
Search for:
 Post subject:
PostPosted: Tue Sep 30, 2008 12:43 am 
Offline
Joined: Thu Dec 01, 2005 4:14 pm
Posts: 78
Location: Ann Arbor, MI
Just throwing this one out there as a brainstorm:

Would it be possible to create a run level - like, say, 7 - have it run the defragmentation in an init script, and then when it is done to return to run level 5? Something like this:

  1. cron (intelligently, somehow) schedules a job to drop to init level X.
  2. The init scripts in level X mimic level 1.
  3. The last init script in level X is a custom script that runs the xfs defrag routine for a predetermined amount of time, and then switches back to run level 5 when done.


Does that sound...sane? :-)


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 14, 2008 8:59 am 
Offline
Joined: Tue Apr 11, 2006 7:44 am
Posts: 287
Location: Los Angeles, CA
graysky wrote:
Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV.

Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size?

_________________
Tim

LinHES 8.4
HDHR3
BioStar A770, AMD X2 4050e, 2GB RAM
GigaByte GeForce 8400, Chaintech AV710
USB-UIRT


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 14, 2008 12:57 pm 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location: /dev/null
opel70 wrote:
graysky wrote:
Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV.

Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size?


Dunno man, capping 4-6 gig SDTV files here. 512 does a great job for me. I just used it based on the suggestion on the wiki I cited.

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 15, 2008 10:08 am 
Offline
Joined: Sun Sep 05, 2004 7:06 pm
Posts: 690
Thanks Graysky for the tip. After I scaled out my setup with multiple drives and storage groups I didn't really realize how fragramented the drives really were. On my old box it was like 52% and 63% and the new box was 97%. I ran the tool for 1.5 hours and it went down to something like 10% or less on the old box and 1% on the new box.

NICE WORK!!! :wink:


Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 15, 2008 3:13 pm 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location: /dev/null
Cool man, glad to hear people are getting mileage out of this thread.

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 25, 2008 7:57 am 
Offline
Joined: Mon Oct 09, 2006 12:18 pm
Posts: 29
Wondering if someone can help me with a little problem I'm having with the output of xfs_db after running xfs_fsr... for some reason xfs_db gives the same numbers before and after... if I reboot then they update.

Code:
# xfs_db -c frag -r /dev/sdb1
# actual 170, ideal 150, fragmentation factor 11.76%
# xfs_fsr -v -t 60
# xfs_db -c frag -r /dev/sdb1
# actual 170, ideal 150, fragmentation factor 11.76%
# reboot
# xfs_db -c frag -r /dev/sdb1
# actual 156, ideal 149, fragmentation factor 4.49%


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 25, 2008 8:15 am 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location: /dev/null
I've seen this before and can't explain it. You can reset the stats by umounting then remounting the filesystem, i.e.

Code:
# umount /myth
# mount /myth

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 25, 2008 11:22 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
opel70 wrote:
graysky wrote:
Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV.

Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size?

Each recording also creates a ".png" file in /myth/tv -- this means that each of those files will use 1GB of disk. Not a show-stopper -- just something you should be aware of.

I have my machine defrag for 10 minutes once a day, just before it shuts down. My fragmentation has remained very low.

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 31, 2008 6:39 am 
Offline
Joined: Tue Mar 29, 2005 6:15 pm
Posts: 79
Location: Plymouth, MI
I'm running R5F27, and recently replaced my media drives with larger ones. During the replacement, I also switched from ext3 to xfs.

I thought I'd see what my fragmentation levels were, and defrag if necessary. Both media drives are showing EXTREMELY high frag levels (98.80% and 92.39%), so I thought I'd defrag them while the box was going to be idle for several hours. Unfortunately, I do not have xfs_fsr on my system... The only xfs_* files available are:

Code:
xfs_admin     xfs_copy      xfs_growfs    xfs_logprint  xfs_quota
xfs_bmap      xfs_db        xfs_info      xfs_mkfile    xfs_repair
xfs_check     xfs_freeze    xfs_io        xfs_ncheck    xfs_rtcp


Can anyone offer any insight as to why I would have xfs_db and not xfs_fsr?

And more importantly, is there a way that I can get xfs_fsr so I can defrag my drives without making large changes to the core OS?

Thank you in advance for any advice you can offer.

_________________
Running Knoppmyth R5F27 since 09/12/07

Hardware Specs


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 31, 2008 8:04 am 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
adoute wrote:
And more importantly, is there a way that I can get xfs_fsr so I can defrag my drives without making large changes to the core OS?

Thank you in advance for any advice you can offer.


Look here

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 31, 2008 8:53 am 
Offline
Joined: Tue Mar 29, 2005 6:15 pm
Posts: 79
Location: Plymouth, MI
Thank you for such a quick response. The required software installed perfectly. Now I just need to find a nice block of time to start defragging...

FWIW, I did try searching - I can't believe I missed the post that you linked to. Thanks again.

_________________
Running Knoppmyth R5F27 since 09/12/07

Hardware Specs


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 31, 2008 10:41 am 
Offline
Joined: Fri Feb 08, 2008 9:19 pm
Posts: 70
My XFS /myth partition was 99% fragmented. I never noticed any
performance issues, but after all the dire warnings in this
thread about fragmentation, I degragmented it anyway (which
took 15-20 hours). I also added the option "allocsize=256m" to
fstab and re-mounted the partition.

After doing the defrag and setting the allocsize, playback now
pauses and stutters when a recording is being deleted. It's
not as bad as my old machine that used ext3.

AFAICT, defragging and using a larger allocsize has caused this
stuttering as files are deleted. I don't suppose there's a
re-fragement utility I can use to get back to the previous
state? I guess if I decrease the allocsize it will eventually
return to "normal", right?

--
Grant

_________________
Grant


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 31, 2008 1:20 pm 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location: /dev/null
@grant - I never heard of that before.... I have been using my XFS system like this (with the 512meg thing in the fstab) and it's been just fine, in fact, it's noticibly faster when deleting 4-5 gig files (usually takes 1-2 sec). I wonder if you have an unrelated problem? Have you tried checking the consistency of the partition for troubles? See this thread for a step-by-step walk through.

Failing that, and if you suspect it's the allocsize statement, do this: take out the allocsize statement in the fstab, umount, remount, THEN try running the defrag util.

Let us know in either case :)

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Nov 01, 2008 2:56 pm 
Offline
Joined: Fri Feb 08, 2008 9:19 pm
Posts: 70
graysky wrote:
@grant - I never heard of that before.... I have been using my
XFS system like this (with the 512meg thing in the fstab) and
it's been just fine, in fact, it's noticibly faster when
deleting 4-5 gig files (usually takes 1-2 sec). I wonder if
you have an unrelated problem?

It's possible it was just a coincidence -- I haven't really
done anything to troubleshoot it since it only stutters for a
second or two and then returns to normal. My ext3 based system
(a different box) used to stutter for 30+ seconds while a file
was being deleted (occasionally the player would even exit
spontaneously).
Quote:
Have you tried checking the consistency of the partition for
troubles?

I ran the XFS file system checker both before and after I
defragged the partition, but I haven't run in since then. I'll
run it again to make sure nothing's gone bad.
Quote:
Failing that, and if you suspect it's the allocsize statement,
do this: take out the allocsize statement in the fstab, umount,
remount, THEN try running the defrag util.

IIRC, I didn't add the allocsize statement until after I had
defragged the partition. I assumed the allocsize statement
would only affect new files that were being created.

_________________
Grant


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 12, 2008 12:15 am 
Offline
Joined: Thu Sep 27, 2007 5:44 pm
Posts: 580
I've tried add a cron job for this and I have been unable to get xfs_fsr to output to a log file. xfs_db does no problem. The weird thing is that if I run the command as a cronjob it runs and the output ends up in /var/log/messages. If I copy and paste the same command into a shell, it runs and puts the output into the proper log file. Any thoughts on what I might be doing wrong?

Code:
02 00 * * 3 (/usr/local/bin/idle.sh && /usr/sbin/xfs_fsr -v -t 600) > /var/log/defrag.log 2>&1
as a cron job (root), results in:
Quote:
Checking what MythTV is doing now or plans within 20 minutes...

mythshutdown returned 0
schemalock 0
running jobs 0
inuse programs 0
potential recordings 0
planned recordings 0

System is idle
in /var/log/defrag.log
and
Quote:
Nov 12 00:02:02 mythtv fsr[9145]: Found 1 mounted, writable, XFS filesystems
Nov 12 00:02:02 mythtv fsr[9145]: xfs_fsr -m /etc/mtab -t 600 -f /var/tmp/.fsrlast_xfs ...
Nov 12 00:02:02 mythtv fsr[9145]: START: pass=0 ino=0 /dev/sda3 /myth
Nov 12 00:02:02 mythtv fsr[9146]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9147]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9148]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9149]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9150]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9151]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9152]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9153]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970593
Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970594
Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970596
Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:02 mythtv fsr[9154]: /myth start inode=0
Nov 12 00:02:02 mythtv fsr[9154]: ino=1124970593
Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:03 mythtv fsr[9154]: ino=1124970594
Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:03 mythtv fsr[9154]: ino=1124970596
Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:03 mythtv fsr[9155]: /myth start inode=0
Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970593
Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970593
Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970594
Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970594
Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970596
Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970596
Nov 12 00:02:03 mythtv fsr[9145]: Completed all 10 passes
in /var/log/messages

If I run
Code:
(/usr/local/bin/idle.sh && /usr/sbin/xfs_fsr -v -t 600) > /var/log/defrag.log 2>&1
in a shell as root, I get:
Quote:
Checking what MythTV is doing now or plans within 20 minutes...

mythshutdown returned 0
schemalock 0
running jobs 0
inuse programs 0
potential recordings 0
planned recordings 0

System is idle
Found 1 mounted, writable, XFS filesystems
xfs_fsr -m /etc/mtab -t 600 -f /var/tmp/.fsrlast_xfs ...
START: pass=0 ino=0 /dev/sda3 /myth
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
/myth start inode=0
ino=1124970593
No improvement will be made (skipping): ino=1124970593
ino=1124970594
No improvement will be made (skipping): ino=1124970594
ino=1124970596
No improvement will be made (skipping): ino=1124970596
Completed all 10 passes
with nothing new in /var/log/messages

What gives? Shouldn't I get the same log results whether a command is run in a shell or a cron job? If it makes any difference I am running R5F27.


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 41 posts ] 
Go to page Previous  1, 2, 3  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 18 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu