View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 7 posts ] 
Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Sat Oct 15, 2011 10:38 am 
Offline
Joined: Wed May 09, 2007 8:47 pm
Posts: 367
Location: Minnesota- Brrrrr!
A source and a target NTFS drive and they are mounted to a LinHES box. The drives are SAMBA accessable. Archlinux automagically mounted the USB HDDs to : /media/DEVICE_VOLUMENAME.

I tried to recursively copy via UNIX CLI , however, only 80% of the files came through. I imagine this is the fastest way to copy because it is USB <=> SAMBA <=> USB.

Identifying the missing 20% was assigned to WinSCP's synchronication tool. Samba allows the source drive to be a 'local' drive.

This process is slow because the data is circulating from LinHES through WinSCP along with encryption overhead. If encryption was disabled or FTP was used, then that might speed it up: I dislike the idea of adding an FTP server just for this exercise.

This seems to be an effective, but kludgey way of doing things. 'There's got to be something better than this!' Suggestions are appreciated. It would preferable to use the existing resources on LinHES and avoid installing additional services \ utilities on LinHES.

_________________
R7.3: 0.22.20091023-1, Hauppauge PVR-500 (Philips FQ1236A MK4), Gigabyte Gigabyte EG45M-UD2H, E5200 2.4Ghz, 2GB RAM, NVIDIA GEFORCE 256MB


Last edited by gatorback on Thu Nov 03, 2011 12:34 pm, edited 2 times in total.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 15, 2011 2:25 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
My first choice tool for any kind of disk to disk directory cloning like this is usually rsync. I've used it to to copy stuff to locally mounted MS-DOS formatted USB drives before, although never to a remote share that I can remember, but it should work.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 15, 2011 4:14 pm 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location: /dev/null
+1 for rsync. It will do what you want so long as you have the shares mounted.

Code:
rsync -avxu --progress --delete-after --exclude "RECYCLER/" --exclude "System Volume Information/" /media/data/ /media/aa_backup/drive_data/

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 15, 2011 7:12 pm 
Offline
Joined: Wed May 09, 2007 8:47 pm
Posts: 367
Location: Minnesota- Brrrrr!
Thanks gentlemen. As usual: excellent contributions.

After reading about the -a (archive) option, I decided to recopy 400 GB tonight and will check the results. Speed is really nice: 14.16MB/s (Megabits / Sec?) => 1.06 GB / Min.

_________________
R7.3: 0.22.20091023-1, Hauppauge PVR-500 (Philips FQ1236A MK4), Gigabyte Gigabyte EG45M-UD2H, E5200 2.4Ghz, 2GB RAM, NVIDIA GEFORCE 256MB


Top
 Profile  
 
 Post subject:
PostPosted: Sun Oct 16, 2011 7:49 am 
Offline
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location: /dev/null
gatorback wrote:
Speed is really nice: 14.16MB/s (Megabits / Sec?) => 1.06 GB / Min.


MB/s = megabytes per second
Mb/s = megabits per second

So, you're doing about 0.83 GB/min. If it were in Mb you'd be getting 0.10 GB/min. Anyway, see, http://en.wikipedia.org/wiki/Data_rate_units

I'm guessing you're on a 100baseT(X) backbone for your LAN or is that wireless? If your LAN is wired and has GLAN potential in your NICs (any modern PC does), a $45 switch can upgrade your to GigaLAN speeds. Assuming the HDDs are >=SATA2 and assuming the PCs are relative modern, you increase the bottleneck from the LAN to the HDD's data read/write. I usually get 80-90 MB/s on drive-to-drive xfers (no LAN) and the same on PC-to-PC xfers using the LAN.

_________________
Retired KM user (R4 - R6.04); friend to LH users.


Top
 Profile  
 
 Post subject:
PostPosted: Sun Oct 16, 2011 9:40 am 
Offline
Joined: Wed May 09, 2007 8:47 pm
Posts: 367
Location: Minnesota- Brrrrr!
@graysky, thanks for clearing up the data rate convention.

The data rate above is within the context of two USB devices connected to the same MythTV box (no networking, which is what I wanted).

For the casual reader, I have added logging of transferred files and timed the transfer:

Code:
date > TicToc_label.txt; rsync -avxu --progress --delete-after --exclude "RECYCLER/" --exclude "System Volume Information/" /media/SOURCE /media/TARGET   | tee ~/rsync_log_label.txt; date >> TicToc_label.txt

_________________
R7.3: 0.22.20091023-1, Hauppauge PVR-500 (Philips FQ1236A MK4), Gigabyte Gigabyte EG45M-UD2H, E5200 2.4Ghz, 2GB RAM, NVIDIA GEFORCE 256MB


Last edited by gatorback on Thu Nov 03, 2011 12:42 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 17, 2011 8:11 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
graysky wrote:
a $45 switch can upgrade your to GigaLAN speeds.

Heck $40 will get you a commercial grade managed switch. ;-)

http://www.roc-noc.com/mikrotik/routerb ... 250gs.html

Or an unmanaged Netgear switch.

http://www.newegg.com/Product/Product.a ... 6833122128


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 


All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 22 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu