I've had KnoppMyth up and running well for several months. I recently went to my parents house and decided to dazzle them with the ability to download a movie from my mythbox across the country, burn it to a dvd, and watch it there. Well, it didn't work, and I just got the old look from my Mom like I struck out in little league.

The symptoms are a little strange. If I try to download a movie (oh, this is through the mythweb/video page) that is over 4.3GB, the download window says the file is only 300-500MB, and stops downloading when it reaches that size. If I try to download a move smaller than 4.3GB, the download window says that it is an unknown file size but downloads the movie properly to completion. Note that this number was determined through an unscientific binary search to the approximate file size. It is possible that the overall file size is not the culprit, but the trend held for the dozen or so tests I ran.

I've tried both firefox and IE, with the same result. I assume there's some file size limit at work, but I can't find reference to this particular limit. I saw several threads on an Apache (I think) limit of 2GB, but it seems that is not the same problem.

My system is fairly standard:
Athlon 3000+
WinTV Theater
(1) 250GB, (2) 160GB Hard drives all in an LVM partition

So, any ideas? BTW, great work on the package!

-Dave

Hmm. Apache could be screwing up on the file size. I can't transfer large files from MythWeb using Firefox in Linux, either. I can, however, stream them with mplayer, since it does its own buffering. You may want to consider doing that next time. You could have probably also used scp to transfer the file.

As for the core problem, you may be hitting a timeout as opposed to a hard limit on file size in Apache. To debug it further, try the same transfer on your LAN, where speeds are surely greater. See if the 4.3GB file size still seems to be there. If not, then it starts to look more like a timeout problem. If so, then it looks more like a file size limit somewhere.

_________________
KnoppMyth R5.5
MythiC Dragon v2.0
Join the KnoppMyth Frappr!

That looks suspiciously like somthing in the chain does not handle file sizes larger than a size that can be stored in 32 bits (2^32 = 4,294,967,296 so the biggest unsigned integer that can be stored in 32 bits is 4,294,967,295). So I doubt it is a timeout issue. The 2G limit is usually because sometimes, signed integers are used and the largest 32 bit signed integer is 2,147,483,647.

Nice math!

So I guess scp or using mplayer's inherent ability to stream anything would be the best workarounds until the cause of that limit is determined.

_________________
KnoppMyth R5.5
MythiC Dragon v2.0
Join the KnoppMyth Frappr!

I like the way you think, Mr. Frost! I played around a little more today. I think the problem is definately somewhere on the http chain. I was able to samba it over to a Windows box with no problems, other than it took 20 mins. I'll try ftp next; I think could change the link in MythWeb to try to download via ftp instead. Hopefully this would allow me to get the entire file without resorting to some sort of stream. Ideally, the end user could get access to the iso for burn purposes.

Thanks for the clues!

-Dave

I've been looking into this, too, and it looks like if you compile Apache with LFS (Large File Support) it will handle the larger files. C.f. http://forum.ev1servers.net/showthread.php?t=38866

If you decide to do this, and it works, be sure to send info to cesman and provide him with the necessary info to roll this into KnoppMyth.

_________________
KnoppMyth R5.5
MythiC Dragon v2.0
Join the KnoppMyth Frappr!

Well, not sure that apache is the total problem here. I set up a quick and dirty ftp server, anonymous read-only only.

I can download the big files when connected via a "typical" ftp client (smartFTP), or linux ftp.

However, if I try to download it with firefox, ala "ftp://192.168.2.100/APOCALYPSE_NOW_REDUX.ISO"

It still limits the download size to 377MB.

I'll look into the LFS, see if that helps.

More info after a little more experimentation...

Actual file size: 4681435136 bytes

File size according to Firefox: 377.4 MB

File size according to IE: 368 MB

Here's the cool one...
File size according to wget: 386,467,840 (unauthoritative)
but it downloaded it to 2149351423 bytes.

Actual size - 2^32 = 386,467,840.

Starting to at least make some sense... Sorry if I'm boring you, but I found it interesting in how many different ways the various software is getting this wrong.

=386,467,840 /1024/1000
=386,467,840 /1024/1024 rounded down.

So firefox thinks 1024000 bytes is a MB

and IE thinks 1048576 bytes is a MB

IE is quoting what it should be calling MiB (mebibytes)
Dont know what you would call what firefox is doing.

I finally got some time to play around with this again... and I at least have a workaround.

I was never able to get an http transfer of over 4.3GB (4GiB). I'm not sure if it's apache or the browser or the http protocol itself, but no luck. I tried several of the "download manager" type programs, they also didn't transfer properly.

But, as I noted before, ftp transfers seemed to work, at least outside of default browsers. I now use GetRight (no endorsement, but it works) to connect to the ftp server. This works.

So, ideally the links on the videos page would use the ftp link to get the files. It was a pretty simple edit:

In video.php, I changed


to


I had to do the usual hoop-jumping to get the ftp server working how I would like, but it's almost there. I still need to get some sort of password protection going before I open it up to the world, but that shouldn't be too hard.

So, hope this helps, the workaround was more convoluted than I would have liked, but what can you do? If there's a better way, though, I'd love to hear about it!

-Dave

Are you trying to download to a fat32 partition on your windows machine? Does it work if you download using a linux machine?

It's probably a long shot, but if your download directory is on a fat32 partition, then you won't be able to get anything larger then 4gigs.

No, on Windows I'm using NTFS, and on Linux I'm using xfs. I've ripped entire dual layer dvd's to the windows box (~8 GB ISO), so I know it's not the restriction of the local filesystem. Besides, I've been able to kind of get it work via ftp...

-Dave

I think that this is a limit of TCP/IP (or HTTP, I forget which. I'm pretty sure it's TCP/IP) It has something to do with the packet identification or some such. LFS support in Apache should be for files on the local system...I think...