Author |
Message |
mattbatt
|
Posted: Wed Oct 24, 2007 12:30 am |
|
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1343
Location:
Orlando FL
|
All the legit search engines abide by a w3 standard detailed here http://www.robotstxt.org/wc/robots.html
Put simply you ask them not to index your site or a portion of your site and they wont.
I don't want my mythtv indexed by Google, Yahoo, Alta Vista, Lycos, Web Crawler, and the like, so I put a "robots.txt" file in the /var/www/ folder. The text file simply states:
Code: User-agent: * Disallow: /
That means
"*" all robots
"/" don't look anywhere
I had to find the folder that the Knoppmyth default page was in so I thought I would document it here. Ya know kinda give back to the community as it were.
Thanks to all those who helped me.
MattBatt
|
|
Top |
|
|
Dale
|
Posted: Wed Oct 24, 2007 1:36 am |
|
|
Site Admin |
Joined: Fri Oct 31, 2003 11:40 pm
Posts: 357
Location:
Irvine, Ca
|
Your site should NOT be accessible from the internet. It is NOT safe, I don't care how much you may think that it is.
|
|
Top |
|
|
graysky
|
Posted: Wed Oct 24, 2007 1:51 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
Agreed... hackers would much rather get into a LINUX box than a windows box, so if you're mythtv box isn't behind a hardware firewall/router, you should have your head examined as it is likely an attractive target to people who know what they're doing. Further, do NOT forward any of the ports to your boxes -- particularly port 80!
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
|
rando
|
Posted: Wed Oct 24, 2007 9:08 am |
|
Joined: Sat Feb 11, 2006 5:26 pm
Posts: 282
Location:
Winnipeg - Canada
|
If you are behind a firewall/router, but have port 80 (or some other port) forwarding your web traffic, would putting a robots file not be a good idea? As at that point your box isn't live on the internet, but yet is still accessible to a crawler, no?
_________________ Currently Running:
Too lazy to update this with my current hardware, I'll redo it during my next install =)
|
|
Top |
|
|
Gibble
|
Posted: Wed Oct 24, 2007 9:21 am |
|
Joined: Thu Sep 30, 2004 12:26 pm
Posts: 468
Location:
Canada
|
Anybody can ignore robots.txt...it's like a "no trespassing" sign, it only keeps the honest folk out.
_________________ "The amount of time needed to solve a problem is inversely proportionate to the complexity of the solution" -- Me
KM: R5.5
CPU: Athlon 3800+
Vid: nvidia 7300GT
Snd: Chaintech av-710
Tuner: PVR150
HD: 1000gb sata + 750gb sata + 500gb usb
|
|
Top |
|
|
Girkers
|
Posted: Wed Oct 24, 2007 3:43 pm |
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
rando, by forwarding port 80 to your KnoppMyth box you are making your box "live" on the internet, just like any other web server.
_________________ Girkers
|
|
Top |
|
|
rando
|
Posted: Wed Oct 24, 2007 4:26 pm |
|
Joined: Sat Feb 11, 2006 5:26 pm
Posts: 282
Location:
Winnipeg - Canada
|
When I said "live" I meant a box that is directly connected to the internet. Having a single port forwarded will be more secure than having a box connected directly and having all services/ports available to be accessed via the internet.
Perhaps the security risks are still pretty big with port 80 forwarded... but certainly they are less big than if all ports were accessible.
_________________ Currently Running:
Too lazy to update this with my current hardware, I'll redo it during my next install =)
|
|
Top |
|
|
mattbatt
|
Posted: Wed Oct 24, 2007 7:48 pm |
|
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1343
Location:
Orlando FL
|
My mythweb page is being port forwarded from a non standard port. For example port 123456 is being port forwarded to port 80 on my mythbox. No that does not make it absolutely hacker proof but it helps. I also followed http://knoppmythwiki.org/index.php?page ... thWebHowTo
and changed my passwords and such so that I'm still less at risk.
One of the main benefits of Mythtv is the ability to change recording options from work. (It also is really cool to show Mac Fanboys why linux rocks.) So closing it off from the web is not a happy option.
Please tell me if I'm still insecure I don't want my baby hacked.
|
|
Top |
|
|
tjc
|
Posted: Wed Oct 24, 2007 8:29 pm |
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
Of course it's still insecure, haven't people ever heard of port scanners? Add a bit of traffic sniffing to target the ports being probed and it can have the bad guys knocking on the right port with the right protocol within seconds. Oh, and the botnets running on at least half your neighbors MS-Windows boxes give them scores of machines in your network neighborhood to do the sniffing and probing...
If you expose any ports at all they should either be encrypted SSL type stuff, preferably with originating address limits, or tied to really bombproof servers isolated in a DMZ.
|
|
Top |
|
|
fra
|
Posted: Wed Oct 24, 2007 10:06 pm |
|
Joined: Fri Sep 07, 2007 11:57 pm
Posts: 166
|
tjc wrote: Of course it's still insecure, haven't people ever heard of port scanners? Add a bit of traffic sniffing to target the ports being probed and it can have the bad guys knocking on the right port with the right protocol within seconds. Oh, and the botnets running on at least half your neighbors MS-Windows boxes give them scores of machines in your network neighborhood to do the sniffing and probing... If you expose any ports at all they should either be encrypted SSL type stuff, preferably with originating address limits, or tied to really bombproof servers isolated in a DMZ.
I second that. I only have a port open for ssh. I then use ssh tunneling to access mythweb and manage the recordings. Moreover, I setup ssh to use a key not a password, just in case I also setup https for mythweb, and I only allow access from my work ip address.
Here's what I've done:
Edited the file /etc/ssh/sshd_config (as root) and made the following changes:
Code: Port [choose something other than 22] PermitRootLogin no ChallengeResponseAuthentication yes UsePAM no ClientAliveInterval 60 ClientAliveCountMax 30
The last two lines are not in the file. I recommend adding them. To figure out what these mean, type "man sshd_config". If then on your router you forward the port you chose above onto your box, you can access it remotely: Code: ssh [your user name on the box]@[your ip address] -p [the port number]
To figure out the ip address of any machine you can use ifconfig or simply open a web browser onto www.whatismyipaddress.comOnce ssh is correctly setup, you can run remotely Code: ssh -L 8080:localhost:80 [your user name]@[your ip address] -p [the port number]
and then on your browser go to http://localhost.8080/mythweb
your web session will be tunneled through your ssh session and securely encrypted.
you can change the number 8080 to any number you want. just have to do it in the ssh command and in the browser. more importantly, you can change port 80 to a different number. but then you have to specify that port to apache on your box. to do this simply edit /etc/apache2/ports.conf and change the port number therein.
using this method, you haven't opened a new port! and the ssh port (which is the ONLY port you have open) is fairly secured, since it requires a password to get through.
of course, requiring key authentication rather than password authentication, restricting access to only a few chosen ip addresses, and setting up SSL (to use https) for mythweb, would make the whole business even more secure. all this exists in the wiki. i'm also in the process of writing my own wiki that integrates all that i've done to my box! it's coming along quite well and i'm hoping to post it soon. it'll include all the above info as well (about key authentication and the such).
|
|
Top |
|
|
Girkers
|
Posted: Thu Oct 25, 2007 6:31 am |
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
Another option is VPN which is the method I use. I can't really comment on the security of this solution, it works and has the appearance of security.
_________________ Girkers
|
|
Top |
|
|
rando
|
Posted: Thu Oct 25, 2007 8:15 am |
|
Joined: Sat Feb 11, 2006 5:26 pm
Posts: 282
Location:
Winnipeg - Canada
|
Fra, I really like the sounds of your setup. I might do something similar. I really do like having access from "out and about" It's handy to check recordings, check my graphs, and just generally take a peek at the system to make sure everything is behaving.
Thanks for posting details of this (even though it sounds like they were/are already posted elsewhere).
_________________ Currently Running:
Too lazy to update this with my current hardware, I'll redo it during my next install =)
|
|
Top |
|
|
rando
|
Posted: Thu Oct 25, 2007 8:32 am |
|
Joined: Sat Feb 11, 2006 5:26 pm
Posts: 282
Location:
Winnipeg - Canada
|
This got me looking around.... and I think I've come across a document that appears to basically be doing what you've described.... I've only glanced over it and haven't given it a full proper read/try yet.
http://souptonuts.sourceforge.net/sshtips.htm
If I get some time today I'll be giving this a try and will report back on what kinda results I get.
_________________ Currently Running:
Too lazy to update this with my current hardware, I'll redo it during my next install =)
|
|
Top |
|
|
rando
|
Posted: Thu Oct 25, 2007 9:04 am |
|
Joined: Sat Feb 11, 2006 5:26 pm
Posts: 282
Location:
Winnipeg - Canada
|
Well, I've closed port 80 on my router, and followed the very easy steps in that URL to setup putty on my WinXP machine to access my MythTV system and it works like a charm. With a simple change to my putty config I'm good to go. As long as I'm SSHd into my box I can browse to http://localhost and have that traffic redirected (securely!) to my mythweb page.
_________________ Currently Running:
Too lazy to update this with my current hardware, I'll redo it during my next install =)
|
|
Top |
|
|
mac
|
Posted: Thu Oct 25, 2007 10:18 am |
|
Joined: Tue Feb 21, 2006 7:24 am
Posts: 396
Location:
Dushanbe, Tajikistan
|
iptables is not rocket science. Not sure why people put so more faith in a closed source firewall than one they have complete control over. iptables is as bulletproof or more than any appliance and in many cases, it is what an appliance is using.
If you have a fairly simple set of rules on the myth box there is no reason that it can't be connected directly to the internet.. For example:
# Allow loopback connections
iptables -A INPUT -p TCP -s 127.0.0.1 -j ACCEPT
# Allow everything from my non-routable IP address
iptables -A INPUT -p TCP -s 10.111.111.111 -j ACCEPT
# Allow the box to get out and do almost anything TCP related is wishes, eth0 faces the outside.
iptables -A INPUT -i eth0 -p TCP -m state --state ESTABLISHED,RELATED -j ACCEPT
# Ignore all TCP not related to connections initiated by me.
iptables -A INPUT -p TCP -j DROP
# Take care of nameservice.
iptables -A INPUT -p UDP -s <my nameserver> --source-port 53 -j ACCEPT
iptables -A INPUT -p UDP -j REJECT
Doing something like forwarding port 80 is not a big deal in that you can restrict the rule to just one incomming IP address. You can even turn the rule on an off as you need it.
etc.. you should run nmap against your own box to make sure you did things correctly.
---- just quick SYN scan from an outside ip address to see any holes :
foo>nmap -sS x.x.x.x
Starting nmap 3.45 ( http://www.insecure.org/nmap/ ) at 2007-10-25 22:38 KGST
All 1657 scanned ports on x.x.x.x are: filtered
|
|
Top |
|
|