View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 14 posts ] 
Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Tue Jan 12, 2016 1:25 pm 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
Running LinHES 8.3

I'm not sure exactly how, but I've broken SSH. To complicate matters, this is on a machine that I don't have physical access to. When I try to SSH into the box, I get the "Last Login" message and then it takes MINUTES before I'm able to actually do anything in the terminal. I've been trying to troubleshoot, but I'm coming up with nothing. I tried running ssh in verbose mode (ssh -lroot -vvv 192.168.x.x) and I've provided that output below. However, even in that mode, it's silent after "Last Login" and then the only thing that comes up after that is the standard splash page starting with Tuner Status. Of course, there's still a long delay before that pops up.

To help troubleshoot, here's what I did. The network was a 192.168.1.x network, and I changed it to a 192.168.28.x network. So, I installed LinHES on a machine with an IP address of 192.168.1.120 and now the IP address of the machine is 192.168.28.120. Everything else appears to be working just fine, just ssh is slow. I'm wondering if the old IP is still hiding somewhere and that's what's slowing everything down.

Code:
Kevins-MacBook-Pro:~ Kevin$ ssh -lroot -vvv 192.168.28.120
OpenSSH_6.9p1, LibreSSL 2.1.8
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: /etc/ssh/ssh_config line 102: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.28.120 [192.168.28.120] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/Kevin/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.9
debug1: match: OpenSSH_6.9 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 192.168.28.120:22 as 'root'
debug3: hostkeys_foreach: reading file "/Users/Kevin/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /Users/Kevin/.ssh/known_hosts:19
debug3: load_hostkeys: loaded 1 keys from 192.168.28.120
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-rsa
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-dss
debug2: kex_parse_kexinit: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: kex_parse_kexinit: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: kex_parse_kexinit: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: kex_parse_kexinit: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug1: kex: server->client chacha20-poly1305@openssh.com <implicit> none
debug1: kex: client->server chacha20-poly1305@openssh.com <implicit> none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:pAeidvPr6SsZRf6uJB2Y76wzoGg8v2fT4SLVsVa4OfE
debug3: hostkeys_foreach: reading file "/Users/Kevin/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /Users/Kevin/.ssh/known_hosts:19
debug3: load_hostkeys: loaded 1 keys from 192.168.28.120
debug1: Host '192.168.28.120' is known and matches the RSA host key.
debug1: Found key in /Users/Kevin/.ssh/known_hosts:19
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/Kevin/.ssh/id_rsa (0x0),
debug2: key: /Users/Kevin/.ssh/id_dsa (0x0),
debug2: key: /Users/Kevin/.ssh/id_ecdsa (0x0),
debug2: key: /Users/Kevin/.ssh/id_ed25519 (0x0),
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/Kevin/.ssh/id_rsa
debug3: no such identity: /Users/Kevin/.ssh/id_rsa: No such file or directory
debug1: Trying private key: /Users/Kevin/.ssh/id_dsa
debug3: no such identity: /Users/Kevin/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /Users/Kevin/.ssh/id_ecdsa
debug3: no such identity: /Users/Kevin/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /Users/Kevin/.ssh/id_ed25519
debug3: no such identity: /Users/Kevin/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
root@192.168.28.120's password:
debug2: we sent a password packet, wait for reply
debug1: Authentication succeeded (password).
Authenticated to 192.168.28.120 ([192.168.28.120]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug2: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug1: Sending environment.
debug3: Ignored env TERM_PROGRAM
debug3: Ignored env SHELL
debug3: Ignored env TERM
debug3: Ignored env TMPDIR
debug3: Ignored env Apple_PubSub_Socket_Render
debug3: Ignored env TERM_PROGRAM_VERSION
debug3: Ignored env TERM_SESSION_ID
debug3: Ignored env USER
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env __CF_USER_TEXT_ENCODING
debug3: Ignored env PATH
debug3: Ignored env PWD
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: Ignored env XPC_FLAGS
debug3: Ignored env XPC_SERVICE_NAME
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env LOGNAME
debug3: Ignored env DISPLAY
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Last login: Tue Jan 12 10:11:58 2016 from 192.168.28.91
Tuner Status:
-------------
    Tuner 1 (HDHOMERUN) on mythtv : Idle
    Tuner 2 (HDHOMERUN) on mythtv : Idle
    Tuner 3 (HDHOMERUN) on mythtv : Idle
    Tuner 4 (HDHOMERUN) on mythtv : Idle

_________________
LinHES 8.3, 1 BE, 3 FE


Last edited by kmkittre on Wed Jan 27, 2016 1:27 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Tue Jan 12, 2016 1:36 pm 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
Oh, and I wanted to test to see if the mythtv user also was experiencing this delay, but I couldn't figure out how to enable mythtv to use ssh. It's not being denied in /etc/ssh/sshd_conf, but I am unable to ssh using the mythtv account.

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 9:33 am 
Offline
Site Admin
Joined: Fri Jun 11, 2004 7:58 am
Posts: 507
This is either a generic DNS timeout or one caused by the login status screen.

To determine which is it login to the box and then run

/etc/profile.d/z_myth_status.sh

If that shows the same long delay then you know it's the problem. The next step would be to figure out what part of the status is causing the delay.


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 10:22 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
Thank You Jams! It is definitely that script that's causing the slow down. Any tips for troubleshooting? Seriously considering just renaming it right now so that I can get back to a reasonable connection time...

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 10:31 am 
Offline
Site Admin
Joined: Fri Jun 11, 2004 7:58 am
Posts: 507
it's safe to disable that script, it doesn't do anything but show myth status on new login.

To disable it the file will need to be moved out of /etc/profile.d or modified so it doesn't run anything. For finding out why it's doing it, that will take more effort. Might be able to look at it later today, but can't promise anything.


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 10:41 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
Well this is just weird. I looked at what the script was pointed at, and it's running /usr/LH/bin/myth_status.py. If I run the shell script, the delay is there. If I run the python script directly (python2 myth_status.py), it runs just fine. This is the entirety of that shell script:

Code:
#!/bin/bash
if [ x$MYTH_RUN_STATUS = x ]
then
    MYTHCONFDIR=/usr/share/mythtv /usr/LH/bin/myth_status.py
fi
MYTH_RUN_STATUS="1"

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 10:46 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
If I remove "MYTHCONFDIR=/usr/share/mythtv ", the script runs snappy.

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 10:57 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
This also has no issues:

Code:
#!/bin/bash
if [ x$MYTH_RUN_STATUS = x ]
then
    MYTHCONFDIR=/usr/share/mythtv
    /usr/LH/bin/myth_status.py
fi
MYTH_RUN_STATUS="1"


That's just super f-ing odd. Why would this all of a sudden become an issue?

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 11:06 am 
Offline
Site Admin
Joined: Fri Jun 11, 2004 7:58 am
Posts: 507
See if there is a difference between
/usr/share/mythtv/config.xml
and
/home/mythtv/.mythtv/config.xml
/root/.mythtv/config.xml

They should be symlinks to /usr/share/mythtv/config.xml


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 11:08 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
Okay, more details. I'm still having issues when logging in, even when I modify this script. I discovered that if I hit ctrl-c three times, I can kill whatever's running and then I get an actual error this time:

Code:
Couldn't connect to MythTV service for status
Exception AttributeError: "'MythBE' object has no attribute 'hostname'" in  ignored

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 11:10 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
jams wrote:
See if there is a difference between
/usr/share/mythtv/config.xml
and
/home/mythtv/.mythtv/config.xml
/root/.mythtv/config.xml

They should be symlinks to /usr/share/mythtv/config.xml


There's the error. That file (config.xml) is pointing to the old IP address.

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 11:13 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
Fixing the IP address in /usr/share/mythtv/config.xml fixed the problem. Thank you so much for your help troubleshooting, jams.

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 11:19 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
So, I say it "fixed the problem," but it's still slower than I'd like. Maybe it's just because the computer is slower. I did notice, that now, when I run the python script directly it's also a bit slower than it was before. Fixing the IP address appears to have slowed down the python script. Does that make sense?

(I just timed it, and it takes a little over 10s to run)

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 
 Post subject: Re: Broken SSH
PostPosted: Wed Jan 13, 2016 11:29 am 
Offline
Joined: Tue Jul 12, 2005 8:01 am
Posts: 670
Location: Salem, MA
I just turned on verbosity (python2 -v myth_status.py) and it pauses in two spots:

Code:
import numbers # precompiled from /usr/lib/python2.7/numbers.pyc

Code:
import unicodedata # dynamically loaded from /usr/lib/python2.7/lib-dynload/unicodedata.so

_________________
LinHES 8.3, 1 BE, 3 FE


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 


All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 29 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu