How to Solve Error SSH Remote Access is not Responding

Introduction

The title of this article is the focus on the content. It is a problem on remote accessing server using ssh command. Another article in this link provides detail description about the factor causing the problem. In this case, a server remote process using ssh command suddenly failed. It happens because of the package removal process.  It erase libgomp and nss-softokn-freebl package using yum utility command. The remote process just stop without any further progress. Terminating the is the only option after waiting for too long. It shows as follows :

user@hostname:~$ ssh myuser@xxx.xxx.xxx.xxx
^C
user@hostname:~$

Assessing the current situation

Fortunately, before the sshd service stop working, there is one active terminal connecting to the remote server. To be able to grasp the root cause of the problem, check the status of the sshd service :

[root@localhost nspr]# systemctl status sshd
● sshd.service - OpenSSH server daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2018-07-16 09:30:49; xx months xx days ago
     Docs: man:sshd(8)
           man:sshd_config(5)
 Main PID: 10481 (sshd)
   CGroup: /system.slice/sshd.service
           └─10481 /usr/sbin/sshd -D

Feb 04 07:36:55 localhost.localdomain sshd[19128]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "xxxxxx"
Feb 04 07:36:57 localhost.localdomain sshd[19128]: Failed password for xxxxxx from xxx.xxx.xxx.xxx port xxxxxx ssh2
Feb 04 07:36:57 localhost.localdomain sshd[19128]: Connection closed by xxx.xxx.xxx.xxx port 42480 [preauth]
Feb 04 07:36:57 localhost.localdomain sshd[19128]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=xxx.xxx.xxx.xxx  user=xxxxxx
Feb 04 07:37:00 localhost.localdomain sshd[19133]: Accepted password for xxxxxx from xxx.xxx.xxx.xxx port 42486 ssh2
Feb 04 08:27:47 localhost.localdomain sshd[10481]: /usr/sbin/sshd: error while loading shared libraries: libfreebl3.so: cannot open shared object file: No such file or directory
Feb 04 08:29:34 localhost.localdomain sshd[10481]: /usr/sbin/sshd: error while loading shared libraries: libfreebl3.so: cannot open shared object file: No such file or directory
Feb 04 08:31:01 localhost.localdomain sshd[10481]: /usr/sbin/sshd: error while loading shared libraries: libfreebl3.so: cannot open shared object file: No such file or directory
Feb 04 08:31:28 localhost.localdomain sshd[10481]: /usr/sbin/sshd: error while loading shared libraries: libfreebl3.so: cannot open shared object file: No such file or directory
Feb 04 08:31:47 localhost.localdomain sshd[10481]: /usr/sbin/sshd: error while loading shared libraries: libfreebl3.so: cannot open shared object file: No such file or directory
[root@localhost nspr]#

Attempt to execute the solution

Unfortunately, restarting the sshd service for a solution is helpless. The execution command for restarting the sshd service fails as follows :

[root@localhost ~]# systemctl restart sshd
Job for sshd.service failed because the control process exited with error code. See "systemctl status sshd.service" and "journalctl -xe" for details.
[root@localhost ~]#

Another additional information regarding the log of the ssh service status after restarting it exists below :

[root@localhost mnt]# systemctl status sshd
● sshd.service - OpenSSH server daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Mon 2019-02-04 09:03:53; 5s ago
     Docs: man:sshd(8)
           man:sshd_config(5)
  Process: 22653 ExecStart=/usr/sbin/sshd -D $OPTIONS (code=exited, status=127)
 Main PID: 22653 (code=exited, status=127)

Feb 04 09:03:53 localhost.localdomain systemd[1]: Failed to start OpenSSH server daemon.
Feb 04 09:03:53 localhost.localdomain systemd[1]: Unit sshd.service entered failed state.
Feb 04 09:03:53 localhost.localdomain systemd[1]: sshd.service failed.
[root@localhost mnt]# journalctl -xe
Feb 04 09:03:03 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:13 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:13 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:13 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:13 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:23 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:23 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:24 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:24 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:34 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:34 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:34 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:34 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:44 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:44 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:44 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:44 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:52 localhost.localdomain polkitd[821]: Registered Authentication Agent for unix-process:22644:1753845907 (system bus name :1.11011 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path 
Feb 04 09:03:52 localhost.localdomain systemd[1]: Stopping OpenSSH server daemon...
-- Subject: Unit sshd.service has begun shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit sshd.service has begun shutting down.
Feb 04 09:03:52 localhost.localdomain sshd[10481]: Received signal 15; terminating.
Feb 04 09:03:53 localhost.localdomain systemd[1]: Starting OpenSSH server daemon...
-- Subject: Unit sshd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit sshd.service has begun starting up.
Feb 04 09:03:53 localhost.localdomain sshd[22653]: /usr/sbin/sshd: error while loading shared libraries: libfreebl3.so: cannot open shared object file: No such file or directory
Feb 04 09:03:53 localhost.localdomain systemd[1]: sshd.service: main process exited, code=exited, status=127/n/a
Feb 04 09:03:53 localhost.localdomain systemd[1]: Failed to start OpenSSH server daemon.
-- Subject: Unit sshd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit sshd.service has failed.
-- 
-- The result is failed.
Feb 04 09:03:53 localhost.localdomain systemd[1]: Unit sshd.service entered failed state.
Feb 04 09:03:53 localhost.localdomain systemd[1]: sshd.service failed.
Feb 04 09:03:53 localhost.localdomain polkitd[821]: Unregistered Authentication Agent for unix-process:22644:1753845907 (system bus name :1.11011, object path /org/freedesktop/PolicyKit1/AuthenticationAge
Feb 04 09:03:54 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:54 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:54 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:03:54 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:04:04 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:04:04 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:04:04 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
Feb 04 09:04:04 localhost.localdomain kernel: [drm] EDID has major version 0, instead of 1
[root@localhost mnt]#

The solution is by far is quite clear. In the about output, sshd service is in the state of error. Because it cannot find the libfreebl3.so file. Therefore, try to do something so that it can be discovered by the sshd service.  For instance, copy the libfreebl3.so file from another server. Do it but the server must have identical operating system or environment. It is quite simple though but it actually works. So, the main question is why bother copy the needed files ?, why don’t just reinstall the package using yum utility ?. Please remember that the ‘yum’ utility is also broken because of this incident. The following is an attempt to copy the file from another server to the server with the problem :>/p>

1. Copy the file with the name of libfreebl3.so. In this case, the operating system is CentOS. It exists in the /usr/lib64 folder. Execute several commands as follows :

[root@hostname lib64]# pwd
/usr/lib64
[root@hostname lib64]# ls -al | grep libfree3.so
[root@hostname lib64]# ls -al | grep libfreebl3.so
-rwxr-xr-x.  1 root root   11448 May 16 2018 libfreebl3.so
[root@hostname lib64]# scp libfreebl3.so root@xxx.xxx.xxx.xxx:/root
root@xxx.xxx.xxx.xxx's password: 
libfreebl3.so                                                                                                                                                             100%   11KB   1.9MB/s   00:00    
[root@hostname lib64]# 

2. Don’t forget to modify the file permission. Execute the following command :

[root@localhost lib64]# chmod +x libfreebl3.so 
[root@localhost lib64]#

3. Finally, test the remote access again using ssh command. Before that, don’t forget to restart the sshd service by executing the following command :

[root@localhost lib64]# systemctl restart sshd
[root@localhost lib64]#