Centos ssh failure

I have a customer who uses a fairly new Centos server to function as an ssh gateway to his ancient SCO box. On Friday he told me that no one could ssh in.

My first thought was the router, but it seemed to be operating correctly. I had him log in at the console and try an "ssh localhost". That complained about syntax errors in ssh_config at a few lines. I had him read me the offending lines; there was nothing wrong.

I needed to login myself and didn't want to drive to Boston. Not only didn't I want to waste the time, but this customer has an impossible parking situation. He actually has a good sized parking lot, but because it was too big for what his employees needed, he rented spsces to a used car dealer! Whenever I have to go there, I cringe because I can almost never find a parking spot anywhere near them.

So, I reprogrammed the router to let my ip in through telnet and had him turn it on. Normally telnetd wouldn't even be installed on a Linux box; I don't even remember why it was here, but having it available certainly saved some time and trouble.

After logging in I confirmed that there really was nothing wrong with the ssh_config. I next looked at /var/log/secure and found entries like this:

 Jun  8 07:31:59 e-smith sshd[24987]: Accepted password for rmtxyz 
 from ::ffff:127.0.0.1 port 53387 ssh2
 Jun  8 07:31:59 e-smith sshd[24989]: pam_unix(sshd:session): session 
 opened for user rmtxyz by (uid=1000)
 Jun  8 07:31:59 e-smith sshd[24989]: pam_loginuid(sshd:session): 
 set_loginuid failed opening loginuid
 Jun  8 07:31:59 e-smith sshd[24989]: pam_loginuid(sshd:session): 
 set_loginuid failed 
 Jun  8 07:31:59 e-smith sshd[24989]: fatal: PAM session setup failed[14]: 
 Cannot make/remove an entry for the specified session
 Jun  8 07:31:59 e-smith sshd[24987]: syslogin_perform_logout: 
 logout() returned an error
 
 

That looks like a PAM problem, but I couldn't see anything wrong with PAM either. I scouted around the web a bit and found references to similar problems from a few years back. These looked like they might have come from a bad update to ssh at that time, but how could that be affecting us now? That didn't make sense.

Nevertheless, I figured that would only take a minute to try, so I did a "yum update openssh" and to my complete surprise ssh logins started working again.

I have no idea what this was, but it's fixed for now. I took the telnet config out of the router and all is back to normalcy.