Was it spying, or something innocent?
I don't know whether I should admit this, but one of my favorite activities as a security manager is incident response.
Sure, incidents can be a security manager's worst nightmare, putting you and your security program on the spot. But they are fairly rare at my company, so when we do have one, it is something of a break from my general routine of audits, compliance activity and meetings. They are usually challenging, and sometimes we catch a bad guy.
At issue: A log shows that two high-level executives' PCs were used to log into a sensitive tool. That seems rather suspicious.
Action plan: Look into the incident, starting by asking the executives what they were doing at the time of the log-ins.
Our most recent event didn't uncover any bad guys, as it turns out, but we did discover a configuration error in our Microsoft DNS servers.
Here's what happened: One of our engineers was using software called Remote Admin (Radmin) to troubleshoot one of the expensive, specialized measurement tools that my company designs and manufactures. While reviewing connection logs from the Radmin server software, he noticed some suspicious activity that had originated from the PCs of two of the most senior executives in the company. That was very strange, since our high-level executives don't normally log into the tools. Why would these executives have done that? I had to wonder. So I asked them. One took a look at the logs and said that at the time of the connection, he had been sleeping. The other executive said that when his machine was supposedly logging into one of our tools, he was high above the Atlantic on his way to Europe. So how could these machines, which were turned off or unattended, be responsible for the suspicious connections?
As it turned out, they weren't. In fact, the log-ins weren't done from two different PCs belonging to two executives but from one PC belonging to an engineer with a legitimate reason to log into the tool.
So how was it that a Domain Name System reverse lookup had fingered the wrong parties?
In our company, we use Dynamic Host Configuration Protocol, or DHCP, which assigns an IP address from a predefined network range. We have DHCP configured so that each IP address assignment expires after two weeks, after which the PC is assigned a different IP address the next time it comes on the network. What I hadn't realized was that our Windows environment keeps the cache information on all these IP address assignments rather than purging the old entries.
I couldn't understand why we would arrange things this way, so I asked our Windows server team, who told me that they had disabled automatic flushing of the DNS cache because it had caused problems. What sort of problems? I wanted to know. Uh, well, no one could remember exactly. In any event, this caching was why our logs had pointed to the two executives, since their PCs had been assigned those two IP addresses in the past.
We'll have to investigate what the reason was for disabling the flushing of the DNS cache; it might not even be a real problem anymore, and we'll certainly find a way around it if it is still a valid problem, so that we can re-enable the automatic flushing.
In the end, we didn't have any executives involved in industrial espionage. But even though this incident was a false positive, it was an interesting diversion. And it provided a good lesson on the importance of reviewing configuration baselines to ensure that DNS servers properly flush information. Of course, I also want to have historical information available to answer questions such as who was assigned a particular IP address at a given date and time. That can be critical information to have, and we'll want to retain it.
This week's journal is written by a real security manager, "Mathias Thurman," whose name and employer have been disguised for obvious reasons. Contact him at firstname.lastname@example.org.
Read more about Security in Computerworld's Security Topic Center.
This story, "Was It Spying, or Something Innocent?" was originally published by Computerworld.