Security Basics: Secure Application Design

(This article was originally published on the Findings From the Field blog.) Siemens received a lot of criticism when people learned that the Stuxnet worm used a hard-coded password to access WinCC databases remotely and compromise them. Some of the criticism was justified, and some was not. In this posting we explore what a secure application looks like, especially control system type applications with intensive server/server communications, and requirements for unattended reboot. Web Server Design Let’s start by looking at a common design paradigm – a farm of web server applications, each connecting to a common set of database servers. The most secure design is one where the database enforces all security policies. In a corporate environment, the web browsers can send the corporate credentials of the user they run as to the web server, the web server can validate those credentials and spawn a sub-process as the user, and that sub-process can pass the user’s credentials to the relational database. The database can be configured to understand which users have access to which tables and rows and enforce security policies. Web Server Architecture This approach breaks down when applied outside of a corporation to millions or tens of millions of end users. Managing those users as Windows accounts in an Active Directory server is impractical. In these circumstances, users expect to be challenged for a password or other means of identification, and the web server applications must enforce some security policies rather than the database. Enforcement of some security policies has to move higher in the solution architecture, closer to the end user. In this scenario, the best practice is to have a different password for each web server, and store the database password in an encrypted file on the web server, outside of the web site filesystem tree. When the web application starts, it reads the password, decrypts the file, sends the password to the database, and then erases the in-memory copy of the password. The encryption is not a strong security measure – it is intended only to hide the credentials from administrators who might casually glance at the file. The real protection for the saved password is that the password file is not accessible with a URL, and filesystem permissions protect the file from unprivileged accounts. Very secure sites go one step further – they encrypt the filesystem storing the credentials, or encrypt the credentials file itself in such a way that the web server application cannot automatically decrypt. At these sites, the web servers need human intervention to complete a reboot process. The servers need to prompt a human user for a password or other means to decrypt the filesystem or the stored password file. This scenario is an example of the need for server-to-server communications. The web servers must talk to the database servers. The database still enforces security to the extent that the web server credentials only allow the web servers to do things that end users are allowed to do through web servers – retrieve their data, modify their data, etc.. The web server application sorts out which data belongs to which user and ensures that one user cannot access another user’s data. To reboot unattended, the web servers must be able to authenticate to the database servers using stored credentials. Some sort of accessible, stored credentials are essential if the server to server communications are to be re-established after an unattended reboot. Control System Security Issues In applying this scenario to control systems, the theme is still that servers enforce security as much as is practical, but there are mitigating factors: PLC’s, RTU’s and other devices are all servers, and there are hundreds or thousands of them to manage, sometimes over a huge geography. These devices typically cannot manage many separate users, nor are they typically integrated with Active Directory or other central user management systems. Often control system components must be able to reboot unattended. Sometimes even those control system components which use conventional operating systems are scattered over a huge geography. Flying personnel out to site each time a machine reboots is impractical. A remote desktop session to the computer to provide a connection password is a possibility that the most secure sites may wish to examine, but is impractical in emergencies where large numbers of computers lose power and restart. Even though there may be a very small number of users authorized to operate or administer a control system, when changing shifts there may not be enough time to destroy all of an old user’s applications and log in as a new user. Many physical processes require constant human supervision and “flying blind” for several minutes while applications are restarted as a new user may be both unsafe and illegal. Unlike home users, HMI workstations can be very physically secure. Often the HMI computers run in a locked rack in the control room, or they maybe located  in a server room directly above or below the control room, with only KVM connections into the control room. As a consequence, best practice in control systems design tends toward the designs used in the web server example, when tens of millions of users are involved. Enforcement of some security policies is pushed closer to the end user, or even into applications running on the HMI workstation. Role-Based Security In spite of the differences between conventional enterprise security models and control system security requirements, role-based security is still something control systems should enforce. Ideally, security systems should provide fine-grained controls governing which users can access which functions and data. These fine-grained controls are then combined into a handful of “roles.” All operators for example, get one kind of access, network administrators get another kind, and process engineers get a third. Ideally, the permissions for each role are configurable, and end users are assigned one or more roles, rather than hundreds of individual permissions. In practice though, control system components and especially data acquisition and control devices do not provide fine-grained controls over security. They really should define a handful of roles though, and roles should be enforced as deep into the tree of servers as is practical. For example: if a worm takes over an operator HMI workstation, that worm should not find itself able to reprogram a PLC, or reprogram the SCADA database, or re-flash PLC firmware. These are not functions control room operators need to carry out, and the credentials for HMI applications should permit only those functions which operators do need to carry out. Stored Credentials As a rule, it is not practical on every shift change to have the old operators “log out” of each of hundreds of PLC’s, and the new shift “log in.” Most control systems are expected to operate continuously and seamlessly across a shift change. HMI servers are expected to remain connected continuously to alarm servers, historians, real-time databases, and OPC servers. Each of these servers must remain continuously connected to hundreds of PLC’s and RTU’s, no matter how many or few operators are logged in. This, along with the unattended reboot requirement, demands a stored-credential model just like the web server scenario. Stored credentials permit these essential and continuously-running servers to “log into” their data sources automatically and as-needed. Ideally though, the credentials are stored with some sort of encryption on the machines that need them. Ideally, the passwords are not hard-coded in the control systems applications. Ideally, there is a seamless and thoroughly reliable way to regularly change all of these credentials at once, so that stolen credentials cannot be used to attack the site indefinitely. Unfortunately, few control system vendors meet these requirements at present. Most distributed control applications do not have a way to seamlessly update credentials. Not all devices have so much as per-user role-based access controls. Looking Forward Control systems are not like most conventional applications. For a small number of end users, control systems have a comparatively large numbers of servers. Security for control systems cannot be compared to the simplest enterprise applications, but are better compared to the most complex, tiered enterprise systems. Saying that an alarm server application should prompt an end user for a different password for every server it connects to, every time the alarm server reboots, is not practical at most sites. That said, there is a great deal of improvement that can be made in control systems security. Role-based access controls must become pervasive at all levels of the architecture. Account and password management functions must extend to even the most remote devices. Stored credentials on everything from the dumbest devices to the smartest servers must be easily update-able, and regularly updated. The Stuxnet worm has resulted in a lot of nonsense being published about the state of control system security. Some of the discussion of the worm though, does point out aspects of control system security designs which really should improve.