Install Apache Tomcat 9 on Ubuntu 16.04 for production
For this production setup we are going with the binary distributions of the Oracle Java Standard Edition version 8 and Apache Tomcat version 9.
It is obviously easier to install the Tomcat and Java packages from Ubuntu's repository and that is also a perfectly viable setup for a production environment, but consider these points:
Security updates
Using a Long Term Service version of Oracle's Java SE such as version 8 will secure access to security updates directly from Oracle for years. A new LTS version is released every 3 years. Once it's superseded by a new one, support will still continue for another 5 years with a Java SE Subscription for a total of 8 years.
Oracle changed their licensing scheme from April 2019 such that while their binary distribution is still free for personal and development use, it does require a Java SE Subscription for use in a server production environment.
For now there is reason to be a little cautious to adopt version 11, the next LTS release of Java SE, because it has undergone a lot of structural changes (all the way since version 8) that affect the build tool chain, IDE's and libraries. And do not use Oracle's "feature" versions released in between LTS versions for production use. Feature releases sport a new major version number every 6 month and security updates end immediately when a major version is superseded by the next one.
Although Ubuntu's Java packages are built from the same OpenJDK as Oracle's Java SE, they are only available through Ubuntu's Universe repository, which means they are maintained by the community, not Canonical itself. There is no guarantee of timely updates and no statement on how long a particular OpenJDK is supported. The same goes for Ubuntu's packaging and distribution of updates for Apache Tomcat.
Redhat is actually doing better than Ubuntu in this regard. Redhat pledged to support OpenJDK 8 until 2023 and help maintain fixes in upstream OpenJDK. Later they updated their OpenJDK Lifecycle and Support Policy to claim support until 2026 with quarterly updates.
Major version lock in
Another reason to use the binary distribution over Ubuntu's packages is that the packages might lock us in to a particular major version of OpenJDK or Apache Tomcat even though newer major versions are available. For instance Apache Tomcat 9 is not available as a package in Ubuntu 16.04 LTS.
No automatic updates
Managing operations without automatic package updates is a drawback, but on the other hand for a production environment, you might need to setup a system where package updates are gated through dev, test and staging environments before they reach production. So we a faced with complexity anyway.
So with all these points out of the way let's get started with the setup.
Tomcat's hotdeployment disabled
Tomcat's hotdeployment feature is not a security vulnerability by itself, but it is often exploited as part of attacks on vulnerable web applications to deploy backdoors into a running Tomcat. We are going to restrict the Tomcat service account from writing to it's hotdeployment folder and disable hotdeployment altogether.
A pattern for strict file permissions
On Linux systems we cannot meaningfully restrict file permissions from a process if the service account running the process owns the file. The owner can simply add permissions back. Group members however cannot change permissions.
If we are to restrict Tomcat's service account from writing to the hotdeployment folder, then we have to set root as owner and only give Tomcat read permission through the folder's group.
A typical pattern for file and folder permissions, that we will use throughout most of the Java and Tomcat installation, is exemplified with the Tomcat hotdeployment folder:
drwxr-x--- 3 root tomcat 4096 Jan 21 11:25 webapps/
Here the root account owns the folder - not Tomcat's service account. The folder is made available to Tomcat through the "tomcat" group with read and execute permission but no write permission. All other users have no permissions.
Let me underline that having root own the folder does not imply that we are running Tomcat as root. As we shall see later, we will use a Systemd unit file to launch Tomcat with the "tomcat" service account and "tomcat" group.
Install Oracle Java SE 8
Download Oracle Java SE 8 from http://www.oracle.com/technetwork/java/javase/downloads/
Hit the terminal ctrl+alt+t. Unpack the tar and move it to /usr/local:
$ cd Downloads $ tar xvzf jre-8_linux-x64_bin.tar.gz $ sudo mv jre-8 /usr/local/
Make root the owner of the files and only provide access to Java through a group called "java". Later we will add the Tomcat service account to this group.
$ sudo addgroup java $ cd /usr/local $ sudo chown -R root:java jre-8/ $ sudo chmod -R o-rwx jre-8/
Create a symbolic link "jre" that links to the current installation folder jre-8. If at some point we want to install a new version of Java, then all we have to do is point the "jre" link to the new installation folder and restart Java. The JRE_HOME environment variable, that we will set up later, need not change if it only refers to the "jre" link and not directly to the physical installation folder.
$ sudo ln -s /usr/local/jre-8/ /usr/local/jre
Finally make sure the Oracle Java installation is used over any other Java installation on path, by giving it a higher priority through the Debian Alternatives system. Even if we don't explicitly install OpenJDK it might sneak in as a dependency of other packages.
$ sudo update-alternatives --install /usr/bin/java java /usr/local/jre/bin/java 2000
Install Apache Tomcat 9
Download the tar.gz distribution of Apache Tomcat 9 from https://tomcat.apache.org/
Set a umask that strips write permission for group and other users. This will take affect on all files that we unpack in a moment.
$ umask 022
Unpack Tomcat and move it to /usr/local
$ tar xvzf apache-tomcat-9.0.14.tar.gz $ sudo mv apache-tomcat-9.0.14 /usr/local/
Create a symbolic link called "tomcat" that links to the physical installation folder apache-tomcat-9.0.14. Just like the Java installation folder we can install new versions of Tomcat alongside the old installation, relink the symbolic tomcat folder to the new installation folder and restart the Tomcat server. The CATALINA_HOME environment variable need not change because it refers to the symbolic link tomcat:
$ sudo ln -s /usr/local/apache-tomcat-9.0.14 /usr/local/tomcat
Create a service account named tomcat and add it to the java group:
$ sudo useradd -r -s /usr/sbin/nologin tomcat $ sudo adduser tomcat java
System administrator expect all services to log to a folder under /var/log. Lets create a folder /var/log/tomcat and redirect Tomcat logs here by replacing Tomcat's own log folder with a symbolic link to the /var/log/tomcat folder.
We are going to set the group on the log folder to "adm". This is the standard group on Ubuntu for administrators that need to read the log files. In addition we set the SETGID permission on the log folder. This insures that new files "inherit" the group "adm" from the folder and not the group of the user that creates the file.
$ sudo mkdir /var/log/tomcat $ sudo chown tomcat:adm /var/log/tomcat $ sudo chmod 750 /var/log/tomcat $ sudo chmod g+s /var/log/tomcat $ cd /usr/local $ rm -rf tomcat/logs $ sudo ln -s /var/log/tomcat /usr/local/tomcat/logs
Now it's time to implement the pattern for secure file permissions described earlier. We restrict Tomcat from writing to its hotdeployment folder, but need to grant write permission to the temp and work folders. The Tomcat service account will not own the installation files and folders, but is granted limited permissions through the tomcat group.
$ sudo chown -R root:tomcat apache-tomcat-9.0.14/ $ sudo chmod g+rx apache-tomcat-9.0.14/conf $ sudo chmod g+r apache-tomcat-9.0.14/conf/* $ sudo chmod g+w apache-tomcat-9.0.14/temp apache-tomcat-9.0.14/work
Tomcat as a Systemd service
Create a Systemd unit file for the Tomcat service:
$ sudo vim /etc/systemd/system/tomcat.service
Press i to enter insert mode, then paste this content to the file:
[Unit] Description=Apache Tomcat After=network.target [Service] Type=simple Environment=CATALINA_HOME=/usr/local/tomcat Environment=JRE_HOME=/usr/local/jre ExecStart=/usr/local/tomcat/bin/catalina.sh run ExecStop=/usr/local/tomcat/bin/catalina.sh stop SuccessExitStatus=143 User=tomcat Group=tomcat [Install] WantedBy=multi-user.target
The press ESC to exit insert mode and type ":wq" to write and quit.
The unit file is set up to launch Tomcat using its standard script catalina.sh. This way all the normal launch features of Tomcat work the way they always do - such as its use of environments variables, the sourced setenv.sh file etc.
It's important to use "catalina.sh run" instead of "catalina.sh start". The latter launched the process in the background and this can cause Systemd to lose track of the main process. If that happens then Systemd will erroneously conclude that the main process is done when the catalina.sh script finishes and consequently call the ExecStop script. This will lead to a premature shutdown of Tomcat.
Since Tomcat is not launched as a background process by the catalina.sh script itself we have to use "Type=simple" not "forking"!
Now activate the Tomcat Systemd service:
$ sudo systemctl daemon-reload $ sudo systemctl enable tomcat
From this point onwards we can start and stop Tomcat using these commands:
$ sudo systemctl start tomcat $ sudo systemctl stop tomcat
Check it's running on http://localhost:8080/
If everything is OK then it's time to remove the bundled web apps:
$ sudo systemctl stop tomcat $ sudo rm -rf tomcat/webapps/*
Tuning Tomcat's internal logging
Tomcat's own internal logging needs a bit of tuning for production use. We are going to disable console logging and make sure its Async logger uses a buffer instead of writing log statements to disk one by one.
Open Tomcat's logging configuration for editing.
$ sudo vim tomcat/conf/logging.properties
The root handlers are configured in a line that begins with ".handlers".
.handlers = 1catalina.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler
Press i to enter insert mode and remove the ", java.util.logging.ConsoleHandler" from the line.
Then add these lines:
1catalina.org.apache.juli.AsyncFileHandler.bufferSize = 8192 2localhost.org.apache.juli.AsyncFileHandler.bufferSize = 8192
Press ESC to exit insert mode and type ":wq" to write and quit.
The catalina.out file should not receive much output after we disabled Tomcat's Console logging. Catalina.out is special because it's not handled by a logging framework, it's just the stdout being appended to the file using the Linux operator ">>". It does not roll over, pack or delete.
First we set up logrotate to roll the file over daily:
$ sudo vim /etc/logrotate.d/tomcat
Press i to enter insert mode and add this content to the file:
/var/log/tomcat/catalina.out {
copytruncate
daily
rotate 30
compress
missingok
create 640 tomcat adm
}
Next set up a Cron job to compress log files every day and delete the files after 30 days.
$ sudo vim /etc/cron.daily/tomcat
Press i to enter insert mode and add this content to the file:
#!/bin/sh MAX_AGE_DAYS=30 if [ -d /var/log/tomcat ]; then find /var/log/tomcat -name 'catalina.*.log' -daystart -mtime +0 -print0 | xargs --no-run-if-empty -0 gzip -9 find /var/log/tomcat -name 'localhost.*.log' -daystart -mtime +0 -print0 | xargs --no-run-if-empty -0 gzip -9 find /var/log/tomcat -name 'catalina.*.log.gz' -mtime +$MAX_AGE_DAYS -print0 | xargs --no-run-if-empty -0 rm -- find /var/log/tomcat -name 'localhost.*.log.gz' -mtime +$MAX_AGE_DAYS -print0 | xargs --no-run-if-empty -0 rm -- fi
Press ESC to exit insert mode and type ":wq" to write and quit.
Set the file permissions of the Cron job:
$ sudo chmod 755 /etc/cron.daily/tomcat
Deployment
Disable hotdeployment:
$ sudo vi tomcat/conf/server.xml
Press i to enter insert mode and search for the <Host> tag and change unpackWARS and autoDeploy to "false":
<Host name="localhost" appBase="webapps" unpackWARs="false" autoDeploy="false">
Search for the <Server> tag and change the shutdown command to anything random:
<Server port="8005" shutdown="83n22ordnyodn24sewrg">
With hotdeployment disabled we have to stop Tomcat for deployments, unzip the WAR into the webapps folder, set appropriate permissions and restart tomcat. Let's imagine we are deploying a WAR file called helloworld.war:
$ sudo su -
$ mv helloworld.war /usr/local/tomcat/webapps/ $ cd /usr/local/tomcat/webapps/ $ systemctl stop tomcat $ unzip helloworld.war -d helloworld $ chown -R root:tomcat helloworld/ $ chmod -R 0640 helloworld/ $ systemctl start tomcat
Security announcements
Sign up to the tomcat-announce mailinglist to get security announcements. When a new security update is released simply install it side by side with the old installation folder, copy the webapps content and any custom configuration over and then relink /usr/local/tomcat to point to the new installation folder and restart tomcat.
