Status Worker Reference

Introduction

Tomcat Connectors has a special type of worker, the so-called status worker. The status worker does not forward requests to Tomcat instances. Instead it allows to retrieve status and configuration information at runtime, and furthermore to change many configuration items dynamically. This can be done via a simple embedded web interface.

The status worker is especially powerful, when used together with load balancing workers.

This document does not explain the HTML user interface of the status worker. Until now it is very simple, so just go ahead and use it. This doc instead tries to explain the less obvious features of the status worker. We also will give a complete coverage of the various request parameters and their meaning, so that you can include the status worker in your automation scripts.

The documentation of the status worker starts with jk 1.2.20

Usage Patterns

Actions

The status worker knows about the following actions:

  • list: lists the configurations and runtime information of all configured workers. The output will be grouped by global information first (version data), then load balancer information, after that AJP worker information and finally the legend. For load balancers, there will be a summary part, and after that details for each member worker. For all workers, we also include the URL mappings (forward definitions).
  • show: the same as list, but only shows data for one chosen worker
  • edit: produces a form to edit configuration data for a chosen worker. There is a special subtype of "edit", that makes it easy to change one attribute for all members of a load balancer, e.g. their activation state.
  • update: commit changes made in an edit form. Caution: the changes will not be persisted to the configuration files. As soon as your restart your web server, all changes made through the status worker will be lost! On the other hand, the changes done by the status worker will be applied during runtime without a restart of the web server.
  • reset: reset all runtime statistics for a worker.
  • recover: Mark a member of a load balancer, that is in error state, for immediate recovery.
  • version: only show version information of the web server and the JK software
  • dump: list the original workers configuration. Caution: the dump will only contain the configuration that was used during startup. Any changes applied later by the dynamic management interface of the status worker itself will not be contained in this dump. The dump action has been added in version 1.2.27.

Output Format

For most actions you can choose between 4 output formats.

  • HTML: Used interactively with a browser
  • XML: Mostly useful for automation, when your scripting environment is XML friendly. This format has rich structure information, but does not work line based, so you would really like to use it together with XML tools.
  • Properties: This format is a line based format, that conforms to the rules of Java property files. Most structure information is contained in the hierarchical key. For information, that is of configuration nature, the format should produce lines very similar to the ones you can use in workers.properties. It will not produce a complete configuration file!
  • Text: A simple textual output format.
The "edit" action does only make sense for the HTML output type.

User Interface Features

In the HTML view, there is an automatic refresh feature, implemented via the meta refresh option of HTML. Once you start the automatic refresh, the UI will will respect it for all actions except edit, update and maintain. Even if you navigate through one of those, the automatic refresh will start again as soon as you come back to one of the other actions.

Many parts of the HTML page can be minimised, if they are not interesting for you. There are a couple of "Hide" links, which will collapse parts of the information. The feature exists for the following blocks of information:

  • Legend: Do not show the legend for the information presented in "list" and "show" actions
  • URI mappings: Do not show the URI mapping for the workers
  • Load Balancing Workers: Do not show workers of type "lb"
  • AJP Workers: Do not show workers of type ajp
  • Balancer Members: Do not show detailed information concerning each member of load balancers
  • Load Balancer Configuration: Do not show configuration data for load balancers
  • Load Balancer Summary: Do not show status summary for load balancers
  • AJP Configuration: Do not show configuration data for ajp workers load balancer members
The last three minimisation features have been added in version 1.2.27.

Special Considerations concerning URL Maps and Virtual Hosts

Note: The following restriction has been removed starting with version 1.2.26.

The Apache module mod_jk makes use of the internal Apache HTTP Server infrastructure concerning virtual hosts. The downside of this is, that the status worker can only show URL maps, for the virtual host it is defined in. It is not able to reach the configuration objects for other virtual hosts. Of course you can define a status worker in any virtual host you are using. All information presented apart from the URL maps will be the same, independent of the virtual host the status worker has been called in.

Logging

The status worker will log changes made to the configuration with log level "info" to the usual JK log file. Invalid requests will be logged with log level "warn". If you want to report some broken behaviour, log file content of level "debug" or even "trace" will be useful.

Configuration

Basic Configuration

The basic configuration of a status worker is very similar to that of a usual ajp worker. You need to specify a name for the worker, and the URLs you want to map to it. The first part of the configuration happens in the workers.properties file. We define a worker named mystatus of type status:

worker.list=mystatus
worker.mystatus.type=status
Then we define a URL, which should be mapped to this worker, i.e. the URL we use to reach the functionality of the status worker. You can use any method mod_jk supports for the web server of your choice. Possibilities are maps inside uriworkermap.properties, an additional mount attribute in workers.properties, or JkMount for the Apache HTTP Server. Here's an example for a uriworkermap.properties line:
/private/admin/mystatus=mystatus
The URI pattern is case sensitive.

As you will learn in the following sections, the status worker is very powerful. You should use the usual authentication and authorisation methods of your web server to secure this URL.

You can also define multiple instances of the status worker, by using different names and URL mappings. For instance you might want to configure them individually and then allow special groups of people to use them

Output Customisation

There are a couple of attributes for the workers.properties entries, which allow to customise various aspects of the output of the status worker.

The attribute css can be set to the URL of a stylesheet:

worker.mystatus.css=/private/admin/static/mystatus.css
When writing HTML output, the status worker then includes the line
<link rel="stylesheet" type="text/css" href="/private/admin/static/mystatus.css" />
There is no sample stylesheet included with the mod_jk release, and by default the attribute css is empty, so no stylesheet reference will be included in the pages. The HTML code of the status worker output pages does not include any class attributes. If you like to contribute a stylesheet or improvements to the HTML layout, please contact us on the tomcat developers list.

The properties output format can be customised via the attribute prefix. The names of all properties the status worker does output, will begin with this prefix. The default is "worker".

Several attributes influence the format when writing XML output. The attribute ns allows to set a namespace prefix, that will be used for every status worker+element. The default is "jk:". Setting it to "-" disables the namespace prefix.

With the attribute xmlns you can map the prefix to a namespace URL. The default value is xmlns:jk="http://tomcat.apache.org". Setting it to "-" disables the output of the URL.

Finally you can specify an XML document type via the attribute doctype. The specified string will be inserted at the beginning of the document, directly after the xml header. The default is empty.

Securing Access

We urge you to use the builtin access control features of your web server to control access to the status worker URLs you have chosen. Nevertheless two configuration attributes of status workers are helpful. The attribute "read_only" disables all features of the status worker, that can be used to change configurations or runtime status of the other workers. A read_only status worker will not allow access to the edit, update, reset or recover actions. The default value is false, ie. read/write. To enable read_only you need to set it to true.

You could configure two status workers, one has read_only and will be made available to a larger admin group, the other one will be used fully featured, but only by fewer people:

worker.list=jk-watch
worker.jk-watch.type=status
worker.jk-watch.read_only=true
worker.jk-watch.mount=/user/status/jk
worker.list=jk-manage
worker.jk-manage.type=status
worker.jk-manage.mount=/admin/status/jk
Starting with version 1.2.21, a read/write status worker can also be switched temporarily into read-only mode by the user via a link in the HTML GUI. The user can always switch it back to read/write. Only a status worker configured as read-only via the "read_only" attribute is completely safe from applying any changes.

The other attribute you can use is user. By default this list is empty, which means no limit on the users. You can set "user" to a comma separated list of user names. If your web server is configured such that it sends the user names with the request, the status worker will check, if the name attached with the request is contained in it's "user" list.

The user list can be split over multiple occurrences of the "user" attribute.

By default, the user names are matched case sensitively. Starting with version 1.2.21 you can set the attribute user_case_insensitive to true. Then the comparison will be made case insensitive.

Service Availability Rating

For load balancing workers the status worker shows some interesting overview information. It categorises the members of the load balancer into the classes "good", "bad" and degraded". This feature can be combined with external escalation procedures. Depending on your global system design and your operating practises your preferred categorisation might vary.

The categorisation is based on the activation state of the workers (active, disabled or stopped), which is a pure configuration state, and the runtime state (OK or ERR with possible substates idle, busy, recovering, probing, and forced recovery) which only depends on the runtime situation.

The runtime substates have the following meaning:

  • OK (idle): This worker didn't receive any request since the last balancer maintenance. By default balancer maintenance runs every 60 seconds. The worker should be OK, but since we didn't have to use it for some time, we can't be sure. This state has been called N/A before version 1.2.24.
  • OK (busy): All connections for this worker are in use for requests.
  • ERROR (recovering): The worker was in error state for some time and is now marked for recovery. The next request suitable for this worker will use it.
  • ERROR (probing): After setting the worker to recovering, we received a request suitable for this worker. This request is now using the worker.
  • ERROR (forced recovery): The worker is in error, but we don't have an alternative worker, so we keep using it.

By default the status worker groups into "good" all members, that have activation "active" and runtime state not equal to "error" with empty substate. The "bad" group consists of the members, that have either activation "stopped", or are in runtime state "error" with empty substate.

Workers that fit neither of the two groups, are considered to be "degraded".

You can define other rules for the grouping into good, bad and degraded. The two attributes "good" and "bad" can be populated by a comma-separated list ob single characters or dot-separated pairs. Each character stands for the first character of one of the possible states "active", "disabled", "stopped", "ok", "idle", "busy", "recovering" and "error". The additional states "probing" and "forced recovery" are always rated equivalent to "recovering". Comma-separated entries will be combined with logical "or", if you combine a configuration and a runtime state with a dot. the are combined with logical "and". So the default value for "good" is "a.o,a.i,a.b,a.r", for "bad" it is "e,s".

The status worker first tries to match against the "bad" definitions, if this doesn't succeed it tries to match against "good", and finally it chooses "degraded", if no "bad" or "good" match can be found.

Request Parameters

This section should help you building automation scripts based on the jk status management interface. This interface is stable in the sense, that we only expect to add further parameters in the future. Existing parameters from previous versions will keep their original semantics. We also expect the output formats XML, Properties and Text to be kept stable. So please use those, if you want to parse status worker output in your automation scripts.

Actions

The action is determined by the parameter cmd. It can have the values "list", "show", "edit", "update", "reset", "recover", "version" and "dump". If you omit the cmd parameter, the default "list" will be used. All actions except for "list", "refresh", "version" and "dump" need additional parameters.

The action "dump" has been added in version 1.2.27.

Output Format

The format is determined by the parameter mime. It can have the values "html", "xml", "txt" and "prop". If you omit the mime parameter, the default "html" will be used. The action "edit" (the edit form) does only make sense for "mime=html".

Worker Selection

Actions that operate on a single worker need one or two additional parameters to select this worker. The parameter w contains the name of the worker from the worker list. If an action operates on a member (sub worker) of a load balancer, the parameter w contains the name of the load balancer worker, and the additional parameter sw contains the name of the sub worker.

Automatic Refresh

During automatic refresh, the parameter re contain the refresh interval in seconds. If you omit this parameter, automatic refresh will be off.

Hide Options

The parameter opt contains a bit mask of activated options. The default is 0, so by default no options are activated. The following options exist:

  • 0x0001: hide members of lb workers
  • 0x0002: hide URL maps
  • 0x0004: hide the legend
  • 0x0008: hide load balancer workers
  • 0x0010: hide ajp workers
  • 0x0020: only allow read_only actions for a read/write status worker.
  • 0x0040: hide load balancer configuration
  • 0x0080: hide load balancer status summary
  • 0x0100: hide configuration for ajp and load balancer member workers
Values 0x0040-0x0100 have been added in version 1.2.27.

Data Parameters for the standard Update Action

You can use the edit action with a final click to the update button, to change settings of workers. But you can also make direct calls to the update action. The following request parameters contain the configuration information, you want to change. First the list for load balancer workers:

  • vlr: retries (number)
  • vlt: recover_time (seconds)
  • vlee: error_escalation_time (seconds)
  • vlx: max_reply_timeouts (number)
  • vls: sticky_session (0/f/n/off=off, 1/t/y/on=on; case insensitive)
  • vlf: sticky_session_force (0/f/n/off=off, 1/t/y/on=on; case insensitive)
  • vlm: method (0/r="Requests", 1/t="Traffic", 2/b="Busyness", 3/s="Sessions", 4/s="Next"; case insensitive, only first character is used)
  • vll: lock (0/o="Optimistic", 1/p="Pessimistic"; case insensitive, only first character is used)
And now the list of parameters you can use to change settings for load balancer members:
  • vwa: activation flag (0/a="active", 1/d="disabled", 2/s="stopped"; case insensitive, only first character is used)
  • vwf: load balancing factor (integer weight)
  • vwn: route for use with sticky sessions (string)
  • vwr: redirect to define simple failover rules (string)
  • vwc: domain to tell JK about your replication design (string)
  • vwd: distance to express preferences (integer)
Finally the list of parameters you can use to change settings for ajp workers and ajp load balancer members:
  • vahst: host (string)
  • vaprt: port (number)
  • vacpt: connection_pool_timeout (number)
  • vact: connect_timeout (number)
  • vapt: prepost_timeout (number)
  • vart: reply_timeout (number)
  • var: retries (number)
  • varo: recovery_options (number)
  • vabl: busy_limit (number)
  • vamps: max_packet_size (number)
Note that changing the host name or port will only take effect for new connections. Already established connections to the old address will still be used. Nevertheless this feature is interesting, because you can provision load balancer members with port "0", which will automatically be stopped during startup. Later when you know the final names and ports, you can set them and they will be automatically activated.

The leading character "v" has been added to the parameters in version 1.2.27. Changing settings for ajp workers has also been introduced in version 1.2.27.

For the details of all parameters, we refer to the workers.properties Reference.

Aspect Editing for Load Balancer Members

You can use the edit action to edit all settings for a load balancer or for a member of a load balancer respectively on one page. If you want to edit one configuration aspect for all members of a load balancer simultaneously, this will be triggered by the parameter att. The value of the parameter indicates, which aspect you want to edit. The list is the same as in the previous section, except for "vahst" and "vaprt": "vwa", "vwf", "vwn", "vwr", "vwc", "vwd", "vacpt", "vact", "vapt", "vart", "var", "varo", "vabl" and "vamps". But here you need to put the name into the parameter att, instead of using it as a request parameter name.

The values of the common aspect for all the load balancer members will be given in parameters named "val0", "val1", ....