In Open Liberty, you have the option to format your server logs in either basic or JSON format. When logs are in JSON format, you have to specify the sources (message, trace, accessLog, ffdc, audit) you want to send to messages.log or console.log and standard-out. Open Liberty 20.0.0.8 adds the option for you to include fields from the accessLogging logFormat attribute in the JSON logs. Previously, only selected fields would be printed in JSON logs. You are now able to include other NCSA (National Center for Supercomputing Applications) access log fields in your JSON logs.

Also, you now have the ability to define which JSON access log fields you want from the accessLogging logFormat attribute, which can then be sent to a log analysis tool, such as the ELK (Elasticsearch, Logstash, Kibana) stack. With this new feature, you can receive more informative logs that suit your needs. For example, you are able to specify that you want the user ID and request time fields in your JSON access logs — then, these can be filtered by user ID in Kibana and track performance on a user-by-user basis. When logs are in JSON format, you can use the new jsonAccessLogFields logging attribute to specify whether you want your access logs to have the default set of fields or a custom set of fields based on the HTTP accessLogging logFormat attribute. In order to receive access logs, the property accessLogging or httpAccessLogging must be set. For example, you can set the following attributes in your server.xml:

<httpEndpoint id="defaultHttpEndpoint" httpPort="9080" httpsPort="9443" host="*">
    <accessLogging logFormat='%R{W} %u %{my_cookie}C %s'/>
</httpEndpoint>
<logging messageFormat="json" messageSource="message,accessLog" jsonAccessLogFields="logFormat"/>

Now, in the messages.log file, your access logs will contain the four fields specified in the accessLogging logFormat attribute (elapsed time, user ID, cookie, and response code).

{
    "type": "liberty_accesslog",
    "host": "192.168.1.15",
    "ibm_userDir": "/you/jennifer.zhen.chengibm.com/libertyGit/open-liberty/dev/build.image/wlp/usr/",
    "ibm_serverName": "defaultServer",
    "ibm_cookie_my_cookie": "example_cookie",
    "ibm_responseCode": 200,
    "ibm_datetime": "2020-06-18T09:30:47.693-0400",
    "ibm_sequence": "1592487047653_0000000000001"
}

This new functionality is also available for the logstashCollector-1.0 feature by adding the following to your server.xml:

<featureManager>
    <feature>logstashCollector-1.0</feature>
</featureManager>

<logstashCollector
    jsonAccessLogFields="logFormat">
</logstashCollector>

This table describes the new fields available with the corresponding logFormat token:

For more information:

In Open Liberty, you have the option to format your server logs in basic or JSON format. When logs are in JSON format, you have to specify the sources (message, trace, accessLog, ffdc, audit) you want to send to messages.log or console.log and standard-out. This feature adds a new option for you to write pre-formatted JSON application logs directly to System.out or System.err. Previously, when Open Liberty was running with JSON logging enabled, Open Liberty embeds anything written to System.out or System.err into the message field of a liberty_message event. Now, you can write JSON directly to System.out or System.err without wrapping in the liberty_message event.

You can now write your JSON application logs directly to System.out or System.err, which can then be sent to a log analysis tool, such as the ELK (Elasticsearch, Logstash, Kibana) stack. Previously, when JSON logging was enabled, pre-formatted JSON application logs would look like this:

{
    "type":"liberty_message",
    "host":"192.168.0.119",
    "ibm_userDir":"\/you\/yushan.lin@ibm.com\/Documents\/archived-guide-log4j\/finish\/target\/liberty\/wlp\/usr\
    ",
    "ibm_serverName":"log4j.sampleServer",
    "message":"{\n   \"timeMillis\" : 1587666082123,\n
            \"thread\" : \"Default Executor-thread-8\",\n
            \"level\" : \"WARN\",\n
            \"loggerName\" : \"application.servlet.LibertyServlet\",\n
            \"message\" : \"hello liberty servlet warning message!\",\n
            \"endOfBatch\" : false,\n
            \"loggerFqcn\" : \"org.apache.logging.log4j.spi.AbstractLogger\",\n
            \"threadId\" : 53,\n
            \"threadPriority\" : 5\n}\r",
    "ibm_threadId":"00000035",
    "ibm_datetime":"2020-04-23T14:21:22.124-0400",
    "module":"SystemOut",
    "loglevel":"SystemOut",
    "ibm_methodName":"",
    "ibm_className":"",
    "ibm_sequence":"1587666082124_000000000001B",
    "ext_thread":"Default Executor-thread-8”
}

Now you can output the logs so that pre-formatted JSON logs are not wrapped in liberty_message events. Visualization tools such as Kibana can be used to analyze certain fields in your JSON logs. You can analyze both custom-formatted JSON application logs and Liberty JSON logs in the same visualization.

You can enable this functionality by setting appsWriteJson="true" in the logging element of the server.xml, or you can have it set from the moment the server starts by setting in bootstrap.properties: com.ibm.ws.logging.apps.write.json=true.

For more information: