Configure Kubernetes log collection on CRD

Log collection is configured on the console by default. Log Service also provides CRD configuration for log collection for Kubernetes microservice development. This allows you to use kubectl to manage configurations.

We recommend you use the CRD method for collection configuration management, as this method is better integrated with the Kubernetes deployment and publishing process.

Implementation principles

Figure 1. Implementation principles

Run the installation command to install the alibaba-log-controller Helm package. The Helm package mainly run the following operations:

Create aliyunlogconfigs CRD (Custom Resource Definition).

Deploy alibaba-log-controller.

Deploy Logtail DaemonSet.

The internal workflow of configuration is as follows:

Use kubectl or other tools to apply the aliyunlogconfigs CRD configuration.

alibaba-log-controller detects configuration update.

alibaba-log-controller automatically submits requests for Logstore creation, configuration creation, and configuration application to machine groups based on the CRD content and server status.

Logtail running in DaemonSet mode periodically sends requests for server configuration, obtains the new or updated configuration, and performs the rapid loading.

Logtail collects standard outputs or files from each container (pod) based on the configuration information.

Logtail sends processed and aggregated data to the Log Service.

Configuration method

Note If you have used the Logtail deployed in DaemonSet mode, you cannot manage configurations in CRD mode. For more information, see
Migration process for the DaemonSet deployment mode in this document.

You must define the CRD of AliyunLogConfig to create configurations, and delete the corresponding CRD resource to delete the configuration. The CRD is configured as follows:

apiVersion: log.alibabacloud.com/v1alpha1 ## Default value, no need for change
kind: AliyunLogConfig ## Default value, no need for change
metadata:
name: simple-stdout-example ## Resource name, which must be unique in the cluster
spec:
logstore: k8s-stdout ## Logstore name, automatically created if no name exists
shardCount: 2 ## [Optional] Number of Logstore shards. The default value is 2. The value range is 1 to 10.
lifeCycle: 90 ## [Optional] Storage period of the Logstore. The default value is 90. The value range is 1 to 7300. The value 7300 indicates permanent storage.
logtailConfig: ## Detailed configuration
inputType: plugin ## Input type of collection. Generally, the value is file or plugin.
configName: simple-stdout-example ## Collection configuration name. The value must the same as the resource name (metadata.name).
inputDetail: ## Detailed configuration information, see the example
...

After the configuration is completed and applied, alibaba-log-controller is created automatically.

Run kubectl get aliyunlogconfigs ${config_name} -o yaml to view the detailed configuration and status.

The status field in the configuration shows the configuration execution result. If the configuration is successfully applied, the value of statusCode is 200 in the status field. If the value of statusCode is not 200, applying the configuration failed.

Configuration example

Container standard output

In the container standard output, set inputType to plugin and fill the detailed information in the plugin field under inputDetail. For more information on the configuration fields, see Containers-standard output.

Simple collection mode

Collect standard outputs (stdout and stdeer) of all containers except for those who has environment variable configuration COLLECT_STDOUT_FLAG=false.

Collect the access log of Grafana and parse the access log into structured data.

Grafana container has environment variable configuration
GF_INSTALL_PLUGINS=grafana-piechart-..... You can set
IncludeEnv to
GF_INSTALL_PLUGINS: '' to enable the Logtail to collect standard outputs from this container only.
Figure 2. Custom collection mode

A log entry may be divided into multiple lines because the log contains error stacking information. Therefore, you must set a regular expression for the beginning of a line. To extract each field, use a regular expression. The detailed configuration is as follows: