In the event there is an issue you would like to report to ScyllaDB support, you need to submit logs and other files which help the support team diagnose the issue. Only the ScyllaDB support team members can read the data you share.

In general, there are two types of issues:

Scylla failure - There is some kind of failure, possibly due to a connectivity issue, a timeout, or otherwise, where the Scylla server or the Scylla nodes are not working. These cases require you to send ScyllaDB support both a Health Check Report as well as Core Dump files (if available).

Scylla performance - you have noticed some type of degradation of service with Scylla reads or writes. If it is clearly a performance case and not a failure, refer to Report a performance problem.

When Scylla fails, it creates core dump which can later be used to debug the issue. The file is written to /var/lib/scylla/coredump. If there is no file in the directory, see Troubleshooting Core Dump.

In the event the /var/lib/scylla/coredump directory is empty, the following solutions may help. Note that this section only handles some of the reasons why a core dump file is not created. It should be noted that in some cases where a core dump file fails to create not because it is in the wrong location or because the system is not configured to generate core dump files, but because the failure itself created an issue where the core dump file wasn’t created or is not accessible.

If the scylla/coredump directory is empty even after you changed the custom configuration file, it might be that Automatic Bug Reporting Tool (ABRT) is running and all core dumps are pipelined directly to it.

Procedure

Check the /proc/sys/kernel/core_pattern file.
If it contains something similar to |/usr/libexec/abrt-hook-ccpp%s%c%p%u%g%t%h%e636f726500 replace the contents with core.

If you are experiencing a performance issue when using Scylla, let us know and we can help. To save time and increase the likelihood of a speedy solution, it is important to supply us with as much information as possible.

There are two types of metrics you need to collect: Scylla Server and Scylla Client(node). The Scylla Server metrics can be displayed using an external monitoring service like Grafana or they can be collected using scyllatop and other commands.

Note

It is highly recommended to use the Scylla monitoring stack so that the Prometheus metrics collected can be shared.

There are several commands you can use to see if there is a performance issue on the Scylla Server. Note that checking the CPU load using top is not a good metric for checking Scylla.
Use scyllatop instead.

Note

To help ScyllaDB support assess your problem, it is best to pipe the results to a file which you can attach with the Health Check report.

Check the SendfilestoScyllaDBsupportgauge-load. If the load is close to 100%, the bottleneck is Scylla CPU.

$ scyllatop *gauge-load

Check if one of Scylla core is busier than the others:

$ sar -P ALL

Check the load on one CPU (0 in this example)

$ perf top -C0

Check if the disk utilization percentage is close to 100%. If yes, the disk might be the bottleneck.

$ ostat -x 1`` to observe the disk utilization.

Collect run time statistics.

$ sudo perf record --call-graph dwarf -C 0 -F 99 -p $(ps -C scylla -o pid --no-headers) -g sleep 10
Alternatively, you can run the ``sudo ./collect-runtime-info.sh```source <https://gist.github.com/tzach/b2cf347a198e63939643a1af910b036e/>`_ which does all of the above, except scyllatop and upload the compressed result to s3. You can also see the results in `./report` dir

If you have not done so already, supply ScyllaDB support with the UUID. Keep in mind that although the ID you supply is public, only ScyllaDB support team members can read the data you share. In the ticket/issue you open, list the documents you have uploaded.

Procedure

Do one of the following:

If you are a Scylla customer, open a Support Ticket and include the UUID within the ticket.

If you are a Scylla user, open an issue on GitHub and include the UUID within the issue.