2018-10-01 05:50:31 UTC SEVERE [<default,0x1>] [PeriodicServiceWorkerManager] Tenant: xxxxxxxxxxxxxxxxxxxxxxxxxxx Periodic worker: [node] updates worker encountered an exception. Will continue to run (throttled for 1 hour):
java.lang.NullPointerException
at com.compuware.apm.server.onprem.rest.management.api.updates.impl.UpdateBatchManagementUtilServiceImpl.getBatchIdentifier(UpdateBatchManagementUtilServiceImpl.java:210)
at com.compuware.apm.server.onprem.updates.UpdatesManagementServiceImpl.matchesWithLocalBatch(UpdatesManagementServiceImpl.java:254)
at com.compuware.apm.server.onprem.updates.UpdatesManagementServiceImpl.lambda$synchronizeWithUpdatesEntryList$3(UpdatesManagementServiceImpl.java:82)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at com.compuware.apm.server.onprem.updates.UpdatesManagementServiceImpl.synchronizeWithUpdatesEntryList(UpdatesManagementServiceImpl.java:83)
at com.compuware.apm.server.onprem.updates.UpdatesWorker.run(UpdatesWorker.java:118)
at com.compuware.apm.platform.PeriodicServiceWorkerManager.run(PeriodicServiceWorkerManager.java:130)
at com.compuware.apm.util.api.threadpool.RunnableThrowableFilterDecorator.run(RunnableThrowableFilterDecorator.java:48)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at com.compuware.apm.util.api.threadpool.AccessibleRunnableScheduledFuture.run(AccessibleRunnableScheduledFuture.java:120)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Attachments:
Up to 10 attachments (including images) can be used with a maximum of 52.4 MB each and 262.1 MB total.

Answer by
Paulina R.·
Oct 02, 2018 at 04:35 PM

Then the easiest way to upgrade this cluster with that bug in place would probably be manually executing installation files on each node. If you have already uploaded them via UI before, they're located by default in "/opt/dynatrace-managed/installer/upgrade" (could be different if you're using custom paths). But you can unpack your server installation package and run the file anywhere on the machine. The file you need to execute will be named "dynatrace-managed-1.142.xxx.sh". No parameters are necessary, as this is an upgrade and will use existing configuration. After you successfully upgrade each server node to 142, you will need to do the same for version 144 and 146.

Starting from 146, you should be able to use the UI again, because the bug from before will be fixed. You can then upload all remaining installation packages (148, 150 and 152) on the Automatic Update page and trigger upgrades with a button on that page. I hope this helps.

Could be because 1.146 installation package is not valid for 1.140 (you cannot skip 142 and 144) and this is a pretty old version, which maybe didn't handle such situations well. In newer versions this problem should not occur (e.g. you can upload 148, 150 and 152 while in version 146)

As for agents etc, their version cannot be higher than server version, which is why they're treated as invalid, too

This timeframe is for automatic upgrades, normally you can also trigger the upgrade manually at any time by clicking the "Upgrade now" button on the Automatic Update page. However, this page is currently not working because of the bug... And this bug may also prevent automatic upgrades from starting.

There is a couple of things we can do, but first can you please tell me if this is an online cluster (connected to Mission Control, automatically downloads update files) or an offline cluster (not connected, update files are uploaded manually)? It's important, because this bug could be caused by update files being only partially uploaded (e.g. copying .sh files without corresponding .config files). Also, do you have access to machines with Dynatrace Managed installed (or in other words, is executing installation files manually an option?)

Answer by
Paulina R.·
Oct 02, 2018 at 03:07 PM

Hi, which version of Dynatrace Managed are you using? There was a bug which produced exactly the same error log, but it was fixed in 1.146