Screenshots

Anonymous (login to see details)

Wolfgang, is a separate license required to create this automated test dashlet? When I create a dashboard with the 'Test Automation' dashlet, I get an error that says my license does not support analyzing 'automated tests'. There was no mention of requiring a separate license for this during the HOT workshop in Orlando?

Anonymous (login to see details)

Anonymous (login to see details)

One more question, is there a minimum version of Jenkins that will support this plugin? We have v1.508 running in our development environment right now but it seems the upload didn't seem to work. However, I ran an install on my local 1.535 and that seemed to be ok.

~~Edited

Ah, nevermind, had to reboot the Jenkins server to see it. Working now.

Anonymous (login to see details)

Hi Wolfgang,

I am running the same build multiple times to see what kind of output I get with the Jenkins plugin and each time I am getting a different value for the # of tests passed. All I am doing is running the same build over and over again. Shouldn't the # passed be the same each time?

Anonymous (login to see details)

Hi,

These numbers should reflect the number of passed tests per build - the plugin gets these numbers from the test automation dashlet. Could you look at one of the failed tests from #36 or #34 and see what the test automation dashlet looks like for this test?

Anonymous (login to see details)

Hi Wolfgang,

That's the thing, there are no failed tests. All the tests pass (it's the same build I keep running over and over with no changes). The Test Automation dashlet does show all the tests and they are all passed. Not sure why only a subset gets returned to the plugin.

Anonymous (login to see details)

I just got this dashboard to work...great stuff!! Just a couple of comments / questions:

Is it possible to suppress the logging? It's dumping the entire report and results, which is quite large for us.

I didn't see it mentioned here, but i had to set up the dashlet properties in dynaTrace to allow for many more results. We have much more than the default 100 tests that return in the default report. Also, setting the timeframe for longer than the test run was helpful. Not a problem, just an issue more people might run into.

Is there any way to add this to a view? We're looking for a landing page in Jenkins that shows all of our critical build information such as Coverage, Test Runs, and we'd like to add the dynaTrace graph too.

Again, great work...this will be a heavily used plugin for us! Thanks!

Anonymous (login to see details)

Hi John,

Sure, I will include a switch to control this behaviour

Thanks for your input, I will include this in the instructions. How long are your tests running? In any case, this will get better with dT 5.6 when the test automation data will be available directly through the API without the detour via the dashboard report.

That sounds like a great idea, I'll have to check what I have to change in the plugin to support this.

Anonymous (login to see details)

Wolfgang-

Thanks for the reply...I look forward to dT 5.6 and the improvements that will be possible due to it. As for #2, our tests are currently running for about 2 minutes, but most are very lightweight. We currently have 250+ tests, and we're just starting to get into 'full productivity' mode, so we're adding 15 - 20 a day at minimum. Not a problem after I realized i had to change the dashlet properties...just took me a bit to figure that out.

Anonymous (login to see details)

FYI, I've just had to disable this plugin from our Jenkins installation. Two reasons:

The inability to skip logging is quite annoying, and many of our developers complain. Minor issue.

I found that our build.xml is ~467k, and the dynaTrace Results are ~456k of that. Under heavy load, it was taking a long time to render simple JUnit reporting graphs (among other things), because it would have to read this entire file. This effectively caused our Jenkins server to crash periodically.

Looking at my last post, we had ~250 tests, and now we're up to 550. So it appears that this plugin just doesn't scale with our current growth. I would suggest a switch to turn off logging (as you suggested before), but also moving the test results out of the main build.xml file. This is the technique that most other plugins use (TestResult, Cobertura, Jacoco, etc...).

Anonymous (login to see details)

Hi John,

Thanks again for your feedback! I have uploaded a new version yesterday that finally contains the switch to suppress logging. Do you think you could give that a try?

On your second remark, I will definitely improve the plugin to use a separate storage file. I didn't expect this slowdown so far with my sample tests, but I understand your concern. I will post update this page once it's done

Anonymous (login to see details)

Hi Wolfgang-

Thanks again for addressing the issues. I'm a bit wary of trying the new plugin until the 2nd issue is addressed, as we couldn't even bring Jenkins up for several hours. This is a bit too disruptive for us, but I'd be glad to try after it is addressed.

Anonymous (login to see details)

Oh, under these circumstances I can totally understand your wariness If I hear you correctly you say that the ~500k file causes Jenkins to stall? That's a behaviour I didn't experience so far...Sorry about that, I'll work on it!

Anonymous (login to see details)

Wolfgang-

Yes, it caused Jenkins to stall. What we saw was that any request to the job's page would read in build.xml and cause a HUGE stack. Also, Jenkins will read these files in a synchronized block, so we would have requests start piling up. So if it took 5 - 10s (un verified times), it would do this on 10 - 20 threads (depending on how many people were accessing our Jenkins Dashboard). This caused Jenkins to get in a state where it was simply unrecoverable.

I actually ended up having to delete our job history (which isn't that big of a deal).

Anonymous (login to see details)

Hi all,

We are recently using this plugin for Jenkins version 1.535. After installation and machine reboot, it shows in the plugin is successfully installed. However, "dynaTrace test automation" can't be found in "Post-Build Action". (I created a dummy job by clicking New Job->Build a free-style software project)

Anonymous (login to see details)

Hi,

You have to use the Jenkins BUILD_ID for the versionBuild parameter passed to dynaTrace in the set test information call. Are you doing that at the moment? Maybe you can share your pom.xml (either here or via mail - wolfgang.gottesheim@compuware.com)?

Anonymous (login to see details)

Hi,

Good to hear

@1: You can, absolutely.

@2: I guess that depends a bit on your module structure. My first thought would be to put the dynaTrace dependency in the parent pom and define a profile for (unit/performance/integration...) tests that attaches the dynaTrace goals to the proper lifecycle phases. Could you share a bit more about your projects?

@3: Interesting question, I haven't tried that so far. You have multiple Jenkins servers building the same project with agents connected to the same dT server? If they map into different system profiles I wouldn't see too much problems (you'd need to create multiple dashboards pointing to the respective profile)

Anonymous (login to see details)

I think I found the reason why I was getting the exception. One thing I also did is that I had to remove the dynaTrace:dtAutomation:3.5:setTestInformation goal in order for my test to be categorized as "UI-Driven". Prior to the mvn command, I invoke an HTTP POST to the REST services on the dynaTrace server before I invoke my mvn command:

Anonymous (login to see details)

Anonymous (login to see details)

hi,

i am getting the following error message while executing ANT target.

============

SetTestInformationForTestAutomation:
BUILD FAILED C:\Program Files (x86)\Jenkins\jobs\Jmeter_TeamA_AddUser\workspace\build.xml:42: Problem: failed to create task or type DtSetTestInformation
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations have taken place.

Anonymous (login to see details)

I would like to explain you what I'm trying to achieve, so maybe you can tell me if it is possible or not with easyTravel application

My final goal is being able to show our clients how Dynatrace can close the full development cicle (development, build, unit tests, and integration tests).

In order to accomplish this goal, I have planned the next milestones:

- Import and compile easyTravel code in an eclipse project.

- Run an existing test file and see the results in Dynatrace.

- Make a change in code (e.g.: any change that increment the DB calls) and then run again the test to be able to compare, in Dynatrace, the differences between these two executions and show the client the benefit of drilling into details like this with Dynatrace.

- Integrate the process for building and running Ant tests with Jenkins, taking advantage of Dynatrace plugin for Jenkins.

- Finally, when pushing any change in code to a Git repo, automatize a build+test run task in Jenkins.

Anonymous (login to see details)

Anonymous (login to see details)

hi Rick

i have tried to options mentioned in that link.. I am able to move forward and with my ANT script i can now create a session, stop it and generate report. but i am not able to display Dynatrace metrics in Jenkins..

Will the plugin mentioned in this post will work only with "Test Automation dashlet " ?

Anonymous (login to see details)

Any specific error you are getting which you can share? I have a hunch you just need to save the Test Automation dashlet in a dashboard and reference this dashboard name in the configuration of the plugin in Jenkins

Anonymous (login to see details)

Hi all,

I have been working with Wolfgang on some of the challenges I have had getting the Jenkins plugin and the test Automation reports to show up properly.

Has anyone else on this forum been able to get the reports to show up properly for Browser agent tests? I have tried and tried, but can never get it to work. I have been able to get JUnit tests to show up ok, but never for Browser agent tests.

Anonymous (login to see details)

Hi Kristof,

I thought I already replied here (or via email?), but for future reference - TeamCity is not supported by this plugin. However, it's definitely something I'd like to cover in the future as well, but currently Bamboo is higher up in the list...

Anonymous (login to see details)

Anonymous (login to see details)

Wolfgang,

No, we have one jenkins project that contains many maven modules.

Each module contains tests, therefore, the surefire plugin is invoked many time.

Each time with a different build number. At the end of the build, the DT jenkins plugin displays an arbitrary number of tests run. For instance, if you have 147 total, the DT plugin will display sometime 79, sometime 30 sometime 120 etc...

We noticed by looking at the XML report that the build number is different for each module...

Anonymous (login to see details)

Hi,

did you find a solution for this issue? I'm facing the same. The Dynatrace client is showing > 2600 tests, but the Jenkins dashboard shows randomly ca. 800 or only 30, depending on which ID it accesses it on the server.

I tried setting the dtTestrunId manually as Maven property like {{<dtTestrunID>${BUILD_TAG}</dtTestrunID>}}, but then the Jenkins dashboard completely failed to get the XML report and the version in Dynatrace was "-.-.-.1 (autogenerated)".

would be great if we could solve this issue, otherwise the Jenkins integration is not really helpful...

Anonymous (login to see details)

Anonymous (login to see details)

Hi,

As far as I know the interfaces used haven't changed since 1.509, although I have to admit I haven't tried it myself. But the plugin itself is built against a certain version (1.535), so I'm not sure if you can actually install it on an older build. Probably the easiest way is to try...

Anonymous (login to see details)

Anonymous (login to see details)

I'm working with the new 2.x version of the plugin. I'm using a NAnt script for the test. It is able to start the test, talk to the DT server and start/stop recording. After I run the test, I see the data in the DT client. The problem comes when Jenkins makes a rest call to pull back the test information. I noticed that when the test runs it has one id, but when Jenkins makes the call it has a second id. That means it pulls back no data. In the plugin doc, it talks about passing the id to the agent, but only for java. Is there a way I should be passing the the id to the .Net agent config?

Anonymous (login to see details)

The test id must be passed to the .net agent through the environment variable DT_TESTRUN_ID (see also .NET Agent Configuration)Feel free to do a push request to update the documentation to include a snipplet for NAnt once you have it running. I'm sure it will be useful to others.

Anonymous (login to see details)

Hi! I filed an issue here per the header in the github repo, but didn't get a response. I'd like to do the refactoring work necessary to get the plugin manageable by Jenkins Pipeline script. Where should I branch from?

Anonymous (login to see details)

I am using test centre edition and got some issue in terms of fetching record from dynatrace, it used to work for sometime and now I started getting below error of retry in jenkins console. I am able to see Testin appmon, and my header is also injected correctly. Please let me know how to debug and find my jenkins is not getting data from dynatrace appMon and display in jenkins ui. I am using jenkins 1.651.3 and my dynatrace plugin version is latest for jenkins.