Elastic Agent stuck at pending

Our Windows elastic agents get stuck at "Pending". I've looked through the various related questions on this site and none of the solutions work for us. When I left work on a Friday, everything seemed to be working fine (and had been for a couple of weeks) but on Monday everything is hosed up. I can ping our bamboo url fine and the Bamboo user is not set to have an expiring password.

Log

2015-08-31 16:44:23,585 FATAL [Thread-1] [AgentBootstrap] Exiting due to fatal exception.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:28)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'buildAgentController' defined in class path resource [applicationContextRemoteAgent.xml]: Cannot resolve reference to bean 'agentQueueAccessor' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentQueueAccessor' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot resolve reference to bean 'connectionFactory' while setting bean property 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:336)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:646)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at org.springframework.context.support.ClassPathXmlApplicationContext.&lt;init&gt;(ClassPathXmlApplicationContext.java:139)
at org.springframework.context.support.ClassPathXmlApplicationContext.&lt;init&gt;(ClassPathXmlApplicationContext.java:105)
at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:227)
at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:70)
at com.atlassian.bamboo.agent.elastic.client.ElasticAgent.start(ElasticAgent.java:31)
... 6 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentQueueAccessor' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot resolve reference to bean 'connectionFactory' while setting bean property 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:336)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
... 25 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:290)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:646)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
... 35 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#29a220e6' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:278)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:276)
... 47 more
Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:164)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:125)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:270)
... 52 more
Caused by: java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
at com.google.common.io.Closeables.close(Closeables.java:80)
at com.google.common.io.Closeables.closeQuietly(Closeables.java:99)
at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.ping(BambooActiveMQConnectionFactory.java:191)
at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.validateUri(BambooActiveMQConnectionFactory.java:145)
at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.setBrokerURL(BambooActiveMQConnectionFactory.java:63)
at org.apache.activemq.ActiveMQConnectionFactory.&lt;init&gt;(ActiveMQConnectionFactory.java:142)
at org.apache.activemq.ActiveMQConnectionFactory.&lt;init&gt;(ActiveMQConnectionFactory.java:138)
at org.apache.activemq.ActiveMQSslConnectionFactory.&lt;init&gt;(ActiveMQSslConnectionFactory.java:75)
at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.&lt;init&gt;(BambooActiveMQConnectionFactory.java:53)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:148)
... 54 more

Looking into this further the system has Java 7 installed also. I manually installed Java 8 and am rebooting the instance. Hopefully that will fix it? I can create a new AMI and try starting it from Bamboo if that's required. If not, I'm kind of hosed. Any thoughts on what I need to do to get this custom image current?

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

Download latest version of the atlassian-bamboo-elastic-image and grabbed the updated agent code from there. Also had to manually edit the .bat file since I was getting command not found errors on java.exe even though JAVA_HOME was defined. After that, build agents are being found and running. Thanks!

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.