Following upgrade to 7.6 DVCS synch appears frozen

Stopped getting BitBucket updates right after the upgrade which seemingly went ok. DVCS will not sync although it appears loaded.

In the atlasian.jira.log we have a message as per the below

2017-11-28 18:57:24,457 WaitForAO ERROR [c.a.j.p.dvcs.scheduler.DvcsScheduler] Exception waiting for AO to initializecom.querydsl.core.QueryException: Caught SQLServerException for select "AO_E8B6CC_MESSAGE"."ID", "AO_E8B6CC_MESSAGE"."ADDRESS", "AO_E8B6CC_MESSAGE"."PAYLOAD", "AO_E8B6CC_MESSAGE"."PAYLOAD_TYPE", "AO_E8B6CC_MESSAGE"."PRIORITY", max("AO_E8B6CC_MESSAGE_QUEUE_ITEM"."RETRIES_COUNT") as "RETRIES_COUNT" from "jiraschema"."AO_E8B6CC_MESSAGE" "AO_E8B6CC_MESSAGE" left join "jiraschema"."AO_E8B6CC_MESSAGE_QUEUE_ITEM" "AO_E8B6CC_MESSAGE_QUEUE_ITEM" on "AO_E8B6CC_MESSAGE"."ID" = "AO_E8B6CC_MESSAGE_QUEUE_ITEM"."MESSAGE_ID" where "AO_E8B6CC_MESSAGE"."ID" = ? group by "AO_E8B6CC_MESSAGE"."ID", "AO_E8B6CC_MESSAGE"."ADDRESS", "AO_E8B6CC_MESSAGE"."PAYLOAD", "AO_E8B6CC_MESSAGE"."PAYLOAD_TYPE", "AO_E8B6CC_MESSAGE"."PRIORITY" at com.querydsl.sql.DefaultSQLExceptionTranslator.translate(DefaultSQLExceptionTranslator.java:50) at com.querydsl.sql.Configuration.translate(Configuration.java:453) at com.querydsl.sql.AbstractSQLQuery.fetch(AbstractSQLQuery.java:502) at com.atlassian.jira.plugins.dvcs.dao.impl.querydsl.MessageDaoQueryDsl.lambda$getById$8(MessageDaoQueryDsl.java:196) at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:66) at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) ... 3 filtered at java.lang.reflect.Method.invoke(Method.java:498) at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) at com.sun.proxy.$Proxy273.runInManagedTransaction(Unknown Source) ... 3 filtered at java.lang.reflect.Method.invoke(Method.java:498) at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) at com.sun.proxy.$Proxy273.runInManagedTransaction(Unknown Source) at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:64) at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:40) at com.atlassian.jira.plugins.dvcs.dao.impl.querydsl.MessageDaoQueryDsl.getById(MessageDaoQueryDsl.java:180) at com.atlassian.jira.plugins.dvcs.dao.impl.querydsl.MessageQueueItemDaoQueryDsl.getByQueueAndMessage(MessageQueueItemDaoQueryDsl.java:180) at com.atlassian.jira.plugins.dvcs.service.MessagingServiceImpl.fail(MessagingServiceImpl.java:346) at com.atlassian.jira.plugins.dvcs.service.MessagingServiceImpl.lambda$initRunningToFail$0(MessagingServiceImpl.java:176) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at com.atlassian.jira.plugins.dvcs.dao.impl.querydsl.util.PseudoStream.consumeAllInTable(PseudoStream.java:50) at com.atlassian.jira.plugins.dvcs.dao.impl.querydsl.MessageQueueItemDaoQueryDsl.getByState(MessageQueueItemDaoQueryDsl.java:233) at com.atlassian.jira.plugins.dvcs.service.MessagingServiceImpl.initRunningToFail(MessagingServiceImpl.java:169) at com.atlassian.jira.plugins.dvcs.service.MessagingServiceImpl.onStart(MessagingServiceImpl.java:498) at com.atlassian.jira.plugins.dvcs.scheduler.DvcsScheduler.lambda$onStart$0(DvcsScheduler.java:68) at java.lang.Thread.run(Thread.java:745)Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The text, ntext, and image data types cannot be compared or sorted, except when using IS NULL or LIKE operator. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258) at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:467) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:409) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199) at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:331) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83) at com.querydsl.sql.AbstractSQLQuery.fetch(AbstractSQLQuery.java:446) ... 41 more

Running the same query as above in the SQL server (same SQL server and instance as before the update), it appears that the query will not execute because it's trying to order by ntext fields ("AO_E8B6CC_MESSAGE"."PAYLOAD"). This is inline with the error shown in the log.

Seemingly we have the wrong statement issued to the SQL server. Not sure if this is a bug or something went wrong in the upgrade. Any ideas/suggestions?

No ETA at this moment, as we're still doing the initial investigation, but it has been escalated internally.

I don't have a public link to provide you but as many of these teams are either in Sydney or the US I will be able to update you tomorrow at least to say whether or not we have found more details on the case, if not a fix.

Shannon

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

To avoid--------Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The text, ntext, and image data types cannot be compared or sorted, except when using IS NULL or LIKE operator----------works for me