If you are upgrading to Confluence 4.0 or later from an older version (From Confluence 3.5.x or earler) then as part of the upgrade an automatic migration of your content will take place. This is a non-destructive process. Your existing content is not overwritten. Instead, the migration process will create a new version of each wiki markup page. The new version will use the new XHTML-based storage format, so that you can edit the page in the Confluence rich text editor.

Note: Even though the process is non-destructive, you must be sure to perform a backup of your database and home directory prior to starting the new version of Confluence, as we recommend for any Confluence upgrade.

Migration process

Depending on the size of your Confluence installation, the migration from wiki markup to the new XHTML-based storage format could prove time consuming. The duration of the migration is difficult to estimate; this is due to a number of site specific factors. As a rough guide, a test dataset we migrated was 130,000 pages, totalling approximately 700Mb, which took six minutes.

The following properties that can be modified to allow finer control over the migration process:

Property

Purpose

Default

confluence.wiki.migration.threads

The number of concurrent worker threads migrating content

4

confluence.wiki.migration.batch.size

The number of items migrated in each batch of work

500

confluence.wiki.migration.versioncomment

The comment associated with the newly migrated version of each piece of content

"Migrated to Confluence 4.0"

(For instructions on setting Confluence system properties see this document.)

Again, due to the large variability in Confluence installations it is hard to give specific recommendations for the above settings. One point to note though that both increasing batch size and the number of threads (or both) will increase the peak memory required for migration. If memory is an issue then as you increase one of these settings consider decreasing the other.

Another factor to be aware of if modifying these defaults is that of the cache settings employed in your site. The migration will quickly populate certain Confluence caches so be sure that if you have customised caches as described here that there is enough memory on the server for these caches should they reach maximum capacity.

Watching the migration logs during the upgrade

To monitor the progress of a site migration you should watch the output in the application log.

Typical logging progress will be shown by multiple log entries at the INFO level of the following format:

There may be a wide array of messages logged from each individual page but any errors are also collected for display in a single migration report once all content has been processed. Here is a typical example of such a report:

Each entry in the report will identify the content that caused migration exceptions as well as displaying the exceptions themselves.

In almost all cases any content reported as errored will have been migrated to the new XHTML-based storage format, but will actually consist of wiki markup content wrapped within an XML 'unmigrated-wiki-markup' macro. This content will still be viewable in Confluence and editable within the new Confluence Editor.

However, in some cases a batch of content may actually have completely failed to migrated. This is most typically due to an unhandled exception causing a database transaction rollback. This would be reported in the log with a message like this:

Unable to start up Confluence. Fatal error during startup sequence: confluence.lifecycle.core:pluginframeworkdependentupgrades (Run all the upgrades that require the plugin framework to be available) - com.atlassian.confluence.content.render.xhtml.migration.exceptions.MigrationException: java.util.concurrent.ExecutionException: org.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only

Confluence provides no further report about this scenario and will also allow Confluence to restart as normal without retrying a migration. If a user tries to view any such unmigrated content they will see an exception similar to this:

java.lang.UnsupportedOperationException: The body of this ContentEntityObject ('Page Title') was 'WIKI' but was expected to be 'XHTML'

The solution is to ensure you manually re-run the site migration after the restart.

Re-running the migration – for content that completely failed the migration

A Confluence Administrator can restart the site migration if there was any content that failed migration (see previous section). Only the content that is still formatted in wiki markup will be migrated, so typically a re-migration will take less time than the original migration.

To manually re-run migration:

Open this URL in your browser: <Confluence Address>/admin/force-upgrade.action

Select wikiToXhtmlMigrationUpgradeTask in the Upgrade task to run dropdown list.

Choose Force Upgrade.

Re-attempting the migration – for content in 'unmigrated-wiki-markup' macro

The previous section was about dealing with the exceptional circumstance where certain content was left completely unmigrated. The most common migration problem is that the content was migrated but remains formatted as wiki markup on the page, within the body of an 'unmigrated-wiki-markup' macro. Any content which is referenced in the migration report will be found in this state. This content is still viewable and editable but since it is wiki markup it cannot be edited using the full feature set of the rich text editor.

The most common reason for content to be in this state is that the page contains an unknown macro, or a macro that is not compatible with Confluence 4.x.

Regardless of the solution you choose, you can then force a re-migration of all the content (including content in templates) that was left wrapped in an 'unmigrated-wiki-markup' macro. This feature is found at <Confluence Address>/admin/unmigratedcontent.action

Notes

We refer to the Confluence storage format as 'XHTML-based'. To be correct, we should call it XML, because the Confluence storage format does not comply with the XHTML definition. In particular, Confluence includes custom elements for macros and more. We're using the term 'XHTML-based' to indicate that there is a large proportion of HTML in the storage format.

"If you are able to view the log output for confluence, you could set com.atlassian.confluence.content.render.xhtml.migration to ALL under <BASE URL>/admin/viewlog4j.action and then run the migration from <BASE URL>/admin/unmigratedwikicontent.action . The log will show you the page title, which is then somewhat easy to search for though Confluence. If you still have the old log files from when you first upgraded to 4.0, it includes the page ID along with the macro error, which is more useful for quick access to the page." - https://answers.atlassian.com/questions/15230/confluence-4-0-finding-pages-that-have-content-with-incompatible-upgraded-macros

That should return all the pages with the wiki markup wrapper (replace with the right macro name, if that isn't it). You can also search for this string in the BODYCONTENT table using SQL (using the right XHTML syntax of course, like '%<ac:macro...%').

Using the feedback from the above ticket, I ran these queries against my database, which showed the culprits:

SELECT * FROM CONTENT WHERE CONTENTID IN (SELECT CONTENTID FROM bodycontent WHERE BODYTYPEID = 0) AND PREVVER IS NULL AND CONTENTTYPE IN ('PAGE','BLOGPOST');SELECT * FROM CONTENT WHERE CONTENTID IN (SELECT CONTENTID FROM bodycontent WHERE BODY LIKE '%unmigrated-wiki-markup%') AND PREVVER IS NULL AND CONTENTTYPE IN ('PAGE','BLOGPOST');

The second queries yielded a number of contentids. We run PostgreSQL, so I changed the query slightly to create a clickable HTML file by running this command (adjust the code to suit your site):

Nevertheless viewing previous versions of a page throw a java.lang.UnsupportedOperationException: The body of this ContentEntityObject ('xyz') was 'BodyType:WIKI' but was expected to be 'BodyType:XHTML'. This exception is caused by every previous page view. How can I migrate the whole version history? Has anybody else such a problem?

Some macros used in Confluence 3 wiki markup are now, arguably, obsolete; they could be replaced by equivalent native Confluence 4 XML.

For example, if, in Confluence 3, you used the {table} macro and its related macros ({tr}, {td} etc.) from the Adaptavist Content Formatting Macros plugin, you might now wish to use the native (XHTML-based) Confluence XML table markup, and the corresponding support in the Confluence 4 rich text editor for editing tables, rather than editing multiple nested macro placeholders for {table} et al.

My Confluence site is small, and we mostly kept to simple wiki markup for tables in Confluence 3, so this is not a problem that I am personally facing.

But I do wonder: is this issue a pain for anyone out there? I hate to think that there are Confluence 4 users who are, in the rich text editor, editing tables that are represented by table macro placeholders, when they could be editing "native" tables.

It occurs to me that I could relatively quickly develop an XSLT stylesheet to transform the <ac:macro> XML elements for {table} et al into the "native" XHTML-based <table> markup. And then perhaps wrap that XSLT stylesheet in some rudimentary user interface to make it easy to use. Or maybe there's already an existing solution to this? (I haven't looked very hard; like I say, it's not a problem that I am personally facing. I really am just responding to the unhappy thought I had that people might be editing tables as multiple nested macro placeholders in the Confluence 4 rich text editor.)

I'm not quite sure how to measure this demand. Perhaps, send me an email? (I won't quote my address here, but I'm not difficult to find.) Or - it's just an idea - maybe "like" this comment? (A dozen likes would be enough to prompt me into action.)

To clarify this offer: you can already convert "macro placeholder"-style table markup (described above) to "native" XHTML <table> markup simply by copying'n'pasting the rendered HTML (what you see when you view the page, rather than edit it) back into the rich text editor. But that method is only practical for piecemeal conversion (one table, or one page, at a time). You could use an XSLT stylesheet as the basis of a "batch" conversion process.

Is it possible to do something similar but with table-plus tables which have been dumped into wiki-markup macros after upgrading? We have squillions and our users aren't comfy using the markup (crazy folks!) to edit the tables. It has been suggested that we attempt to re-migrate but that seems awfully risky if a script could do it?

Bob Swift would be the best person to answer this, but I'll have a go.

No and yes.

No: it is not possible (okay, almost anything is possible; I guess I really mean "it is not feasible") to use what I offered (an XSLT stylesheet) to convert table-plus macros into "native" Confluence 4+ XML markup (with the tables in XHTML-like <table> markup rather than Confluence 3.x wiki markup). XSLT is not an ideal language for converting wiki markup into XML. (I'm not saying it's impossible, but there are better languages than XSLT for string manipulation.)

Yes: I can at least offer some ideas for converting table-plus tables into "native" <table> markup. But the initial ideas that I'm going to present here are based on using the Confluence editor, not an automated conversion script. We can discuss the idea of automated scripts later, if you like.

The table-plus macro offers features that are currently not available in Confluence 4 tables. Depending on the table-plus features that you use, you might decide - perhaps on a case-by-case basis - whether you want to keep using the table-plus macro (inside the "Wiki Markup" macro, <ac:macro ac:name="unmigrated-wiki-markup">) or discard the table-plus macro, and migrate its contents to native Confluence 4+ XML markup.

For example, suppose you have a page that contains the following Wiki Markup macro (containing a table-plus macro). Here, I am deliberately showing the XML markup:

You might decide that you can live without the features introduced by table-plus (such as auto total); you just want the table to be editable as a "native" table in Confluence.

To do that:

Select (click) the Wiki Markup macro in the editor (exactly how the editor indicates that the macro is selected might depend on your browser, but that's another story).

Copy the selection to the clipboard (in Windows, press Ctrl+C).

Select Insert > Wiki Markup (or press Shift+Ctrl+D).

Paste (press Ctrl+V) the contents of the clipboard (the table-plus macro) into the Wiki Markup dialog.

Delete the table-plus macro start and end tags, leaving only the table wiki markup.

Click Insert.

The table appears as a "native" Confluence table, replacing the Wiki Markup macro.

I know that this is not a good solution for converting squillions of table-plus macros, but it is a relatively quick procedure if you decide, while editing a page, that you can live without a particular instance of a table-plus macro.

Another possible solution might involve asking Bob to develop a native Confluence 4 version of the table-plus macro; that is, a wrapper for native Confluence 4 table markup (editable using the Confluence 4 editor). To which Bob (not wishing to put words in his mouth) might reasonably respond, "How much are you willing to pay?"

Regarding migrations: these occur on a regular basis automatically by Confluence whenever it sees new macros installed and un-migrated macros. There is nothing you normally need to do except perhaps monitor for migration errors in the log and/or restart Confluence to get other migrations going. You also need to ensure the xhtml enabled macros are enabled of course. In some cases they can be disabled on install. In UPM, expand out the modules associated with the plugin and make sure they are enabled. For my plugins, 3.x compatible modules have -legacy in the name, the rest are xhtml (Confluence 4.x) macros.

However, what I failed to specify in my original question is that I'm wanting to migrate the "wiki-markup" table-plus tables to proper table-plus tables, for some reason they haven't migrated correctly!

Anonymous

There is no way of doing this at the moment. However it would be possible to implement it. Could you explain a little about your full use case e.g. have you tried to fix something on the individual page and now you want to see if it is "migrate-able"?

I'm in the process of upgrading from 3.2 to 5.3. I upgraded from 3.2 to 3.5 to 4.3 to 5.3.

I now have 233 pages with unmigrated-wiki content. Unfortunately there is no /admin/unmigratedwikicontent.action in 5.3, I guess. Is there any way to update content with incompatible macros in 5.3? Otherwise I'd have to do the update to 4.3 once again, update all the plugins in 4.3, do the update content, update to 5.3 and update all the plugins again. Not something to look forward to

Hi Sergei, you can find information on the best upgrade path from Confluence 3.1 on this page Upgrading Confluence. If you have questions or concerns you can also contact Support for assistance in planning your upgrade.