This plugin provides a generic framework that supports editing of topics using any browser-based HTML editor. It works by transforming TML into HTML for the editor, and then transforming HTML back into TML on save.

Features

Details

What's in the package

The package includes the following pieces:

TML to HTML translator

HTML to TML translator (with stand-alone script)

Generic plugin for automating the translation during editing

How it works

The plugin works by translating the topic text into HTML when someone edits a topic. The HTML is then fed to the WYSIWYG editor. On save, the edited HTML is run through the reverse translation before saving to the topic. TML is used in preference to HTML in the stored topic wherever possible, though HTML may be used if the translator can't find a suitable TML equivalent.

The default rendering that Foswiki uses to generate HTML for display in browsers is 'lossy' - information in the TML is lost in the HTML output, and a round-trip (recovering the original TML from the HTML) is impossible. To solve this problem the plugin instead uses its own translation of TML to XHTML. The generated XHTML is annotated with CSS classes that support the accurate recovery of the original TML.

Before you ask the obvious question, yes, the translator could be used to replace the Foswiki rendering pipeline for generating HTML pages. In fact, the translator is taken almost directly from the implementation of the rendering pipeline for the TWiki-4 release

Translation of the HTML back to TML uses the CPAN:HTML::Parser. This parser is used in preference to a more modern XML parser, because the WYSIWYG editor may not generate fully compliant XHTML. A strict parser would risk losing content. CPAN:HTML::Parser is better at handling malformed HTML.

There is also the advantage that the translator can be used to import HTML from other sources - for example, existing web pages. Due to the simple nature of TML and the potential complexity of web pages, this translation is often lossy - i.e. there will be HTML features that can be entered by editors that will be lost in this translation step. This is especially noticeable with HTML tables.

Using the translators from Perl scripts

Both translators can be used directly from Perl scripts, for example to build your own stand-alone translators.

A stand-alone convertor script for HTML to TML is included in the installation. It can be found in tools/html2tml.pl. Run it with a --help parameter to find out how to use it i.e. perl -I bin tools/html2tml.pl --help

There is also a stand-alone translator for TML to HTML, in tools/tml2html.pl.

Integrating a HTML Editor

The plugin can be used to integrate an HTML editor in a number of different ways.

The HTML for the content-to-be-edited can be generated directly in the standard edit template.

The HTML for the content-to-be-edited can be generated directly in a specialised edit template.

A URL can be used to fetch the content-to-be-edited from the server, for use in an IFRAME.

REST handlers can be called from Javascript to convert content.

Generating content directly in the standard edit template

This is the technique used by WYSIWYG editors that can sit on top of HTML
textareas, such as TinyMCE. The topic content is pre-converted to HTML before inclusion in the standard edit template. These editors use plugins that have a beforeEditHandler and an afterEditHandler. These handlers are responsible for the conversion of topic text to HTML, and post-conversion of HTML back to TML.

Your plugin should set the textareas_hijacked context id, to signal to skins to suppress their textarea manipulation functions.

This is the recommended integration technique, if your editor can support it.

Generating content directly in a specialised edit template

This technique is useful when the editor requires the topic content in a variety of different formats at the same time. In this scenario the editor uses a custom edit template. The WYSIWYG content is made available for instantiation in that template in a number of different formats. WYSIWYGPLUGIN_WYSIWYGSKINmust be set for this to work.

The flow of control is as follows:

User hits "edit" with the skin (or cover) set the same as WYSIWYGPLUGIN_WYSIWYGSKIN.

The WysiwygPlugin beforeEditHandler determines if the topic is WYSIWYG editable, and vetos the edit if not by redirecting to the standard edit skin. the edit

The edit template containing the JS editor is instantiated.

The following macros are available for expansion in the template:

%WYSIWYG_TEXT% expands to the HTML of the content-to-be-edited. This is suitable for use in a textarea.

%JAVASCRIPT_TEXT% expands to the HTML of the content-to-be-edited in a javascript constant.

User edits and saves

The afterEditHandler in the WyswiygPlugin sees that wysiwyg_edit is set, which triggers the conversion back to TML.

The HTML form in the edit template must include an <input called wysiwyg_edit and set it to 1, to trigger the conversion from HTML back to TML.

WYSIWYGPLUGIN_WYSIWYGSKIN must be set to the name of the skin used for WYSIWYG editing. This is often the name of the editor e.g. xinha.

Fetching content from a URL

In this scenario, the edit template is generated without the content-to-be-edited. The content is retrieved from the server using a URL e.g. from an IFRAME.

The flow of control is as follows:

As Generating content directly in a specialised edit template

As Generating content directly in a specialised edit template

As Generating content directly in a specialised edit template

When the document loads in the browser, the JS editor invokes a content URL (using an IFRAME or a XmlHttpRequest) to obtain the HTML document to be edited

The content URL is just a Foswiki view URL with the wysiwyg_edit parameter set.

The WysiwygPlugin recognises the wysiwyg_edit parameter and uses the TML2HTML translator to prepare the text, which is then returned as text/plain to the browser.

Two macros, %OWEB% and %OTOPIC%, can be used in the content URL in the edit template to refer to the source topic for the content.

After edit handling is as for Generating content directly in a specialised edit template

Other techniques

Asynchronous saves

Editors can use XmlHttpRequest to perform saves, by POSTing to the Foswiki save script with the wysiwyg_edit parameter set to 1. This parameter tells the beforeSaveHandler in the WysiwygPlugin to convert the content back to TML. See CommandAndCGIScripts for details of the other parameters to the save script.

Once the save script has completed it responds with a redirect, either to an Oops page if the save failed, or to the appropriate post-save URL (usually a view). The editor must be ready to handle this redirect.

Handling Attachments

Attachment uploads can be handled by URL requests from the editor template to the Foswiki
upload script. The upload script normally redirects to the containing topic; a behaviour that you usually don't want in an editor! There are two ways to handle this:

If the uploads are done in an IFRAME or via XmlHttpRequest, then the 302 redirect at the end of the upload can simply be ignored.

You can pass noredirect to the upload script to suppress the redirect. In this case you will get a text/plain response of OK followed by a message if everything went well, or an error message if it did not.

REST handlers

If you are confident in Javascript you can use REST handlers with XmlHttpRequest to convert content from TML to HTML and back again.

Plugin Configuration Settings

Translator control

The globalpreference settingWYSIWYG_EXCLUDE can be set to make the plugin sensitive to what is in a topic, before allowing it to be edited. The comma separated list to fall back to text edit can include:

html - HTML tags (e.g. <div>, not including <br>), or

macros - simple macros (e.g. %VAR%) or

calls - macros with parameters e.g. %MACRO{...}%

pre blocks (<pre>)

HTML comments (<!-- ... -->)

script = inline HTML Script tags - default

style = inline Css style tags - default

table = inline html tables (=<table ..>. TML tables are not excluded)

If the plugin detects an excluded construct in the topic, it will refuse to allow the edit and will redirect to the default editor.

WYSIWYG_EDITABLE_CALLS - Exceptions to WYSIWYG_EXCLUDE

If you excluded calls in WYSIWYG_EXCLUDE, you can still define a subset of macros that do not block edits. this is done in the globalpreference settingWYSIWYG_EDITABLE_CALLS, which should be a list of macro names separated by vertical bars, with no spaces, e.g: * Set WYSIWYG_EDITABLE_CALLS = COMMENT|CALENDAR|INCLUDE

WYSIWYGPLUGIN_PROTECT_EXISTING_TAGS - Protect specific tags originally in the topic text

The WYSIWYGPLUGIN_PROTECT_EXISTING_TAGS preference tells the translator
that certain HTML tags which were originally in the topic text should remain as HTML tags;
the translator will not try to convert them to TML. This protects the tags
themselves, and not the contents enclosed between the <tag> and </tag>

The default setting for this preference is defined within the plugin.
It corresponds to div, span.

This feature may be disabled by setting the preference to a single comma.
This does not guarantee that HTML markup will be removed; the conversion
of HTML tags to TML markup remains subject to the other controls provided
by the WysiwygPlugin, including the WYSIWYGPLUGIN_STICKYBITS and
WYSIWYGPLUGIN_IGNOREATTRS preferences,
<sticky> blocks, <literal> blocks and the rules applied
to tables and lists.

The WYSIWYGPLUGIN_PROTECT_TAG_BLOCKS preference tells the translator
that certain HTML tag blocks which were originally in the topic text should remain as HTML blocks;
the translator will not try to convert them to TML.

The default setting for this preference is defined within the plugin.
It corresponds to script, style.

As an example, individual html tables can be protected by surrounding them
with =<sticky> .. </sticky> block. However,if you want to have all
=<table> markup preserved as entered into topics by default, rather than subject to
WYSIWYG editing, add table to this list, and =<table> markup will become
automatically sticky.

This feature may be disabled by setting the preference to a single comma.

WYSIWYGPLUGIN_STICKYBITS - Protect tags based upon their arguments

You can define the global preference WYSIWYGPLUGIN_STICKYBITS to stop the
plugin from ever trying to convert specific HTML tags into
TML when certain specific attributes are present on the tag. This is most
useful when you have styling or alignment information in tags that must be
preserved.

This preference setting is used to tell the translator which attributes, when present
on a tag, make it "stick" i.e. block conversion back to TML.

For example, setting it to
table=background,lang;tr=valign will stop the translator from trying to
convert any table tag that has background or lang attributes, and any
tr tag that has a valign attribute back to Foswiki | table | column |
markup (regardless of where that table tag comes from).

This setting is used only after the page has been processed by the editor. If
the editor does not support a particular tag or attribute and the editor corrupts the
tag, this setting will not be helpful. It is only used to prevent
an HTML tag from being converted back to TML.

Format of the setting is tag1=attrib,attrib;tag2=attrib. Attributes delimited by
comma, and tags delimited by semicolon.

The left side of the equal sign is the tag.

The right side of the equal sign is a comma delimited list of attributes to be matched.

If a matching tag is found, that matches any of the attributes listed,
the tag will not be converted back to TML.
You can use perl regular expressions to match tag and attribute names, so
.*=id,on.* will ensure that any tag with an id or on* event handler is kept as
HTML.

Note: HTML has been gradually deprecating HTML attributes, replacing
them with equivalent styles. For example, the table, row and cell background
color attribute: bgcolor="#123456" has been deprecated, replaced with style="background-color: #123456;"
A limited subset of tags ( table, th, tr, td ) will match against both the
attributes and the style components. Styles have been added to the default
setting.

The default setting for this preference are hard coded in the plugin. If you
wish to change the settings, the following list is the default setting coded
in the plugin:

If you edit using the plain-text editor, you can use the <sticky>..</sticky> tags to delimit HTML (or TML) that you do not want to be WYSIWYG edited.

WYSIWYGPLUGIN_IGNOREATTRS - Ignore tag attributes when deciding whether to keep a tag or not when converting HTML to TML. This is most useful when you

have specific styling that you want to make sure you strip off.

This preference takes the same format as WYSIWYGPLUGIN_STICKYBITS. It
specifies tags and their attributes that are to be ignored when making the
decision whether to keep the tag or not. For example, a <font face="Open Sans"> tag will normally be maintained in the TML. However setting
WYSIWYGPLUGIN_IGNOREATTRS to font=face will result in it being removed.

By default WYSIWYGPLUGIN_IGNOREATTRS is empty. WYSIWYGPLUGIN_STICKYBITS
takes precedence over this setting.

Implementors note if you are using your own before/after edit handlers, you can call Foswiki::Plugins::WysiwygPlugin::isWysiwygEditable() to check these controls.

Overlapping styles

Because Foswiki uses a "best guess" approach to some formatting, it allows overlapping of tags in a way forbidden by HTML, and it is impossible to guarantee 100% that formatting in the original Foswiki document will still be there when the same document is loaded and then saved through the WysiwygPlugin. The most obvious case of this is to do with styles. For example, the sentence

*bold _bold-italic* italic_

is legal in TML, but in HTML is represented by

<strong>bold <em>bold-italic</em></strong> <em>italic</em>

which gets translated back to TML as

*bold _bold-italic_* _italic_

which is correct by construction, but does not render correctly in Foswiki. This problem is unfortunately unavoidable due to the way TML works.

WysiwygPlugin is able to convert tables with cells that span rows into TML. This requires syntax provided by the TablePlugin (that is, the | ^ | markup). WysiwygPlugin will therefore only perform row-span related conversion if TablePlugin is enabled. TablePlugin is enabled by default and hence WysiwygPlugin converts tables with cells that span rows between TML and HTML by default.

If TablePlugin is not enabled, then TML table cells containing only ^ are not converted to rowspans, and HTML tables containing rowspans are not converted to TML.

Description: Sometimes tables will fail to be converted to TML syntax (will stay as HTML) because there are attributes on the table (such as alignment or border decorations) that WysiwygPlugin does not know how to preserve. If such attributes are necessary, please use VarTABLE instead.

Work-around:

Click inside the offending table

Click the table toolbar button (usually used to create a new table)

With the exception of Cols and Rows, delete/reset all content from the fields on the 'General' and 'Advanced' tabs.

Foswikitask:Item5221: Use Wysiwyg transition to remove usually unwanted paragraph html tags in table cells, which are otherwise impossible to remove in TinyMCE up to at least 3.3.6Foswikitask:Item8274: Fix problem where Wysiwyg transition merges two consecutive lists (a result of work on Foswikitask:Item2254)