Author: Andrew StaceyFormat: MarkdownFollowing on from the discussion on the cafe, I've discovered that tikz (a fairly sophisticated LaTeX picture package) can be converted to SVG. Basically, running htlatex (from the TeX4ht package) on a latex document with tikz pictures in produces SVG versions of the pictures. There are a few limitations, mainly to do with text, but it seems to work.
However, including SVGs in Instiki doesn't seem to work exactly. Compare the picture in the sandbox with the original SVG:
[Sandbox](http://ncatlab.org/nlab/show/Sandbox)
[Original SVG](http://www.math.ntnu.no/~stacey/documents/smooth.1-1.svg)
For more examples, see the page of my talk on Comparative Smootheology at the Ottawa conference which is [here](http://www.math.ntnu.no/~stacey/Seminars/ottawa.html).
The trick, from the pgf manual, is to load the correct output driver _before_ loading the Ti_k_Z package. The line is:
\def\pgfsysdriver{pgfsys-tex4ht.def}
I got this hint originally from someone's user page on Wikipedia, but it's covered fully in the PGF manual.

Following on from the discussion on the cafe, I've discovered that tikz (a fairly sophisticated LaTeX picture package) can be converted to SVG. Basically, running htlatex (from the TeX4ht package) on a latex document with tikz pictures in produces SVG versions of the pictures. There are a few limitations, mainly to do with text, but it seems to work.

However, including SVGs in Instiki doesn't seem to work exactly. Compare the picture in the sandbox with the original SVG:

Author: Andrew StaceyFormat: MarkdownCross-posted from the n-Lab: Bruce Bartlett said:
Been a while since I thought about getting a good graphics engine here in inStiki, but these pictures have got me excited again. Right now I am thinking that the following procedure could work well:
* You draw a picture using your favourite method (TikZ, Inkscape, etc.)
* You export it as an svg file.
* You copy and paste this svg text into the nLab page you are editing.
* Now you click "Submit" as usual.
* Here's the trick: the next time someone clicks "Edit page" in the nLab, the Instiki software is intelligent and displays the SVG text as a javascript "expand box". In other words, there's a little command in the text "Expand SVG code (+)" and if you click the (+), it expands all the code out. If you click "(-)", the code collapses again. This way we don't get large swathes of SVG code cluttering up the text area.
* Perhaps the software can be more advanced and remember if someone has altered any of the SVG segments of code. If not, it shouldn't be resubmitted when someone clicks "Submit", otherwise it will make for slow uploads.

Cross-posted from the n-Lab: Bruce Bartlett said:

Been a while since I thought about getting a good graphics engine here in inStiki, but these pictures have got me excited again. Right now I am thinking that the following procedure could work well:

You draw a picture using your favourite method (TikZ, Inkscape, etc.)

You export it as an svg file.

You copy and paste this svg text into the nLab page you are editing.

Now you click "Submit" as usual.

Here's the trick: the next time someone clicks "Edit page" in the nLab, the Instiki software is intelligent and displays the SVG text as a javascript "expand box". In other words, there's a little command in the text "Expand SVG code (+)" and if you click the (+), it expands all the code out. If you click "(-)", the code collapses again. This way we don't get large swathes of SVG code cluttering up the text area.

Perhaps the software can be more advanced and remember if someone has altered any of the SVG segments of code. If not, it shouldn't be resubmitted when someone clicks "Submit", otherwise it will make for slow uploads.

Author: Andrew StaceyFormat: MarkdownOne method that would solve a lot of that would be to have the SVGs as external files that are then included at the appropriate point. I don't know whether or not that is possible, though.
Another thing is that exported files can often contain a lot of "junk" (cafeful examination of my picture will reveal that it uses 5 decimal places; perhaps slightly over-accurate). Some way of automatically trimming this down would be helpful.
I think it's okay to add another step into your scheme, between the export and copying into Instiki, but it needs to be an easy one, say a script that modifies things a little to be acceptable to Instiki. However, what that needs to do will only become apparent with a little experimentation.

One method that would solve a lot of that would be to have the SVGs as external files that are then included at the appropriate point. I don't know whether or not that is possible, though.

Another thing is that exported files can often contain a lot of "junk" (cafeful examination of my picture will reveal that it uses 5 decimal places; perhaps slightly over-accurate). Some way of automatically trimming this down would be helpful.

I think it's okay to add another step into your scheme, between the export and copying into Instiki, but it needs to be an easy one, say a script that modifies things a little to be acceptable to Instiki. However, what that needs to do will only become apparent with a little experimentation.

Author: BruceFormat: TextI tried posting a diagram on the nLab, via LaTex -> Inkscape (via textext) -> SVG -> nLab. I found that the text came out as spline primitives (which is the way textext will convert TeX output into SVG). Obviously we don't want that, since it takes up too much space - we'd want the text to come out in the standard MathML font.
I was under the impression this was somehow impossible, due to a fundamental incompatibility between SVG and MathML (they don't speak to each other).
Now I realize that Andrew did exactly that with his diagram. Andrew, can you spell out exactly what you did, for us idiots? I'm running a Windows machine.

I tried posting a diagram on the nLab, via LaTex -> Inkscape (via textext) -> SVG -> nLab. I found that the text came out as spline primitives (which is the way textext will convert TeX output into SVG). Obviously we don't want that, since it takes up too much space - we'd want the text to come out in the standard MathML font.

I was under the impression this was somehow impossible, due to a fundamental incompatibility between SVG and MathML (they don't speak to each other).

Now I realize that Andrew did exactly that with his diagram. Andrew, can you spell out exactly what you did, for us idiots? I'm running a Windows machine.

Author: UrsFormat: TextHi guys,
this is awsome that you are looking into this.
I don't think I'll have the time to help ypou much with it, but I want to let you know that I think having a good transfer mechanism from LaTeX coded diagrams used in articles to to SVG used on the wiki would be a major point in case of the wiki.
I hope once you figured it all out and the dust has settled, we'll be able to add a short description on the HowTo page that describes what one has to do, or even provides a script or the like that does it automatically.
Best,
Urs

Hi guys,

this is awsome that you are looking into this.

I don't think I'll have the time to help ypou much with it, but I want to let you know that I think having a good transfer mechanism from LaTeX coded diagrams used in articles to to SVG used on the wiki would be a major point in case of the wiki.

I hope once you figured it all out and the dust has settled, we'll be able to add a short description on the HowTo page that describes what one has to do, or even provides a script or the like that does it automatically.

Author: BruceFormat: HtmlOk, I've done some more reading up... it's giving me "Application errors".
It seems that the TikZ ->SVG method might well work. Sadly, I don't think the xypic -> SVG method can work (so I don't see a way of using the standard commutative diagrams package xymatrix from xypic... this is the standard one, right?). I'm not sure -why- it can't work, but when you run htlatex on it it just produces a png file rather than a svg file.
But with TikZ, you can actually produce SVG files. This might be very exciting, because TikZ has the capacity to produce nice commutative diagrams. There's a nice manual available at <a href="http://www.felixl.de/commu.pdf">Commutative diagrams in TikZ</a>. However, when I attempted to convert one of these diagrams into SVG, I noticed the arrows came out fine but the text was garbled. Andrew, how did you get the "psi" and "phi" on your diagram to work?
If this TikZ thing indeed works out, then perhaps we can ask Jacques to install this basic capability into Instiki. Namely, you can just type
\begin{tikzpicture}
\end{tikzpicture}
and Jacques tex2mml compiler will automatically run the htlatex for you behind the scenes... so that SVG will be displayed, although you edit it in user-friendly LaTex form.

It seems that the TikZ ->SVG method might well work. Sadly, I don't think the xypic -> SVG method can work (so I don't see a way of using the standard commutative diagrams package xymatrix from xypic... this is the standard one, right?). I'm not sure -why- it can't work, but when you run htlatex on it it just produces a png file rather than a svg file.

But with TikZ, you can actually produce SVG files. This might be very exciting, because TikZ has the capacity to produce nice commutative diagrams. There's a nice manual available at Commutative diagrams in TikZ. However, when I attempted to convert one of these diagrams into SVG, I noticed the arrows came out fine but the text was garbled. Andrew, how did you get the "psi" and "phi" on your diagram to work?

If this TikZ thing indeed works out, then perhaps we can ask Jacques to install this basic capability into Instiki. Namely, you can just type

\begin{tikzpicture}\end{tikzpicture}

and Jacques tex2mml compiler will automatically run the htlatex for you behind the scenes... so that SVG will be displayed, although you edit it in user-friendly LaTex form.

Author: Andrew StaceyFormat: MarkdownI found that I couldn't just input the SVG that TikZ+htlatex gave me straight into Instiki. Even if the SVG file displayed correctly by itself then it still wasn't guaranteed to work. What gave me most grief was the text, as Bruce has found out. Basically, text boxes in TikZ (nodes and the like) can be extremely complicated but text in SVG is extremely simple. I think that if you prepare your pictures knowing that you are going to output them to SVG then it should be a little simpler, but I drew my pictures before I knew that one could do SVG (not that that would have made any difference, I drew them for my talk not for Instiki). So I've edited the text by hand in all of them.
If anyone wants some examples to play around with, then there are 9 pictures from my talk in Ottawa available on my webpage [here](http://www.math.ntnu.no/~stacey/Seminars/ottawa.html). You can download the source for the pictures, and the resulting SVG. The SVGs have been mildly hacked to get the text right. If you do the 'htlatex' hack and compare the output with the SVG from my webpage then you'll see what I had to do. (Note: before running 'htlatex' on any of the LaTeX files make sure to uncomment the `\def\pgfsys` line).
However, even that editting is vastly easier than generating the SVG by hand, or even using a "graphical" program to produce it. I'm much more of a "programmer" than a "graphic designer" so I far prefer using something like TikZ than, say, Inkscape or XFig.
I've been emailing Jacques about this a little as well and he seems keen to make it easy to go from LaTeX (TikZ) to Instiki (SVG) so I think it's a Good Time to investigate and try to find out how to make it all work well.
Just a side point, having stumbled upon TikZ (all credit to the SBS boys and girls) then I'm not using xypic again. In fact, I'm thinking of writing a little script that would convert a (simple!) xymatrix diagram into TikZ so that I can convert all my old diagrams into the Brave New Format.
However, it's late Friday evening here and the weekend beckons.
By the way, when experimenting then make sure that the SVG is valid _before_ you try importing it into Instiki. Either load the file up in an SVG-capable browser (Firefox) or something like Inkscape. Firefox is good because it'll tell you where the (first) error is - that's how I found out that the text boxes weren't working well (and remember that in valid XML then all tags have to be either balanced or end with a slash). Doing this saves a lot of time.

I found that I couldn't just input the SVG that TikZ+htlatex gave me straight into Instiki. Even if the SVG file displayed correctly by itself then it still wasn't guaranteed to work. What gave me most grief was the text, as Bruce has found out. Basically, text boxes in TikZ (nodes and the like) can be extremely complicated but text in SVG is extremely simple. I think that if you prepare your pictures knowing that you are going to output them to SVG then it should be a little simpler, but I drew my pictures before I knew that one could do SVG (not that that would have made any difference, I drew them for my talk not for Instiki). So I've edited the text by hand in all of them.

If anyone wants some examples to play around with, then there are 9 pictures from my talk in Ottawa available on my webpage here. You can download the source for the pictures, and the resulting SVG. The SVGs have been mildly hacked to get the text right. If you do the 'htlatex' hack and compare the output with the SVG from my webpage then you'll see what I had to do. (Note: before running 'htlatex' on any of the LaTeX files make sure to uncomment the \def\pgfsys line).

However, even that editting is vastly easier than generating the SVG by hand, or even using a "graphical" program to produce it. I'm much more of a "programmer" than a "graphic designer" so I far prefer using something like TikZ than, say, Inkscape or XFig.

I've been emailing Jacques about this a little as well and he seems keen to make it easy to go from LaTeX (TikZ) to Instiki (SVG) so I think it's a Good Time to investigate and try to find out how to make it all work well.

Just a side point, having stumbled upon TikZ (all credit to the SBS boys and girls) then I'm not using xypic again. In fact, I'm thinking of writing a little script that would convert a (simple!) xymatrix diagram into TikZ so that I can convert all my old diagrams into the Brave New Format.

However, it's late Friday evening here and the weekend beckons.

By the way, when experimenting then make sure that the SVG is valid before you try importing it into Instiki. Either load the file up in an SVG-capable browser (Firefox) or something like Inkscape. Firefox is good because it'll tell you where the (first) error is - that's how I found out that the text boxes weren't working well (and remember that in valid XML then all tags have to be either balanced or end with a slash). Doing this saves a lot of time.

Author: Andrew StaceyFormat: MarkdownBruce wrote in the sand:
> Well, you can certainly upload files here in Instiki, and I think that method would be possible. But then we’re back to the Fundamental Conundrum:
> If we’re going to start including files, you might as well export your graphic as a png file and include it the old-fashioned way.
> It’s the old problem: SVG and MathML have not been built so that one can use them both simultaneously (I think).
Not necessarily. SVG has some distinct advantages over PNGs, scalability for one, and it's lossless (sort of). Web pages are generally made up of lots of little bits stuck together so putting the SVGs in separate files just to tidy up the content is no great issue.
I also think that SVG and MathML can be used together. Jacques said something about 'foreignObject's in SVG. But as I said above, it's late so I'm not investigating now.

Bruce wrote in the sand:

Well, you can certainly upload files here in Instiki, and I think that method would be possible. But then we’re back to the Fundamental Conundrum:

If we’re going to start including files, you might as well export your graphic as a png file and include it the old-fashioned way.

It’s the old problem: SVG and MathML have not been built so that one can use them both simultaneously (I think).

Not necessarily. SVG has some distinct advantages over PNGs, scalability for one, and it's lossless (sort of). Web pages are generally made up of lots of little bits stuck together so putting the SVGs in separate files just to tidy up the content is no great issue.

I also think that SVG and MathML can be used together. Jacques said something about 'foreignObject's in SVG. But as I said above, it's late so I'm not investigating now.

Author: Mike ShulmanFormat: MarkdownI'm not sure what exactly causes all the text in the SVG produced by htlatex to be stacked on top of itself at one spot in the picture, but it seems that it can be avoided by using more basic TikZ syntax rather than the matrix stuff suggested in the commutative diagrams tutorial. For instance, running this through `htlatex`:
\begin{tikzpicture}[auto,scale=2]
\node (A) at (0,0) {$A^2$};
\node (B) at (2,0) {$\int B$};
\node (C) at (1,1) {$\sum_0^\infty C$};
\draw[->] (A) to node[swap] {$\phi$} (B);
\draw[->] (A) to node {$\psi$} (C);
\draw[->] (C) to node {$\theta$} (B);
\draw[->] (1,0.5) to node {$\mu$} (1,0.3);
\end{tikzpicture}
produces an SVG that seems to display fine in firefox and instiki. There is no MathML by default, so that labels including math like $A^2$ or $\sum_0^\infty C$ do not display correctly, but that can then be fixed up by hand: namely, find the `<text>` tags that contain the labels and replace their contents (usually one or more `<tspan>`s) by something like
`<foreignObject markdown="1" x="-5pt" y="-12pt" width="10pt" height="15pt">$A^2$</foreignObject>`
I don't know any automatic way to decide what the `x`, `y`, `width`, and `height` values should be; it depends on how big the MathML is going to end up. But if we can figure out a way to automate that, then perhaps Jacques can make Instiki read the TikZ code and parse it into this sort of SVG+MathML for the browser. I think that would be the ideal solution: the diagrams would be stored in more or less readable TikZ format in the instiki source and could easily be changed by future editors of the page, without needing to recreate the whole thing.
Apparently if you call `xhmlatex` instead of `htlatex` it is supposed to produce MathML from your formulas. The resulting MathML doesn't work for me, though; I'm not sure what's wrong. And it doesn't even try to do anything with formulas inside TikZ pictures.

I'm not sure what exactly causes all the text in the SVG produced by htlatex to be stacked on top of itself at one spot in the picture, but it seems that it can be avoided by using more basic TikZ syntax rather than the matrix stuff suggested in the commutative diagrams tutorial. For instance, running this through htlatex:

produces an SVG that seems to display fine in firefox and instiki. There is no MathML by default, so that labels including math like $A^2$ or $\sum_0^\infty C$ do not display correctly, but that can then be fixed up by hand: namely, find the <text> tags that contain the labels and replace their contents (usually one or more <tspan>s) by something like

I don't know any automatic way to decide what the x, y, width, and height values should be; it depends on how big the MathML is going to end up. But if we can figure out a way to automate that, then perhaps Jacques can make Instiki read the TikZ code and parse it into this sort of SVG+MathML for the browser. I think that would be the ideal solution: the diagrams would be stored in more or less readable TikZ format in the instiki source and could easily be changed by future editors of the page, without needing to recreate the whole thing.

Apparently if you call xhmlatex instead of htlatex it is supposed to produce MathML from your formulas. The resulting MathML doesn't work for me, though; I'm not sure what's wrong. And it doesn't even try to do anything with formulas inside TikZ pictures.

Author: BruceFormat: HtmlHey this is great! Mike has shown us a good method.
All we need are those displacement numbers (x="-5pt" y="-12pt") and the width/height of our math formulas (width="10pt" height="15pt"), and we're away. Perhaps this will turn out to be a tough nut to ultimately crack though, from reading Jacques' <a href="http://golem.ph.utexas.edu/~distler/blog/archives/001475.html">previous posts</a> on this mattter... but maybe not!
What we really need is for Till Tantau to upgrade his htlatex program to output text as embedded mathml via the foreignobject tag. Jacques and Till should join forces to revolutionize math on the web! The htlatex program should use the itex2mml translator to translate LaTeX text formulas formulas into mathml, which then get inserted into the svg via foreignobject tags!
Until that time, maybe Jacques can indeed program a quick fix for us here. Just guess the x and y displacement values (keep the default values x="-5pt" y="-12pt" which seems to work) and the default width and height as width="10pt" height="15pt". Then he can include the htlatex engine directly into Instiki, so that we can start making pictures and commutative diagrams in TikZ!
Then all we need is for Andrew to indeed write that "little script" which translates xymatrix diagrams into TikZ format, and for Jacques to include this into Instiki. This will enable Urs to copy and paste his gimungous xymatrix diagrams directly into the nLab with glee!

Hey this is great! Mike has shown us a good method.

All we need are those displacement numbers (x="-5pt" y="-12pt") and the width/height of our math formulas (width="10pt" height="15pt"), and we're away. Perhaps this will turn out to be a tough nut to ultimately crack though, from reading Jacques' previous posts on this mattter... but maybe not!

What we really need is for Till Tantau to upgrade his htlatex program to output text as embedded mathml via the foreignobject tag. Jacques and Till should join forces to revolutionize math on the web! The htlatex program should use the itex2mml translator to translate LaTeX text formulas formulas into mathml, which then get inserted into the svg via foreignobject tags!

Until that time, maybe Jacques can indeed program a quick fix for us here. Just guess the x and y displacement values (keep the default values x="-5pt" y="-12pt" which seems to work) and the default width and height as width="10pt" height="15pt". Then he can include the htlatex engine directly into Instiki, so that we can start making pictures and commutative diagrams in TikZ!

Then all we need is for Andrew to indeed write that "little script" which translates xymatrix diagrams into TikZ format, and for Jacques to include this into Instiki. This will enable Urs to copy and paste his gimungous xymatrix diagrams directly into the nLab with glee!

Author: BruceFormat: HtmlMmm... I'm a touch confused, since it seems that htlatex system <a href="http://minimals.contextgarden.net/current/modules/t-tikz/tex/generic/pgf/systemlayer/pgfsys-tex4ht.def">does include</a> mathml in the svg output via foreignobject tags. I quote from the file:
"initial support of Mathml and xhtml inside svg through the svg:foreignelement tag
it'll allow us to have complicated text nodes in the tex4ht driver"

Mmm... I'm a touch confused, since it seems that htlatex system does include mathml in the svg output via foreignobject tags. I quote from the file:

"initial support of Mathml and xhtml inside svg through the svg:foreignelement tagit'll allow us to have complicated text nodes in the tex4ht driver"

Author: Mike ShulmanFormat: MarkdownThe version of `pgfsys-tex4ht.def` that you linked to is the [CVS](http://pgf.cvs.sourceforge.net/viewvc/pgf/pgf/generic/pgf/systemlayer/pgfsys-tex4ht.def?view=log) version, the one the developers are currently working on. The claimed mathml support was added after the most recent official release of PGF (version 2.00), which is the version I have installed. Maybe we should try using the CVS version, although doing that with anything always gives me the creeps.
I cannot extract something relevant from your other link, can you explain what you got from it?
By the way, according to the PGF manual, one of the limitations of the `pgfsys-tex4ht.def` driver is that "Matrices do not work." So I guess that "explains" why we can't get it to work that way. `(-:`

The version of pgfsys-tex4ht.def that you linked to is the CVS version, the one the developers are currently working on. The claimed mathml support was added after the most recent official release of PGF (version 2.00), which is the version I have installed. Maybe we should try using the CVS version, although doing that with anything always gives me the creeps.

I cannot extract something relevant from your other link, can you explain what you got from it?

By the way, according to the PGF manual, one of the limitations of the pgfsys-tex4ht.def driver is that "Matrices do not work." So I guess that "explains" why we can't get it to work that way. (-:

Author: BruceFormat: HtmlHi Mike,
Sorry, the link I should have referred to when I wrote "this user" above was <a href="http://article.gmane.org/gmane.comp.tex.pgf.user/1599">this one</a>. I'm not sure if anything useful can be extracted from it!

Hi Mike,

Sorry, the link I should have referred to when I wrote "this user" above was this one. I'm not sure if anything useful can be extracted from it!

Author: Andrew StaceyFormat: MarkdownHmm, looking at my LaTeX files I see that I'm not even using the most up to date version of PGF, let alone CVS. I have no objection to trying out the CVS - it's probably enough to check out the pgfsys-tex4ht.def file. I suspect that actually one can drop this in to a stable release of PGF. In fact, if you have your own TEXMF tree then TeX searches there first so you can just stick the current version somewhere there and if it doesn't work, delete it and you won't have damaged anything.
By the way, as far as I know Till Tantau has nothing to do with TeX4ht (that is, htlatex). As part of the PGF project he's written a "translator" for how TeX4ht should interpret PGF code. It's that translator that we're mucking about with.
Taking the advice of "walking before running", I think the first "stage" is to figure out a "by hand" method for producing SVG that is acceptable for Instiki from suitable input. For diagrams designed specifically for inclusion in an Instiki installation, this should go something like:
1. Design the picture using TikZ
2. Export using htlatex
3. Extremely minor post-processing, all of a reasonably standard variety
Diagrams, such as xy stuff, that weren't drawn explicitly for inclusion may need a little pre-processing (say, to convert to TikZ as I suggested above) and a little more post-processing.
My point is, before we think about automating all this we need to establish exactly what it is we want to automate. Also, automation may not be a desirable goal. As nice as it may be to think that you can take a paper and just stuff it in to Instiki and it will look "okay", we should expect to do a little adjustment to get things to look right. Paper and screen are different media, after all, and what looks right on one may not look right on the other (I'm counting PDF as "paper" here; the point being that it is static whereas screen display can be more interactive).
Lest that sound too cautionary, let me say "well done" for tracking down stuff like the modified pgfsys file!

Hmm, looking at my LaTeX files I see that I'm not even using the most up to date version of PGF, let alone CVS. I have no objection to trying out the CVS - it's probably enough to check out the pgfsys-tex4ht.def file. I suspect that actually one can drop this in to a stable release of PGF. In fact, if you have your own TEXMF tree then TeX searches there first so you can just stick the current version somewhere there and if it doesn't work, delete it and you won't have damaged anything.

By the way, as far as I know Till Tantau has nothing to do with TeX4ht (that is, htlatex). As part of the PGF project he's written a "translator" for how TeX4ht should interpret PGF code. It's that translator that we're mucking about with.

Taking the advice of "walking before running", I think the first "stage" is to figure out a "by hand" method for producing SVG that is acceptable for Instiki from suitable input. For diagrams designed specifically for inclusion in an Instiki installation, this should go something like:

Design the picture using TikZ

Export using htlatex

Extremely minor post-processing, all of a reasonably standard variety

Diagrams, such as xy stuff, that weren't drawn explicitly for inclusion may need a little pre-processing (say, to convert to TikZ as I suggested above) and a little more post-processing.

My point is, before we think about automating all this we need to establish exactly what it is we want to automate. Also, automation may not be a desirable goal. As nice as it may be to think that you can take a paper and just stuff it in to Instiki and it will look "okay", we should expect to do a little adjustment to get things to look right. Paper and screen are different media, after all, and what looks right on one may not look right on the other (I'm counting PDF as "paper" here; the point being that it is static whereas screen display can be more interactive).

Author: Mike ShulmanFormat: MarkdownI definitely agree that we should not expect to be able to just stuff ordinary latex into instiki. As I said before, I think the ideal setup would be something like the current setup with itex in instiki: there is a human-readable and -writable syntax for creating pictures and diagrams, which is at least close to a standard latex syntax, which is automatically processed to produce portable output (SVG and/or MathML) viewable in a browser. But the source code of the page remains in the human-read-write-able syntax (presumably TikZ or like it), so that subsequent people editing the page can modify it without having to recreate it entirely. That's the level of automation I would like; I wouldn't be bothered if the syntax is a bit different from standard latex. I think this would be a significant advantage over attaching files (either images or SVG) or including raw SVG in the page, even if we have a (semi-)automated way to produce SVG from latex, since generated SVG is not very human-read-write-able. Do other people have different thoughts about what the ideal setup would be?
I think I have already proposed a "by hand" method for producing instiki-acceptable SVG. It goes as you suggest, with the "minor post-processing" being replacement of the `<tspan>` tags for text by `<foreignObject>` ones containing the actual math. A sample result can be seen at the bottom of the [SVG Sandbox](http://ncatlab.org/nlab/show/SVG+Sandbox) page.
I guess there are sort of two approaches one could take. One would be to try to use the MathML translation in htlatex to insert the math in the right place in the SVG. The other is to use itex2MML, which is the approach I used. At present, itex2MML appears to me to be more promising; I currently can't get htlatex to output MathML that my firefox can understand.
Has anyone been able to get TikZ to make an acceptable-looking double-shafted arrow (for a 2-cell)? Its "double lines" appear to be produced using a hack that breaks arrowheads.

I definitely agree that we should not expect to be able to just stuff ordinary latex into instiki. As I said before, I think the ideal setup would be something like the current setup with itex in instiki: there is a human-readable and -writable syntax for creating pictures and diagrams, which is at least close to a standard latex syntax, which is automatically processed to produce portable output (SVG and/or MathML) viewable in a browser. But the source code of the page remains in the human-read-write-able syntax (presumably TikZ or like it), so that subsequent people editing the page can modify it without having to recreate it entirely. That's the level of automation I would like; I wouldn't be bothered if the syntax is a bit different from standard latex. I think this would be a significant advantage over attaching files (either images or SVG) or including raw SVG in the page, even if we have a (semi-)automated way to produce SVG from latex, since generated SVG is not very human-read-write-able. Do other people have different thoughts about what the ideal setup would be?

I think I have already proposed a "by hand" method for producing instiki-acceptable SVG. It goes as you suggest, with the "minor post-processing" being replacement of the <tspan> tags for text by <foreignObject> ones containing the actual math. A sample result can be seen at the bottom of the SVG Sandbox page.

I guess there are sort of two approaches one could take. One would be to try to use the MathML translation in htlatex to insert the math in the right place in the SVG. The other is to use itex2MML, which is the approach I used. At present, itex2MML appears to me to be more promising; I currently can't get htlatex to output MathML that my firefox can understand.

Has anyone been able to get TikZ to make an acceptable-looking double-shafted arrow (for a 2-cell)? Its "double lines" appear to be produced using a hack that breaks arrowheads.

Author: Mike ShulmanFormat: MarkdownOk, never mind my complaints about the MathML produced by `xhmlatex`. It wasn't displaying correctly in my firefox because `xhmlatex` by default produces an `.html` file, but firefox won't display MathML correctly unless it is in XHTML mode. Changing the extension of the `xhmlatex` output to `.xhtml` fixed the problem.

Ok, never mind my complaints about the MathML produced by xhmlatex. It wasn't displaying correctly in my firefox because xhmlatex by default produces an .html file, but firefox won't display MathML correctly unless it is in XHTML mode. Changing the extension of the xhmlatex output to .xhtml fixed the problem.

Author: Mike ShulmanFormat: MarkdownPutting the CVS version of pgf in my `~/texmf`, by which I mean checking it out and then copying the contents of `generic/pgf` to `~/texmf/tex/generic/pgf`, causes htlatex to complain:
! Undefined control sequence.
\pgfsys@endpicture ->\HtmlParOn \Par
l.15 \end{tikzpicture}
I don't understand why it is complaining, since `\HtmlParOn` is a perfectly good command defined by tex4ht. Any ideas?

Putting the CVS version of pgf in my ~/texmf, by which I mean checking it out and then copying the contents of generic/pgf to ~/texmf/tex/generic/pgf, causes htlatex to complain:

Author: BruceFormat: HtmlMike, perhaps your error
"I don't understand why it is complaining, since \HtmlParOn is a perfectly good command defined by tex4ht."
is related to that line of code included by the user Olivier Binda on <a href="http://article.gmane.org/gmane.comp.tex.pgf.user/1599">that page</a>:
"{\everypar={}%<----- this is important to produce valid xhtml code in tikzpicture with text nodes (in vboxes a.k.a. with the text width option)"
On the "ideal solution": I think it's a big step up to have editable human-readable text like TikZ or something in the Instiki page as opposed to a big snippet of SVG (whether it is inline or included via a link to an uploaded file). That's what would get me very excited. On the other hand, Andrew I'm happy with your proposal; the trouble is that Instiki doesn't have an "include" command like Mediawiki does (I think... can't quite remember now). At the moment, one can upload files like pdf files or images, and access them from the page, but you can't include text into the page, as far as I know.
Unfortunately the machine I'm at right now has quite an old version of Miktex and so on, so I can't try out the cutting edge ideas till tomorrow.
I don't think it's okay to have huge swathes of MathML cluttering up pages, so we are not yet at the level of a solution. I'm hoping Jacques and Till Tantau get together for the PGF translating of Tex4ht (thanks Andrew for explaining how it works). We are epsilon away from a solution here....!

Mike, perhaps your error

"I don't understand why it is complaining, since \HtmlParOn is a perfectly good command defined by tex4ht."

is related to that line of code included by the user Olivier Binda on that page:

"{\everypar={}%<----- this is important to produce valid xhtml code in tikzpicture with text nodes (in vboxes a.k.a. with the text width option)"

On the "ideal solution": I think it's a big step up to have editable human-readable text like TikZ or something in the Instiki page as opposed to a big snippet of SVG (whether it is inline or included via a link to an uploaded file). That's what would get me very excited. On the other hand, Andrew I'm happy with your proposal; the trouble is that Instiki doesn't have an "include" command like Mediawiki does (I think... can't quite remember now). At the moment, one can upload files like pdf files or images, and access them from the page, but you can't include text into the page, as far as I know.

Unfortunately the machine I'm at right now has quite an old version of Miktex and so on, so I can't try out the cutting edge ideas till tomorrow.

I don't think it's okay to have huge swathes of MathML cluttering up pages, so we are not yet at the level of a solution. I'm hoping Jacques and Till Tantau get together for the PGF translating of Tex4ht (thanks Andrew for explaining how it works). We are epsilon away from a solution here....!

Author: Mike ShulmanFormat: MarkdownIt looks like the support for generating MathML in SVG in the pgf tex4ht driver was added _after_ Olivier Binda's email, and quite possibly he was the one who added it, see [this followup](
http://sourceforge.net/mailarchive/message.php?msg_id=F05182A8-5664-4E74-AD72-394A3F7CAB87%40tcs.uni-luebeck.de).
Also, it looks as though his comment was about producing valid xhtml by not including `<p>` tags inside foreign elements, whereas my error is about htlatex not even compiling the code.

It looks like the support for generating MathML in SVG in the pgf tex4ht driver was added after Olivier Binda's email, and quite possibly he was the one who added it, see this followup.

Also, it looks as though his comment was about producing valid xhtml by not including <p> tags inside foreign elements, whereas my error is about htlatex not even compiling the code.

Author: BruceFormat: HtmlI have emailed Olivier Binda, and he responded thus:
"Well, I implemented the mechanism and committed the changes to the cvs version and wrote a very short text in the pgf manual explaining how to use it.
In the archive attachment you can find a xml page (open it with firefox 3) of a comparative rendering of some picture with the foreignelements method and the previous method.
I obtained this result by applying mzxetex with the option "xhtml,3" to the tex file in attachment (some macros are missing but you'll get the point)
Anyway, this should be documented in the pgf/tikz manual (the cvs version).
I haven't worked at all on this mechanism since then because:
1. I'm not using xetex anymore, I'm only using luatex now (which is way more powerful) and there is no support for luatex in tex4ht (eitan gurari hasn't implemented it yet, to the best of my knowledge... and I should know: I'm the one who asked him to implement initial support for xetex in tex4ht)
2. Once luatex supplants the other tex engine, tikz/pgf will have to be rewritten in lua and there will probably be a luatex driver that writes direcetly mathml+svg without using tex4ht
3. tex to mathml+svg translation won't ever be perfect, because of the svg specification and the differences in browsers so I thought it wasn't worth the effort / time to perfect what I already implemented (which should be just fine for 99% of pictures as you can manually fix them)
Olivier Binda
"
I'm not sure if this helps. Unfortunately I can't link to the example tex file he mentions because of a weird thing going on with googlemail at the moment.

I have emailed Olivier Binda, and he responded thus:

"Well, I implemented the mechanism and committed the changes to the cvs version and wrote a very short text in the pgf manual explaining how to use it.

In the archive attachment you can find a xml page (open it with firefox 3) of a comparative rendering of some picture with the foreignelements method and the previous method.

I obtained this result by applying mzxetex with the option "xhtml,3" to the tex file in attachment (some macros are missing but you'll get the point)

Anyway, this should be documented in the pgf/tikz manual (the cvs version).

I haven't worked at all on this mechanism since then because:

1. I'm not using xetex anymore, I'm only using luatex now (which is way more powerful) and there is no support for luatex in tex4ht (eitan gurari hasn't implemented it yet, to the best of my knowledge... and I should know: I'm the one who asked him to implement initial support for xetex in tex4ht)

2. Once luatex supplants the other tex engine, tikz/pgf will have to be rewritten in lua and there will probably be a luatex driver that writes direcetly mathml+svg without using tex4ht

3. tex to mathml+svg translation won't ever be perfect, because of the svg specification and the differences in browsers so I thought it wasn't worth the effort / time to perfect what I already implemented (which should be just fine for 99% of pictures as you can manually fix them)

Olivier Binda"

I'm not sure if this helps. Unfortunately I can't link to the example tex file he mentions because of a weird thing going on with googlemail at the moment.

Author: Mike ShulmanFormat: MarkdownThe CVS version of the PGF manual suggests that you need to add the
option `tex4ht node/escape=true` in order to activate the escape to
foreignObjects. Adding this to my code produces the error
Runaway argument?
{\EndDviMath \ShowPar
! Paragraph ended before \c:$$: was complete.
<to be read again>
\par
l.8 \node[tex4ht node/escape=true]
(A) at (0,0) {$A^2$};
In addition to the same "Undefined control sequence" error I got before.
What exactly did you ask Olivier Binda? Did you mention my error?

The CVS version of the PGF manual suggests that you need to add the
option tex4ht node/escape=true in order to activate the escape to
foreignObjects. Adding this to my code produces the error

Author: Andrew StaceyFormat: MarkdownI've created a new category on the forum: diagrams; and I've shifted this discussion into it. Please feel free to start a new discussion in that category if you feel that it is warranted. In fact, if you're not sure whether to add to an old discussion or start a new one, then I'd say start a new one. It's much easier to quickly scan through a lot of discussion titles than search through long lists of comments.
For example, it might be useful to gather all the stuff relevant to the pgfsys-tex4ht.def file in one place, and separate out the "post-processing" into another thread.
Unlike Bruce, I don't think that we're within an epsilon of solving this and so keeping the discussion easy to scan through will help.

I've created a new category on the forum: diagrams; and I've shifted this discussion into it. Please feel free to start a new discussion in that category if you feel that it is warranted. In fact, if you're not sure whether to add to an old discussion or start a new one, then I'd say start a new one. It's much easier to quickly scan through a lot of discussion titles than search through long lists of comments.

For example, it might be useful to gather all the stuff relevant to the pgfsys-tex4ht.def file in one place, and separate out the "post-processing" into another thread.

Unlike Bruce, I don't think that we're within an epsilon of solving this and so keeping the discussion easy to scan through will help.

Author: TobyBartelsFormat: HtmlYes, Instiki does have a method to include files!<p>See <code>[[latest changes]]</code> for a typical example: <code>[[!include all changes]]</code>. Note that <code>[[all changes]]</code> has the code for shifting the thing to the right; <code>[[!include </code>…<code>]]</code> is a pure inclusion.

This comment is invalid XML; displaying source.Yes, Instiki does have a method to include files!<p >See <code ><a href="http://ncatlab.org/nlab/show/latest+changes">latest changes</a></code> for a typical example: <code ><a href="http://ncatlab.org/nlab/show/%21include+all+changes">!include all changes</a></code>. Note that <code ><a href="http://ncatlab.org/nlab/show/all+changes">all changes</a></code> has the code for shifting the thing to the right; <code ><a href="http://ncatlab.org/nlab/show/%21include+%3C%2Fcode%3E%E2%80%A6%3Ccode+%3E">!include </code>…<code ></a></code> is a pure inclusion.</p>

Author: TobyBartelsFormat: HtmlThere's now an example of an SVG inclusion in the <a href="http://ncatlab.org/nlab/show/SVG+Sandbox">SVG Sandbox</a>, included from the new <a href="http://ncatlab.org/nlab/show/Inclusion+Sandbox">Inclusion Sandbox</a>.

Author: BruceFormat: HtmlToby, that's great! Now we at least have a working "drop and load" model, which we can implement immediately until the time we get the svg+foreignobject hack in htlatex working. We just need to decide on naming conventions for the uploaded svg files: I suggest [[!include name of page/name of diagram.svg]], eg. [[!include colimit/limit_diagram.svg]].
In the email I received from Olivier Binda, he gave quite a comprehensive test suite for his SVG+foreignobject patch into Tex4ht. I will load it on the nlab for others to try out, as soon as I stop getting "application errors". For what it's worth, it doesn't really display right on my system (Windows XP + Firefox 3), as Olivier Binda indeed warned --- the text seems always a touch displaced down and to the right, but it's pretty close to working... I don't know how to actually compile the stuff yet, Mike/Andrew might be able to do that.

In the email I received from Olivier Binda, he gave quite a comprehensive test suite for his SVG+foreignobject patch into Tex4ht. I will load it on the nlab for others to try out, as soon as I stop getting "application errors". For what it's worth, it doesn't really display right on my system (Windows XP + Firefox 3), as Olivier Binda indeed warned --- the text seems always a touch displaced down and to the right, but it's pretty close to working... I don't know how to actually compile the stuff yet, Mike/Andrew might be able to do that.

Author: Andrew StaceyFormat: MarkdownBruce, how do I get those files from the SVG Sandbox? It looks like a "create page" link to me and when I click on it then I get an error. If it's not easy to upload them, could you email them to me? I can stick them somewhere here and provide a link on this forum.
(I presume that Olivier is happy for you to redistribute his files ...)
Toby, I like the include idea. Can you write up what you did? Most logical place is the HowTo on the n-Lab but if you feel that it's just a step on the way rather than part of the Proper Way of Doing Things then put it here for us to try out (start a new discussion in the "diagram" category).

Bruce, how do I get those files from the SVG Sandbox? It looks like a "create page" link to me and when I click on it then I get an error. If it's not easy to upload them, could you email them to me? I can stick them somewhere here and provide a link on this forum.

(I presume that Olivier is happy for you to redistribute his files ...)

Toby, I like the include idea. Can you write up what you did? Most logical place is the HowTo on the n-Lab but if you feel that it's just a step on the way rather than part of the Proper Way of Doing Things then put it here for us to try out (start a new discussion in the "diagram" category).

Author: TobyBartelsFormat: Html@ Bruce:<p>I would leave off the <code>.svg</code> in the page name, since in principle we might choose to replace it with something other than SVG without editing the page that it appears in.<p>There's also an argument to be made that the including page's name shouldn't appear in the included page's name, since the latter might be included in more (or fewer) than one page at any given time. But conversely, if this is a step towards including TikZ code directly into the page, then it might be better to use page names that reflect that intent.

This comment is invalid XHTML+MathML+SVG; displaying source.<div>
@ Bruce:<p>I would leave off the <code>.svg</code> in the page name, since in principle we might choose to replace it with something other than SVG without editing the page that it appears in.<p>There's also an argument to be made that the including page's name shouldn't appear in the included page's name, since the latter might be included in more (or fewer) than one page at any given time. But conversely, if this is a step towards including TikZ code directly into the page, then it might be better to use page names that reflect that intent.</p></p>
</div>

Author: TobyBartelsFormat: HtmlWhen linking to a file in Instiki, remember to use <code>[[</code>…<code>:file]]</code>. Unfortunately, now I am getting application errors.

This comment is invalid XML; displaying source.When linking to a file in Instiki, remember to use <code ><a href="http://ncatlab.org/nlab/show/%3C%2Fcode%3E%E2%80%A6%3Ccode+%3E%3Afile"></code>…<code >:file</a></code>. Unfortunately, now I am getting application errors.

Author: TobyBartelsFormat: HtmlI only get application errors when I try to use <code>:file</code>! (Or when I try to go directly to <code>http://ncatlab.org/nlab/files/LD@Svg@Tests_files.zip</code>, where it ought to be.) Very annoying …

I only get application errors when I try to use :file! (Or when I try to go directly to http://ncatlab.org/nlab/files/LD@Svg@Tests_files.zip, where it ought to be.) Very annoying …

Author: Andrew StaceyFormat: MarkdownIf we're shifting the SVGs off the main pages (which I think is a very good idea) then we should have something on those main pages which links to the included page, a bit like the "edit sidebar" link on the table of contents. Obviously this shouldn't be part of the diagram itself but maybe a list of includes right at the bottom of the page.
Looking at the diagram on the adjunction page I can see why it's best not to mix MathML and SVG too much.

If we're shifting the SVGs off the main pages (which I think is a very good idea) then we should have something on those main pages which links to the included page, a bit like the "edit sidebar" link on the table of contents. Obviously this shouldn't be part of the diagram itself but maybe a list of includes right at the bottom of the page.

Looking at the diagram on the adjunction page I can see why it's best not to mix MathML and SVG too much.

Author: Andrew StaceyFormat: MarkdownBy the way, including just SVGs breaks the TeX export. You need to upload a PDF as well and put an `\includegraphics{file}` after the `\end{svg}`. Take a look at Jacques' [page](http://golem.ph.utexas.edu/instiki/show/SVG) on this for details.

By the way, including just SVGs breaks the TeX export. You need to upload a PDF as well and put an \includegraphics{file} after the \end{svg}. Take a look at Jacques' page on this for details.

Author: Mike ShulmanFormat: Markdown> By the way, including just SVGs breaks the TeX export.
Another good reason to aim for being able to put TikZ in the source, rather than raw SVG. I haven't had time to work on this for a bit but I hope to soon.
I think the adjunction diagrams would look better if the whole thing were in SVG, rather than just the loopy arrows around the outside.

By the way, including just SVGs breaks the TeX export.

Another good reason to aim for being able to put TikZ in the source, rather than raw SVG. I haven't had time to work on this for a bit but I hope to soon.

I think the adjunction diagrams would look better if the whole thing were in SVG, rather than just the loopy arrows around the outside.

Author: Peter HeinigFormat: TextCreated [[nonisometricregular20170614]] and [[nonregularmono20170614]], for further use in [[category of simple graphs]], partly to at least once see how the include-an-svg-from-another-page-functionality works. Using
<div style="float:left;width:4em;height:25em;margin: 10px 10px 10px 0;">
<svg xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
width="100%" height="100%" viewBox="0 0 400 300" >
<g>
[[!include nonregularmono20170614]]
</g>
</svg>
</div>
to get text flowing around an illustration may be mildly interesting to others.
The result is not yet the intended one, but close, and the intent should be achievable along these lines.
The complaint about the figure not rendering
in at least one browser is not forgotten,
just could not yet find a better solution.
Have to interrupt now.