Posts Tagged grapheme-colour synaesthesia

On Thursday I was thinking about interesting things I could do if I made a book of some of my writing and typeset it myself, and one of the things I thought of was running Synaesthete’s Blues (a poem about discovering my ‘S’ fridge magnet was blue, when according to my grapheme → colour synaesthesia it should be orange) through Synaesthetist (an app I wrote to display text in a given synaesthetist’s colours) so that it would be displayed in my colours. This would annoy other grapheme → colour synaesthetes in exactly the manner described in the poem (since their colours are almost certainly different from mine) and would probably be jarring for non-synaesthetes too.

If I did make such a book, it would probably be very expensive and time-consuming to produce, so for now, I will just put a coloured version of Synaesthete’s Blues here. The letters are coloured with the colours I associate with them, and outlined with the colour of the first letter of the word they’re in, because the first letter of a word tends to dominate, and the outline makes it clearer what’s going on and easier to read than simply mixing the colours would — see the post about Synaesthetist for more about that. It’s in bold text, which means the outline is thinner than usual relative to the letters, and makes its effect less true to how I see the words, but it makes the text easier to read, and should make it easier for you to see the colours of the individual letters without them being so dominated by the first letters. The title, URL and my name at the bottom are all done by blending the first letter’s colour in with the others in a fraction that looks just right to me.

Coincidentally, Friday’s xkcd comic was about grapheme → colour synaesthesia. Neat as a tree. In case you’re wondering, I couldn’t see either big number (though they are there.) I have to look at a few graphemes at a time to get colours from them; I can’t see a colour picture just by glancing at it.

A couple of dictionaries tell me that ‘immersive’ applies to electronics which engage several senses apart from sound and vision, whereas ‘immersion’ does have the meaning I’m after but isn’t an adjective. I’m going to stick with calling this immersive until I think of a better idea. You’re immersed (but not embedded) in synaesthesia and the frustration of seeing graphemes in the wrong colours.

Like this:

21 hours and already twoposts ago, I linked to my post about the program I wrote for exploring grapheme colour synaesthesia. So I happened to look at that post again, and realised that despite the fact I learnt the words to Chicken Monkey Duck without any reference to the personally-coloured lyrics (Mac users: download the file and open it in TextEdit to see the text with coloured outlines matching the initial letters, which the box.net preview doesn’t show. Windows users: sorry, I’ve noticed Windows rtf viewers don’t show outlines) I made, I could still start to answer some of the questions. I asked, among other things:

Could grapheme-colour synaesthetes learn to look at a sequence of colours that correspond to letters in their synaesthesia, and read a word?

Could colours be used to help grapheme-colour synaesthetes learn to read a new alphabet, either one constructed for the purposes of secret communication, or a real script they will be able to use for something?

And I included this picture of some text in a supposedly-Klingon font (though it does not seem to be quite the same as plqaD; I am not sure why I didn’t find this font at the time) with the letters coloured according to my own grapheme-colour synaesthesia for the corresponding letters in the latin alphabet:

Looking at it today, I realised I couldn’t remember what the text was; I wrote that post ten months ago. It definitely wasn’t ‘How razorback-jumping frogs can level six piqued gymnasts!’ So I decided to see if I could figure it out based on the colours.

The first and second words I figured out in seconds: Swimming is. S, W and I are pretty distinctive colours, and the Ms even look like Ms. Then… teemncgc? No, that’s not a g. I can imagine a capital G in that sort of colour, but I’m using small-g colour for the ‘g’s, and anyway the letter is a different shape. It’s an H! And that’s not an n but a u; they’re fairly similar. Actually I’d say u is closer to a c-yellow than to n-yellow, so maybe I put that one in wrong. So Swimming is tee? much like (‘like’ is easy to read once I figure out it’s an l rather than a c; I love the colour of the letter K) mathing? No, bathing! M and B are pretty similar colours. So that’s not tee… it’s slightly grayer… logically it must be too. I guess I don’t think too much about the colour of the letter O, but that’s about right. Swimming is too much like… bathing. Okay, I’m googling that; I assume I took a Klingon quote from the internet. Yep, it’s Worf.

So yes, at least one grapheme-colour synaesthete can look at a sequence of colours that correspond to letters in their synaesthesia, and read a word. As for the second question, I haven’t really learnt much of the Klingon ‘alphabet’ from this, because I’ve been reading the colours and only paying attention to the shapes of the letters when two letters are of similar colour. The colours mean I can cheat and not even look at the shapes, so they might actually make it harder for me to learn the shapes in the short term. On the other hand, I can read more, more quickly, since I’ll never have to refer to a chart to look up which letters the shapes correspond to. I suspect that practice reading the colours (perhaps gradually fading to black) would help me learn the shapes in the long run.

I haven’t touched the Synaesthetist app since I wrote it; maybe I should go back and implement some of the features I was thinking about.

I’ve mentionedbefore that I have grapheme-colour synaesthesia. That means that I intuitively associate each letter or number with a colour. The colours have stayed the same throughout my life, as far as I remember, and they are not all the same colours that other grapheme-colour synaesthetes (such as my father and brother) associate with the same letters. I still see text written in whichever colour it’s written in, but in my mind it has other colours too. If I have to remember the number of a bus line, there’s a chance I’ll remember the number that goes with the colour it was written in rather than the correct letter, or I’ll remember the correct letter and look in vain for a bus with a number written in that colour.

Well, I’ve been wondering whether it could work the other way.

Could grapheme-colour synaesthetes learn to look at a sequence of colours that correspond to letters in their synaesthesia, and read a word?

Could this be used to send code messages that only a single synaesthete can easily read?

Could colours be used to help grapheme-colour synaesthetes learn to read a new alphabet, either one constructed for the purposes of secret communication, or a real script they will be able to use for something?

What would be the difference in learning time for a grapheme-colour synaesthete using their own colours for the replacement graphemes, a grapheme-colour synaesthete using random colours, and a non-synaesthete?

I know that for me, there are quite a few letters with similar colours, and a few that are black or white, so reading a novel code wouldn’t be infallible, but I suspect I would be able to learn a new alphabet a little more easily or read it more naturally if it were presented in the ‘right’ colours. I wonder whether the reason the Japanese symbol for ‘ka’ seemed so natural and right to me was that it seemed to be the same colour as the letter k.

It occurred to me that, as a programmer and a grapheme-colour synaesthete, I could test these ideas, or at least come up with some tools that scientists working in this area could use to test them. So I wrote a little Mac program called Synaesthetist. You can download it from here. In it, you choose the colours that you associate with different letters (or just make up some if you don’t have grapheme-colour synaesthesia and you want to know what it’s like) and save them to a file.

Then you can type in some text, and you’ll see the text with the letters in the right colours, like so:

But even though this sample is using the ‘right’ colours for the letters, it still looks all wrong to me. When I think of a word, usually the colour of the word is dominated by the first letter. So I added another view with a slider, where you can choose how much the first letter of a word influences the colours of the rest of the letters in the word.

This shows reasonably well what words are like for me, but sometimes the mix of colours doesn’t really resemble either original colour. It occurred to me that an even better representation would be to have the letters in their own colours, but outlined in the colour of the first letter. So I added that:

Okay, so that gives you some idea of what the words look like in my head. And maybe feeding text through this could help me to memorise it. Here’s an rtf file of the lyrics to Mike Phirman‘s song ‘Chicken Monkey Duck‘ in ‘my’ colours, with initial letter outline. I’ll study these and let you know it it helps me to memorise them. To be scientific about it, I really should recruit another synaesthete (who would have different colours from my own, and so might be hindered by my colours) and a non-synaesthete to try it as well, and define exactly how much it should be studied and how to measure success. But I’m writing a blog, not running a study, so if you want to try it, download the file. (I’d love it if somebody did run a study to answer some of my questions, though. I’d add whatever features were necessary to the app.)

But these functions don’t go too far in answering the questions I asked earlier. How about reading a code? Well, I figured I’d be more likely to intuit letters from coloured things if they looked a little bit like letters: squiggles rather than blobs. So first I added a view that simply distorts the letters randomly by an amount that you can control with the slider. I did this fairly quickly, so there are no spaces or word-wrapping yet.

I can’t read it when it gets too distorted, but perhaps it’s easier to read at low-distortion than it would be if the letters were all black. Maybe I’d be able to learn to ‘read’ the distorted squiggles based on colour alone, but I doubt it. This randomly distorts the letters every time you change the distortion amount of change the text, and it doesn’t keep the same form for each occurrence of the same letter. Maybe if it did, I’d be able to learn and read the new graphemes more easily than a non-synaesthete would. Okay, how about just switching to a font that uses a fictional alphabet? Here’s some text in a Klingon font I found:

I know that Klingon is its own language, and you can’t just write English words in Klingon symbols and call it Klingon. But the Futurama alien language fonts I found didn’t work, and Interlac is too hollow to show much colour.

Anyhow, maybe with practice I’ll be able to read that ‘Klingon’ easily. I certainly can’t read it fluently, but even having never looked at a table showing the correspondence between letters and symbols, I can figure out some words if I think about it, even when I copy some random text without looking. I intend to add a button to fetch random text from the web, and hide the plain text version, to allow testing of reading things that the synaesthete has never seen before, but I didn’t have time for that.

Another thing I’ll probably do is add a display of the Japanese kana syllabaries using the consonant colour as the outline and the vowel colour as the fill.

Here’s a screenshot of the whole app:

As I mentioned, you can download it and try it for yourself. It works on Mac OS X 10.7, and maybe earlier versions too. To use it, either open my own colour file (which is included with the download) or create a new document and add some characters and colours in the top left. Then enter some text on the bottom left, and it will appear in all the boxes on the right side. If you change the font in the bottom left, say to a Klingon font, it will change in all the other displays except the distorted one.

This is something I’ve coded fairly hastily on the occasional train trip or weekend, usually forgetting what I was doing between stints, so there are many improvements that could be made, and several features already halfway developed. It could do with an icon and some in-app help, too. I’m still working on this, so if you have any ideas for it, I’m all ears.

Just over twelve hours to write something. I should have started sooner. I’ll start by reading the section on short short stories in Susan Tiberghian’s book, because it’s about time I wrote some prose. She says, ‘A story, be it short or book length, creates a dream in the reader’s mind.’ Can I create a universe in your head in twelve hours? How much of the real universe had been created after twelve hours? It didn’t take much more than seventeen minutes for the newly created protons and neutrons to band together into light nuclei.

Things go a little slower now, but perhaps I can do something similar in the time I have. First, I need some protons to start from. That’s easy. Take three random cards from my pile of sixes of clubs. With any luck, they’ll be different enough that merely finding a link between them will give me an entire story, but not so different that I can’t find a link. Three quarks to form a proton or neutron, two the same, one different.

An ordinary six of clubs. Why do the boring cards always come up when I do this? A close-up of a black spotted cow in Holland. Well, cows eat clovers. Spreading phlox in Canada. Sounds like something made up by Dr. Seuss. Too similar. Do the phlox and clovers vie for the cow’s attention? Can I write an interesting story about a perfectly ordinary cow eating clovers? Susan quotes Eunice Scarfe as saying, ‘If we have lived, we each have a story.’ What is the cow’s story? Perhaps the letter of the week can help me. H, from the Semitic letter ח. According to wikipedia, the form of the letter probably stood for a fence or posts. There are none, in the field where this Dutch cow lived.

Green clovers and phlox

I do not like this spreading phlox,
I would not like it with an ox.
I’d rather risk a mad cowpox,
by joining all the other stocks
and munching on a tasty clover,
but alas I can’t get over,
Thank goodness I’ve a bale of stover,
some for me and some left over.

No, this isn’t going anywhere. I quite like the CERN card this week though: formation of nuclei, or nucleosynthesis: Temperature is low enough to allow protons and neutrons to combine to form nuclei (deuterium, helium, lithium) Conditions similar to interior of stars. It could be an analogy for so many things.

Nuclear Bonds

At first, I was friends with everyone. Any kid who would play with me for five minutes was my friend for five minutes, maybe six. Later on, they tired of bouncing between playmates, and formed more lasting friendships. I flew through them alone, at times kicked here and there by their repulsion, at times accepted temporarily into a more neutral group. Finally I collided with another lone spark, and we bonded.

Not bad, I guess. But I don’t know how long I could continue it. What’s the letter of the week again? Ah… H is for hydrogen, which has the lightest nucleus of all, a single proton, which would have existed even before nucleosynthesis started. What can I say about hydrogen? I may not have much of a story, but I have the best title ever.

Big Bang Nucleosynaesthesia

Hydrogen’s green,
Helium too.
I didn’t know how,
but somehow I knew.

I used to think hydrogen was green. The letter H was as green as they come, and I didn’t know where else I would have got that association from.

My family had several old cars, often referred to as ‘old bombs’. One was exactly the colour of H, and I was burning to make a joke about it being an H-bomb. I always stopped just short of saying anything, because I couldn’t figure out what made H green. Was hydrogen green? It ought to be. Eventually, the frustration of not being able to tell this joke got to me, and I asked my dad whether hydrogen was green. It wasn’t.

Some time later, I gathered the courage to ask him whether the letter H was green. I don’t remember what colour he said it was, but it was not green. He said that perhaps the colours we associated with numbers and letters came from fridge magnets or alphabet books we had as children. A is for apple, so maybe that’s why it was red. Only, it’s more of a pinkish red.

When I was a teenager, I heard about something called synaesthesia, where people could taste colours, see sounds, and all sorts of other weird and wonderful combinations. How strange it must be to see a red apple and taste
a steak and cheese pie. How amazing it must be to see an entire symphony laid out like an intricately knotted carpet. How enlightening it must be to feel a graph tingling on the back of the neck, and linking intuitively with other information like a massage from a well-trained masseuse.

Synaesthetes were real-world superheroes, until I found out I was one. A few years ago I read about something called grapheme-colour synaesthesia, which means that people automatically associate letters and numbers with colours. Like all kinds of synaesthesia, it runs in families. Different people have different colours for each letter and number, although ‘A’ is quite frequently reported to be red. It does not seem to depend on the fridge magnets the synaesthetes were exposed to. Nor does it reveal any deep truths about the universe outside my head. On the other hand, people are talking a lot about hydrogen as a green alternative to fossil fuels these days…

Perhaps this idea would just about cut it. Perhaps not. The H fridge magnet which I’ll have to use to illustrate it is an incongruous red. An H in disguise; it took me a while to find.

Sunset. The faintly fading photons remind me that it’s time to fuse all these proto-ideas into the nucleus of a story. Perhaps if I force myself to write them, a link will reveal itself. But they stubbornly stay separate, isolated and inadequate. Perhaps that’s how it should be. Most of the universe today is made of hydrogen, those lone protons which slipped through the nucleosynthesis stage unaffected. I just need to embellish them with electrons, and send them electronically across the globe.