Posted
by
kdawson
on Friday July 30, 2010 @11:13AM
from the do-what-i-mean dept.

kkleiner writes "BlindType has created a new touchscreen keyboard program of the same name that changes size, orientation, and position to match your wandering fingers as they type. BlindType also features some of the most impressive typing correction software I've ever seen. The result is a practical touchscreen interface that knows what you meant to type, even if you make mistakes. Lots of them. In fact, you can type without looking at the screen at all."

Just add 'lol' 'omgwtfbbq' 'brb' 'afk' and 'QQ' to the dictionary. I highly doubt that they didn't realize that people would want to add words, names, and places that they don't know about. Same system to add those can be used to add all the annoying text speak you want.

Clippy: "Hey! It looks like you're trying to string some letters together unimaginatively. I can help you with:-picking some letters-putting them in a document in order-buying an English prose style guide online..."

Sounds like almost reasonable Smalltalk [wikipedia.org]. Alternatively, I'm sure you could "tell" BlindType you wanted it to be code-friendly (and state a language)... seems like the AI should be able switch out dictionaries and valid syntax and symbols... come to think of it, it might be able to create a more impressive IntelliSense-like IDE feature.

That's exactly what I came here to ask. In addition I already know that autocorrective software, as currently installed on 'smart'phones is totally useless: I alternate between 3 languages daily, one of which is not on the phone, and the other 2 I need to go 5 menus deep into the phone to change, and it changes not only the typing language but the interface as well. At least on a PC, for instance with Kmail, I get the language change in a single click.

On my Android phone, I'm very fond of Scandinavian Keyboard*, where a sideways swipe will toggle between Danish and English layouts and dictionaries. If I had a third, or non-Scandinavian language, I would be very frustrated (and tired of the menus!).

I wonder what happens when you want to type garbage or inane abbreviations: e.g. lol omgwtfbbq brb afk QQ.

A virtual hand probably materializes and slaps you.

Or how about quotations? Or parenthetical usage (as most online writers are keen to these days). Any symbol usage or numbers even. And for the biggy - international language support. Don't get me wrong. I find this fascinating to the point of "Doh! why didn't I think of that!" and I am really glad these 2 guys will be set for life financially very soon.

No where near the same. I can type without looking on my droid, it has a slide out keyboard and I am sure the keys have less stroke than that. On the screen I have to look to make sure the correct key is hit.

That was my reaction too, until I RTFA and realized that it is about phones with a touchscreen.
Summaries published by kdawson ought to have some kind of warning label, perhaps something like "Warning: This summary may not reflect the contents of the related article!"

I really can't believe it took so long for them to make something like that, I figured that the Android/iPhone keyboard would look at finger movements on each key to try to see if you pressed in the center like you wanted that letter or far to the side like you didn't and adjust accordingly much like this. But I guess not.

I often have other people read text messages aloud to me while driving and then dictate the response. I don't think this "amazing keyboard of the future" is for me. Next thing you know its either trying to account for someone else's waggling on your device, or its entirely unusable to someone else because it accounts for your waggling too much

True, but the phones I've been buying and have really enjoyed are the ones with a retractable keyboard that I only use the software keyboard if I'm too lazy to open up the phone to use the physical keyboard.

From all information that has been posted so far, there isn't any learning going on, except for custom words. The demonstrations they've given in person don't seem to have involved any prep either. So it would seem that it would work fine for your use case.

SwiftKey comes close, and has really good predictive text guesses that are usually right and will let me pick the exact thing I typed from the displayed list if I am typing a formerly unknown proper noun, for example, but it has some minor problems on my Nexus One that are really frustrating.

First, it tends to recognize keypresses near the lower right corner as backspace or enter and on the bottom row as space keys. It's very smart about character keys, but doesn't apply the same analysis to punctuation keys - it should be obvious that I wouldn't type three characters then space then a garbled half of the word - it's like it stops processing when it thinks I meant to hit a space bar. Duh. Blindtype seems to be smarter about that.

And second, it occasionally seems to fail to register or filters out the first 1-2 characters of a word I am typing if I type too fast. It's almost like it thinks they were so quick that they were mistakes. This is very frustrating and needs to be a configurable option for fast thumb typists like myself.

These are the reasons I'm looking forward to BlindType, or at least a fixed version of SwiftKey. It seems from the BlindType demo videos to at least address the first problem because it processes a set of keypresses as a batch. Whether I'm faster in practice with BlindType, SwiftKey or something else will remain to be seen.

And yes, I've tried Swype too and have it installed but don't use it everyday. It's nice and quite fast if I'm sitting there and looking at my screen, but I can't use it at all when I'm walking around or otherwise multitasking, which is why I'm so eager for BlindType.

I figured that the Android/iPhone keyboard would look at finger movements on each key to try to see if you pressed in the center like you wanted that letter or far to the side like you didn't and adjust accordingly much like this

The iPhone keyboard does actually take a lot of slop into account - if you type and shift to accidentally press other keys, the final word will correct based on keys that were almost where you hit, so that you don't have to be totally precise - the correction is pretty good on the i

The iphone does do corrections like you're saying, this just seems to do a better job of it. It also seems to be able to take the relative positions of all your keypresses and look for likely word mappings, even if the presses aren't anywhere near the actual keyboard position.

I wouldn't necessarily say it does a better job, but, rather, a different job. iOS's and, from what I understand, Android's built-in typing correction is designed around fixing spelling errors that are either the result of either misplaced fingers or outright misspelt words. BlindType does a much better job at handling the first problem at the expense of the second. If you read the article, they readily admit that it does not compensate well for words that the user misspells, since their algorithms work aro

It didn’t take long to make. It just took a while for somebody to conceive of it.

It’s not just checking finger position relative to the keys, either. It’s creating a “pattern” for the word that you type, then comparing that pattern to the patterns of known words and picking the best match. It is insensitive to scale or angle, so I’m guessing they’re using distances and angles between each tap. But how...

Hey, somebody has to be the first to do it. Like many profitable ideas, once presented it's a "duh", but someone had to have - and act on - that "duh" moment first.

I figured that the Android/iPhone keyboard would look at finger movements on each key to try to see if you pressed in the center like you wanted that letter or far to the side like you didn't and adjust accordingly much like this.

The iPhone keyboard's spelling correction takes finger drift into account. If you type an unrecognized word, but one/s

So I saw that it's got a patent pending, but don't recognize the format of the number, "61361522." Searching various dbs turns up nothing, and searching the web turns up only links back to the blindtype website. I really want to read this patent app.

I even searched for a few likely keywords, and found other, competing patent apps from other companies that purport to do something similar like this one [google.com]...

With the auto correction on most touch phones these days typing out complete words and sentences isn't really a problem once you get used to it. Especially hard to get past for me was wanting to hit backspace to correct it when you see mistakes, if you just plow on through it will usually fix the errors as you go. The problem is trying to type things that aren't complete words; email addresses, URLs, abbreviations, uncommon names and punctuation are all still very difficult to type and I don't see how thi

or needing to use words from two (or more) languages. On most systems, switching between dictionaries is rather roundabout -- if a dictionary even exists. And sometimes, a different keyboard layout is involved as well, although in that case I grant that a touchscreen has the advantage.

I think this kind of technology will be very useful when someone is dealing with several "wall-sized" displays (you move around the room, and the "keyboard" moves with you). Or think of a doctor typing stuff in a "chart".

If you had watched the video you would have noticed that this is mainly for mobile devices with no physical keyboard (read: zero tactile feedback). If you can take a mobile device with an on screen keyboard and type perfectly fine without looking at it you've got great motor skills. But what happens if it is slightly askew since your fingers are drifting slightly due to no tactile feedback? This fixes that and allows you to literally have no keyboard displayed on the screen and still type coherent words and sentences.

This isn't about learning to type on a real keyboard, it's about not needing one all together. Just know the basic layout and type away on the screen with no keyboard visual.

I'm seriously hoping for one to come out with a small camera you mount on the screen that watches your fingers and allows you to type without the need to project a keyboard. Spacing wouldn't matter anymore and you could just type away with your eyes closed and the software is smart enough to guess what word you meant.

So BlindType is pretty spot on. You can't see a keyboard (blind) and you can't feel one yet you're typing just fine.

Gee, that's new! Typists from decades ago were able to do just that. It was called "training" and "expertise".

Seriously, though, I expect two distinct problems with this:

1) How well will it handle "non-US slopiness"? Sloppy typing in Spanish (etc.) is quite different from typical english-language slopiness.

2) IT'S NOT A MODEL M KEYBOARD!!! There, I said it. I don't care for "the keyboard of the future" if the "keyboard from the past" is still alive and well and functioning nicely. Actually, make that "the keyboard from the past and present".:-P

So true, the craft of touch typing is seldom taught now, nearly everyone I hire recently have been two finger peck typists. While they are rather quick to type, you can see that few of them can compose while typing, they continually watch their index fingers and marvel that I can sit and type notes while engaging them in a meeting. Don't presume that I'm slamming them, I honestly am sad they were not taught proper touch typing skills and had the benefit of it through their careers. Knowing where the keys re

We actually just buy non touch typists typing tutor or some similar program and make them learn as part of their training. I've seen typists so slow they go look for things to copy and paste because its faster. Guess what kind of bug that leads to? Touch typing improves programming productivity, so its worthwhile asking an interviewee. When I think of all the things I learned in high school...well, that class probably is the one I use the most today. It should probably be mandatory before graduation.

Ready for the car analogy? You're bashing an article about an 8-ton dump truck that gets 30mpg by saying cars have been getting that for years. That's great, but what do you have to say that's relevant?

From what they've said, it appears to be language-independent. It's more to do about interpreting why you touched the screen in a certain place, so what language you're trying to type... it's just a different dictionary to match against.

Gee, that's new! Typists from decades ago were able to do just that. It was called "training" and "expertise".

Not looking at the "screen" (i.e. platen) while typing was generally due to the fact that you were reading off copy and typing it in to create another (typically more legible) copy. Now that computers can do copy and paste without a man-in-the-middle, we do what manual typists did when coming up with original work: we look at what we're typing so we catch our own mistakes. Reading off paper copy still occurs, but is less common. (Though sometimes you're reading off electronic copy that prevents copy & p

From reading the article, it sounds like all the software is doing is running next-cell error analysis against a dictionary. It's not exactly a keyboard so much as a slightly hopped-up spell checker. Assuming spell checking software doesn't already take QWERTY layout into account, that is.

The only way that I would get rid of my Model M Keyboard (and the stash of spares), would be if someone came out with a keyboard that had the same key action/feel as the old IBM 327x keyboards (which you could use as an anti-tank barrier). The Selectric keyboard had just about the same tactile feel as the 3270s, not quite the same, but close.
so yes -- I fully understand your feeling about your keyboard... they can have my Model M keyboard when they can pry it from my cold dead hands.

As standard spell check has already "corrected" my spelling into sending out requests for dates to my entire department, telling my boss I was flatulent and created numerous marital misunderstandings, I'm not sure I'm at all wanting to use this. Sometimes its best just to leave the typo so the reader can wonder what I meant, and not think I meant what the computer wondered.

My Android has come up with some of the most... amazing... interpretations of what I wrote. When I'm remembering to watch its "guesses" like a hawk, and never press the punctuation key until I'm sure it's picked the right word, it's actually nice. Sometimes I only have to type a few characters of a fairly long word.

But if I'm looking at what letters my finger is hitting... Yeah. Very whimsical results. Not as bad as the "Tablespoons" parody of "Jabberwocky as writ

Swype is great, and comes pre-loaded on many phones [Droid X, Galaxy S series of phones]. Only problem is that it adds to your dictionary anything that you tap out. That includes garbage from URLs like the end of a bit.ly link. Without manually deleting garbage like that, it starts to become unusable.

They should allow you to see and edit your dictionary like you can with T9.

So if you start to (touch)typing something it don't understand (foreing names or languages, i.e.) he relocates itself and "corrects" what you typed ? What could possibly go wrong?Also, the blindness part... in normal keyboards you should see the monitor, you get some input both from the physical keys and of course, from what is being displayed in the monitor, and that helps writing without watching the keyboard. But in a tablet (as shown in the article) you only have the monitor, you see the keyboard becaus

As this program appears designed to interpret what you want to say based on actual, English-language words, it would be interesting to see how it would handle poor spellers.

Personally, I am probably somewhat of a spelling nazi, as I cannot stand how inept some persons (seemingly the younger generation--get off my lawn!) appears to be at spelling. If this is released, I would imagine that poor spellers would either (a) be forced to finally learn how to spell (again, get off my lawn!), or regrettably more

As this program appears designed to interpret what you want to say based on actual, English-language words, it would be interesting to see how it would handle poor spellers.
Personally, I am probably somewhat of a spelling nazi, as I cannot stand how inept some persons (seemingly the younger generation--get off my lawn!) appear to be at spelling. If this is released, I would imagine that poor spellers would either (a) be forced to finally learn how to spell (again, get off my lawn!), or regrettably more li

This would work fine if the typist's fingers are accurate in the sense that they move the correct fingers in the right sequence. However, not all typos are of this variety-- using my habits as an example, sometimes I type out of seuqence (faster fingers tend to beat the weaker fingers to the punch), sometimes I dont put enough pressue on the keys/touchscreen for them to register, sometimes I get the finger right, but on teh wuong hand. (I'm also horrible with grammar, and tend to write in free verse more th

...sometimes I type out of seuqence (faster fingers tend to beat the weaker fingers to the punch), sometimes I dont put enough pressue on the keys/touchscreen for them to register, sometimes I get the finger right, but on tehwuong hand...

If it works as well as depicted, then I hope it gets incorporated in the iOS. If it's not incorporated into the iOS then it would only be marginally useful. Queue the Android vs. iOS comments now... but I love my iPhone anyway. If it works, Apple ought to buy it from them and hire them.

When you look at things like Swype and BlindType, it really shows the power of having a device with a virtual keyboard. It lets you have data entry mechanisms that can be a lot easier than working very physically small keys.

It also shows the limitation of Apple's approach in that a user cannot replace the system keyboard (unless they jailbreak), although I think you'd have to root Android to replace the keyboard system wide which is similar in nature. Third party apps could individually include the keybo

Every time I see tools like this I wonder if QWERTY should still be relevant. "We're using a QWERTY keyboard, because everyone knows how to use a QWERTY keyboard." IIRC QWERTY was originally designed to keep typebars in manual typewriters from jamming together, and its kept on keeping on through Selectrics to PCs to today. It's all so roman-chariots-and-railroad-gauges (why abandon a good metaphor even if its not really true?) Are there finger-based text-inputting tools for touchscreens and/or smartphones t

I'm generally fine with QWERTY on the desktop/laptop keyboards as I didn't find Dvorak to be significantly better, but QWERTY is completely suboptimal for input when you're using one finger/stylus or, say, two thumbs. I've played around with the FITALY [wikipedia.org] keyboard on my WM PDA, and although there was learning process, it did seem to be a somewhat faster. Also the Metropolis [ibm.com] layout, based on a Monte Carlo method of the same name, looked very promising, but I don't recall ever finding an implementation for WM,

Steve Jobs' primary directive, (operating on the subconscious layer. . , I hope), is to dumb down humans so that they degrade into malleable monied children who don't think about any reality other than the bloopy, candy-coated dream world presented to them by Apple.

Part of that master plan was to surgically remove user interactivity from the internet, so that people can only fingerpaint, play games and communicate in twittery bursts of retard-speak.

Let's think of how they might have designed the algorithm for this. In the videos, it looks like it is treating one word at a time, so let's consider that scenario here as well. I would define the problem as assigning to every possible word the probability that it is the word the user intended. I would use Bayes' theorem [wikipedia.org] to achieve this.

First, assume a prior probability distribution over all words. Words not in our dictionary, and words of the wrong length, we give probability zero. The remaining words can be assigned equal probability or, better, a probability proportional to their frequency in the language. If you want to be fancy, you could have more sophisticated models that knows which words are likely to come after others and such things.

Second, for each candidate word, what is the probability that the user would tap the screen as they just did? A model for this could be that the location of each tap is drawn from a Gaussian probability distribution centred at the intended letter with a known standard deviation and that each tap's deviation from its target is independent of the others'.

Finally, Bayes' theorem states that the posterior probability (the one we want to calculate) of each word is the word's prior probability (from step one) times its likelihood (the probability of step two).

To implement the arbitrary position, orientation and size of the keyboard, we redefine the problem from finding the probability of each candidate word to finding the probability of each tuple (intended word, keyboard position, keyboard orientation, keyboard size). Make it simple; have each element of this tuple to be independent of the other and use flat distributions for all keyboard parameters. To choose the most likely word, you could either pick the word of the most likely tuple or, more correctly, for each candidate word, integrate over all possible keyboard parameters (weighted with their prior probabilities) to get the probability of that word. Likewise, you could introduce the standard deviation of the taps as another element of the tuple, with its own prior distribution.

I suspect this method is a bit to heavy on computation cost and power consumption, so if you cannot find a clever way to do it fast, you might have to cut corners in the rigor (or do something completely different).

...bayes...
And in the end you end up with something looking like chinese: you type (or rather, draw) some quick symbol for a very common word, a more complex symbol for a rarer word (and you need to define and then memorize them all), and for any 'foreign' word, you spell it the old way. Asians are used to type a lot faster than we do on cell phones. And they also take notes faster on paper (a chinese student friend in the US would write down the physics courses in chinese, which allowed her to write down the entire course, while the US student could only grad words here and there).

I write 4 different languages and the nifty spell correction thingies always need to be told which language I am currently writing in. And you also can forget about messages written in multiple languages.

Hence for me a keyboard is good when I can accurately type what I need.