Dr. C. Scott Ananianhttp://cananian.livejournal.com/
Dr. C. Scott Ananian - LiveJournal.comThu, 03 Oct 2013 07:52:59 GMTLiveJournal / LiveJournal.comcananian123187personalhttp://l-userpic.livejournal.com/11312513/123187Dr. C. Scott Ananianhttp://cananian.livejournal.com/
100100http://cananian.livejournal.com/69096.htmlThu, 03 Oct 2013 07:52:59 GMTRust is not fasthttp://cananian.livejournal.com/69096.html
<p>There are plenty of safe high-level languages in the world; JavaScript, for example. <a href="https://en.wikipedia.org/wiki/Rust_(programming_language)">Rust</a> is different: it's supposed to be safe <b>and fast</b>.</p>
<p>But Rust is slow. (And its type system hates you.)</p>
<p>Rust is slow because there is lots of hidden indirection ("smart dereferencing") and other hidden costs (ref counting, etc). In low-level C code I can look at a line of code and know roughly how many (slow) memory accesses are present. Not so in Rust.</p>
<p>Further, Rust's type system leads to extra unnecessary copying, just to get your code to compile without massive refactoring of the standard library. When writing <a href="https://github.com/cscott/rusty-turtle">rusty-turtle</a> I found myself having to add <code>~</code> or <code>@</code> pointers to my types (forcing extra layers of dereferencing) just to work around the type system. Further, the APIs have a genericity problem: there are lots of duplicate methods, since <code>&amp;</code>-pointers aren't truely generic/orthogonal. (And you will find yourself duplicating methods in your own APIs as well, in order to be able to pass in parameters with different reference types -- or else just throw up your hands and wrap an extra <code>@</code> layer around everything.)</p>
<p>The ownership type system also fights against typical APIs like <code>find_and_insert</code> for maps, since you don't know (before you do the find) whether or not you will be giving up ownership of the parameter (in order to do an insert). So you just copy the inserted value, always! Cycles are cheap, right?</p>
<p>Rust is also slow because it is not built to be parallel. The language is <i>concurrent</i>, but this is a word game: in the past few years the terms have been redefined such that "concurrent" is (roughly) non-blocking cooperative multitasking (such is implemented by <a href="http://nodejs.org/">node.js</a> and <a href="http://www.gnu.org/software/pth/">GNU Pth</a>), and "parallel" is reserved for actually doing more than one thing simultaneously (whether on separate CPUs or separate cores of a single CPU). Rust's memory model doesn't help: there is no shared memory, and ownership types make fork/join parallelism difficult. All inter-task communication is explicit message passing, with the races that entails. (Perhaps I'm spoiled: the <a href="https://en.wikipedia.org/wiki/Cilk_(programming_language)">Cilk</a> <a href="http://supertech.csail.mit.edu/cilk/manual-5.4.6.pdf">work-stealing nano/microscheduler</a> is my benchmark for speed.)</p>
<p>Some possible improvements:</p>
<ul><li>Get rid of smart dereferencing; make it clear when performance is impacted by memory references.
</li><li>Fix bugs with small objects/ABI limitations to avoid unnecessary parameter wrapping.
</li><li>Make <code>&amp;</code> pointers truely generic (or invent a new pointer which is)
and do template expansion/method splitting to generate the proper specialized version of the method automatically (although this will exacerbate existing problems with code caching).
</li><li>Better support fast refcounting/fast gc (update coalescing, generations).
</li><li>Support fork/join parallelism and work-stealing.
</li></ul>
<p>This post is written from my experience with Rust in May 2013. Some of these problems are known, and some may eventually be fixed. But it makes me wonder what the language is really supposed to be good at. There are already plenty of slow safe languages.</p>http://cananian.livejournal.com/69096.htmlrustprogrammingpublic0http://cananian.livejournal.com/68747.htmlThu, 03 Oct 2013 06:53:22 GMTJavaScript in asm.js (and a little rust)http://cananian.livejournal.com/68747.html
<p>Over on twitter, Tim Caswell mentioned, <a href="https://twitter.com/creationix/status/385471322408493056">"I think high-level scripting language on top of something like rust.zero would make for an amazing OS."</a> and that <a href="https://twitter.com/cscottnet/status/385540671379365888">set me off a bit</a>. Twitter isn't a great place to write a reasoned discussion of programming languages or implementation strategies, so let's take a shot at it here.</p>
<p>As I've <a href="http://cananian.livejournal.com/tag/turtlescript">written about on this blog</a>, I've been tinkering for years with TurtleScript, a small learnable JavaScript subset in the spirit of Alan Kay. Over in that twitter conversation <a href="https://twitter.com/littlecalculist/status/385464517120884736">David Herman</a> mentioned <a href="https://github.com/cscott/rusty-turtle">rusty-turtle</a>, my TurtleScript bytecode interpreter written in Rust. The <code>rusty-turtle</code> codebase includes a REPL which runs TurtleScript's tokenizer, parser, bytecode compiler, and standard library (all written in TurtleScript) through the Rust interpreter. It's quite cute, and I implemented much more of the JavaScript semantics than I strictly needed to (with the side-effect that the behaviors in the <a href="https://www.destroyallsoftware.com/talks/wat">JavaScript wat</a> talk now appear quite sane and sensible to me).
</p>
<p>I wrote <code>rusty-turtle</code> as a personal warm-up: I was considering taking a job with the fine folks at Mozilla (<a href="https://en.wikipedia.org/wiki/OLPC">OLPC</a> having run out of money again) and wanted to understand the technology better. I described a number of further research projects I thought would be interesting to pursue <a href="https://github.com/cscott/rusty-turtle/blob/master/README.md#other-research-ideas">in the rusty-turtle README</a>, including <a href="https://en.wikipedia.org/wiki/Cilk">cilk</a>-style fork/join parallelism or transactional memory support (the latter being the subject of <a href="http://cscott.net/Publications/phd-thesis.pdf">my thesis</a>), and a JIT backend using rust's llvm library bindings.</p>
<p>But the true turtles-all-the-way-down approach would be to rewrite the backend using <a href="http://asmjs.org/">asm.js</a>, which can be trivially JIT'ed (using llvm bindings). Then you've have an entire system from <span style="white-space:nowrap">(pseudo-)assembly</span> code up, all written in a consistent manner in JavaScript. To that end, I wrote <a href="http://turtlescript.github.cscott.net/asmjs.html">single-pass type-checker/verifier</a> for <code>asm.js</code> in TurtleScript, discovering lots of issues with the spec in the process (sigh). (I justified this as more "Mozilla interview preparation"! Besides, it was fun.)</p>
<p>Tim Caswell, to finally answer your question: I think that this "JavaScript all the way" system would make an amazing OS. The Rust stuff is just a distraction (except as needed to bootstrap).</p>
<p>In the next post I'll rant a bit more about Rust.</p>
<p>ps. In the end I over-prepared (!): Mozilla's feedback was that I seemed to "know too much about <a href="https://en.wikipedia.org/wiki/Rust_(programming_language)">Rust</a> to work on <a href="https://en.wikipedia.org/wiki/Servo_(layout_engine)">Servo</a>" (Mozilla's experimental web layout engine, written in Rust). Mozilla seems to have reached that awkward size where it can not longer hire smart people and find places for them to contribute; new hires need to fit into precise job descriptions a priori. That sort of place is not for me.</p>http://cananian.livejournal.com/68747.htmlrustjavascriptturtlescriptprogrammingpublic0http://cananian.livejournal.com/68088.htmlThu, 22 Nov 2012 19:29:56 GMTSDR 0.7http://cananian.livejournal.com/68088.html
<p>Happy Thanksgiving! Here's <a href="http://cscott.net/Projects/SDR/">SDR 0.7</a> to celebrate the holiday. Version 0.7 incorporates a number of dance engine refactorings completed shortly after the <a href="http://cananian.livejournal.com/59475.html">previous release promised them</a>, as well as (more recently) a number of new call and concept definitions (and <a href="http://cscott.net/Projects/SDR/sdr-latest/doc/net/cscott/sdr/doc-files/">test cases</a>) inspired by the <a href="http://www.lynette.org/recordings/c4teach/index.html">C4 calls I am currently studying</a>. I also updated Google App Engine and Google Web Toolkit to the latest versions for the <a href="http://square-dance.appspot.com/">web app</a>, although <a href="http://jmonkeyengine.com/">jMonkeyEngine</a> is still stuck at 2.0 &mdash; we might get an Android version of SDR if I manage to rebase to jMonkeyEngine 3.0 for the next release.</p>
<p>Breathing the square properly is still a challenge. Other square dance programs only treat starting and ending formations, but SDR has to consider all of the intermediate positions along the dancers' paths. This leads to some very unusual formations being breathed. As mentioned in the notes for the <a href="http://cananian.livejournal.com/59475.html">last release</a>, SDR formulates breathing as a solution to a mixed integer linear programming problem&mdash;but there are still a few bugs lurking which cause the constraint solver to blow up from time to time (especially from columns, for some reason).
Hopefully I'll be able to dig into this for the next release.</p>http://cananian.livejournal.com/68088.htmlsdrsquare dancepublic0http://cananian.livejournal.com/67703.htmlSun, 21 Oct 2012 22:51:05 GMTReading Project Talk (and slides)http://cananian.livejournal.com/67703.html
<p>An unruly tag team of OLPC folks gave a long talk on <a href="http://wiki.laptop.org/go/Literacy_Project">the Literacy Project</a> today for attendees at this year's <a href="http://www.olpcsf.org/CommunitySummit2012">OLPC-SF Community Summit</a>. It was streamed live on Ustream: <a href="http://www.ustream.tv/recorded/26344794">Part 1</a> (Matt Keller, Richard Smith), <a href="http://www.ustream.tv/recorded/26345580">Part 2</a> (Richard Smith, Ed McNierney, C. Scott Ananian, Chris Ball, questions from the audience). We've posted the slides: <a href="http://dev.laptop.org/~cscott/literacy-public/Keller-Reading.ppt">Matt Keller</a>, <a href="https://docs.google.com/presentation/d/18S9eSMMZeCxiCRxXZl4VgVAxQFTLIWQCOkO2GVGjk8g/edit">Richard Smith</a>, <a href="https://docs.google.com/presentation/d/1CSJBu_11YKc_3H20Zi5maDPym9tcYo7gdmK9Hj1Wc30/edit">C. Scott Ananian</a>.</p>
<p>You can try out some of the apps mentioned in the talk. <a href="http://nell-balloons.github.cscott.net">Nell's Balloons</a> and <a href="http://nell-colors.github.cscott.net">Nell's Colors</a> will run in any reasonably-recent Google Chrome or Mozilla Firefox browser. They will also run as a <a href="https://marketplace.mozilla.org/developers/">Firefox webapp</a> on Android devices, using the <a href="http://nightly.mozilla.org/">latest Firefox nightly for Android</a>. For deployment we use a slightly-tweaked build of Firefox (adding expanded webapp storage quotas and the ability to use plugins from inside webapps), and a <a href="https://github.com/cscott/intent-addon">custom plugin</a> to hook up the <a href="http://funf.org/">Funf logging framework</a>. Source code is available on github: <a href="http://github.com/cscott/nell-balloons">nell-balloons</a>; <a href="http://github.com/cscott/nell-colors">nell-colors</a>. In addition, Chris Ball's "Matching" app for Android is available: <a href="http://dev.laptop.org/~cscott/literacy-public/Matching-1.4.0.apk">apk</a>; <a href="http://dev.laptop.org/git/users/cjb/android-matching/">source</a>.</p>http://cananian.livejournal.com/67703.htmljavascriptsugarolpcnellandroidpublic3http://cananian.livejournal.com/67390.htmlWed, 22 Aug 2012 16:29:11 GMTThe Importance of Sensing Distancehttp://cananian.livejournal.com/67390.html
<p>At <a href="http://www.dimeb.de/idc2012">IDC 2012</a> in June, Arnan Sipitakiat and Nusarin Nusen discussed how they are using <a href="http://dx.doi.org/10.1145/2307096.2307108">Robo-Blocks</a>&mdash;a turtle robot and &ldquo;tangible <a href="http://wiki.sugarlabs.org/go/Activities/Turtle_Art">Turtle Blocks</a>&rdquo;&mdash;to teach problem solving and debugging skills to 5- through 12-year-olds.</p>
<p>One of the things I learned from their presentation was that children had difficulty reasoning about relative angles. The Robo-Blocks robot does not have any distance feedback on its motors, so &ldquo;the result of a program will change depending on the roughness of the surface and the battery level of the robot.&rdquo; They worked around this issue by developing a protractor tool to guide the children's reasoning about the relationship between the (arbitrary) numbers entered and the amount the robot turned, but some kids still had difficulty. The researchers &ldquo;often had to insist on trying the protractor&rdquo; and &ldquo;some children preferred to keep increasing the turn amount even if a small decrease would have fixed the problem&rdquo; resulting in programs that had the robot making multiple complete rotations before setting off in the correct direction. The kids were also dissatisfied with polygon-drawing tasks (&ldquo;<a href="http://en.wikipedia.org/wiki/Turtle_Geometry">turtle geometry</a>&rdquo;) because the inaccuracies of open-loop control of the robot means that the polygons often didn't close completely, and &ldquo;[t]his small error turned out to be unacceptable to children.&rdquo;</p>
<p>So I designed the XOrduino turtle robot from the start to have distance sensors so that it can do accurate turns with closed-loop control. Here's a little video showing how they work in the current (A1.5 / B1) revision of the board:</p>
<lj-embed id="18" />
<p>Some bonus pictures of the speed sensor on the workbench:</p>
<ul>
<li>The robot on the workbench with probes.<br />
<a href="http://ic.pics.livejournal.com/cananian/123187/2055/original.jpg"><img alt="Speed sensor test setup" border="0" title="Speed sensor test setup" src="http://ic.pics.livejournal.com/cananian/123187/2055/300.jpg" width="300" /></a>
</li>
<li>Signal from the motor speed sensor. 5ms/div .5v/div. Motor is running at full speed, unloaded. Two dips are seen: the larger is from a piece of white paper glued to the rim of the gear; the smaller is from a spot made with a white paint marker (the paint didn't stick very well). White-out worked much better (as shown in the video above).<br />
<a href="http://ic.pics.livejournal.com/cananian/123187/1814/original.jpg"><img alt="Oscilloscope trace" border="0" title="Oscilloscope trace" src="http://ic.pics.livejournal.com/cananian/123187/1814/300.jpg" width="300" /></a>
</li>
<li>Oscilloscope settings<br />
<a href="http://ic.pics.livejournal.com/cananian/123187/1575/original.jpg"><img alt="Oscilloscope settings" border="0" title="Oscilloscope settings" src="http://ic.pics.livejournal.com/cananian/123187/1575/300.jpg" width="300" /></a>
</li>
</ul>http://cananian.livejournal.com/67390.htmlsugarolpcxorduinoarduinopublic2http://cananian.livejournal.com/67093.htmlSat, 18 Aug 2012 05:13:16 GMTXO Turtle Bot drives aroundhttp://cananian.livejournal.com/67093.html
<p>Here's a first look at an XOrduino Turtle bot driving around:</p>
<lj-embed id="13" />
<p>I've checked out all of the functionality on the A1.5 board except the step-up voltage regulator now. I'm optimistic the B1 boards (being made now in Taipei) will be clean.</p>
<p>It will be great when we've got lesson plans written up so kids can learn how to control the bot with Turtle Blocks, and play with the different possible behaviors. Instead of just bumping around ("like a Roomba, except it doesn't vaccuum" a friendly 6-year-old beta-tester told me), you can trace patterns you design, or use the Scratch Sensor Board sensors to make the robot "afraid of sound", "attracted to light", or add your own sensors and behaviors.</p>http://cananian.livejournal.com/67093.htmlolpcxorduinopublic0http://cananian.livejournal.com/66895.htmlWed, 15 Aug 2012 20:34:02 GMT"Hello, World" from XOrduino/XO Stickhttp://cananian.livejournal.com/66895.html
<p>Here's a quick look at the next versions of the XOrduino and XO Stick boards. These were assembled from a small quantity of "pre-B1" boards I had made at <a href="http://batchpcb.com">BatchPCB</a>.</p>
<lj-embed id="11" />
<p>I've uploaded some more pictures to <a href="https://plus.google.com/photos/109541946294746531763/albums/5772322118805954321">the XOrduino album</a> as well.</p>
<p>Here's a little table relating the board versions pictured with those I've previously discussed.</p>
<table>
<tr><td>Build</td><td>XOrduino</td><td>XO Stick</td></tr>
<tr><td>A1</td><td>v4</td><td>v5</td></tr>
<tr><td>This video</td><td>v6</td><td>v7</td></tr>
<tr><td>B1</td><td>v8</td><td>v14</td></tr>
</table>
<p>B1 is "the next run" of boards, already released to the fab house but not yet in hand.</p>
<p>The big feature added to XOrduino after A1 was a motor driver, to allow using the XOrduino as a Turtle robot. The big feature added to XO Stick after A1 was the shield form factor, allowing it to ride piggy back on the XOrduino. This makes it easier to share a single turtle robot with a classroom: there may be only one XOrduino robot base, but each student can have their own low-cost XO Stick "brains". They can take turns snapping their brains on top of the base to drive it.</p>
<p>I haven't finished testing all the functionality of these new boards yet, but it looks like I haven't made any major mistakes! Help still wanted with software, documentation, etc; send email to xorduino@gmail.com if you're interested.</p>http://cananian.livejournal.com/66895.htmlsugarolpcxorduinoarduinopublic0http://cananian.livejournal.com/66654.htmlThu, 02 Aug 2012 06:39:39 GMTXO Bot joins the XOrduino and XO Stickhttp://cananian.livejournal.com/66654.html
<p>Free things first: I've got parts for 20 copies of the "Mk I"
<a href="http://cananian.livejournal.com/66129.html">XOrduino and XO Stick</a>. <b>I'm mailing them out for free</b> (!) in exchange
for your development help. Send me an email at
xorduino@gmail.com describing what you'd like to do with the
XOrduino/XO Stick, and your full mailing address. Best 20 or so get kits.</p>
<p align="center"><img alt="XOrduino A1" border="0" title="XOrduino A1" src="http://ic.pics.livejournal.com/cananian/123187/847/original.jpg" /></p>
<p>Here are some of the projects which you might be able to help with:</p>
<ul>
<li><b>Assemble an XOrduino / XO stick with an 8-12 year old</b> and document
the process. What parts were tricky to solder? Where did polarity
matter? How much of the function of the different devices did you
find worth explaining? Photos or video of children assembling the device would be great for future publicity, with their permission. (We're not crazy: kids can <a href="http://www.youtube.com/watch?v=Pus_fA1Tv9w">repair XOs</a> and <a href="http://www.sparkfun.com/news/555">solder</a>.)</li>
<li><b>Test different configurations</b> of the boards. What are the
fewest components necessary for a functional XO Stick?
What capacitors are really needed? What's the smallest number of
components needed to get the arduino IDE to talk to the XOrduino?
Then add the components for the <a href="http://info.scratch.mit.edu/Sensor_Boards">Scratch Sensor Board</a> functionality,
and test that with this <a href="https://github.com/osbock/ScratchSensors">Arduino sketch</a> (some minor porting required). Try out whatever
Arduino shields/old Arduino code you have lying around, and see if there
are any gotchas there. Document it all, take photos and video, let me know about bugs and pitfalls.</li>
<li><b>Write some killer <i>education</i> apps</b>! These
boards are meant specifically for teaching kids&mdash;take the <a href="http://wiki.sugarlabs.org/go/Activities/Turtle_Art/Using_Turtle_Art_Sensors">Turtle Art with Sensors</a> ideas as examples, and write up some
lessons to teach science. Or take inspiration from the old school
"fun with electronics" kits from Radio Shack and recreate some of the
popular standbys: a burglar alarm for kids' tree fort,
a light-sensitive alarm they can hide in their sibling's drawer,
etc. Or a document how to program a robot (more on the robot below) with simple emergent behaviors&mdash;avoiding walls, turning toward light, fleeing loud sounds, etc. The <a href="https://www.sparkfun.com/products/11074">Cubelets</a> examples may give you ideas. Take photos and video.</li>
<li><b>Arduino support for the XO Stick</b>. There are a number of projects
which add support for the ATtiny85 and friends to the Arduino IDE (for example, <a href="http://provideyourown.com/2011/arduino-program-attiny/">this one</a>).
Ideally we'd like to make the XO Stick as Arduino-compatible as
possible, so we can reuse the excellent Arduino IDE, etc.
This involves (a) porting an arduino-compatible bootloader (like
<a href="http://embedded-creations.com/projects/attiny85-usb-bootloader-overview/avr-jtag-programmer/">usbAspLoader-tiny</a>), as well as (b) porting the Arduino libraries to match the
pinout/peripherals of the ATtiny85 and ATtiny861 (<a href="http://hlt.media.mit.edu/?p=1695">this page</a> is a good start).</li>
<li><b>Program an XO Stick from an XOrduino</b> and vice versa. Ideally we'd
like to bootstrap the initial chip programming, so that one
programmed XOrduino (or XO Stick) can be used to put the initial
bootloaders on the others. For technical reasons the XO Stick
is probably best as a "clone tool": without interacting with the
USB bus it would just copy its internal memory to another
XO Stick. The XOrduino is a little easier, just a matter of
adapting the existing <a href="http://arduino.cc/en/Tutorial/ArduinoISP">Arduino sketches and documentation</a>.</li>
<li><b>Debrick an XO</b> from the XO Stick. The XO Stick can talk to the
EC programming bus to recover a bricked XO; it can probably also
<a href="http://wiki.laptop.org/go/SPI_FLASH_Recovery">reprogram OpenFirmware</a>. We need to write a bit of code to make it
pain-free and document the process. This would make the XO Stick a useful repair accessory for XO deployments.</li>
<li><b><a href="http://catroid.org">Scratch</a>/<a href="http://wiki.sugarlabs.org/go/Activities/TurtleArt">Turtle Blocks</a> support</b> for the <a href="http://wiki.sugarlabs.org/go/Activities/Turtle_Art/Plugins#Arduino">XOrduino</a> and/or <a href="http://youtu.be/qfU_f8HF-78">XO turtle bot</a> (see below).</li>
</ul>
<p>Here's the exciting part two: I'm already working on the XOrduino and XO Stick "Mk II". The latest schematics/boards
are in github (<a href="https://github.com/cscott/xostick">xostick</a>, <a href="https://github.com/cscott/xorduino">xorduino</a>). The kits I'll be sending out this week correspond to the "A1" tag in those repositories; the "Mk II" revision is on the master
branch.</p>
<p>The XO Stick gets a minor change with big implications: instead of
using a 20-pin header matching the ATtiny861 pinout, I've widened the
board to <b>give the XO Stick a standard Arduino shield connector</b> (and some
prototyping area). This opens the way for a port of the Arduino IDE
(mentioned above), but it also means that the XO Stick can be mounted on top of
an XOrduino. In a cost-conscious classroom environment, this allows a teacher to buy/make one copy of the XOrduino with all of its
fancy peripherals (scratch sensors, robot support) and then give each student a copy of the
cheaper XO Stick. The students share the XOrduino and swap out their XO Stick "brains" on top to control it or use its peripherals. Mating the two
boards also makes it straightforward to program an XO Stick from
an XOrduino, or to use the XO Stick's prototyping area to hack together
a shield for the XOrduino.</p>
<p>The XOrduino gets a more exciting feature (hinted at above) -- enough peripherals to become the <b>XO Turtle Bot</b>! This is a very
low-cost <a href="http://en.wikipedia.org/wiki/Turtle_%28robot%29">turtle robot</a> based on a <a href="http://www.tamiyausa.com/product/item.php?product-id=70097">Tamiya motor assembly</a>. All of the
extra robot components are optional&mdash;you can populate just the parts you want&mdash;but a classroom can now make
their XOrduinos (or XO Stick + XOrduino base) into standalone turtle
robots, controlled by Scratch, Turtle Art, or Arduino code. The XO Turtle Bot revision adds a motor driver, two bump switches, a simple 3-cell
power supply, and rotation sensors for the motors to the XOrduino. (Arnan Sipitakiat and Nussarin Nusen in their Robo-Blocks presentation for <a href="http://dimeb.informatik.uni-bremen.de/idc2012/program.htm">IDC 2012</a> explained that children find "turn for two seconds" hard to understand; we include motor sensors so that we can "turn 90 degrees" instead.) And of course because the robot is based on XOrduino, you can add
whatever other sensors you like and write arduino/Scratch/Turtle Blocks code for it.</p>
<p align="center"><img alt="XOrduino A1 board on top of Tamiya Twin Motor Gearbox." border="0" title="XOrduino A1 board on top of Tamiya Twin Motor Gearbox." src="http://ic.pics.livejournal.com/cananian/123187/1350/300.jpg" width="300" />&nbsp;&nbsp;&nbsp;<img alt="XOrduino A1 plugged into USB port; prototype XO Turtle Bot in the background." border="0" title="XOrduino A1 plugged into USB port; prototype XO Turtle Bot in the background." src="http://ic.pics.livejournal.com/cananian/123187/1278/300.jpg" width="300" /></p>
<p>I'm excited about the potential of low-cost robotics and the Arduino platform for
education. If you are, too, let me send you a kit so you can help
out!</p>http://cananian.livejournal.com/66654.htmlsugarolpcxorduinoandroidarduinopublic1http://cananian.livejournal.com/66444.htmlTue, 12 Jun 2012 06:39:52 GMTNell's Tinkrbook in Omohttp://cananian.livejournal.com/66444.html
<p>
This week I will be at the 2012 <a href="http://dimeb.informatik.uni-bremen.de/idc2012/">Interaction Design and Children
conference</a> in Bremen, Germany. I will be presenting the
<a href="http://cananian.livejournal.com/66008.html">Growing Up With Nell</a>
paper as well as discussing the OLPC Foundation's literacy pilots in
Ethiopia.
</p>
<p>
The <a href="http://wiki.laptop.org/go/Literacy_Project">Literacy
Project</a> is a collaboration between four different groups (as
alluded to by the title of this post): the One Laptop per Child
Foundation (&ldquo;Nell&rdquo;), the MIT Media Lab
(&ldquo;Tinkrbook&rdquo;), the School of Education, Communication and
Language Sciences at Newcastle University, and the Center for Reading
and Language Research at Tufts University (&ldquo;Omo&rdquo;).
The goal is to reach children even further from educational
infrastructure than OLPC has ventured to date. In particular, the
Ethiopia pilots are complete child-led bootstraps, attempting to teach
kids to read English (an official language of Ethiopia) who neither
speak English nor read in any language yet. There are no teachers in the village,
and no literate adults either.
</p>
<p>Adapting Nell to this environment has some challenges: how do we
guide students through pedagogic material with stories if they don't
yet understand the language of the stories we want to tell? But the
essential challenge is the same: we have hundreds of apps and videos
on the tablets and need to provide scaffolding and guidance to the
bits most appropriate for each child at any given time, just as Nell seeks to
guide children through the many activities included in Sugar. In the
literacy project there is also a need for automated assessment tools: how can we tell that the project is
working? How can we determine what parts of our content are effective
in their role?</p>
<p>I'll write more about the Literacy Project in the coming weeks.
As we've started to get data back, some of the lessons learned are
familiar: kids do the strangest things! They learn how to do things
we never knew they could do (or meant for them to) and often are
motivated by pleasures which surprise us. For example, one app we
deployed had a sphere which deflated with a sort of farting noise when
the child picked the wrong answer. It turns out that the kids liked making the farting noise
much more than they liked the response to the correct answer! Obvious
in retrospect, but the lesson reminds us why we are pursuing an incremental development and data collection approach. Happily, the hardware itself
has been a success: low hardware failure rates, solar powered charging
is successful (although they prefer to charge the devices during the
middle of the day; we'd expected them to do so overnight from storage batteries charged during the day), and they've
mastered the touch interface very quickly on their own. The pilots
have been running since February, and the kids are still very engaged
with the content. So far, so good!</p>
<p align="center">
<img alt="Smiling boy in Ethiopia Literacy Pilot" border="0" title="Smiling boy in Ethiopia Literacy Pilot" src="http://ic.pics.livejournal.com/cananian/123187/387/300.jpg" height="225" />&nbsp;<img alt="Two girls in Ethiopia Literacy Pilot" border="0" title="Two girls in Ethiopia Literacy Pilot" src="http://ic.pics.livejournal.com/cananian/123187/749/300.jpg" width="300" /></p>http://cananian.livejournal.com/66444.htmlsugarolpcnellpublic0http://cananian.livejournal.com/66129.htmlSat, 09 Jun 2012 16:06:32 GMTIntroducting the XOrduino! (and XO Stick)http://cananian.livejournal.com/66129.html
<p>I banged out two <a href="http://freedomdefined.org/OSHW">open hardware</a> designs this week, designed for use with the <a href="http://en.wikipedia.org/wiki/OLPC_XO-1">OLPC XO</a> laptops.</p>
<p>The first is the <b>XOrduino</b>, a stripped down low-cost
<a href="http://arduino.cc/">Arduino</a>-compatible board that plugs right into the XO's USB ports.
But wait, there's more: it's also compatible with the <a href="http://info.scratch.mit.edu/Sensor_Boards">Scratch Sensor
Board</a>, so you can use this device to control <a href="http://scratch.mit.edu/">Scratch</a> (and <a href="http://wiki.sugarlabs.org/go/Activities/TurtleArt#Arduino">Turtle Art</a>, once <a href="http://firmata.org/">Firmata</a> is ported).
It should be compatible with the <a href="http://en.wikipedia.org/wiki/Arduino#Software">Arduino IDE</a> and all <a href="http://arduino.cc/en/Main/ArduinoBoardLeonardo">Arduino
Leonardo</a>-compatible shields.</p>
<p>The board uses mostly through-hole parts, with one exception, and
there are only 20 required components for the basic Arduino
functionality, costing about $5 (from digikey, quantity 100). It is
reasonable for local labor or even older kids to assemble by hand.</p>
<p>It's open hardware: <a href="http://en.wikipedia.org/wiki/Eagle_(program)">Eagle</a> design files are <a href="https://github.com/cscott/xorduino">on github</a> (<a href="https://github.com/cscott/xorduino/blob/A1/XOrduino-sch.pdf?raw=true">schematic PDF</a>,
<a href="https://github.com/cscott/xorduino/blob/A1/XOrduino-brd.pdf?raw=true">pcb PDF</a>). I expect to have a small number of boards in a few weeks; let
me know if you'd like one in exchange for help with hardware and
software bring-up. Schematic and layout review also appreciated (I did the PCB routing late at night under time pressure leaning heavily on autoroute, it's certainly not the prettiest). And feedback from Arduino and Arduino shield hackers would also be welcome.</p>
<p>If $5 per student is too much money, there's also the <b>XO Stick</b>, my
second board. It's based on the <a href="http://www.sparkfun.com/products/9147">AVR Stick</a> using the <a href="http://www.atmel.com/devices/attiny85.aspx">ATtiny85</a> processor and costs only
$1/student. It's not quite as user-friendly as the Arduino-compatible
board, but it can also be used to teach simple lessons in embedded
electronics. For $0.12 more you can populate an <a href="http://www.atmel.com/devices/ATTINY261A.aspx">ATtiny261A</a> (though a '461 or '861 would be better) and get 13 I/O ports; this variant should be powerful enough to program other XO Sticks and perform XO maintenance tasks (accessing the serial
console, debricking a laptop via SPI flash). The XO Stick is even easier for a
kid to assemble themself: only 8 required components, all through-hole.
(Sadly, my desire to shave every penny off the cost of this design meant that I couldn't use some of the symmetry tricks I invented for <a href="http://www.mit.edu/~puzzle/12/ben_bitdiddle/investigators_report/solution/">a 2012 Mystery Hunt puzzle</a> to make the circuit <i>impossible</i> to assemble incorrectly.)
</p>
<p>Same deal as the XOrduino: design files <a href="https://github.com/cscott/xostick">on github</a> (<a href="https://github.com/cscott/xostick/blob/master/XO-Stick-sch.pdf?raw=true">schematic PDF</a>,
<a href="https://github.com/cscott/xostick/blob/master/XO-Stick-brd.pdf?raw=true">pcb PDF</a>); I expect to have a few boards available to people who want to
help make some software for them. Schematic and layout review is also appreciated!</p>http://cananian.livejournal.com/66129.htmlsugarolpcxorduinoarduinopublic0http://cananian.livejournal.com/66008.htmlMon, 12 Mar 2012 20:55:35 GMTGrowing Up With Nellhttp://cananian.livejournal.com/66008.html
<p>Chris, Michael, and I just submitted a short paper describing our work on the XO-3 Nell project to this year's <a href="http://dimeb.informatik.uni-bremen.de/idc2012/">Interaction Design and Children</a> conference. Give the preprint a read: <a href="http://cscott.net/Publications/OLPC/idc2012.pdf"><i>Growing Up With Nell: A Narrative Interface for Literacy</i> [pdf]</a>.</p>
<p>We're hard at work implementing Nell. We could use help in a number of areas: art, animation, story, user interface, javascript hacking, and probably others. For example, at the moment Chris and I are: drawing the characters, drawing user interface concepts, writing silly alphabet stories, animating the silly alphabet stories, doing the CSS/HTML layout to mock up designs, making some of the mock ups into functional demos, implementing HMM-based handwriting recognition in JavaScript, porting pyaiml to JavaScript, implementing an Apple Guide-style contextual help system on top of HTML widgets, and writing a integrated story editor (and stories about the story editor). I'd welcome volunteers for any of these tasks!</p>http://cananian.livejournal.com/66008.htmlsugarolpcnellpublic1http://cananian.livejournal.com/65380.htmlTue, 01 Nov 2011 22:28:14 GMTA collection of Nell demoshttp://cananian.livejournal.com/65380.html
<p>Here are some banged-together demos of various pieces of
<a href="http://laptop.org">One Laptop per Child</a>'s
<a href="http://cananian.livejournal.com/65077.html">Project Nell</a>.
The ultimate goal is a Nell <a href="http://wiki.laptop.org/go/Nell">demo</a>
for <a href="http://en.wikipedia.org/wiki/Consumer_Electronics_Show">CES</a>
in January 2012, but these bits should be considered as tech demos,
benchmarks, and proofs of concept, not actual pieces of that demo (yet).
</p>
<p>Most of these demos require WebGL support. Visit <a href="http://get.webgl.org/">get.webgl.org</a> for information about
enabling WebGL in your browser; there is WebGL support in Chrome,
Firefox, Safari, and Opera&mdash;although it often requires enabling
experimental features in the browser preferences.</p>
<ul>
<li><a href="http://dev.laptop.org/~cscott/nelldemo.20111101/tile_test.html">Tiles</a>. Performance benchmark
for a tile-based home screen. "Apps" are "locations" on your world map,
which you can customize as you like. (Here's an interesting
<a href="http://vocamus.net/dave/?p=1168">blog entry</a> discussing
world-creation for kids.) Day/night would ultimately reflect current
time, although they've been greatly sped up in this demo. Lots of
rough edges and missing UI, but all the textured triangles are
present, so it should be an accurate benchmark.<br />
(Drag with left mouse button to rotate, middle mouse button to zoom,
right mouse button to pan.)
</li>
<li><a href="http://dev.laptop.org/~cscott/nelldemo.20111101/nell_home_test.html">Nell at home</a>. Basic idea
(including transition) for activities which include dialog with
Nell or story-telling.<br />
Standalone model viewers:
<a href="http://dev.laptop.org/~cscott/nelldemo.20111101/watchtower_test.html">Castle</a> (from <a href="http://www.blendswap.com/3D-models/architecture/watchtower/">blendswap</a>),
<a href="http://dev.laptop.org/~cscott/nelldemo.20111101/sintel_test.html">"Nell"</a> (Sintel, from <a href="http://www.blendswap.com/3D-models/characters/sintel-lite-2-57b/">blendswap</a>),
<a href="http://dev.laptop.org/~cscott/nelldemo.20111101/sintel_lite_test.html">Alternate (lightweight) Nell model</a>,
<a href="http://dev.laptop.org/~cscott/nelldemo.20111101/maison_test.html">Alternate (heavyweight) house model</a>
(from <a href="http://www.blendswap.com/3D-models/architecture/little-house/">blendswap</a>).<br />
In model viewers: drag with left mouse button to rotate, middle
mouse button to zoom, right mouse button to pan.
</li>
<li><a href="http://dev.laptop.org/~cscott/nelldemo.20111101/music.html">Music maker</a>. Uses WebGL and the Web
Audio APIs to let you draw and perform music.<br />
Inspired by Andr&eacute; Michelle's
<a href="http://lab.andre-michelle.com/tonematrix">ToneMatrix</a> and
<a href="http://lab.andre-michelle.com/karplus-strong-guitar">Karplus-Strong
Guitar</a> (see also
<a href="http://en.wikipedia.org/wiki/Karplus-Strong_string_synthesis">wiki</a>
and this 2008 <a href="http://lac.linuxaudio.org/">Linux Audio Conference</a>
<a href="https://ccrma.stanford.edu/realsimple/faust_strings/faust_strings.pdf">paper</a>),
as well as <a href="http://labs.dinahmoe.com/">DinahMoe</a>'s
<a href="http://labs.dinahmoe.com/ToneCraft/">ToneCraft</a> and
the <a href="http://www.global.yamaha.com/tenori-on/">Tenori-on</a>.
</li>
<li><a href="http://www.youtube.com/watch?v=4Mr27yY8-A8">Quake on XO-1.75</a>
(video).
Of course we need to actually run WebGL with good performance on XO
hardware. Jon Nettleton has been working hard on our GL drivers,
enabling the GPU on the XO-1.75 hardware for the first time.
This Quake demo shows his progress&mdash;don't worry, Quake is not
actually part of the Nell demo! (We have a GPU in the XO-1.5 as
well, which hasn't yet been utilized.)
</li>
<li><a href="http://twolivesleft.com/Codify/">Codify</a>&mdash;not
one of our demos (it's a commercial iPad app) but it demonstrates
the direction we'd like to push <a href="http://wiki.laptop.org/go/Pippy">Pippy</a>.</li>
</ul>
<p>Coming soon: TurtleArt and Implode for the web. We've started
converting them to GTK3 in preparation for hoisting them bodily onto
the interwebs. Here's the <a href="http://git.sugarlabs.org/~cscott/turtleart/cscott-gtk3/commits/gtk3">source
code repository for the TurtleArt port</a> if you'd like to watch or
participate in this hackage. (See <a href="http://repl.it">repl.it</a>
for one of the more unusual ways to get Python running in the web
context.) The rest of the demo source code is on <a href="https://github.com/cscott/nelldemo">github</a> (or just "View Source" in your browser).</p>http://cananian.livejournal.com/65380.htmlolpcnellpublic1http://cananian.livejournal.com/65077.htmlSat, 01 Oct 2011 05:18:05 GMTIntroducing Nellhttp://cananian.livejournal.com/65077.html
<p>Between now and January <a href="http://en.wikipedia.org/wiki/Consumer_Electronics_Show">CES</a>, Chris Ball and I will be building <i>Nell</i> for the <a href="http://en.wikipedia.org/wiki/OLPC">OLPC</a> <a href="http://blog.laptop.org/2011/07/22/olpc-xo-3-design-update/">XO-3</a> tablet.
Nell is a name, not an acronym, but if you want to pronounce it as
"Narrative Environment for <a href="http://wiki.laptop.org/go/Learning_Learning">Learning Learning</a>," I won't stop you.</p>
<p>Nell's development will be demo-oriented&mdash;we're going to try to write the most
interesting bits first and learn as we go&mdash;so don't be upset if you
don't see support right away for legacy Sugar activities ("Sweet
Nell"), robust sharing support, mesh networking, or whatever
your favorite existing feature is. They'll come, but the new
crazy stuff is what we need to evaluate first.</p>
<p>Here are four of the big ideas behind Nell, along with pointers to some of our sources of inspiration.</p>
<p><b>Narrative.</b> I probably don't need to restate that Neil
Stephenson's "The Diamond Age" has been hugely influential, and we also owe
a large debt to interactive fiction and the <a href="http://pr-if.org/">Boston IF group</a> in particular. (Check out
the talks from our
"<a href="http://blog.printf.net/articles/2011/06/18/narrative-interfaces">Narrative
Interfaces day</a>" at OLPC.)
<a href="http://users.soe.ucsc.edu/~jskorups/wiki/wide_ruled/wide_ruled_v2">Wide
Ruled</a> (<a href="http://eis.ucsc.edu/sites/default/files/Skorupski_DAC09_WideRuledLessonsLearned.pdf">conference
paper</a>) and <a href="https://research.cc.gatech.edu/inc/mark-riedl">Mark Riedl</a>
at Georgia Tech have demonstrated interesting approaches to story representation.
I'm also looking forward to the results of the Experimental Game Play
group's September <a href="http://experimentalgameplay.com/blog/2011/09/story-game-in-september-2011/">Story
Game</a> competition.</p>
<p><b>Emotion.</b> The Radiolab podcast "<a href="http://www.radiolab.org/2011/may/31/">Talking To Machines</a>" crystallized my thinking about emotionally-attractive environments. The discussion with Caleb
Chung, the creator of Furby, is particularly apropos. Caleb's goal is
to make things which kids want to "play with for a long time," and he
contributes his three rules for creating things which "feel alive":
it must (1) feel and show emotions, (2) be aware of itself and its
environment, and (3) have behaviors which change over time. Furby's
pursuit of these goals include expressive eyes and ears, crying when
held upside down, reacting to loud noises, and gradually switching from <a href="http://en.wikipedia.org/wiki/Furby#Furbish-English_phrases">Furbish</a>
to English for its utterances. A living thing emits a
constant stream of little surprises. Expect to see Nell put the
XO-3's microphone and accelerometer to good use.</p>
<p><b>Talking and Listening.</b> The "Talking To Machines" podcast also discusses <a href="http://en.wikipedia.org/wiki/ELIZA">ELIZA</a> and <a href="http://en.wikipedia.org/wiki/Cleverbot">Cleverbot</a>, which
dovetails with my interest in the popular <a href="http://wiki.sugarlabs.org/go/Activities/Speak">Speak
activity</a> for Sugar and related toys like <a href="http://outfit7.com/apps/talking-tom-cat/">Talking Tomcat</a> for
mobile phones. The key insight here is that a little bit of "cheap
trick" AI can go a long way toward making a personable and engaging
system. We want Nell to feel like a friend. Recent work by
the <a href="http://csc.media.mit.edu/">Common Sense Computing
Initiative</a> at MIT's Media Lab shows how we can reset this on a sounder
basis and use mostly-unstructured input to allow the system to grow
and learn (creating "behaviors changing over time"). In particular, I'll cite <a href="http://csc.media.mit.edu/conceptnet">ConceptNet</a> for its
database and practical NLP tools, and inspiration from
"Empathy Buddy," "StoryFighter," and the other projects described in their
<a href="http://web.media.mit.edu/~push/Beating-Common-Sense.pdf">Beating
Common Sense</a> paper. It's also worth noting that open source
speech tools are good and getting better (the <a href="http://voxforge.org/">VoxForge</a> site points to most of them);
also interesting is <a href="http://festvox.org/transform/transform.html">this technique</a>
for matching a synthesized voice to that of the user.</p>
<p><b>Collecting, nurturing, and rewarding.</b> Collector games such as <a href="http://en.wikipedia.org/wiki/Pocket_Frogs">Pocket Frogs</a> and
<a href="http://www.facebook.com/iphoneflowergarden">Flower Garden</a>
are sticky activities which
encourage kids to come back to the device and continue working toward a goal over a long period of time.
<a href="http://www.technologyreview.com/printer_friendly_article.aspx?id=37874">Memrise</a>
is educational software illustrating this technique: its users tend a garden of flowers by
mastering a set of flash cards. Nell will incorporate the sticky aspects of such games, possibly also integrating the Mozilla <a href="http://openbadges.org/">Open Badges infrastructure</a> into an achievement/reward system.</p>
<p>I hope this has given you a general sense of the direction of our Nell project. In future
blog posts I'll drill down into implementation details, demonstration storyboards, and other more concrete facets of Nell.</p>http://cananian.livejournal.com/65077.htmlsugarolpcnellifpublic4http://cananian.livejournal.com/64747.htmlWed, 15 Jun 2011 21:27:50 GMTNarrative Interfaceshttp://cananian.livejournal.com/64747.html
<p>One Laptop per Child creates student-centric learning experiences. Our current software stack, however, is somewhat "shallow". When you turn on the XO, all the content is immediately available but there is no path or guidance provided. Nothing suggests what you should try first, or indicates an order to progress through the activities provided. Everything is available, but there's no built-in journey. No plot. How can we improve this?</p>
<p><a href="http://www.timeanddate.com/worldclock/fixedtime.html?msg=Narrative+Interfaces+Talks+at+OLPC&amp;iso=20110617T14&amp;p1=43&amp;ah=2&amp;am=30">This Friday (June 17) at 2pm Eastern</a> we're inviting some folks over to OLPC's new offices at the <a href="http://www.americantwine.com/">American Twine</a> building to discuss <i>Narrative Interfaces</i>, as part of the proposed XO-3 software stack. <a href="http://nickm.com/">Nick Montfort</a> will give a short talk on <a href="http://curveship.com/"><i>Curveship</i></a>, his model-based interactive fiction system, and <a href="http://printf.net/">Chris Ball</a> will present some related recent hacking. <a href="http://web.media.mit.edu/~anjchang/">Angela Chang</a> will present her <i>Tinkerbooks</i> early-literacy platform, which allows kids to interactively change the written story on the page. And I'll discuss Neal Stephenson's novel <a href="http://en.wikipedia.org/wiki/The_Diamond_Age"><i>The Diamond Age</i></a> (a recap of <a href="http://www.dailymotion.com/video/xip6ep_en-the-diamond-age_tech">a short talk I gave at EduJAM in Uruguay</a>), and give concrete suggestions for how Diamond Age's <i>Primer</i> might influence the software architecture for the XO-3. (I might even reveal how to make software testing semantically indistinguishable from writing a game!) Chris Ball and I have also been collecting best-of-breed "comic books that teach you something" as examples of educational narrative; we'll pass those around and post a reading list after the event.</p>
<p>The real point of this meeting isn't the talks, per se, but the discussions to follow. We're trying to gather folks who know a lot more about this stuff than we do, in order to learn from them and be inspired. We don't have a lot of space, unfortunately, so I'm going to have to ask for RSVPs from those who wish to attend. If you're in the Boston area and feel like you have something to contribute (and especially if you have created/could create <a href="http://creativecommons.org/">Creative Commons</a>-licensed content for education), drop me a line at cscott at laptop dot org. Describing what you can contribute to the discussion will help break ties if space is inadequate.</p>
<p>We will also live-stream the meeting at <a href="http://www.ustream.tv/channel/cscottnet">ustream.tv/channel/cscottnet</a>. Afterwards we'll post higher-quality video and a list of cited works. Thanks in advance to everyone who will participate, online and off!</p>
<p><b>UPDATE:</b> video now up; see <a href="http://blog.printf.net/articles/2011/06/18/narrative-interfaces">this writeup on Chris Ball's blog</a>.</p>http://cananian.livejournal.com/64747.htmlsugarolpcnellifgamescurveshippublic0http://cananian.livejournal.com/64330.htmlTue, 24 May 2011 16:22:51 GMTSmall systems... and distributed oneshttp://cananian.livejournal.com/64330.html
<p>Today I stumbled across some very interesting projects by <a href="https://github.com/SamuraiJack">Nickolay Platonov</a> which I'd like to discuss in an OLPC context.</p>
<p>I've been hacking away at <a href="http://cscott.net/Projects/TurtleScript">TurtleScript</a> fueled partly by a drive for minimalism: a small system is a learnable system. To that end, the language is based on <a href="http://www.crockford.com/">Douglas Crockford</a>'s "Simplified JavaScript" (as recognized by <a href="http://javascript.crockford.com/tdop/tdop.html">this top-down operator precedence parser</a>) which tries hard to include only JavaScript's "<a href="http://www.livestream.com/etsy/video?clipId=pla_1463e546-47ed-4a93-b59a-bd52b236e8b8">Good Parts</a>". For example, the <a href="https://github.com/cscott/TurtleScript/blob/beeba5c138d88af40297f93689ecbe7721724819/crender.js#L333">initial code</a> for the TurtleScript system uses <a href="http://javascript.crockford.com/prototypal.html">prototype inheritance</a> (via <code>Object.create</code>) rather than classical class-style inheritance. In fact, the <code>new</code> operator isn't even included in the Simplified JavaScript/TurtleScript language.</p>
<p>In a discussion of TurtleScript Alan Kay mentioned, "It is very tricky to retain/maintain readability (so the first Smalltalk was also an extensible language not just semantically but syntactically)." And today I stumbled across the <a href="http://joose.it/">Joose</a> library, which gives a very <a href="http://joose.github.com/Joose/doc/html/Joose.html">readable syntax</a> for traditional class-based inheritance. It backs this up with a robust meta-object protocol, introspection, and lots of nice features borrowed from Perl 6, CLOS, and Smalltalk. The syntax ought to work fine with the limited tile set of TurtleScript, although I might have to add a tile for the <code>new</code> operator.</p>
<p>However, adding Joose raises some questions. Is the increase in readability worth the addition of such a large library to the system? What impact will this have on understanding problems and debugging? Is a return to class-based inheritance a positive change? (There have been arguments that the make-a-clone-and-change-it practice of prototype inheritance is easier to understand for new learners.) Can a larger overall system actually be easier to understand?</p>
<p>And once we're looking at libraries... Nickolay Platonov is now working on <a href="http://joose.it/blog/2011/04/08/syncler-distributed-applications-for-human-beings/">Syncler</a>, based on the <a href="http://joose.it/blog/wp-content/uploads/2011/03/Bayou-updates-propagation.pdf">Bayou</a> system. Unobstructive replication mechanisms would certainly make it easier to construct the sorts of collaborative applications we've wanted for <a href="http://en.wikipedia.org/wiki/Sugar_(desktop_environment)">Sugar</a>. I have two concerns with Syncler's current state. First, the use of <a href="http://joose.it/blog/2011/02/14/joosex-cps-tutorial-part-i/">explicit continuation-passing style</a> greatly impairs readability. The JavaScript <code>yield</code> keyword <a href="http://blog.ometer.com/2010/11/28/a-sequential-actor-like-api-for-server-side-javascript/">helps a lot</a> when writing asynchronous code. (It's not supported by all JavaScript engines, but <code>yield</code> wouldn't be hard to add to TurtleScript.) Second, Syncler's event model uses explicit callbacks. I've been greatly impressed with the <a href="http://www.flapjax-lang.org/">Flapjax</a> event model (and its <a href="http://lamp.epfl.ch/~imaier/pub/DeprecatingObserversTR2010.pdf">strongly-typed functional cousin</a>). Both of these changes ought to make asynchronous code much more readable&mdash;and isn't that an important part of <a href="http://en.wikipedia.org/wiki/Grok">grokking</a> a system?</p>http://cananian.livejournal.com/64330.htmljavascriptsugarolpcturtlescriptprogrammingpublic3http://cananian.livejournal.com/64140.htmlFri, 20 May 2011 06:06:15 GMTTurtles All The Way Downhttp://cananian.livejournal.com/64140.html
<p><img src="http://cscott.net/Projects/TurtleScript/images/hello-world.png" /></p>
<p>Inspired by <a href="http://croquetweak.blogspot.com/">Bert Freudenberg</a>, <a href="http://piumarta.com/software/">Ian Piumarta</a>, and <a href="http://wiki.sugarlabs.org/go/User:Walter">Walter Bender</a>, I started hacking on "Turtles All The Way Down" (aka <a href="http://cscott.net/Projects/TurtleScript/">TurtleScript</a>) on the plane back from Uruguay.
Now there's a nice <a href="http://cscott.net/Projects/TurtleScript/ctiles.html">rendering demo</a> to show what a tile-based editor for JavaScript might look like, as well as a <a href="http://cscott.net/Projects/TurtleScript/tdop.html">bytecode compiler and interpreter</a> for the language. The <a href="https://github.com/cscott/TurtleScript/blob/f977f782bb2bda06d26e5eeb1259db95dd48f2ca/bytecode_table.js">bytecode instruction set</a> is still too large; encouraged by Craig Chambers' work on <a href="http://selflanguage.org/documentation/published/implementation.html">SELF</a> I think I ought to be able to replace all the unary and binary operators, conditional jumps, and slot selectors by a single <code>mapof</code> operator. I can put a better <a href="http://piumarta.com/software/cola/objmodel2.pdf">object model</a> on the interpreter, too;
I've written some <a href="http://cscott.net/Projects/TurtleScript/#simplifying-the-environment">notes on the matter</a>.</p>
<p>The question is: does this really have educational value? "Turtles all the way down" is a great slogan, and a fine way to teach a graduate-level class on compiler technology, but I feel that the higher-level UI for tile-based program editing is the really useful thing for tablet computing. I'm a compiler geek and love the grungy underbelly of this stuff, but I keep reminding myself I should really be spending more time building a beautiful fluffy surface.</p>http://cananian.livejournal.com/64140.htmljavascriptsugarolpcturtlescriptturtlespublic1http://cananian.livejournal.com/63783.htmlFri, 29 Apr 2011 15:47:57 GMTNext Steps for New Technologieshttp://cananian.livejournal.com/63783.html
<p>I've reached the end of the month. I've accomplished my
Android and NativeClient-related <a href="http://cananian.livejournal.com/62667.html">goals</a>, but didn't get the time to do
as much mesh and python investigation as I'd wanted. Here are some
ideas for next month's work. (Next week I'll be in Uruguay for <a href="http://wiki.sugarlabs.org/go/Uruguay_Summit_2011">EduJAM</a>.)</p>
<h3>GObject Introspection (Android or NaCl)</h3>
<ol>
<li>Start by porting <a href="http://en.wikipedia.org/wiki/Libffi">libffi</a>. An android port would be
straightforward, but since libffi involves code generation
(<a href="https://github.com/atgreen/libffi/blob/master/src/arm/ffi.c#L557">ARM</a>, <a href="https://github.com/atgreen/libffi/blob/master/src/x86/ffi.c#L475">x86</a>), this is going to require a bit of assembly
magic and the new "JIT"/"shared library" support in the NaCl plugin.</li>
<li>Then port <a href="http://blogs.gnome.org/johan/2008/06/01/introduction-to-gobject-introspection/">gobject-introspection</a>. GObject-Introspection relies on
libffi for its guts, but the <strong>hard</strong> part of this port will be refactoring
g-i's build process, which is <a href="http://mail.gnome.org/archives/gtk-devel-list/2009-March/msg00062.html">not cross-compilation
friendly</a>. Might need to rewrite some tools. If targeting NaCl, you might consider finishing the <a href="https://groups.google.com/group/native-client-discuss/msg/1e61ab5f7bd71797">code allowing execution of unsandboxed NaCl binaries</a>.</li>
<li>Turn gobject-introspection on its head: generate GIR and a
C binding for the platform "native" interface. For NaCl,
this would be a <a href="http://www.w3.org/DOM/Bindings">GObject-using C-level binding
of the browser-native DOM</a>; for
Android, this would be a GIR binding of the native Android APIs. These bindings should be mostly
automatically generated, since they will need to continue tracking
successive native platform releases/HTML5 features.</li>
<li>Demos! Change browser DOM from Python, write native Android apps
in Python. Add a gobject-introspection binding to <a href="http://dev.laptop.org/git/users/wmb/cforth">cforth</a>,
then do the same from forth. (Forth might be a simpler place to
start than Python. Or not.)</li>
</ol>
<h3>GTK (Android or NaCl)</h3>
<ol>
<li> Build on the cairo/pango port to proceed to a full GTK backend for
Android/NaCl. These backends ought to be upstreamable. The NaCl
port should be based on the <a href="http://blogs.gnome.org/alexl/2011/03/15/gtk-html-backend-update/">broadway</a> work: the cairo canvas
would be drawn to more directly, but a lot of the mechanism which
captures JavaScript events and translates them into the GTK event loop
could probably be reused.</li>
<li> Demo: "Hello GTK world" in Android/NaCl.</li>
</ol>
<h3>Sugar partitioning.</h3>
<p>Bring Sugar closer to being a true
multi-language multi-library platform.</p>
<ol>
<li> Refactor sugar modules (for example, sugar toolbar widget) as
standalone C libraries. Basic idea is to embed Python and export
a C API, while preserving as much of the code as possible. Python
libraries now invoke this library via g-i-r instead of directly.
The python embedding tool is probably a useful standalone product.</li>
<li> Rewrite "Hello, Sugar" activity in C (or <a href="http://en.wikipedia.org/wiki/Vala_(programming_language)">vala</a>), using <code>#include</code> for <code>import</code>
and <a href="http://en.wikipedia.org/wiki/GObject">GObject</a> inheritance instead of python inheritance. Use this as
a guide to pull apart sugar into modules (as above) to make this
code actually work as written.</li>
</ol>
<h3>Miscellanous topics</h3>
<ol>
<li> <a href="http://en.wikipedia.org/wiki/Google_Chrome_OS">ChromeOS</a> w/ touch support.
<p>Find an appropriate machine, do an installation, what are the
roadblocks/rough spots? Can we install on <a href="http://www.engadget.com/2011/01/09/marvell-powered-olpc-xo-1-75-only-draws-2-watts-of-power-finall/">XO-1.75</a> as a testbed?</p>
</li>
<li>TurtleArt as JavaScript viewer/editor.
<p>Revisit <a href="https://github.com/cscott/TurtleScript#readme">TurtleScript</a> work, but skip over the
time-consuming "construct an editor" step by reusing the
(excellent) <a href="http://wiki.laptop.org/go/Turtle_Art">TurtleArt</a> code.</p>
</li>
<li> Mesh: android <a href="http://lists.olsr.org/pipermail/olsr-dev/2011-April/004439.html">olsrd</a> frontend, build testbed, research 802.11 DCF issues.</li>
</ol>
<h3>Summary</h3>
<p>There are four rough topics here; I might try to continue the
breadth-first search by spending a week on each. It might be more
satisfying to downselect two of these issues and spend two weeks on
each.</p>http://cananian.livejournal.com/63783.htmlnativeclientjavascriptsugarolpcandroidprogrammingmeshpublic0http://cananian.livejournal.com/63595.htmlFri, 29 Apr 2011 15:08:56 GMTPango/Android -vs- Pango/NaClhttp://cananian.livejournal.com/63595.html
<p>At the end of <a href="http://cananian.livejournal.com/62756.html">my Sugar/Android</a> week, I had a simple
<a href="http://en.wikipedia.org/wiki/Pango">Pango</a>-on-<a href="http://en.wikipedia.org/wiki/Cairo_(graphics)">Cairo</a> demo running. This was built on a stack
of ported libraries, including <a href="http://en.wikipedia.org/wiki/Gettext">gettext</a>, <a href="http://cgit.freedesktop.org/pixman/tree/README">pixman</a>,
<a href="http://en.wikipedia.org/wiki/FreeType">freetype</a>, <a href="http://en.wikipedia.org/wiki/Libxml2">libxml2</a>, <a href="http://en.wikipedia.org/wiki/Fontconfig">fontconfig</a>, and <a href="http://en.wikipedia.org/wiki/Glib">glib</a>,
as well as <a href="http://en.wikipedia.org/wiki/Cairo_(graphics)">cairo</a> and <a href="http://en.wikipedia.org/wiki/Pango">pango</a>. You can run the demo
yourself by <a href="http://en.wikipedia.org/wiki/Sideloading">sideloading</a> <a href="http://dev.laptop.org/~cscott/Android/Pango/pango-demo.apk">pango-demo.apk</a> onto your Android
device (tested on a Motorola <a href="http://en.wikipedia.org/wiki/Motorola_Xoom">Xoom</a>), and you can <a href="http://dev.laptop.org/git/users/cscott/android-libs/tree/">browse the
source code</a> to see what it entailed (here's the <a href="http://dev.laptop.org/git/users/cscott/android-libs/tree/jni/Makefile.devel">scariest
part</a>). (I was inspired by Akita Noek's <a href="https://github.com/anoek/android-cairo">android-cairo project</a>,
but I ended up reworking the build scheme and redoing most of the
ports.)</p>
<a href="http://dev.laptop.org/~cscott/Android/Pango/screenshot.png"><img src="http://dev.laptop.org/~cscott/Android/Pango/screenshot.png" width="300" alt="Screenshot of Pango demo on Android" title="Pango demo on Android" /></a>
<p>It made sense to start my Sugar/<a href="http://en.wikipedia.org/wiki/Google_Native_Client">NaCl</a> investigation by porting
the same demo application to Native Client. The same stack of ported
libraries was involved, although it was easy to include more
functionality in the Native Client ports, including threading and
PNG/PS/PDF support in cairo. The <a href="http://dev.laptop.org/git/users/cscott/naclports">source code</a> is a fork from
the upstream <a href="http://code.google.com/p/naclports/">naclports</a> project, and the process was generally much cleaner.
(But see my previous post for some <a href="http://cananian.livejournal.com/63325.html">caveats regarding naclports</a>.)
If you're using Chrome 10 or 11, you can <a href="http://dev.laptop.org/~cscott/NaCl/Pango/cairo.html">run the demo in your
browser</a> (follow the instructions on that page). The
Wesnoth team has a parallel project which <a href="https://groups.google.com/group/native-client-discuss/msg/8bfa6401e4951551?pli=1">ported some of these libraries as well</a>, but
not in an upstreamable manner.</p>
<a href="http://dev.laptop.org/~cscott/NaCl/Pango/screenshot.png"><img src="http://dev.laptop.org/~cscott/NaCl/Pango/screenshot.png" width="300" alt="Screenshot of Pango demo on Native Client" title="Pango demo on NaCl" /></a>
<p>The demo app uses cairo to draw the background, an animated X, and
some basic text in the center; it uses Pango's advanced international
text support to draw properly-shaped Persian text in a circle
around it. The center text is the "proper" bilingual Greek/Japanese
written form of "pango"; the text around the edges is the Persian name of
the internationalization library, "harfbuzz". Note that the Persian
text is written right-to-left&mdash;and that I didn't put a full CJK
font in the NaCl app, so the Japanese "go" character is missing.
The Android port rebuilds the font cache at each startup, so it loads
rather slowly; the NaCl port contains a prebuilt font cache so it starts
more quickly.</p>
<p><b>Both ports took about two weeks.</b> I blew my <a href="http://cananian.livejournal.com/62667.html">original schedule</a>,
partly due to the Patriot's day holiday, and partly because I'd given
Android about a week's head start by tinkering on it before my
original schedule post. The framerate of the demo is much better on
NaCl (so fast that the edges of the animated X look choppy in the
screenshot), but the hardware isn't easily comparable, so the
comparison doesn't really tell us much. The porting effort was
certainly more pleasant on NaCl, since newlib is a much more complete
libc than Android's "Bionic"&mdash;but having gdb available made
debugging on Android easier. (There is an unintegrated NaCl branch that
<a href="https://groups.google.com/group/native-client-discuss/browse_thread/thread/e11fdea235fa3702">integrates NaCl gdb in the browser</a>, though!)</p>
<p>Much of the GNOME/POSIX library stack assumes access to a filesystem
tree and does file-based configuration. In our demo application,
fontconfig was the most culpable party: it wanted to load a
configuration file describing font locations and naming, then to load
the fonts themselves from the file system, and finally to write a
cache file describing what it found back to the file system. Most
ported software is going to want similar access&mdash;even if you store
the user's own documents in a <a href="http://wiki.laptop.org/go/Journal_Activity">Journal</a>, software still expects
to find configuration, caches, and other data in a filesystem.</p>
<p>Android provides the <b>POSIX filesystem APIs</b>, but the filesystem an app can
touch is segmented and sandboxed. As discussed previously, Android's
<a href="http://developer.android.com/reference/android/os/storage/StorageManager.html">Opaque Binary Blob</a> feature may allow you to create a app-specific
filesystem, but this doesn't let you share (for example) fonts and
font configuration between activities. NaCl might eventually provide
a similar unshared mechanism based on the HTML5 <a href="http://www.html5rocks.com/tutorials/appcache/beginner/">AppCache</a>.</p>
<p>The preferred solution is more limited, but more flexible: no built-in
filesystem APIs are used (or in NaCl's case, provided!) at all.
Instead, you provide your own implementation of the POSIX file APIs
(either via the <a href="https://groups.google.com/group/native-client-discuss/msg/4b84cd47910430e2">--wrap linker indirection</a> or through an appropriate
backend to newlib/glibc/glib). For the NaCl demo app, I wrote a
rather-elaborate <a href="http://dev.laptop.org/git/users/cscott/naclports/tree/src/examples/graphics/cairo/cairo_files.c">in-memory filesystem</a> --- only to find that an
even-more-elaborate one <a href="http://dev.laptop.org/git/users/cscott/naclports/tree/src/packages/scripts/memory_filesys">already existed in naclports</a>. But the
longer-term solution uses message-passing (<a href="https://sites.google.com/a/chromium.org/dev/nativeclient/how-tos/simple-rpc">SRPC</a> in NaCl, <a href="http://developer.android.com/guide/topics/intents/intents-filters.html">intents</a> in
Android) to implement these POSIX APIs. In Native Client, the
implementation would be in browser-side JavaScript, which would then
allow you to share parts of the filesystem tree between activities
and/or map it into (cached) web-addressed resources. In either case,
your application still sees the bog-standard POSIX API it expects.</p>
<p>More problematic are the <b>networking APIs</b>. Here Android provides
a pretty standard socket library, while Native Client provides
nothing at all. Using a browser-based implementation, as for
the file APIs, will work fine for HTTP, <a href="http://en.wikipedia.org/wiki/WebSockets">WebSockets</a> and even P2P via the
<a href="http://rtc-web.alvestrand.com/home/papers/juberti-p2ptransport-api.pdf">HTML5 P2P APIs</a>. But it's not clear that (for example) glib's
elaborate <a href="http://git.gnome.org/browse/glib/tree/gio/libasyncns/asyncns.c">asynchronous DNS name resolver implementation</a> can (or
should!) be implemented in a NaCl port.</p>
<p>In the end, the porting effort and abstraction shifts needed for
Native Client and Android are <b>roughly comparable</b>. I expect
Native Client will hold a strong edge in allowing close
integration with web standards and web technologies. Android will
probably continue to hold an edge in third-party application support and
platform maturity.</p>http://cananian.livejournal.com/63595.htmlnativeclientsugarolpcandroidprogramminggnomepublic1http://cananian.livejournal.com/63325.htmlFri, 29 Apr 2011 14:24:36 GMTSugar-on-Native Client investigationhttp://cananian.livejournal.com/63325.html
<p>This post will describe the state of Native Client in general, based on
week 2 of my <a href="http://cananian.livejournal.com/62667.html">original four week plan</a>. In the next post, I'll
link to my work so far, and compare the Native Client and the Android
efforts. Recapping, the end goal of these explorations is a platform for the next generation of the
<a href="http://en.wikipedia.org/wiki/Sugar_(desktop_environment)">Sugar learning environment</a>.</p>
<p>To begin, the Native Client (NaCl) plugin is fairly mature in a number of
areas. Version <a href="http://code.google.com/chrome/nativeclient/docs/releasenotes.html">0.2 of the NaCl SDK</a> was recently released (a
version number which substantiates the "fairly" in my previous
sentence), and the NativeClient plugin is currently shipping in Chrome
(versions 10 and 11), although you have to manually turn on a
preference in the <code>about:flags</code> dialog to enable it. The
NaCl toolchain is much more standard than the Android NDK toolchain <a href="http://cananian.livejournal.com/62756.html">I
discussed previously</a>, and the robust <a href="http://code.google.com/p/naclports/">naclports</a> tree shows
that the patches required for NaCl ports of common packages <a href="http://code.google.com/p/naclports/source/browse/trunk/src/packages/scripts/pixman-0.16.2/nacl-pixman-0.16.2.patch">tend not
to be too evil</a>. The <a href="http://wiki.tcl.tk/28211">Tcl
interpreter</a> and <a href="http://dev.laptop.org/~cscott/NaCl/Qt/">Qt tookit port</a> demos show that fairly complex pieces of code can be deployed today on NaCl.</p>
<p>On the other hand, there are three main difficulties:</p>
<ol>
<li> The default NaCl toolchain uses <a href="http://en.wikipedia.org/wiki/Newlib">newlib</a> as its standard C library.
This is consistent with Google's preference for BSD-licensed code in
SDKs they provide to the public (see the <a href="http://codingrelic.geekhold.com/2008/11/six-million-dollar-libc.html">discussion of Bionic in the
Android SDK</a>). However, there also exists <a href="https://groups.google.com/group/native-client-discuss/msg/16bcc685269fa09e">a branch of the SDK which
uses glibc</a>. The glibc branch supports several additional
features, like <a href="http://code.google.com/p/nativeclient/issues/detail?id=565">shared library support</a>. However, it is unclear
whether this will ever be a "supported" part of the SDK. If glibc does
become supported, it is unlikely ever to be the <i>only</i> supported libc;
the BSD-licensed newlib will need to remain available as an option.
(Yes, the LGPL license of glibc shouldn't inspire such paranoia, but
Google has elected not to undertake the education of all prospective
third-party developers.)</li>
<li> The naclports project, although fairly robust, is driven between the
<a href="http://en.wikipedia.org/wiki/Between_Scylla_and_Charybdis">Scylla and Charybdis</a> of compatibility. The goal is that all
the code in naclports be buildable at all times on all three major
platforms: Windows, Mac, and Linux. Further, it should support both
x86_32 and x86_64 backends, and ideally ARM and <a href="http://nativeclient.googlecode.com/svn/data/site/pnacl.pdf">pNaCl</a> as well.
It's <a href="http://build.chromium.org/p/client.nacl.ports/console">auto-built</a> against the latest SDK sources, but should also work
on the latest <i>released</i> SDK. And with the addition of the
glibc/newlib split discussed above, the possible build targets are
multiplied further. Needless to say, keeping the tree building
against such a large number of variants is not an easy task, and
naclports is usually broken in some way. In practice, most developers
seem to pay attention to some subset (say, x86_32/newlib/Linux host),
but it's hard to push patches upstream without worrying about breaking
some obscure target. It might be best to base future work on a
proper package technology, like (say) dpkg-cross.</li>
<li> In general, a lot of interesting NaCl development has occurred on
branches that are not easily integrated. I've already mentioned glibc
support, which is a toolchain branch; shared library support is on
another branch that requires a new chromium plugin as well. At
various times different means have been implemented to run NaCl
binaries "natively" outside the sandbox (for example, in order to test
some feature at build time, or auto-generate some piece of code via
introspection). These efforts live on abandoned branches, while the
"official" means to do this is incomplete. Similarly, a lot of
interesting NaCl work used the now-abandoned legacy "<a href="http://en.wikipedia.org/wiki/NPAPI">NPAPI</a>" plugin
interface to interact with the browser. It was followed by the
"Pepper" plugin interface, which was <a href="https://wiki.mozilla.org/NPAPI:Pepper">itself abandoned</a>. Current
work uses the <a href="http://code.google.com/p/ppapi/">Pepper2</a> browser plugin APIs, which
(unfortunately) have not yet been implemented in non-Chrome browsers
and continue to flux about. Many interesting browser interactions
exist only in deprecated Pepper APIs, not having yet been built into
Pepper2. ARM and pNaCl work also appears to be on unintegrated
branches. There are a number of different <a href="https://sites.google.com/a/chromium.org/dev/nativeclient/how-tos/debuggingtips">gdb support
strategies</a>.</li>
</ol>
<p>None of these difficulties is insurmountable&mdash;and in fact, some
are side-effects of the desirable active development and
productization of Native Client. To date I've done my work on the
(more compatible) SDK v0.1 and the (more upstreamable) newlib library.
So far newlib has not been a huge obstacle, and this basis allows my
patches and ports to be more broadly useful. This might change in the
future&mdash;certainly at some point we need to move to ARM and/or
pNaCl for XO-3, which will probably require building chrome and the NaCl
toolchain from scratch. At that point, it may be worth further
exploring the non-mainstream branches.</p>http://cananian.livejournal.com/63325.htmlnativeclientsugarolpcprogrammingpublic1http://cananian.livejournal.com/63130.htmlWed, 27 Apr 2011 18:30:23 GMTHow does the iPad "use the iPhone's GPS"?http://cananian.livejournal.com/63130.html
<p>A few months ago, a number of stories came out covering the iPad's remarkable-seeming ability to <a href="http://blog.urbanape.com/post/3798485232/show-me-the-way">share the GPS of a tethered iPhone</a>. Apple's latest <a href="http://www.apple.com/pr/library/2011/04/27location_qa.html">location database FAQ</a> confirms the suspicions I voiced at the time: there's no actual GPS sharing involved. Instead, Apple is using the simultaneous GPS and Wifi radios on your iPhone to "crowd-source" what I'll call a "skyhook" database (after the <a href="http://www.skyhookwireless.com/">first company to publicly use the technique</a>). This correlates Wifi base station identifiers with their GPS locations <i>in real time</i> -- including (most likely) the real time location of the "base station" created by the iPhone when it is in tethering mode. All nearby Apple devices use this database to compute their location (based on all visible wifi base stations). Since the nearby device sees the iPhone's "base station" and the iPhone is busily updating the position of that "base station" in real time (along with all the other base stations the iPhone can see), the iPad (lacking a GPS of its own) gains the apparent magical ability to compute a very accurate position for itself.</p>
<p>The real interesting part of this story involves user consent and privacy&mdash;do iPhone users generally know that their devices are registering their location in Apple's database in real time whenever tethering is turned on? Any device which can query Apple's location database for the MAC address of your iPhone can track the position of your iPhone whenever you are tethering. That's basically what the magical ability of the iPad/iPhone pair tells us. Did you know that?</p>http://cananian.livejournal.com/63130.htmliphonepublic0http://cananian.livejournal.com/62756.htmlTue, 12 Apr 2011 17:40:48 GMTSugar-on-Android, week onehttp://cananian.livejournal.com/62756.html
<p>Last week I described a four-week plan to survey key technologies for
One Laptop Per Child's forthcoming XO-3 tablet computer. Here I'll
describe the results of the first week of work, which dove into
Google's Android operating system. <i>Warning: technical content ahead...</i></p>
<h3>Basic design of Sugar-on-Android</h3>
<ol>
<li>Cross-compile gobject/GTK/gobject-introspection/cairo/dbus for Android;
distribute these key libraries as <a href="http://developer.android.com/sdk/ndk/">NDK</a> libraries. This is what I spent
most of my time on this week: I've managed so far to port libiconv, gettext,
glib, pixman, freetype, fontconfig, cairo, libxml2, and pango. (<a href="http://dev.laptop.org/git/users/cscott/android-libs">Source code</a>)</li>
<li>Use cairo or OpenGL backends of GTK3 to render legacy Sugar activities directly to Android
canvas.</li>
<li>Modularize sugar; use <a href="http://www.freedesktop.org/wiki/Software/dbus">D-Bus</a> for inter-module communication.
Interprocess communication mechanism is Android <a href="http://developer.android.com/guide/topics/intents/intents-filters.html">'intents'</a>; these can
<a href="http://developer.android.com/resources/articles/can-i-use-this-intent.html">redirect to the web or the Android Market for missing dependencies</a>. (<a href="http://blogs.gnome.org/uraeus/2011/04/12/gstreamer-and-android/">Collabora reportedly already has a D-Bus implementation for Android</a>.) Sugar components can also become Android <a href="http://developer.android.com/guide/topics/fundamentals/services.html">Services</a>.</li>
<li>Implement Sugar <a href="http://wiki.sugarlabs.org/go/Human_Interface_Guidelines/The_Laptop_Experience/Zoom_Metaphor">Home/Groups/Neighborhood</a> views and <a href="http://wiki.sugarlabs.org/go/Human_Interface_Guidelines/The_Laptop_Experience/The_Journal">Journal</a> as four separate Android
<a href="http://developer.android.com/guide/topics/appwidgets/index.html">App Widgets</a>. These could also be implemented by <a href="http://developer.android.com/resources/samples/Home/index.html">providing a new Android home application</a>, but I think the finer-grained modularity afforded by splitting these functions would yield a better design and make it easier to incorporate upstream improvements to the Android launcher.(Android <a href="http://developer.android.com/resources/samples/CubeLiveWallpaper/index.html">Live Wallpaper</a> is also similar in function, but not as good a fit.)</li>
<li>The Sugar <a href="http://wiki.sugarlabs.org/go/Human_Interface_Guidelines/The_Laptop_Experience/The_Journal">Journal</a> becomes an Android <a href="http://developer.android.com/guide/topics/providers/content-providers.html">"Content Provider"</a>, which
stores/retrieves content for other Sugar activities. (There is special Android support for <a href="http://developer.android.com/resources/samples/WeatherListWidget/index.html">"collection-based Widgets"</a> and <a href="http://developer.android.com/resources/articles/live-folders.html">Live Folders</a> which may be helpful.)</li>
<li>Use <a href="https://live.gnome.org/GObjectIntrospection">gobject-introspection</a> to build a multi-language environment.
Use <a href="http://live.gnome.org/JGIR">JGIR</a> to expose Sugar APIs to "native"
Dalvik apps; use something like the <a href="http://code.google.com/p/android-scripting/">Android Scripting Environment</a> to expose Android native
APIs to GIR languages (Python, JavaScript, C, etc).</li>
<li>[opportunity] Use the <a href="http://www.olsr.org/?q=node/30">Android port of OLSRd</a> to implement a <a href="http://wiki.sugarlabs.org/go/Human_Interface_Guidelines/The_Laptop_Experience/Zoom_Metaphor#Neighborhood">Neighborhood view</a>.
Alternatively, investigate <a href="http://code.google.com/p/adhoc-on-android/">AODV routing on Android</a> and/or <a href="https://www.alljoyn.org">AllJoyn</a> (which also requires root access, see <a href="https://www.alljoyn.org/sites/default/files/80-BA001-1_C_AllJoyn-Android-NDK-Developer-Guide.pdf">pg 24-25 of the manual</a>).</li>
</ol>
<h3>Key Benefits</h3>
<ul>
<li>Sugar is integrated into Android environment; use native Android
education apps, or apps like <a href="http://www.gottabemobile.com/2011/02/18/android-3-0-honeycomb-movie-studio-enables-movie-editing-on-the-go/">Movie Studio</a> which have no Sugar
equivalents yet.</li>
<li>Android APIs and customization hooks are good, and provide a
more-extensible framework for development.</li>
</ul>
<h3>Open challenges (general)</h3>
<ol>
<li>The web integration story is cloudy. <a href="http://code.google.com/p/apps-for-android/source/browse/trunk/Samples/WebViewDemo/src/com/google/android/webviewdemo/WebViewDemo.java">Java and JavaScript can call each other inside a bundled WebView widget</a>,
but this isn't supported in standard Browser app. Browser plugin interface would help.</li>
<li>No good story for building 'native' <a href="http://en.wikipedia.org/wiki/Dalvik_(software)">Java/Dalvik</a> or C apps on the device.
Writing a simple Dalvik compiler would help. <a href="http://en.wikipedia.org/wiki/Dalvik_(software)#External_links">Dalvik specs are available</a>, and people have
written <a href="http://benlynn.blogspot.com/2009/02/compiler-compilers_11.html">Dalvik compilers for toy languages</a>.</li>
<li>"View source" requests can be implemented as an Android 'intent' message,
but no good story for implementing this functionality other than on a case-by-case basis in each activity.</li>
<li>Although the Amazon Marketplace for Android indicates that it can be done, it appears that there is no "blessed" mechanism for creating .apk files on the device and installing them. (<a href="http://code.google.com/p/android/issues/detail?id=545">Android bug</a>, <a href="http://groups.google.com/group/android-developers/browse_thread/thread/fe0978b471d4bef4/72507bfb7586aded?lnk=gst">discussion</a>)</li>
</ol>
<h3>Current technical issues/bugs</h3>
<ol>
<li>Cross-compiling for Android is currently a miserable
experience. The Android NDK appears to have been put together by a
team which had never seen a proper cross-compiler before. Since I
only had a week for this exploration, I mostly <a href="http://dev.laptop.org/git/users/cscott/android-libs/tree/jni/Makefile.devel">kludged things together</a>
to get past this, but any serious work with Android should start by
defining and upstreaming proper autoconf "target triplets" for
Android-on-{ARMv5, ARMv7, x86} and building a proper cross-compiler.
Then patches to various tools and libraries could start being
upstreamed. Using the bespoke <code>Android.mk</code> build system of the NDK is
a non-starter. No serious obstacles here, just work to
do.</li>
<li>Xoom hardware is ARMv7, but <a href="http://groups.google.com/group/android-ndk/browse_thread/thread/a19fc6df3d661d79">Android emulator is ARMv5 only</a>.
Unfortunately, gdb is broken on the Xoom. So we're
building for ARMv5 at the moment, so we can debug in the (slow) emulator.</li>
<li>No good support for shared libraries may cause activity bloat.
May be able to be worked around using the <a href="http://developer.android.com/reference/android/os/storage/StorageManager.html">new Opaque Binary Blob (OBB) feature</a>.</li>
<li>Much existing code (fontconfig, gettext, gtk, etc) expects to read
configuration files from the filesystem. Currently we are using the
default fall-back configurations. OBB support may help here as well.
There are a number of different <a href="http://developer.android.com/guide/topics/data/data-storage.html">storage APIs in Android</a>, but none seems quite right.
</li>
<li>It would be nice to implement a ring-style XO home screen without
completing replacing the android Launcher. No clear way to constrain
app layout on home screen w/o completely replacing the Launcher.
Is it worth hacking the <a href="http://android.git.kernel.org/?p=platform/packages/apps/Launcher2.git;a=summary">Launcher source</a>?</li>
<li>Mesh on Android using OLSRd current requires root access. In order
to run on unrooted Android devices, we need (a) proper power
management for Ad Hoc mode wifi, (b) APIs to enable Ad Hoc mode, and
(c) APIs to manipulate kernel routes.</li>
<li>We're building libraries without thread support because Android's
"Bionic" libc has an eccentric thread library. Linking with
<code>-lpthread</code> fails because the thread functionality is bundled into <code>-lc</code>.
Probably just providing an empty <code>libpthread.so</code> would help a lot.</li>
<li>Some work has been done to build GNU libc for Android. This bloats
activities even further, but might help ease library porting.</li>
<li>Porting gobject-introspection will be painful because <a href="http://mail.gnome.org/archives/gtk-devel-list/2009-March/msg00062.html">its makefiles
are not set up for cross-compiling</a>. Some steps want to run on the
target hardware, which is difficult in the Android environment.</li>
</ol>
<h3>Bottom line</h3>
<p>I can see how the whole Sugar stack can be put together on the Android
platform. The hardest part is probably just setting up packaging
and a good and repeatable build environment for the different
components, and getting enough adoption of this that patches to
support Android can be pushed upstream. Many of the important pieces
can be developed in parallel (Theme, Journal, Mesh, Friends, Home,
library porting, etc). A little early to tell how hard it will be
to port existing Sugar activities to the new Python/pygobject/GTK3
framework.</p>http://cananian.livejournal.com/62756.htmlsugarolpcandroidpublic5http://cananian.livejournal.com/62667.htmlMon, 04 Apr 2011 21:48:52 GMTExploring New Technologieshttp://cananian.livejournal.com/62667.html
<p>Last Monday I rejoined <a href="http://laptop.org">One Laptop Per Child</a> as Director, New
Technologies. My mandate is hardware and software for the XO-3,
OLPC's upcoming ARM-based tablet computer for education in the
developing world. The new machine should be lower cost, lower power,
more indestructible, more powerful, and potentially more expandable
than ever. There are <a href="http://wiki.laptop.org/go/Deployments">about two million machines</a> in the XO-1 family
(XO-1, XO-1.5) in the hands of kids
today. The XO-3 will build upon
this impressive foundation to reach further into the poorest and
least-connected regions of the world.</p>
<p>I will kick-off my work with a series of <b>four week-long sprints</b>
between now and <a href="http://wiki.sugarlabs.org/go/Uruguay_Summit_2011">eduJAM Uruguay</a>
to investigate a
number of possible directions for the educational software stack on
the XO-3 tablet. On the XO-1&mdash;series machines OLPC ships <a href="http://en.wikipedia.org/wiki/Sugar_(desktop_environment)">Sugar</a>, an
impressive collection of educational software developed by
<a href="http://sugarlabs.org/">Sugar Labs</a>. How can we best keep
the best of Sugar while yanking the UI forward into a touch-friendly
tablet world?</p>
<ol>
<li>This week (April 4-8) I'll begin by working on a <b>port of the <a href="http://gtk.org">GTK3</a> UI library to
<a href="http://en.wikipedia.org/wiki/Android_(operating_system)">Android</a></b>. The GTK3 library contains touch support missing from the
GTK2 library on which Sugar is currently based. The end goal here
would be a full port of the Python/GTK-based Sugar APIs, running on
something like the Honeycomb Android OS. Our existing educational
activities could be ported to the new APIs without too much
difficulty, but we'd largely use the existing Android OS facilities
instead of the parts of Sugar concerned with low-level system
management. To clarify: this is a preliminary exploration&mdash;we
haven't decided to base the tablet software on Android (or anything
else) yet.</li>
<li>The next week brings a new direction. During the week of April 11-15
I will start <b>porting Python/GTK3 to <a href="http://en.wikipedia.org/wiki/Google_Chrome">Chrome</a> or <a href="http://en.wikipedia.org/wiki/ChromeOS">ChromeOS</a> via the Google
<a href="http://en.wikipedia.org/wiki/Google_Native_Client">NativeClient plugin</a></b>. This path would result in activities which more
fully integrate with web technologies&mdash;even in disconnected regions
of the world. On desktop machines, Sugar activities could be run
inside the Chrome browser, while ChromeOS (or another embedded OS
running chrome/webkit) would provide the system management functions
on tablet machines like the XO-3. As with the Android port, this is
an exploration, not a definite software direction.</li>
<li>The week of April 18-22 I hope to <b>focus on <a href="http://en.wikipedia.org/wiki/Wireless_mesh_network">mesh networking</a></b>. This has
a somewhat checkered history in our deployments; I hope to identify
the remaining roadblocks and map a way forward to make this a flagship
feature of the XO-3 software.</li>
<li>The week of April 25-29 is for the <b>existing Python-based Sugar
codebase</b>. In order to continue moving forward, it needs to migrate to
GTK3, <a href="http://live.gnome.org/GObjectIntrospection">gobject-introspection</a>, and some other key enabling technologies.
I believe it would also benefit from language-independent APIs and
better modularization to allow a more incremental migration path.</li>
</ol>
<p>The following week is <b><a href="http://wiki.sugarlabs.org/go/Conozco_Uruguay_Tour">Conozco Uruguay</a> and the <a href="http://wiki.sugarlabs.org/go/Uruguay_Summit_2011">Uruguay EduJAM</a></b> where I'll present my
progress on these initial exploratory projects and discuss the path ahead
with the wider OLPC and Sugar communities. Clearly, a week each is not
enough time to finish any of these projects! But the focused effort should
help to better identify the promise, roadblocks, and challenges in
each of these paths, which then in turn will help us to plan the future.</p>http://cananian.livejournal.com/62667.htmlsugarolpcchromeosnetworkingandroidgooglepythonnativeclientsugarlabssugarcampjobsprogrammingmeshpublic10http://cananian.livejournal.com/62444.htmlMon, 21 Feb 2011 03:48:25 GMTTaste, user experience, and engineeringhttp://cananian.livejournal.com/62444.html
<p>A recent <a href="http://speedbird.wordpress.com/2011/02/19/nokia-culture-will-out/">article on Nokia's internal culture</a> contained this interesting quote:</p>
<blockquote>
Designers are also, by training and predilection, inclined to design for the usual, where engineers are taught a kind of rigor that compels them to account for, and overweight, low-probability events.
</blockquote>
<p>This does seem to me to often be a fundamental problem in not only interaction and UI design, but also internal programming APIs and interfaces. Good engineering is a clever balance; as <a href="http://en.wikiquote.org/wiki/Larry_Wall">Larry Wall has said</a>: "Easy things should be easy, and hard things should be possible." An engineering mindset often fixates on the hard things (the "interesting part of the problem"!) and tries to make the hard things easy (or easier), at the risk of making the easy things hard. The end result is failure.</p>
<p>Truly elegant engineering involves finding a view of the problem where the hard parts of the problem disappear. We're not always fortunate enough to find that solution. In falling back to a practical/possible solution, we must be careful to ensure that <b>we keep the easy things easy</b> &mdash; it's fine if the hard things are difficult, so long as they are possible. Effort spent making the hard things easier is wasted if it makes the easy things harder. The goal is not a uniform mediocrity of design.</p>http://cananian.livejournal.com/62444.htmlprogrammingpublic1http://cananian.livejournal.com/61968.htmlTue, 18 Jan 2011 06:53:12 GMTImproving Hunt Software/Improving Google Docshttp://cananian.livejournal.com/61968.html
<p>There's been a lot of discussion about publishing and sharing the software that Mystery Hunt teams use to collaborate to solve puzzles. This is mostly misguided, IMO: teams are very different, and they organically grow solutions to fit their unique processes. On the other hand, an increasing number of teams (including my team, Codex), are building their collaboration software on top of Google services, especially Google Docs and Spreadsheets. Rather than trying to collaborate on One True Hunt Team Software, I think it would be far more useful (for all teams) to lobby for improvements to Google's services. These raise the bar for everyone, and indirectly benefit many other people with collaborative processes.</p>
<p>So here's my list of improvements I'd like to see in Google services:</p>
<ol>
<li>Integrating Docs and Spreadsheet. If we had an initial "sheet" of
the spreadsheet with editable formatted text (not spreadsheet cells)
we could actually do away with the wiki we use for capturing free form
thoughts and links related to a puzzle.</li>
<li>Integrating Spreadsheet chat, Google Talk, and Jabber. We could
just use the chat in the spreadsheet if it were open and accessible, instead of creating our own per-puzzle chat rooms.</li>
<li>Making publish and "setAnonymous" access available via APIs that
<a href="http://code.google.com/a/google.com/p/apps-api-issues/issues/detail?id=2364">actually work</a>. We need to use Google Doc Script to do the
<a href="http://code.google.com/googleapps/appsscript/service_spreadsheet.html">setAnonymousAccess call</a>, which is not exported via the otherwise-more-complete <a href="http://code.google.com/apis/gdata/docs/directory.html">GData APIs</a>, and drive a headless Firefox 2 instance via <a href="http://seleniumhq.org">Selenium</a> to get
the publish bits enabled for the spreadsheet. That's ridiculous.</li>
<li>Providing a way for Google Doc Scripts on a spreadsheet to export
data more easily. We can use a Google Form to create a spreadsheet
for a puzzle, but no easy way to provide a link to that spreadsheet on
the page that results after form submission, or <a href="http://www.google.com/support/forum/p/Google%20Docs/thread?tid=6bf51c58b611f412&amp;hl=en">redirect from there</a>.</li>
<li>IIRC Google Talk support for multi-user chat is still <a href="http://developer.pidgin.im/ticket/3360">barely supported</a>. It doesn't use the standard Jabber
protocols, for one. If this were a better supported/more
standard service, we wouldn't have to run our own Jabber server.</li>
<li>And, of course, the "next generation" of all this would integrate
audio and video into the chat as well. Video is probably more useful,
as it communicates human emotional cues. Audio isn't easily archived
or searchable, and doesn't work well in crowded rooms, so it is less
useful to us. But it would be great if we could actually <i>see</i>
some/many/most of the particants in ringhunters, maybe little live
video icons next to their faces.</li>
</ol>
<p>Any further suggestions from other teams?</p>http://cananian.livejournal.com/61968.htmlmystery huntcollaborationgooglepublic11http://cananian.livejournal.com/61568.htmlThu, 02 Dec 2010 21:02:49 GMTWhat's Wikileaks up to?http://cananian.livejournal.com/61568.html
<p>A recent <a href="http://www.economist.com/blogs/democracyinamerica/2010/12/after_secrets">article in the Economist</a> points out that Wikileaks is not unique: modern network tools have made anonymous communication ubiquitous. You can't stop "wikileaks" by attacking Julian Assange alone. The article is incorrect, however, in claiming that anonymity is <i>easy</i> &mdash; in some sense <a href="http://en.wikipedia.org/wiki/Federalist_Papers">anonymous leafleteers in Colonial America</a> were better off. <a href="http://en.wikipedia.org/wiki/Bradley_Manning">Bradley Manning currently sits in jail</a>. <a href="http://boingboing.net/2010/09/14/haystack-is-burning.html">Haystack was fundamentally flawed</a>. There continues to be a role for organizations who desire to facilitate anonymous speech to identify and provide trustworthy and user-friendly tools and procedures.</p>
<p>Aaron Brady achieves a more fundamental insight by <a href="http://zunguzungu.wordpress.com/2010/11/29/julian-assange-and-the-computer-conspiracy-%E2%80%9Cto-destroy-this-invisible-government%E2%80%9D/">examining Julian Assange's aims</a>. Assange's goal is to hobble "conspiracies", that is, the small cliques of power and secrecy embedded in most organizations, and he seeks to do this by causing them to fear information sharing. By this metric, Wikileaks <a href="http://swampland.blogs.time.com/2010/11/29/state-pulls-the-plug-on-siprnet/">seems to be succeeding</a>. (Read Aaron Brady's essay for the details.)</p>
<p>But it's worth pausing to consider: are open organizations truly better? Is openness practically achievable? This is an organizational problem which was on the front burner at <a href="http://en.wikipedia.org/wiki/OLPC">OLPC</a> while I worked there: OLPC pledged an open development and governance model, but was continually charged with being closed, insular, and secretive in practice. We reorganized previously-internal mailing lists and pledged to conduct all important business on public archived lists. Yet there was continual backsliding. Sometimes private email was used merely to prevent embarrassment or confusion&mdash;to fact-check before making a public statement. Other times it was claimed that <i>some</i> measure of secret/private communication was a fundamental part of business or negotiation, necessary for interacting with external entities. In order to evaluate the latest components/plans/schedules of our partners, we had to sign NDAs. The secrecy requirements of the third-party then contaminated related discussions. In the end, even an organization with a goal of openness ended up embedding pockets of secrecy, which always threatened to grow and spread unless they were occasionally beaten back. Attempting to stand for open principles was often claimed to make OLPC "uncompetitive," as in: we couldn't hope to get the best deals/access to the latest components/whatever if we insisted on being open about everything.</p>
<p>The quest for openness in business seems to parallel the role of Wikileaks in national affairs. As with OLPC's business negotiations, we are being told that secrecy is an essential part of the diplomatic process, and that publishing internal cables hobbles America's ability to achieve its goals. The claim is that Wikileaks threatens to make America "uncompetitive."</p>
<p>Is this true? Openness is an ethical position, but not a black-and-white one. Very few people argued that OLPC (or America) should have <i>no</i> secrets &mdash; the debate was always "how many?" In practice if the desired answer was "as few as possible", there was always a Wikileaks-like need to continually drag private content to public forums in order to combat the creep of secrecy. Perhaps the same is true of governments.</p>
<p>Then again, over-reaching openness threatens <i>individual privacy</i> &mdash; where to draw the line? Must all our personal mistakes be made in public? Must all our <i>national</i> mistakes be made in public?</p>http://cananian.livejournal.com/61568.htmlsecurityolpcechelonlegalwikileakspublic0