Archive for category Uncategorized

Around a month has passed since our talk at mloc.js. There we had a great oppor­tu­nity to talk about the tech­nol­ogy behind the duetto C++ com­piler for the Web. We also made our first pub­lic announce­ment of our promise based sup­port for asyn­chro­nous type safe RPC, which fits in our vision of C++ as the plat­form lan­guage for the Web, both on the client and on the server side.

Today, we announce the new, feature-packed release of Duetto — 0.9.4! This time, we really splurged, and added some qual­ity fea­tures that we’re sure you’ll find interesting:

Ini­tial sup­port for promise-based async RPC

Strong inter­op­er­abil­ity with JavaScript

__asm__ to exe­cute JavaScript from C++ code

[[jsex­port]] attribute to use C++ objects from JavaScript

Sup­port for C++ mutex/atomic/thread headers

As usual, you can find the full source code of the release here, plus binary pack­ages for Debian/Ubuntu on our PPA, a binary setup for win­dows and a binary archive for Mac OS X here.

Asyn­chro­nous RPC using promises

With duetto you can “tag” a method to let the com­piler know that the code should be com­piled for the server side. The com­piler also takes care of gen­er­at­ing RPC boil­er­plate includ­ing type safe seri­al­iza­tion of para­me­ters and dese­ri­al­iza­tion of the return value. While the basic model is syn­chro­nous, you can take advan­tage of promises to use fully type safe asyn­chro­nous RPC in duetto. See e.g. the fol­low­ing code:

// Pending requests for messages
vector<Promise<string>*> pendingRequests;
Promise<string>* getChatMessageRemote() [[server]]
{auto ret=new Promise<string>();// Store the new promise, you can use it later to complete // the request in asynchronous mannerpendingRequests.push_back(ret);return ret;
}
void sendMessageRemote(const string& str) [[server]]
{for (auto p: pendingRequests)p->done(str);pendingRequests.clear();
}
void messageHandler(const string& newMessage) [[client]]
{client::console.log("Message received",newMessage.c_str());
}
void webMain() [[client]]
{ // First wait for messagesauto promise = getChatMessageRemote();// Add a callback to the promisepromise->then(messageHandler);// Tell the promise that no more callbacks will be addedpromise->complete();// Now send a message, it will be echoed backsendMessageRemote("Test message");
}

The code is a sim­ple but work­ing exam­ple of a multi-user chat appli­ca­tion. The getChatMes­sageR­e­mote method returns a promise imme­di­ately after being called, with­out wait­ing for the server answer. On the server side we cre­ate a promise and return it. We also need to store it some­where to even­tu­ally ful­fill the promise by send­ing a value.

The client code needs to set a call­back to receive the value when the promise will be even­tu­ally ful­filled. Later on the client send a chat mes­sage using the sendMes­sageR­e­mote server method. This method is syn­chro­nous and will return only after the server side exe­cu­tion is com­pleted. The server echoes the mes­sage back to all con­nected clients by ful­fill­ing all pend­ing promises.

Strong JS interoperability

This release of duetto includes two fea­tures that pro­vide a supe­rior inte­gra­tion with man­u­ally writ­ten JavaScript code: the [[jsex­port]] class attribute and __asm__ sup­port for inline JavaScript.

[[jsex­port]] class attribute

We have intro­duced pre­lim­i­nary sup­port for a new attribute: [[jsex­port]]. This attribute is designed to be applied on C++ classes, doing so will cause make the class avail­able to man­u­ally writ­ten JavaScript code. The fol­low­ing exam­ple shows its typ­i­cal use:

The [[jex­port]] attribute places some lim­i­ta­tions on the class. In par­tic­u­lar, the object must have a triv­ial destruc­tor and no over­loaded meth­ods. More­over oper­a­tors can’t be exported to JavaScript. Hav­ing a triv­ial destruc­tor is nec­es­sary to make sure that the reg­u­lar Garbage Col­lec­tion mech­a­nism is suf­fi­cient to reclaim the object. We plan to expand this fea­ture in the future to reduce any lim­i­ta­tions as much as possible.

Inline JavaScript using __asm__

This release also adds sup­port for insert­ing arbi­trary JavaScript code inside C++ code using the __asm__ func­tion­al­ity. Our design choice has been to put no limit on what you can do inside __asm__, includ­ing break­ing the pro­gram, so be care­ful when tak­ing advan­tage of this capability.

At the moment only this sim­ple form of JS code inser­tion is sup­ported. It’s not cur­rently pos­si­ble to pass vari­ables from sur­round­ing C++ code or return a value from __asm__, but we will extend this sup­port in the future.

Sup­port for C++ mutex/atomic/thread headers

Some C++ func­tion­al­i­ties, espe­cially the ones related to multi-threading, do not have a direct map­ping on the browser plat­form. That said, we want to pro­vide as much com­pat­i­bil­ity as pos­si­ble with exist­ing C++ code to reduce the effort when using duetto for porting.

As of duetto 0.9.4, we sup­port the fol­low­ing C++ thread­ing primitives:

mutex: Imple­mented as plain coun­ters, mutexes can­not block exe­cu­tion but report an error if a non-recursive mutex is acquired twice.

atomic: Imple­mented as plain inte­gers, since the browser is not actu­ally con­cur­rent there is no real sup­port for atomic operations.

thread: Mod­ern browsers sup­port Web­Work­ers, but the thread­ing model does not map to the one expected by C++ threads. We chose to make the user aware that C++ threads are not sup­ported by adding an explicit error mes­sage when the thread header is included in code com­piled with duetto.

Today Lean­ing Tech­nolo­gies Ltd. releases duetto, a com­piler designed to develop inte­grated (back­end and fron­tend) web appli­ca­tions in C++. duetto is now avail­able to the pub­lic as FOSS here, and will be offered in a com­mer­cial pack­age with closed-source-friendly licens­ing soon.

Browser-based appli­ca­tions will undoubt­edly play a big role in the future of both con­sumer and busi­ness appli­ca­tions. Web apps have some clear advan­tages com­pared to native appli­ca­tions, like being natively cloud-based and col­lab­o­ra­tive, while also being inher­ently portable, espe­cially thanks to the increased atten­tion that browser ven­dors have ded­i­cated to web stan­dard com­pli­ancy. Mod­ern web stan­dards, that usu­ally go under the umbrella term HTML5, pro­vide incred­i­ble close-to-native capa­bil­i­ties and this gap will be shrink­ing with time.

Sev­eral tools have been devel­oped to help pro­gram­mers in writ­ing large-scale Web appli­ca­tion fron­tends, reduc­ing the lim­i­ta­tions and obsta­cles given by Javascript: Coffe­Script, Microsoft Type­Script, Google Dart, Google GWT and, more recently, emscripten, which com­piles LLVM byte­code (and there­fore C++) to Javascript. On the server side, it is com­mon to use lan­guages such as PHP, Python and Ruby, lead­ing to a sep­a­rate back­end code­base in a lan­guage other than Javascript. This causes dupli­ca­tion of code (and new bugs), hav­ing to deal with RPC man­u­ally, and over­all takes time away from proper devel­op­ment. A very pop­u­lar solu­tion for this has been to use Javascript on the server as well with node.js.

We present duetto, our C++ com­piler for the Web, which is now avail­able to the pub­lic. Our solu­tion inte­grates and sur­passes fea­tures of exist­ing tools, and allows to pro­gram both the fron­tend and the back­end of a Web appli­ca­tion in an inte­grated C++ code­base, com­pil­ing them respec­tively to JavaScript and native code.

Duetto com­bines the advan­tages of emscripten and node.js by allow­ing the pro­gram­mer to:

write web appli­ca­tions in C++, reusing exist­ing code and mak­ing port­ing of whole appli­ca­tions and games to the browser plausible.

code both the fron­tend and the back­end of a web appli­ca­tion in the same lan­guage and codebase

In addi­tion to this, duetto pro­vides some nice features:

Bring the robust­ness and proven scal­a­bil­ity of C++ pro­gram­ming to the Web

You can access all browser APIs directly. Duetto inher­its the C++ phi­los­o­phy of expos­ing the plat­form capa­bil­i­ties (and lim­i­ta­tions) to the users. There is no mid­dle man.

Duetto is based on LLVM/clang. An indus­try stan­dard C++ com­piler is a programmer’s best friend: code san­ity is ver­i­fied as com­pile time, includ­ing RPC signatures.

The LLVM tool­chain also guar­an­tees that a mind-blowing set of opti­miza­tions is run at com­pile time, gen­er­at­ing highly effi­cient code. This decreases the bur­den on JavaScript JIT com­piler at runtime.

Con­trar­ily to emscripten we do not try to emu­late a tra­di­tional address space using typed arrays, but directly map C++ objects to JS objects. This reduces mem­ory con­sump­tion since the garbage col­lec­tor can delete unused object.

Are you inter­ested in using duetto? You can now. We have just released the whole tech­nol­ogy as a FOSS project, avail­able here. Also, we’ve stuck to our promise of releas­ing in six months from our first announce­ment in April

The com­piler itself, derived from LLVM/clang is released with the same license as LLVM/clang (UI/NCSA) and we plan to con­tribute gen­er­ally use­ful code upstream. Accom­pa­ny­ing head­ers and libraries will be released as GPLv2+. We will be also offer­ing closed source friendly licens­ing of such com­po­nents for a fee. Here are links to the var­i­ous repos­i­to­ries on github:

Release tar­balls will be avail­able shortly on the Launch­pad page of the project: https://launchpad.net/duetto. Launch­pad will also be our bug report­ing sys­tem, so please use it to report any prob­lem you might find. We will be also pro­vid­ing builds of duetto for Ubuntu linux using a PPA. We plan to pro­vide binary builds for Mac OS X and Win­dows as well, but we are still work­ing on automat­ing those build.

Please sub­scribe to our Launch­pad group/mailing list to dis­cuss bugs, desired fea­tures and usabil­ity issues on https://launchpad.net/~duetto-users. You can also fol­low us on twit­ter (@leaningtech) and visit our web­site to be updated on duetto related news:

I got as a Christ­mas gift an exter­nal USB audio device: an Asus Xonar U1. It’s a nice device with a decent audio qual­ity. The audio itself, both in and out works per­fectly with the stan­dard snd-usb-audio ker­nel mod­ule. Unfor­tu­nately the audio con­trols on the device are not stan­dard, so I devel­oped a sim­ple dae­mon and accom­pa­ny­ing udev and pm-utils script to get the device to work. I’ve pub­lished every­thing under the GPL hop­ing that they might be use­ful for some­one else. Feel free to clone and fork my repos­i­tory on github

I’m very happy to annouce a new major release for Lightspark, the open source flash player imple­men­ta­tion. This release includes quite a lot of fixes, both inter­nal and vis­i­ble to users. The most impor­tant ones from a user point of view are prob­a­bly the improved sup­port for PulseAu­dio flat vol­umes, which makes it impos­si­ble for lightspark to manip­u­late the sys­tem vol­ume and the newly added sup­port for the BBC video player. To use BBC site you might need to use AdBlock, which is gen­er­ally rec­om­mended since more often than not flash adver­tise­ments are not sup­ported by Lightspark and causes fail­ures in oth­er­wise sup­ported sites.

chal­lenge 4:

This chal­lenge is a lit­tle more dif­fi­cult, but if you know python you have all the skills nec­es­sary to beat it.
Question:

Hello Anony­mous,it has come to our knowl­edge that the Lity­van Secret Ser­vice (note: this was part of a more com­plex story that ran across all the CTF)is installing a remote con­trol sys­tem on their sub­marines. This wouldallow a hand­ful of man to launch a mas­sive attack from a secureloca­tion.This threat is not accept­able: your mis­sion is to break into onesub­ma­rine remote con­trol sys­tem and launch a mis­sile against Navy Con­trol Center,so to shut down the sys­tem for good.The details of this sys­tems are unknown, but we have recov­ered from astolen usb device two files that might give some insights to you: theyare attached. We dis­cov­ered what we sus­pect to be one of the entry pointof this sys­tem: it is located at 10.15.42.42, on TCP port 5000.Should you com­plete this mis­sion, you will be lav­ishly remunerated.Good luck!X

To make it run, unpack this file and run server.py (it’s all python, you can check it for back­doors). To start the chal­lenge, you should read only the con­tent of the “pub” direc­tory: that was the mate­r­ial that was given in the ctf.

Points: 500
Teams that have com­pleted it: 0, as most of the dif­fi­cult chal­lenges — because of the struc­ture of the iCTF 2010, it turned out that it was more con­ve­nient to focus only on the easy chal­lenges. We’ll have to fix it next year!

Solu­tion: in an html com­ment fol­low­ing this line. The given file also con­tains a script that can solve the chal­lenge (the test_* files)

For the solu­tion of another dif­fi­cult chal­lenge, head over to Bryce’s blog.

Finally, after a long delay caused by sev­eral issues Lightspark 0.4.5 is out. Here it is a brief Changelog.

Include the new Advanced Graph­ics Engine, that should pro­vide smoother and faster graph­ics and sup­port for clipping

More robust input sup­port (makes it pos­si­ble to use Play/Pause on YouTube)

The work on this released has been espe­cially slowed down by an issue found in libxml++ and another one found in mesa. Pack­agers should pay atten­tion to ful­fill the fol­low­ing depen­dency requirements:

Libxml++ ver­sion 2.33.1 or bet­ter. If an older ver­sion on libxml++ must be sup­ported some com­mits must be back­ported. More info about this in the README file

Mesa should include the fix for this bug. The issue affects some radeon and maybe intel cards.

Pack­ages will be avail­able for Fedora Rawhide and are already uploaded on the PPA for Ubuntu Mav­er­ick

Me and most of my engi­neer­ing col­leagues here at Sant’Anna School for Advanced Stud­ies are leav­ing tomor­row morn­ing for a week long tour of uni­ver­si­ties and labs in Bel­gium and Nether­lands. This means I’ll have no time to work on Lightspark, sorry

Any­way the 0.4.5 release will be prob­a­bly out in the last days of the month. The crit­i­cal bug in libxml++ is now fixed as my patch was accepted upstream. If (as I believe) a new ver­sion of the library will be released and pack­aged before i get back there will be no rea­son to hold the release any­more.

Another week, another bug­fix release for Lightspark! Apart from restor­ing the sup­port for YouTube this release fea­tures the new plu­gin based audio frame­work that makes it pos­si­ble to sup­port other back­ends beside PulseAu­dio. At the moment both an ALSA and Ope­nAL plu­g­ins are being worked on.

Hello planet!
I’m back from GUADEC. It was my first con­fer­ence about open source and it was great.
I’ve found par­tic­u­larly inspir­ing the talk by Guil­laume Desmottes about Telepa­thy and Epiphany, which can be great to extend GTG pos­si­bil­i­ties in col­lab­o­ra­tion.
The talk by Jake Edge about pro­mot­ing free soft­ware projects was also very inter­est­ing, in par­tic­u­lar for the young Lightspark project (that went com­pletely unno­ticed for a few months before show­ing up on planet GNOME).

Thanks to the excit­ing talks and peo­ple at GUADEC, the GTG team (even the peo­ple who were not there!) has been work­ing fer­vently on a nice rewrite of some parts of GTG core, along with a lot of unit-tests. Hope­fully, a lot of bugs will be closed thanks to this, and GTG will be nicer to code.

Some of the GTG peo­ple at GUADEC. From left to right: Bertrand Rousseau, Karlo Jez, Lionel Dri­cot and me.

As for my Google Sum­mer of Code on Get­ting Things Gnome sup­port for mul­ti­ple back­ends, this week has seen:

a port of my Evo­lu­tion plu­gin as a back­end (that was the last one planned)

refac­tor­ing of the Twit­ter plu­gin to get autho­riza­tion through Oauth (using the tweepy library, thanks Tante for the hint)

docs, docs, docs

Next week, I’ll keep doc­u­ment­ing and test­ing. I should also write a guide on how to write new back­ends. See you next week!

Try­ing to keep up with the old rule “Release early, release often” I’ happy to announce release 0.4.2.2 open source flash player.

This appar­ently small point release actu­ally includes the biggest fea­ture plan­nend for the upcom­ing 0.4.3 release, namely Gnash fall­back on older SWF clips. Lightspark cur­rently relies no Gnash for any Flash con­tent that does not require AVM2 (Action­Script 3) support.

I would also like to explain an issue that many users and testers reported. Fire­fox is not able to han­dle mul­ti­ple plu­gin for the same file type! Not even if only one of those plu­g­ins is actu­ally enabled. So, if lightspark is installed along­side adobe’s player or Gnash no flash con­tent will be dis­played. This is a fire­fox bug, I’ve reported the bug and pro­posed a patch that is cur­rently wait­ing the review.

The source of the release is as always avail­able on launch­pad. Binary pack­ages for Ubuntu Lucid and Mav­er­ick will be avail­able on the usual PPA (in a cou­ple hours from now, Launch­pad seems pretty busy at the moment). More­over, since the last announce­ment lightspark has been also included in the debian exper­i­men­tal suite (thanks to Didier Raboud and Luca Falavigna).