Streaming the web, binary json and streaming in javascript

Streaming is an old concept in software, the reason why Unix works so well. It is important that we align with this philosophy when we do web development. During work of my part-time master project, I focused on how to make streaming large files to elastic search efficiently.

Below is outline of talk I would like to share.

Why do we prefer streaming

philosophy

advantages

Common transformation on stream

compression: GZIP/Deflate, when to use

encoding: Discuss overhead of different binary-textual encoding, e.g. base64 is NOT optimal. Base64/Base85/basE91/YEnc

Making it more efficient

Binary JSON

Smile /UBJSON

comparison

Platform-specific

javascript: What we can learn from the streaming model in nodejs

HTML5 stream api

Others for overview, including web socket, SPDY

Speaker: Mr Vincent Lau Chun Yin (Hong Kong)

Developer passionate in technology and the open source community
Speaker of “Introduction to Elastic Search” of last HK open source conference 2013

Graduated from HKU BBAIS / Msc(CS). Full-time java developer in investment bank, meanwhile more interested in node.js and web development.
Starting up the @javascripthk meetup and the @code4hk, which is an organization to gather hackers to work on social-aware applications and meanwhile improve HK’s IT industry.
Keep busy on side projects, include hong kong light pollution map, which is a crowd sourcing open-data site which gather information on light pollution spots and call for action. Also Cheeatz.com, which is now an editor allow user to type markdown to generate cheeatsheet with gist, then sync to evernote.

(This is a a session of Hong Kong Open Source Conference 2014 on 29 March, HKOSC is an annual international conference in Hong Kong for global open source community, it is aimed to demonstrate the use of open source software technology, open hardware technology and latest development in the world. HKOSC is targeted for developers, users and promoters.)