Topics

Featured in Development

Peter Alvaro talks about the reasons one should engage in language design and why many of us would (or should) do something so perverse as to design a language that no one will ever use. He shares some of the extreme and sometimes obnoxious opinions that guided his design process.

Featured in AI, ML & Data Engineering

Today on The InfoQ Podcast, Wes talks with Katharine Jarmul about privacy and fairness in machine learning algorithms. Jarul discusses what’s meant by Ethical Machine Learning and some things to consider when working towards achieving fairness. Jarmul is the co-founder at KIProtect a machine learning security and privacy firm based in Germany and is one of the three keynote speakers at QCon.ai.

Featured in Culture & Methods

Organizations struggle to scale their agility. While every organization is different, common patterns explain the major challenges that most organizations face: organizational design, trying to copy others, “one-size-fits-all” scaling, scaling in siloes, and neglecting engineering practices. This article explains why, what to do about it, and how the three leading scaling frameworks compare.

NPM Adopted Rust to Remove Performance Bottlenecks

Npm's exponential growth drove the npm engineering team to switch from Node.js to Rust to handle CPU-bound tasks that were going to become a performance bottleneck. A recent white paper overviews the experience of developing the new service in Rust and running it in production for more than one year.

Most of npm operations are network-bound, which is exactly the scenario that Node.js was designed for. However, npm engineers discovered that the authorization service that ensures only authorized users can publish packages showed a worrying CPU-bound limitation. Npm engineer Chris Dickinson explains that since the authentication service was slightly outdated, the decision to rewrite it from scratch was an easy one to take. Accordingly, npm engineers set out to identify the best language for the task. C and C++ were quickly excluded due to the lack of memory safety, while Java was ruled out due to its dependence on a virtual machine. The two languages that were left were Go and Rust. To better evaluate the final solution, the npm team decided to rewrite their authentication service in both of those languages, as well as in Node.js to have a reference for comparison.

Admittedly, npm authentication service was not extremely complex and the rewrite in Node.js took just one hour. On the other hand, Go required two days, while it took a whole week to rewrite it in Rust, mostly due to a steeper learning curve for Rust and to the intrinsic complexities of the language, explain Dickinson.

You will write a correct program, but you will have to think about all the angles of that correct program.

In the end, the npm team decided to deploy the Rust version of the authentication service mostly thanks to the strong support they got from the Rust community and to the superiority of Rust package manager Cargo, in comparison with what Go offered at that time.

The good news for the npm team is that the Rust service has been running for more than one year in production without a single alert. This is in stark contrast to the usual experience of deploying a Node.js service at npm which includes extensive monitoring to keep errors and resource usage under control. On the negative side of things, the biggest downside of introducing Rust at npm, according to Dickinson, was having to deal with a second stack, including monitoring and logging.