Topics

Featured in Development

Peter Alvaro talks about the reasons one should engage in language design and why many of us would (or should) do something so perverse as to design a language that no one will ever use. He shares some of the extreme and sometimes obnoxious opinions that guided his design process.

Featured in AI, ML & Data Engineering

Today on The InfoQ Podcast, Wes talks with Katharine Jarmul about privacy and fairness in machine learning algorithms. Jarul discusses what’s meant by Ethical Machine Learning and some things to consider when working towards achieving fairness. Jarmul is the co-founder at KIProtect a machine learning security and privacy firm based in Germany and is one of the three keynote speakers at QCon.ai.

Featured in Culture & Methods

Organizations struggle to scale their agility. While every organization is different, common patterns explain the major challenges that most organizations face: organizational design, trying to copy others, “one-size-fits-all” scaling, scaling in siloes, and neglecting engineering practices. This article explains why, what to do about it, and how the three leading scaling frameworks compare.

Caching, NoSQL & Grids - What the Banks Can Teach Us

Summary

John Davies shares insight into SQL, NoSQL, grid, virtualization and caching technologies from his personal experience using them in financial institutions.

Bio

John Davies is Co-founder and CTO of Incept5 and has been intimately involved in implementing Visa's new capabilities. He was previously Chief Architect at JP Morgan and BNP Paribas, co-founder of C24 (sold to Iona) and Technical Director at Progress Software. John has co-authored several enterprise Java and architecture books and is a frequent speaker at banking and technology conferences.

About the conference

Software is changing the world; QCon aims to empower software development by facilitating the spread of knowledge and innovation in the enterprise software development community; to achieve this, QCon is organized as a practitioner-driven conference designed for people influencing innovation in their teams: team leads, architects, project managers, engineering directors.

Community comments

OOM with -Xmx4G -Xms4G

Your message is awaiting moderation. Thank you for participating in the discussion.

Very interesting talk, however if Davies' code threw an OOM despite having 4 gigs of heap, then he has not written it in a streaming fashion. He must be holding on to all the data.

Not only would that explain his OOM, it would also render his timing exercise (1.2 seconds) and thus his comparison to unix cmd line tools (that indeed rock, god bless em!) useless, as there will probably be a lot of wasteful memory allocation and heavy gc action going on when the JVM desperately tries to free up memory while someone is holding on to everything.

Re: Powershell

Your message is awaiting moderation. Thank you for participating in the discussion.

On most OSes, you can ran whatever shell you want.

That said, the standard gnu sort command allows you to sort numerically, alphabetically, and by whatever column you want. There are tons of other options: linux.die.net/man/1/sort

One thing Powershell has that you don't get with the unix tools is proper types. I sometimes wish they had, but then again it would probably make it much less easy to use. You can get amazingly far with just text and conventions on how e.g. columns are separated (tabs? whitespace?), no worrying about types, hierarchies (elements in xml) etc.

Btw, bash and sort and the other guys run on what you seem to consider a decent OS, too. Does that make it an indecent OS?

Re: Powershell

Your message is awaiting moderation. Thank you for participating in the discussion.

"One thing Powershell has that you don't get with the unix tools is proper types" that's what I meant by "piping objects". It's great that sort can take the params for typing - my apologies for the ignorance, but having the type system means all my cmdlets can take advantage of it, w/o having to implement it.

The question of decency should be forwarded to the author of the presentation, I only used his word to make a point. Personally, it's whatever floats your boat, I just hate those not-so-subtle 'my preference is better than yours' comments. I like my git shell for source control, but for manipulating data (the context of the presentation), I'd say having the type system would be a big deal.