Planet = erlang.http://www.planeterlang.com/atom.xml2017-12-14T02:07:33+00:00http://intertwingly.net/code/venus/Erlang/OTP 20.2 is releasedhttp://www.erlang.org/news/1162017-12-13T00:00:00+00:00img src=http://www.erlang.org/upload/news/
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">Erlang/OTP 20.2 is the second service release for the 20 major release.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">The service release contains mostly bug fixes and characteristics</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">improvements but also some new features.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">Some highlights for 20.2</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<ul>
<li class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60, 60, 60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">crypto, ssl</li>
</ul>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> The crypto API is extended to use private/public keys</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> stored in an Engine for sign/verify or encrypt/decrypt</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> operations.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> The ssl application provides an API to use this new</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> engine concept in TLS.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">
<ul>
<li class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(0, 0, 0); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">ssh</li>
</ul>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(0,0,0); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(0,0,0); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> SSH can now fetch the host key from the private keys</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(0,0,0); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> stored in an Engine. See the crypto application for</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(0,0,0); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> details about Engines.</div>
</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<ul>
<li class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60, 60, 60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> ssl</li>
</ul>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> A new command line option -ssl_dist_optfile has been</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> added to facilitate specifying the many options needed</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> when using SSL as the distribution protocol.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<ul>
<li class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60, 60, 60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">stdlib</li>
</ul>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> Improve performance of the new string functionality</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> when handling ASCII characters.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">You can find the README and the full listing of changes for this</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">service release at</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="http://www.erlang.org/download/otp_src_20.2.readme" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/download/otp_src_20.2.readme</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">The source distribution and binary distributions for Windows can be</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">downloaded from</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="http://www.erlang.org/download/otp_src_20.2.tar.gz" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/download/otp_src_20.2.tar.gz</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="http://www.erlang.org/download/otp_win32_20.2.exe" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/download/otp_win32_20.2.exe</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="http://www.erlang.org/download/otp_win64_20.2.exe" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/download/otp_win64_20.2.exe</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">Note: To unpack the TAR archive you need a GNU TAR compatible program.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">For installation instructions please consult the README file that is</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">part</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">of the distribution.</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">The Erlang/OTP source can also be found at GitHub on the official</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">Erlang</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">repository,<span class="m_1732679952816234514gmail-Apple-converted-space"> </span><a href="https://github.com/erlang/otp" style="color: rgb(45,113,184);" target="_blank">https://github.com/erlang/otp</a><span class="m_1732679952816234514gmail-Apple-converted-space"> </span>with tag OTP-20.2</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">The on-line documentation can be found at:<span class="m_1732679952816234514gmail-Apple-converted-space"> </span><a href="http://www.erlang.org/doc/" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/doc/</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">You can also download the complete HTML documentation or the Unix</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">manual files</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="http://www.erlang.org/download/otp_doc_html_20.2.tar.gz" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/download/otp_doc_html_20.2.tar.gz</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="http://www.erlang.org/download/otp_doc_man_20.2.tar.gz" style="color: rgb(45,113,184);" target="_blank">http://www.erlang.org/download/otp_doc_man_20.2.tar.gz</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">Please report any new issues via Erlang/OTPs public issue tracker</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"><a href="https://bugs.erlang.org" style="color: rgb(45,113,184);" target="_blank">https://bugs.erlang.org</a></div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">We want to thank all of those who sent us patches, suggestions and bug</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">reports!</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">Thank you!</div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;"> </div>
<div class="m_1732679952816234514gmail--x-evo-paragraph" style="color: rgb(60,60,60); font-family: monospace; font-size: 17px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; white-space: normal; width: 71ch;">The Erlang/OTP Team at Ericsson</div>Erlang.org Newshttp://www.erlang.org/rss/newsErlang.org News RSSHot and fresh Erlang.org RSS newshttp://www.erlang.org/rss/newsAlvaro Videla Talks with Brian Troutwine about His Code Mesh Talkyt:video:zJrJptqziHA2017-12-12T13:24:45+00:00Erlang Solutionshttps://www.youtube.com/channel/UCKrD_GYN3iDpG_uMmADPzJQErlang Solutionsyt:channel:UCKrD_GYN3iDpG_uMmADPzJQZomp/zx: Yet Another Repository Systemhttp://zxq9.com/?p=14082017-12-12T08:59:14+00:00<p>I’ve been working on a from-source repo system for <a href="https://www.erlang.org/">Erlang</a> on and off for the last few months, contributing time to it pretty much whenever real-life is not interfering. I’m getting close to making a release. Now that my main data bits are worked out, the rest isn’t all that hard. I need to figure out what I want to say in an announcement.</p>
<p>The problem is that I’m really horrible at announcements and this system does things in a pretty different way to other repository systems out there, so I’m not sure what things are going to be important about it to users (worth putting into an announcement) and what things are going to be important to only <em>me</em> because I’m the one who wrote it (and am therefore obsessed with its externally inconsequential internals). What is internally interesting about a project is almost never what is externally interesting about it. Marketing; QED. So I need to sort that out, and writing sometimes helps me sort that kind of thing out.</p>
<p>I’m making this deliberately half-baked, disorganized, over-long post public because <a href="http://armstrongonsoftware.blogspot.jp/">Joe Armstrong</a> gave me some food for thought the other day. I had written him my thoughts on a subject posted to a mailing list but sent the message in private. I made my message to him off-list for two reasons: first, I wasn’t comfortable with my way of expressing the idea just yet; and second, I am busy with real-life stuff and side projects, including the repo system, and don’t want to get sucked into online chatter that might amount to nothing more than <a href="http://bikeshed.com/">bikeshedding</a>. (I’m a world-class bikeshedder!) Joe wrote me back asking why I made the reply private, I told him my reasons, and he made me change my mind. He hopes that more people will publish their ideas all the time, good or bad, fully baked or still soggy — because that’s the only way we can ever find any other interesting ideas these days is by searching for them, usually in text, on the net somewhere. It isn’t like we can’t go back and <em>revise</em>, but whether or not we do go back and clean up our literary messes, the availability of core ideas and exposure of thought processes are more important than polish. He’s been on a big drive to make sure that he posts most of his thoughts to public mailing lists or blogs so that his ideas get at least indexed and archived. On reflection I agree with him.</p>
<p>So here I am, trying to publicly organize my thoughts on my repository system.</p>
<p>I should start with the goals of the system.</p>
<p>This system is intended to smooth over a few points of pain experienced when trying to get a new Erlang project off the ground, and in particular avert the path of pain peculiar to Erlang newcomers when they encounter the “how to set up a project” problem. Erlang’s tooling is great but a bit crufty (<em>deeply</em> featured, but <em>confusing</em> to interface with) and not at all what the kool kids expect these days. And anyway I’m really just trying to scratch my own itch here.</p>
<p>At the moment we have two de facto standards for publishing Erlang systems: <a href="https://erlang.mk/">erlang.mk</a> and <a href="https://github.com/rebar/rebar">Rebar</a>. I like both of these, especially erlang.mk, but they do one thing that annoys me and never seems to quite fit my need: they build <a href="http://erlang.org/doc/design_principles/release_structure.html">Erlang releases</a>.</p>
<p>Erlang releases are great. They cut all the cruft of a release out and pack <em>everything</em> needed to actually run a system into a single blob of digits that you can move, in a single shot, to a new target system — including the Erlang runtime itself. Awesome! Self-contained deployment <em>and it never misses</em>. This has been an Erlang feature since before people even realized that they needed repeatable deployment infrastructure outside of the classic “let’s build a monolithic, static binary executable” approach. (Erlang is perpetually ahead of its time, even by today’s standards. I look at the poor kids stubbing their toes with <a href="https://thehftguy.com/2016/11/01/docker-in-production-an-history-of-failure/comment-page-1/">Docker</a> and language du jour and just shake my head — though part of that is because many shops are using Docker to solve concurrency issues that they haven’t even become cognizant of, thinking that they are experiencing “scaling” problems but missing the point entirely.)</p>
<p>Erlang releases are awesome when the deployment target is an embedded system, but not so awesome if the target is a full-blown operating system, VM, container, or virtual environment fully stocked with gobs of memory and storage and flush with system utilities and resources. Erlang releases sort of kitchen-sink the deployment itself. What if you want to run several different Erlang programs, all delivered as releases, all depending on the same library? You’ve got <em>tons</em> of copies of that library. Which is OK, but still sort of weird, because you <em>also</em> have <em>tons</em> of copies of the runtime (among other things). Each release is self-contained and lean, but in aggregate this is a bit odd.</p>
<p>Erlang releases make sense when you’re deploying to a phone switch or a sensor device in the middle of nowhere and the runtime is basically acting as its own operating system. Erlang releases are, in that context, analogous to putting a <a href="https://wiki.gentoo.org/wiki/Stage_tarball">Gentoo stage 3</a> binary image on a system to leapfrog most of the toolchain process. Very cool when you’re in that situation, but a bit tinker-tacky when you’re just trying to run, say, a client program written in Erlang or test a web front-end for something that uses <a href="http://yaws.hyber.org/">YAWS</a> or <a href="https://ninenines.eu/">Cowboy</a>.</p>
<p>So that’s the siloed-kitchen-sink issue. The other issue is that <em>newcomers are perpetually confused about releases</em>. This makes teaching elementary Erlang hard. In my view we should really focus on escript for beginner code — just let the new guy run something out of a single file <a href="http://principles-wiki.net/principles:principle_of_least_surprise">the way he is used to doing</a> when <a href="http://www.programmingforbeginnersbook.com/blog/when_you_know_the_basics_but_you_still_cant_code/">learning a new language</a> instead of showing him pages of really slick code, then some interpreter stuff, and then leaping straight from that to a complex and advanced packaging setup necessarily tailored for conducting embedded deployments to slim hardware devices. Seriously. WTF. <a href="http://erlang.org/doc/man/escript.html">Escripts</a> give beginners all the power of Erlang necessary for exploring the more interesting bits of code and refactoring needed to learn <a href="http://erlang.org/doc/getting_started/seq_prog.html">sequential Erlang</a> with the <em>major advantage</em> of being able to interface with the system the same way programmers from other environments are used to dealing with langauge runtimes like <a href="https://www.linux.com/learn/writing-simple-bash-script">Bash</a>, <a href="http://www.grymoire.com/Unix/Awk.html">AWK</a>, <a href="http://ged.msu.edu/angus/tutorials/programming-scripts.html">Python</a>, <a href="https://stackoverflow.com/questions/166347/how-do-i-use-ruby-for-shell-scripting">Ruby</a>, <a href="https://learn.perl.org/first_steps/">Perl</a>, etc.</p>
<p>But what about that gap between scripts and full-blown production deployments for embedded hardware?</p>
<p>Erlang has… nothing.</p>
<p>That’s right! There is no agreed-upon way to deploy or even run Erlang code in the same manner a Python coder would expect to execute a python program. There is no <a href="http://docs.python-guide.org/en/latest/dev/virtualenvs/">virtualenv</a> type system, there is no standard answer to the question “if I’m in the project directory and type <code>./do_thingy</code> it will just work, right?” The answer is always “Well, it depends…” and what actually winds up happening is that people either roll a <em>whole release</em> just to crank a trivial amount of code up or (quite often) implement an ad hoc way to get the same effect in a lighter-weight way. (erlang.mk shines here, actually.)</p>
<p>Erlang <em>does</em> provide a number of ways to make a system run locally from source of .beam files — and has actually quite reasonable built-in resources for this — but nothing has been built around these tools that also deals with external dependencies, argument passing in a standard way, or any of the other little things you really need if you want to claim a complete solution. Hence all the ad hoc solutions that “<a href="https://blog.codinghorror.com/the-works-on-my-machine-certification-program/">work on my machine</a>” but certainly aren’t something you expect your users to use (not with broad success, anyway).</p>
<p>This wouldn’t be such a big problem if it weren’t for the fact that not having any standard way to “just run a program” <em>also</em> means that there really isn’t any standard way to deal with <em>client side code</em> in Erlang. This is a big annoyance for me because much of what I do is client-side code. In Erlang.</p>
<p>In fact, it totally boggles my mind that client-side Erlang isn’t more common, especially considering that AMD is already fielding zillion-core processors for <em>desktops,</em> yet <em>most languages are fundamentally single-threaded</em>. That doesn’t mean you can’t do concurrency and parallelism in other languages, but most problems are <em>not</em> parallel in nature to begin with (parallel problems are easy to write solutions to in any language) while most real-world problems <em>are</em> concurrent. But <em>concurrent</em> systems are hard to write in almost <em>every language</em>. Concurrent problems are the bulk of the interesting problems we’re still not very good at solving with computers. AMD is moving to make the tools available to make much more interesting concurrent processing tools available on the client side (which means Intel will soon start pouring it gajillions worth of blood diamond money into a similar effort), but most languages and environments have no good way to make use of that on the client side. (Do you see why I hear Lady Fortune knocking?)</p>
<p>Browsers? Oh yeah. That’s a great plan. Have you noticed that most sites slowly move toward the “Single Page App” design over time (read as: the web sucks, so now we write full-but-crippled client-programs and deliver them over the web), invest heavily in do-sneaky-things-without-telling-you JavaScript and try to hog every core your system has if you allow it the slightest permission to do so? No. In the age of bitcoin miners embedded in nearly every ad this is not the direction I think we should be envisioning things going.</p>
<p>I want to take better advantage of the cores users have available, and that doesn’t necessarily mean make more efficient use of every cycle as much as it means to <em>make scheduling across processes more efficient</em> to reduce <em>latency</em> throughout the system overall. That’s something users care about quite a lot. This is the problem Erlang has already solved in a way no other runtime out there has. So I want to capitalize on it.</p>
<p>And yet, there is still not standardish way of dealing with code from source, running it locally, declaring or resolving dependencies, <em>or even launching a client-side program at all</em>.</p>
<p>So… how am I approaching it?</p>
<p>I have a project called “zomp” which is a repository system. It is a distributed repository system, so not everything has to be held in one place. Code in the zomp universe is held in little semantic silos called “realms”. Each realm can have whatever packages the owner (sysop) wants it to have. Each realm must have one server node somewhere that is its “prime” — the node in charge of that realm. That node is where system operator tasks for that realm take place, packagers and maintainers submit code for inclusion, where the package index is built, where the canonical copy of everything is stored. Other nodes configured to see that realm connect to the prime node and receive a copy of the current indexes and are tested for availability and published as available resources for querying indexes or downloading packages.</p>
<p>When too many subordinate nodes connect to a prime the prime will redirect a new node to a subordinate, when a subordinate gets “full” of subordinates itself, it picks a subordinate for new redirects itself, etc. so each realm winds up forming a resource tree of mirror nodes that connect back to the realm prime by a single path. A single node might be prime for several realms, or other nodes may act as prime for different realms — and any node can be configured to become a part of any number of realm trees.</p>
<p>That’s the high-level code division.</p>
<p>The zomp constellation is interfaced with via the “zx” program (short for “zomp explorer”, or “zomp exchanger”, or “Zomp eXtreem!”, or homage to the <a href="http://oldcomputers.net/zx81.html">Sinclair ZX-81</a>, or whatever else might lend itself to the letters “zx” that you might want to make up — I actually forget what it originally stood for, but it is remarkably convenient to type so it’s staying that way)</p>
<p>zx is configured to have visibility on zomp realms the same way a zomp node is (in fact, they use the same configuration files and it isn’t weird to temporarily host a zomp node on your desktop the same way you might host a torrent node for a while — the only extra effort is that you <em>do</em> have to open a port, zomp doesn’t (yet) do hole punching magic).</p>
<p>You can tell zx to run a program using the highly counter-intuitive command:</p>
<pre>zx run Realm-ProgramName[-Version]</pre>
<p>It breaks the program name down into:</p>
<ul>
<li>Realm (optional, defaulting to the main realm of public FOSS packages called “otpr”)</li>
<li>Name (necessary — sort of the whole point)</li>
<li>Version (which is optional and can also be partial: “1.0.3” vs just “1.0” or “1”, defaulting to the latest in a series or latest overall)</li>
</ul>
<p>With those components it then contacts any zomp node it knows provides the needed realm, resolves the latest version number of the requested program, downloads and unpacks it, checks and downloads any missing dependencies, builds the program, and launches it. (And if it doesn’t know any active mirrors it asks the prime node and is seeded with known mirror nodes in addition to getting its query answered.)</p>
<p>The packages are kept in a local cache stored at the <em>user</em> level, not the system level (sort of like how browsers keep their JS and page caches) — though if you want to daemonize zomp and run it as a permanent service (if you run a realm prime, for example) then you would want to create an unprivileged system user specifically for the purpose. If you specify a fully-qualified “realm-name-version” for execution and the packages already exist and are built, zx just launches the code directly (which is the majority case, so no delay there — fast startup).</p>
<p>All zomp nodes carry a complete index of their configured realms and can answer queries with very little overhead, but only the prime node has a copy of all the packages for that realm</p>
<p> </p>
<p>Zomp realms are write-only. There is no facility for removing a package from a realm entirely, only for upgrading the versions of packages whenever necessary. (Removal is, of course, possible, but requires manual intervention by the sysop.)</p>
<p>When a zx client or zomp node asks an upstream node for a package and the upstream node does not have a copy it will query its upstream until the request reaches a node that does have a copy. Once found a “found” notice goes back down to the client telling it how many hops away the package is, and new “hops away” notices are sent as the package is passed downstream toward the original requestor (avoiding timeouts and allowing the user to get some feedback about what is going on). The package is cached at each node along the way, so subsequent requests for that same package will be handled immediately without any more relay downloading.</p>
<p>Because the tree of nodes is expected to be relatively ephemeral and in a constant state of flux, the tendency is for package stores on mirror nodes to be populated by only the latest, most popular packages. This prevents the annoying problem with old realms having gobs of packages that nobody uses but mirror hosts being burdened with maintaining them all anyway.</p>
<p>But why not just keep the latest of everything and ditch old packages?</p>
<p>Ever heard of “version shear”? Yeah. Me too. It sucks. That’s why.</p>
<p>There are no “up to” or “greater than” or “abstract version 3” type dependency declarations in zomp package metadata. As a package maintainer you must explicitly declare the complete version of each dependency in your system. In the case of diamond-shaped dependencies (where two packages in your system depend on slightly different versions of the same package) the burden is on the packagers to declare a version that works for a given release of <em>that</em> package. There are no dependency trees for this reason. If your package depends on X, and X depends on Y and Z then your package must be defined as depending on X, Y and Z — and fully specify the versions involved.</p>
<p>Semver is strictly enforced, by the way. That is, all release numbers are “Major.Minor.Patch”. And that’s it. No more, no less. This is one of the primary criteria for inclusion into a public realm and central to the way both zx and zomp interpret package semantics. If an upstream project has some other numbering scheme the packager will <em>need to create a semver standard of his own</em>. And actually, this turns out to not be very hard in practice. There is <em>one</em> weird side-effect of full, static dependency version declarations and semver: updating dependencies results in incrementing your package’s patch number, so even if you don’t change anything in a program for a long time, a program with many dependencies under heavy development may wind up on version 2.3.257 without much change other than the <code>{deps, PackageIDs}.</code> line in the package meta file.</p>
<p>zx helps make you aware of these situations, so solving them has not been particularly difficult in practice.</p>
<p>Why do things this way?</p>
<p>The “static dependencies forever and ever, amen” decision is a tradeoff between the important feature of <em>fully repeatable builds</em> Erlang releases are famous for (to the point of bug-compatibility between deployment sites — which is critical in production) and the <em>flexibility</em> users and developers have come to expect from source repository systems like pip, pypi, CPAN, etc. Because each realm is write-only there is no danger that a package will be superceded and disappear. The way trickle-down caching works for mirror zomp nodes does not unduly burden the subordinate realm mirrors, and the local caching behavior of zx itself at launch time tends to make all of this mostly delay-free for zx clients and still gives them the option to always run “latest available version” if they want.</p>
<p>And on the note of “latest version”…</p>
<p>Client-side programs are not expected to be run too terribly long at a time. People shut desktop programs down, restart computers, update their kernels, etc. So even if a client program runs a long time (on the order of web, email, IRC, certain games, crypto wallets/miners, torrent nodes, Freenode, Tor, etc) it will still have a chance to restart every few days or weeks to check for a new version (if invoked in a way that omits the version number so that it always queries the latest version).</p>
<p>But what about for long-running server-side type programs? When zx starts a script checks the initial environment and then starts the erlang runtime with zx as its target application, passing it the package ID of the desired program to run and its arguments as arguments. That last sentence was odd. An example is helpful:</p>
<pre>zx run foo-bar arg1 arg2 arg3</pre>
<p>zx invokes the launching script (a Bash script on Linux, BSD and OSX, a batch file on Windows — so actually the command is <code>zx.bash</code> or <code>zx.cmd</code>) with the arguments <code>run foo-bar arg1 arg2 arg3</code>. zx receives the instruction “run” and then breaks “foo-bar” into <code>{Realm, Name} = {"foo", "bar"}</code>. Everything after that is passed in as strings which wind up being the input arguments to the program being run: “foo-bar”.</p>
<p>zx registers a process called zx_daemon which remains resident in the runtime and waits for a subscription request or zomp query. Any Erlang program written with the intention of being used with zx can send a message to zx_daemon and ask it to maintain a connection to the program’s parent realm and enroll for update notifications. If the target program itself is the subject of a realm index update then it will get a message letting it know what has changed. The program can respond any way the author wants to such a notification.</p>
<p>In this way it is possible to write a client-side or server-side application that can enroll to become aware of updates to itself without any extra infrastructure and a minimal amount of code. In some programs I’ve used this to cause a pop up notification to appear to desktop users so they know that a new version has become available and they should restart the program (the way Firefox does on Windows). It could also be used to initiate a restart on its own, or whatever else you might come up with.</p>
<p>There are several benefits to developers of using this system as well.</p>
<p>As a developer I can start a new project by doing <code>zx init app [Realm-Name]</code> or <code>zx init lib [Realm-Name]</code> in an existing project root directory and a zomp.meta file will be generated for it, or a new project template directory will be created (populated with a functioning sample skeleton project). I can do <code>zx dailyze</code> and zx will make sure a generally relevant PLT exists or is built (if not up to date) and used to check the typespecs of the project and its dependencies. <code>zx create package [Path]</code> will create a zomp package, sign it, and populate the metadata for it. <code>zomp keygen</code> will generate the kind of keys necessary to interact with a zomp server. <code>zomp submit PackageFilePath</code> will submit a package for review.</p>
<p>And so on.. It is a lot easier to do most things now, and that’s the main point.</p>
<p>(There are commands for reviewing, approving, or rejecting package submissions, adding packagers and maintainers to package projects, adding dependencies to projects, X.Y.Z version incrementing, etc. as well.)</p>
<p>This is about 90% of the way I want it to be, but that means about 90% of the effort remains (pessimistically assuming the 90/10 rule, because life sucks and nobody cares). Most of that is probably going to be finagling some network lunacy, but a <em>lot</em> of the effort is going to be in putting polish to it.</p>
<p>Zomp/zx is based on a similar project I wrote for use within Tsuriai a few years ago that has much sparser features but does basically the same thing: eases packaging and repeatable deployment from source to client systems. I would never release that version publicly because it has a lot of “<a href="https://www.urbandictionary.com/define.php?term=works+for+me">works for me!</a>” level functionality, but <em>very</em> little polish and requires manually diddling quite a few settings files in error-prone ways (which is fine because it was just <em>us</em> diddling them).</p>
<p>My intention here is to Cadillac this out a bit so that newcomers can slide into the new language and just focus on that language after learning a minimum of tooling commands or environmental details. I think <code>zx init app foo-bar</code> and <code>zx runlocal</code> are a low enough bar for entry.</p>zxq9http://zxq9.comerlang – The Intellectual WildernessOn Government: "There is nothing more useless than doing efficiently that which should not be done at all."http://zxq9.comVagrant for Erlang Developmenthttp://stratus3d.com/blog/2017/12/10/vagrant-for-erlang-development2017-12-11T01:46:12+00:00<p>I typically like to do development work on my local machine. Locally I’ve got all my favorite tools, scripts, and aliases along with custom mappings for my editor. Local development is much more pleasant than SSH’ing into a server and running commands. Without all my custom tools and configurations the environment feels foreign to me. Because of this I generally try to avoid solutions to development problems that involve a virtual machine. Even though the VM is running on my laptop it’s really not that much easier to develop on than a regular server.</p>
<p>I’ve known about <a href="https://www.vagrantup.com/">Vagrant</a> for a long time, but I really wasn’t interested in using it because it was easy to setup development environments on my laptop with <a href="https://github.com/asdf-vm/asdf">asdf</a>. Then I encountered a project at work that I wasn’t able to get working on my laptop. I spent hours trying to figure out what was misconfigured, but to no avail. I reluctantly figured I would give Vagrant a try. It seemed like a better option than using a plain VM. It turned out to be very effective. My development with Vagrant is almost seamless now.</p>
<p>In this blog post I’ll cover a few of the issues I ran into when setting up Vagrant for my Erlang project as well as some things I discovered that improved my workflow with Vagrant.</p>
<h2>Installation</h2>
<p>First off you’ll need to install Vagrant and a hypervisor for running the actual VM. I like <a href="https://www.virtualbox.org/">VirtualBox</a> because it is free and open source.</p>
<p>If you are on linux you may be able to use your package manager to install VirtualBox. Version 5.0 is the latest version that Vagrant supports.</p>
<pre><code># Install VirtualBox 5, I'm on Debian so I'm using apt-get
$ sudo apt-get install virtualbox-5.0
</code></pre>
<p>Then install vagrant. You can download it from the <a href="https://www.vagrantup.com/">Vagrant website</a> or if you are on linux you can use your package manager to install it.</p>
<h3>Installation Hiccup</h3>
<p>After installing Vagrant and trying to start it up for one of my projects I realized there was an issue with my VirtualBox installation. It turned out to be due to an option called <code>VT-x</code> being disabled in my BIOS. The error I got when I tried to boot the VM looked like this:</p>
<pre><code>There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.
Command: ["startvm", "dc1a0388-9aab-4ce9-9343-0778af7d1f1d", "--type", "headless"]
Stderr: VBoxManage: error: VT-x is disabled in the BIOS for all CPU modes (VERR_VMX_MSR_ALL_VMX_DISABLED)
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component ConsoleWrap, interface IConsole
</code></pre>
<p>I rebooted my machine, went into the BIOS, and enabled that option. When I booted up my VirtualBox installation worked and no errors were printed.</p>
<p><img alt="Intel VT-d Feature Enable" src="http://stratus3d.com/images/posts/vagrant-erlang/intel-vt-d-feature-enable.jpg" title="Intel VT-d Feature Enable" /></p>
<h2>Setting Up the Environment on the VM</h2>
<p>Once you have Vagrant installed you can begin setting it up for your Erlang project. Navigate to your project on the command line and run <code>vagrant init &lt;box&gt;</code> to generate a <code>Vagrantfile</code> for the project. For this blog post I chose the <code>hashicorp/precise64</code> box, which is Ubuntu 12.04 and seems to be the default box that is used in the Vagrant documentation. Boxes are the package format for Vagrant environments. Boxes contain the base VM image and other metadata. Available boxes are listed on the <a href="https://app.vagrantup.com/boxes/search">Vagrant website</a>. The <code>Vagrantfile</code> in your project root is where you can specify configuration values for your project’s box. Typically there isn’t much that needs to change, but there are plenty of options available. You can set options for network interfaces, synced folders, and the base box image that is used by the VM. I’m not going to cover all that here. The <a href="https://www.vagrantup.com/docs/">Vagrant documentation</a> and the book <a href="http://shop.oreilly.com/product/0636920026358.do">Vagrant: Up and Running</a> are great resources.</p>
<p>Once you have a VM up and running you will need to provision the box. For Erlang development you will need Erlang, Rebar/Rebar3, and optionally Elixir. The <code>Vagrantfile</code> allows us to specify a provisioning script that can be run when the VM is created to install all the tools you will need. I use asdf locally, so I figured I would use it on the VM as well.</p>
<p>The provision script I needed would need to install asdf, install all the necessary asdf plugins, and then install the correct versions of Erlang, Rebar, and Elixir for the project. The script I came up with does all of this:</p>
<pre><code class="sh provision.sh">#!/usr/bin/env bash
# Unoffical Bash "strict mode"
# http://redsymbol.net/articles/unofficial-bash-strict-mode/
set -euo pipefail
#ORIGINAL_IFS=$IFS
IFS=$'\t\n' # Stricter IFS settings
# Install Git and other asdf dependencies
sudo apt-get install -y git automake autoconf libreadline-dev libncurses-dev \
libssl-dev libyaml-dev libffi-dev libtool unixodbc-dev \
build-essential autoconf m4 libncurses5-dev curl
# Install asdf
git clone https://github.com/asdf-vm/asdf.git $HOME/.asdf
(cd $HOME/.asdf; git checkout v0.4.0)
echo -e '\n. $HOME/.asdf/asdf.sh' &gt;&gt; $HOME/.bashrc
echo -e '\n. $HOME/.asdf/completions/asdf.bash' &gt;&gt; $HOME/.bashrc
# Make asdf available in this script
set +u
source "$HOME/.asdf/asdf.sh"
set -u
# Install all the necessary asdf plugins
asdf plugin-add erlang https://github.com/asdf-vm/asdf-erlang.git
asdf plugin-add rebar https://github.com/Stratus3D/asdf-rebar.git
asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git
# Navigate to the directory containing the project (/vagrant is the directory
# that is synced with the project dir on the host)
cd /vagrant
# Make the versions defined .tool-versions file the versions used by the vagrant
# user in any directory
cp .tool-versions $HOME
# Install all correct versions of these packages for the project
asdf install
echo "Completed setup of Erlang environment!"
</code></pre>
<p>asdf expects a <code>.tool-versions</code> file in the project root, so before you have Vagrant run the provision script the <code>.tool-versions</code> file must exist in the project. For my project I needed the latest Erlang and Rebar3 versions but not Elixir, so mine looked like:</p>
<pre><code class="sh .tool-versions">erlang 20.1
rebar 3.4.7
</code></pre>
<p>Now you just need to tell Vagrant to use this script to provision your VM. The <code>config.vm.provision</code> parameter allows us to specify the provision method for the Vagrant box. For a shell script like this you need to add a <code>config.vm.provision</code> line like this:</p>
<pre><code class="ruby Vagrantfile">Vagrant.configure(2) do |config|
# ... omitted other options
config.vm.provision "shell", path: "provision.sh", privileged: false
end
</code></pre>
<p>Vagrant will run the provision script after creating the VM, so if you already have a Vagrant box running run <code>vagrant destroy</code> and <code>vagrant up</code> to have Vagrant setup a new VM and then run the provision script. If the provision script finishes without errors you should have a running Vagrant VM configured for Erlang development!</p>
<h2>Tighter Integration with My Local Environment</h2>
<p>SSH’ing onto the VM to run commands is something I wanted to avoid and it turns out it’s easy to avoid running commands directly on the VM. Vagrant provides the <code>vagrant ssh</code> command which can be used to ssh onto the server, but it can be treated as a regular SSH client, meaning you can use it to run arbitrary commands on the server just like you could with a regular SSH client. To run arbitrary commands use:</p>
<pre><code>$ vagrant ssh -- '&lt;command&gt;'
</code></pre>
<p>For example, to see the IP addresses of the VM run:</p>
<pre><code>$ vagrant ssh -- 'ip address'
</code></pre>
<p>You can also run scripts on the VM like this:</p>
<pre><code>$ vagrant ssh -- &lt; &lt;script&gt;
</code></pre>
<p>This is a lot to type out for simple things so I was eager to find a better way of doing this. It would be nice to not have to type out so much. After <a href="https://stackoverflow.com/questions/47423980/proper-way-to-run-commands-on-vagrant-box">asking some questions</a> I found three ways to make running commands on the VM easier.</p>
<h3>Shell Alias</h3>
<p>The first way to simplify commands is to just create a shell alias for <code>vagrant ssh --</code>. It’s easy to do and makes the commands a lot shorter:</p>
<pre><code># Add this to your .bashrc
alias vc="vagrant ssh --"
# Then you can use it to run commands on the VM:
$ vc 'ip address'
</code></pre>
<p>The downside to this is that you still have to quote the command you want to run.</p>
<h3>vagrant-exec</h3>
<p><a href="https://github.com/p0deje/vagrant-exec">vagrant-exec</a> is a very nice Vagrant plugin that aims to make it easier to run commands on the VM. It offers some very nice features:</p>
<ul>
<li>Uses synced folders to map commands to the right directory on the VM, allowing you navigate around your local environment and run commands in the equivalent on the VM.</li>
<li>It has options for generating shims, which you can add to your <code>$PATH</code> and then run commands locally without a prefix.</li>
<li>It has options for prepending commands with other commands. For example prepend <code>apt-get</code> with <code>sudo</code>.</li>
</ul>
<p>vagrant-exec is a much better choice than shell aliases. It offers more features and tighter integration. The downside is it often requires more work to configure.</p>
<p>What vagrant-exec does isn’t that complicated so I wanted to see if I could write a simplified version of it as a shell script.</p>
<h3><code>va</code> script</h3>
<p>I was able to write a simple Bash script that works similar to vagrant-exec. It lacks many of the features provided by vagrant-exec, but still makes running commands very easy. I named the script <code>va</code> to make it short enough that no alias would be needed. Using the script is very easy. Going back to the IP address example it would just be:</p>
<pre><code>$ va ip address
</code></pre>
<p>Basically all the script does is look at the synced folder mappings configured for the project, and then maps the current directory on the host machine to equivalent directory on the VM. This allows you to easily run directory-specific commands on the host without having to worry about the directory being used on the VM. The output from the command is printed just as if it was run locally.</p>
<p>The source for the script can be found in my <a href="https://github.com/Stratus3D/dotfiles/blob/master/scripts/tools/va">dotfile repo on GitHub</a>. All you need to is put it on your <code>$PATH</code>.</p>
<h2>Conclusion</h2>
<p>Overall Vagrant has been a big help. I was surprised at how much searching I had to do to find a good way of seamlessly running commands on the VM from my local environment. With my <code>va</code> script I’m pretty happy, and I can always use <code>vagrant-exec</code> in the future if I find my <code>va</code> script insufficient.</p>
<p>I still really like developing locally but for times when I can’t run a project locally I’m going to use Vagrant. It’s hard to beat the ease of use and tight integration that Vagrant provides.</p>
<h2>References</h2>
<ul>
<li>asdf: <a href="https://github.com/asdf-vm/asdf">https://github.com/asdf-vm/asdf</a></li>
<li>Vagrant: <a href="https://www.vagrantup.com/">https://www.vagrantup.com/</a></li>
<li>StackOverflow question on running commands: <a href="https://stackoverflow.com/questions/47423980/proper-way-to-run-commands-on-vagrant-box">https://stackoverflow.com/questions/47423980/proper-way-to-run-commands-on-vagrant-box</a></li>
<li>vagrant-exec: <a href="https://github.com/p0deje/vagrant-exec">https://github.com/p0deje/vagrant-exec</a></li>
<li>va script: <a href="https://github.com/Stratus3D/dotfiles/blob/master/scripts/tools/va">https://github.com/Stratus3D/dotfiles/blob/master/scripts/tools/va</a></li>
<li>My dotfiles: <a href="https://github.com/Stratus3D/dotfiles">https://github.com/Stratus3D/dotfiles</a></li>
</ul>Stratus3Dhttp://stratus3d.com/Category: Erlang | Stratus3Dhttp://stratus3d.com/Tomas Petricek Talks with Brian Troutwine about His Code Mesh Talkyt:video:lv2N6RZ5FlM2017-12-09T13:57:58+00:00Erlang Solutionshttps://www.youtube.com/channel/UCKrD_GYN3iDpG_uMmADPzJQErlang Solutionsyt:channel:UCKrD_GYN3iDpG_uMmADPzJQSequential Erlang II by Simon Thompson | 4/13 of Erlang Express Courseyt:video:UREGRvK96e82017-12-08T06:18:58+00:00Erlang Solutionshttps://www.youtube.com/channel/UCKrD_GYN3iDpG_uMmADPzJQErlang Solutionsyt:channel:UCKrD_GYN3iDpG_uMmADPzJQMacro Madness: How to use `use` wellhttps://dockyard.com/blog/2017/12/07/macro-madness-how-to-use-use-well2017-12-07T08:30:21+00:00<p>In Elixir, macros are used to define things that would be keywords in other languages: <code class="inline">defmodule</code>, <code class="inline">def</code>, <code class="inline">defp</code>, <code class="inline">defmacro</code>, and <code class="inline">defmacrop</code> are all macros defined by the standard library in <code class="inline">Kernel</code>. In ExUnit, <code class="inline">assert</code> is able to both run the code passed to it to see if the test is passing, but also print that code when it fails, so that you don’t need a custom DSL to show what was being tested. In GenServer, <code class="inline">use GenServer</code> defines default implementations for all the required callbacks.</p>
<p>If you want a head-trip, look at the implementation of <code class="inline">defmacro</code>, which is defined using <code class="inline">defmacro</code>:</p>
<pre><code class="elixir language-elixir">defmacro defmacro(call, expr \\ nil) do
define(:defmacro, call, expr, __CALLER__)
end</code></pre>
<p><em><a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/kernel.ex#L3547-L3549">Kernel.defmacro/2</a></em></p>
<p>Don’t worry, like all languages defined in themselves, <code class="inline">defmacro</code> is defined using a “bootstrap” library that’s written in the underlying language, in Elixir’s case <a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/src/elixir_bootstrap.erl"><code class="inline">:elixir_bootstrap</code></a> defines minimal versions of <code class="inline">@</code>, <code class="inline">defmodule</code>, <code class="inline">def</code>, <code class="inline">defp</code>, <code class="inline">defmacro</code>, <code class="inline">defmacrop</code> in Erlang: just enough for <code class="inline">Kernel</code> to be parsed once and then it defines the full version. This way, you don’t need the last version of Elixir to build the next version, just Erlang.</p>
<pre><code class="elixir language-elixir">import Kernel, except: [@: 1, defmodule: 2, def: 1, def: 2, defp: 2,
defmacro: 1, defmacro: 2, defmacrop: 2]
import :elixir_bootstrap</code></pre>
<p><em><a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/kernel.ex#L4-L6">Kernel</a></em></p>
<p>Macros allow us to generate code dynamically at compile time. One of the reasons they were added to Elixir was to reduce the amount of boiler plate needed to be written for behaviours, such as <a href="http://erlang.org/doc/man/gen_server.html"><code class="inline">:gen_server</code></a>. In Erlang, this boiler plate was manually added to each file using Emacs templates.</p>
<p>Before the introduction of <code class="inline">-optional_callbacks</code> attribute in Erlang 20, there was no way to add new callbacks without having everyone update their code to add their own copy of the default implementation.</p>
<p>GenServer has 6 <a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#callbacks">callbacks</a> you need to implement. Every GenServer you use would need to have the correct signature and return values for all those callbacks.</p>
<ul>
<li><a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#c:code_change/3"><code class="inline">code_change/3</code></a>
</li>
<li><a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#c:handle_call/3"><code class="inline">handle_call/3</code></a>
</li>
<li><a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#c:handle_cast/2"><code class="inline">handle_cast/2</code></a>
</li>
<li><a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#c:handle_info/2"><code class="inline">handle_info/2</code></a>
</li>
<li><a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#c:init/1"><code class="inline">init/1</code></a>
</li>
<li><a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#c:terminate/2"><code class="inline">terminate/2</code></a>
</li>
</ul>
<p>So, to implement the bare minimum, we can get away with one-liners in most cases, but we need to remember the shape of each of the returns even if we don’t care about <code class="inline">code_change/3</code> for hot-code upgrades. Additionally, the one-liners with <code class="inline">raise</code> won’t type check with dialyzer: it will warn about non-local return, which is just dialyzer’s way of saying you’re raising an exception or throwing. The <a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/gen_server.ex#L576-L638">real code in GenServer</a> is doing more to <a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/gen_server.ex#L597-L600">make dialyzer happy</a> and to give you <a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/gen_server.ex#L591-L594">more helpful error messages that are easier to debug</a>.</p>
<pre><code class="elixir language-elixir">def init(args), do: {:ok, args}
def handle_call(msg, _from, state), do: raise "Not implemented"
def handle_info(msg, state) do
:error_logger.error_msg(
'~p ~p received unexpected message in handle_info/2: ~p~n',
[__MODULE__, self(), msg]
)
{:noreply, state}
end
def handle_cast(msg, state), do: raise "Not implemented"
def terminate(_reason, _state), do: :ok
def code_change(_old, state, _extra), do: {:ok, state}</code></pre>
<p>But, if you read <a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html">the docs for GenServer</a> and know that you don’t need to implement all the callbacks, <a href="https://hexdocs.pm/elixir/1.5.2/GenServer.html#module-use-genserver-and-callbacks">you can put <code class="inline">use GenServer</code> in your callback module and all those default implementation will be defined for you</a>. So, you go from having to hap-hazardly copy default implementations to each callback module to a single line.</p>
<p>Just like <code class="inline">defmodule</code> and the various <code class="inline">def*</code> for call definitions, <code class="inline">use</code> is not a keyword in Elixir, it is <a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/kernel.ex#L4214-L4225">a macro in <code class="inline">Kernel</code></a>, so think of <code class="inline">use</code> as a convention, not a keyword.</p>
<p><code class="inline">use</code> is not magic. It’s very short piece of code that is only complex to give some convenience:</p>
<ol>
<li>It automatically does <code class="inline">require</code>, as <code class="inline">__using__</code> is a macro and macros can’t be used without an explicit <code class="inline">require</code> first
</li>
<li>It uses <code class="inline">Enum.map</code>, so you can pass multiple aliases (<code class="inline">use Namespace.{Child1, Child2}</code>)
</li>
<li>It raises an <code class="inline">ArgumentError</code> if you called it wrong.
</li>
</ol>
<pre><code class="elixir language-elixir">defmacro use(module, opts \\ []) do
calls = Enum.map(expand_aliases(module, __CALLER__), fn
expanded when is_atom(expanded) -&gt;
quote do
require unquote(expanded)
unquote(expanded).__using__(unquote(opts))
end
_otherwise -&gt;
raise ArgumentError, "invalid arguments for use, expected a compile time atom or alias, got: #{Macro.to_string(module)}"
end)
quote(do: (unquote_splicing calls))
end</code></pre>
<p><em><a href="https://github.com/elixir-lang/elixir/blob/v1.5.2/lib/elixir/lib/kernel.ex#L4214-L4225">Kernel.use/2</a></em></p>
<p>If <code class="inline">use</code> just calls the <code class="inline">__using__</code> macro, what is the <code class="inline">__using__</code> macro supposed to do? The only requirement is that it behaves like any other macro: it returns <code class="inline">quote</code>d code. The rest is up to the conventions and best practices in the docs for <code class="inline">Kernel.use</code>.</p>
<h2>Example</h2>
<p>Let’s look at an example of using <code class="inline">__using__</code> and the misteps you can make along the way and how to fix them.</p>
<p><a href="http://lovecraft.wikia.com/wiki/Elder_Thing?file=Screenshot_20171018-090430.jpg"><img alt="An Old One" src="https://i.imgur.com/ueBTmT9.jpg" /></a></p>
<p>While working at <a href="http://lovecraft.wikia.com/wiki/Miskatonic_University">Miskatonic University</a>, <a href="http://lovecraft.wikia.com/wiki/William_Dyer">William Dyer</a> started a compendium of various species the university had encountered. The university’s not mad enough to try to bring them to Earth, so we use a <code class="inline">Client</code> library to establish communication with grad students working in the field.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.OldOnes do
def get(id) do
with {:ok, client_pid} &lt;- client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
defp client_start_link do
Miskatonic.Clients.Portal.start_link(entrance: "witch-house")
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/william-dyer/lib/miskatonic/old_ones.ex"><code class="inline">Miskatonic.OldOnes</code>@william-dyer</a></em></p>
<p><a href="http://lovecraft.wikia.com/wiki/Great_Old_One?file=The_great_old_ones_by_tentaclesandteeth-d7xmcz0.jpg"><img alt="The heads of multiple Great Old Ones merge organically with Cthulhu's head at the base" src="https://i.imgur.com/US6RiZQ.jpg" /></a></p>
<p>While researching the <a href="http://lovecraft.wikia.com/wiki/Elder_Thing">Old Ones</a>, Miskatonic grad students found some of their records referring to greater species that the Old Ones were studying. Because <a href="https://martinfowler.com/bliki/TwoHardThings.html">naming is hard</a>, Miskatonic has started to call them <a href="http://lovecraft.wikia.com/wiki/Great_Old_One">Great Old Ones</a>.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.GreatOldOnes do
def get(id) do
with {:ok, client_pid} &lt;- client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
defp client_start_link do
Miskatonic.Clients.Boat.start_link(
latitude: -47.15,
longitude: -126.72
)
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/gustaf-johansen/lib/miskatonic/great_old_ones.ex"><code class="inline">Miskatonic.GreatOldOnes</code>@gustaf-johansen</a></em></p>
<p>So, we have two modules, that both have a <code class="inline">get</code> function for getting the research on a resource, but how we can communicate with the grad students in the fields differ. We want to make communicating with new and exciting things that want to drive us mad easier because we keep losing grad students, so we need to refactor our two modules and extract the common pieces. Here’s the general shape. There’s a <code class="inline">get/1</code> function that takes an <code class="inline">id</code> and then internally there’s <code class="inline">client_start_link/0</code> function that hides the different ways we communicate with the realms of the different species.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
def get(id) do
with {:ok, client_pid} &lt;- client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
defp client_start_link do
??
end
end</code></pre>
<h3>Using <code class="inline">use</code></h3>
<p>Using the <code class="inline">use</code> convention, we can move <code class="inline">get/1</code> definition into a quote block in the <code class="inline">__using__</code> macro for a new, general <code class="inline">Miskatonic.Species</code> module. We can move <code class="inline">get/1</code> into it, but we can’t move client_start_link in it.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
defmacro __using__([]) do
quote do
def get(id) do
with {:ok, client_pid} &lt;- client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/bob-howard/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Species</code>@bob-howard</a></em></p>
<p>Now we can <code class="inline">use Miskatonic.Species</code> allow us to get rid of the duplicate <code class="inline">get/1</code> code in each module, but we still need the <code class="inline">client_start_link</code> since it differs in each.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.OldOnes do
use Miskatonic.Species
defp client_start_link do
Miskatonic.Clients.Portal.start_link(entrance: "witch-house")
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/bob-howard/lib/miskatonic/old_ones.ex"><code class="inline">Miskatonic.OldOnes</code>@bob-howard</a></em></p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.GreatOldOnes do
use Miskatonic.Species
defp client_start_link do
Miskatonic.Clients.Boat.start_link(latitude: -47.15,
longitude: -126.72)
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/bob-howard/lib/miskatonic/great_old_ones.ex"><code class="inline">Miskatonic.GreatOneOne</code>@bob-howard</a></em></p>
<p><a href="http://thelaundryfiles.wikia.com/wiki/Robert_%2522Bob%2522_Howard?file=BobHoward.jpg"><img alt="Bob Howard in a tactical turtleneck holding a glow hand-held device" src="https://i.imgur.com/PREpfTO.jpg" /></a></p>
<p><a href="http://thelaundryfiles.wikia.com/wiki/Robert_%2522Bob%2522_Howard">Bob Howard</a> gets pulled off the project and sent to <a href="http://thelaundryfiles.wikia.com/wiki/The_Laundry">The Laundry</a>, so a new grad student, Carly Rae <a href="https://aphyr.com/tags/jepsen">Jepsen</a> needs contact with the <a href="http://lovecraft.wikia.com/wiki/Great_Race_of_Yith">Yithians</a>, who Old Ones fought.</p>
<p><a href="http://lovecraft.wikia.com/wiki/Great_Race_of_Yith?file=012.jpeg"><img alt="Great Race of Yith" src="https://i.imgur.com/LxN1d9r.jpg" /></a></p>
<p>Seeing how useful <code class="inline">use Miskatonic.Species</code> was in the other modules, the Carly Rae Jepsen tries the same, but she get a cryptic error message that <code class="inline">client_start_link/0</code> is undefined.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Yithians do
use Miskatonic.Species
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/carly-rae-jepsen-compilation-error/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Yithians</code>@carly-rae-jepsen-compilation-error</a></em></p>
<pre><code class=" language-">== Compilation error in file lib/miskatonic/yithians.ex ==
** (CompileError) lib/miskatonic/yithians.ex:2: undefined function client_start_link/0
(stdlib) lists.erl:1338: :lists.foreach/2
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6</code></pre>
<p><em><code class="inline">mix compile</code></em></p>
<p>Carly Rae tracks down that <code class="inline">Miskatonic.Species</code> depends on <code class="inline">client_start_link/0</code> being defined, but <code class="inline">Miskatonic.Species</code> isn’t currently making the best use of the compiler to tell developers that. Using <code class="inline">@callback</code>, to declare that <code class="inline">client_start_link/0</code> is required by <code class="inline">@behaviour Miskatonic.Species</code> that Carly Rae adds to the <code class="inline">quote</code> block.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
@callback client_start_link() ::
{:ok, pid} | {:error, reason :: term}
defmacro __using__([]) do
quote do
@behaviour Miskatonic.Species
def get(id) do
with {:ok, client_pid} &lt;- client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/carly-rae-jepsen-client-start-link-callback/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Species</code>@carly-rae-jepsen-client-start-link-callback</a></em></p>
<p>So, great, Carly Rae gets a compiler warning now, that’s more specific about why Carly Rae needs <code class="inline">client_start_link</code> in <code class="inline">Miskatonic.Yithians</code>, but it looks like <code class="inline">@callback</code> implementations need to be public, so change all the <code class="inline">defp client_start_link</code> to <code class="inline">def client_start_link</code></p>
<pre><code class=" language-">warning: undefined behaviour function client_start_link/0 (for behaviour Miskatonic.Species)
lib/miskatonic/great_old_ones.ex:1
warning: undefined behaviour function client_start_link/0 (for behaviour Miskatonic.Species)
lib/miskatonic/yithians.ex:1
warning: undefined behaviour function client_start_link/0 (for behaviour Miskatonic.Species)
lib/miskatonic/old_ones.ex:1</code></pre>
<p><em><code class="inline">mix compile</code></em></p>
<p>With the switch to public <code class="inline">client_start_link/0</code>, we can learn about the Old Ones, Great Old Ones, and Yithians, but the code could be better. Although we’re not writing the <code class="inline">def get</code> in every file, it’s being stored in each, which we can see if we ask for the debug info. For one function, this isn’t a big deal, but if we add more and more functions, this is unnecessary bloat, we know it’s exactly the same code. Code loading still takes time with the BEAM even if it’s faster than languages that need to be interpreted from source first.</p>
<pre><code class="elixir language-elixir">iex&gt; {:ok, {module, [debug_info: {_version, backend, data}]}} = :beam_lib.chunks('_build/dev/lib/miskatonic/ebin/Elixir.Miskatonic.Yithians.beam',[:debug_info])
iex&gt; {:ok, debug_info} = backend.debug_info(:elixir_v1, module, data, [])
iex&gt; {:ok, %{definitions: definitions}} = backend.debug_info(:elixir_v1, module, data, [])
iex&gt; List.keyfind(definitions, {:get, 1}, 0)
{:get, 1}, :def, [line: 2, generated: true],
[{[line: 2, generated: true],
[{:id, [counter: -576460752303423100, line: 2], Miskatonic.Species}], [],
{:with, [line: 2],
[{:&lt;-, [line: 2],
[{:ok,
{:client_pid, [counter: -576460752303423100, line: 2],
Miskatonic.Species}}, {:client_start_link, [line: 2], []}]},
[do: {{:., [line: 2], [Miskatonic.Client, :show]}, [line: 2],
[{:client_pid, [counter: -576460752303423100, line: 2],
Miskatonic.Species},
{:id, [counter: -576460752303423100, line: 2],
Miskatonic.Species}]}]]}}]}</code></pre>
<p>The general approach you want to take when making functions in your <code class="inline">__using__</code> <code class="inline">quote</code> block to be as short as possible. To do this, I recommend immediately calling a normal function in the outer module that takes <code class="inline">__MODULE__</code> as an argument.</p>
<p>The reason I recommended always passing in the <code class="inline">__MODULE__</code> is illustrated well here, <code class="inline">module</code> is needed, so that <code class="inline">client_start_link/0</code> can be called in <code class="inline">get/2</code> because it’s outside the <code class="inline">quote</code> block and won’t be in the module that calls <code class="inline">use Miskatonic.Species</code> anymore.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
@callback client_start_link() ::
{:ok, pid} | {:error, reason :: term}
defmacro __using__([]) do
quote do
@behaviour Miskatonic.Species
def get(id), do: Miskatonic.Species.get(__MODULE__, id)
end
end
def get(module, id) do
with {:ok, client_pid} &lt;- module.client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/get-module/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Species</code>@get-module</a></em></p>
<p>Carly Rae Jepsen is doing such a good job on the code that the university doesn’t want to risk her going mad in the field, so Miskatonic University has decided to fund another graduate position on the team. <a href="http://lovecraft.wikia.com/wiki/The_Shadow_Out_of_Time">Nathaniel Wingate Peaslee</a> joins the team and discovers that the Yithian psychic link isn’t limited to just swamping location, but can be used to swap in time. This means to study more of Yithians, the <code class="inline">Miskatonic.Yithians</code> module should try mind transferring to a Yithian in a different time, if getting info on a Yithian fails.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Yithians do
use Miskatonic.Species
def client_start_link(keywords \\ [yithian: "Librarian"]) do
Miskatonic.Clients.Psychic.start_link(keywords)
end
def get(id) do
case Miskatonic.Species.get(__MODULE__, id) do
{:error, :not_found} -&gt;
with {:ok, pid} &lt;- client_start_link(yithian: "Coleopterous") do
Miskatonic.Client.show(pid, id)
end
found -&gt;
found
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/clause-cannot-match/lib/miskatonic/yithians.ex"><code class="inline">Miskatonic.Yithians</code>@clause-cannot-match</a></em></p>
<p>Ah, but Nathaniel seems unable to override <code class="inline">get/1</code> that the <code class="inline">use Miskatonic.Species</code> is inserting. Line 2 is the line where <code class="inline">use Miskatonic.Species</code> is called while line 8 is where Nathaniel wrote the <code class="inline">def get</code>.</p>
<pre><code class=" language-">warning: this clause cannot match because a previous clause at line 2 always matches
lib/miskatonic/yithians.ex:8</code></pre>
<p><em><code class="inline">mix compile</code></em></p>
<p>We can use <code class="inline">defoverridable</code> to any function defined above in a <code class="inline">quote</code> block as overridden if the outer scope defines the same name and arity, instead of the outer scope appending clauses to the same name and arity. Although mixing clauses from <code class="inline">quote</code> blocks and the outer scope is allowed, it’s mostly going to cause confusing bugs, so I recommend always marking any functions defined in a <code class="inline">quote</code> block.</p>
<table>
<thead>
<tr>
<th>defoverridable</th>
<th>No</th>
<th>Yes</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>quote</code> clauses</td>
<td><code>quote</code> clauses</td>
<td><code>quote</code> clauses</td>
</tr>
<tr>
<td><code>defmodule</code> clauses</td>
<td>Both</td>
<td><code>defmodule</code> clauses</td>
</tr>
</tbody>
</table><p>So, Nathaniel marks <code class="inline">get/1</code> as overridable, and the override works without warnings.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
@callback client_start_link() ::
{:ok, pid} | {:error, reason :: term}
defmacro __using__([]) do
quote do
@behaviour Miskatonic.Species
def get(id), do: Miskatonic.Species.get(__MODULE__, id)
defoverridable get: 1
end
end
def get(module, id) do
with {:ok, client_pid} &lt;- module.client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/defoverridable/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Species</code>@defoverridable</a></em></p>
<p>But, he’s able to do more, when you override a <code class="inline">defoverridable</code> function, you can call the overridden function with <code class="inline">super</code>. This allows users of your <code class="inline">__using__</code> macro to not have to look at the implementation of the function they are overriding, which means their code is more likely to continue working if you change implementation details.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Yithians do
use Miskatonic.Species
def client_start_link(keywords \\ [yithian: "Librarian"]) do
Miskatonic.Clients.Psychic.start_link(keywords)
end
def get(id) do
case super(id) do
{:error, :not_found} -&gt;
with {:ok, pid} &lt;- client_start_link(yithian: "Coleopterous") do
Miskatonic.Client.show(pid, id)
end
found -&gt;
found
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/defoverridable/lib/miskatonic/yithians.ex"><code class="inline">Miskatonic.Yithians</code>@defoverridable</a></em></p>
<p>Miskatonic University’s library is doing really well, but it still has some slight bugs: every module has a <code class="inline">get/1</code> and it’s overridable, but it’s not a callback. It may seem weird to mark <code class="inline">get/1</code> as a callback, since only client code calls <code class="inline">get/1</code>, but if we want to make test mocks, to test code that depends on <code class="inline">Miskatonic.Species</code> we really need a <code class="inline">get/1</code> callback. By making <code class="inline">get/1</code> a callback, we can also use the compact form of <code class="inline">defoverridable</code>, that takes the name of the behaviour whose callbacks are overridable, instead of listing each function’s name/arity.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
@callback client_start_link() ::
{:ok, pid} | {:error, reason :: term}
@callback get(id :: String.t) :: term
defmacro __using__([]) do
quote do
@behaviour Miskatonic.Species
def get(id), do: Miskatonic.Species.get(__MODULE__, id)
defoverridable Miskatonic.Species
end
end
def get(module, id) do
with {:ok, client_pid} &lt;- module.client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/defoverridable-behaviour/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Species</code>@defoverridable-behaviour</a></em></p>
<p>One final check that Elixir 1.5 gives us is <code class="inline">@impl</code>. <code class="inline">@impl</code> is like <code class="inline">@Override</code> in Java, but better.</p>
<ol>
<li>Mark which functions are implementations of callbacks
</li>
<li>Document which behaviour a function is for, which makes finding docs and source easier for readers
</li>
<li>Force all other callbacks for the same behaviour to use <code class="inline">@impl</code> to maintain consistent documentation.
</li>
</ol>
<p>In <code class="inline">Miskatonic.Species</code>, there is only one behaviour, but if it was a stack of behaviours, such as building on top of <code class="inline">GenServer</code>, then marking which callbacks are for <code class="inline">GenServer</code> and which are for other behaviours can be very helpful.</p>
<pre><code class="elixir language-elixir">defmodule Miskatonic.Species do
@callback client_start_link() ::
{:ok, pid} | {:error, reason :: term}
@callback get(id :: String.t) :: term
defmacro __using__([]) do
quote do
@behaviour Miskatonic.Species
@impl Miskatonic.Species
def get(id), do: Miskatonic.Species.get(__MODULE__, id)
defoverridable Miskatonic.Species
end
end
def get(module, id) do
with {:ok, client_pid} &lt;- module.client_start_link() do
Miskatonic.Client.show(client_pid, id)
end
end
end</code></pre>
<p><em><a href="https://github.com/KronicDeth/miskatonic/blob/impl/lib/miskatonic/species.ex"><code class="inline">Miskatonic.Species@impl</code></a></em></p>
<h2>TL;DR</h2>
<p>Let’s review Miskatonic University’s finding and thank the graduate students for turning mad, so we don’t have to.</p>
<ol>
<li>We can use <code class="inline">use</code>, which calls <code class="inline">__using__</code>, which calls <code class="inline">quote</code> to inject default implementations
</li>
<li>All <code class="inline">def</code>s in the <code class="inline">quote</code> block should be declared as <code class="inline">@callback</code>s in the outer module where <code class="inline">defmacro __using__</code> is.
</li>
<li>Put <code class="inline">@behaviour</code> with the outer module as the behaviour name at the top of quote block
</li>
<li>The default functions should be one-liners that call functions with the same name in the outer module with <code class="inline">__MODULE__</code> as a prepended argument.
</li>
<li>Mark all default functions with <code class="inline">@impl</code>, as it will force other callbacks for the behaviour to also use <code class="inline">@impl</code> and double check you got the name and arity right between the <code class="inline">@callbacks</code> and implementation in the quote block.
</li>
<li>Use that passed in <code class="inline">__MODULE__</code> whenever you need to call another callback from the outer module functions, so that overrides for any callback will always be called. Don’t call other outer module functions directly!
</li>
<li>Use <code class="inline">defoverridable</code> with the outer module so that you don’t have confusing errors with clauses mixing from the quote block and the <code class="inline">use</code> using module.
</li>
</ol>Luke Imhoffhttps://dockyard.com/blogDockYard bloghttps://dockyard.com/blogWorld, meet Code Sync Conferenceshttp://erlang-solutions.com/blog/world-meet-code-sync-conferences.html2017-12-06T16:41:46+00:00<p>I attended my first Erlang User Conference in 1995. It was my first conference ever. I was an intern at the Computer Science Lab, working on my Master’s thesis with Joe Armstrong. The conference was opened by Erlang System’s manager Roy Bengtson, my future boss. In his opening talk, he announced two new libraries, the <strong>Erlang Term Storage</strong>, and the <strong>Generic Server Module</strong>, as well as the tools which were eventually merged to give us the <strong>Observer</strong>. When attendees complained over the lack of documentation for these tools, Klacke at the CS Lab suggested they write it themselves. </p>
<p>The two day conference had doubled in numbers from its first installation the previous year, with presentations from the Computer Science Laboratory, Erlang Systems, Ericsson and Universities around the world. It was the beginning of something you do not get to experience often. </p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/On2lRba.jpg" /><br /><i>Opening slide from the proceedings of the Second Erlang User Conference 1995</i></p>
<h1>The journey to launching Code Sync</h1>
<p>By 2009, the conference had outgrown the Ericsson conference center in Älvsjö, and the OTP team did not have the infrastructure and flexibility needed to expand the event. We at Erlang Solutions had gained experience in events by running the Erlang eXchange in 2008 followed by the first Erlang Factory in Palo Alto in early 2009. Ericsson asked us to help, so we took over the logistics and worked with them to put together the program. </p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/Hjn5VLL.jpg" /><br /><i>Can you spot me at Erlang User Conference 2006?</i></p>
<p>From these humble beginnings, a conference focused on Erlang expanded to include OTP. Use cases of trade-offs in distributed systems. Talks on cloud infrastructure, orchestration and micro services before the terms were invented. And attempts to make Erlang OO (Not the way Alan Kay intended it) were described and forgotten. The discussions in the hallway track were on the unsuitability of C++ for certain types of problems and around an emerging language called Java. </p>
<p>Fast forward to 2017, the focus from Java has moved to the JVM and its ecosystem. It is Scala, Akka, Groovy, Grails, Clojure and Spring. The same happened with .NET, giving it an ecosystem for C#, F# and Visual Basic to thrive. Erlang’s natural progression was no different. As time progressed, the BEAM came along, and new languages were created to run on it. Reia, by Tony Arcieri was the first (who ever said that a Ruby Flavoured Erlang was a bad idea?) and Efene, a C-flavoured language by Mariano Guerra first presented at the Erlang Exchange in 2008 is still used in production today! </p>
<p>The conferences evolved from a languages conference to a conference on the Erlang Ecosystem, where the BEAM and OTP were used to build scalable and resilient systems. Conferences where communities were exchanging experiences, inspiring and learning from one another. And as we started looking outside of the Erlang ecosystem, our events expanded to include talks on functional programming, concurrency, multi-core and distributed systems. </p>
<p>As a result, the Erlang User Conference, Erlang Factory, and Code Mesh have grown to a roster of global Erlang, Elixir and Alternative Tech conferences which have gone from strength to strength. Who can forget Mike, Joe and Robert on stage bickering together, Martin Odersky joking on how Scala influenced Erlang, Simon Peyton Jones talking about Erlang and Haskell, two childhood friends grew up together or Joe Armstrong interviewing Alan Kay! As of today, we organise five tentpole conferences every year, as well as numerous satellite conferences and a thriving partnership with ElixirConf and Lambda Days.</p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/P4Si9cL.jpg" /><br /><i>Joe Armstrong and Alan Kay in conversation at Code Mesh 2016</i></p>
<p>Last month we took Erlang Factory Lite to the Indian Subcontinent for the first time! This was on the back of a successful event in Buenos Aires this March and a sold out Factory Lite in Rome. This happened alongside some of the best conferences we’ve ever put on, from Erlang and Elixir Factories in San Francisco, Erlang User Conferences in Stockholm, Code Mesh in London to co-organising ElixirConf EU in Barcelona. </p>
<h1>Introducing Code Sync</h1>
<p>On the eve of 2018, the tenth anniversary of our first event, we’re ready for the next phase. I’m excited to announce that all of our conferences are joining the newly launched family of global conferences called <strong><a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz47">Code Sync</a></strong>. Each conference will retain its own personality and stay true to one vision of creating the space for developers and innovators to come together as a community to share their ideas &amp; experiences, learn from one another and invent the future. New name and brand, new colleagues and speakers joining our existing roster of contributors, speakers, and attendees. </p>
<p>Scheduled for next year, we have:</p>
<p style="text-align: center;"><a href="https://www.erlang-solutions.com/news.rss"><img align="centre" src="https://i.imgur.com/yigUhhZ.jpg" /></a></p>
<p style="text-align: center;"><a href="https://www.erlang-solutions.com/news.rss"><img align="centre" src="https://i.imgur.com/duHjiL2.jpg" /></a></p>
<h3>Code BEAM - Discovering the Future of the Erlang Ecosystem</h3>
<p>Previously Erlang and Elixir Factory<br />
<a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz5m">Code BEAM SF, San Francisco - 15 - 16 March 2018</a><br />
<a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz5p">Code BEAM STO, Stockholm - 31 May - 1 June 2018</a><br /><br /></p>
<p style="text-align: center;"><a href="https://www.erlang-solutions.com/news.rss"><img align="centre" src="https://i.imgur.com/YO5UHWO.jpg" /></a></p>
<h3>Code BEAM Lites - Satellite conferences of Code BEAM</h3>
<p>Previously Erlang and Elixir Factory Lite<br />
Various dates &amp; locations<br />
<a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz5r">Milan - 6 April 2018</a><br />
<a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz5r">Berlin - 12 October 2018</a><br /><br /></p>
<p style="text-align: center;"><a href="https://www.erlang-solutions.com/news.rss"><img align="centre" src="https://i.imgur.com/bLjK7T9.jpg" /></a></p>
<h3>Code Elixir - Connecting the Elixir Community</h3>
<p>Previously ElixirLDN<br />
<a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz5t">London - 16 August 2018</a><br /><br /></p>
<p style="text-align: center;"><a href="https://www.erlang-solutions.com/news.rss"><img align="centre" src="https://i.imgur.com/OMSERg9.jpg" /></a></p>
<h3>Code Mesh - Exploring Alternative Tech</h3>
<p>Name unchanged<br />
<a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz4h">London - 8-9 November 2018</a><br /><br /></p>
<p>We are in the early stages of planning Code BEAM Lite events in New York, Budapest, Bangalore, and Bogota. If interested, join <a href="mailto:info@codesync.global">our mailing list</a> and stay tuned. </p>
<p>The creation of the Code Sync family of tech conferences is part of the commitment we have made to open our conferences to a wider audience and to spread the culture to <strong>Learn, Share &amp; Inspire</strong> globally. The Code Sync team has grown from a single person, to a group of five full time employees and an ever growing number of local partners, programme committee members and volunteers. All this wouldn’t have happened without your continuous support - so we hope you will join our Code Sync conferences and become a member of one global community!</p>
<p>The very first Code Sync conference is <strong><a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz5m">Code BEAM SF</a></strong> taking place in San Francisco on 15-16 March. <a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz49">Call For Talks</a> and <a href="http://www2.erlang-solutions.com/l/23452/2017-12-06/56rz4c">Very Early Bird tickets</a> are already open so we hope to see you there!</p>
<ul>
<li>- Francesco<br /><br /></li>
</ul>Erlang SolutionsBlog RSShttp://erlang-solutions.com/news.rssAdvanced RabbitMQ Support Part II: Deeper Insight into Queueshttp://erlang-solutions.com/blog/advanced-rabbitmq-support-part-ii-deeper-insight-into-queues.html2017-12-06T10:05:35+00:00<p><a href="http://www2.erlang-solutions.com/l/23452/2017-12-05/56rkz7"><img src="https://i.imgur.com/kBekpKs.png" /></a></p>
<p>Before you go any further, you should know that you can test WombatOAM out today with a <strong><a href="http://www2.erlang-solutions.com/l/23452/2017-12-05/56rkwh">45 day free trial for WombatOAM 3.0.0</a></strong></p>
<hr />
<h1>Introduction</h1>
<p>The most important and critical elements of any RabbitMQ installation are the Queues. Queues retain messages specific to different use cases across various industrial sectors such as telecommunications, financial systems, automotive, and so forth. Queues, and their adherence to <a href="https://www.rabbitmq.com/amqp-0-9-1-reference.html">AMQP</a> are essentially “<strong>why</strong>” RabbitMQ exists. Not only do they retain messages till consumption, but internally, they are also an implementation of some of the most complex mechanisms for guaranteeing efficient message propagation through the fabric, while catering for additional requirements such as high availability, message persistence, regulated memory utilisation, and so forth.</p>
<p>So queues are general, the main focal point of any RabbitMQ installation. Which is why all RabbitMQ users and support engineers often find themselves having to do regular checks around queues, as well ensuring their host Rabbit nodes have been precisely configured to guarantee efficient message queueing operations. Typical questions that tend to arise from RabbitMQ users and support engineers are;</p>
<p><em><strong>… how many messages are in Queue “A”?</strong></em></p>
<p><em><strong>… how many messages are pending acknowledgement in Queue “K”?</strong></em></p>
<p><em><strong>… how many consuming clients are subscribed to Queue “R”?</strong></em></p>
<p><em><strong>… how much memory is Queue “D” using?</strong></em></p>
<p><em><strong>… how many messages in Queue “F” are persisted on disk?</strong></em></p>
<p><em><strong>… is Queue “E” alive?</strong></em></p>
<p>Within RabbitMQ, the implementation of a queue is a combination of multiple aspects such as the behaviour specification governing its operation (e.g. internally, what is known as the backing queue behaviour), the transient/persistent message store components, and most importantly, the queue process dictating all the logic and mechanics involved in the queueing logic. From these, a number of attributes exist, which give an indication of the current state of the queue. Some of these queue attributes are illustrated below:</p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/2WfLKUS.png" /></p>
<p><em>Fig 1: RabbitMQ Queue Attributes</em></p>
<h1>WombatOAM</h1>
<p>As of <strong><a href="http://www2.erlang-solutions.com/l/23452/2017-12-05/56rkyp">WombatOAM</a></strong> 2.7.0, the WombatOAM-RabbitMQ plugin now ships with an additional agent, the <strong>RabbitMQ Queues</strong> agent. This RabbitMQ Queues agent has been precisely designed and developed to allow monitoring and acquisition of metrics specific to Queues, as well as presenting them in a user friendly manner to RabbitMQ users. Two modes of operation are supported:</p>
<p>Dynamic operation: Queues existing on the monitored node, with names matching to a user defined regex are dynamically loaded by WombatOAM for monitoring.
Static operation: Specific queues are configured and monitored as defined in the WombatOAM RabbitMQ</p>
<h1>Configuration</h1>
<p>The manner in which this agent operates and presents metrics is solely dependant on the way in which it has been configured.</p>
<h2>1. Dynamic operation</h2>
<p>Dynamic mode of monitoring Queues may be configured by defining a match specification, from which queue names are matched against as follows, and the particular, desired attribute/metric from each matched queue. For example, to monitor <code>memory</code> usage of all queues, the following configuration may be defined in the <code>wombat.config</code> file:</p>
<pre class="codehilite"><code>{set, wo_plugins, plugins, rabbitmq_queues, dynamic_queues, [{match_spec, ".*"},
{metric, memory}]
}.
</code></pre>
<p>This will capture all queues on the node being monitored and present <code>memory</code> metrics from queues.</p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/U2CBZK2.png" /></p>
<p><em>Fig 2: RabbitMQ Dynamic Queue Metrics</em></p>
<h2>2. Static operation</h2>
<p>In static mode of operation, users explicitly specify Queues and corresponding attribute/metric they would like to monitor in the <code>wombat.config</code> . A complete static configuration entry would consist of the <em>Queue Name</em>, <em>Virtual Host</em>, and the <em>Attribute</em> being measured. For example, to monitor the number of <code>messages</code> , <code>consumers</code> and amount of <code>memory</code> utilisation from the <code>SERVICES.QUEUE</code> , and number of <code>messages</code> only, from the <code>EVENTS.QUEUE</code>, a user may specify the following configuration from the <code>wombat.config</code> file:</p>
<pre class="codehilite"><code>{set, wo_plugins, plugins, rabbitmq_queues, static_queues,
[{&lt;&lt;"SERVICES.QUEUE"&gt;&gt;, &lt;&lt;"/"&gt;&gt;, messages},
{&lt;&lt;"SERVICES.QUEUE"&gt;&gt;, &lt;&lt;"/"&gt;&gt;, memory},
{&lt;&lt;"SERVICES.QUEUE"&gt;&gt;, &lt;&lt;"/"&gt;&gt;, consumers},
{&lt;&lt;"EVENTS.QUEUE"&gt;&gt;, &lt;&lt;"/"&gt;&gt;, messages}]}.
</code></pre>
<p><strong>Configuring Static Queues is of extreme importance if your mission critical queues which you need continuous visibility of metrics such as messages counts and memory usage</strong></p>
<p>The following illustrates an example of static mode:</p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/0XGkABS.png" /></p>
<p><em>Fig 3: RabbitMQ Static Queue Metrics</em></p>
<h1>Taking “things” further!</h1>
<p>Coupling together our discussion of monitoring Queues, together with discussion with <a href="http://www2.erlang-solutions.com/l/23452/2017-12-05/56rkpf">Part-1</a> of this series of carrying out <strong>advanced alarming operations for RabbitMQ</strong> operations, imagine how many alarming cases we could achieve by defining alarms specific to Queue metrics?</p>
<p>Not only does WombatOAM provide us with a huge spectrum of alarming cases we could handle, but useful metrics. Imagine how useful the following alarms would be:</p>
<p><em><strong>“an alarm which when triggered would send your team email notifications indicating that the number of messages in your most critical SERVICE.QUEUE has just reached the 500 000 message, limit without messages being consumed?”</strong></em></p>
<p>Or:</p>
<p><em><strong>“an alarm configured to issue email notifications when the number of consuming clients falls below a certain minimum permissible number, indicating there’s a critical service affecting problem on the client end”</strong></em></p>
<p>or even more interesting:</p>
<p><em><strong>“an alarm and email notification issued when a queues individual memory usage exceeds a certain cap value, beyond which would be an indication of one or more problems manifesting in the cluster.”</strong></em></p>
<p>Defining such alarms could be as simple as configuring the following in <code>wombat.config</code> as illustrated here.</p>
<p style="text-align: center;"><img align="centre" src="https://i.imgur.com/esZGdVF.png" /></p>
<p><em>Fig 4: RabbitMQ Queue Alarms</em></p>
<h1>Conclusion</h1>
<p>So with these capabilities in mind, imagine the total number Queue specific metrics attainable for monitoring on WombatOAM? The number can be immense, and only limited by the total number of queues you have running, along with the number of attributes you have configured/setup for monitoring. All this is dependant on your configuration. To be precise, a total of 16 attributes are configurable per queue on WombatOAM, meaning a total of <strong>16 x <em>N</em></strong> queue specific metrics are <strong>attainable</strong> (Wow!). So imagine a queue count of ~50 or more queues on a RabbitMQ installation? The number of attainable metric capabilities becomes crazy! That’s ~50 x 16 = a staggering <strong>800 metrics!!!</strong></p>
<p>WombatOAM also provides ability to order queues as desired since the number of available queue metrics has the potential to be extremely large. The rate at which metrics are acquired is also configurable. If you desire to reduce frequency of which metrics are gathered (which is recommended when you have an extremely large number of queues, and queue metrics configured), this can be carried out by simply updating configuration.</p>
<hr />
<p><strong>Erlang Solutions offers world-leading RabbitMQ consultancy, support &amp; tuning solutions. <a href="http://www2.erlang-solutions.com/l/23452/2017-12-05/56rkyf">Learn more &gt;</a></strong></p>Erlang SolutionsBlog RSShttp://erlang-solutions.com/news.rssWhat's new in Elixir - Dec/17http://feeds.feedburner.com/blog/2017/12/05/whats-new-in-elixir2017-12-05T00:00:00+00:00<p>Today’s post marks the first in a new series bringing you the latest changes to the Elixir language.
We’d love to hear from you about what you’d like to see in future posts so join the conversation on <a href="https://elixirforum.com/t/whats-new-in-elixir-discussion-dec-17/10605">the Elixir Forum thread</a>.</p>
<p>So what’s in master? Let’s have a look:</p>
<ol>
<li>
<p>Disagreements about formatting are a thing of the past!
As part of 1.6 we’ve added <a href="https://hexdocs.pm/elixir/master/Code.html#format_string!/2">a code formatter to Elixir</a>.
The formatter is available in projects via <a href="https://hexdocs.pm/mix/master/Mix.Tasks.Format.html#content">the mix task <code class="highlighter-rouge">format</code></a>. <a href="https://github.com/elixir-lang/elixir/issues/6643">The community already helped format all files in the Elixir codebase</a> and you can <a href="https://hashrocket.com/blog/posts/format-your-elixir-code-now">give the formatter a try now</a>.</p>
</li>
<li>
<p>The all new <code class="highlighter-rouge">DynamicSupervisor</code> behaviour is now available on master.
Unlike the traditional <code class="highlighter-rouge">Supervisor</code> strategies, the <code class="highlighter-rouge">DynamicSupervisor</code> allows children to be added dynamically via <code class="highlighter-rouge">start_child/2</code>.
For more on the <code class="highlighter-rouge">DynamicSupervisor</code> check out the <a href="https://hexdocs.pm/elixir/master/DynamicSupervisor.html">documentation</a>.</p>
</li>
<li>
<p>Look for changes in compiler diagnostics as part of this new release that make integration with editors easier.
An all new <code class="highlighter-rouge">Mix.Task.Compiler</code> behaviour will ensure existing and future compilers meet a common specification and return adequate diagnostics.
These changes will enable editors to provide better support for Elixir code compilation.
Jake Becker, one of the features contributors, outlined these benefits in his blog post <a href="https://medium.com/@JakeBeckerCode/elixirls-0-2-better-builds-code-formatter-and-incremental-dialyzer-be70999ea3e7">ElixirLS 0.2: Better builds, code formatter, and incremental Dialyzer</a>.</p>
</li>
<li>
<p>Improvements to the <code class="highlighter-rouge">mix xref</code> task should make it easier for developers to make sense of the output.
These improvements include the new <code class="highlighter-rouge">graph --format stats</code> command and a new option for all xref commands <code class="highlighter-rouge">--include-siblings</code>, for umbrella projects.
For more information on xref changes checkout the CHANGELOG <a href="https://github.com/elixir-lang/elixir/blob/0e72d4839cda97edce75ca0c537555ce4ead7a6a/CHANGELOG.md#mix-xref">entry</a>.</p>
</li>
<li>
<p>Stream data and property testing will be joining Elixir core in a future release. Not only will these be useful to users of Elixir but they’ll be used to make Elixir itself better! <a href="https://elixir-lang.org/blog/2017/10/31/stream-data-property-based-testing-and-data-generation-for-elixir/">See our previous announcement for more information</a> and give the <a href="https://github.com/whatyouhide/stream_data">stream_data library</a> a try.</p>
</li>
</ol>
<p>Think we missed something? Let us know <a href="https://elixirforum.com/t/whats-new-in-elixir-discussion-dec-17/10605">at the Elixir Forum</a>.</p>Sean Callanhttp://elixir-lang.orgElixir Langhttp://elixir-lang.org