Carmen La
http://carmenla.me/
Carmen's Blog
Tue, 24 Apr 2018 05:22:04 +0000
clj-rss
http://carmenla.me/blog/2016-06-11-scraping-youtube-haikus/
http://carmenla.me/blog/2016-06-11-scraping-youtube-haikus/
Scraping Youtube Haikus from Reddit
<p>While browsing Reddit the other day, I stumbled upon <a href='https://www.reddit.com/r/youtubehaiku'>/r/youtubehaiku</a>. As with any seemingly popular subreddit that I discover, I decided to check out the top posts. The next day I showed the subreddit to my coworker and jokingly suggested that he write a script to scrape the links from Reddit and create a YouTube playlist from them. We spent the next couple of hours doing just that. Here's how I wrote mine.</p><h2 id="the&#95;setup">The Setup</h2><p>I used the <code>app</code> template as the base of the project by running the following command in the terminal:</p><pre><code>lein new app youtubehaikus
</code></pre><p>I knew I'd need dependencies for making http requests, encoding/decoding JSON, and URL parsing so I added the relevant libraries to my <code>project.clj</code></p><pre><code class="clojure">:dependencies &#91;&#91;org.clojure/clojure &quot;1.8.0&quot;&#93;
&#91;clj-http &quot;2.2.0&quot;&#93;
&#91;cheshire &quot;5.6.1&quot;&#93;
&#91;com.cemerick/url &quot;0.1.1&quot;&#93;&#93;
</code></pre><p>Next, I created a new playlist on YouTube and <a href='https://developers.google.com/youtube/v3/guides/auth/server-side-web-apps'>created an OAuth 2.0 access token</a>. These tokens expire after 60 minutes.</p><h2 id="getting&#95;the&#95;reddit&#95;post&#95;data">Getting the Reddit Post Data</h2><p><a href='https://www.reddit.com/dev/api#GET_top'>This</a> is the Reddit API endpoint that we want to use to get the post data. To get the first 100 top posts of all time, we need to specify the query params <code>t=all</code> and <code>limit=100</code> in our request. If you specify the <code>after=FULLNAME</code> param then you can retrieve data about posts that come after a certain post. I be using this optional parameter in order to scrape more than just the top 100 posts of all time.</p><p>We'll start by defining our endpoint as follows:</p><pre><code class="clojure">&#40;def reddit-url &quot;https://www.reddit.com/r/youtubehaiku/top.json?t=all&amp;limit=100&quot;&#41;
</code></pre><p>The body of the response looks something like this:</p><pre><code class="clojure">{:kind &quot;Listing&quot;,
:data {:modhash &quot;&quot;,
:children &#91;...&#93;}}
</code></pre><p>The post data is stored under the <code>:children</code> key so we'll want to retrieve that:</p><pre><code class="clojure">&#40;-&gt; reddit-url
&#40;http/get {:headers {&quot;User-Agent&quot; &quot;thing by /u/me&quot;
&quot;Accept&quot; &quot;application/json&quot;}}&#41;
:body
&#40;json/parse-string true&#41;
&#40;get-in &#91;:data :children&#93;&#41;&#41;
</code></pre><p>Next, we want to extract the YouTube video ids from the links in the posts. Each object in the <code>:children</code> vector is structured like so:</p><pre><code class="clojure">{:kind &quot;t3&quot;,
:data { ...
:url &quot;...&quot;
:name &quot;t3&#95;...&quot;}}
</code></pre><p>and we'll obtain the video ids as follows:</p><pre><code class="clojure">&#40;defn get-path &#91;{:keys &#91;host path query&#93;}&#93;
&#40;if &#40;= &quot;youtu.be&quot; host&#41;
&#40;subs path 1&#41;
&#40;or &#40;get query &quot;v&quot;&#41;
&#40;get query &quot;amp;v&quot;&#41;&#41;&#41;&#41;
&#40;map #&#40;-&gt; % &#40;get-in &#91;:data :url&#93;&#41; url get-path&#41; post-data&#41;
</code></pre><h2 id="creating&#95;the&#95;playlist">Creating the Playlist</h2><p>The last step is to iterate through the video ids and add them to our playlist using the <a href='https://developers.google.com/youtube/v3/docs/playlistItems/insert'>YouTube API</a>.</p><p>However, I wanted to scrape the top 500 videos so I did that by using <code>loop/recur</code> and the <code>after</code> query param in the Reddit endpoint. The <code>FULLNAME</code> that you want to use as the <code>after</code> param is the value associated with the <code>:name</code> key in the last entry of the post data.</p><p>Putting it all together, this was my entire namespace:</p><pre><code class="clojure">&#40;ns youtubehaikus.core
&#40;:gen-class&#41;
&#40;:require &#91;clj-http.client :as http&#93;
&#91;cheshire.core :as json&#93;
&#91;cemerick.url :refer &#91;url&#93;&#93;&#41;&#41;
&#40;def reddit-url &quot;https://www.reddit.com/r/youtubehaiku/top.json?t=all&amp;limit=100&quot;&#41;
&#40;defn get-path &#91;{:keys &#91;host path query&#93;}&#93;
&#40;if &#40;= &quot;youtu.be&quot; host&#41;
&#40;subs path 1&#41;
&#40;or &#40;get query &quot;v&quot;&#41;
&#40;get query &quot;amp;v&quot;&#41;&#41;&#41;&#41;
&#40;defn -main &#91;&amp; args&#93;
&#40;loop &#91;reddit-url reddit-url
pages 5&#93;
&#40;let &#91;post-data &#40;-&gt; reddit-url
&#40;http/get {:headers {&quot;User-Agent&quot; &quot;thing by /u/me&quot;
&quot;Accept&quot; &quot;application/json&quot;}}&#41;
:body
&#40;json/parse-string true&#41;
&#40;get-in &#91;:data :children&#93;&#41;&#41;
last-post-id &#40;get-in &#40;last post-data&#41; &#91;:data :name&#93;&#41;
haiku-ids &#40;map #&#40;-&gt; % &#40;get-in &#91;:data :url&#93;&#41; url get-path&#41; post-data&#41;&#93;
&#40;doseq &#91;id haiku-ids&#93;
&#40;try &#40;http/post &quot;https://www.googleapis.com/youtube/v3/playlistItems&quot;
{:query-params {:access&#95;token &quot;ACCESS&#95;TOKEN&quot;
:part &quot;snippet&quot;}
:content-type :json
:body &#40;-&gt; {:snippet {:playlistId &quot;PLAYLIST&#95;ID&quot;
:resourceId {:kind &quot;youtube#video&quot;
:videoId id}}}
json/generate-string&#41;}&#41;
&#40;catch Exception &#95;&#41;&#41;&#41;
&#40;when-not &#40;= 1 pages&#41;
&#40;recur &#40;str reddit-url &quot;&amp;after=&quot; last-post-id&#41;
&#40;dec pages&#41;&#41;&#41;&#41;&#41;&#41;
</code></pre><p>You can run this from the root directory of the project on the command line by doing <code>lein run</code> or from your repl.</p><p>I wrapped the <code>post</code> in a <code>try/catch</code> block that eats the exception because I was getting some bad responses when attempting to add some of the videos to my playlist - I think it was because some of the videos no longer exist. The link to my playlist is <a href='https://www.youtube.com/watch?v=7DRL6ocEK-M&list=PLJLxMUV2EaMRtYRm5xMsnxquhFzQguB3T'>here</a>.</p>
Sat, 11 Jun 2016 00:00:00 +0000
http://carmenla.me/blog/2015-06-23-reagent-live-markdown-editor/
http://carmenla.me/blog/2015-06-23-reagent-live-markdown-editor/
Reagent Live Markdown Editor
<p>In this post, we'll create a simple live Markdown editor with <a href='https://github.com/reagent-project/reagent'>Reagent</a>.</p><p>The Reagent Cookbook recipe can be found <a href='https://github.com/reagent-project/reagent-cookbook/tree/master/recipes/markdown-editor'>here</a>.</p><h2 id="setting&#95;up&#95;the&#95;project">Setting up the Project</h2><p>Let's start by creating a new reagent-figwheel project.</p><pre><code>lein new reagent-figwheel markdown-editor
</code></pre><p>First, open up the <code>index.html</code> file and add the following scripts below the app div</p><pre><code>&lt;link rel=&quot;stylesheet&quot; href=&quot;//cdnjs.cloudflare.com/ajax/libs/highlight.js/8.6/styles/default.min.css&quot;&gt;
&lt;link rel=&quot;stylesheet&quot; href=&quot;https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.1.1-1/css/united/bootstrap.min.css&quot;&gt;
&lt;script src=&quot;//cdnjs.cloudflare.com/ajax/libs/highlight.js/8.6/highlight.min.js&quot;&gt;&lt;/script&gt;
&lt;script src=&quot;//cdnjs.cloudflare.com/ajax/libs/highlight.js/8.6/languages/clojure.min.js&quot;&gt;&lt;/script&gt;
&lt;script src=&quot;//cdnjs.cloudflare.com/ajax/libs/marked/0.3.2/marked.min.js&quot;&gt;&lt;/script&gt;
</code></pre><p>We'll be using Bootstrap for basic styling and Highlight.js for code syntax highlighting.</p><h2 id="creating&#95;the&#95;editor">Creating the Editor</h2><p>Next, open up the <code>core.cljs</code> file in the <code>src/cljs</code> folder and remove the <code>defonce</code> statement; we won't be needing that.</p><p>The first thing we'll do is set up the editor and create an atom for the Markdown content.</p><pre><code>&#40;defn editor &#91;content&#93;
&#91;:textarea.form-control
{:value @content
:on-change #&#40;reset! content &#40;-&gt; % .-target .-value&#41;&#41;}&#93;&#41;
&#40;defn page &#91;&#93;
&#40;let &#91;content &#40;reagent/atom nil&#41;&#93;
&#40;fn &#91;&#93;
&#91;:div
&#91;:h1 &quot;Live Markdown Editor&quot;&#93;
&#91;:div.container-fluid
&#91;:div.row
&#91;:div.col-sm-6
&#91;:h3 &quot;Editor&quot;&#93;
&#91;editor content&#93;&#93;&#93;&#93;&#93;&#41;&#41;&#41;
</code></pre><p>This code should be familiar to anyone that's used Reagent before.</p><p>Next, we'll add the preview section to the right of the editor. Just add the following under the first <code>col-sm-6</code> div and that's all we need for the UI.</p><pre><code>&#91;:div.col-sm-6
&#91;:h3 &quot;Preview&quot;&#93;
&#91;preview content&#93;
</code></pre><p>The preview component needs to display the parsed and compiled Markdown so let's do that next. We'll be using <a href='https://github.com/chjj/marked'>Marked</a> to handle the Markdown.</p><pre><code>&#40;defn markdown-component &#91;content&#93;
&#40;fn &#91;&#93;
&#91;:div {:dangerouslySetInnerHTML
{:&#95;&#95;html &#40;-&gt; content str js/marked&#41;}}&#93;&#41;&#41;
&#40;defn preview &#91;content&#93;
&#40;when &#40;not-empty @content&#41;
&#40;markdown-component @content&#41;&#41;&#41;
</code></pre><p>This will display the compiled Markdown but it's still missing syntax highlighting for code blocks. The following code will traverse the code nodes in the preview and apply the necessary syntax highlighting.</p><pre><code>&#40;defn highlight-code &#91;html-node&#93;
&#40;let &#91;nodes &#40;.querySelectorAll html-node &quot;pre code&quot;&#41;&#93;
&#40;loop &#91;i &#40;.-length nodes&#41;&#93;
&#40;when-not &#40;neg? i&#41;
&#40;when-let &#91;item &#40;.item nodes i&#41;&#93;
&#40;.highlightBlock js/hljs item&#41;&#41;
&#40;recur &#40;dec i&#41;&#41;&#41;&#41;&#41;&#41;
</code></pre><p>However, we can't just call this function inside <code>markdown-component</code> immediately after the Markdown is compiled. The code syntax cannot be highlighted until the preview component has been mounted on the DOM.</p><p>So in order to do this, we need to post-process the HTML after the preview component has been mounted. We can do this by adding metadata to the <code>fn</code> in <code>markdown-component</code> using <code>with-meta</code>.</p><pre><code>&#40;defn markdown-component &#91;content&#93;
&#91;&#40;with-meta
&#40;fn &#91;&#93;
&#91;:div {:dangerouslySetInnerHTML
{:&#95;&#95;html &#40;-&gt; content str js/marked&#41;}}&#93;&#41;
{:component-did-mount
&#40;fn &#91;this&#93;
&#40;let &#91;node &#40;reagent/dom-node this&#41;&#93;
&#40;highlight-code node&#41;&#41;&#41;}&#41;&#93;&#41;
</code></pre><p>With this metadata, <code>component-did-mount</code> will be called after the HTML has been generated and the node is mounted in the browser DOM.</p><p>Lastly, if we want to build the ClojureScript with advanced compilation we'll have to specify some externs. This is because the compiler munges variable names that come from external libraries, making them unavailable inside the ClojureScript.</p><p>To overcome this, create a file with the following</p><pre><code>var hljs = {};
hljs.highlightBlock = function&#40;&#41;{};
marked = function&#40;&#41;{};
</code></pre><p>and specify it inside your <code>:compiler</code> map in your <code>project.clj</code>. For example,</p><pre><code>:externs &#91;&quot;externs/syntax.js&quot;&#93;
</code></pre><p>For the full source code visit my <a href='https://github.com/lacarmen/reagent-markdown-editor'>GitHub</a> or see the live, styled up demo <a href='https://rawgit.com/lacarmen/reagent-markdown-editor/master/demo/editor.html'>here</a>.</p>
Tue, 23 Jun 2015 00:00:00 +0000
http://carmenla.me/blog/2014-12-31-fizzbuzz/
http://carmenla.me/blog/2014-12-31-fizzbuzz/
Unconditional FizzBuzz
<p>Ah FizzBuzz, the age old programming interview question. For those of you that don't know what FizzBuzz is, here's a quick rundown:</p><p>For numbers 1 through 100</p><ul><li>if the number is divisible by 3, print "Fizz"<ul><li>if the number is divisible by 5, print "Buzz"</li><li>if the number is divisible by 3 and 5, print "FizzBuzz"</li><li>otherwise, print the number </li></ul></li></ul><p>Seems easy right? You'd be surprised at how many people can't do this. I'd probably be surprised too if I hadn't sat on the interviewing side of the table at my previous placement when they were looking for their next co-op student. </p><p><!&ndash;more&ndash;></p><p>A basic solution in Clojure could be as follows:</p><pre><code>&#40;map #&#40;cond &#40;zero? &#40;mod % 15&#41;&#41; &quot;FizzBuzz&quot;
&#40;zero? &#40;mod % 5&#41;&#41; &quot;Buzz&quot;
&#40;zero? &#40;mod % 3&#41;&#41; &quot;Fizz&quot;
:else %&#41;
&#40;range 1 101&#41;&#41;
</code></pre><p>Now in my previous post I talked about Project Euler and how you can solve the problems in numerous different ways. The same goes for FizzBuzz. </p><p>Some popular modifications to the FizzBuzz challenges include: writing the solution recursively, writing it in just one (reasonably long) line of code, using different paradigms, etc. A friend of mine actually mentioned this particular one to me and I thought it was slightly more interesting than the alternatives I've been presented with so far - the unconditional FizzBuzz.</p><p>In other words, write FizzBuzz without using any conditionals such as <code>if</code> or <code>cond</code>. Of course I wrote my solution in Clojure. It's so handy for manipulating data :)</p><pre><code>&#40;let &#91;words &#91;nil &quot;Fizz&quot; &quot;Buzz&quot; &quot;FizzBuzz&quot;&#93;
fizz &#91;1 0 0&#93;
buzz &#91;2 0 0 0 0&#93;&#93;
&#40;map #&#40;&#40;assoc words 0 %&#41; &#40;+ &#40;fizz &#40;rem % 3&#41;&#41; &#40;buzz &#40;rem % 5&#41;&#41;&#41;&#41;
&#40;range 1 101&#41;&#41;&#41;
</code></pre><p>You could write this in one line if you wanted to by getting rid of that let statement but of course, that removes some of the readability. </p><pre><code>&#40;map #&#40;&#40;assoc &#91;nil &quot;Fizz&quot; &quot;Buzz&quot; &quot;FizzBuzz&quot;&#93; 0 %&#41; &#40;+ &#40;&#91;1 0 0 &#93; &#40;rem % 3&#41;&#41; &#40;&#91;2 0 0 0 0&#93; &#40;rem % 5&#41;&#41;&#41;&#41; &#40;range 1 101&#41;&#41;
</code></pre><p>The one liner version of this solution is Twitter friendly as well - 113 characters :)</p><p><em>ps. for extra laughs see <a href='https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpriseEdition'>FizzBuzz - The Enterprise Edition</a></em></p>
Wed, 31 Dec 2014 00:00:00 +0000
http://carmenla.me/blog/2014-12-24-post4/
http://carmenla.me/blog/2014-12-24-post4/
Clojure and Project Euler
<p>A few years ago I really enjoyed doing programming contests - practice ones and real ones. It was a really good way for me to get comfortable with both the concepts and the language when I started programming. Nowadays the contests that I frequented are usually too easy to be a challenge but they also present the same type of challenges year after year.</p><p>This was when I was in high school so I was using Visual Basic at the time. When I entered university and learned Java and Python I decided to take on Project Euler to become familiar with the languages. Although it's been 4 months since I started working with Clojure and I'm not exactly a beginner anymore, I thought it would still be fun to Project Euler since Clojure is my first FP language.</p><h2 id="problem&#95;1&#95;-&#95;<a href='https://projecteuler.net/problem=1'>Multiples of 3 and 5</a>">Problem 1 - <a href='https://projecteuler.net/problem=1'>Multiples of 3 and 5</a></h2><blockquote><p> If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23. Find the sum of all the multiples of 3 or 5 below 1000. </p></blockquote><p>On a side note, when I first saw the title of this problem I thought it was going to be FizzBuzz :)</p><p>A straightforward solution:</p><pre><code>&#40;reduce + &#40;filter #&#40;or &#40;zero? &#40;mod % 3&#41;&#41;
&#40;zero? &#40;mod % 5&#41;&#41;&#41;
&#40;range 1000&#41;&#41;&#41;
=&gt; 233168
</code></pre><p>This generates a vector of numbers up to 1000 and then <code>filter</code> returns a lazy sequence of the elements that satisfy the predicate function. Finally, <code>reduce</code> is used to sum up the remaining numbers.</p><p>The beauty of Project Euler problems is that they can be solved with simple language basics but you can usually rewrite them in more elegant ways. </p><pre><code>&#40;reduce + &#40;set &#40;concat &#40;range 0 1000 3&#41; &#40;range 0 1000 5&#41;&#41;&#41;&#41;
=&gt; 233168
</code></pre><p>By using the third parameter for Clojure's <code>range</code>, you can generate sequences of multiples of 3 and 5. By concatenating these sequences and then turning them into a set, you get rid of the duplicate values. </p><h2 id="problem&#95;2&#95;-&#95;<a href='https://projecteuler.net/problem=2'>Even Fibonacci numbers</a>">Problem 2 - <a href='https://projecteuler.net/problem=2'>Even Fibonacci numbers</a></h2><blockquote><p> Each new term in the Fibonacci sequence is generated by adding the previous two terms. By starting with 1 and 2, the first 10 terms will be: 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ... By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms. </p></blockquote><p>First, define the Fibonacci Sequence:</p><pre><code>&#40;defn fib
&#40;&#91;&#93; &#40;fib 1 2&#41;&#41;
&#40;&#91;x y&#93; &#40;cons x &#40;lazy-seq &#40;fib y &#40;+ x y&#41;&#41;&#41;&#41;&#41;&#41;
</code></pre><p>Using <code>lazy-seq</code> allows us to generate an infinite sequence.</p><p>Now just add up the even elements under 4m.</p><pre><code>&#40;reduce + &#40;take-while &#40;partial &gt;= 4000000&#41;
&#40;filter even? &#40;fib&#41;&#41;&#41;&#41;
=&gt; 4613732
</code></pre><p>First, we use <code>filter</code> to get a sequence of the even Fibonacci numbers. Then we use <code>take-while</code> to return a sequence of elements for which the supplied predicate is true. Here, our predicate is <code>&#40;partial &gt;= 4000000&#41;</code>. Partial takes a function f and fewer than normal arguments to f and returns a function that takes the additional args.</p><h2 id="problem&#95;3&#95;-&#95;<a href='https://projecteuler.net/problem=3'>Largest prime factor</a>">Problem 3 - <a href='https://projecteuler.net/problem=3'>Largest prime factor</a></h2><blockquote><p> What is the largest prime factor of the number 600851475143? </p></blockquote><p>Okay there's a few factorization algorithms out there that can requre a sequence of all the prime factors of a number. If you choose to implement one of these algorithms then you could simply take the max for the answer.</p><p>Here's how I did it:</p><pre><code>&#40;defn largest-prime &#91;num factor&#93;
&#40;if &#40;or &#40;&gt; factor &#40;int &#40;Math/sqrt num&#41;&#41;&#41; ; Don't need to check further than this
&#40;= num factor&#41;&#41;
num
&#40;if &#40;zero? &#40;mod num factor&#41;&#41;
&#40;recur &#40;/ num factor&#41; factor&#41; ; If the number is divisible by the factor, divide and iterate
&#40;recur num &#40;inc factor&#41;&#41;&#41;&#41;&#41; ; Otherwise try with a bigger factor
&#40;largest-prime 600851475143 2&#41;
=&gt; 6857
</code></pre><blockquote><p> <strong>Edit:</strong> A redditor has kindly informed me that <code>&#40;&gt; &#40;&#42; factor factor&#41; num&#41;</code> is faster than <code>&#40;&gt; factor &#40;int &#40;Math/sqrt num&#41;&#41;&#41;</code> since taking a square root is slower than squaring. </p></blockquote><p>I start off with <code>factor = 2</code> since 2 is the smallest prime. If <code>factor</code> is greater than the square root of <code>num</code> or if they're equal then the search stops there. (The floor of the square root of a number is the upper bound for its largest prime factor.)</p><p>Next, if <code>num</code> is divisible by <code>factor</code> then call <code>largest-prime</code> again with <code>&#40;/ num factor&#41;</code> and <code>factor</code>. Otherwise, increment <code>factor</code> and keep trying.</p><h2 id="problem&#95;4&#95;-&#95;<a href='https://projecteuler.net/problem=4'>Largest palindrome product</a>">Problem 4 - <a href='https://projecteuler.net/problem=4'>Largest palindrome product</a></h2><blockquote><p> A palindromic number reads the same both ways. The largest palindrome made from the product of two 2-digit numbers is 9009 = 91 × 99. Find the largest palindrome made from the product of two 3-digit numbers. </p></blockquote><p>First, a little helper function to determine if a number is a palindrome:</p><pre><code>&#40;defn palindrome? &#91;n&#93;
&#40;= &#40;reverse &#40;str n&#41;&#41; &#40;seq &#40;str n&#41;&#41;&#41;&#41;
</code></pre><p>Then filter out the palindrome products and find the max:</p><pre><code>&#40;apply max
&#40;filter palindrome?
&#40;for
&#91;a &#40;range 100 1000&#41;
b &#40;range 100 1000&#41;&#93;
&#40;&#42; a b&#41;&#41;&#41;&#41;
=&gt; 906609
</code></pre><p>The <code>for</code> structure shown here will iterate through b for each iteration of a. In other words, it's a nested for-loop.</p><p><code>apply</code> is an interesting function. Calling <code>&#40;apply max &#91;1 2 3&#93;&#41;</code> is equivalent to <code>&#40;max 1 2 3&#41;</code>. </p><h2 id="problem&#95;5&#95;-&#95;<a href='https://projecteuler.net/problem=5'>Smallest multiple</a>">Problem 5 - <a href='https://projecteuler.net/problem=5'>Smallest multiple</a></h2><blockquote><p> 2520 is the smallest number that can be divided by each of the numbers from 1 to 10 without any remainder. What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20? </p></blockquote><p>This problem is pretty straightforward and can be solved with high school math.</p><p>The greatest common divisor of two non-zero integers is the largest positive integer that divides the numbers without a remainder. </p><pre><code>&#40;defn gcd &#91;a b&#93;
&#40;if &#40;zero? a&#41;
b
&#40;recur &#40;mod b a&#41; a&#41;&#41;&#41;
</code></pre><p>The lowest common multiple of two numbers is the smallest number (not zero) that is a multiple of both.</p><pre><code>&#40;defn lcm &#91;a b&#93;
&#40;/ &#40;&#42; a b&#41; &#40;gcd a b&#41;&#41;&#41;
</code></pre><p>Finally, use <code>reduce</code> to find the lowest common multiple of all the numbers from 1 to 20 inclusively.</p><pre><code>&#40;reduce lcm &#40;range 1 21&#41;&#41;
=&gt; 232792560
</code></pre><p>This will find the LCM of 1 and 2, then the LCM of the result of that and 3 and so on. Thus, the LCM of all the numbers from 1 to 20.</p><p>I'll leave off here before this post gets too lengthy. Project Euler (and a lot of other stuff) is a lot more enjoyable when working with Clojure so I'll likely do a part two eventually.</p>
Wed, 24 Dec 2014 00:00:00 +0000
http://carmenla.me/blog/2014-12-08-post3/
http://carmenla.me/blog/2014-12-08-post3/
Splitting Out Cryogen's Compiler
<p> As Cryogen continued to grow over the past few weeks, a couple of users - including myself - found it increasingly cumbersome to update our blogs with the newest features in the template. It seemed the general approach was to do another <code>lein new cryogen</code> and then copy over the updates. Obviously this is very error prone and merging is a pain for those who have their own modifications to the template. </p><p> Seeing as how Cryogen is pretty stable now, pulling out the compiler code into a library was the clear direction to go in and was also agreed upon by the current users. </p><p> With that, I'm happy to report that I've migrated everything except for <code>server.clj</code> from Cryogen's <code>src</code> folder to <a href='https://github.com/lacarmen/cryogen-core'>Cryogen-core</a>. This library is also on <a href='https://clojars.org/cryogen-core'>clojars</a> so you can simply add the dependency to your project. You can still create a new Cryogen template with <code>lein new cryogen my-blog</code>, but this will be the new file structure: </p><pre><code> my-blog
|---.gitignore
|---project.clj
|---resources
| |---templates
| | |---css
| | | |---screen.css
| | |---js
| | | |---highlight.pack.js
| | |---html
| | | |---layouts
| | | |---archives.html
| | | |---base.html
| | | |---home.html
| | | |---page.html
| | | |---post.html
| | | |---tag.html
| | |---md
| | |---pages
| | | |---about.md
| | | |---another-page.md
| | |---posts
| | |---10-03-2014-first-post.md
| | |---11-04-2014-second-post.md
| | |---13-11-2014-docs.md
| |---404.html
| |---config.edn
|---src
| |---cryogen
| | |---server.clj
</code></pre> <p> If you have your own modifications to the compiler, you can do a redef in your project to override the default behaviour. When there are updates to the compiler you can simply bump up the version in your project while keeping your modifications. </p><p> On one more note, there was an issue where local resource links would have to be absolute such as: </p><pre><code> !&#91;pic01&#93;&#40;/blog/img/pic01.png&#41;
</code></pre><br /><p> Where <code>/blog</code> is your blog-prefix specified in your config. If you changed your blog prefix though you would have to manually search and replace all of the <code>/blog</code>'s in your markdown files so I've added a simple transformer function, <code>rewrite-hrefs</code>, in the compiler amend this. </p><pre><code> &#40;defn rewrite-hrefs
&#91;{:keys &#91;blog-prefix&#93;} text state&#93;
&#91;&#40;clojure.string/replace text #&quot;href=.?/|src=.?/&quot; #&#40;str &#40;subs % 0 &#40;dec &#40;count %&#41;&#41;&#41; blog-prefix &quot;/&quot;&#41;&#41;
state&#93;&#41;
&#40;defn parse-content
&#91;rdr config&#93;
&#40;md-to-html-string
&#40;-&gt;&gt; &#40;java.io.BufferedReader. rdr&#41;
&#40;line-seq&#41;
&#40;s/join &quot;\n&quot;&#41;&#41;
:heading-anchors true
:replacement-transformers &#40;conj transformer-vector &#40;partial rewrite-hrefs config&#41;&#41;&#41;&#41;
</code></pre> <p> When <code>parse-content</code> is called, the replacement transformer will process the html generated by <code>md-to-html-string</code> and inject the blog prefix in front of any local resource links in your markdown. For example, if your blog prefix was <code>/blog</code>, </p><pre><code> !&#91;pic01&#93;&#40;/img/pic01.png&#41;
</code></pre> <p> would become </p><pre><code> &lt;img src=&quot;/blog/img/cryogen.png&quot;&gt;
</code></pre> <p> once the post gets compiled. Do note that the leading slash in front of the local resource is required. </p><p> These are the things that have been happening with Cryogen lately. And as usual, if you want new features added please feel free to submit an issue or pull request. :) </p>
Mon, 08 Dec 2014 00:00:00 +0000
http://carmenla.me/blog/2014-11-26-post2/
http://carmenla.me/blog/2014-11-26-post2/
Updates and Architecture
<p>It's been about two weeks since I released Cryogen to the public and the feedback has been great so I'd like to go over some of the changes that have been made thus far and a how everything fits together behind Cryogen.</p><p><!&ndash;more&ndash;></p><h2 id="updates">Updates</h2><p>Someone pointed out to me that posts or pages that were deleted from <code>resources/templates</code> would have to be manually deleted from <code>resources/public</code> in order for the sitemap and RSS feed to be updated correctly since these two files are generated based on the contents of the <code>public</code> folder.</p><p>To remedy this, I updated the compiler to wipe out the <code>public</code> folder and then generate all of its contents again with each compile. I was actually worried that this would increase compile time but <a href='http://yogthos.net/'>Yogthos's blog</a> has actually moved to Cryogen and found no problems with performance even with his 60+ existing posts.</p><p>Previously, any asset folders like css and javascript were stored under <code>public</code> but then have now been moved to <code>templates</code>. <em>Everything</em> in the public folder is now generated by the compiler. I've added the <code>:resources</code> key to the config that lets you specify which asset folders you would like to copy over from <code>templates</code> to <code>public</code>.</p><pre><code class="clojure">:resources &#91;&quot;css&quot; &quot;js&quot; &quot;img&quot;&#93;
</code></pre><p>The copying is done by this simple function in <code>src/cryogen/io.clj</code> using <a href='https://github.com/Raynes/fs'>fs</a>.</p><pre><code class="clojure">&#40;defn copy-resources &#91;{:keys &#91;blog-prefix resources&#93;}&#93;
&#40;doseq &#91;resource resources&#93;
&#40;fs/copy-dir
&#40;str &quot;resources/templates/&quot; resource&#41;
&#40;str public blog-prefix &quot;/&quot; resource&#41;&#41;&#41;&#41;
</code></pre><p>Cryogen also has <a href='https://disqus.com/'>Disqus</a> support now. Simply register your blog on Disqus, add your <code>disqus-shortname</code> to the config and set <code>disqus?</code> to <code>true</code> and Selmer will inject the info into the following script in the post template.</p><pre><code>{% if disqus-shortname %}
&lt;div id=&quot;disqus&#95;thread&quot;&gt;&lt;/div&gt;
&lt;script type=&quot;text/javascript&quot;&gt;
&#40;function&#40;&#41; {
var dsq = document.createElement&#40;'script'&#41;; dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = '//{{disqus-shortname}}.disqus.com/embed.js';
&#40;document.getElementsByTagName&#40;'head'&#41;&#91;0&#93; || document.getElementsByTagName&#40;'body'&#41;&#91;0&#93;&#41;.appendChild&#40;dsq&#41;;
}&#41;&#40;&#41;;
&lt;/script&gt;
{% endif %}
</code></pre><p>For long posts or pages with numerous headings, you can now generate a table of contents. Just add <code>:toc true</code> to the metadata of the post or page where you want to generate a toc. For example:</p><pre><code class="clojure">{:title &quot;Updates and Architecture&quot;
:layout :post
:tags &#91;:cryogen :clojure&#93;
:toc true}
</code></pre><p>Some other updates:</p><ul><li>Can now customize the number of recent posts shown<ul><li>Sass support by <a href='https://github.com/turbopape'>turbopape</a></li><li>Archive sorting fixed for users with a locale that isn't "en"</li></ul></li></ul><h2 id="architecture">Architecture</h2><p>Sometimes things are obvious to you when you design them but not necessarily to others. I've received a few questions regarding templating on the Cryogen repo so I thought I'd go over the basic architecture here.</p><p>The reason I created Cryogen was because I wanted a site generator (in Clojure!) that had a clear separation between content and layout. The idea is that you create HTML templates under <code>resources/templates/html/layouts</code> with whatever layout/theme you want and the content of your posts and pages from <code>resources/templates/md</code> gets injected by the compiler through Selmer.</p><p><strong> Post/Page Layouts </strong></p><p>Each markdown file representing a post or page must contain metadata about the layout in the <code>:layout</code> key that corresponds to an HTML file under <code>html/layouts</code>. For example, this post's metadata is as follows:</p><pre><code class="clojure">{:title &quot;Updates and Architecture&quot;
:layout :post
:tags &#91;:cryogen :clojure&#93;}
</code></pre><p>When the site gets built, this is what puts the post together:</p><pre><code class="clojure">&#40;spit &#40;str public &#40;:uri post&#41;&#41;
&#40;render-file &#40;str &quot;templates/html/layouts/&quot; &#40;:layout post&#41;&#41;
&#40;merge default-params
{:servlet-context &quot;../&quot;
:post post
:disqus-shortname disqus-shortname}&#41;&#41;&#41;
</code></pre><p>Selmer will render the post and the compiler will spit it out into the <code>public</code> folder.</p><p>The second argument passed into <code>render-file</code> is a map of items that Selmer will inject into the specified layout. In this case, that will be <code>post.html</code>.</p><pre><code class="django">&lt;div id=&quot;post&quot;&gt;
...
&lt;h2&gt;{{post.title}}&lt;/h2&gt;
...
&lt;div&gt;
{{post.content|safe}}
&lt;/div&gt;
...
&lt;/div&gt;
</code></pre><p>Here, <code>post</code> is another map of items so <code>{{post.title}}</code> and <code>{{post.content}}</code> will render the values of <code>:title</code> and <code>:content</code> from <code>post</code>. Pages and tags are compiled in a similar fashion.</p><p><strong> Templates </strong></p><p>This segues into how inheritance works in the templates. In the <code>layouts</code> folder, <code>base.html</code> contains the header, sidebar and footer of the site. These are the things that remain constant no matter what page or post you are viewing. The page/post content gets injected into the main reading pane with <code>{% block content %}</code>:</p><pre><code>&lt;body&gt;
&lt;!--header--&gt;
&lt;!--sidebar--&gt;
...
&lt;div id=&quot;content&quot;&gt;
{% block content %}
{% endblock %}
&lt;/div&gt;
...
&lt;!--footer--&gt;
&lt;/body&gt;
</code></pre><p>To inherit the base template, all the other html files in the <code>layouts</code> folder start with a line of code that specifies the path of the base layout it should inherit followed by the layout for that page or post wrapped in <code>{% block content %}</code>.</p><p>A short example is the layout for pages.</p><pre><code>{% extends &quot;templates/html/layouts/base.html&quot; %}
{% block content %}
&lt;div id=&quot;custom-page&quot;&gt;
&lt;div id=&quot;page-header&quot;&gt;
&lt;h2&gt;{{page.title}}&lt;/h2&gt;
&lt;/div&gt;
{% if page.toc %}{{page.toc|safe}}{% endif %}
{{page.content|safe}}
&lt;/div&gt;
{% endblock %}
</code></pre><p>And that's it! If you want to change the header, sidebar or footer - change it in <code>base.html</code>. If you want to add more post or page layouts, create a new html file under <code>resources/templates/html/layouts</code> and follow the above structure. If you want to add different filters to your content, please check out the <a href='https://github.com/yogthos/selmer#usage'>Selmer docs</a>.</p>
Wed, 26 Nov 2014 00:00:00 +0000
http://carmenla.me/blog/2014-11-12-post1/
http://carmenla.me/blog/2014-11-12-post1/
Another Year, Another Blog
<p>Welcome to the first post of what is probably my 3rd or 4th attempt at maintaining a blog. The first few times didn't blow over too well. I'm not one for creative writing and my lack of motivation eventually led to the abandonment of my previous blogging attempts.</p><p>The last time I tried to maintain a blog was probably two to three years ago and I was working with WordPress. The templates weren't very palateable and neither was the code. The admin interface was nice but it just wasn't worth it if I only wanted to serve up static content. I recall trying out blogspot but that wasn't my cup of tea either.</p><h2 id="enter&#95;misaki">Enter Misaki</h2><p>Originally, I tried to use <a href='https://github.com/liquidz/misaki'>Misaki</a>, a <a href='http://jekyllrb.com/'>Jekyll</a> inspired static site generator written in Clojure. It was simple at first. There were a few included <a href='https://github.com/weavejester/hiccup'>hiccup</a> layouts and a few example posts (also written in hiccup). But I soon found that this was going to be much more complicated than I had anticipated.</p><p><!&ndash;more&ndash;></p><p>I encountered my first problem when I wanted to display the most recent blog entry on the home page. After searching through the <a href='http://liquidz.github.io/misaki/api/index.html'>API</a>, I concluded that this was the best and only way to display the contents of the latest post: </p><pre><code class="clojure">&#91;:div
&#40;-&gt;&gt; &#40;:posts site&#41;
first
:lazy-content
force&#41;&#93;
</code></pre><p>Except... this wasn't quite right. Misaki's compiler escaped html tags when it populated the lazy content. I guess this was an easy fix though. I simply modified function from the line of code where the lazy content was populated. </p><p>My biggest complaint was that Misaki didn't have out of the box support for Markdown posts. There's actually another version called <a href='https://github.com/liquidz/misaki-markdown'>Misaki Markdown</a> but any pages that I wanted to add had to be written in markdown as well. What I wanted was to be able to create custom pages with html/css and write my posts in markdown. </p><p>I ended up altering <code>misaki.server</code> in order to preprocess markdown posts into html posts before compiling the entire site. The first three functions below are what I added.</p><pre><code class="clojure">&#40;defn- find-assets &#91;f ext&#93;
&#40;-&gt;&gt; f
file-seq
&#40;filter &#40;fn &#91;file&#93; &#40;-&gt; file .getName &#40;.endsWith ext&#41;&#41;&#41;&#41;
rest&#41;&#41;
&#40;defn preprocess &#91;file&#93;
&#40;with-open &#91;rdr &#40;clojure.java.io/reader file&#41;&#93;
&#40;-&gt;&gt; &#40;doall &#40;line-seq rdr&#41;&#41;
&#40;split-with #&#40;.startsWith &#40;.trim %&#41; &quot;;&quot;&#41;&#41;
&#40;map #&#40;clojure.string/join &quot;\n&quot; %&#41;&#41;&#41;&#41;&#41;
&#40;defn compile-markdown &#91;&#93;
&#40;doseq &#91;file &#40;find-assets &#40;clojure.java.io/file &quot;md&quot;&#41; &quot;md&quot;&#41;&#93;
&#40;spit &#40;str &quot;my-blog/template/posts/&quot;
&#40;first &#40;clojure.string/split &#40;.getName file&#41; #&quot;\.&quot;&#41;&#41;
&quot;.html.clj&quot;&#41;
&#40;let &#91;&#91;header content&#93; &#40;preprocess file&#41;&#93;
&#40;str header &quot;\n&quot; &quot;&#91;:div &quot; &#40;with-out-str &#40;pprint &#40;md/to-html content&#41;&#41;&#41; &quot;&#93;&quot;&#41;&#41;&#41;&#41;&#41;
; =do-all-compile
&#40;defn do-all-compile
&#91;&#93;
&#40;compile-markdown&#41;
&#40;print-compile-result &quot;all templates&quot; &#40;call-all-compile&#41;&#41;
&#40;println &quot; &#42; Finish Compiling&quot;&#41;&#41;
</code></pre><p>Setting up my site to my specifications took way longer than it should have and I couldn't help but think there must be another way.</p><h2 id="introducing&#95;cryogen">Introducing Cryogen</h2><p>All this resulted in <a href='https://github.com/lacarmen/cryogen'>Cryogen</a>, a simple static site generator written in Clojure by <a href='https://github.com/yogthos'>Yogthos</a> and I.</p><p><strong>Features</strong></p><ul><li>Blog posts and pages with Markdown<ul><li>Twitter Bootstrap theming</li><li>Plain HTML page templates with <a href='https://github.com/yogthos/selmer'>Selmer</a></li><li>Code syntax highlighting with <a href='https://highlightjs.org/'>Highlight.js</a></li><li>Tags</li><li>Sitemap</li><li>RSS feed</li></ul></li></ul><p><strong>Setting Up Cryogen</strong></p><p>The easiest way to get Cryogen set up is to use <a href='https://github.com/technomancy/leiningen/'>Leiningen</a>. Once you have Leiningen 2.5.0 or above installed, simply run the following command to create a new cryogen template.</p><pre><code>$ lein new cryogen &lt;project name&gt;
</code></pre><p>Alternatively, you can put <code>cryogen</code> in the <code>Leningen Template to use:</code> field if you are creating a new Leiningen project in Eclipse with Counterclockwise.</p><p><strong>Project Structure</strong></p><p>The new project will have the following structure:</p><pre><code>my-blog
|---.gitignore
|---project.clj
|---resources
| |---config.edn
| |---templates
| | |---css
| | | |---screen.css
| | |---js
| | | |---main.js
| | |---img
| | | |---favicon.png
| | |---html
| | | |---layouts
| | | |---archives.html
| | | |---base.html
| | | |---home.html
| | | |---page.html
| | | |---post.html
| | | |---tag.html
| | |---md
| | |---pages
| | | |---about.md
| | | |---another-page.md
| | |---posts
| | |---10-03-2014-first-post.md
| | |---11-04-2014-second-post.md
| | |---13-11-2014-docs.md
|---src
| |---cryogen
| | |---compiler.clj
| | |---io.clj
| | |---rss.clj
| | |---server.clj
| | |---sitemap.clj
| | |---watcher.clj
</code></pre><p>The skeleton site has a few example posts and pages in it. </p><ul><li><code>templates/md</code> - blog posts written in Markdown<ul><li><code>templates/pages</code>- custom pages written in Markdown</li><li><code>templates/css</code> - save your custom CSS files</li><li><code>templates/js</code> - save your custom js files here</li></ul></li></ul><p><strong> Compiling Posts and Running the Server </strong></p><p>The server can be started using the <code>lein-ring</code> plugin. Running the following in the root folder of your Cryogen project will compile your resources to <code>resources/public</code> then start the site at <a href='http://localhost:3000'>http://localhost:3000</a></p><pre><code>$ lein ring server
</code></pre><p>That's it for getting the basic template up and running. Additional docs can be found on the <a href='https://github.com/lacarmen/cryogen'>github page</a>.</p><hr/><p>So here we go! Another attempt at maintaining a blog - this time using Cryogen. It's actually my first noteworthy project using Clojure and hopefully that will encourage me to write here regularly. I'd also like to give a huge thanks to <a href='http://yogthos.net/'>Yogthos</a> for helping me out with my first project!</p>
Wed, 12 Nov 2014 00:00:00 +0000