If you are a new or unverified account, and are unable to post comment replies, please click here to verify your account. Otherwise, read the Solution Submission Tutorial for a walkthrough of submitting a solution, or click below to message the moderators for assistance.

Write your own challenge!

To help the community and write your own challenge to be submitted, head on over to /r/DailyProgrammer_Ideas and share your project - read the sidebar in that subreddit for more information.

The web is full of creepy stories, with Reddit's /r/nosleep at the top of this list. Since you're a huge fan of not sleeping (we are programmers, after all), you need to amass a collection of creepy stories into a single file for easy reading access! Your goal is to write a web-crawler that downloads all the text submissions from the top 100 posts on /r/nosleep and puts it into a simple text-file.

Formal Inputs & Outputs:

Input Description:

No formal input: the application should simply launch and download the top 100 posts from /r/nosleep into a special file format.

Output Description:

Your application must either save to a file, or print to standard output, the following format: each story should start with a title line. This line is three equal-signs, the posts's name, and then three more equal-signs. An example is "=== People are Scary! ===". The following lines are the story itself, written in regular plain text. No need to worry about formatting, HTML links, bullet points, etc.

Sample Inputs & Outputs:

If I were to run the application now, the following would be examples of output:

=== Can I use the bathroom? ===

Since tonight's Halloween, I couldn't... (your program should print the rest of the story, I omit that for example brevity)

=== She's a keeper. ===

I love this girl with all of my... (your program should print the rest of the story, I omit that for example brevity)

Next time use the reddit APIs, either the XML or the JSON API to grab the data and format it correctly. That will make your code a lot nicer to look at.
The code you wrote is pretty good, however the logic needs to be split up a more. The printStories method is to vague, and takes too much on. The printStories method is taking the responsibility of reading from http, parsing the http and printing out the http to file! There are lots of ways you could split the logic out but I feel each of these should have there own method and return the appropriate response. One rule I try to use is, if I can't explain what my method does in a single statement then its probably doing to much.

There are lots of good JSON java libraries out there, just have google. You will need to download the jar file and add that file to your classpath to use it in IDE. As for XML you can either use the parsers which comes with Java (javax.xml), or go online to find another XML library. Checkout the other Java examples on this thread too!

Plain Nokogiri. Didn't look at the other solutions here as I didn't want to feel too inspiried by you guys. Found out that my solution isn't as effective and is really slow compared ankederosine and the_mighty_skeetadon's awesome solutions! Didn't even catch the json part! :/