Meta

accounts

So here we are, the culmination of the Accounts system. This final blog post in the series will be on the Password Reset system. It will be a continuation of the Password Recovery system, and takes in concepts from the SignUp Verification system, so if you haven’t read those two yet, I strongly recommend you do so.

From the Password Recovery system, we now know how to accept an email ID from the user (whose email needs to be recovered), and send him an email with a Password Reset link. What we now need to do, is implement the reset link, and make a system which actually resets the password. Again, we’re gonna do this using Servlets.

First up, we needed to figure out the reset link. Here, we’ll do exactly as we did in the SignUp verification: use tokens. We give them a link to the Reset Password page (will talk about it after this) and send in a token so that the link is unique. So in the PasswordRecoveryService servlet, I added the tokens feature as well.

The way I did it is the same as what I wrote in the last article, so I’ll just show the code here, for the explanation please refer to the last article:

We basically get the forgotemail (i.e email ID whose password user forgot which he entered in the Password Recovery page). We check if the email even exists in the Authentication database: if it does, the recovery email is sent to him. The token generation is obvious (resetpass is the Password Reset page as I mentioned), and like in the SignUp verification, we again kept a file reset-mail.txt which is the body of the email we have to send. But, there’s another portion in that code which I redacted. What did I do in there?

Now, the user gets the mail, and he clicks the link. He is led to localhost:9000/apps/resetpass/index.html?token=<random-token> (localhost in this case). What we want now is the program to figure out from the token what email ID’s password needs to be reset, and then once the user resets the password from that Email ID, make the changes to the database. Now comes the main part i.e resetauth

I had to figure out what email ID the token refers to, so I figured out, the best solution would be to have another JsonTray in DAO, which maps token names with email IDs and also contains their expiry times. The reason why I did so, and didn’t do the other way round, will be explained later in this article.

So, I decided to add another enum to ClientCredential.Type, named resetpass_token which stores the keys (i.e tokens) for the JsonTray (which I named passwordreset and it accesses passwordreset.json). And I made the JsonTray non-persistent, i.e it has a definite size.

So here’s what we do once we get forgotemail (i.e the Email ID whose password is forgotten): We create a random token, add it as a ClientCredential.Type.access_token key to passwordreset.json, and then to the JSONObject we add its expiry time, and the ID etc. Basically, resetauth is an Authentication object which points to the JSONObject with the appropriate token. So, this is the code to be added in place of //code redacted as written above:

This completes most of PasswordRecoveryService, except one part, which I’ll now come to.

So now, when the email is sent, we store the token and the corresponding email ID in a non-persistent database. And now, the user clicks on the link in the email. The link has a token and it leads to the Password Reset page. We now need to figure out whose email ID’s password needs to be reset, given the token and the database, and then display that email ID to the user (so that he knows he’s resetting the right email ID). So this is the last part of the code of PasswordRecoveryService:

In the Password Reset page (resetpass), I made an AJAX GET call to the PasswordRecoveryService servlet, and sent them two parameters: getParameters = true and the token of the url. This is how the AJAX call looks like:

This code is pretty obvious: we retrieve the token, and if it exists in the passwordreset.json AND the current time is before the expiry date of the token (we get the expiry time using Instant.now.getEpochSecond(), I’m not going into the entire code of the expiry time because it’s a bit long), we retrieve the token’s email ID and display it to the user. Else, it gives an error (Expired Token, Invalid Token or No token as per the condition).

So now at the resetpass page (it’s a simple HTML page so I’ll focus on the JavaScript rather than the page design etc), the user can see his email ID on the screen, he sets a new password, and clicks on Submit. On clicking of the button, we need to reset the password in the actual database, which is Authentication (i.e DAO.authentication).

So the final part of the entire Password Recovery / Reset system comes here, the way to actually reset the password. For this, we create another servlet, the PasswordResetServlet. It accesses the DB, resets the password if it can, and displays the appropriate message to the user. This is the code of PasswordResetServlet: (API endpoint set as /api/resetpassword.json)

On clicking of the submit button, we send the newly set password (in encoded URI format) AND the token from the URL (remember, we had done the token processing earlier in the getParameters ONLY to show the user the email ID he’s resetting, we hadn’t actually done the actual resetting so we send the token back to the PasswordResetServlet). We retrieve both the new password and the token. Then straightaway, we look out for the token in the passwordreset.json, get the email ID from there, then go to Authentication.json. We remove the salt and the passwordHash from there, create a new salt, and store another passwordHash by encoding the newpass (i.e the new password) with the salt. Once this is done, we then go back to passwordreset.json and remove the token from there, since it was one-time, so the token can’t be used again. This finishes the process, and now, the user’s password is reset.

Phew, that was long!

So finally we have seen how the accounts system of loklak_depot was developed. This was just the mere beginning of what’s next to come and was a relatively trivial part (which was only complicated because it was all servlet based and we developed all the security measures ourselves). The next thing will be when loklak_depot’s functionality actually begins: Q&A apps, apps which use the Twitter data and give the user valuable information by using the APIs available. This is what I’ll be working on now, and will start talking about it in my next article. Feedback, as always, is welcome. 🙂

So here I am with another blog post. This, my fourth post in this series, will focus on SignUp verification and how I implemented it. The fifth and the final post in this series, will be on Password Resetting. This will finish the Accounts system, and I’ll then move onto talking about some Twitter analysis applications I’m working on (the Q&A apps I spoke about in the previous posts).

I had covered the Sign Up page before, but there was still a problem. What if the page were to be spammed by bots / unauthorised users? It would then create unnecessary number of accounts and DAO.authentication will then overflow. That’s why it was deemed necessary to have a SignUp verification system.

The SignUp verification system is simple: send email to user once he signs up. The email consists of a verification link which has a token which can be used to verify. Clicking on that will verify his account.

Without further ado, let’s dive into the code. The following code snippet should be self explanatory through the added comments, though I’ll explain at the bottom:

This code is from a class SignUpService.java which again extends AbstractAPIHandler (please see my previous blogposts and Robert’s posts if you aren’t clear about it), so post is obviously the POST request, and is of type Query, and this code returns a JSON (as AbstractAPIHandler implements an abstract method called serviceImpl which returns JSON). We use APIException (a predefined exception class we made) to make it more convenient to report errors with codes (earlier we used JSON). The signup and password posted from the webpage are encoded URI components, so we decode them. The rule for password and email is that they should match the regex scheming + they can’t be equal to each other and that email should be something not in the DB i.e DAO.authentication. The latter is checked by simply using the ClientCredential keys (passwd_login enum) to access DAO.auth and checking.

Here comes the verification part. We first encode the password in SHA256 using a random salt. If the user is permitted to send the email (I’ll come to that in just a bit), we set an access token for it (random int), give it an expiry time, keep it one-time, sent the token as a GET parameter to the base URL i.e localhost (in this case), and mail the url to him using loklakemailhandler.

This is how the getVerificationMailContent function looks like. It makes up the URL and fixes the email body.

where the value of verificationLinkPlaceholder is "%VERIFICATION-LINK%". We take up the token, and send it as a GET parameter along with two more params (validateEmail=true and request_session=true) to the signup servlet. The body of the email contains this URL, which we are reading from a custom file which has the placeholder. The file is as follows: (verification-mail.txt)

You just signed up for Loklak!
To finish the process, you have to verify you email address by clicking on the following link:
%VERIFICATION-LINK%
If you didn't sign up for Loklak, please ignore this email.

As is self explanatory, the link is inserted in the place of %VERIFICATION-LINK%. We read the file using IO.readFileCached and replace using replace as is seen.

Now the last part. What happens when the user clicks on the link? It still leads to the SignUp servlet, isn’t it? So what happens? The following code snippet answers that very question:

Just a reminder: all the code before this one, was also a part of serviceImpl function, so the function gets completed now.

We GET the validateEmail variable from the POST request (if the access level of the user is above anonymous i.e he / she is authorised), and if it is true, we simply convert the access_token key (which was used to store the email ID) into passwd_login, so that it is like all the other email IDs (all verified email IDs have key passwd_login, and access_token key is for those for which we need to verify). Also, as in the last article, we had four options for users.public.signup: false, admin, email and true. The default (i.e true) is that the account is activated and no email will be sent (i.e SignUp page will directly lead to activation). For admin (since admin is for testing purposes), the account won’t really be activated but he can check if the system works. For sending the verification email, we set the config property to “email”, making sendEmail (spoken about earlier) true for “email”. That’s all there is.

And just one last question you may be having: what is the regex code for (written above the validation code)? We made a small improvement to the SignUp system: instead of hardcoding password strength by ourselves, we let the admin to decide for himself, so we kept password strength and password strength description as fields in the config file. And we modified the JavaScript of SignUp page, we added a JQuery AJAX call as follows:

We simply send an AJAX call with the getParameters variable to the SignUp servlet (which again retrieves this variable and returns a JSON which specifies the regex descriptions). We use this JSON and retrieve the regex and the regex descriptions from it, and display that on the webpage (we modified the Password Strength function accordingly as well) Additionally, we also kept a status-box to display errors (if there is any error i.e Public SignUp disabled / emails disabled) which will display the error on the screen itself and block all the fields of the SignUp form.

So that’s it for the SignUp verification. My next and final article on the Accounts system will be on the Password Reset system and how we finally got the Password Recovery system to a full circle. Feedback, as always, is welcome. 🙂

So this is my third post in this five part series on loklak_depot’s early beginnings and the Accounts system. Once I am done detailing the Accounts system, we will get into the more advanced usages of loklak_depot wrt data collection and Q&A applications, so it won’t really be a ‘beginning’ anymore.

I’ll be focusing on the Emailing system (for Password Recovery) in this post and talking about how I implemented it. The next two posts will focus on using the Emailing system for SignUp verification, and Password Resetting.

As shown in the previous two posts, I had implemented the login and SignUp page. But there was a functionality not yet implemented: the “Forgot Password” aka Password Recovery, which obviously is important for the entire system to function the way it should. The idea was to lead the user to a page when he clicked on the “Forgot Password?” button on the Sign Up page. This page asks for the user’s email ID, and once this email ID is verified to be in the database, a verification email is sent to it.

Again, as had been done till now, I implemented a servlet to do all this, but along with that, I felt that probably the emailing system could be used elsewhere as well (it turns out it did get used, in the SignUp verification, about which I’ll talk in a later blog post).

So I created a generalised class which can be used to send emails, given some configuration inputs in the config file. I named it the LoklakEmailHandler class.

Let’s check out the configuration inputs. The following snippet is from the config.properties file:

# this is the message server configuration file
# to customize these settings place a file 'customized_config.properties' to
# the path data/settings/
# link to this installation.
# used for e.g. verification emails
# TODO: should be set during installation
# TODO: should replace shortlink url
host.url=http://localhost:9000
# allow public signup (false, email, admin, true)
# admin means the admin has to enable the account
# email means a validation email will be send to the user
users.public.signup=false
# Regular expression for passwords (warning: has to be a valid java string, escape characters need to be double escaped e.g. \d instead of d
users.password.regex=^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{6,64}$
users.password.regex.tooltip=Enter a combination of atleast six characters, containing lower- and uppercase letters and numbers
# SMTP emailing switch (for password recovery and signup verification etc)
smtp.mails.enabled=false
# SMTP password recovery server host (change this as per need)
smtp.host.name=smtp.gmail.com
# sender email (change this as per need)
[email protected]
# depending on security of recovery host server, set encryption as none, starttls or tls
# also make changes on server side: eg if server is gmail, turn POP3 and SMTP on
smtp.host.encryption=tls
# incase encryption = starttls or tls, set sender password and starttls/tls port
smtp.host.senderpass=randomxyz
smtp.host.port=465

For sending emails, since we did not set up a server by ourselves yet, we let the user set it up for themselves. The config fields are thus self-explanatory and are typical specified fields for SMTP servers. Additionally, for specifying password strengths in the Sign Up page (as spoken about before), we now have a config fields where we specify the regex of password and the tooltip message (of the input box) to be shown, so that the user by itself has control over the password security. And to toggle the entire email sending process (be it in SignUp verification or Password Recovery), we have another field, smtp.mails.enabled.

Now comes the fun part, and the heart of the Email Handling process: the LoklakEmailHandler class. The code looks like this:

The idea is simple. The sendEmail function takes in the Receiver’s ID (subject to regex verification by the EMAIL_PATTERN variable), Subject and Text of email, reads the config values, and depending upon the encryption level (none, starttls or tls) , adds one or more properties to the email properties. Once that is done, it uses the MimeMessage and Transport classes to send the mail. So one just has to specify three fields in total (which are obviously typical of an email) and the email gets sent. Simple, right?

So now that the main part of email sending was covered by LoklakEmailHandler, now comes the relatively small task of using this class in action in the Password Recovery servlet. Here’s how it works: (PasswordRecoveryService.java)

Simple as it is, the servlet receives the forgotemail i.e the email entered for the Password Recovery (which was POSTed from the Password Recovery page), and then tries sending the email to that email address.

As of now, since the actual Password Reset option is under development, we did not specify the reset link (which goes inside the body of the email). For the Password Reset, there were changes to made to the Authentication object too, so I will cover that in a separate blog post.

I’ll be back with the final two posts in this series, both in the coming week, where I’ll be talking about Sign Up verification and the Password Reset system. As always, feedback is duly welcome. 🙂

So here I’m back with another blog post on loklak_depot, this time on the Login page. If you have not read my previous blog post, I suggest you do so here.

For the login page, the idea is simple: there’s a field for Email ID and Password, and a checkbox to “Remember me” (in the case when users don’t want to be logged out). The Email and Password are then verified from the database via a servlet, and the user logs in. We are implementing login by two ways: by POSTing directly to servlet (i.e on clicking on the Login button), and via parameters (using things like curl). On the face of it, this is the functionality. Behind the scenes, it’s a bit more complicated.

For the posting (JavaScript) of the Login and Password to servlet part, that is trivial. It goes something like this:

Here, the $('#login') is the login button. The email and password is POSTed as a URI to prevent any errors on non-UTF8 characters. In addition, there are two more fields: request_session and request_cookie. The former by default is true while the latter is true or false depending upon whether the “Remember Me” checkbox is checked. If the servlet returns an ok status and reason, then you are redirected to the applist page (for now, although we will get onto implementation of the dashboard soon).

Now for the backend code. It has two parts: a servlet and an added logic in AbstractAPIHandler (again, if you don’t know what the latter is, go read my previous blogpost). Here it is:

The servlet extends AbstractAPIHandler. Type is an enum which contains a variable email.This code calls the getIdentity() method from AbstractAPIHandler, which contains the main backend code as follows:

Long, right? The logic behind it though is pretty simple. As shown in this code, if request_session is true, it verifies the email and password from authentication.json. If request_cookie is set to true, the email ID and cookie is written to ClientCredential along with an expiry time, and once that times goes up, the user is logged out. And as in the signup, the passwords all have a hash and a salt. If the password encoded in the salt equals the hash, the login is verified. And the login system thus works fine.

The AbstractAPIHandler is a very useful class for most loklak operations, because it encapsulates most of the important backend operations under one class, and here it has come in handy again.

My next post will be on the Password Recovery system for the accounts, and the LoklakEmailHandler class. Feedback and suggestions for improvement are welcome. 🙂

The next few posts from my side will be on loklak_depot (name tentative), a new product that we’re trying to build, and the services it may provide, and our steps in that direction.

loklak_depot aims to be a marketplace for all things loklak. Call it DiffBot, but only that it’s much more. Our aim is to offer the crawling services of loklak, the dataset dumps and the APIs to authenticated users in the community. We could use the datasets for various purposes. For a university loklak peer for example, we could have a large dataset with special access for students. For shops, we could have datasets tailored for customer use. We also have the apps system which we could use to offer various other services tailored for the user, or commercially sell them like the crawlers at DiffBot. All of these features, at one place: loklak_depot.

For the initial approach to loklak_depot, it was decided to implement an accounts system, such that authenticated users can access features that they cannot on a normal peer. This grants them special rights using a process called provisioning. The provisioning system is implemented through a concept called AAA: Authentication, Authorization and Accounting.

Authentication asks the question, “Who or what are you?”Authorization asks, “What are you allowed to do?”Accounting wants to know, “What did you do?”

AAA is the basis of the permissions and the accounts system, and will be used in further blog posts as well, so make a note of it.

In the accounts system, I have implemented both sign-up and login page as apps, and coded the backend logic in Java. I will cover the sign-up page implementation in this post.

The sign-up page validates the user’s email ID and password and signs him up to loklak. Validation for email was simple regex. For password, there was a counter which specified the strength of the password, also passwords below 6 letters were not accepted and they had to be a mixture of alphanums, letters and numbers. A small code snippet of the strength level is shown below: (written in JQuery)

It must be noted that in the SignUpServlet code, the class extended AbstractAPIHandler.java (a custom class we implemented to cover User Identities, the AAA concept code, as well as the code for receiving the POST requests), and the API Path was set to "/api/signup.json".

Once the information is POSTed to the servlet, it returns a JSON containing a status and a reason depending on the situation. In normal conditions, both have the value ok.

For security, passwords were hashed in SHA256 using salts, and then the hashed passwords are stored in the file. Also, the email and password sent are encoded (from the client side) in URI using encodeURIComponent and sent to the servlet, and decoded back using URLDecoder.decode(post.get("signup",null),"UTF-8"); . The purpose of this was because SHA can throw an exception for characters not in UTF-8, such as those used in foreign languages.

Once this was done, the servlet was added on the MainActivity i.e LoklakServer.java:

servletHandler.addServlet(SignUpServlet.class, "/api/signup.json").

and the page worked like a charm.

Through this, I became really familiar with using servlets first-hand, and coding in JQuery, things I never coded before. I feel really comfortable with using both of them now.

What next in this? I’m considering rewriting the client-side code in Angular so that we can have a Sign Up option on the top menu on the homepage of every loklak peer, instead of as an app. I shall be doing this at a later stage.

In my next two posts, I’ll cover the implementation of the login page, as well as email services for Password Recovery and User Verification. Feedback and suggestions welcome. 🙂