I was looking at NMD website, for some reference and on the slideshow
I noticed "Lorem Ipsum". Now for a design school website that itself
is very damning. Further, on clicking on one of those sliders, a new
tab was opening for me, taking me to a random site. This was worrying,
as the site belonged to a government institute and is serving or doing
something malicious. I reached out to Nandeep, to see if he was also
seeing similar behaviour, he was not. I tried different browsers, with
Ad blocks and the anomaly was gone. Nandeep joked that maybe I got
pawned, which got me a bit worried. On one side I was feeling, nah,
some random behavior, ghost in the shell, and on other side I was
cursing myself for not being able to understand what just happened on
my own system.

I tried to reproduce the behaviour again, disabled all adblockers,
and opened up inspector and looked at the network requests. At the
bottom of the requests, I noticed a few weird http requests being made
to IP 172.205.13.171 on port 3000. Here is a screenshot of
request/network over http and https:

Some context here, Indian broadband service providers have often been
caught with their hands stuck in cookie jars(Airtel, BSNL). And their
official response has been a shrug and something on the lines of "This
is a standard solution deployed by telcos globally, meant to improve
customer experience and empower them to manage their usage". It is one
thing reading such incidents happening to others and a totally
different thing noticing it first hand. Though the MITM was working as
it was explained in classes of cryptography-101, but experiencing it
in person, on live system was giving me a head rush.

I looked at params the request was sending and it included fields like
subscriberId, subscriberIP:

At this point I reached out to punch bhai for comments and feedback. I
shared screenshots of network requests and after comparing them to his
network requests, he helped me zero in to the origin of these dubious
requests. After scrolling through the requests, we finally came to
request made for JQuery-1.6.1.min.js and the ISP was shorting the
request and passing me another Javascript file which internally did
the shady requests to it's own IP and also loaded original JS
file.

This was nuts. We tried couple of other sites, like learn.jquery.com
and there too, BSNL was injecting script.

I enabled https everywhere browser extension and with that in place
the whole interaction with the site was clean. Given that in spite of
previously reported incidents, ISPs don't care and keep behaving the
way they like, we as consumers don't have much choice left. Unless,
our parliament passes a privacy law as directed by Supreme Court,
something on the lines of GDPR, and usable and powerful like
RTI. There are already draft bills available, SaveOurPrivacy is
backed by Internet freedom foundation, which had worked really hard
to push TRAI for NetNeutrality in India. There is nice video compiled
by people behind this
draft:https://www.youtube.com/watch?v=GkSVMA8hkSY. Let us keep pushing
for such a law which empowers uss, the customers, to demand the ISPs
to furnish data they are collecting, with whom they are sharing it and
how much they are sharing and if they are invading our privacy, to
follow a legal course of action.

I started working on SocialMedia Feed project from last August. And
time after time I had found myself in a slump, looking for motivation,
head rush to finish a task. Try to watch movie, find some song-of-day,
something, anything, and before I realize, its the next day
already. They say, “Show up, show up, show up, and after a while the
muse shows up, too” and personally while trying it, self motivation
drags, in really bad way.

Last week I was trying to put together a POC for a possible paid
work. Task was to create a chat bot around different columns of Excel
sheet for users to be able to chat to and get analytical results in
conversational manner. So instead of a technical person running a
query raised by sales team, user can directly chat with a bot and get
answer to a question like "How many tasks finished successfully
today", tasks could be something like email campaign or nightly
aggregation of data or results from a long running algorithm.
Airtifical Intelligence Markup Language - AIML and its concepts are
fairly popular to put together such a chat bot. Many of popular
platforms like pandora bots, api.ai, wit.ai help create such interface
to "train" a bot. Like in case of the above message - How many tasks
finished successfully todaytasks, successful, today gives us
the context on which column to run the query, what value to look for
and duration over which we want to get the count.

I didn't have prior experience of using api.ai platform so I had
planned to read up some docs and train the system to identify context,
maybe a couple of them - ~2hrs of effort. I ended up starting to work
on it only past 12 pm, browsing through random doc links, exploring
sdk for examples, trying to find my way through setting up the
pipeline. By evening 6, I had clocked around 3 hours and 30 minutes on
this task, I got context in place, api.ai provided webhook to call
third party API call so I used their github demo code and got result
from the excel sheet which was shared, though very minimal work but
still, I was able to sort out first level of unknowns.

It was frustrating, tiring, but I felt this pressure to finish
it. Though the POC worked, I didn't get the work but what is really
sad is the way I was able to stick to deadline when pressure of
proving myself, convincing someone else was there. For SocialMedia
Feed project, I have this task, on top of basic gensim topic words,
implement algorithms from Termite and LDAvis paper to identify better
topic representation and get this demo in place. I have reference code
available(paper work is available on github), I know what I have to
do, but still through last 3 days I haven't clocked single minute on
this task. It will happen eventually, but I think idea is to get fired
up personally, take on things, spend time on them and then mark them
done, be professional, not just for others.

With SoFee major work is done in background using celery, polling
twitter for latest status, extract the links, fetch their content and
eventually the segregation of content would also be done this way. I
was looking for a way to keep things updated on user side and concepts
of Progressive web app were really appealing.

What does it do?

Browsers(google chrome, firefox et all) are becoming more capable as
new web standards are rolling out, like having offline cache, push
notifications, accessing hardware(physical web). With these features
now HTML based websites can also work as an native app working on your
phone(android, iPhone) or desktop.

Stumbling Block #1: Scope and caching

I am using Django and with it all static content(css, JS, fonts) gets
served from /static. And for service workers, if we do that, its scope
gets limited to /static, that is, it would be able to handle requests
getting served under /static. This limits access to API calls I am
making. I Looked around and indeed there was a stack-overflow
discussion around the same issue. Its a hacky solution and I added on
to it by passing on some get PARAMS which I can use in template
rendering for caching user specific URLs.

Beyond this I had a few head scratchers while getting cache to work. I
struggled quite a bit to short the fetch request and return cached
response but it just won't work. I Kept on tweaking the code,
experimenting things until I used Jake's trained-to-thrill demo as
base to setup things from scratch and then build on top.

Stumbling Block #2: Push Notifications

Service worker provides access to background Push notification. In
earlier releases, browsers would register for this service and return
a unique Endpoint for subscription, a unique capability URL which is
used by server to push notification to. While this endpoint provided
by Firefox works out of box, for chromium and google chrome browser,
it still returned an obsolete GCM based URL. Now google has started
using Firebase SDK and GCM is no longer supported. Beyond this on
service side PyFCM library worked just fine to push notifications and
it works with firefox too.