Algorithmic pricing with Redis
The algorithmic pricing is an exciting new area, and it combines engineering and mathematics. Chen’s paper has introduced the algorithmic pricing on Amazon Marketplace. This post is to discuss the implementation of an algorithmic pricing based on Redis from the perspective of the sellers.The backgroundEach of Amazon’s ASINs will have many sellers that compete each other.Amazon has a ranking mechanism for Buy Box, say, to punish a new seller. But for the same ASIN, the seller who has the lowest price usually wins the Buy Box. For each ASIN, each of many sellers will have an optimal price (the price ...

Good math, bad engineering
As a formal statistician and a current engineer, I feel that a successful engineering project may require both the mathematician’s ability to find the abstraction and the engineer’s ability to find the implementation.For a typical engineering problem, the steps are usually - - 1. Abstract the problem with a formula or some pseudocodes - 2. Solve the problem with the formula - 3. Iterate the initial solution until it achieves the optimal time complexity and space complexityI feel that a mathematician would like dynamic programming or DP questions most, because they are too similar to the typical deduction question in ...

Use Elasticsearch and Kibana for large BI system
Nowadays Elasticsearch is more and more popular. Besides it original search functionalities, I found Elasticsearch can be used as a logging container. That is what the ELK stack is created for.utilized as a JSON server with richful APIs, which can be combined with its Kibana as BI servers. That is the data store I see everyday 10PB stocking dataaverage 30TB incoming data everydayvarious data sources including binary files such PDFincluding very complicated SQL queries (fortunately no stored procedures)millions of JSON creations daily People want to know what is going on with such data. So a business intelligence or an OLAP ...

Use Slack bot to monitor the server
I used to install Datadog or other SaaS to monitor my Linux boxes on the cloud. Most times they are just overkill for my tiny servers with only 1GB or 2GB memory. Actually what I am most interested is the up-and-running processes, or/and the exact memory usage. And I need a mobile solution to monitor on-the-go. Now with the coming of Slack bot, and its real time Python client,&nbsp;I can just use a simple Python script to realize the purposes. from slackclient import SlackClientfrom subprocess import getoutputimport loggingimport timemessage_channel = '#my-server-001'api_key = 'xoxb-slack-token'client = SlackClient(api_key)if client.rtm_connect(): while True: last_read = ...

Deploy edx spark environment to DigitalOcean
This summer I took the Spark courses at edx CS100 and CS190, and had wonderful experience. The two classes apply a Vagrant virtual machine containing Spark and all teaching materials. There are two challenges with the virtual machine —The labs usually take long time to finish, say 8-10 hours. If the host machine is closed, the RDDs will be lost and the pipeline has to be run again.Some RDD operations take a lot computation/communication powers, such as groupByKey and distinct. Many of my 50k classmates complained about the waiting time. And my most used laptop is a Chromebook and doesn’t ...