PrefaceSometimes it’s hard to do web scraping because: 1. You need to handle login page 2. You need to click on some button to get what you want. 3. The data in some table is dynamic loaded, you won’t get it if you ignore it’s JavaScript file. Fin...…

PrefaceIt is very convenient for me to use slack to supervise the status of running program, so I developed a simple slack-r api. Once some errors are catched, I’ll get a slack message send by slack bot, it is good for debuging and problem solving...…

PrefaceRecently the in-house server in our company crashed several times because of the heavy duty, so we finally decide to move important cronjobs to AWS. Running java on linux is easy, but for R it’s a little different.Run R script on Linux with...…

PrefaceBuild up your own web app with Python + AWSAWS ConfigurationRecommended software: WinSCP: Use this to move files between local and server. PuTTY: Use .pem file to generate .pkk file, which is easier to login with. Git Bash: Run s...…

PrefaceRecently, my company need a daily report to show the position we keep and the margin call we need to prepare, so our manage team can use the money more efficient. We used to do it in java, but the old java-version daily report is not straig...…