Year: 2010

Parallel Database for OLTP and OLAP Just a survey article on materials on parallel database products and technologies for OLTP/OLAP applications. It mainly covers major commercial/academic efforts on developing parallel dbms to solve the ever growing large amount of relational data processing problem. Part I – Parallel DBMSs 1.1 Parallel Database for OLAP (Shared-Nothing/MPP)…

I really want this blog to be a place to express my own ideas and thoughts, but I don’t refuse reference other people’s great ideas, especially when they are really helpful for me or potential readers. The following content is copied from a MSDN blog post named- Product Development Disciplines at Microsoft, I just highlighted…

Jeff Dean gave a keynote Building Large Scale Information Retrieval Systems at WSDM 2009. It’s actually a presentation on how Google search engine evolves during the past 10 years. Here are my notes for this lecture. Part I – Overview of Search Engine Evolution: 1999 VS 2009 Factors to Consider when Designing a Information Retrieval…

Part I – .Net Networking Briefing All .Net networking facilities are build on top of Winsock2 subsystem, so concepts such as non-blocking I/O, async I/O and I/O multiplexing can all be applied to the managed world. .Net network framework just wrapped Winsock2 and added some extra abstraction. Most of the wrapping stuff is done through…

Behind Avatar’s Fancy Special Visual Effect Since 12/18/2009, Avatar has attracted all people’s eyes on this planet. I watched a 3D IMAX version of this film some time ago. After seeing those fancy special visual effect, I can’t help wondering how they accomplish that and what computer hardware/software they had used.The special effect of Avatar…

Database Technologies for Decision Support System Database technologies can be applied into two types of scenarios:- Transaction Processing(OLTP)- Analytic Processing, using statistical method(OLAP) or machine/computational learning method(Data Mining)OLTP, which is based on E.F. Codd’s relation model, is the traditional (maybe most popular) application type of DBMS and most people are very familiar with it. This…

Parallel Programming – Using PVM PVM is an inactive direction in HPC community, but there are many lessons can be learned from its programming model, its architecture design/implementation and how/why it failed to be the dominate system. Part I – What’s PVM?PVM (Parallel Virtual Machine) is a software framework for heterogeneous parallel computing in networked…

Parallel Programming – Using MPI MPI is a message passing programming model standards[2], it defines various Terms/Concepts, Data Structures and Function Signatures that are used to passing messages among computer processes.1. Terms and ConceptsA MPI application consists of multiple Processes(Tasks), each has a unique identifier called Rank. A Process belongs to some Groups and some…

Parallel Programming – Using OpenMP OpenMP is a parallel programming model for shared memory parallel computers.It’s based on Fork-Join parallel execution pattern and is suitable for Data Parallel and Task Parallel applications.Fork-Join Pattern- OpenMP programs begin as a single thread – the Master thread, which executes sequentially until the first parallel region construct is encountered.-…

Parallel Programming – Using POSIX Threads Pthreads (a.k.a POSIX Threads), is another parallel programming model over Shared Memory Computers, which is categorized to Threads Based Model (the other is message passing based model).Pthreads provides Threads by means of pure C style APIs, while OpenMP does it through language compiler directives.As the Process/Thread concepts are very…