Gather and collect the data, store it, do batch processing or real-time processing on it, and serve it via an API to a data scientist/engineer who can easily query it.

You typically work on solutions that scale to multiple systems.

A good big data engineer has extensive knowledge on databases and best engineering practices. These include handling and logging errors, monitoring the system, building human-fault-tolerant pipelines, understanding what is necessary to scale up, addressing continuous integration, knowledge of database administration, maintaining data cleaning, and ensuring a deterministic pipeline.

Bedrijfsprofiel

BigData Republic is a consultancy company in the area of big data engineering, data science and data architectures. We provide consultancy, training & support and help customers in their transition from previous generation information architectures to the new generation of Big Data technologies and analytics.

We don’t see our area of expertise as a playground, where we do only R&D at our customers expense, but we rather go for tangible results. Our consultants have a scientific background but are pragmatic in the execution and understand it all comes down to business value. Our Consultants are industry experts and thought-leaders who have worked extensively with Hadoop, NoSQL databases like Cassandra, CouchDB, search technologies like Elasticsearch, Solr and related technologies like R, Python etc.