We are looking for a Big Data Engineer that will work on the collection, storing, processing, and analysis of huge sets of data from heterogeneous domains.

The primary focus will be on researching for optimal solutions to appropriate for the aforementioned purposes, then maintaining, implementing, and monitoring them. The successful candidate will also be responsible for integrating the said solutions with the architecture used across the company.

Responsibilities:

  • Researching, designing and developing appropriate algorithms for Big Data collection, processing and analysis
  • Selecting and integrating any Big Data tools and frameworks required to enable new and existing product capabilities
  • Collaborate closely with product team to define the requirements and set milestones that relate to Big Data features
  • Detect anomalies and perform audit on raw and processed data
  • Monitoring performance and advising any necessary infrastructure changes
  • Defining data retention policies
  • Present data findings to internal and external stakeholders

Skills and Qualifications:

  • Proficient understanding of distributed computing principles
  • Management of Elasic Search cluster, with all included services
  • Proficiency with Hadoop v2, MapReduce, HDFS
  • Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
  • Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
  • Experience with Spark
  • Experience with integration of data from multiple heterogeneous data sources
  • Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
  • Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
  • Good understanding of Lambda Architecture, along with its advantages and drawbacks
  • Experience with Cloudera/MapR/Hortonworks

Other:

  • Experience with DevOps, TDD and CI practices
  • Knowledge in networking principles
  • Minimum 3 years’ experience
  • English language is a must

Applications to be sent to careers@trgint.com