Mohammed Siddiq

Mohammed Siddiq

Software/Data Engineer

About Me

I am a computer science graduate student from UIC, with interests in backend engineering and large scale data pipelines. I enjoy designing systems and engineering them. The moments where you see your thoughts and your engineered code creating value in real world, drives me to push through challenges and technical obstacles.

When laptop power is done, I would be journaling or reading about the lives of the greats who are impacting the world. I believe there is purpose for life, which I am constantly refining to make my existence on this earth impactful. Lately, I have started playing chess. I play blitz like its an un-timed chess game and always end up making dumb moves in the last 30 seconds.

Work Experience

Graduate Teaching Assistant - Engineering distributed objects in Cloud computing
- UIC (Fall 2019)

Teaching Assistant for Graduate course CS 441 under prof. Mark Grechanik

Graduate Teaching Assistant - Object oriented languages and environments - UIC (Spring 2020)

Teaching Assistant for Graduate course CS 474 under prof. Mark Grechanik

Software Engineer Intern- Molex (May 2019 - August 2019)

Was hired as an intern for the backend engineering team, which was handling streams of high volume data from various sensors on manufacturing and gas plants. This data was collected, processed and exposed to data science teams which was doing real time predictive analysis on it. All this real time data and analysis was exposed to the different users via dashboards.

My contribution to the above project:
Different sensor's that were sending the data followed different protocols/configs to communicate with the backend service. This became hassle, as maintainance or updation of the configs had a cascading effect on the data and its analysis. Any addition of new sensors config was manually updated in the backend's .conf file and subsequent ETL jobs were also updated post that. I worked on the project to streamline all the sensor configs. Created a micro-service that was a single point of access to all the configs. Exposed these configs through RestAPIs, that were housed and maintained on AWS s3. All the backend's .conf file were just pointed to this s3's configuration file which was maintained and updated in real-time. This reduced atleast 20 hours of manual effort every week and drastically eliminated the chances of backend failure due to data mishandling.

Proposed and Implemented Version Controlling to the above service - Designed and implemented a service to audit all the data operations done within the micro-service and exposed roll-back features. Preventing data corruption and reducing system downtime.

Software Development Engineer 1 - Ola cabs, Bangalore, India (July 2017 - August 2018)

Ola cabs is the India's larges ride sharing platform. Uber is competing with Ola for dominating the India's ride sharing market.

I have closely worked with pricing and location platform at Ola.

Single handedly drove the feature “Hotspot and pickup point” into production. A critical feature which enabled creation and modification of geo-shapes in real time, which drastically enhanced the user experience to select pick up points. This feature resulted in 0.5x reduction in ride cancellations.

Introduced rate limiter library in a high throughput location service. Used Redis cache to track the request rates from different clients based on their priorities. Improved the response time of the service by 0.3x during peak hours.

Software Engineer Intern - Delta x, Bangalore, India (January 2017 - May 2017)

DeltaX is an advertisement platform that aggregates different ad-agencies and provides consolidated access to users ad campaigns

Projects

REST Based Chess Engine

Exposed a chess game via REST APIs. Also includes auto-play feature - where an instance (multiple instances) play chess games with other instances through rest api's without any human interaction

Project Demo project name

This project was developed as part of my master's course CS441- Engineering distributed objects in cloud computing. In this project,an existing chess engine was taken, its code was studied and API's were analysed. Modified this engine to play chess over Http requests. The state of the game was maintained through sessionKeys and the state was persisted on MongoDB. The whole project was bundled into a Docker image (Link to docker hub) and deployed on amazon EC2 instances. The interesting and the challenging part was to add and maintain multiple game states - Essentially, a single instance of the engine could play with multiple other opponents at the same time.

Star my project on github

Cloud DataCenter Simulation

In this project I aim to analyze and build various cloud architectures with multiple datacenters (according to availability region). Considering the network they use to connect, with each datacenter having multiple hosts. Each host having its own build and allocation policies of VMs.

project name

Map-Reduce Job simulation Workflow: Job in split into into three different tasks: Mapper, Reducer and ResourceManager.
Mappers: Defines an individual mapper which will work on a particular file split and executes the map task [CloudletExecutionTask]. After done with execution it will emit/write the map result simulated using the [CloudletSendTask].
Reducers: Defines an individual reducer task which will take the output written by the mapper from the intermediate storage, processes it and writes/persists to the output node.
Resource Manager: Models a resource manager, which takes in the Job requests allocated/schedules the jobs among the different available nodes as specified by the job conf.
Worker Nodes and VMs: Each of the above cloudlets are executed on these worker nodes which are nothing but the VMs allocated to the job by the resource manager.

Star my project on github

Extracting Author's publications and co-authorship using Hadoop

Hadoop map reduce implementation for describing author's publication and their relationship with other authors (Co-authorship) in a huge dataset.

Project Demo project name

Created a map/reduce program for parallel processing of the publicly available DBLP dataset that contains entries for various publications at many different (e.g., conferences and journals).
Each entry in the dataset describes a publication, which contains the list of authors, the title, and the publication venue and a few other attributes.

Read More & Star on github

Page Rank In Spark

Page rank implementation in SPARK to rank authors and venues based on their publications in the DBLP dataset.

Project Demo project name

DBLP dataset (represented in XML format) describes the publications, authors and the venues of the publication. This file is hosted on HDFS and logical splits of each publication is done using the customized implementation of XMLInputFormat supporting multiple start and end tags in an XML.

RDD is constructed using this file on HDFS. The Following operations are performed on each logical input/entry of the dataset Authors and venues are extracted. Node links are generated between the co-authors and the venues (bi-directional graph is considered). Every link in the graph is given am initial probability (Initial page rank). These links are cached since page rank runs on the basis of these links and every worker/task needs access to it. Page rank is applied iteratively over the nodes/links in the graph. The damping/teleportation factor is considered to be 0.15. Refer to this link for more details on page rank in the spark

After the algorithm stabilizes the page rank values, the ranked nodes are sorted based on the page rank values. The authors and venues are filtered out accordingly based on their ranks.

Read More and Star project on github

My GitHub