An SUTDent’s fulfilling summer internship Indeed!

What is Indeed?

Indeed is the #1 job site in the world with over 250 million unique visitors each month. Started in the US, they are now in more than 60 countries. Indeed helps job seekers all over the world by giving them free access to jobs from company websites and job boards. Their mission is simple yet inspiring – We Help People Get Jobs.

How did you learn about Indeed?

It was during our SUTD Learning Celebration Carnival in 2018. Jordan, Site Director for Singapore Engineering Centre, and Max, my mentor, who was also a Software Engineering Manager, were there. Max introduced me to the mission of Indeed, as well as the engineering challenges they were engaged with in Singapore. The impassioned way he spoke about what an Indeed Software Engineer does left a deep impression on me.

Prior to that, I have not heard of Indeed as it was not big in Singapore then. When seeking internships, I had frequented the use of social media and our school’s portal more. Hence, it was with curiosity and the joy of novelty that I learnt more about Indeed from Max.

One year on, LCC 2019!

Tell us about your work and projects you handled at Indeed

There are two main projects I worked on and both went into Production. When your code goes into production, it means your work is published. For a public product like Indeed, it means that all Indeed users will be able to experience the change. Deploying code and features into production is often a bittersweet experience – it can be nerve-racking for the high stakes, but also exhilarating and fulfilling!

On my first two weeks, I was onboarded with a Gong project, we will come to why it is called that later. My Gong project was to run an A/B test (also known as a split test) on the main Indeed website. Simply, an A/B test is an experiment. For websites like Indeed where user traffic is high, we can conduct statistically significant experiments by showing different users different versions of the website, and measure an outcome for each experiment. As with every experiment, we start with a hypothesis, control and test groups and we measure the results of each group. One of the most legendary A/B tests was conducted by Google, where they reportedly tested 40 shades of blue on their website to ascertain the best shade of blue to use!

For my experiment, I was targeting European markets that Indeed is trying to localise for, specifically in Germany and the Netherlands. The hypothesis was that jobseekers in these markets had less familiarity and trust towards Indeed. Hence, by telling a job seeker that the job poster has previously hired through Indeed, we can bolster trust and reduce the friction towards applying. Implementing an A/B test from scratch can be extremely challenging. Fortunately, there was already a comprehensive software infrastructure built around testing and measuring, reducing the complexity of the task and making this an apt get-your-feet-wet project.

This was the A/B test I conducted, where some users will see this line and some won’t

After six days, with a lot of help, I finally managed to deploy the experiment to production. Following that, I sounded the gigantic Gong in the middle of the office which is meant to celebrate an Indeed engineer’s first deployment (meant to be done in their first week!) or any other significant milestone.

This is me beaming right before sounding the Gong

A/B testing is a data-driven way of justifying changes to something that already works. In our case, we saw an 8% increase in applications with just a simple line introduced – a significant win!

My second project involved the job posting creation process on employers.indeed.com. When employers post jobs, the dropdown suggestions usually include legally questionable and low quality job terms that are usually irrelevant. The most problematic suggestions are those that connote gender, which can be potentially discriminating. Examples include “Waitress” or “Hostess”. Instead, gender-neutral terms such as “Server” are preferred.

The autocomplete suggestions that I worked on

Hence, my task was to develop a data engineering pipeline that removes or normalizes low quality and inappropriate job terms. While this abstract representation of the task seems simple, it required making changes to three separate code bases and communication with two different teams in Japan and U.S. For this project, I worked closely with an engineer in Japan to productionize the changes. Throughout the process, I learnt to appreciate the different technologies developed and used within Indeed, and the many engineering decisions they have made to enhance reliability, fault tolerance and speed.

After about 8 weeks of development, the feature was fully developed, and subsequently deployed to production.

What I had learnt from the second project was communication and coordination in projects. The project required much more communication and coordination as I was making changes to code bases owned by remote teams. I also realised that reading code is significantly more difficult than writing it. While we only write code once, many others working on our code base have to read it often. I suppose therein lies the importance of writing good code, even if the machine executes it almost all the same way. More importantly, I learnt that being an engineer entails far more than technical skill. It is also about communicating, managing expectations and as my mentor would reiterate – execution. Of course, execution in general encompasses more than executing your program.

What was the culture like at Indeed?

It is hard to describe culture, and it is one of those aspects where you have to be there to feel it. I will try to give concrete examples though, so hopefully you’d be able to experience it as much as I did!

Indeed has a very carefree and trusting environment. As a company that connects job seekers to jobs, Indeed is unsurprisingly hyper-conscious about the job experience and culture they create.

One of the highlights is the policy of Open Paid Time Off (PTO). Of course, there are some fundamental caveats to this, but for the most part, employees are given flexibility and the trust to take leave or work from home whenever they want to.

People at Indeed are smart, cool and most importantly, nice! It is easy to strike up conversations with colleagues from other teams. Have a question that nobody in your team knows the answer to? Feel free to walk over to any colleague who can help.

Within teams, we have daily standups and fortnightly scrums where we share wins and set goals. For engineering work, we have coding styles to follow, as well as a code review and qualitative assurance process before any code is accepted. We work on design documents to conceptualise software architecture before diving into coding. We discuss system design decisions on whiteboards way before we start drumming at our keyboards. In terms of professional growth, we go through an exercise called LevelUp where on a quarterly basis, we receive feedback and review from our peers and superiors, and then from our direct manager for the work we have done.

Hierarchy wise, it feels surprisingly flat. For one of my projects, Andrew, a Senior Product Director, spent a good amount of time to explain the motivation to me. Even Jordan, our Site Director often brought the interns and fresh graduate hires to dinner and weekend outings!

Intern farewell lunch

As an office, there are many things done to ensure growth, welfare and happiness. Indeed has a great office space, endearing support personnel, amazing lunch provided every day, free breakfast once a week, happy hour every Friday, senior leadership dialogues every few months, weekly one-on-one check-in sessions with your mentor, festive celebrations and not to mention, great ground-up initiatives like Tuesday Runs and Futsal, Whiskey Wednesday, Thursday Tennis and the occasional board games on Fridays. Most importantly, we share and celebrate one another’s wins every time something significant is achieved – with the Gong!

What were the key skills you picked up during your internship?

I think I have become a better engineer after the internship. For one, I have become more nuanced – I learnt that often, there’s more than one way to do something and we don’t always have to pursue the best way. Max shared with me something about being smart and getting things done – and I think that is one of my biggest takeaways. Being smart and getting things done is essentially about resource optimisation. When should we double down on something and when should we pull back in the face of diminishing returns? What can we compromise on and what are the guarantees that we must fiercely clinch onto? In trying to answer this question, one would realise that engineering does not happen in a vacuum. Your engineering decisions must consider the context richly – such as its business, product and legal implications.

Being an engineer is more than making engineering or design decisions. The more aware you become, the more you will realise that there are meta-decisions that surround your work every day. With that awareness, and the conscious effort to evaluate those decisions, I believe we can all become more effective at getting things done.

The second significant takeaway is communication. It is often tempting to assume that all other parties are somehow magically kept in the loop, or has access to the same information as you do. When in doubt, it never hurts to provide more background information – doing so can potentially speed up communication. Even something as simple as code review that has a turnaround time of half a day can be sped up dramatically if we spent just 15 minutes on a call or face-to-face with the reviewer, to explain the logic in the code and the rationale behind certain lines of code. At the end of the day, writing software as a team is a relatively new activity that people are still desperately trying to optimize. Much has been said, agreed and disagreed upon about communication within teams. From my experience, it never hurts to be thorough in your communication.

There is a laundry list of other conventional useful things that I have picked up on, that at the risk of being cliché, I shall be very brief about. Considering the lifespan of your code and balancing writing code for the present and for the future can enhance your efficiency and ensure you do not over or under-engineer. When working in a team, it also helps to be aware of optimism bias, and avoid over-promising or being too ambitious simply to achieve a transient, short-term reward. These are just two in said laundry list!

How has SUTD helped you during your stint?

SUTD has a lot of project work that has helped me develop my communication skills. As a result, I was able to build on this soft skill to apply myself effectively in the workplace.

Next, it is definitely the technical grounding. My experience in Indeed was technically challenging, and I felt the need to tap on all the different things I have learnt in my pillar years, and then some. More specifically, Information Systems and Programming for the foundation in object-oriented programming (OOP); Introduction to Algorithms which crucially helped me think about ways to optimize my code in terms of time and space complexity; Elements of Software Construction for Concurrency considerations as well as writing better code and lastly, Computer System Engineering for the knowledge about OS and networks that wasn’t directly applicable, but useful background knowledge to understand more complex systems nonetheless.

Me sharing about my research work in SUTD

What was a memorable experience from your internship at Indeed?

The Summer Hackathon takes the cake for this!

Every year at Indeed, we organise two runs of an internal hackathon – once in Summer and once in Winter. The idea is to bring everyone together, away from the usual work, and innovate for our current products and everything else we would like to. This extends beyond the Product and Engineering teams, and any employee is free to join!

As a hackathon enthusiast and having participated in a few, I volunteered to organise this internal event! I was joined by a motley crew of colleagues comprising HR employees, Office Managers and fellow Engineers. We were also supported by a global community of other Indeedians who were organising their respective site events.

The team hard at work designing an ideas board for the Hackathon

I’ll share three highlights to this experience.

First, there was almost no reporting required to the higher-ups! The team was entrusted to make our own decisions and we worked efficiently with everyone’s help. It was almost like clockwork – everyone chipped in ideas and their time to make things happen. Whatever we wanted, we just had to research and spend the time to achieve it. No bureaucracy, the budget’s the limit!

Finished!

Second, organising has its privileges – we get to choose our favourite food to cater for the event! This included a live ice cream, coffee and grill station throughout the two-day event! We also had bagels for breakfast, Garrett Popcorn, as well as beer and pizza for the evening!

Third, the hacking and the hacks! It was a great experience hacking as I could choose the team of my preference. I joined a team that wanted to create an in-house solution to potentially save USD 1.5 million annually for Indeed. Trying to prove that we can create a competitive solution in 30 hours certainly stretches the definition of hacking to its limits. The exposure to experienced engineers and product directors from other teams was also extremely inspiring. At the end of two days, there were more than fifteen solutions presented – in which most, if not all, were impressive and ingenious!

My final presentation on my last day at Indeed

Lionell Loh is a Junior from Information Systems Technology and Design (ISTD) and a Hwa Chong Institution alumnus.