Testing with the Lights On: An Interview with Alon Eizenman

In this interview, Alon Eizenman, the CTO and cofounder at SeaLights Technologies, discusses his many experiences with startup companies, how software teams are adapting to the current demand for speed, and why you need data before you take testing actions.

Jennifer Bonine: All right, we are back with another exciting interview. Alon, nice to have you with us for the first time ...

Alon Eizenman: Thank you.

Jennifer Bonine: ... here interviewing. A little bit of background on you, maybe for the folks that don't know you yet, but they may know of some of the tools that you've worked on. Maybe give them a little background on yourself. You founded recently, most recently I should say, a company called SeaLights. Maybe tell us a little bit about your background.

Alon Eizenman: Yeah, so I founded SeaLights two years ago. I founded also Nolio. That was acquired by CA. I was part of Conduct. That was acquired by Mercury Interactive. And I founded Coridan. That was acquired by BA.

Jennifer Bonine: Wow.

Alon Eizenman: Yup.

Jennifer Bonine: So you are a serial founder of startup companies.

Alon Eizenman: It happens.

Jennifer Bonine: Yeah. It happens. It's happened quite a few times.

Alon Eizenman: Yeah.

Jennifer Bonine: That's great.

Alon Eizenman: Some I always get attracted back to it. I don't know.

Jennifer Bonine: Yeah. Did you start when you were twelve? I mean, that's a lot of startups, so—

Alon Eizenman: Do I look that young?

Jennifer Bonine: Yeah. I was like, "Did you start when you were in grade school?" You're just really quick at founding and starting.

Alon Eizenman: No, actually, I started after army service back in Israel, and now I—

Jennifer Bonine: Wow, you even had time to go to the army first, too, in Israel?

Alon Eizenman: That's what we do in Israel.

Jennifer Bonine: Yeah. I mean, that's amazing.

Alon Eizenman: Yeah, for three years.

Jennifer Bonine: Three years. Wow.

Alon Eizenman: Three years, and then I joined a very large US company, and after two years, I ... actually, it was so early when I decided to go to the first startup. Everyone told me, "How do you leave such a big company to start ... to go for ... " I joined the first startup company, it was three founders. They were just talking about the idea and we didn't even have seed money, and I left big, big company with very nice position and everything, but then the ... Everyone told me, "What are you doing?" It was so weird to go for ... that was back in '96.

Jennifer Bonine: Wow.

Alon Eizenman: But three years after, we ran so fast and then we got acquired by Mercury Interactive, and then I spent five years. It was back ... although it was larger company, it was still behaving like a startup, so I felt comfortable.

Alon Eizenman: But it was really nice journey, and then I decided to start another one and another one and another one.

Jennifer Bonine: So now with the current one that you have, I'm hearing that there's kind of a challenge that you can help people with their identifying with ... everyone wants to go really fast now, right? It's all about velocity and speed and the whole revolution around CI/CD and the pipelines and pushing stuff quickly. How have you seen it be successful in identifying where your bottlenecks are, where your challenges are in that process?

Alon Eizenman: Yeah, actually, it's starting with my previous company, Nolio, and we built release automation for ... and that was becoming continuous delivery and I was involved with many very large projects of very large companies running into continuous delivery, building automation, automating all their CI/CD and testing. I saw something that is repeating, so everyone that is fully automating their builds and deployments and testing, they start releasing really, really fast ... the release is becoming smaller and smaller, but then the first thing happening is that quality start dropping, and that repeated itself project after project.

Jennifer Bonine: So you could see the pattern?

Alon Eizenman: Yeah. I could see it. Then, that was already ... after the acquisition, we decided to leave CA and build a solution for that, so what ... Actually, when we were trying to identify what the problem is, we saw that when actually releasing, the time, when it's becoming very short, the time is not enough to make the analysis, or to get better knowledge of what the quality is. So usually people are only watching their test results and based on that, they release, but then it's not enough. For example, what happens if you just release a code and none of your testers are actually testing it and they are all green, you would release. And then basically you're releasing code that is not tested by any of your test and that's what happening, and then there are lots of surprises ...

Jennifer Bonine: Oh, yeah.

Alon Eizenman: ... in production. So we saw that, and then we started validating the market, and we saw that it's a real pain and we decided that we go for that.

Jennifer Bonine: Yeah. That's amazing. I mean, because I know there are probably companies out there who are saying, "We're being pushed into going faster and we have to do more and release quicker." But we're seeing that same thing. It's just sprinting towards failure, right? We're sprinting faster to code that doesn't work and ...

Alon Eizenman: Exactly.

Jennifer Bonine: ... issues and problems, so how do we stop that cycle? You know? What is your first approach when you're working with organizations or companies? How do you help them change that pattern, or how long does it take to identify the bottleneck, the problem, and then change it for those organizations?

Alon Eizenman: First, it's a matter of what information they get. You first need to get the tools to better understand, or to get to the level of confident, and then release. Once you have the data, it's not even enough. You need to go into ... sometimes it's a bit of cultural change, but not always. It depends on the organization, but usually it's taking the data that they didn't have before ... because identifying, for example, whether your code is covered or not, you need some other tools to do that because today you don't always have it. There are some coverage tools ... for example, for unit test ... but it's not enough.

Jennifer Bonine: Right.

Alon Eizenman: Then, once you provide the tools or the information, then they need to know how to use it, and then, usually we work with the customers and we take it step-by-step with them so they would be able to start using the data in order to actually improve. So, for example, if they would be able to guide the teams, for example, to reduce the risks and we show them how to do that, then they can increase their confident when they release. The outcome would be that the number of incidents in production start dropping ...

Jennifer Bonine: Yeah, you see that result.

Alon Eizenman: ... and then they see that it's actually working.

Jennifer Bonine: Yeah. They get more confidence, right?

Alon Eizenman: Action, yes.

Jennifer Bonine: Because now they're seeing improvement. I notice you guys have shirts that say, "Testing with the lights on," as opposed to testing in the dark, right, I'm assuming? Right?

Alon Eizenman: Yeah.

Jennifer Bonine: Where you don't have data or information. Actually being able to see what you're doing and make informed decisions.

Alon Eizenman: Yup, yeah.

Jennifer Bonine: Is that kind of the concept?

Alon Eizenman: Yeah. So if you don't know ...

Jennifer Bonine: Right.

Alon Eizenman: You don't know what ...

Jennifer Bonine: Right. Exactly.

Alon Eizenman: If you don't have the data, you cannot actually take actions.

Jennifer Bonine: Yup, you can't take actions based on that data. Now, interestingly enough ... So with that piece, are you seeing people build in those pipelines besides ... you know, there's the pyramid that people talk about, about you should focus most of your automation on the unit level and then move up. We've heard some different thoughts on that, right? People have different theories on that, that that doesn't actually cover everything, and that you also need to add in and include performance, you need to include security, you need to include some other things into those pipelines to make sure those are being checked as well, continuously. Any thoughts on that from your side of your philosophy or approach to it?

Alon Eizenman: Yeah, usually, it's the easiest ... easiest, I would say, action is to build unit test. Developers are actually releasing the code with unit test in the best case, not always. But I do see ... and then, the other level of testing, it's more complicated, so building a test, it's much more difficult and people are more struggling into it, and it's very ... I would say it's a long cycle, much more expensive, and very hard to maintain it.

Jennifer Bonine: Yup.

Alon Eizenman: So the more you go up and the ... up to the top, then there are some organizations that are even doing manual testing, but I see it very often that organizations are going into the transformation, or moving from ... there are lots of organizations hardly have unit test, or they don't have any automation, and they still ... And I meet lots of them even doing only manual testing.

Jennifer Bonine: Right.

Alon Eizenman: But then the question is, how you move from manual and start building automation? What do you do first?

Jennifer Bonine: Yeah, where do you start?

Alon Eizenman: Where do you start? Some are going and trying to replace the manual with integration test, and some are doing it ... regardless what they have with manual testing, they start from the bottom and they build ... you know, start investing with unit test, but I don't see a trend or everyone are doing first unit test and then integration. It depends, but even if you go through the transition, what is the right time to say, "I'm done with my manual test and I'm totally covering it by automation and then I can get rid of manual testing"? No one knows it. When is the right time? Actually, we know how to show them also the data so they can take decisions.

Jennifer Bonine: Yeah, and actually make a decision, so sounds like a lot of it's based on giving people information to make informed decisions as opposed to emotional decisions or decisions not based on fact, right?

Alon Eizenman: Yeah.

Jennifer Bonine: And giving them the data they need. Our time went really quickly. We're already almost up. If someone has more questions and we didn't get a chance to get to it today, how can they contact you?

Alon Eizenman: Yeah, they can go to LinkedIn. I'm on LinkedIn. Or to SeaLights' websites. So it's www.sealights.io. We have webinars, we have blogs, and ...

Jennifer Bonine: Perfect.

Alon Eizenman: ... they can contact me in LinkedIn any time.

Jennifer Bonine: Great! Thanks for being here with us.

Alon Eizenman: Yeah. Thank you ...

Jennifer Bonine: We're glad to have you.

Alon Eizenman: ... for having the opportunity to talk with you. Thank you.

Jennifer Bonine: Thanks, everyone. Stay tuned for our next interview.

Alon Eizenman is the CTO and cofounder at SeaLights Technologies. Before his current role, Alon was the VP of strategy and application delivery at CA Technologies, the CTO and co-founder of Nolio (acquired by CA Technologies), VP R&D and VP customer services at Coridan (acquired by BEA), senior manager, R&D, at Mercury Interactive (acquired by HP), and a part of the foundation team at Conduct (acquired by Mercury Interactive).

About the author

Jennifer Bonine is a VP of global delivery and solutions for tap|QA, Inc., a global company that specializes in strategic solutions for businesses. Jennifer began her career in consulting, implementing large ERP solutions. She has held executive level positions leading development, quality assurance and testing, organizational development, and process improvement teams for Fortune 500 companies in several domains. In a recent engagement for one of the world’s largest technology companies, Jennifer served as a strategy executive and in corporate marketing for the C-Suite. In her career, she has had several opportunities to build global teams from the ground up and has been fortunate to see how many of the world’s top companies operate from the C-Suite viewpoint.