What a CTO Learned at Nvidia About Managing Engineers
Xun Wang started his career at Nvidia, the tech company that has seen its fortunes rise with the emergence of generative AI and accompanying computing needs. Wang has taken the lessons he learned there into his role as CTO of Bloomreach, an e-commerce personalization company whose tech is driven by generative AI.
“The way that I think about organizing our organization is very much in line with this common computer science axiom, where the product you end up building reflects the organizational boundaries of your team,” Wang told Alex Williams, founder and publisher of The New Stack, in this episode of TNS Makers.
What Wang tries to do, he said “is understand what we’re trying to build as product, and to decompose that architecturally in a way that you know are the major components of this product that we’re building, and then to solely mold the organization towards that model.”
He learned that roll-up-your-sleeves approach from Jensen Huang, Nvidia’s founder and CEO. As an engineer himself, Huang modeled the importance of digging into the details.
“He said to me once that you cannot test something into existence,” Wang said. “That’s something that I’ll also remember. What he meant by that is that you need to work on fundamental architecture and platform, and that you can’t just iteratively improve on things and then test it, and then, you know, refine a little bit, and test and refine a little bit.”
In other words, “think about building something right that will last a long time.”
The curiosity about details, Wang said, would make Nvidia’s engineers feel understood and supported. “He would dig so deep down into a particular issue that you feel like he understands it as well as we do.”
That is a level of empathy that leaders with strictly a business background can’t offer their engineering teams, Wang told the Makers audience.
“As an engineering leader, you tend to understand where they’re coming from, because you’ve been through that problem before, and then you can relate to it,” he said. “And then you can either help them out, or sometimes you just accept that, hey, this is a problem that’s not feasible to solve. And let’s move on and try another, different way of thinking.“
‘You Can’t Be Afraid to Learn New Things’
Wang and Williams’ Makers conversation ranged widely, encompassing not just engineering management but also the impact generative AI is having on companies like Bloomreach, the demands the new tech will make on engineers and developers.
Bloomreach was founded 15 years ago by former Google engineers with extensive background in machine learning and AI, Wang said. In 2020, it began adopting generative AI.
“Search is a key module in our product offering, and search essentially is an AI problem,” Wang said. “It’s about understanding human language in terms of the query that you type when you’re searching for a product and then matching that with a corpus of information. So there’s a lot of natural language processing that is needed to be able to understand the intent that is in your mind and translate it into something that a computer can understand.”
GenAI has drastically changed how applications are developed, Wang said. “Now we’re dealing with how do you manipulate models, rather than write the deterministic code? Now you still need to do quite a lot of sort of classical development that still runs on CPU. So you kind of need both skill sets, but you do need a new layer of knowledge and skills in being able to manipulate generative AI to do what you want to do.”
To survive and thrive in this new world, Wang said, is that both newbies and veterans “can’t be afraid to learn new things.”
“It’s very common, I think, to get into this mode where you’re just doing what you’ve done for the last whatever number of years, or what you were taught in school, and you’re not keeping up with things because, you know, educating yourself is a hard thing actually.”
He’s taking his own advice: “ I’m reading like, five white papers a week right now, and just keeping up and learning all of these things. And it’s a fire hose of information.”
Watch or listen to the full episode of Makers to find out what Wang has learned about GenAI and large language model training and what he thinks about Apple’s conversational AI interface.
