Bringing genetic insights to everyone with Invitae's Head of AI, Matthew Davis
Matthew explains how combining machine learning and computational biology can provide mainstream medicine with better diagnostics and insights.
Listen on these platforms
Matthew Davis is Head of AI at Invitae, the largest and fastest growing genetic testing company in the world. His research includes bioinformatics, computational biology, NLP, reinforcement learning, and information retrieval. Matthew was previously at IBM Research AI, where he led a research team focused on improving AI systems.
Connect with Matthew
0:00 Sneak peek, intro
1:02 What is Invitae?
2:58 Why genetic testing can help everyone
7:51 How Invitae uses ML techniques
14:02 Modeling molecules and deciding which genes to look at
22:22 NLP applications in bioinformatics
27:10 Team structure at Invitae
36:50 Why reasoning is an underrated topic in ML
40:25 Why having a clear buy-in is important
Note: Transcriptions are provided by a third-party service, and may contain some inaccuracies. Please submit any corrections to email@example.com. Thank you!
There's lots of genetic defect in everyone. We think the average healthy person has a couple of hundred genes with defective function in their genome. And there's probably lots of what we think of as subclinical symptoms wandering around, across the whole population. So I think that's the future vision of the company is everyone would benefit from having a full understanding of their genetic background. And it's a complicated problem that the doctors don't understand. The patients certainly don't understand, and that we're really at the frontier of understanding.
You're listening to Gradient Dissent, a show about machine learning in the real world. And I'm your host Lukas Biewald.
Matthew Davis is the Head of AI at Invitae, a medical genetic testing company. He applies a really wide range of machine learning techniques to the genetic testing problem, which I think is one of the most interesting applications of ML today. I'm super excited to talk to him.
Invitae is actually a household name in my house because my wife runs a start-up that sells Invitae and I run a startup that also sells Invitae. So we're one of the very few overlapping customers. So I feel like I know Invitae very well, but I was thinking if that wasn't the case, I would definitely not know Invitae. So I was wondering if you could describe what Invitae does and how I might interact with your products as a consumer.
Yeah, sure. So for starters, we're a medical genetic diagnostics company and pretty sure by volume of tests, we're the biggest in the world. We're-
Which is amazing, because you're fairly new for a public company. Didn't you start in 2010 or something like that.
Yeah, that's about right. So the company itself is about a decade old and that's not a coincidence because the availability of high throughput, low cost to genomic sequencing really came online in 2008 with Illumina making it a scalable platform. And at that point it became clear that instead of analyzing one or two genes at a time, that you could be analyzing lots of genes for less money.
And I think the strategy that was clear to the founders was, it's a very narrow market with a very high margin that actually should be an addressable market of everyone with access to modern medicine. And instead the cost could be low and the volume could be high. And that if you pursued that strategy, there was actually way bigger benefits to mankind and also shareholders of a company, because you'd start to learn things about medical genetics, and disease, and probably most importantly relationships to treatments that you weren't going to learn if you took a small, addressable market strategy. So that was the vision-
I think you live in this, but for someone who hasn't had a genetic test for a medical reason, what would be a scenario where you'd actually want that, and what would it do for you?
Yeah. So classically, diagnostics were not about genetics. They were about your cholesterol is high or some other hormone is low or whatever. Genetic diagnostics were first proven en masse at breast cancer where we know there are genetic predispositions that would change your treatment strategy. Where the risk is high enough, that if your mother or your sister or your grandmother, your aunt had breast cancer at age 70... People get cancer when they're old. But if it's at age 35, that's way scarier. And when we added genetic analysis on top of that, we could further partition that to, "Oh, because they had this variant, they were early cancer patients." And then your doctor can help you make the best practice decisions about how to avoid it. At the extreme level that's a prophylactic mastectomy, but there are lots of other intermediates including like which type of drug would be most effective for you.
Should you risk the downsides of chemo? Should you just wait? So it worked with breast cancer. And then as we started being able to analyze more and more genes, we started discovering more and more things that it works for. I think one thing to keep in mind is that human geneticists, in general, they historically study horrific things. Like when you're studying fruit flies or mice or humans, it's some big effects. There's this old saying, big mutations have big effects. Meaning we study things that could give you a heart attack at an early age or make you grow tumors or have crippling nervous system diseases. And there are lots of people at risk for that who don't know it. But I think the real future is there's lots of genetic defect in everyone.
We think the average healthy person has a couple of hundred genes with defective function in their genome. And there's probably lots of what we think of as subclinical symptoms wandering around, across the whole population. So I think that's the future vision of the company is everyone would benefit from having a full understanding of their genetic background. And it's a complicated problem that the doctors don't understand, the patients certainly don't understand. And that we're really at the frontier of understanding. So that's a little bit of the history and a little bit of the future mission statement.
And so I'd imagine that a lot of people listening to this have done one of the consumer tests of maybe Ancestry or 23andMe. So how is what you do different from what happens there?
Yeah. I mean, it's a great question. And it's one that pre-COVID, riding on a plane, someone asks you what you do. And they're "Oh, like 23andMe". And you're "Ah, man, no". I mean, the obvious difference to the interaction with our customers is that historically it goes through a doctor. It's a medical test and you want that provisioned and administered by a medical professional. And 23andMe, it's a fascinating company, but they've focused on things whether or not you like cilantro, not whether or not you're at risk for disease.
And they have tried to move into a diagnostic space, but they're not built for that. And we finally acknowledged that a couple of years ago, after many years of not wanting to offer a medical diagnostic procedure directly to patients, because we didn't want people to go with information in hand, but not understanding an explanation that they could get from a medical caregiver.
So that's the real differences, like we have medical caretakers in place, but we now have a strategy where we let patients initiate their orders. And that's really because there's a lot of the country that doesn't have access to one of the few thousand genetic counselors in the US, so there are places where it's a six months wait. There's places where you're just not going to go. And thanks to telemedicine, thanks to software engineering, it's easier now for us to let a patient whose mother had breast cancer start the process themselves.
We refer them to a telemedicine genetic counselor who helps them by being their medical caregiver, without them having to wait six months to go to a medical center that's a hundred miles away. And we think that's great. Does it lead to a little more confusion? Well, we used to say, we're not consumer-facing, and now we have patient-facing order forms, but that's really the difference. We are trying to help people with a complicated medical problem, not find out their ancestry, unless of course your ancestry has direct bearing on the medical risk.
Got it. That makes sense. I mean, how does AI fit into this? You talk about a broad range... When I've talked to you in the past, I've been shocked by the number of different ML fields that you draw from. So I was thinking maybe you could give me an overview of the different problems, where machine learning techniques can help with what you all are doing.
Yeah. So we say AI and it was an easy-to-adopt term, especially for the last few years, but when I think of AI, I really think of every chapter of a textbook of a field of computer science, it's been around for many decades. And a lot of it, thankfully is machine learning and some of it's not. So some of it is optimization algorithms and robotic planning and so forth. It has been around for a long time and it's still making rapid advance in those fields, but maybe there's a little less well known to machine learning folks. And then a bunch of it is machine learning approximations that can make a problem tractable that wasn't tractable before.
So I mean, the actual applications. We have a key scaling problem. In a lot of ways we're a manufacturing company and our volume tends to almost double every year. And we have a laboratory that has to run assays with actual robots. We have rather complicated standard operating procedures and business process models that need careful execution and not to mention audit logging and accounting stuff. It's the medical field, so we have to be compliant and follow not just HIPAA laws, which are complex, but also contractual obligations to insurance companies and things like that.
There's a lot of complicated process modeling. And then there's a lot of knowledge worker problems. So we have on staff dozens of PhD geneticists and biologists who have done this Herculean task of curating the medical genetic literature for any scrap of evidence that could inform whether this variant that seems to be breaking the function of a gene and a patient could actually be the causal factor that puts them at higher risk, because it's still unknown. Most of the variants that are analyzed in medical genetics, we're still uncertain what their eventual effect would be.
That involves literature mining, all the most contemporary NLP methods for entity extraction, relationship modeling, linking ontologies. We don't get into things like summarization because even the fanciest most expensive model, it's not confident enough to write a medical report for someone. But the sort of language modeling that goes into something like GPT-3 we can use that for concept embeddings for extraction, for classification, for recommendation engines. So we have a lot of that NLP work that a lot of the rest of the world thinks of, and then we've got a fair chunk of computer vision problems, whether they're things like document processing, computer vision, or they're computational biology problems.
And about half of my team is devoted to more core research advancing future products, doing academic collaborations with folks. So they're really trying to struggle with the problem I stated before, which is geneticists traditionally focus on big diseases with big mutations, but there's a lot more subtle signal going on for almost everyone on the planet. And, in a sense it's a signal detection problem. It's really a high order of complexity. It's silly to think of it this way, but if you just imagine we have 25,000 genes working in combinations or anything, how do you search a space of 25,000 factorial combinations? So the hope is that things that were completely intractable before by enumeration could be tractable by approximation. And so that's one of the great hopes for computational biology is that we can produce a search base with machine learning. We covered computational biology, knowledge, and operations. That's a big breadth of stuff to worry about.
And then on top of that, I think there are things like graph embeddings for heterogeneous networks, where there's lots of reasons to believe that heterogeneous entities out in the literature shouldn't be just treated as word tokens that you learn with a language model. But instead you can layer on causality and known relationships. Biology is this kind of fascinating field because if you really cared about Newtonian mechanics, then you probably don't need a neural network approximator to tell you how fast the ball is going to roll down the incline plane with a certain coefficient of friction and whatever right, because you can physically model it really accurately. And in biology, if you open a biology textbook, there are all these cartoons of "This protein binds to this protein and they both bind to the DNA, and then the RNA is made" and whatever.
And they're not just cartoons that you've memorized when you're a biology undergrad, they're actual physical models of material process of the universe. But the uncertainty is way higher. They are rough drafts, and because it's a tiny little submicroscopic machines, historically we don't just take the picture. I guess that's less and less true, because electron microscopy is now getting really good, and x-ray crystallography in some ways is really good at that. But for the most part, you do it by inference.
You do some experiment and the readout is like, you look at a different colored of like, jelly of agarose in a tray, and it's all by inference. So when you look and see one of those CSI TV shows, and they're looking at the big bands of DNA, that's a very abstract version of the actual physical process. And that's where it's great for machine learning because there's enough structure to that cartoon that you don't have to imagine every possible force vector.
You have some constraints, but it's uncertain enough that it's not Newtonian mechanics. So modeling it with uncertainty and then using those indirect observations to guide your search, in a lot of ways, it's a perfect field for using model-based machinery.
Well, okay. So right, I'm taking like a mental note of all the different applications that you mentioned. I have so many questions on each one, but maybe we should start with the last one, because it seems very intriguing.
Why would a company like yours care about modeling the chemistry of molecules? What does that do for you?
Yeah. So, I mean, we know that if you put a change in this DNA sequence, that there's a high likelihood that it's going to change what amino acid is put in the protein, very predictably. We can predict that from basic biology knowledge, but we don't necessarily know that's going to affect the function of protein. And the easier ways historically to make computational estimates of that, were "Compare the sequence of that gene across a thousand related species or 10,000 humans and see, is it always the same letter?"
And it's probably important because if it's not important then evolution will let it float around, but there's actually quite a lot of flexibility in those proteins where they're still functional. So there might be a subset of people where it's different and it doesn't actually matter. If you were an actual biochemist, then you might go do experiments in the laboratory, seeing how the proteins actually touch each other and discovering that the enzyme works better or it doesn't work as well.
And it's really expensive and time consuming to do that at that slow process and scale. But if you had molecular models of those physical properties, then you could do in silico experiments and say well, I can't be sure that the enzymes not going to be as efficient, but based on a whole lot of-
You say in silico, you mean like, in silicon? I don't know, that's a new term to me but I love it.
Yeah. I mean, that's not me, that's biology. Biology loves Latin and so that's a well-tested phrase in computational biology for a long time, but yeah, that's the right answer. That's what it means.
So you're doing the simulation and then you can say, "Well with some certainty, based on the parameterization of those actual biochemical experiments that other people have done, this looks like a big change and therefore it's going to affect the function of the gene". And therefore we have more reason to believe in a very Bayesian sense, our belief increases that this is the cause of someone's disease.
And is this something you'd really do in the future? Or is it in use now? Is this something that everyone would have to do to make a realistic model of... I guess how in use is this kind of modeling, for deciding what genes to look at? This is something you do every day?
I mean, this is definitely a thing that our company does. There's a team that does this and I think it's also an interesting example where I think it's a case where industrial research has more potential than traditional academic research, just because of the volume. The biggest academic collaborations for genome sequencing don't actually get to the same number of people as come through our samples. And they're not as enriched for people who actually have disease. Like the big population, genome sequencing centers in China, and the UK, and the US, they're not generally systematically going after people with disease.
We have an ascertainment bias. It's actually a benefit if we want to study disease because people with disease in their families come through the door. And that means that we can do stuff that you can't do if you're working at the Broad Institute at MIT and Harvard or in Cambridge with European Bioinformatics Institute, like we have access to data that you can use these methods on that no one else can.
And how do you actually set up this problem? How do you formulate it? My mind is just how would I set this up as a machine learning problem that I could actually train on? Is it standard, like what the loss function is? What can you actually observe to put into this?
Yeah, right, I mean, I don't think there's a canonically true answer to that question, but we can talk a little bit about the pros and cons of the approaches. So, I mean, one thing is it's not a consumer recommender system where you recommend products and people click on them and buy them, or they don't. In fact, that's diagnostics, in general, has this problem with no ground truth. People die of symptoms on hospital beds and their doctors don't actually know in some sort of Plato, Aristotle sort of way, why they died. It's just that we have a stronger belief about the causality. So you can take a labeled dataset and say these people were diagnosed and they had that variant and I'll make a model that can predict that outcome with a supervised learning with it.
But you're not actually dealing with ground truth because some of those people had the disease, they had the variant and they had the disease, but they actually had the disease because they smoked cigarettes for 50 years or because they were 90 years old or some other confounding factor was there. So if you want to try and think of it as belief, then you can go down to Bayesian probabilistic graphical model, causality, Judea Pearl-explainable AI path that I think people are excited about talking about, but you have to know a lot of human knowledge goes into that. And it's not as simple as I have some labeled data and I'm going to train an arbitrarily deep neural network to approximate the soft max or something.
So you end up working a lot with "How do I take those physical models of what I think is going on in biology?", and you're trying to design the algorithm to do that, whether it's with causal graphical models or it's just knowing, "From these feature vectors I can learn an auto encoded representation that should in theory account for these factors we know from the physical model are important". And then I'm going to let the neural networks set the weights by showing it what are the observations that are the closest to ground truth people will ever have.
But it sounded there's sort of subproblems here that people work on. Like you talked about people looking at proteins and mixing them together and seeing what happens. Is that a subproblem of this bigger problem where you could have different observations and build a different model around it?
Yeah. Right. So, I mean, we do have a wet lab team that is collecting basic molecular biology data. But one of the awesome things about biology is that lots of other people are doing that. So lots of professors and lots of universities and lots of their grad students are collecting and publishing data in a way that is ingestible for us to learn some of those things. But there are places where you may identify key deficiency in that knowledge that are like well, it's worth it for us to do this experiment because it would really help us parameterize what we think is missing in this model. And so then you have from an industrial research perspective, you have to think about the cost benefit. Is it worth, spinning up a wet lab initiative to do stuff that's hard to do at scale? It's a feeling that you want that feature vector, it better be worth it because it's not cheap.
Sure, sure. Although it sounds like it's evocatively similar to exploring a space of hyperparameters for you know...
I mean, maybe it's more if you had a product recommender and you knew everything that everyone had ever clicked on, but it just still doesn't seem to have that much accuracy. So you send out some design researchers to talk to your customers and sit in their house with them and talk to them like, "Oh, that's weird. Everyone's buying Adidas. I didn't notice that before. Is that in the model? Let's go find out all the shoes everyone buys and then see if that improves the accuracy."
Got it. That makes sense.
The thing is if you had the whole life history of me as an individual and everything I'd ever done, then you might be able to start down the path of that modeling, but that's crazy. No one would do that. You just look at my ad click data and make some recommender that if it has 38% accuracy is going to make a bunch of money for an ad company. But if you're talking about someone's health and complex things like biology, then you want it to be higher accuracy and you got to go actually model stuff out deeper.
So I guess another whole field that you talked about doing is sort of the, what do you call it, sort of medical NLP or bioinformatics. I mean, this is one thing I've been curious about, you've seen a lot of progress, very visible progress in NLP, notably GPT-3, but also these word embeddings becoming super popular. Has that influenced bioinformatics? Does that directly apply? Can you fine tune these models on medical text domains? Or what's the state of the art there today?
Yeah. Right. So, I mean, I think there's two big problems in industry that people would love to solve. One of them is comprehending medical records and the other one is comprehending the medical literature. When you state the problems, they sound the same.
It's like, I went to extract the entities and map the relationships and then link them to ontologies so that I can structure the data and then make queries over it. And if you can do that, then the practical challenges are things like "Can I show to one of our clinical scientists the right piece of literature at the right time to help them make the right insight about this genetic variant that's never been observed in someone before?" And then if you look at the medical records, it's how do I take this allegedly structured unstructured data and turn it into something that's actually structured so that we can make trajectories of people's disease progression or predictive risk.
And so it turns out that training language models on Google books and New York Times articles and Wikipedia does not actually help that much, but also surprisingly, several years ago, I did some experiments when I used to be at IBM research where I had a research group. We did some experiments where we had domain specific corpuses and general corpuses. We would train the same models and to my surprise, the bigger general corpus helped more than the specific corpus. And that was an early transfer learning insight. It was like, "Take the biggest corpus you can get" and then transfer learning is a good idea.
What's hard is, the concepts are the same to humans, but when you look into a medical record, it says MGM, BRCA and that means maternal grandmother had breast cancer. And you look in the medical literature that's published academically, it doesn't even talk about the relatives, and it doesn't even say "breast cancer". It says "malignant neoplasm of the tissue". You don't even know it's talking about the same thing. So mapping the concepts across this is tricky and just the syntax, right? The medical abbreviations, and it's almost like it needs its own language model.
So I mean those are some of the hard problems for contemporary methods to actually work on, especially out of the box. So we do take encoder-decoder based sort of Transformer models and adapt them pretty readily with supervised training. And it's definitely better than starting from scratch, but it still requires domain experts labeling stuff to get there, or it takes some weak supervision, data programming sort of methods where people were writing rules that make a lot of sense to weekly label the data. And it's not as good as human-labeled expert data, but you can bootstrap yourself into having a better data set to train on. So some of those methods work really well in biology.
That's really interesting. Do you have the sense that, well I don't know, I have the sense that recently NLP methods have improved a lot. When I look at scores that I'm used to from a decade or two ago, they just seem much better over the last couple of years. Has the same thing happened in the medical field?
Kind of. So if you take the scores on question-answer datasets, some model is better at answering Stanford question-answer questions than I am, right. Super, very impressive. But I don't think you would expect the same thing to be true with a medical question-answer and a bunch of specialists, doctors in whatever domain. So no one expects a chat bot powered by a GPT-3 to be better at giving medical advice. But that said, the language model that's learned could be extremely useful for facilitating a human expert. And so I think that's where the hope is at this point, it's just like AI assistant, better information retrieval, better support for the expert is the current hope.
Got it. I guess a general problem that a lot of people ask me about, and I know a lot of people listening to this would wonder about is, how do you think about structuring your team? You talked about half the people doing core research, but then also it seems what you're doing is very connected to what the company is doing. Do you try to literally separate the people that are doing the applied stuff and research stuff? Or do you separate it by the field of work? Or how do you think about that?
Yeah, it's a really good question. And I think I suspect the answer that I have today will be different than the answer I have in a few years, which I know is different from the answer I had a few years ago. And it feels one of those things that we'll keep reinventing. We'll keep reinventing how to deploy software and we'll keep reinventing how to provision infrastructure. And we'll come back to the same basic principles that people thought of a few decades ago, but we'll keep refining it. And so right now, the company is effectively, the company still works like a startup with very clear product driven vertical teams. And the idea that we want to imbue a machine learning capability to the company, it's hard to figure that out.
And it's a little different if you're a Google and well, the company is built on machine learning based information retrieval. So we expect everyone to take a machine learning approach to something. So I guess the direct answer is we have a functional team. Everyone goes to meetings together, hangs out together, checks in together, but people have different projects. And it has definitely been hard for some of the team members. A common source of feedback is "I don't know what everyone else is doing because everyone is working on something else. I'm used to working with like four people on a specific project. And we talk every day in a standup meeting, and in this team everyone's doing something different." And the people on the team who went to grad school and experienced what that's like to get a PhD where it's ultimately up to you to do your thing, they're more comfortable with it because they're like, "Yeah, of course we're all doing our own thing."
In reality, I really hope everyone is not doing their own thing. I hope that there is cross-fertilization and support and it's inherently matrixed. But the goal is, we reserve some of the people's time for research because if you don't explicitly set aside the commitment, then it will be absorbed by whatever demand of the product team in the short term. And then we set aside some people's time to develop platforms that are modular and reusable with the hopes that we continue to imbue that throughout the rest of the engineering teams. And then we set aside some people who are then functionally assigned to specific engineering projects, whether it's to realize one of the research projects into production or it's to leverage one of the platforms for a problem, or maybe it's just someone has a pretty straight forward problem, and they need a scikit-learn model, and it's going to take someone an afternoon to prototype and three weeks to get it to production, so we stick someone in there for a sprint or two and make sure it happens. I guess in some sense it's a very zone defense strategy, it has to be flexible.
Right. And do you then hire people who have sort of knowledge of multiple topics, they seem such deep fields that are different. Is it possible to find someone that knows about multiple of these applications?
Yeah. So we hire people with specific expertise, for sure. I am actually just extremely fortunate. I was a software engineer who went to get a graduate degree in computational biology at a time when doing that probably also meant that you're going to do biology. And then I went and worked at IBM in a research division with just this huge diversity of industrial interests. So I was exposed to lots of different AI methods and that was not something that I knew was going to happen to me, but was really fortunate. And what that means is I met people in different industries at different conferences to understand, "Oh, there's this boutique thing that was popular two decades ago, but continues to be a core technology for NASA or Toyota. And not a lot of people pay attention to it, but man, it can solve a lot of problems." But you're just not going to find a Coursera course on [it].
So it's great because we can find people with that expertise. And if they're CS PhDs, kind of fortunately, right, so Computer Science PhDs are generally interested in stuff. If you practiced NLP algorithms, you're probably still interested in plain computer vision. So I think that's a fortunate thing. I can find someone with the expertise in information retrieval and they can still make really meaningful contributions to other types of problems and other subject domains. One of the harder things is getting the biology knowledge solid enough that they can talk to the biologist and the other stakeholders and quickly understand the problem statement.
That makes sense. And I guess, one of the things that you talk about a lot, I think is the importance of engineering to making all this stuff work. Do you hire just pure engineers on your team or do you rely on outside teams to provide that?
Do you have a ratio that you shoot for? I'm just curious about this as sort of algorithms to implement or something.
I think it just depends, but we've tried to maintain a bench of depth so that we can recombine it. I think a lot of really high impact projects can be done with one algorithms person prototypes a thing, hands it off to one or two engineers who implement the thing. And then we further hand it off after it's implemented to an engineering team that's going to love and care for it in the long term and maybe come back to us if they need new features, but to them it looks software that could have come from anywhere.
Other projects you need... We have some of our more challenging algorithmic problems where we'd like the approach of probabilistic programming and there's not a lot of mature frameworks out there for that, like Google and Uber AI, both popularized some, but you need some pretty heavy lifting on algorithm development, some fearless backend engineering chops to make anything happen.
And then once you have the ability to make anything happen, then you also want to layer in the computational biology expertise to make sure the right modeling steps that I described before is happening. So that could be a several person team just to make the prototype, because it's complicated and the tooling requires help. And it's not as simple as a web backend and a React frontend or something.
One of the things that I've been noticing, at my company we've seen more and more interest in customers coming in from pharma and the medical stuff. And it always feels to me like, of all of our customers, it's the biggest culture clash. Basic stuff that I feel I haven't discussed in a long time, they'll be suspicious of open source software, and so I'm like, "Oh my God, is it like 1995?"
Does that happen at Invitae? Because it's sort of a newer company and sort of more maybe CS-focused or do you also feel that working with biologists?
No, I don't think it's a problem here. And certainly I saw that problem with IBM customers at times. I was lucky working at IBM, they were huge investors in Linux, 20 years ago. And it was clear to everyone why that continued to be the case, but I would see it from other companies who were like, "I would prefer the lower performance, more expensive proprietary thing, thank you."
I mean, I think one of the virtues of Invitae is it does have a Bay Area ethos and "Get there faster, get there cheaper" is a good idea.
So I don't think there's any skepticism there, but sometimes you collaborate with the insurance agencies, or the insurance payers or Medicare and then you're into a whole ballpark of... It's not even individual skepticism. And it's not even institutional skepticism. It's codifying contracts. Yeah. So, I mean for us, it's not a problem at all. I think I would imagine the bigger the company, the older the company, it's probably true in every sector, but a lot of the big old companies in technology got over it a long time.
Right, right. Yeah. That makes sense.
Well we always end with two questions. I want to make sure we have time for them. They're broad and feel free to expound a little bit, but one thing we always ask people is, when you look at what people are seeing or what people are doing an ML, what's a topic that you think people don't pay enough attention to maybe a skillset that you'd like to hire for, but nobody's studying or something that you'd to spend more time on if you could.
Yeah. I mean, so just reasoning in general. And I think this happens. If you go to a general AI conference, whether it's one in recent favor, like ICML or NeurIps, or it's the oldest, AAAI sort of standard conferences, keynote speakers will talk about fast and slow AI, or a system one and system two or whatever. But I think no one ever actually wants to do reasoning, because it's so hard. But then you see communities of self-flagellating academics, lamenting that they're only competing to get a higher F1 score on some published data set that's been around forever and what's the actual use of it all. And I think this conversation is also often turned to "Oh, if we were doing some more complex reasoning thing than it would be more valuable for mankind", but it's just hard.
So that's why I said earlier, we're into the probabilistic programming ideas because you can take a causal graphical model that can be highly explainable and you can not have to Monte Carlo sample it until the end of time. Thanks to variational inference and frameworks like Edward and Pyro that make it easier. I think that's going to push our ability to reason about really complex things and bring human expertise in and let people help correct the models and do a lot of the things that would just, frankly, feel like people talk about doing, but are hard to do.
I think there's also a bias against systems at academic conferences. No one wants to write a quote unquote systems paper in a workshop. They want to write an algorithms paper, that's going to get cited 10,000 times. But, that work is probably more important. Or like, putting together a thing that solves a problem is really valuable. And I wish we trained grad students to think about that instead of to think about hyperparameter tuning effectively. And that's if I could snap my fingers and change one thing about this field, I think that would get us like "Pay attention to complicated systems, because it will help you build things like reasoning engines".
Interesting. I guess I have not made a connection between reasoning engines and systems. Those two both seem separate tracks. Is there something about making working systems in your experience that really requires reasoning?
So if you take an example, the word embeddings or graph embeddings, once you have that representation of similarity, you can rank documents and calculate a F1 score, but you can also give it to an expert and say, "I found this thing for you. Do you think it's the right thing or not?" And if they say yes, you can further process it and extract some more information out of it for a specific purpose. And if they say no, then you could ask them why not? And reason about the introducing of relationships you've extracted and actually auto refine your model from the feedback of the user, but that's a HCI problem. That's an interaction problem that you're not even going to start to touch, unless you're open to the idea of building some boring system that ties together a user interface and some backend systems that are not all machine learning.
Totally. Okay. So final question is, so when you look at, in your career taking stuff from the sort of prototype version to deploy it in the real world and useful, where do you see the sort of biggest bottlenecks or biggest problems?
I think the biggest fundamental problem is when you work in an industry and you have an existing company, but probably also when you have a startup and you're trying to get funding for it, to have buy-in from the product philosophy, from the outset. And to have some willingness that the prototypes might not work. So you need a foundational, definitely going-to-work plan to make a product. But to have a "I'm going to reserve 20% of the resources to try this crazier thing, we'll prototype it, and if it works, it'll be great", you gotta have the person who's going to take it to market care about that idea.
When you have a bunch of researchers hanging out, making cool prototypes, and then they take it around like a toddler who made a thing, and they're like, "Oh, look at this thing I made you, don't you love me?" I think almost every researcher I've known in industry could identify with what I just said as the toddler, because we all think we have some brilliant idea and we make a thing and we take it to people and they're like, "I'm sorry, I have a deadline right now. I don't understand, I already have a thing that recommends papers. I think it uses a regular expression?"
They don't care and they don't see the value. So you have to really get the buy-in at the beginning or you can spend a lot of time making a hard thing, and probably an expensive thing, happen and then it doesn't actually go anywhere. And it's more emotional than strategic. You have to be open to the idea that they might not see the value in what you want to do, and that helps you prioritize what to do.
Interesting. We've not heard that answer yet, but that really resonates. That makes a lot of sense. Thank you so much. This is a lot of fun. I really appreciate your openness.
You're welcome. Really appreciate it.
Really appreciate it.
Thanks for listening to another episode of Gradient Dissent, doing these interviews are a lot of fun and it's especially fun for me when I can actually hear from the people that are listening to these episodes. So if you wouldn't mind leaving a comment and telling me what you think, or starting a conversation that would make me inspired to do more of these episodes. And also if you wouldn't mind liking and subscribing, I'd appreciate that a lot.