How Do You Identify Machine Learning Opportunities?
This article provides a playbook for machine learning practitioners on identifying new machine learning opportunities so they can be a part of the next big thing.
Created on May 11|Last edited on February 9
Comment
A few years ago, I was asked by my manager to figure out a way to save millions of dollars with machine learning.
If you scratch your head (I did at the time), the context was a large enterprise with huge operational costs. By that time, we’d standardized most of the business processes generating those costs, went through several cycles of loss elimination, and implemented more traditional IT systems and automation to reduce human effort. Machine learning was seen as the next big thing.
What can you do with such an ask? First, think about the types of ideas that satisfy this request. Then, execute. In this article, we explore a framework to help guide you.
Table of Contents
Two Types of Ideas1. Optimize operational decisions2. Triage and automate repetitive tasksThe Playbook1. Engage Business Stakeholders2. Evaluate Ideas3. Prototype4. Execute ProjectsConclusion
Two Types of Ideas
You can increase the chances of identifying potential ML opportunities by focusing on certain types of ideas. In my experience, I found that optimizing operational decisions and triaging and automating repetitive tasks are most likely to result in successful implementations.
1. Optimize operational decisions
In a large enterprise, hundreds of decisions are made on a daily basis: How much product A should we produce? What should be the price for product B? Should we give that discount to customer C? These decisions are often taken by people without access to all relevant data.
Try framing these decisions as prediction or optimization problems. A forecasting model may outperform human planners. A pricing model may suggest more optimal prices than a human expert. Even better: design systems where these algorithms and human experts work together to come up with optimal solutions.
2. Triage and automate repetitive tasks
If some of your employees get frustrated about work that is boring or repetitive, this may be a signal of a potential automation opportunity. This usually involves some unstructured data (PDF invoice, email, text message, etc.) and may involve processing invoices and handling IT tickets or customer service requests.
The problem is that in most of these instances, there is usually a long tail of rare cases that could be mishandled by a machine learning solution. This is where an intelligent triage (or human-in-the-loop) solution can help.
When building a machine learning solution to handle unstructured data, try to calibrate its confidence in handling a particular input. If the confidence is low, route it to a human expert. That way, you will benefit from people working on the most challenging cases while having a model process for the boring ones.
The Playbook
How do you go from ideas to production? While implementing new technologies in the past, we developed a playbook that could be applied to machine learning as well. Here are the key steps we followed:
1. Engage Business Stakeholders
You can’t come in as an outsider and tell people in an organization what they should do. They know their business, have their own goals, and work hard to achieve them. What you can do, however, is educate them about new technology and partner with them to implement it and help them achieve their goals.
We ran an education program targeting all levels of the organization, from operators to executives, explaining the basic concepts and the ways machine learning can help them. This resulted in a lot of positive energy and ideas that came back to our team.
2. Evaluate Ideas
Having a bunch of ideas for potential projects, the next step is to evaluate them. Here are some questions you should ask to determine if an idea is worth pursuing further:
- What is the business value? You probably want to prioritize ideas that generate the most value for your organization.
- Is it feasible? Specifically for machine learning project ideas, can this be framed as a machine learning problem, such as classification, regression, or clustering? Can we get data to build a model?
- What is the risk if something goes wrong?
- Is this the right thing to do?
After you’ve filtered the ideas that are most likely to be successful, you can charter them as projects or spend some more time prototyping with the business.
3. Prototype
I personally saw a lot of value in quickly prototyping a solution with the business partners before formally starting a project. This helped me better understand the idea and business constraints. It forced me to get a sample of data and see its quality. It allows a discussion on the success criteria - what metric should be used and what level of performance is satisfactory.
Prototyping is also a great way of fostering and testing the engagement of your business partners. If they get excited about the problem, it will be easier to take the project off the ground. If they are skeptical, the project will be much harder to execute.
4. Execute Projects
Getting a model from idea to production involves a lot of work! This is where project management can be helpful. If you are interested in best practices for machine learning project management, let us know in the comments - this topic definitely deserves another article!
Conclusion
If you follow this process, you’re likely to start several machine learning projects. Machine learning engineers will run hundreds of experiments to find the best solution to your problem. You should optimize their workflow in the same way you’re optimizing your business processes. This is exactly where the Weights & Biases platform can help. Learn more today.
Add a comment
Tags: Articles, Domain Agnostic
Iterate on AI agents and models faster. Try Weights & Biases today.