PyGlove, an ML collaboration tool
Improving the exchange in ML ideas via code with PyGlove.
Created on February 6|Last edited on February 6
Comment
Developed by Google, PyGlove is a library for symbolic object-oriented programming in Python. It was published in NeurIPS 2020 with additional functionality and efficiency demonstrated in their latest paper.
What is symbolic object-oriented programming?
Object-oriented programming or OOP for short is a paradigm or way of programming that revolves around the use of classes and objects. From Wikipedia, symbolic programming is "a programming paradigm in which the program can manipulate its own formulas and program components as if they were plain data". PyGlove's repo provides a minimal example of this:
Their recent paper shows how to apply this tool to expedite collaboration in ML located here. In their notebook example, they define a function called experiment .
The difference between simply passing in all the components of the experiment as parameters and PyGlove is that PyGlove lets you modify specific components without having to redefine it. As they show, instead of redefining another model with SeparableConv layers, they use PyGlove functions to substitute those modules into the function without having to redefine everything.
This intuitive approach means easy access for researchers and practitioners to share code, research, and designs. In their examples, this substitution is called patching.
But then I ask, can't those things also be simply parameterized?
Yes, they could be. But I think the simplicity and intuitiveness of this method really shines when you have dozens of parameters. Instead of clogging up the function declaration, you can simply modify exactly what you want, replacing explicitly what you need instead of redefining an entire set of configurations every function call.
What do they show in their paper?
They have results but also limitations. This mutable programming allows for objects to be edited downstream and may make it harder to read code and maintain. You sacrifice a bit of this safety for an improvement in defining and running lots of experiments.
Patches does save lots of code-writing! I can imagine large experiments with perhaps a parameter count nearing the hundreds would probably be reduced to a tenth of its size with this library.
References
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.