Automatically filtering data targeting MMLU Subsets
Created on May 14|Last edited on May 14
Comment
To answer the question: Can we automatically filter data to solve for certain domains?
Result: We see good promise in the STEM areas: mathematics and science clearly do better in the STEM portions. They also do better on MMLU 5 shot overall. We see the reverse trend in MMLU humanities where the humanities and social science splits do better on that respective exam, which is expected. The MMLU social science exam split makes less sense - mathematics and science splits perform the best on this exam, which is counter intuitive since we would expect the social science or humanities split to perform better.
Ran MEDU on several subsets:
- mathematics
- engineering
- science
- social science
- humanities
MMLU Scores
Run set
5
Add a comment