Sentiment classification with the Reddit Praw API and GPT-4o-mini

Learn how to build a Reddit sentiment analysis pipeline that uses GPT-4o-mini to extract opinions from real discussions across subreddits—filtering, summarizing, and classifying posts and comments at scale.
Brett Young
Created on April 3|Last edited on April 16
Comment
Have you ever tried to gauge public opinion on a new technology or controversial issue by reading through Reddit discussions, only to feel overwhelmed by the sheer volume and disorganized nature of the conversation? Reddit contains a wealth of opinions, but manually extracting meaningful patterns or trends from these threads can be difficult. It can be easy to become overly influenced by a handful of charismatic or heavily upvoted responses, mistakenly assuming these represent the broader community’s viewpoint, while ignoring the quieter but potentially far more representative majority.
This article examines how automated sentiment analysis can address these challenges by providing clearer and more balanced insights from Reddit discussions. By using GPT-4o-mini for sentiment classification, this method helps uncover the prevailing opinions hidden within vast and complex comment threads, enabling a more accurate understanding of public perspectives beyond the loudest voices.
﻿
Setting up for Reddit sentiment analysisInstalling dependencies and setting up your environmentAuthenticating with the Reddit APIFetching and processing Reddit contentClassifying Reddit sentiment in PythonAnalyzing Sentiment with GPT-4o-mini Tracking sentiment analysis with W&B Weave Real-world applications and use casesConclusion 
﻿
﻿
Setting up for Reddit sentiment analysisBefore diving into sentiment analysis, you'll need to set up your Python environment and authenticate with the Reddit API. This next section guides serves as a guide to walk you through installing dependencies, obtaining API credentials, and fetching relevant Reddit content for analysis.
Installing dependencies and setting up your environmentFirst, let's install the required libraries for our Reddit sentiment analysis pipeline:
pip install praw litellm asyncio weave
Authenticating with the Reddit APITo access Reddit data, you'll need to create a Reddit developer account and register an application:
1. Go to https://www.reddit.com/prefs/apps
2. Click "Create App" or "Create Another App" at the bottom
3. Fill in the required information:
Name: (your app name, e.g., "Sentiment Analyzer")
Select "script" as the app type
Set "redirect uri" to http://localhost:8000
Add a description
Click "Create app"
Note your client ID (under the app name) and client secret
﻿
Now you can authenticate with the Reddit API using PRAW:
import praw
﻿
REDDIT_CLIENT_ID = 'YOUR_CLIENT_ID'
REDDIT_CLIENT_SECRET = 'YOUR_CLIENT_SECRET'
REDDIT_USER_AGENT = 'sentiment_analyzer/0.1 by YOUR_USERNAME'
﻿
# Initialize the Reddit API client - using read-only mode for simplicity
reddit = praw.Reddit(
    client_id=REDDIT_CLIENT_ID,
    client_secret=REDDIT_CLIENT_SECRET,
    user_agent=REDDIT_USER_AGENT
)
﻿
# test: print the titles of the 5 hot posts in r/Python
for submission in reddit.subreddit('Python').hot(limit=5):
    print(submission.title)
Fetching and processing Reddit contentRaw Reddit data presents unique challenges for traditional sentiment analysis approaches. Posts and comments often contain slang, emojis, hyperlinks, markdown formatting, quoted text, and nested discussions that can confuse simple classifiers. Historically, sentiment analysis pipelines required extensive preprocessing to clean this noisy data—removing URLs, stripping formatting tags, normalizing text case, removing stop words, and handling special characters.
However, in the age of generative AI, large language models have transformed our approach to text preprocessing. Unlike traditional NLP pipelines that required meticulous cleaning to function effectively, modern LLMs can understand and interpret raw Reddit content with minimal preprocessing. These models have been trained on diverse internet text, including social media content similar to Reddit, and can handle the platform's unique linguistic patterns.
I will briefly show how we can access reddit posts and comments for a given subreddit, which will be the foundation of our project: 
import praw
import os
import json
from datetime import datetime
﻿
# Reddit API credentials
# Initialize the Reddit API client - using read-only mode for simplicity
REDDIT_CLIENT_ID = 'YOUR_REDDIT_CLIENT_ID'
REDDIT_CLIENT_SECRET = 'YOUR_REDDIT_CLIENT_SECRET'
REDDIT_USER_AGENT = 'your_app_name/0.01 by YOUR_REDDIT_USERNAME'
REDDIT_USERNAME = 'YOUR_REDDIT_USERNAME'
REDDIT_PASSWORD = 'YOUR_REDDIT_PASSWORD'
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
﻿
# Init Reddit
reddit = praw.Reddit(
    client_id=REDDIT_CLIENT_ID,
    client_secret=REDDIT_CLIENT_SECRET,
    user_agent=REDDIT_USER_AGENT,
    username=REDDIT_USERNAME,
    password=REDDIT_PASSWORD
)
﻿
def search_subreddits(topic, subreddits=None, limit=10, time_filter='month'):
    """
    Search for posts about a specific topic across one or more subreddits.
    
    Args:
        topic (str): The search query
        subreddits (list): List of subreddit names to search. If None, searches all of Reddit
        limit (int): Maximum number of posts to retrieve per subreddit
        time_filter (str): 'hour', 'day', 'week', 'month', 'year', or 'all'
        
    Returns:
        list: List of post data dictionaries
    """
    results = []
    
    # If no specific subreddits provided, search all of Reddit
    if not subreddits:
        print(f"Searching all of Reddit for: {topic}")
        for post in reddit.subreddit('all').search(topic, sort='relevance', 
                                                 time_filter=time_filter, limit=limit):
            post_data = extract_post_data(post)
            results.append(post_data)
            print(f"Found post: {post.title[:60]}...")
    else:
        # Search each specified subreddit
        for sub_name in subreddits:
            try:
                print(f"Searching r/{sub_name} for: {topic}")
                subreddit = reddit.subreddit(sub_name)
                
                for post in subreddit.search(topic, sort='relevance', 
                                           time_filter=time_filter, limit=limit):
                    post_data = extract_post_data(post)
                    results.append(post_data)
                    print(f"Found post: {post.title[:60]}...")
            except Exception as e:
                print(f"Error searching r/{sub_name}: {str(e)}")
    
    print(f"Retrieved {len(results)} posts total")
    return results
﻿
def extract_post_data(post):
    """
    Extract relevant data from a Reddit post and its comments.
    
    Args:
        post: A PRAW post object
        
    Returns:
        dict: Dictionary containing post data and comments
    """
    # Extract basic post information
    post_data = {
        "id": post.id,
        "title": post.title,
        "body": post.selftext,
        "author": str(post.author),
        "score": post.score,
        "upvote_ratio": post.upvote_ratio,
        "url": post.url,
        "created_utc": post.created_utc,
        "created_date": datetime.fromtimestamp(post.created_utc).strftime('%Y-%m-%d %H:%M:%S'),
        "num_comments": post.num_comments,
        "subreddit": post.subreddit.display_name,
        "is_self": post.is_self,  # True if text post, False if link post
        "permalink": f"https://www.reddit.com{post.permalink}",
        "comments": []
    }
    
    # Get comments (limiting to top-level comments for simplicity)
    post.comments.replace_more(limit=0)  # Remove "load more comments" objects
    
    for comment in post.comments[:20]:  # Get top 20 comments
        try:
            comment_data = {
                "id": comment.id,
                "author": str(comment.author),
                "body": comment.body,
                "score": comment.score,
                "created_utc": comment.created_utc,
                "created_date": datetime.fromtimestamp(comment.created_utc).strftime('%Y-%m-%d %H:%M:%S')
            }
            post_data["comments"].append(comment_data)
        except Exception as e:
            print(f"Error processing comment: {str(e)}")
    
    return post_data
﻿
def save_to_json(data, filename="reddit_data.json"):
    """Save the Reddit data to a JSON file"""
    with open(filename, 'w', encoding='utf-8') as f:
        json.dump(data, f, ensure_ascii=False, indent=2)
    print(f"Data saved to {filename}")
﻿
# Example usage
if __name__ == "__main__":
    # Define search parameters
    TOPIC = "artificial intelligence"
    SUBREDDITS = ["technology", "MachineLearning", "futurology", "ArtificialIntelligence"]
    POST_LIMIT = 5  # Posts per subreddit
    
    # Search for posts
    results = search_subreddits(TOPIC, SUBREDDITS, limit=POST_LIMIT)
    
    # Save data to file
    save_to_json(results, f"{TOPIC.replace(' ', '_')}_reddit_data.json")
This script uses PRAW to collect Reddit posts and comments for a given topic. It starts by authenticating with Reddit’s API, then defines search_subreddits, which searches either all of Reddit or specific subreddits for posts related to a keyword. For each post, it calls extract_post_data to pull fields like title, body, author, score, timestamps, and up to 20 top-level comments. These are stored as dictionaries. save_to_json writes the full dataset to a JSON file. At the end, it runs a sample query for “artificial intelligence” across selected subreddits, then saves the results as artificial_intelligence_reddit_data.json. The output can be used for sentiment analysis or other text processing.
Classifying Reddit sentiment in PythonTo judge sentiment from Reddit posts and comments, we use a script that combines Reddit data extraction with prompt-based classification via an LLM. Instead of relying on lexicons or pretrained classifiers, this method uses natural prompts and GPT-4o-mini to determine whether the sentiment of a post or comment is Good, Bad, or Neutral in relation to a specific topic. This approach is more robust against sarcasm, slang, and context ambiguity—things that are common on Reddit.
I’ll first share the full script for our app, then I will cover some of the core details of how it works. In our script, we will use Weave, which is a lightweight observability and logging tool designed for LLM apps. It can help track which prompts were run, what responses were returned, and make debugging easier as the analysis scales. You can run the app without it, but it’s useful for tracing what the model was thinking at each step.
The script begins by using get_subreddits() to identify relevant communities where the topic is likely being discussed. This function takes the user’s query and prompts an LLM to list five subreddit names based on where people might naturally talk about that subject. Once those subreddits are identified, the script generates search terms and fetches threads from each one. From there, it filters posts for relevance—meaning it doesn’t just analyze everything it finds. Instead, it uses an LLM to score how closely each post actually relates to the original query. Posts that fall below a relevance threshold (0.6 by default) are skipped. This ensures that the script only runs sentiment classification on threads that are clearly on-topic. Once filtered, long posts are summarized, sentiment is classified on both the main post and a limited number of top-level comments, and the output is printed progressively. The run finishes with a breakdown of Good, Neutral, and Bad opinions across posts and comments, along with a confidence score for the dominant sentiment.
import os
import re
import asyncio
import praw
from litellm import acompletion
from collections import Counter
import weave; weave.init('reddit_sentiment')
﻿
# Reddit creds
REDDIT_CLIENT_ID = 'YOUR_REDDIT_CLIENT_ID'
REDDIT_CLIENT_SECRET = 'YOUR_REDDIT_CLIENT_SECRET'
REDDIT_USER_AGENT = 'your_app_name/0.01 by YOUR_USERNAME'
REDDIT_USERNAME = 'YOUR_USERNAME'
REDDIT_PASSWORD = 'YOUR_PASSWORD'
﻿
﻿
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
﻿
# Init Reddit
reddit = praw.Reddit(
    client_id=REDDIT_CLIENT_ID,
    client_secret=REDDIT_CLIENT_SECRET,
    user_agent=REDDIT_USER_AGENT,
    username=REDDIT_USERNAME,
    password=REDDIT_PASSWORD
)
﻿
@weave.op 
async def get_subreddits(query, num=2):
    """Get relevant subreddits for a query"""
    prompt = f"""List {num} relevant subreddits where people would discuss and answer the following question:
﻿
{query}
﻿
Return only the subreddit names, no hashtags or explanations."""
    res = await acompletion(
        model="gpt-4o-mini",
        api_key=OPENAI_API_KEY,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.5,
        max_tokens=200,
    )
    content = res["choices"][0]["message"]["content"]
    return [
        re.sub(r"^\d+[\.\)]\s*", "", line.strip().replace("r/", "").replace("/r/", ""))
        for line in content.splitlines() if line.strip()
    ]
﻿
@weave.op 
async def summarize_post(post_title, post_body):
    """Summarize a post into 30-50 words"""
    if len(post_body) < 200:  # If post is already short, no need to summarize
        return post_title + " " + post_body
﻿
    prompt = f"""Summarize the following Reddit post in 30-50 words, capturing the main points and sentiment:
﻿
Title: {post_title}
Body: {post_body}
﻿
Provide ONLY the summary."""
    
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0.3,
            max_tokens=100,
        )
        
        summary = res["choices"][0]["message"]["content"].strip()
        return summary
    except Exception as e:
        print(f"Error summarizing post: {str(e)}")
        # Fallback: just use the title and first 40 words of body
        words = post_body.split()
        short_body = " ".join(words[:40]) + ("..." if len(words) > 40 else "")
        return post_title + " " + short_body
﻿
﻿
﻿
@weave.op 
async def generate_search_terms(query):
    """Generate effective search terms from the query"""
    prompt = f"""Generate 3 effective Reddit search terms for finding discussions about this question:
﻿
{query}
﻿
The terms should be effective for Reddit's search function (keywords, not full sentences).
Return only the search terms, one per line, no numbering or explanations."""
    
    res = await acompletion(
        model="gpt-4o-mini",
        api_key=OPENAI_API_KEY,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7,
        max_tokens=100,
    )
    content = res["choices"][0]["message"]["content"]
    return [line.strip() for line in content.splitlines() if line.strip()]
﻿
async def assess_post_relevance(post_title, post_body, topic):
    """Determine if a post is relevant to the given topic"""
    prompt = f"""Assess if the following Reddit post is relevant to this topic: "{topic}"
﻿
Post title: "{post_title}"
Post body: "{post_body}"
﻿
Respond with a number between 0 and 1 indicating relevance:
0 = Not at all relevant to the topic
0.5 = Somewhat relevant
1 = Highly relevant to the topic
﻿
Respond with only a number between 0 and 1."""
    
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=10,
        )
        
        score_text = res["choices"][0]["message"]["content"].strip()
        try:
            score = float(score_text)
            return score
        except ValueError:
            # If we can't parse as float, make a rough estimate based on text
            if "1" in score_text:
                return 1.0
            elif "0.5" in score_text:
                return 0.5
            else:
                return 0.0
    except Exception:
        return 0.5  # Default to maybe relevant on error
﻿
async def fetch_threads(subs, topic, limit=5, comments_per_post=7):
    """Fetch threads related to the topic, filtering for relevance"""
    results = []
    processed_urls = set()  # Track processed posts to avoid duplicates
    
    # Generate search terms from the topic
    search_terms = await generate_search_terms(topic)
    print(f"Search terms generated: {search_terms}")
    
    # Extract keywords for fallback
    topic_keywords = topic.lower().strip()
    search_terms.append(topic_keywords)
    
    for sub in subs:
        try:
            for term in search_terms:
                print(f"Searching r/{sub} for: {term}")
                
                for post in reddit.subreddit(sub).search(term, limit=limit, sort='relevance'):
                    # Skip if we've already processed this post
                    if post.permalink in processed_urls:
                        continue
                    processed_urls.add(post.permalink)
                    
                    # Check if post is relevant to the topic
                    relevance_score = await assess_post_relevance(post.title, post.selftext, topic)
                    if relevance_score < 0.6:  # Threshold for relevance
                        print(f"Skipping irrelevant post: {post.title} (relevance: {relevance_score:.2f})")
                        continue
                    
                    # Generate a concise summary of the post
                    post_summary = await summarize_post(post.title, post.selftext)
                    
                    post.comments.replace_more(limit=0)
                    comments = []
                    
                    for c in post.comments[:comments_per_post]:
                        comments.append({
                            "text": c.body,
                            "score": c.score
                        })
                        
                    if comments:
                        results.append({
                            "subreddit": sub,
                            "title": post.title,
                            "body": post.selftext,
                            "summary": post_summary,
                            "url": f"https://www.reddit.com{post.permalink}",
                            "post_score": post.score,
                            "comments": comments,
                            "relevance_score": relevance_score
                        })
                        print(f"Added relevant post: {post.title} (relevance: {relevance_score:.2f})")
                        print(f"Summary: {post_summary}\n")
        except Exception as e:
            print(f"Error processing subreddit {sub}: {str(e)}")
            continue
            
    return results
﻿
﻿
@weave.op 
async def analyze_post_sentiment(post, query):
    """Analyze post sentiment towards a specific topic"""
    # Use the summary instead of the full body if available
    if "summary" in post and post["summary"]:
        combined_text = post["summary"]
    else:
        combined_text = f"Title: {post['title']}\nBody: {post['body']}"
    
    # If text or query is missing, return Neutral
    if not (combined_text and query):
        return "Neutral"
    
    prompt = f"""You are a sentiment classifier analyzing a Reddit post to determine opinions about a specific topic.
﻿
Topic: "{query}"
Post: "{combined_text}"
﻿
Classify the sentiment ONLY if the post directly discusses the topic:
﻿
- "Good": The post clearly expresses positive sentiment about the topic
- "Bad": The post clearly expresses negative sentiment about the topic 
- "Neutral": The post does not clearly discuss the topic, expresses mixed feelings, or the relevance to the topic is unclear
﻿
If you're uncertain whether the post is directly relevant to the topic, classify as "Neutral".
﻿
Respond with exactly one word - either "Good", "Bad", or "Neutral"."""
    
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=10,
        )
        
        label = res["choices"][0]["message"]["content"].strip()
        # Normalize output to ensure we get one of our three labels
        if "good" in label.lower():
            return "Good"
        elif "bad" in label.lower():
            return "Bad"
        else:
            return "Neutral"
    except Exception:
        return "Neutral"  # Return Neutral on any error
﻿
﻿
﻿
@weave.op 
async def analyze_comment_sentiment(comment, post_context, query):
    """Analyze comment sentiment towards a specific topic with post context"""
    # Use the summary instead of the full post content if available
    if "summary" in post_context and post_context["summary"]:
        post_context_text = post_context["summary"]
    else:
        post_context_text = f"Post title: {post_context['title']}\nPost body: {post_context['body']}"
    
    # If any required parameter is missing, return Neutral
    if not (comment["text"] and post_context_text and query):
        return "Neutral"
    
    prompt = f"""You are a sentiment classifier analyzing a Reddit comment to determine opinions about a specific topic. Dont classify the post, ONLY the comment
﻿
Topic: "{query}"
Original Post Context: "{post_context_text}"
The Comment to classify: "{comment["text"]}"
﻿
Classify the sentiment ONLY if the comment directly discusses the topic:
﻿
- "Good": The comment clearly expresses positive sentiment about the topic
- "Bad": The comment clearly expresses negative sentiment about the topic 
- "Neutral": The comment does not clearly discuss the topic, expresses mixed feelings, or the relevance to the topic is unclear
﻿
If you're uncertain whether the comment is directly relevant to the topic, classify as "Neutral".
﻿
Respond with exactly one word - either "Good", "Bad", or "Neutral"."""
    
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=10,
        )
        
        label = res["choices"][0]["message"]["content"].strip()
        # Normalize output to ensure we get one of our three labels
        if "good" in label.lower():
            return "Good"
        elif "bad" in label.lower():
            return "Bad"
        else:
            return "Neutral"
    except Exception:
        return "Neutral"  # Return Neutral on any error
﻿
async def main():
    query = input("Enter a topic to analyze sentiment on Reddit (e.g., 'ChatGPT for coding'): ")
    print(f"Topic: {query}")
    
    print("Finding relevant subreddits...")
    subreddits = await get_subreddits(query)
    print(f"Subreddits found: {subreddits}")
    
    print("Fetching and filtering relevant threads...")
    threads = await fetch_threads(subreddits, query)
    print(f"Found {len(threads)} relevant threads\n")
    
    if not threads:
        print("No relevant threads found. Try a different topic.")
        return
    
    # Track all sentiment labels
    post_labels = []
    comment_labels = []
    all_labels = []
    
    # Analyze each thread
    for thread in threads:
        print(f"\nAnalyzing: {thread['title']} [Score: {thread['post_score']}, Relevance: {thread.get('relevance_score', 'N/A')}]")
        print(f" → {thread['url']}")
        
        # Analyze post sentiment
        post_sentiment = await analyze_post_sentiment(thread, query)
        post_labels.append(post_sentiment)
        all_labels.append(post_sentiment)
        
        print(f"Post sentiment about topic: [{post_sentiment}]")
        
        # Post summary
        print(f"Post summary: {thread.get('summary', 'N/A')}")
        
        # Analyze comments
        print("\nComments:")
        for c in thread["comments"]:
            comment_sentiment = await analyze_comment_sentiment(c, thread, query)
            comment_labels.append(comment_sentiment)
            all_labels.append(comment_sentiment)
            
            # Comment preview
            text_preview = c["text"][:100] + "..." if len(c["text"]) > 100 else c["text"]
            print(f"[{comment_sentiment}] (score: {c['score']}) {text_preview}")
    
    # Generate sentiment summaries
    post_count = Counter(post_labels)
    comment_count = Counter(comment_labels)
    overall_count = Counter(all_labels)
    
    print("\n" + "="*50)
    print(f"SENTIMENT ANALYSIS FOR: {query}")
    print("="*50)
    
    print("\nPost Sentiment:")
    for k in ["Good", "Neutral", "Bad"]:
        pct = (post_count.get(k, 0) / max(len(post_labels), 1)) * 100
        print(f"  {k}: {post_count.get(k, 0)} ({pct:.1f}%)")
    
    print("\nComment Sentiment:")
    for k in ["Good", "Neutral", "Bad"]:
        pct = (comment_count.get(k, 0) / max(len(comment_labels), 1)) * 100
        print(f"  {k}: {comment_count.get(k, 0)} ({pct:.1f}%)")
    
    print("\nOverall Sentiment:")
    for k in ["Good", "Neutral", "Bad"]:
        pct = (overall_count.get(k, 0) / max(len(all_labels), 1)) * 100
        print(f"  {k}: {overall_count.get(k, 0)} ({pct:.1f}%)")
    
    # Calculate relevant sentiment (excluding neutral)
    relevant_counts = {k: v for k, v in overall_count.items() if k != "Neutral"}
    if relevant_counts:
        dominant_sentiment = max(relevant_counts.items(), key=lambda x: x[1])[0]
        dominant_count = relevant_counts[dominant_sentiment]
        total_relevant = sum(relevant_counts.values())
        confidence = (dominant_count / total_relevant) * 100
        
        print(f"\nOverall sentiment about '{query}': {dominant_sentiment}")
        print(f"Confidence: {confidence:.1f}% (based on {total_relevant} relevant opinions)")
    else:
        print(f"\nNot enough relevant opinions about '{query}' were found.")
    
    # Log to Weights & Biases
﻿
﻿
if __name__ == "__main__":
    asyncio.run(main())
With the full script in place, the full pipeline runs from topic input to sentiment breakdown. It starts by using GPT-4o-mini to suggest subreddits where the topic is likely being discussed. From there, it generates search terms, pulls posts from each subreddit, and filters out anything irrelevant using a relevance score generated by the model.
Relevant posts are summarized if they’re long, then analyzed for sentiment. The script also classifies a set number of top-level comments—these are the direct replies to the post, not replies to other comments. This keeps the analysis focused on the core discussion rather than every side thread.
Each post and comment is labeled as Good, Bad, or Neutral depending on how clearly it expresses an opinion about the topic. At the end, the script prints a sentiment breakdown across all posts and comments, along with a confidence score showing how dominant the leading sentiment is.
﻿
﻿
To fix this, I added post summarization as a preprocessing step. Instead of passing the full post body as context, the script summarizes each post down to 30–50 words. This stripped out unnecessary noise while still preserving enough context for the classifier to understand what the comment was reacting to: 
async def summarize_post(post_title, post_body):
    if len(post_body) < 200:
        return post_title + " " + post_body
﻿
    prompt = f"""Summarize the following Reddit post in 30-50 words, capturing the main points and sentiment:
﻿
Title: {post_title}
Body: {post_body}
﻿
Provide ONLY the summary."""
    
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0.3,
            max_tokens=100,
        )
        
        summary = res["choices"][0]["message"]["content"].strip()
        return summary
    except Exception as e:
        words = post_body.split()
        short_body = " ".join(words[:40]) + ("..." if len(words) > 40 else "")
        return post_title + " " + short_body
Summarizing the posts first makes the overall pipeline faster and improves model performance. Short summaries give the model cleaner, more focused context while preserving meaning. If the post is already short, summarization is skipped and the raw text is used directly. 
Analyzing Sentiment with GPT-4o-mini For analyzing the actual sentiment, the script uses prompt-based classification via GPT-4o-mini. This is done separately for both posts and comments. The model is instructed to respond with only one of three possible labels—Good, Bad, or Neutral—based on whether the text expresses a clear opinion about the user-provided topic.
For posts, the classifier is given the summary (or raw text if short) along with the topic. The prompt explicitly tells the model to only classify the sentiment if the post clearly discusses the topic. If it’s unclear or unrelated, it should default to “Neutral.” This avoids false positives from vague or off-topic content that might otherwise be misclassified as opinionated.
async def analyze_post_sentiment(post, query):
    if "summary" in post and post["summary"]:
        combined_text = post["summary"]
    else:
        combined_text = f"Title: {post['title']}\nBody: {post['body']}"
﻿
    if not (combined_text and query):
        return "Neutral"
﻿
    prompt = f"""You are a sentiment classifier analyzing a Reddit post to determine opinions about a specific topic.
﻿
Topic: "{query}"
Post: "{combined_text}"
﻿
Classify the sentiment ONLY if the post directly discusses the topic:
﻿
- "Good": The post clearly expresses positive sentiment about the topic
- "Bad": The post clearly expresses negative sentiment about the topic 
- "Neutral": The post does not clearly discuss the topic, expresses mixed feelings, or the relevance to the topic is unclear
﻿
If you're uncertain whether the post is directly relevant to the topic, classify as "Neutral".
﻿
Respond with exactly one word - either "Good", "Bad", or "Neutral"."""
﻿
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=10,
        )
        label = res["choices"][0]["message"]["content"].strip()
        if "good" in label.lower():
            return "Good"
        elif "bad" in label.lower():
            return "Bad"
        else:
            return "Neutral"
    except Exception:
        return "Neutral"
This function builds a prompt using either the post’s summary or its full text and asks the model to classify sentiment only if the post clearly discusses the topic. If it’s off-topic, vague, or unrelated, it gets labeled as “Neutral.” This prevents the model from over-interpreting generic Reddit content.
For comments, the analysis works similarly but adds a layer of context: the sentiment classifier sees both the comment and the summary of the original post it’s replying to: 
async def analyze_comment_sentiment(comment, post_context, query):
    if "summary" in post_context and post_context["summary"]:
        post_context_text = post_context["summary"]
    else:
        post_context_text = f"Post title: {post_context['title']}\nPost body: {post_context['body']}"
﻿
    if not (comment["text"] and post_context_text and query):
        return "Neutral"
﻿
    prompt = f"""You are a sentiment classifier analyzing a Reddit comment to determine opinions about a specific topic. Don't classify the post, ONLY the comment
﻿
Topic: "{query}"
Original Post Context: "{post_context_text}"
The Comment to classify: "{comment["text"]}"
﻿
Classify the sentiment ONLY if the comment directly discusses the topic:
﻿
- "Good": The comment clearly expresses positive sentiment about the topic
- "Bad": The comment clearly expresses negative sentiment about the topic 
- "Neutral": The comment does not clearly discuss the topic, expresses mixed feelings, or the relevance to the topic is unclear
﻿
If you're uncertain whether the comment is directly relevant to the topic, classify as "Neutral".
﻿
Respond with exactly one word - either "Good", "Bad", or "Neutral"."""
﻿
    try:
        res = await acompletion(
            model="gpt-4o-mini",
            api_key=OPENAI_API_KEY,
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=10,
        )
        
        label = res["choices"][0]["message"]["content"].strip()
        if "good" in label.lower():
            return "Good"
        elif "bad" in label.lower():
            return "Bad"
        else:
            return "Neutral"
    except Exception:
        return "Neutral"
After running the script, we will see the results printed to the console: 
﻿
Tracking sentiment analysis with W&B Weave Weave is used throughout the script to monitor how the app is performing during execution. Each call to the model, whether it’s for subreddit discovery, summarization, relevance scoring, or sentiment classification, is automatically logged. This makes it easy to inspect inputs and outputs for any step and trace back where things might be going wrong.
During development, Weave was especially useful for debugging relevance scoring and sentiment classification. By looking at how specific prompts were handled, I could quickly catch cases where the model was misclassifying off-topic posts or being influenced by irrelevant parts of the input. This helped tune prompt phrasing to get more consistent outputs. 
Inside Weave, it's easy to filter by each function, and quickly analyze each input and output to our model. This ultimately serves as a LLM "debugger" that gives us a full visualization over how the model is performing: 
﻿
﻿
After filtering by a given function, I can let see exactly how each function is performing: 
﻿
Weave isn’t just useful during development—it gives you visibility when your application is in production. You can inspect every prompt,  model response, and decision the app makes in real time. That means when something looks off, you don’t have to guess—you can trace exactly what the model saw and why it responded the way it did. 
Real-world applications and use casesSentiment analysis has clear value across real-world use cases. For companies, tracking Reddit sentiment can help surface early signs of brand issues, customer frustration, or PR risks before they escalate. Teams can monitor how users react to feature launches, pricing changes, or unexpected outages by analyzing the tone of organic discussion.
It’s also useful for product feedback. Reddit threads often contain honest, detailed opinions that don’t show up in traditional user surveys. By classifying sentiment across those threads, teams can identify trends in praise or criticism, helping them prioritize improvements or understand how a launch is being received.
But it’s worth noting that Reddit content comes with baked-in biases. The platform skews toward certain communities and user types, and sentiment from Reddit won’t always match broader public opinion. Insights drawn from this kind of analysis should be treated as one perspective—not a complete picture.
Conclusion Reddit is one of the most active and diverse platforms for real discussion online. People use it to share experiences, offer feedback, ask questions, and debate ideas across nearly every topic imaginable. With the right approach, that content becomes a powerful source of insight.
By combining GPT-4o-mini with relevance filtering, summarization, and sentiment classification, this pipeline makes it possible to turn Reddit threads into structured data. You can quickly understand how users are reacting to a topic—whether it’s a product, service, or idea—based on the conversations they’re already having. While the results reflect the biases of the platform’s user base, they offer a fast and flexible way to gauge public response across different communities.
﻿
Add a comment
Tags: Articles, Sentiment Analysis, Tutorial, Weave, Evaluations, Agents
Iterate on AI agents and models faster. Try Weights & Biases today.