Methods and Data Sources

Methods

This page documents the methodology behind the project and provides access to the underlying data. The analysis draws on public discussion from Reddit to examine how Punjabi and South Asian communities talk about identity, language, heritage, the memory of Partition, and belonging across the India–Pakistan border and the global diaspora. All data collection, cleaning, and visualization was carried out in Python using a Google Colab notebook, which made it possible to run the entire workflow in the browser without installing software locally and to keep the code, data, and outputs together in one reproducible document.

Tools and Environment

The project was built entirely in Google Colab, a free cloud-based notebook environment that runs Python. Working in Colab allowed the analysis to be developed step by step, with each block of code producing visible output that could be checked before moving on. The main Python libraries used were PRAW (the Python Reddit API Wrapper) for collecting posts and comments, pandas for organizing the text into structured tables, the re and nltk libraries for cleaning and filtering the text, and wordcloud together with matplotlib for generating and displaying the final visualizations.

Data Collection

Reddit data was gathered programmatically through the Reddit API using the PRAW library in Colab. After authenticating with API credentials, the notebook queried threads connected to five recurring themes: the diaspora experience, Lahore and regional heritage, language, the memory of Partition, and shared or contested identity. For each theme, the relevant submissions and their comment threads were pulled, and the text of each post and comment was extracted and stored in a pandas DataFrame. Each row recorded the text along with basic metadata so the dataset could later be exported to a spreadsheet for review.

Data Cleaning and Processing

Once collected, the text was cleaned inside the Colab notebook using Python. Regular expressions (the re library) were used to strip out URLs, punctuation, numbers, and other non-alphabetic characters, and all text was converted to lowercase so that the same word would not be counted twice. Common “stop words” (such as the, and, or is) were removed using a standard stop-word list, supplemented with a few custom words specific to the dataset that added noise without meaning. The cleaned words were then tokenized and counted by frequency using pandas, producing a ranked list of the most common terms for each theme.

Visualization

The frequency results were turned into word clouds using Python’s wordcloud library and rendered with matplotlib. In each word cloud the size of a word reflects how often it appeared in that theme’s discussion: the larger the word, the more central it was to the conversation. A separate color scheme was applied to each theme to make the five clouds visually distinct, and the finished images were exported as PNG files from Colab and uploaded to this site.

Limitations

Reddit data reflects only the views of users who choose to post and is not a representative sample of any community. Word frequency captures what is discussed most often but not tone, sarcasm, or context, so each visualization should be read alongside its written interpretation. API rate limits and the structure of Reddit threads also mean the dataset is a sample of the available discussion rather than a complete record.

Data and Spreadsheets

The spreadsheets below contain the raw and processed data exported from the Colab notebook. c