---

Open Source and You- How James Pustejovsky Increases Research Accessibility Through Open Source

While presenting research, it is important that the audience understands what they are reading. No one knows this better than James Pustejovsky, a statistician and associate professor in the University of Wisconsin-Madison department of Educational Psychology. Pustejovsky advocates for transparency in research through his development of open source software. Every part of the research process, says Pustejovsky, should be accessible for those interested.

Graduating from Boston College with an undergraduate degree in economics, Pustejovsky had a passion for working with quantitative data. After working in the economics field for a few years, he returned to school in pursuit of a PhD. Connecting with an advisor that analyzed education statistics, Pustejovsky was motivated to enter the same field.

“I don’t want to think of it as stats just for educational psychology. The work we do is relevant to education. It’s relevant to psychology. It’s relevant to sociology… this department is an interesting place to be because you get all of these different disciplinary perspectives that converge on thinking about the same problems.”

Finishing his PhD in 2013, Pustejovsky entered the academic world in a time where research transparency was less of a norm- a period commonly known as the replication crisis. 

“The way academics had been doing quantitative empirical work in psychology had substantial flaws. Big research findings were not replicable…when researchers attempted to replicate something, it didn’t work. It was a big crisis that spread throughout the social sciences.”

The answer to the crisis was transparency. Without transparency, the research being done could not be analyzed for potential flaws. As the academic world began to grapple with the issue, open source became an obvious solution.

“The norm is increasingly that we should share the full data sets we’ve collected and the code we’ve created. R has become a really big language for sharing that sort of code…in R, code can be easily shared and accessed by pretty much anyone. That’s why I have devoted so much effort to open source software development, because it facilitates efforts to improve the transparency of empirical research.”

Why it took so long for the research world to realize the importance of transparency, Pustejovsky does not know. 

“My youngest is in third grade. Right now, she is learning how to collect data and tell people about it. Being transparent about your process and sharing the results in an open way is the basic norm of science. Making things more open is just common sense!”

Pustejovsky and his fellow PhD candidates relied on a closed source program for their research. While the program was efficient, its $100 price tag was a high bar for entry. When he found the free, open source R program, Pustejovsky immediately made the switch. 

“Getting introduced to the world of R was just kind of mind-blowing for me. It was free, and I had some previous coding experience, so I really liked it.”

Pustejovsky’s enthusiasm for open source programming and the R coding language only continues to grow. His most notable project is clubSandwich, a meta-analytic statistical model. 

“One of my grad school friends and I both do a lot of work in an area called meta-analysis. Meta-analysis is all about taking the results from a bunch of different studies and kind of putting them together to make sense of the sort of broader patterns.”

The clubSandwich package focuses on a specific kind of meta-analysis called robust variance estimation. 

“The formula for calculating robust variance estimations looks like a sideways sandwich…so most statisticians call it a sandwich estimator.”

As his colleague, Beth Tipton, furthered her work on sandwich estimators, Pustejovsky realized he could program them. His realization inspired the clubSandwich project, an entirely open source R package.

“I was looking at some stuff [Beth Tipton] did and I was like, oh, I think I can write some code that will do that automatically. We started by thinking about these models that are used for meta-analysis. Eventually, we realized the techniques that we were developing applied a lot more broadly. Not just in meta-analysis, but into a lot of other types of models that are used in different parts of statistics.”

Pustejovsky knows that clubSandwich is a complicated statistical system. That’s why he is taking steps to make the program easier to understand, increasing its accessibility. He recently enlisted the help of an OSPO intern to program teaching materials. 

“We’ve actually received grant funding to help develop educational materials that go along with the packages. As part of that, I’ve realized there’s some tweaks we can make to the software itself. We can tweak the software to make it easier to explain to users.”

Pustejovsky has been sure to open his research to interested students. In the Spring of 2025, the R expert became a mentor for the Open Source Program Office. He works with interns on the clubSandwich project. There is no better way for a student to gain experience in open source, Pustejovsky insists, than to let them work directly on a project.

“I haven’t had a lot of opportunities to work with undergraduates. It’s great because I have learned a lot from them. I’m at the point now where I’m more interested in figuring out how I can do some regular classes for undergraduate students.”

Pustejovsky is excited to see open source grow in the coming years. The technology has built a great community, especially on the University of Wisconsin-Madison campus. As transparency and accessibility increase, so too will the presence of open source beyond UW’s campus. 

“I have people contacting me from all over the world with questions about some of my work, especially around the software. In some cases, they come with questions that I don’t know the answer to. It’s created some really interesting methodological questions. These conversations have spurred more methods research, given me new project ideas, and reinforced the value of open source to me.”