March 4, 2019

A Critical Assessment of Open Science

A Critical Assessment of Open Science

We are creating a framework for discussing open science and we need your feedback


By Lara Mangravite and John Wilbanks

At Sage Bionetworks, we think a lot about open science. Our organization was founded explicitly to use open practices to promote integration of large-scale data analytics into the life sciences. We were guided by a very specific definition of open science: the idea that scientific “teams of teams” working together on a growing commons of open data can unleash substantial increases in scientific throughput and capacity.

We have learned a lot over the past 10 years regarding best practices for successful application of open science in this context. We are curious to understand how our observations may overlap with those from others in the field. Is there a common set of guidelines that can help support the effective use of openness in the life sciences? On the flip side, is there a set of common mistakes that keep getting repeated? We quickly realized in our work, for example, that each project has a window of time where “openness” is optimally effective. Early in a project, openness can sometimes hinder creativity as people hold back on sharing ideas that are still immature.

With this in mind, we decided to convene a few collaborators for a small workshop to critically assess openness in the life sciences. The goal wasn’t to speak for the field or to declare consensus – we know that we have an unrepresentative sample and can speak no universal truths. The goal was to get candid and clear feedback from this initial group about what’s working and what’s not.

What did we learn?

  1. Open science is a general term that is used to represent many different ideas. While most open-science approaches are based on a common premise that sharing, transparency, and/or collaboration will lead to better science, there is neither a universal definition of “open” nor a universal way to apply “open approaches.” Diverse groups use the term to represent a wide variety of activities designed to achieve distinct goals. These are not well distinguished in the language used to describe them, and this can lead to confusion amongst open-science proponents who don’t always feel represented in each other’s ideas. It becomes very difficult to define best practices or to evaluate the success of the open science movement as a whole.
  2. Open science is not an isolated movement. These open approaches are but one part of a much larger scientific – and social – ecosystem. They don’t operate in isolation. Neither should we. We will work with the workshop attendees to provide a full briefing of this meeting – to formally publish as an open-access paper later this year.

In the meantime, we’d like to use this channel to work through some of these topics with a larger group of people. We’re going to think critically about what we mean when we talk about open science in the life sciences and we urge you to engage with us – tell us what you see, how it’s working, what could be done better. We hope (and expect) you to challenge us so that together we can get to a more cohesive and compelling way of talking about open science.

We will post here regularly as we develop our thinking. Some workshop attendees will contribute guest posts, drawing out themes that are important to their work, such as reuse, transparency, collaboration, policy implications, and more. We also want to know your perspectives on open science and encourage you to write a blog post and publish on your preferred platform. Tweet us the link to your post, and stay tuned as we continue this conversation.

About this series: In February 2019, Sage Bionetworks hosted a workshop called Critical Assessment of Open Science (CAOS). This series of blog posts by some of the participants delves into a few of the themes that drove discussions – and debates – during the workshop.

Voices from this series:

Read here or on

Introduction by Lara Mangravite and John Wilbanks

Voices From the Open Science Movement by Lara Mangravite and John Wilbanks

Recognizing the Successes of Open Science by Titus Brown

Open Science is About the Outcomes, Content, and Process of Research by Brian Nosek

Do Scientists Reuse Open Data? by Irene Pasquetto

Open Science: To What End? by Cyndi Grossman