Changes in Round 6

We have big changes to announce!  Read to learn about…

  • simpler definition of replication
  • more markets paying out
  • simpler surveys

Simpler Definition of Replication

SCORE has adopted a simpler, unified definition of replication:

Replication is testing the same claims using data that was not used in the original study.

That required some changes from us. Starting in Round 6, Replication Markets will no longer distinguish between “data replication” and “direct replication.” 

Studies selected for replication will use the most appropriate type, and all will count for market resolutions.

More Markets Pay Out!

The new definition of replication increases the expected market replications from about 50 in Rounds 1-5 to about 125 in Rounds 6-10.

Simpler Surveys

Survey-Takers, rejoice! The new definition also means we can simplify our surveys! 

Surveys now have only 4 questions (rather than 7). In essence, they ask:

  1. What is the probability that a high-power replication would succeed?
  2. What fraction of participants will give an estimate larger than 50% on the previous question?
  3. How plausible is this claim?
  4. Is there anything else we should know?
The shorter surveys are back to a single page. And we updated to a cleaner, journal-like format (see right).
redacted claim
New claim format

With the change, the other three questions became redundant. We considered keeping them anyway, but they were providing little information: Old Q3 was 93% correlated with old Q1, with a slope of 1. (We will make this data available to explore after claims resolve.)

2 thoughts on “Changes in Round 6”

Leave a Reply to Changes to Market Payouts – Replication Markets Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Want to receive updates about Replication Markets? Share your contact information below.

We’re sorry to see you go! Please visit our social media sites.

This site uses cookies to provide you with a better browsing experience.

Visit our Privacy Policy for more information.