C19 Preprint Resolution Details

By Charles Twardy

Review In late 2020 we asked forecasters to predict the fate of 400 popular COVID-19 preprints posted to bioRxiv and medRxiv between January and September of that year. For each we asked: Would it be published within a year of posting, and if so, would it be in a high-impact (JIF≥10) journal?  [No / High / Low]…

12-Nov: Almost…

By Charles Twardy

We expected to announce prizes this week, but co-pilot efforts just ran slowly and after verifying Rounds 1-11, today we found a small discrepancy in our Round 0 (meta-claims) calculations that wasn’t sorted by end of day. This will certainly be embarrassingly obvious with a fresh look on Monday. In the meantime, we can say…

OCT-2021 UPDATE

By Charles Twardy

SCORE Markets (Rounds 1-10, plus 0 & 11): I waited as long as possible to maximize the number of completed replications, but we now have about two months to resolve claims, pay prizes, and wrap up. There are now max 88 qualifying claims: better than in February, but less than we would like. It is…

Status Update

By Charles Twardy

Hello forecasters! A quick status update Survey prizes paid out as planned, but we’re still awaiting final December resolutions to let us pay out the main SCORE markets. COS says we should have it shortly. There will be far fewer resolutions than expected, but we cannot wait a year for pandemic-halted experiments. Updated what-to-expect:• We…

What a Year at Replication Markets!

By Charles Twardy

Dear Forecasters: Wow!  What a year it’s been! Did you know you completed 54,000 surveys and 42,000 trades on 3,000 studies?  Plus the current COVID-19 Round 11? That’s stunning. Replication Markets Recap: The View From 3,000 Claims The forecasts are in. The results are not. But there’s quite a bit we can say – if you don’t mind preliminary results.…

Round 8 Fraud Investigation Summary

By Charles Twardy

Happily, we have seen little evidence of fraud during RM. However, due diligence requires we check at least monthly, and in Round 8 we disqualified 11 accounts for improper collaboration. We cannot offer details, but we can say several would have won survey prizes, and that warning flags included clear ties to a single established…

Exploring the new trade widget

By Charles Twardy

The new trade widget has the same functionality as the one you are familiar with from Rounds 1-5, with an added dimension.  You will have a better sense of the point cost/gain of a trade as you adjust the slider. The balloon value shows the adjusted probability of replication; the red curves show how many…

Science is Hard

By Charles Twardy

Cross-posted from RM’s PI’s blog. Nature Communications has an inspiring piece about a DARPA project baking replication into the original study. The DARPA Biological Control program applies independent validation & verification (IV&V) to synthetic biology. The article is a Lessons Learned piece. Although DARPA oversight presumably mitigated the element of competition, the IV&V teams had at least…

Changes to Market Payouts

By Charles Twardy

SCORE’s change in definition of “replication” (see our blog post on this topic) more than doubled the number of claims that will resolve in Round 6 and later. This has affected our project’s budget. We have adjusted market payouts as described below. However, a change to incentives has to be approved by our Institutional Review…

Changes in Round 6

By Charles Twardy

We have big changes to announce!  Read to learn about… simpler definition of replication more markets paying out simpler surveys Simpler Definition of Replication SCORE has adopted a simpler, unified definition of replication: Replication is testing the same claims using data that was not used in the original study. That required some changes from us.…

We’re sorry to see you go! Please visit our social media sites.

This site uses cookies to provide you with a better browsing experience.

Visit our Privacy Policy for more information.