Two Wrongs

Quarterly Cup 2024 Q1 Retrospective

Quarterly Cup 2024 Q1 Retrospective

Alternate title: the quarter I found out I suck at forecasting.

I have continued participating in the Metaculus Quarterly Cup, and as in the previous two quarters, I will summarise my experience and lessons.

Just the numbers

My average blind1 As in without seeing the community prediction. Brier score2 A lower Brier score is better, and the range people practically fall in is something like 0.07 (best) to 0.25 (random guessing.) was 0.25 – which is about the worst it could be!

Part of this is probably overconfidence from doing so well the previous tournament, and maybe this tournament was a little harder than the past two? I’m not sure.

What I did differently

I’m not going to share the rest of the results because it was just bad. It doesn’t matter exactly how bad. But! Aside from hubris, there were reasons. I did two things differently this quarter compared to the previous two.

  1. I did not look at the community prediction.
  2. I did not update very actively.

Basically, I wanted to participate in this tournament in hardcore mode, to see how much of my past result was actual forecasting skill, and how much of it was that I had learned to smooth out the noise of the community forecast. It turns out a large portion of it was about smoothing out the community forecast!

So now I have a new challenge ahead of me: learn forecasting again, this time without the crutches. I have a long way to go.