Business Analytics · Calibrated forecasting market

📈 How to play Signal or Noise

You are on the analytics team of a Dubai mall, forecasting each weekend as honest probabilities and paying for data that might help.

Join a game Run it as facilitator Print this guide

Your goal

Score as many forecast points as you can over three weekends. Points come from the Brier score, which rewards probabilities that match how often things really happen. Bold calls that miss cost far more than careful ones, and data only pays if it moves your forecast.

You are one forecaster or forecasting team for Marasi Grand Mall. Everyone forecasts the same weekend. You compete with the other forecasters and with the crowd: the average of everyone's forecasts.

Join in 30 seconds

Scan the QR code on the big screen with your phone camera, or open the game page and type the 4-letter code.
Type your first name. No app, no account.
Join a team or create one and name it. Your teammates get a role each. In individual mode you run your own forecaster.
If your phone locks or the page reloads, open it again: you return to the same seat.

Team roles

Lead: Chairs the team and submits the final decision. If the Lead's phone dies, any teammate can take over. In this game: Turn the team's discussion into one probability split per question.
Analyst: Owns the numbers and the team's prediction for the round. In this game: Read the feeds and estimate footfall with the equation on the model card.
Treasurer: Watches the budget and costs before the team commits. In this game: Data costs points: buy a feed only if it can move your forecast.
Risk and ethics officer: Flags harms and pushes back before the team commits. In this game: Watch for overconfidence: a 90% that misses costs far more than a 50%.
Voice: Talks to other teams and presents the team's story at the debrief. In this game: Compare your split with the crowd's after each reveal and explain any gap.

The three rounds

Each round you predict your own result, make your decisions within about 8 minutes, and then the model reveals what happened. An event card lands between rounds.

1
Weekend 1 · Season opener
Everyone gets the free weather forecast and events calendar. You forecast footfall, best promotion and car park. Any data you buy now arrives next weekend.
Watch for: Compare your probability on each outcome with the crowd's. Note which question cost you the most points.
2
Weekend 2 · Festival weekend
Revenue replaces the promotion question. The free pack is gone: you see only the feeds you bought. Last weekend's outcomes and the crowd's forecast are on screen.
Watch for: Check whether your bought data improved your score and whether your forecast moved the way the data pointed.
3
Weekend 3 · Final weekend
The data shop is closed; you use the feeds bought in weekend 2. You forecast footfall, best promotion and revenue, and pick one question to count double.
Watch for: See whether your double-weight question paid off. Compare the crowd with the best individual across all three weekends.

Your decisions

What each control on your phone does, and how to think about it.

Footfall next weekend	Every weekend	Spread 100 points over four bands: under 105k, 105k–120k, 120k–135k, over 135k. Points are your probabilities. Unassigned points count as 0%. Use the model card: start from base footfall, add heat and events, subtract rain. Put most weight near your estimate, some on the neighbours.
Promotion with the biggest sales lift	Weekends 1 and 3	Spread 100 points over the gold souk raffle, the kids' zone and the food court 2-for-1. The winner is whichever lifts sales most versus a normal weekend. Think about which promotion weather and events would help. Last year's lifts and social buzz are evidence, but noisy.
When the main car park fills	Weekends 1–2	Spread 100 points over full before 7 pm, full 7–8 pm, or not full by 8 pm. More visitors, events and rain make it fill earlier. Link it to your footfall forecast. If you expect a crowd, shift weight towards an earlier fill.
Weekend retail revenue	Weekends 2–3	Spread 100 points over four bands from under AED 21m to over AED 29m. Revenue is footfall times spend per visitor; events raise spend and rain lowers it. Build it from your footfall view. Uncertainty in footfall carries straight into revenue, so keep the split honest.
Buy data for next weekend	Weekends 1–2	Up to 3 feeds within an 8-point budget: weather 2, events 3, last-year 1, social buzz 2, regression 4. The cost comes off this weekend's score. Feeds arrive next weekend. Buy a feed only if it could change your split. Ask what you would do differently with it, then compare that with its cost.
Question that counts double	Weekend 3 only	Pick footfall, best promotion or revenue, or none. That question's Brier score gets twice the weight in your final score. Choose the question where your data is best and your split is honest, not the one you feel most excited about.

What you see on your phone

Your data: The feeds you hold this weekend. Weekend 1 gives a free weather forecast and events calendar. After that you see only the feeds you bought last weekend.
Public information: The model card for footfall, plus any public news: a storm warning, a confirmed concert or a pundit's certain call. Everyone sees this panel.
Last weekend: what happened: From weekend 2: each question's outcome, the probability you put on it and the probability the crowd put on it. Use it to check your calibration.
Your track record: From weekend 2: your last score, your mean Brier score last weekend and your calibration level (R).

How you are scored

Every game in the library uses the same scoreboard. Your round result is your Forecast score (points); the leaderboard adds resilience and trust.

Score = 100 × (0.5 × performance + 0.25 × resilience + 0.25 × trust) + prediction points

Performance: Forecast scoreForecast score in points each weekend: 100 × (1 − mean Brier/2), minus the data you bought, plus any outage refund. A perfect forecast scores 100; an even spread scores about 65; full confidence on a wrong band scores 0 for that question.

Resilience (0–100)Resilience here is calibration. It compares your stated probabilities with how often those outcomes happened. It stays near 45 if you never commit, and drops 8 for each question where you put 0% on what happened.

Trust (0–100)Trust rises 1 each weekend. It gains 5 if your forecast moved the way your bought data pointed, and loses 5 if you ignored it. It loses 7 for each 85%+ call that missed, and 4 for copying the pundit or swinging wildly without data.

Performance is your total Forecast score over the rounds, scaled from the lowest to the highest in the room. Before each round you also predict “Your forecasting score this weekend”: land close and earn up to 10 bonus points; miss by the whole amount and earn none. The facilitator may change the weights before the game starts.

Tips

Always place all 100 points. Unassigned points count as 0% and simply lower your score.
Never put 0% on a band that could happen. The Brier score punishes it hard, and a log score would be infinitely bad.
Use the model card to turn weather and events into a footfall number, then spread probability around it.
Value of information: a feed is worth buying only if it could change your split by more than it costs.
Check each feed against the others and the model card. Data can be biased, and a feed that disagrees with everything deserves doubt.
Compare your split with the crowd after each reveal. If the crowd keeps beating you, ask what it knows that you ignore.

Common mistakes

Putting 90–100% on one band. When it misses, that question scores close to zero and trust falls.
Buying several feeds and then not using them. You pay the cost and lose trust for ignoring the data.
Forgetting that the free feeds end after weekend 1. Without a purchase you forecast weekend 2 on public data only.

A worked example

In weekend 1 a team puts 10/50/30/10 on footfall, 35/35/30 on the promotion and 20/60/20 on the car park. Footfall lands at 112k, the raffle wins and the car park fills between 7 and 8 pm. Their Brier scores are 0.36, 0.64 and 0.24, giving about 79.4 points. They bought the regression estimate and events calendar for 7 points, so they score about 72.4.

Shocks that can hit

The facilitator draws one event card before rounds 2 and 3. It changes the market from that round on.

One data feed is biased. An auditor warns that one paid feed has drifted since the metro station works began. Nobody yet knows which one.
Storm warning for next weekend. A rare storm front is heading for Dubai. Forecasters give a 70% chance of heavy rain. Roads may flood and families may stay home.
A pundit is certain. A popular retail influencer posts: 'I am 100% sure about next weekend's footfall.' Their call is on everyone's screen. Their record is right about half the time.
Big concert announced. A global pop star adds a show at the mall's arena next weekend. Expect crowds, late evenings and very busy food courts.
Sensor outage. A firmware update knocks out part of the analytics stack. The regression estimate will not be delivered next weekend; buyers are refunded.

The model behind the game

Brier score on a known data-generating model. The weekend really follows the footfall equation, so data feeds carry signal. The Brier score is a proper scoring rule: you score best by reporting what you truly believe. The logarithmic market scoring rule used by prediction markets works the same way but punishes a 0% on the true outcome without limit.

Footfall F = base + 1.8·(heat − 33) + 14·event − 22·rain + ε ; BS = (1/N)·Σ_q Σ_k (f_qk − o_qk)² ; score = 100·(1 − BS/2) − data cost

Brier score: The squared gap between your probabilities and what happened, summed over bands. 0 is perfect; lower is better.
Calibration: Your probabilities match reality: things you call 70% happen about 70% of the time.
Sharpness: How far your forecasts commit away from an even spread. Good forecasters are sharp and still calibrated.
Proper scoring rule: A scoring rule where you score best, on average, by reporting what you truly believe.
Value of information: How much a piece of data could improve your decision, compared with what it costs.
Wisdom of crowds: Averaging many independent forecasts often beats most individuals, because their errors partly cancel.
Biased data: A data source that is consistently off in one direction, so it misleads even when it looks precise.
Log score: Another proper scoring rule. It punishes a 0% on the true outcome without limit.

For facilitators 3 rounds · about 50 minutes

Set-up

Works well in individual or team mode; teams of 2–4 suit the forecasting debate. Allow 8 minutes per weekend.
Bots fill up to four forecasters. They forecast from the real model with some noise, so they are a fair benchmark.
Before weekend 1, show why 90% on a wrong band costs far more than 50% on a wrong band.
Remind teams that data bought this weekend only arrives next weekend, and the shop closes in weekend 3.

Modes

Team: Each team submits one forecast per question. The Analyst builds the footfall estimate, the Treasurer runs the data budget and Risk guards against overconfidence.
Individual: Each student forecasts alone, which gives the cleanest wisdom-of-crowds result. Ask students to commit before discussing with neighbours.
Hybrid: Each student forecasts individually but sits with a team to share reasoning. The team leaderboard averages members' scores, a mini crowd in itself.

Run sheet

Before class	Open the facilitator console on the projector laptop, choose the mode in Setup, and check the QR shows.
0–5 min	Students scan and join. Read the situation aloud while teams form. Point out the roles on their phones.
5–15 min	Round 1: 8 minutes to decide and predict, then Reveal. Read the headline and one surprising line from a team.
Between rounds	Draw or choose the event card. Give teams 30 seconds to read it before opening the next round.
15–25 min	Round 2, then Reveal.
27–37 min	Round 3, then Final results: podium, awards and the score chart.
Last 12 min	Debrief with the questions below, then open reflections (4 minutes). Download the CSV.

What usually happens

Many students put everything on one band in weekend 1. Use the reveal to show what that did to their Brier score.
Teams often buy data out of anxiety. After weekend 2, ask which feeds actually moved their forecast.
The crowd's average forecast often beats most individuals. Show the crowd versus best individual chart after weekend 2.
If the biased-feed card is drawn, the results lines name the drifted feed after the reveal. Before that, ask how teams would spot it.
The pundit card tempts teams to copy a 100% call. Their record is right about half the time.

Debrief questions

Did buying data improve your Brier score? Which feed was worth its cost, and which only made you feel confident?
When did the crowd beat the best individual? Why does averaging many forecasts often win?
One feed was biased in some games. How could you have spotted it from the numbers alone?

Follow-up probes

What does a 70% forecast mean if it misses? Was it wrong?
Which feed changed your split the most, and was it worth its cost?
How would you check a data feed for bias using only the numbers you had?
Why does averaging forecasts often beat the best individual over several weekends?
Where in your job do people reward confident calls over calibrated ones?
How did you pick the question to count double, and would you pick it again?

Reflection and evidence

Pick one forecast you got wrong. Say what probability you gave, what happened, and what data would have changed your split. Use the words calibration and Brier score.

Students write 80–120 words on their phones. The Data tab exports every decision, result, prediction and reflection as one CSV for grading.

Console controls

Pause, extend or set the timer; step back a round
Draw, choose or write the event card
Move players, make a Lead, add computer players
Award or fine with a reason that appears on phones
Message every screen; hide names; spotlight a team
Keys: P projector mode, Q join QR