Beta v3 Simulation ‘Outage’ / Mar 26 2023
An update on our findings and the actions we’ve taken.
Dear Players and Community Members,
We experienced an issue after the season 2 rollover (Midnight UTC of March 26), which resulted in nine races getting ‘stuck’ in a state of not processing while still holding the race entry fees. We apologize for any inconvenience this may have caused and would like to provide an update on our findings and the actions we have taken to address the issue.
Root Cause
Our investigation revealed that there was a mismatch between how our “Oracle” (the dynamic AI-driven race creator) and our existing calculation systems handled races. The Oracle system created races whose “open time” fit within the current season, while the calculation system focused on races whose “race time” fit within the same season. This mismatch of some races opening in season 2 and racing in season 3 led to the calculation system believing that the races were out of range, and therefore skipped simulating them. Your entry fees were always (and will always be) safe, but the race was stuck in a ‘purgatory’ of sorts.
Resolution
Addressing these nine ‘stuck’ races required some manual effort. After careful investigation, we found the easiest and safest path forward was to just move these races back in time before the season rollover – Doc Brown style – to be simulated again. All nine have now simulated and payouts have been processed. The links are below.
- Frontier Maiden
- White Water Maiden
- Harlem Allowance
- Seneca Maiden
- Harbor East Handicap
- Manatee Rescue Maiden
- Carnival Allowance
- Phoenix Handicap
- Holly Maiden
Moving Forward
As we are now in a new season, we do not expect to see any new instances of this error. Our engineering team has updated the Oracle system to prevent similar issues in the future. Alongside this, any time we have issues like this appear, we create additional early-detection systems and unit-tests to help catch problems before they can ever appear to the player.
We understand the importance of a seamless, secure, and trustworthy gaming experience for our players in Photo Finish™ LIVE, and we are laser-focused on our commitment to continuously improving our platform. We will always closely monitor the game to make any necessary adjustments to ensure a smooth and enjoyable experience for all players.
We appreciate your understanding and patience as we work through this issue. If you have any further questions or concerns, please do not hesitate to reach out to our support team at support@thirdtimegames or here on Discord.
Thank you for being a part of our community, and we look forward to providing you with an even better gaming experience in the future.
Sincerely,
Third Time Team
