Ergast API is being deprecated at the end of 2024

[pesaventofilippo]

As per http://ergast.com/mrd/:

The API will be updated until the end of the 2024 season then shutdown at the end of the year.

It's obviously a long-term problem, but what should we do?
If it's only the API being deprecated, then someone could just run a mirror (which I'm planning to do at some point) by cloning the database every time it updates. If however they stop updating the DB, do we have an alternative to Ergast?

[theOehrly]

Thanks for the hint. I guess there's plenty of time still to tackle this issue. My first reaction is that the community around FastF1 would be perfect to take this over. Maybe get the folks from the Multiviewer community involved as well (although they don't really depend on it, there are a lot of tech-savvy nerds there). I can easily contact the relevant people there.

We don't want Ergast to die. For multiple reasons. First, it's a unique database of Formula 1 history. Second, Chris has spent huge amounts of time to build it and collect the data. It would be a pity if this effort is not retained. I'll go ahead and contact him to make sure it's OK to use his database. And then, it would be nice to build a team to get this done by the end of next year at the latest. Doing this all alone, along with maintaining and updating FastF1, might be a bit of a challenge for me.

An API server for FastF1 was planned anyway, although nothing is really underway yet. It's all in a very rough planning stage still.

From the way I understand it, there will be no further updates to the data. Therefore, it would be necessary to figure out a (preferably) at least semi-automatic way to populate the database with future data.

[pesaventofilippo]

Fully agree. Do you know how the Ergast DB gets populated? Is it still a manual process or does it use data from F1 livetiming (or other sources)?
If it's at least somewhat automatic, it would be great if the current Ergast maintainer could open-source or at least give his system to you, so we don't start from scratch.

What I suspect though is that is a heavily manual process, and in that case I don't know how much we could do automatically, because there aren't many other data sources apart from the F1 livetiming and Ergast itself.

[theOehrly]

Just a very short update here. I've talked to Chris, who runs Ergast. He's totally Ok with it, if we try to build an open source successor to Ergast and use his database.

I currently tend towards rewriting the API server completely. The current Ergast runtime is available on Github, but the code itself is fairly old. I'm positive that a more modern implementation can scale better. One possible option would be Cloudflare's D1 database with Cloudflare Pages functions. When creating this as an open source project, it's possible to apply for project support to get a premium tier for free. This has the advantage that hosting would be free, with virtually unlimited load capacity. Of course, some more thought should be given to all of this before starting anything.
I still have some exams to deal with, but I'd like to start pushing forwards with this somewhere in October.

[mdsakalu]

I'd be interested in helping out with this effort, just let me know what you need.

[theOehrly]

I've set up a Slack workspace for now to better discuss how we might want to tackle this project. Everybody who wants to help out can join here:
https://join.slack.com/t/fastf1/shared_invite/zt-25osjniqp-TsBgxY_7alSMar5wEgWcXQ

(Note that this workspace is only intended for discussions regarding the deprecation of the Ergast API and development work around that, but not for user support questions or similar)

Some other people contacted Chris from Ergast independently, and they will be joining the Slack workspace as well.

(ping for those I know may be interested in helping @mdsakalu @Casper-Guo @harningle @pesaventofilippo)

[pbulsink]

I'm curious as to if there are any updates on this project - over at f1dataR we also rely on Ergast but hope to be able to migrate to a new option by years' end. Neither of us there are strong web/server devs but if you need help with testing I'm sure we could lend a hand.

[theOehrly]

There has been a lot of progress. We are aiming to announce a beta testing phase next. In theory, all the code is already publicly available, and the service is live right now. But we want to fix some more bugs and also add a bit more documentation and information for potential contributors before we tell people where to look. But the plan is to "open" beta testing this summer.

[marcokreeft87]

It seams to be a long term problem but the current season isnt displayed in the same way as it used to be. So its kind off a now problem

[theOehrly]

The problem is actively being worked on. But everybody who is working on it is doing so in their spare time. There's no reasonable way to make things go faster. Aiming for a full replacement in 2025 is the goal and it is realistic. Anything before that is an optional bonus.
And let's see how things really turn out once the season actually starts. Ergast has never supported Pre-Season Testing.

[marcokreeft87]

Yes I know. I wish I had time to help but my current work doesnt leave much room to do stuff after work hours.

Ergast used to have the practice,sprint,qualifying times in the schedule, but they don't have that in the 2024 season data :(

[theOehrly]

I'm aware that those times are still missing. But as I said, the season hasn't started yet. This stuff isn't automated in current Ergast. It's not unlikely that they will be added still.

[quasibasi]

Hello all!
The drivers, constructors, qualifying data seems to be missing for 2024, even though the Bahrain qualy happened today. Any ideas of whether they will be updated?
Thanks in advance!

[marcokreeft87]

Please read this thread

[quasibasi]

Oh I see, you mean it's the same question which has been answered by:

The problem is actively being worked on. But everybody who is working on it is doing so in their spare time.

Thanks!

[SmCTwelve]

Per the Ergast FAQ page: "a season is assumed to begin when results become available for the first race of the season". So just wait and see if the data becomes available after the race results.

[quasibasi]

Thanks a lot @SmCTwelve!